Generative AI for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations - an ISPOR Working Group Report.

Abstract

Objective: To provide an introduction to the uses of generative Artificial Intelligence (AI) and foundation models, including large language models (LLMs), in the field of health technology assessment (HTA).

Methods: We reviewed applications of generative AI in three areas: systematic literature reviews, real world evidence (RWE) and health economic modeling.

Results: (1) Literature reviews: generative AI has the potential to assist in automating aspects of systematic literature reviews by proposing search terms, screening abstracts, extracting data and generating code for meta-analyses; (2) Real World Evidence (RWE): generative AI can facilitate automating processes and analyze large collections of real-world data (RWD) including unstructured clinical notes and imaging; (3) Health economic modeling: generative AI can aid in the development of health economic models, from conceptualization to validation. Limitations in the use of foundation models and LLMs include challenges surrounding their scientific rigor and reliability, the potential for bias, implications for equity, as well as nontrivial concerns regarding adherence to regulatory and ethical standards, particularly in terms of data privacy and security. Additionally, we survey the current policy landscape and provide suggestions for HTA agencies on responsibly integrating generative AI into their workflows, emphasizing the importance of human oversight and the fast-evolving nature of these tools.

Conclusions: While generative AI technology holds promise with respect to HTA applications, it is still undergoing rapid developments and improvements. Continued careful evaluation of their applications to HTA is required. Both developers and users of research incorporating these tools, should familiarize themselves with their current capabilities and limitations.