Conversational AI with large language models to increase the uptake of clinical guidance

Abstract

The rise of large language models (LLMs) and conversational applications, like ChatGPT, prompts Health Technology Assessment (HTA) bodies, such as NICE, to rethink how healthcare professionals access clinical guidance. Integrating LLMs into systems like Retrieval-Augmented Generation (RAG) offers potential solutions to current LLMs’ problems, like the generation of false or misleading information. The objective of this paper is to design and debate the value of an AI-driven system, similar to ChatGPT, to enhance the uptake of clinical guidance within the National Health Service (NHS) of the UK. Conversational interfaces, powered by LLMs, offer healthcare practitioners clear benefits over traditional ways of getting clinical guidance, such as easy navigation through long documents, blending information from various trusted sources, or expediting evidence-based decisions in situ. But, putting these interfaces into practice brings new challenges for HTA bodies, like assuring quality, addressing data privacy concerns, navigating existing resource constraints, or preparing the organization for innovative practices. Rigorous empirical evaluations are necessary to validate their effectiveness in increasing the uptake of clinical guidance among healthcare practitioners. A feasible evaluation strategy is elucidated in this research while its implementation remains as future work.