A use case of ChatGPT: summary of an expert panel discussion on electronic health records and implementation science.

Objective: Artificial intelligence (AI) is revolutionizing healthcare, but less is known about how it may facilitate methodological innovations in research settings. In this manuscript, we describe a novel use of AI in summarizing and reporting qualitative data generated from an expert panel discussion about the role of electronic health records (EHRs) in implementation science.

Materials and methods: 15 implementation scientists participated in an hour-long expert panel discussion addressing how EHRs can support implementation strategies, measure implementation outcomes, and influence implementation science. Notes from the discussion were synthesized by ChatGPT (a large language model-LLM) to generate a manuscript summarizing the discussion, which was later revised by participants. We also surveyed participants on their experience with the process.

Results: Panelists identified implementation strategies and outcome measures that can be readily supported by EHRs and noted that implementation science will need to evolve to assess future EHR advancements. The ChatGPT-generated summary of the panel discussion was generally regarded as an efficient means to offer a high-level overview of the discussion, although participants felt it lacked nuance and context. Extensive editing was required to contextualize the LLM-generated text and situate it in relevant literature.

Discussion and conclusions: Our qualitative findings highlight the central role EHRs can play in supporting implementation science, which may require additional informatics and implementation expertise and a different way to think about the combined fields. Our experience using ChatGPT as a research methods innovation was mixed and underscores the need for close supervision and attentive human involvement.