Extension of the Consolidated Criteria for Reporting Qualitative Research Guideline to Large Language Models (COREQ+LLM): Protocol for a Multiphase Study.

IF 1.5 Q3 HEALTH CARE SCIENCES & SERVICES
Leonard Fehring, Julian Frings, Paul Rust, Christian Kempny, Petra A Thürmann, Sven Meister
{"title":"Extension of the Consolidated Criteria for Reporting Qualitative Research Guideline to Large Language Models (COREQ+LLM): Protocol for a Multiphase Study.","authors":"Leonard Fehring, Julian Frings, Paul Rust, Christian Kempny, Petra A Thürmann, Sven Meister","doi":"10.2196/78682","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Qualitative research provides essential insights into human behaviors, perceptions, and experiences in health sciences. The COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist, published in 2007 and endorsed by the Enhancing the Quality and Transparency of Health Research Network, advanced transparency of qualitative research reporting. However, the recent integration of large language models (LLMs) into qualitative research introduces novel opportunities and methodological challenges that existing guidelines do not address. LLMs are increasingly applied to research design as well as processing, analysis, interpretation, and even direct interaction (\"conversing\") with qualitative data. However, their probabilistic nature, dependence on underlying training data, and susceptibility to hallucinations necessitate dedicated reporting to ensure transparency, reproducibility, and methodological validity.</p><p><strong>Objective: </strong>This protocol outlines the methodological development process of COREQ+LLM, an extension to the COREQ checklist, to support transparent reporting of LLM use in qualitative research. The three main objectives are to (1) identify and categorize current applications of LLMs used as qualitative research tools, (2) assess how LLM use in qualitative studies in health care is reported in published studies, and (3) develop and refine reporting items for COREQ+LLM through a structured consensus process among international experts.</p><p><strong>Methods: </strong>Following the Enhancing the Quality and Transparency of Health Research Network guidance for reporting guideline development, this study comprises 4 main phases. Phase 1 is a systematic scoping review of peer-reviewed literature from January 2020 to April 2025, examining the use and reporting of LLMs in qualitative research. The scoping review protocol was registered with the Open Science Framework on June 6, 2025, and will adhere to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. Phase 2 will use a Delphi process to reach consensus on candidate items for inclusion in the COREQ+LLM checklist among an interdisciplinary international panel of experts. Phase 3 includes pilot testing, and phase 4 involves publication and dissemination.</p><p><strong>Results: </strong>As of September 2025, the steering committee has been established, and the initial search strategy for the scoping review has identified 5049 records, with 4201 (83.20%) remaining after duplicate removal. Title and abstract screening is underway and will inform the initial draft of candidate checklist items. The COREQ+LLM extension is scheduled for completion by December 2025.</p><p><strong>Conclusions: </strong>The integration of LLMs in qualitative research requires dedicated reporting guidelines to ensure methodological rigor, transparency, and interpretability. COREQ+LLM will address current reporting gaps by offering specific guidance for documenting LLM integration in qualitative research workflows. The checklist will assist researchers in transparently documenting LLM use, support reviewers and editors in evaluating methodological quality, and foster trust in LLM-supported qualitative research. By December 2025, COREQ+LLM will provide a rigorously developed tool to enhance the transparency, validity, and reproducibility of LLM-supported qualitative studies.</p><p><strong>International registered report identifier (irrid): </strong>DERR1-10.2196/78682.</p>","PeriodicalId":14755,"journal":{"name":"JMIR Research Protocols","volume":"14 ","pages":"e78682"},"PeriodicalIF":1.5000,"publicationDate":"2025-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12508663/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"JMIR Research Protocols","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2196/78682","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Background: Qualitative research provides essential insights into human behaviors, perceptions, and experiences in health sciences. The COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist, published in 2007 and endorsed by the Enhancing the Quality and Transparency of Health Research Network, advanced transparency of qualitative research reporting. However, the recent integration of large language models (LLMs) into qualitative research introduces novel opportunities and methodological challenges that existing guidelines do not address. LLMs are increasingly applied to research design as well as processing, analysis, interpretation, and even direct interaction ("conversing") with qualitative data. However, their probabilistic nature, dependence on underlying training data, and susceptibility to hallucinations necessitate dedicated reporting to ensure transparency, reproducibility, and methodological validity.

Objective: This protocol outlines the methodological development process of COREQ+LLM, an extension to the COREQ checklist, to support transparent reporting of LLM use in qualitative research. The three main objectives are to (1) identify and categorize current applications of LLMs used as qualitative research tools, (2) assess how LLM use in qualitative studies in health care is reported in published studies, and (3) develop and refine reporting items for COREQ+LLM through a structured consensus process among international experts.

Methods: Following the Enhancing the Quality and Transparency of Health Research Network guidance for reporting guideline development, this study comprises 4 main phases. Phase 1 is a systematic scoping review of peer-reviewed literature from January 2020 to April 2025, examining the use and reporting of LLMs in qualitative research. The scoping review protocol was registered with the Open Science Framework on June 6, 2025, and will adhere to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. Phase 2 will use a Delphi process to reach consensus on candidate items for inclusion in the COREQ+LLM checklist among an interdisciplinary international panel of experts. Phase 3 includes pilot testing, and phase 4 involves publication and dissemination.

Results: As of September 2025, the steering committee has been established, and the initial search strategy for the scoping review has identified 5049 records, with 4201 (83.20%) remaining after duplicate removal. Title and abstract screening is underway and will inform the initial draft of candidate checklist items. The COREQ+LLM extension is scheduled for completion by December 2025.

Conclusions: The integration of LLMs in qualitative research requires dedicated reporting guidelines to ensure methodological rigor, transparency, and interpretability. COREQ+LLM will address current reporting gaps by offering specific guidance for documenting LLM integration in qualitative research workflows. The checklist will assist researchers in transparently documenting LLM use, support reviewers and editors in evaluating methodological quality, and foster trust in LLM-supported qualitative research. By December 2025, COREQ+LLM will provide a rigorously developed tool to enhance the transparency, validity, and reproducibility of LLM-supported qualitative studies.

International registered report identifier (irrid): DERR1-10.2196/78682.

大型语言模型定性研究综合报告准则(COREQ+LLM)的扩展:多阶段研究协议。
背景:定性研究提供了对健康科学中人类行为、观念和经验的基本见解。2007年出版并得到提高卫生研究质量和透明度网络认可的COREQ(报告定性研究综合标准)核对表提高了定性研究报告的透明度。然而,最近将大型语言模型(llm)集成到定性研究中带来了新的机会和方法上的挑战,而现有的指导方针并没有解决这些问题。法学硕士越来越多地应用于研究设计以及处理、分析、解释,甚至与定性数据直接互动(“对话”)。然而,它们的概率性质、对潜在训练数据的依赖性以及对幻觉的易感性需要专门的报告来确保透明度、可重复性和方法有效性。目的:本协议概述了COREQ+LLM (COREQ清单的扩展)的方法学发展过程,以支持在定性研究中透明地报告法学硕士的使用。三个主要目标是:(1)识别和分类法学硕士作为定性研究工具的当前应用,(2)评估法学硕士在医疗保健定性研究中的使用如何在已发表的研究中报告,以及(3)通过国际专家之间的结构化共识过程开发和完善COREQ+法学硕士的报告项目。方法:本研究遵循《提高卫生研究网络报告指南的质量和透明度》的指导方针,分为4个主要阶段。第一阶段是对2020年1月至2025年4月的同行评议文献进行系统的范围审查,检查法学硕士在定性研究中的使用和报告。该范围评价方案于2025年6月6日在开放科学框架注册,并将遵循PRISMA-ScR(范围评价的系统评价和元分析扩展首选报告项目)指南。第二阶段将使用德尔菲过程,在跨学科的国际专家小组中就COREQ+LLM清单中的候选项目达成共识。第三阶段包括试点测试,第四阶段包括出版和传播。结果:截至2025年9月,指导委员会已经成立,范围评审的初步检索策略已确定5049条记录,重复删除后剩余4201条(83.20%)。标题和摘要筛选正在进行中,并将通知候选清单项目的初步草案。COREQ+LLM延期计划于2025年12月完成。结论:法学硕士在定性研究中的整合需要专门的报告指南,以确保方法的严谨性、透明度和可解释性。COREQ+LLM将解决目前的报告差距,提供具体的指导,记录法学硕士整合定性研究工作流程。该清单将帮助研究人员透明地记录法学硕士的使用,支持审稿人和编辑评估方法质量,并促进法学硕士支持的定性研究的信任。到2025年12月,COREQ+LLM将提供一个严格开发的工具,以提高法学硕士支持的定性研究的透明度、有效性和可重复性。国际注册报告标识符(irrid): DERR1-10.2196/78682。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
2.40
自引率
5.90%
发文量
414
审稿时长
12 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信