Leveraging evaluation of quality on medical education research with ChatGPT.

IF 3.3 2区教育学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Medical Teacher Pub Date : 2025-03-01 Epub Date: 2024-08-04 DOI:10.1080/0142159X.2024.2385678

Javier Alejandro Flores-Cohaila, Peter Garcia-Portocarrero, Deysi A Saldaña-Amaya, Brayan Miranda-Chavez, Cesar Copaja-Corzo

{"title":"Leveraging evaluation of quality on medical education research with ChatGPT.","authors":"Javier Alejandro Flores-Cohaila, Peter Garcia-Portocarrero, Deysi A Saldaña-Amaya, Brayan Miranda-Chavez, Cesar Copaja-Corzo","doi":"10.1080/0142159X.2024.2385678","DOIUrl":null,"url":null,"abstract":"What is the educational challenge? The Medical Education Research Study Quality Instrument (MERSQI) is widely used to evaluate the quality of quantitative research in medical education. It has strong evidence of validity and is endorsed by guidelines. However, the manual appraisal process is time-consuming and resource-intensive, highlighting the need for more efficient methods. What are the proposed solutions? We propose to use ChatGPT to evaluate the quality of medical education research with the MERSQI and compare its scoring with those of human evaluators. What are the potential benefits to a broader global audience? Using ChatGPT to evaluate medical education research with the MERSQI can decrease the resources required for quality appraisal. This allows faster summaries of evidence, reducing the workload of researchers, editors, and educators. Furthermore, ChatGPTs' capability to extract supporting excerpts provides transparency and may have the potential for data extraction and training new medical education researchers. What are the next steps? We plan to continue evaluating medical education research with ChatGPT using the MERSQI and other instruments to determine its feasibility in this realm. Moreover, we plan to investigate which types of studies ChatGPT performs best in.","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"401-403"},"PeriodicalIF":3.3000,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical Teacher","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/0142159X.2024.2385678","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/4 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

Abstract

What is the educational challenge? The Medical Education Research Study Quality Instrument (MERSQI) is widely used to evaluate the quality of quantitative research in medical education. It has strong evidence of validity and is endorsed by guidelines. However, the manual appraisal process is time-consuming and resource-intensive, highlighting the need for more efficient methods. What are the proposed solutions? We propose to use ChatGPT to evaluate the quality of medical education research with the MERSQI and compare its scoring with those of human evaluators. What are the potential benefits to a broader global audience? Using ChatGPT to evaluate medical education research with the MERSQI can decrease the resources required for quality appraisal. This allows faster summaries of evidence, reducing the workload of researchers, editors, and educators. Furthermore, ChatGPTs' capability to extract supporting excerpts provides transparency and may have the potential for data extraction and training new medical education researchers. What are the next steps? We plan to continue evaluating medical education research with ChatGPT using the MERSQI and other instruments to determine its feasibility in this realm. Moreover, we plan to investigate which types of studies ChatGPT performs best in.

查看原文本刊更多论文

利用 ChatGPT 对医学教育研究质量进行评估。

什么是教育挑战？医学教育研究质量工具（MERSQI）被广泛用于评估医学教育定量研究的质量。它的有效性证据确凿，并得到了相关指南的认可。然而，人工评估过程耗时耗力，需要更高效的方法。建议的解决方案是什么？我们建议使用 ChatGPT，用 MERSQI 评估医学教育研究的质量，并将其评分与人类评估者的评分进行比较。对更广泛的全球受众有什么潜在好处？使用 ChatGPT 评估医学教育研究的 MERSQI 可以减少质量评估所需的资源。这样可以更快地总结证据，减轻研究人员、编辑和教育工作者的工作量。此外，ChatGPTs 能够提取支持性摘录，从而提高了透明度，并有可能用于数据提取和培训新的医学教育研究人员。下一步计划是什么？我们计划继续使用 MERSQI 和其他工具对 ChatGPT 的医学教育研究进行评估，以确定其在这一领域的可行性。此外，我们还计划调查 ChatGPT 在哪类研究中表现最佳。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Medical Teacher 医学-卫生保健

CiteScore

7.80

自引率

8.50%

发文量

396

审稿时长

3-6 weeks

期刊介绍： Medical Teacher provides accounts of new teaching methods, guidance on structuring courses and assessing achievement, and serves as a forum for communication between medical teachers and those involved in general education. In particular, the journal recognizes the problems teachers have in keeping up-to-date with the developments in educational methods that lead to more effective teaching and learning at a time when the content of the curriculum—from medical procedures to policy changes in health care provision—is also changing. The journal features reports of innovation and research in medical education, case studies, survey articles, practical guidelines, reviews of current literature and book reviews. All articles are peer reviewed.