人工智能教授医学生外科诊断推理：使用全自动低成本反馈系统的实验证据

IF 2.1 3区医学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Journal of Surgical Education Pub Date : 2025-08-05 DOI:10.1016/j.jsurg.2025.103639

Yavuz Selim Kıyak , Emre Emekli , Tuğba İş Kara , Özlem Coşkun , Işıl İrem Budakoğlu

{"title":"人工智能教授医学生外科诊断推理：使用全自动低成本反馈系统的实验证据","authors":"Yavuz Selim Kıyak , Emre Emekli , Tuğba İş Kara , Özlem Coşkun , Işıl İrem Budakoğlu","doi":"10.1016/j.jsurg.2025.103639","DOIUrl":null,"url":null,"abstract":"<div><h3>OBJECTIVE</h3><div>While AI-generated feedback has shown promise in medical education, prior studies have only used AI for feedback, with question design handled by human experts, and the process required human involvement. This study aimed to evaluate the effectiveness of a fully automated AI-based system that generates both multiple-choice questions (MCQs) and personalized feedback, without any human input, on improving diagnostic reasoning in preclinical medical students.</div></div><div><h3>DESIGN</h3><div>A prospective, parallel-group, interventional study. The intervention group (Year-1 students) received AI-generated MCQs and feedback over 5 days using a web platform, coded via “vibe coding,” with spaced repetition. The diagnoses covered included 5 abdominal pain conditions: acute appendicitis, acute cholecystitis, acute pancreatitis, acute gastroenteritis, and nephrolithiasis. Diagnostic performance was assessed via an Objective Structured Video Examination (OSVE), immediately and 2 weeks postintervention. The control group (Year-2 students) completed the OSVE once.</div></div><div><h3>SETTING</h3><div>Gazi University Faculty of Medicine, Ankara, Turkiye; institutional academic setting focused on undergraduate medical education.</div></div><div><h3>PARTICIPANTS</h3><div>Thirty-eight Year-1 medical students completed the intervention. Thirty-three Year-2 students served as a non-randomized control group. All intervention participants completed the immediate assessment; 30 completed the delayed assessment.</div></div><div><h3>RESULTS</h3><div>Intervention participants outperformed the control group in diagnosing the 5 abdominal pain conditions immediately after the intervention (p < 0.001) and at the 2-week follow-up (p < 0.001). Postintervention expert review confirmed the accuracy of all AI-generated questions and identified minimal issues in 0.6% of feedback statements. Total AI cost was $0.51.</div></div><div><h3>CONCLUSIONS</h3><div>A fully automated, low-cost AI system without human in the loop during content generation can significantly enhance illness scripts in preclinical medical students. Early engagement with such tools may help students strengthen surgical diagnostic skills and derive greater benefit from clinical environments. This kind of tools may transform how clinical reasoning is taught in resource-limited or high-volume educational settings.</div></div>","PeriodicalId":50033,"journal":{"name":"Journal of Surgical Education","volume":"82 10","pages":"Article 103639"},"PeriodicalIF":2.1000,"publicationDate":"2025-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"AI Teaches Surgical Diagnostic Reasoning to Medical Students: Evidence from an Experiment Using a Fully Automated, Low-Cost Feedback System\",\"authors\":\"Yavuz Selim Kıyak , Emre Emekli , Tuğba İş Kara , Özlem Coşkun , Işıl İrem Budakoğlu\",\"doi\":\"10.1016/j.jsurg.2025.103639\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>OBJECTIVE</h3><div>While AI-generated feedback has shown promise in medical education, prior studies have only used AI for feedback, with question design handled by human experts, and the process required human involvement. This study aimed to evaluate the effectiveness of a fully automated AI-based system that generates both multiple-choice questions (MCQs) and personalized feedback, without any human input, on improving diagnostic reasoning in preclinical medical students.</div></div><div><h3>DESIGN</h3><div>A prospective, parallel-group, interventional study. The intervention group (Year-1 students) received AI-generated MCQs and feedback over 5 days using a web platform, coded via “vibe coding,” with spaced repetition. The diagnoses covered included 5 abdominal pain conditions: acute appendicitis, acute cholecystitis, acute pancreatitis, acute gastroenteritis, and nephrolithiasis. Diagnostic performance was assessed via an Objective Structured Video Examination (OSVE), immediately and 2 weeks postintervention. The control group (Year-2 students) completed the OSVE once.</div></div><div><h3>SETTING</h3><div>Gazi University Faculty of Medicine, Ankara, Turkiye; institutional academic setting focused on undergraduate medical education.</div></div><div><h3>PARTICIPANTS</h3><div>Thirty-eight Year-1 medical students completed the intervention. Thirty-three Year-2 students served as a non-randomized control group. All intervention participants completed the immediate assessment; 30 completed the delayed assessment.</div></div><div><h3>RESULTS</h3><div>Intervention participants outperformed the control group in diagnosing the 5 abdominal pain conditions immediately after the intervention (p < 0.001) and at the 2-week follow-up (p < 0.001). Postintervention expert review confirmed the accuracy of all AI-generated questions and identified minimal issues in 0.6% of feedback statements. Total AI cost was $0.51.</div></div><div><h3>CONCLUSIONS</h3><div>A fully automated, low-cost AI system without human in the loop during content generation can significantly enhance illness scripts in preclinical medical students. Early engagement with such tools may help students strengthen surgical diagnostic skills and derive greater benefit from clinical environments. This kind of tools may transform how clinical reasoning is taught in resource-limited or high-volume educational settings.</div></div>\",\"PeriodicalId\":50033,\"journal\":{\"name\":\"Journal of Surgical Education\",\"volume\":\"82 10\",\"pages\":\"Article 103639\"},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2025-08-05\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Surgical Education\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S193172042500220X\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION, SCIENTIFIC DISCIPLINES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Surgical Education","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S193172042500220X","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

摘要

虽然人工智能生成的反馈在医学教育中显示出前景，但之前的研究仅使用人工智能进行反馈，问题设计由人类专家处理，并且过程需要人类参与。本研究旨在评估基于人工智能的全自动系统的有效性，该系统在没有任何人工输入的情况下生成多项选择题（mcq）和个性化反馈，以提高临床前医学院学生的诊断推理能力。设计一项前瞻性、平行组、干预性研究。干预组（一年级学生）在五天内通过网络平台接受人工智能生成的mcq和反馈，通过“vibe编码”进行编码，间隔重复。诊断包括5种腹痛情况：急性阑尾炎、急性胆囊炎、急性胰腺炎、急性胃肠炎和肾结石。在干预后立即和2周，通过客观结构化视频检查（OSVE）评估诊断表现。对照组（二年级学生）完成一次OSVE。加齐大学医学院，土耳其安卡拉；以本科医学教育为重点的机构学术设置。参与者：38名一年级医学生完成了干预。33名二年级学生作为非随机对照组。所有干预参与者都完成了即时评估；30人完成了延迟的评估。结果干预组在干预后对5种腹痛症状的诊断优于对照组(p <；0.001)和2周随访时(p <；0.001)。干预后的专家审查确认了所有人工智能生成的问题的准确性，并在0.6%的反馈声明中发现了最小的问题。人工智能总成本为0.51美元。结论在内容生成过程中无需人工参与的全自动化、低成本人工智能系统可以显著提高临床前医学院学生的病情脚本。早期使用这些工具可以帮助学生加强外科诊断技能，并从临床环境中获得更大的好处。这种工具可能会改变临床推理在资源有限或高容量教育环境中的教学方式。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

AI Teaches Surgical Diagnostic Reasoning to Medical Students: Evidence from an Experiment Using a Fully Automated, Low-Cost Feedback System

OBJECTIVE

While AI-generated feedback has shown promise in medical education, prior studies have only used AI for feedback, with question design handled by human experts, and the process required human involvement. This study aimed to evaluate the effectiveness of a fully automated AI-based system that generates both multiple-choice questions (MCQs) and personalized feedback, without any human input, on improving diagnostic reasoning in preclinical medical students.

DESIGN

A prospective, parallel-group, interventional study. The intervention group (Year-1 students) received AI-generated MCQs and feedback over 5 days using a web platform, coded via “vibe coding,” with spaced repetition. The diagnoses covered included 5 abdominal pain conditions: acute appendicitis, acute cholecystitis, acute pancreatitis, acute gastroenteritis, and nephrolithiasis. Diagnostic performance was assessed via an Objective Structured Video Examination (OSVE), immediately and 2 weeks postintervention. The control group (Year-2 students) completed the OSVE once.

SETTING

Gazi University Faculty of Medicine, Ankara, Turkiye; institutional academic setting focused on undergraduate medical education.

PARTICIPANTS

Thirty-eight Year-1 medical students completed the intervention. Thirty-three Year-2 students served as a non-randomized control group. All intervention participants completed the immediate assessment; 30 completed the delayed assessment.

RESULTS

Intervention participants outperformed the control group in diagnosing the 5 abdominal pain conditions immediately after the intervention (p < 0.001) and at the 2-week follow-up (p < 0.001). Postintervention expert review confirmed the accuracy of all AI-generated questions and identified minimal issues in 0.6% of feedback statements. Total AI cost was $0.51.

CONCLUSIONS

A fully automated, low-cost AI system without human in the loop during content generation can significantly enhance illness scripts in preclinical medical students. Early engagement with such tools may help students strengthen surgical diagnostic skills and derive greater benefit from clinical environments. This kind of tools may transform how clinical reasoning is taught in resource-limited or high-volume educational settings.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Journal of Surgical Education EDUCATION, SCIENTIFIC DISCIPLINES-SURGERY

CiteScore

5.60

自引率

10.30%

发文量

261

审稿时长

48 days

期刊介绍： The Journal of Surgical Education (JSE) is dedicated to advancing the field of surgical education through original research. The journal publishes research articles in all surgical disciplines on topics relative to the education of surgical students, residents, and fellows, as well as practicing surgeons. Our readers look to JSE for timely, innovative research findings from the international surgical education community. As the official journal of the Association of Program Directors in Surgery (APDS), JSE publishes the proceedings of the annual APDS meeting held during Surgery Education Week.