A Randomized Controlled Trial of a Deep Language Learning Model-Based Simulation Tool for Undergraduate Medical Students in Surgery

IF 2.1 3区医学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Journal of Surgical Education Pub Date : 2025-07-28 DOI:10.1016/j.jsurg.2025.103629

Cathleen A. McCarrick, Philip D. McEntee, Patrick A. Boland, Suzanne Donnelly, Yvonne O’Meara, Helen Heneghan, Ronan A. Cahill

{"title":"A Randomized Controlled Trial of a Deep Language Learning Model-Based Simulation Tool for Undergraduate Medical Students in Surgery","authors":"Cathleen A. McCarrick, Philip D. McEntee, Patrick A. Boland, Suzanne Donnelly, Yvonne O’Meara, Helen Heneghan, Ronan A. Cahill","doi":"10.1016/j.jsurg.2025.103629","DOIUrl":null,"url":null,"abstract":"<div><h3>Introduction</h3><div>Effective communication is a critical skill for surgeons that commences often with history-taking. While simulation-based training is utilized to enhance these skills, recent advancements in artificial intelligence (AI), especially deep language learning models (DLM), offer new opportunities. This study evaluates the integration of DLM as a simulated patient (SP) into surgical history-taking training for senior medical students during clinical rotations.</div></div><div><h3>Methods</h3><div>A randomized controlled trial was conducted with surgery module students. Participants were divided into control and intervention groups, the former receiving standard experiential learning and the latter adding 3 structured sessions with DLM (ChatGPT, Open AI) as SP (with interaction texts submitted for tutor evaluation). All students underwent Objective Structured Clinical Examination (OSCE) of history-taking with a human SP and blinded assessor blinded by group for baseline competency ascertainment and again after either intervention or a similar time of standard learning. Intervention group students were anonymously surveyed to assess communication confidence and perspectives on DLM as SP.</div></div><div><h3>Results</h3><div>After initial pilot trialing, ninety students participated formally with 45 assigned to each arm via randomized cluster sampling. DLM-content was uniformly appropriate. Baseline scores were similar but significantly increased in the intervention group alone (p < 0.001, 0.37v0.19 Cohen D education effect size). 62% of students completed the survey, a majority (57%) articulating increased confidence, rich detail in DLM histories (72%) and would use again (95%).</div></div><div><h3>Conclusions</h3><div>DLM effectively enhanced surgical history-taking skills. These findings indicate AI can serve as a valuable tool for student development alongside clinical learning.</div></div>","PeriodicalId":50033,"journal":{"name":"Journal of Surgical Education","volume":"82 9","pages":"Article 103629"},"PeriodicalIF":2.1000,"publicationDate":"2025-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Surgical Education","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1931720425002107","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction

Effective communication is a critical skill for surgeons that commences often with history-taking. While simulation-based training is utilized to enhance these skills, recent advancements in artificial intelligence (AI), especially deep language learning models (DLM), offer new opportunities. This study evaluates the integration of DLM as a simulated patient (SP) into surgical history-taking training for senior medical students during clinical rotations.

Methods

A randomized controlled trial was conducted with surgery module students. Participants were divided into control and intervention groups, the former receiving standard experiential learning and the latter adding 3 structured sessions with DLM (ChatGPT, Open AI) as SP (with interaction texts submitted for tutor evaluation). All students underwent Objective Structured Clinical Examination (OSCE) of history-taking with a human SP and blinded assessor blinded by group for baseline competency ascertainment and again after either intervention or a similar time of standard learning. Intervention group students were anonymously surveyed to assess communication confidence and perspectives on DLM as SP.

Results

After initial pilot trialing, ninety students participated formally with 45 assigned to each arm via randomized cluster sampling. DLM-content was uniformly appropriate. Baseline scores were similar but significantly increased in the intervention group alone (p < 0.001, 0.37v0.19 Cohen D education effect size). 62% of students completed the survey, a majority (57%) articulating increased confidence, rich detail in DLM histories (72%) and would use again (95%).

Conclusions

DLM effectively enhanced surgical history-taking skills. These findings indicate AI can serve as a valuable tool for student development alongside clinical learning.

查看原文本刊更多论文

基于深度语言学习模型的外科本科医学生模拟工具的随机对照试验

对外科医生来说，有效的沟通是一项至关重要的技能，这通常始于历史记录。虽然基于模拟的培训被用来提高这些技能，但人工智能（AI）的最新进展，特别是深度语言学习模型（DLM），提供了新的机会。本研究评估了在临床轮转期间，将DLM作为模拟病人（SP）纳入高年级医学生的手术记录训练。方法采用随机对照试验对外科模块学生进行研究。参与者被分为对照组和干预组，前者接受标准的体验式学习，后者增加了3个结构化的DLM （ChatGPT, Open AI）作为SP（互动文本提交给导师评估）。所有学生都接受了客观结构化临床检查（OSCE），由一名人类SP和盲法评估员进行历史记录，以确定基线能力，并在干预或类似的标准学习时间后再次进行。干预组学生进行匿名调查，以评估沟通信心和对DLM作为sp的看法。结果经过初步的试点试验，90名学生正式参与，随机整群抽样将45名学生分配到每组。dlm内容一致合适。基线得分相似，但单独干预组显著增加（p <； 0.001,0.37v0.19 Cohen D教育效应量）。62%的学生完成了调查，大多数学生（57%）表达了增强的信心，DLM历史的丰富细节（72%），并会再次使用（95%）。结论sdlm可有效提高手术记录技能。这些发现表明，人工智能可以作为学生发展和临床学习的宝贵工具。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Surgical Education EDUCATION, SCIENTIFIC DISCIPLINES-SURGERY

CiteScore

5.60

自引率

10.30%

发文量

261

审稿时长

48 days

期刊介绍： The Journal of Surgical Education (JSE) is dedicated to advancing the field of surgical education through original research. The journal publishes research articles in all surgical disciplines on topics relative to the education of surgical students, residents, and fellows, as well as practicing surgeons. Our readers look to JSE for timely, innovative research findings from the international surgical education community. As the official journal of the Association of Program Directors in Surgery (APDS), JSE publishes the proceedings of the annual APDS meeting held during Surgery Education Week.