Assessing AI in Various Elements of Enhanced Recovery After Surgery (ERAS)-Guided Ankle Fracture Treatment: A Comparative Analysis with Expert Agreement.

IF 2.7 3区 医学 Q2 HEALTH CARE SCIENCES & SERVICES
Journal of Multidisciplinary Healthcare Pub Date : 2025-03-19 eCollection Date: 2025-01-01 DOI:10.2147/JMDH.S508511
Rui Wang, Xuanming Situ, Xu Sun, Jinchang Zhan, Xi Liu
{"title":"Assessing AI in Various Elements of Enhanced Recovery After Surgery (ERAS)-Guided Ankle Fracture Treatment: A Comparative Analysis with Expert Agreement.","authors":"Rui Wang, Xuanming Situ, Xu Sun, Jinchang Zhan, Xi Liu","doi":"10.2147/JMDH.S508511","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>This study aimed to assess and compare the performance of ChatGPT and iFlytek Spark, two AI-powered large language models (LLMs), in generating clinical recommendations aligned with expert consensus on Enhanced Recovery After Surgery (ERAS)-guided ankle fracture treatment. This study aims to determine the applicability and reliability of AI in supporting ERAS protocols for optimized patient outcomes.</p><p><strong>Methods: </strong>A qualitative comparative analysis was conducted using 35 structured clinical questions derived from the Expert Consensus on Optimizing Ankle Fracture Treatment Protocols under ERAS Principles. Questions covered preoperative preparation, intraoperative management, postoperative pain control and rehabilitation, and complication management. Responses from ChatGPT and iFlytek Spark were independently evaluated by two experienced trauma orthopedic specialists based on clinical relevance, consistency with expert consensus, and depth of reasoning.</p><p><strong>Results: </strong>ChatGPT demonstrated higher alignment with expert consensus (29/35 questions, 82.9%), particularly in comprehensive perioperative recommendations, detailed medical rationales, and structured treatment plans. However, discrepancies were noted in intraoperative blood pressure management and preoperative antiemetic selection. iFlytek Spark aligned with expert consensus in 22/35 questions (62.9%), but responses were often more generalized, less clinically detailed, and occasionally inconsistent with best practices. Agreement between ChatGPT and iFlytek Spark was observed in 23/35 questions (65.7%), with ChatGPT generally exhibiting greater specificity, timeliness, and precision in its recommendations.</p><p><strong>Conclusion: </strong>AI-powered LLMs, particularly ChatGPT, show promise in supporting clinical decision-making for ERAS-guided ankle fracture management. While ChatGPT provided more accurate and contextually relevant responses, inconsistencies with expert consensus highlight the need for further refinement, validation, and clinical integration. iFlytek Spark's lower conformity suggests potential differences in training data and underlying algorithms, underscoring the variability in AI-generated medical advice. To optimize AI's role in orthopedic care, future research should focus on enhancing AI alignment with medical guidelines, improving model transparency, and integrating physician oversight to ensure safe and effective clinical applications.</p>","PeriodicalId":16357,"journal":{"name":"Journal of Multidisciplinary Healthcare","volume":"18 ","pages":"1629-1638"},"PeriodicalIF":2.7000,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11930842/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Multidisciplinary Healthcare","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.2147/JMDH.S508511","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/1/1 0:00:00","PubModel":"eCollection","JCR":"Q2","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Objective: This study aimed to assess and compare the performance of ChatGPT and iFlytek Spark, two AI-powered large language models (LLMs), in generating clinical recommendations aligned with expert consensus on Enhanced Recovery After Surgery (ERAS)-guided ankle fracture treatment. This study aims to determine the applicability and reliability of AI in supporting ERAS protocols for optimized patient outcomes.

Methods: A qualitative comparative analysis was conducted using 35 structured clinical questions derived from the Expert Consensus on Optimizing Ankle Fracture Treatment Protocols under ERAS Principles. Questions covered preoperative preparation, intraoperative management, postoperative pain control and rehabilitation, and complication management. Responses from ChatGPT and iFlytek Spark were independently evaluated by two experienced trauma orthopedic specialists based on clinical relevance, consistency with expert consensus, and depth of reasoning.

Results: ChatGPT demonstrated higher alignment with expert consensus (29/35 questions, 82.9%), particularly in comprehensive perioperative recommendations, detailed medical rationales, and structured treatment plans. However, discrepancies were noted in intraoperative blood pressure management and preoperative antiemetic selection. iFlytek Spark aligned with expert consensus in 22/35 questions (62.9%), but responses were often more generalized, less clinically detailed, and occasionally inconsistent with best practices. Agreement between ChatGPT and iFlytek Spark was observed in 23/35 questions (65.7%), with ChatGPT generally exhibiting greater specificity, timeliness, and precision in its recommendations.

Conclusion: AI-powered LLMs, particularly ChatGPT, show promise in supporting clinical decision-making for ERAS-guided ankle fracture management. While ChatGPT provided more accurate and contextually relevant responses, inconsistencies with expert consensus highlight the need for further refinement, validation, and clinical integration. iFlytek Spark's lower conformity suggests potential differences in training data and underlying algorithms, underscoring the variability in AI-generated medical advice. To optimize AI's role in orthopedic care, future research should focus on enhancing AI alignment with medical guidelines, improving model transparency, and integrating physician oversight to ensure safe and effective clinical applications.

人工智能在ERAS引导下踝关节骨折治疗中增强术后恢复各要素的评估:与专家一致的比较分析。
目的:本研究旨在评估和比较ChatGPT和科大讯飞Spark这两种人工智能驱动的大型语言模型(llm)在生成临床建议方面的性能,这些建议与专家共识一致,即在手术后增强恢复(ERAS)引导下的踝关节骨折治疗。本研究旨在确定人工智能在支持ERAS方案以优化患者预后方面的适用性和可靠性。方法:采用来自ERAS原则下优化踝关节骨折治疗方案专家共识的35个结构化临床问题进行定性比较分析。问题包括术前准备、术中处理、术后疼痛控制和康复以及并发症处理。ChatGPT和科大讯飞Spark的回复由两位经验丰富的创伤骨科专家根据临床相关性、与专家共识的一致性和推理深度进行独立评估。结果:ChatGPT与专家共识的一致性较高(29/35,82.9%),特别是在全面的围手术期建议、详细的医疗理由和结构化的治疗计划方面。然而,术中血压管理和术前止吐药的选择存在差异。科大讯飞Spark在22/35个问题(62.9%)中与专家共识一致,但回答往往更笼统,缺乏临床细节,有时与最佳实践不一致。在23/35个问题(65.7%)中,ChatGPT和科大讯飞的意见一致,ChatGPT在其建议中通常表现出更大的特异性、及时性和准确性。结论:人工智能驱动的llm,特别是ChatGPT,在支持eras引导的踝关节骨折治疗的临床决策方面显示出前景。虽然ChatGPT提供了更准确和上下文相关的响应,但与专家共识的不一致性突出了进一步改进、验证和临床整合的必要性。科大讯飞Spark的一致性较低,表明训练数据和底层算法存在潜在差异,凸显了人工智能生成的医疗建议的可变性。为了优化人工智能在骨科护理中的作用,未来的研究应侧重于增强人工智能与医疗指南的一致性,提高模型透明度,并整合医生监督,以确保安全有效的临床应用。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Multidisciplinary Healthcare
Journal of Multidisciplinary Healthcare Nursing-General Nursing
CiteScore
4.60
自引率
3.00%
发文量
287
审稿时长
16 weeks
期刊介绍: The Journal of Multidisciplinary Healthcare (JMDH) aims to represent and publish research in healthcare areas delivered by practitioners of different disciplines. This includes studies and reviews conducted by multidisciplinary teams as well as research which evaluates or reports the results or conduct of such teams or healthcare processes in general. The journal covers a very wide range of areas and we welcome submissions from practitioners at all levels and from all over the world. Good healthcare is not bounded by person, place or time and the journal aims to reflect this. The JMDH is published as an open-access journal to allow this wide range of practical, patient relevant research to be immediately available to practitioners who can access and use it immediately upon publication.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信