评估胃肠道手术抗凝管理中使用和不使用检索增强型 ChatGPT4 的情况。

IF 2.1 Q3 GASTROENTEROLOGY & HEPATOLOGY
Annals of Gastroenterology Pub Date : 2024-09-01 Epub Date: 2024-08-19 DOI:10.20524/aog.2024.0907
Sheza Malik, Himal Kharel, Dushyant S Dahiya, Hassam Ali, Hanna Blaney, Achintya Singh, Jahnvi Dhar, Abhilash Perisetti, Antonio Facciorusso, Saurabh Chandan, Babu P Mohan
{"title":"评估胃肠道手术抗凝管理中使用和不使用检索增强型 ChatGPT4 的情况。","authors":"Sheza Malik, Himal Kharel, Dushyant S Dahiya, Hassam Ali, Hanna Blaney, Achintya Singh, Jahnvi Dhar, Abhilash Perisetti, Antonio Facciorusso, Saurabh Chandan, Babu P Mohan","doi":"10.20524/aog.2024.0907","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>In view of the growing complexity of managing anticoagulation for patients undergoing gastrointestinal (GI) procedures, this study evaluated ChatGPT-4's ability to provide accurate medical guidance, comparing it with its prior artificial intelligence (AI) models (ChatGPT-3.5) and the retrieval-augmented generation (RAG)-supported model (ChatGPT4-RAG).</p><p><strong>Methods: </strong>Thirty-six anticoagulation-related questions, based on professional guidelines, were answered by ChatGPT-4. Nine gastroenterologists assessed these responses for accuracy and relevance. ChatGPT-4's performance was also compared to that of ChatGPT-3.5 and ChatGPT4-RAG. Additionally, a survey was conducted to understand gastroenterologists' perceptions of ChatGPT-4.</p><p><strong>Results: </strong>ChatGPT-4's responses showed significantly better accuracy and coherence compared to ChatGPT-3.5, with 30.5% of responses fully accurate and 47.2% generally accurate. ChatGPT4-RAG demonstrated a higher ability to integrate current information, achieving 75% full accuracy. Notably, for diagnostic and therapeutic esophagogastroduodenoscopy, 51.8% of responses were fully accurate; for endoscopic retrograde cholangiopancreatography with and without stent placement, 42.8% were fully accurate; and for diagnostic and therapeutic colonoscopy, 50% were fully accurate.</p><p><strong>Conclusions: </strong>ChatGPT4-RAG significantly advances anticoagulation management in endoscopic procedures, offering reliable and precise medical guidance. However, medicolegal considerations mean that a 75% full accuracy rate remains inadequate for independent clinical decision-making. AI may be more appropriately utilized to support and confirm clinicians' decisions, rather than replace them. Further evaluation is essential to maintain patient confidentiality and the integrity of the physician-patient relationship.</p>","PeriodicalId":7978,"journal":{"name":"Annals of Gastroenterology","volume":null,"pages":null},"PeriodicalIF":2.1000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11372545/pdf/","citationCount":"0","resultStr":"{\"title\":\"Assessing ChatGPT4 with and without retrieval-augmented generation in anticoagulation management for gastrointestinal procedures.\",\"authors\":\"Sheza Malik, Himal Kharel, Dushyant S Dahiya, Hassam Ali, Hanna Blaney, Achintya Singh, Jahnvi Dhar, Abhilash Perisetti, Antonio Facciorusso, Saurabh Chandan, Babu P Mohan\",\"doi\":\"10.20524/aog.2024.0907\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>In view of the growing complexity of managing anticoagulation for patients undergoing gastrointestinal (GI) procedures, this study evaluated ChatGPT-4's ability to provide accurate medical guidance, comparing it with its prior artificial intelligence (AI) models (ChatGPT-3.5) and the retrieval-augmented generation (RAG)-supported model (ChatGPT4-RAG).</p><p><strong>Methods: </strong>Thirty-six anticoagulation-related questions, based on professional guidelines, were answered by ChatGPT-4. Nine gastroenterologists assessed these responses for accuracy and relevance. ChatGPT-4's performance was also compared to that of ChatGPT-3.5 and ChatGPT4-RAG. Additionally, a survey was conducted to understand gastroenterologists' perceptions of ChatGPT-4.</p><p><strong>Results: </strong>ChatGPT-4's responses showed significantly better accuracy and coherence compared to ChatGPT-3.5, with 30.5% of responses fully accurate and 47.2% generally accurate. ChatGPT4-RAG demonstrated a higher ability to integrate current information, achieving 75% full accuracy. Notably, for diagnostic and therapeutic esophagogastroduodenoscopy, 51.8% of responses were fully accurate; for endoscopic retrograde cholangiopancreatography with and without stent placement, 42.8% were fully accurate; and for diagnostic and therapeutic colonoscopy, 50% were fully accurate.</p><p><strong>Conclusions: </strong>ChatGPT4-RAG significantly advances anticoagulation management in endoscopic procedures, offering reliable and precise medical guidance. However, medicolegal considerations mean that a 75% full accuracy rate remains inadequate for independent clinical decision-making. AI may be more appropriately utilized to support and confirm clinicians' decisions, rather than replace them. Further evaluation is essential to maintain patient confidentiality and the integrity of the physician-patient relationship.</p>\",\"PeriodicalId\":7978,\"journal\":{\"name\":\"Annals of Gastroenterology\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2024-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11372545/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annals of Gastroenterology\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.20524/aog.2024.0907\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/8/19 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q3\",\"JCRName\":\"GASTROENTEROLOGY & HEPATOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Gastroenterology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.20524/aog.2024.0907","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/19 0:00:00","PubModel":"Epub","JCR":"Q3","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}
引用次数: 0

摘要

背景:鉴于胃肠道(GI)手术患者的抗凝管理日益复杂,本研究评估了 ChatGPT-4 提供准确医疗指导的能力,并将其与之前的人工智能(AI)模型(ChatGPT-3.5)和检索增强生成(RAG)支持模型(ChatGPT4-RAG)进行了比较:ChatGPT-4 根据专业指南回答了 36 个与抗凝相关的问题。九位消化科专家对这些回答的准确性和相关性进行了评估。还将 ChatGPT-4 的性能与 ChatGPT-3.5 和 ChatGPT4-RAG 进行了比较。此外,还进行了一项调查,以了解消化科医生对 ChatGPT-4 的看法:结果:与 ChatGPT-3.5 相比,ChatGPT-4 的回答在准确性和连贯性方面有明显提高,30.5% 的回答完全准确,47.2% 的回答基本准确。ChatGPT4-RAG 整合当前信息的能力更强,完全准确率达到 75%。值得注意的是,对于诊断性和治疗性食管胃十二指肠镜检查,51.8% 的回答完全准确;对于带或不带支架的内镜逆行胰胆管造影术,42.8% 的回答完全准确;对于诊断性和治疗性结肠镜检查,50% 的回答完全准确:结论:ChatGPT4-RAG 为内窥镜手术中的抗凝管理提供了可靠而精确的医疗指导,大大提高了内窥镜手术中的抗凝管理水平。然而,出于医疗法律方面的考虑,75% 的完全准确率仍不足以用于独立的临床决策。人工智能可能更适合用来支持和确认临床医生的决策,而不是取代他们。进一步的评估对于维护病人的保密性和医患关系的完整性至关重要。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Assessing ChatGPT4 with and without retrieval-augmented generation in anticoagulation management for gastrointestinal procedures.

Background: In view of the growing complexity of managing anticoagulation for patients undergoing gastrointestinal (GI) procedures, this study evaluated ChatGPT-4's ability to provide accurate medical guidance, comparing it with its prior artificial intelligence (AI) models (ChatGPT-3.5) and the retrieval-augmented generation (RAG)-supported model (ChatGPT4-RAG).

Methods: Thirty-six anticoagulation-related questions, based on professional guidelines, were answered by ChatGPT-4. Nine gastroenterologists assessed these responses for accuracy and relevance. ChatGPT-4's performance was also compared to that of ChatGPT-3.5 and ChatGPT4-RAG. Additionally, a survey was conducted to understand gastroenterologists' perceptions of ChatGPT-4.

Results: ChatGPT-4's responses showed significantly better accuracy and coherence compared to ChatGPT-3.5, with 30.5% of responses fully accurate and 47.2% generally accurate. ChatGPT4-RAG demonstrated a higher ability to integrate current information, achieving 75% full accuracy. Notably, for diagnostic and therapeutic esophagogastroduodenoscopy, 51.8% of responses were fully accurate; for endoscopic retrograde cholangiopancreatography with and without stent placement, 42.8% were fully accurate; and for diagnostic and therapeutic colonoscopy, 50% were fully accurate.

Conclusions: ChatGPT4-RAG significantly advances anticoagulation management in endoscopic procedures, offering reliable and precise medical guidance. However, medicolegal considerations mean that a 75% full accuracy rate remains inadequate for independent clinical decision-making. AI may be more appropriately utilized to support and confirm clinicians' decisions, rather than replace them. Further evaluation is essential to maintain patient confidentiality and the integrity of the physician-patient relationship.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Annals of Gastroenterology
Annals of Gastroenterology GASTROENTEROLOGY & HEPATOLOGY-
CiteScore
4.30
自引率
0.00%
发文量
58
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信