Luis A Hernández-Flores, José B López-Martínez, Jesús J Rosales-de-la-Rosa, Daniel Aillaud-De-Uriarte, Sergio Contreras-Garduño, Rubén Cortés-González
{"title":"具有挑战性的肿瘤病例评估：ChatGPT， Gemini和多学科肿瘤委员会之间的比较分析。","authors":"Luis A Hernández-Flores, José B López-Martínez, Jesús J Rosales-de-la-Rosa, Daniel Aillaud-De-Uriarte, Sergio Contreras-Garduño, Rubén Cortés-González","doi":"10.1002/jso.28121","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>Since its introduction in 2022, public-access conversational AI, exemplified by ChatGPT and Gemini, has been increasingly utilized in medical decision-making, though its impact is questionable. This study aims to evaluate its efficacy in assessing complex oncologic cases compared to a multidisciplinary tumor board (MTB) comprising experts from various specialties.</p><p><strong>Methods: </strong>A 2-year retrospective analysis was conducted on 98 oncological cases at a reference medical center in Mexico City. A MTB comprising surgical oncologists, medical oncologists, radio-oncologists, pathologists, among others, reviewed and discussed each case to determine management strategies. We evaluated four key decision points, dichotomized as either affirmative or negative: the need for new imaging studies, radiation therapy, chemotherapy, and surgery. Comprehensive medical documentation accompanied each case. We then compared AI's decisions with those of the MTB using the same criteria and conducted a Cohen's Kappa test to assess agreement.</p><p><strong>Results: </strong>Agreement between ChatGPT (4o) and Gemini (1.5 Flash), and the MTB ranged from none to slight for additional imaging studies (Gemini: κ = 0.100, p = 0.087; ChatGPT 4o: κ = 0.024, p = 0.592) and chemotherapy (Gemini: κ = 0.089, p = 0.316; ChatGPT 4o: κ = 0.336, p = 0.001). Moderate agreement was observed for decisions regarding surgery (Gemini: κ = 0.194, p = 0.046; ChatGPT 4o: κ = 0.467, p = < 0.001) and radiotherapy (Gemini: κ = 0.214, p = 0.012; ChatGPT 4o: κ = 0.525, p = < 0.001).</p><p><strong>Conclusions: </strong>Both ChatGPT and Gemini showed moderate agreement with the multidisciplinary tumor board on decisions regarding surgery and radiotherapy. ChatGPT also showed moderate agreement in chemotherapy, but further assessment is needed for other interventions. ChatGPT proved to be superior to Gemini in most key points. The potential of these public access AI in oncology warrants continued exploration to refine its utility in clinical practice.</p>","PeriodicalId":17111,"journal":{"name":"Journal of Surgical Oncology","volume":" ","pages":""},"PeriodicalIF":2.0000,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Assessment of Challenging Oncologic Cases: A Comparative Analysis Between ChatGPT, Gemini, and a Multidisciplinary Tumor Board.\",\"authors\":\"Luis A Hernández-Flores, José B López-Martínez, Jesús J Rosales-de-la-Rosa, Daniel Aillaud-De-Uriarte, Sergio Contreras-Garduño, Rubén Cortés-González\",\"doi\":\"10.1002/jso.28121\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Introduction: </strong>Since its introduction in 2022, public-access conversational AI, exemplified by ChatGPT and Gemini, has been increasingly utilized in medical decision-making, though its impact is questionable. This study aims to evaluate its efficacy in assessing complex oncologic cases compared to a multidisciplinary tumor board (MTB) comprising experts from various specialties.</p><p><strong>Methods: </strong>A 2-year retrospective analysis was conducted on 98 oncological cases at a reference medical center in Mexico City. A MTB comprising surgical oncologists, medical oncologists, radio-oncologists, pathologists, among others, reviewed and discussed each case to determine management strategies. We evaluated four key decision points, dichotomized as either affirmative or negative: the need for new imaging studies, radiation therapy, chemotherapy, and surgery. Comprehensive medical documentation accompanied each case. We then compared AI's decisions with those of the MTB using the same criteria and conducted a Cohen's Kappa test to assess agreement.</p><p><strong>Results: </strong>Agreement between ChatGPT (4o) and Gemini (1.5 Flash), and the MTB ranged from none to slight for additional imaging studies (Gemini: κ = 0.100, p = 0.087; ChatGPT 4o: κ = 0.024, p = 0.592) and chemotherapy (Gemini: κ = 0.089, p = 0.316; ChatGPT 4o: κ = 0.336, p = 0.001). Moderate agreement was observed for decisions regarding surgery (Gemini: κ = 0.194, p = 0.046; ChatGPT 4o: κ = 0.467, p = < 0.001) and radiotherapy (Gemini: κ = 0.214, p = 0.012; ChatGPT 4o: κ = 0.525, p = < 0.001).</p><p><strong>Conclusions: </strong>Both ChatGPT and Gemini showed moderate agreement with the multidisciplinary tumor board on decisions regarding surgery and radiotherapy. ChatGPT also showed moderate agreement in chemotherapy, but further assessment is needed for other interventions. ChatGPT proved to be superior to Gemini in most key points. The potential of these public access AI in oncology warrants continued exploration to refine its utility in clinical practice.</p>\",\"PeriodicalId\":17111,\"journal\":{\"name\":\"Journal of Surgical Oncology\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":2.0000,\"publicationDate\":\"2025-02-12\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Surgical Oncology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1002/jso.28121\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"ONCOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Surgical Oncology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1002/jso.28121","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0
摘要
导语：自2022年推出以来，以ChatGPT和Gemini为代表的公共访问会话式人工智能已越来越多地用于医疗决策，尽管其影响值得怀疑。本研究旨在评估其在评估复杂肿瘤病例方面的效果，并与由不同专业专家组成的多学科肿瘤委员会（MTB）进行比较。方法：对墨西哥城某参考医疗中心98例肿瘤病例进行2年回顾性分析。由外科肿瘤学家、内科肿瘤学家、放射肿瘤学家、病理学家等组成的MTB审查和讨论了每个病例，以确定管理策略。我们评估了四个关键的决策点，分为肯定或否定：需要新的影像学研究，放射治疗，化疗和手术。每个病例都附有全面的医疗记录。然后，我们使用相同的标准将人工智能的决策与MTB的决策进行比较，并进行科恩卡帕测试来评估一致性。结果：ChatGPT（40）和Gemini （1.5 Flash）之间的一致性，以及额外影像学研究的MTB从无到轻微不等(Gemini: κ = 0.100, p = 0.087；ChatGPT 4 o:κ= 0.024,p = 0.592)和化疗(双子:κ= 0.089,p = 0.316;ChatGPT 40: κ = 0.336, p = 0.001)。在手术决策方面，观察到中度一致性(Gemini: κ = 0.194, p = 0.046；ChatGPT 40: κ = 0.467, p =结论：ChatGPT和Gemini在多学科肿瘤委员会关于手术和放疗的决定上表现出中度一致。ChatGPT在化疗中也显示出适度的一致性，但需要进一步评估其他干预措施。事实证明，ChatGPT在大多数关键点上都优于Gemini。这些公共访问人工智能在肿瘤学中的潜力值得继续探索，以完善其在临床实践中的应用。
Assessment of Challenging Oncologic Cases: A Comparative Analysis Between ChatGPT, Gemini, and a Multidisciplinary Tumor Board.
Introduction: Since its introduction in 2022, public-access conversational AI, exemplified by ChatGPT and Gemini, has been increasingly utilized in medical decision-making, though its impact is questionable. This study aims to evaluate its efficacy in assessing complex oncologic cases compared to a multidisciplinary tumor board (MTB) comprising experts from various specialties.
Methods: A 2-year retrospective analysis was conducted on 98 oncological cases at a reference medical center in Mexico City. A MTB comprising surgical oncologists, medical oncologists, radio-oncologists, pathologists, among others, reviewed and discussed each case to determine management strategies. We evaluated four key decision points, dichotomized as either affirmative or negative: the need for new imaging studies, radiation therapy, chemotherapy, and surgery. Comprehensive medical documentation accompanied each case. We then compared AI's decisions with those of the MTB using the same criteria and conducted a Cohen's Kappa test to assess agreement.
Results: Agreement between ChatGPT (4o) and Gemini (1.5 Flash), and the MTB ranged from none to slight for additional imaging studies (Gemini: κ = 0.100, p = 0.087; ChatGPT 4o: κ = 0.024, p = 0.592) and chemotherapy (Gemini: κ = 0.089, p = 0.316; ChatGPT 4o: κ = 0.336, p = 0.001). Moderate agreement was observed for decisions regarding surgery (Gemini: κ = 0.194, p = 0.046; ChatGPT 4o: κ = 0.467, p = < 0.001) and radiotherapy (Gemini: κ = 0.214, p = 0.012; ChatGPT 4o: κ = 0.525, p = < 0.001).
Conclusions: Both ChatGPT and Gemini showed moderate agreement with the multidisciplinary tumor board on decisions regarding surgery and radiotherapy. ChatGPT also showed moderate agreement in chemotherapy, but further assessment is needed for other interventions. ChatGPT proved to be superior to Gemini in most key points. The potential of these public access AI in oncology warrants continued exploration to refine its utility in clinical practice.
期刊介绍：
The Journal of Surgical Oncology offers peer-reviewed, original papers in the field of surgical oncology and broadly related surgical sciences, including reports on experimental and laboratory studies. As an international journal, the editors encourage participation from leading surgeons around the world. The JSO is the representative journal for the World Federation of Surgical Oncology Societies. Publishing 16 issues in 2 volumes each year, the journal accepts Research Articles, in-depth Reviews of timely interest, Letters to the Editor, and invited Editorials. Guest Editors from the JSO Editorial Board oversee multiple special Seminars issues each year. These Seminars include multifaceted Reviews on a particular topic or current issue in surgical oncology, which are invited from experts in the field.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
