ChatGPT vs. surgeons on pancreatic cancer queries: accuracy & empathy evaluated by patients and experts

IF 2.7 3区医学 Q2 GASTROENTEROLOGY & HEPATOLOGY

Hpb Pub Date : 2025-03-01 DOI:10.1016/j.hpb.2024.11.012

Rosa Klotz , Thomas M. Pausch , Jörg Kaiser , Maximilian C. Joos , Rüdiger Hecktor , Azaz Ahmed , Colette Dörr-Harim , Arianeb Mehrabi , Martin Loos , Susanne Roth , Christoph W. Michalski , Christoph Kahlert

{"title":"ChatGPT vs. surgeons on pancreatic cancer queries: accuracy & empathy evaluated by patients and experts","authors":"Rosa Klotz , Thomas M. Pausch , Jörg Kaiser , Maximilian C. Joos , Rüdiger Hecktor , Azaz Ahmed , Colette Dörr-Harim , Arianeb Mehrabi , Martin Loos , Susanne Roth , Christoph W. Michalski , Christoph Kahlert","doi":"10.1016/j.hpb.2024.11.012","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Artificial intelligence (AI) offers potential support in patient–clinician interactions, but its impact on such communication remains unexplored.</div></div><div><h3>Methods</h3><div>In this study, ChatGPT was compared with two pancreatic surgeons in responding to ten pancreatic cancer surgery-related questions, co-designed with the Patient Advisory Board of the Surgical Society's Study Center. A blind evaluation of these responses, considering content congruency and clarity for non-specialists, was conducted by patients and surgeons.</div></div><div><h3>Results</h3><div>From June 23 to July 21, 2023, 24 patients and 25 surgeons participated, of which eleven patients and ten surgeons completed the survey in full. Utilizing a quantitative scale from 1 (strong-disagreement) to 5 (full-agreement), consensus was observed among patients and specialists concerning the content delivered by ChatGPT. The metrics for comprehensibility to a non-specialist audience consistently showed positive reception. In the evaluation of empathetic resonance, ChatGPT's responses mirrored those of the surgeons in the patient's view. A significant proportion ranked Surgeon 1's contributions foremost, followed closely by ChatGPT.</div></div><div><h3>Discussion</h3><div>This study demonstrates that surgeons and ChatGPT answer common queries from patients regarding pancreatic cancer surgery comparable regarding reliability, lay comprehension and empathy as evaluated by patients and surgical experts. These findings highlight the potential of AI in enhancing patient-provider interactions.</div></div>","PeriodicalId":13229,"journal":{"name":"Hpb","volume":"27 3","pages":"Pages 311-317"},"PeriodicalIF":2.7000,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Hpb","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1365182X24024274","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Artificial intelligence (AI) offers potential support in patient–clinician interactions, but its impact on such communication remains unexplored.

Methods

In this study, ChatGPT was compared with two pancreatic surgeons in responding to ten pancreatic cancer surgery-related questions, co-designed with the Patient Advisory Board of the Surgical Society's Study Center. A blind evaluation of these responses, considering content congruency and clarity for non-specialists, was conducted by patients and surgeons.

Results

From June 23 to July 21, 2023, 24 patients and 25 surgeons participated, of which eleven patients and ten surgeons completed the survey in full. Utilizing a quantitative scale from 1 (strong-disagreement) to 5 (full-agreement), consensus was observed among patients and specialists concerning the content delivered by ChatGPT. The metrics for comprehensibility to a non-specialist audience consistently showed positive reception. In the evaluation of empathetic resonance, ChatGPT's responses mirrored those of the surgeons in the patient's view. A significant proportion ranked Surgeon 1's contributions foremost, followed closely by ChatGPT.

Discussion

This study demonstrates that surgeons and ChatGPT answer common queries from patients regarding pancreatic cancer surgery comparable regarding reliability, lay comprehension and empathy as evaluated by patients and surgical experts. These findings highlight the potential of AI in enhancing patient-provider interactions.

查看原文本刊更多论文

胰腺癌查询的ChatGPT vs.外科医生：由患者和专家评估的准确性和同理心。

背景：人工智能（AI）为医患互动提供了潜在的支持，但其对这种交流的影响仍未被探索。方法：在本研究中，与外科学会研究中心患者咨询委员会共同设计的ChatGPT与两位胰腺外科医生对10个胰腺癌手术相关问题的回答进行了比较。患者和外科医生对这些反应进行了盲评估，考虑到内容的一致性和非专业人员的清晰度。结果：2023年6月23日至7月21日，共有24名患者和25名外科医生参与调查，其中11名患者和10名外科医生完整完成了调查。利用从1（强烈反对）到5（完全同意）的定量量表，观察到患者和专家对ChatGPT提供的内容的共识。非专业受众的可理解性指标始终显示出积极的接受度。在共情共振的评估中，ChatGPT的反应反映了外科医生在患者眼中的反应。很大比例的人认为外科医生1的贡献最重要，其次是ChatGPT。讨论：本研究表明，经患者和外科专家评估，外科医生和ChatGPT在可靠性、外行理解和移情方面回答了患者关于胰腺癌手术的常见问题。这些发现突出了人工智能在加强医患互动方面的潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Hpb GASTROENTEROLOGY & HEPATOLOGY-SURGERY

CiteScore

5.60

自引率

3.40%

发文量

244

审稿时长

57 days

期刊介绍： HPB is an international forum for clinical, scientific and educational communication. Twelve issues a year bring the reader leading articles, expert reviews, original articles, images, editorials, and reader correspondence encompassing all aspects of benign and malignant hepatobiliary disease and its management. HPB features relevant aspects of clinical and translational research and practice. Specific areas of interest include HPB diseases encountered globally by clinical practitioners in this specialist field of gastrointestinal surgery. The journal addresses the challenges faced in the management of cancer involving the liver, biliary system and pancreas. While surgical oncology represents a large part of HPB practice, submission of manuscripts relating to liver and pancreas transplantation, the treatment of benign conditions such as acute and chronic pancreatitis, and those relating to hepatobiliary infection and inflammation are also welcomed. There will be a focus on developing a multidisciplinary approach to diagnosis and treatment with endoscopic and laparoscopic approaches, radiological interventions and surgical techniques being strongly represented. HPB welcomes submission of manuscripts in all these areas and in scientific focused research that has clear clinical relevance to HPB surgical practice. HPB aims to help its readers - surgeons, physicians, radiologists and basic scientists - to develop their knowledge and practice. HPB will be of interest to specialists involved in the management of hepatobiliary and pancreatic disease however will also inform those working in related fields. Abstracted and Indexed in: MEDLINE® EMBASE PubMed Science Citation Index Expanded Academic Search (EBSCO) HPB is owned by the International Hepato-Pancreato-Biliary Association (IHPBA) and is also the official Journal of the American Hepato-Pancreato-Biliary Association (AHPBA), the Asian-Pacific Hepato Pancreatic Biliary Association (A-PHPBA) and the European-African Hepato-Pancreatic Biliary Association (E-AHPBA).