ChatGPT能否为患者提供骨科相关问题的答案？ChatGPT与医疗保障人员的比较

IF 3.1 2区医学 Q2 PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH

Patient Education and Counseling Pub Date : 2025-09-17 DOI:10.1016/j.pec.2025.109333

Maud Jacobs , Walter van der Weegen , Hans Savelberg , Rob de Bie , Rienk van Beek , Joost Kuipers , Peter Pilot

{"title":"ChatGPT能否为患者提供骨科相关问题的答案？ChatGPT与医疗保障人员的比较","authors":"Maud Jacobs , Walter van der Weegen , Hans Savelberg , Rob de Bie , Rienk van Beek , Joost Kuipers , Peter Pilot","doi":"10.1016/j.pec.2025.109333","DOIUrl":null,"url":null,"abstract":"<div><h3>Introduction</h3><div>Patient Engagement Platforms, particularly chat functionalities, potentially improve communication but may also heighten workload, contributing to burnout among healthcare professionals. Natural Language Processing advancements, like ChatGPT and Med-PaLM, offer human-like responses to various questions, but concerns about their use in healthcare remain. This study evaluates whether Large Language Models can respond to patient questions as well as support staff in terms of quality and empathy.</div></div><div><h3>Methods</h3><div>In this cross-sectional study, 111 patient questions on lower limb arthroplasty, answered by support staff via an app, were selected. These questions were put into ChatGPT 3.5 to generate responses, and were collected on July 2 and 3, 2024. Two blinded healthcare professionals, an orthopaedic surgeon and an anesthetist, evaluated both the responses generated by ChatGPT and support staff, on quality, empathy, and risk of potential adverse events, selecting their preferred responses and identifying what they thought was ChatGPT’s response. A Patient Panel (n = 29) also assessed responses on empathy, preference, and source of the responses.</div></div><div><h3>Results</h3><div>Fifty questions were available for a comparative analysis between ChatGPT and support staff responses. No quality difference was found (p = 0.075) between ChatGPT and support staff, though ChatGPT was rated as more empathetic (p < 0.001). No difference was found between the two responses in the risk of incorrect treatment (p = 0.377). Physicians identified ChatGPT’s responses in 84–90 % of cases. The Patient Panel found ChatGPT to be more empathetic (p < 0.001) but showed no preference for ChatGPT (p = 0.086). Patients accurately identified ChatGPT’s responses in 34.5 % of cases (p = 0.005). Three ChatGPT responses showed high-risk errors.</div></div><div><h3>Conclusion</h3><div>This study shows ChatGPT generated high quality and empathetic responses to patient questions about lower limb arthroplasty. Further investigation is needed to optimize clinical use, but high appreciation for ChatGPT responses highlights the potential for use in clinical practice in the near future.</div></div>","PeriodicalId":49714,"journal":{"name":"Patient Education and Counseling","volume":"142 ","pages":"Article 109333"},"PeriodicalIF":3.1000,"publicationDate":"2025-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Can ChatGPT provide responses to patients for orthopaedic-related questions? A comparison between ChatGPT and medical support staff\",\"authors\":\"Maud Jacobs , Walter van der Weegen , Hans Savelberg , Rob de Bie , Rienk van Beek , Joost Kuipers , Peter Pilot\",\"doi\":\"10.1016/j.pec.2025.109333\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Introduction</h3><div>Patient Engagement Platforms, particularly chat functionalities, potentially improve communication but may also heighten workload, contributing to burnout among healthcare professionals. Natural Language Processing advancements, like ChatGPT and Med-PaLM, offer human-like responses to various questions, but concerns about their use in healthcare remain. This study evaluates whether Large Language Models can respond to patient questions as well as support staff in terms of quality and empathy.</div></div><div><h3>Methods</h3><div>In this cross-sectional study, 111 patient questions on lower limb arthroplasty, answered by support staff via an app, were selected. These questions were put into ChatGPT 3.5 to generate responses, and were collected on July 2 and 3, 2024. Two blinded healthcare professionals, an orthopaedic surgeon and an anesthetist, evaluated both the responses generated by ChatGPT and support staff, on quality, empathy, and risk of potential adverse events, selecting their preferred responses and identifying what they thought was ChatGPT’s response. A Patient Panel (n = 29) also assessed responses on empathy, preference, and source of the responses.</div></div><div><h3>Results</h3><div>Fifty questions were available for a comparative analysis between ChatGPT and support staff responses. No quality difference was found (p = 0.075) between ChatGPT and support staff, though ChatGPT was rated as more empathetic (p < 0.001). No difference was found between the two responses in the risk of incorrect treatment (p = 0.377). Physicians identified ChatGPT’s responses in 84–90 % of cases. The Patient Panel found ChatGPT to be more empathetic (p < 0.001) but showed no preference for ChatGPT (p = 0.086). Patients accurately identified ChatGPT’s responses in 34.5 % of cases (p = 0.005). Three ChatGPT responses showed high-risk errors.</div></div><div><h3>Conclusion</h3><div>This study shows ChatGPT generated high quality and empathetic responses to patient questions about lower limb arthroplasty. Further investigation is needed to optimize clinical use, but high appreciation for ChatGPT responses highlights the potential for use in clinical practice in the near future.</div></div>\",\"PeriodicalId\":49714,\"journal\":{\"name\":\"Patient Education and Counseling\",\"volume\":\"142 \",\"pages\":\"Article 109333\"},\"PeriodicalIF\":3.1000,\"publicationDate\":\"2025-09-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Patient Education and Counseling\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0738399125007001\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Patient Education and Counseling","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0738399125007001","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH","Score":null,"Total":0}

引用次数: 0

摘要

患者参与平台，特别是聊天功能，可能会改善沟通，但也可能增加工作量，导致医疗保健专业人员的倦怠。自然语言处理技术的进步，如ChatGPT和Med-PaLM，为各种问题提供了类似人类的回答，但人们仍然担心它们在医疗保健领域的应用。本研究评估了大型语言模型是否能够在质量和同理心方面回应患者问题以及支持人员。方法在横断面研究中，选择111例患者关于下肢关节置换术的问题，由支持人员通过应用程序回答。这些问题被放入ChatGPT 3.5中生成响应，并于2024年7月2日和3日收集。两名医疗保健专业人员，一名骨科医生和一名麻醉师，对ChatGPT和支持人员产生的反应进行了评估，包括质量、移情和潜在不良事件的风险，选择他们喜欢的反应，并确定他们认为ChatGPT的反应。患者小组（n = 29）也评估了对移情、偏好和反应来源的反应。结果在ChatGPT和支持人员的回答之间有50个问题可供比较分析。ChatGPT和支持人员之间没有发现质量差异（p = 0.075），尽管ChatGPT被评为更具同理心（p <； 0.001）。两组患者不正确治疗的风险无差异（p = 0.377）。医生在84 - 90%的病例中确定了ChatGPT的反应 %。患者小组发现ChatGPT更有同理心（p <； 0.001），但对ChatGPT没有偏好（p = 0.086）。34.5% %的患者准确识别出ChatGPT的反应（p = 0.005）。三个ChatGPT响应显示了高风险错误。结论该研究显示ChatGPT对患者关于下肢关节置换术的问题产生了高质量的共情反应。需要进一步的研究来优化临床应用，但对ChatGPT反应的高度评价突出了在不久的将来在临床实践中的应用潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Can ChatGPT provide responses to patients for orthopaedic-related questions? A comparison between ChatGPT and medical support staff

Introduction

Patient Engagement Platforms, particularly chat functionalities, potentially improve communication but may also heighten workload, contributing to burnout among healthcare professionals. Natural Language Processing advancements, like ChatGPT and Med-PaLM, offer human-like responses to various questions, but concerns about their use in healthcare remain. This study evaluates whether Large Language Models can respond to patient questions as well as support staff in terms of quality and empathy.

Methods

In this cross-sectional study, 111 patient questions on lower limb arthroplasty, answered by support staff via an app, were selected. These questions were put into ChatGPT 3.5 to generate responses, and were collected on July 2 and 3, 2024. Two blinded healthcare professionals, an orthopaedic surgeon and an anesthetist, evaluated both the responses generated by ChatGPT and support staff, on quality, empathy, and risk of potential adverse events, selecting their preferred responses and identifying what they thought was ChatGPT’s response. A Patient Panel (n = 29) also assessed responses on empathy, preference, and source of the responses.

Results

Fifty questions were available for a comparative analysis between ChatGPT and support staff responses. No quality difference was found (p = 0.075) between ChatGPT and support staff, though ChatGPT was rated as more empathetic (p < 0.001). No difference was found between the two responses in the risk of incorrect treatment (p = 0.377). Physicians identified ChatGPT’s responses in 84–90 % of cases. The Patient Panel found ChatGPT to be more empathetic (p < 0.001) but showed no preference for ChatGPT (p = 0.086). Patients accurately identified ChatGPT’s responses in 34.5 % of cases (p = 0.005). Three ChatGPT responses showed high-risk errors.

Conclusion

This study shows ChatGPT generated high quality and empathetic responses to patient questions about lower limb arthroplasty. Further investigation is needed to optimize clinical use, but high appreciation for ChatGPT responses highlights the potential for use in clinical practice in the near future.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Patient Education and Counseling 医学-公共卫生、环境卫生与职业卫生

CiteScore

5.60

自引率

11.40%

发文量

384

审稿时长

46 days

期刊介绍： Patient Education and Counseling is an interdisciplinary, international journal for patient education and health promotion researchers, managers and clinicians. The journal seeks to explore and elucidate the educational, counseling and communication models in health care. Its aim is to provide a forum for fundamental as well as applied research, and to promote the study of organizational issues involved with the delivery of patient education, counseling, health promotion services and training models in improving communication between providers and patients.