Does ChatGPT provide safe and reliable patient information related to hip and knee arthroplasty?

IF 2 4区医学 Q2 SURGERY

Canadian Journal of Surgery Pub Date : 2026-09-29 Print Date: 2026-09-01 DOI:10.1503/cjs.018325

Joëlle Deschênes-Bilodeau, Peter Staunton, Charles Desgagné, John Antoniou

{"title":"Does ChatGPT provide safe and reliable patient information related to hip and knee arthroplasty?","authors":"Joëlle Deschênes-Bilodeau, Peter Staunton, Charles Desgagné, John Antoniou","doi":"10.1503/cjs.018325","DOIUrl":null,"url":null,"abstract":"Background: Patients have been resorting to online content as their main source of medical knowledge, notably on orthopedic interventions. However, Web-based research findings present a reverse correlation between quality of content and popularity. We sought to evaluate whether ChatGPT could provide an alternative and safe source of medical information for arthroplasty patients.Methods: We gave 5 commonly Googled questions related to hip and knee arthroplasty to board-certified arthroplasty surgeons, fellows, and orthopedic surgery residents, as well as to ChatGPT 4.0. We anonymized all answers, which were then analyzed by an independent board-certified arthroplasty surgeon. We scored the answers for accuracy of content (6-point Likert scale) and completeness (3-point Likert scale), then compared the performance of all groups.Results: Among human responders, the mean accuracy grade was 75% (standard deviation [SD] 17%), with a mean completeness grade of 69% (SD 22%). The fellows represented the strongest subgroup, with 75% of their answers scored above 5/6 for accuracy (mean 82%, SD 16%). We found that all human-generated answers had a statistically significant correlation between accuracy and completeness. ChatGPT had a mean accuracy grade of 93% (SD 12%), with a mean completeness of 93% (SD 14%).Conclusion: ChatGPT appears to be a safe tool for patients to access general arthroplasty information online. It outperformed all human responders on both accuracy and completeness of answers. The strongest human responder group was the arthroplasty fellows. Further work is required to clarify the tool's performance against other easily accessible online patient information sources.","PeriodicalId":9573,"journal":{"name":"Canadian Journal of Surgery","volume":"69 5","pages":"E402-E407"},"PeriodicalIF":2.0000,"publicationDate":"2026-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13632767/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Canadian Journal of Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1503/cjs.018325","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/9/1 0:00:00","PubModel":"Print","JCR":"Q2","JCRName":"SURGERY","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Patients have been resorting to online content as their main source of medical knowledge, notably on orthopedic interventions. However, Web-based research findings present a reverse correlation between quality of content and popularity. We sought to evaluate whether ChatGPT could provide an alternative and safe source of medical information for arthroplasty patients.

Methods: We gave 5 commonly Googled questions related to hip and knee arthroplasty to board-certified arthroplasty surgeons, fellows, and orthopedic surgery residents, as well as to ChatGPT 4.0. We anonymized all answers, which were then analyzed by an independent board-certified arthroplasty surgeon. We scored the answers for accuracy of content (6-point Likert scale) and completeness (3-point Likert scale), then compared the performance of all groups.

Results: Among human responders, the mean accuracy grade was 75% (standard deviation [SD] 17%), with a mean completeness grade of 69% (SD 22%). The fellows represented the strongest subgroup, with 75% of their answers scored above 5/6 for accuracy (mean 82%, SD 16%). We found that all human-generated answers had a statistically significant correlation between accuracy and completeness. ChatGPT had a mean accuracy grade of 93% (SD 12%), with a mean completeness of 93% (SD 14%).

Conclusion: ChatGPT appears to be a safe tool for patients to access general arthroplasty information online. It outperformed all human responders on both accuracy and completeness of answers. The strongest human responder group was the arthroplasty fellows. Further work is required to clarify the tool's performance against other easily accessible online patient information sources.

查看原文本刊更多论文

ChatGPT是否提供安全可靠的与髋关节和膝关节置换术相关的患者信息？

背景：患者一直将在线内容作为其医学知识的主要来源，尤其是骨科干预方面。然而，基于网络的研究结果显示，内容质量与受欢迎程度之间存在反向相关性。我们试图评估ChatGPT是否可以为关节置换术患者提供另一种安全的医疗信息来源。方法：我们将5个与髋关节和膝关节置换术相关的常见谷歌问题提供给委员会认证的关节置换术医生、研究员和骨科外科住院医师，以及ChatGPT 4.0。我们将所有的答案匿名化，然后由独立的委员会认证的关节置换外科医生进行分析。我们对答案的内容准确性（6分李克特量表）和完整性（3分李克特量表）进行评分，然后比较所有组的表现。结果：在应答者中，平均准确度等级为75%（标准差[SD] 17%），平均完整性等级为69% （SD 22%）。研究员代表了最强的亚组，他们的答案中75%的准确性得分在5/6以上（平均82%，标准差16%）。我们发现，所有人工生成的答案在准确性和完整性之间具有统计上显著的相关性。ChatGPT的平均准确度等级为93% (SD为12%)，平均完整性为93% （SD为14%）。结论：ChatGPT似乎是一个安全的工具，为患者访问一般关节置换术的信息在线。它在答案的准确性和完整性方面都优于所有人类应答者。反应最强的是关节成形术患者。需要进一步的工作来澄清该工具与其他易于访问的在线患者信息源的性能。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Canadian Journal of Surgery 医学-外科

CiteScore

3.00

自引率

8.00%

发文量

120

审稿时长

6-12 weeks

期刊介绍： The mission of CJS is to contribute to the meaningful continuing medical education of Canadian surgical specialists, and to provide surgeons with an effective vehicle for the dissemination of observations in the areas of clinical and basic science research.