Diagnosis, treatment, and prevention of ankle sprains: Comparing free chatbot recommendations with clinical guidelines.

IF 1.9 3区医学 Q2 ORTHOPEDICS

Foot and Ankle Surgery Pub Date : 2024-12-13 DOI:10.1016/j.fas.2024.12.003

Friederike Eva Roch, Franziska Melanie Hahn, Katharina Jäckle, Marc-Pascal Meier, Hartmut Stinus, Wolfgang Lehmann, Ronny Perthel, Paul Jonathan Roch

{"title":"Diagnosis, treatment, and prevention of ankle sprains: Comparing free chatbot recommendations with clinical guidelines.","authors":"Friederike Eva Roch, Franziska Melanie Hahn, Katharina Jäckle, Marc-Pascal Meier, Hartmut Stinus, Wolfgang Lehmann, Ronny Perthel, Paul Jonathan Roch","doi":"10.1016/j.fas.2024.12.003","DOIUrl":null,"url":null,"abstract":"Background: Free chatbots powered by large language models offer lateral ankle sprains (LAS) treatment recommendations but lack scientific validation.Methods: The chatbots-Claude, Perplexity, and ChatGPT-were evaluated by comparing their responses to a questionnaire and their treatment algorithms against current clinical guidelines. Responses were graded on accuracy, conclusiveness, supplementary information, and incompleteness, and evaluated individually and collectively, with a 60 % pass threshold.Results: The collective analysis of the questionnaire showed Perplexity scored significantly higher than Claude and ChatGPT (p < 0.001). In the individual analysis, Perplexity provided significantly more supplementary information than the other chatbots (p < 0.001). All chatbots met the pass threshold. In the algorithm evaluation, ChatGPT scored significantly higher than the others (p = 0.023), with Perplexity below the pass threshold.Conclusions: Chatbots' recommendations generally aligned with current guidelines but sometimes missed crucial details. While they offer useful supplementary information, they cannot yet replace professional medical consultation or established guidelines.","PeriodicalId":48743,"journal":{"name":"Foot and Ankle Surgery","volume":" ","pages":""},"PeriodicalIF":1.9000,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Foot and Ankle Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.fas.2024.12.003","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ORTHOPEDICS","Score":null,"Total":0}

引用次数: 0

Abstract

Background: Free chatbots powered by large language models offer lateral ankle sprains (LAS) treatment recommendations but lack scientific validation.

Methods: The chatbots-Claude, Perplexity, and ChatGPT-were evaluated by comparing their responses to a questionnaire and their treatment algorithms against current clinical guidelines. Responses were graded on accuracy, conclusiveness, supplementary information, and incompleteness, and evaluated individually and collectively, with a 60 % pass threshold.

Results: The collective analysis of the questionnaire showed Perplexity scored significantly higher than Claude and ChatGPT (p < 0.001). In the individual analysis, Perplexity provided significantly more supplementary information than the other chatbots (p < 0.001). All chatbots met the pass threshold. In the algorithm evaluation, ChatGPT scored significantly higher than the others (p = 0.023), with Perplexity below the pass threshold.

Conclusions: Chatbots' recommendations generally aligned with current guidelines but sometimes missed crucial details. While they offer useful supplementary information, they cannot yet replace professional medical consultation or established guidelines.

查看原文本刊更多论文

踝关节扭伤的诊断、治疗和预防：比较免费聊天机器人推荐与临床指南。

背景：由大型语言模型驱动的免费聊天机器人提供外侧踝关节扭伤（LAS）治疗建议，但缺乏科学验证。方法：通过比较聊天机器人claude、Perplexity和chatgpt对问卷的回答以及它们针对当前临床指南的治疗算法，对它们进行评估。根据回答的准确性、结论性、补充信息和不完整性进行评分，并单独和集体评估，通过阈值为60 %。结果：对问卷的集体分析显示，Perplexity的得分明显高于Claude和ChatGPT （p ）。结论：聊天机器人的建议总体上与当前的指导方针一致，但有时会遗漏关键细节。虽然它们提供了有用的补充信息，但它们还不能取代专业的医疗咨询或既定的指导方针。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Foot and Ankle Surgery ORTHOPEDICS-

CiteScore

4.60

自引率

16.00%

发文量

202

期刊介绍： Foot and Ankle Surgery is essential reading for everyone interested in the foot and ankle and its disorders. The approach is broad and includes all aspects of the subject from basic science to clinical management. Problems of both children and adults are included, as is trauma and chronic disease. Foot and Ankle Surgery is the official journal of European Foot and Ankle Society. The aims of this journal are to promote the art and science of ankle and foot surgery, to publish peer-reviewed research articles, to provide regular reviews by acknowledged experts on common problems, and to provide a forum for discussion with letters to the Editors. Reviews of books are also published. Papers are invited for possible publication in Foot and Ankle Surgery on the understanding that the material has not been published elsewhere or accepted for publication in another journal and does not infringe prior copyright.