Microsoft Copilot提供比ChatGPT和谷歌Gemini更准确可靠的前交叉韧带损伤和修复信息然而，没有资源是整体上最好的

Q3 Medicine

Arthroscopy Sports Medicine and Rehabilitation Pub Date : 2025-04-01 DOI:10.1016/j.asmr.2024.101043

Suhasini Gupta B.S. , Rae Tarapore M.D. , Brett Haislup M.D. , Allison Fillar M.D.

{"title":"Microsoft Copilot提供比ChatGPT和谷歌Gemini更准确可靠的前交叉韧带损伤和修复信息然而，没有资源是整体上最好的","authors":"Suhasini Gupta B.S. , Rae Tarapore M.D. , Brett Haislup M.D. , Allison Fillar M.D.","doi":"10.1016/j.asmr.2024.101043","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><div>To analyze and compare the quality, accuracy, and readability of information regarding anterior cruciate ligament (ACL) injury and reconstruction provided by various artificial intelligence AI interfaces (Google Gemini, Microsoft Copilot, and OpenAI ChatGPT).</div></div><div><h3>Methods</h3><div>Twenty questions regarding ACL reconstruction were inputted into ChatGPT 3.5, Gemini, and the more precise subinterface within Copilot and were categorized on the basis of the Rothwell criteria into Fact, Policy, and Value. The answers generated were analyzed using the DISCERN scale, JAMA benchmark criteria, and Flesch-Kincaid Reading Ease Score and Grade Level. The citations provided by Gemini and Copilot were further categorized by source of citation.</div></div><div><h3>Results</h3><div>All 3 AI interfaces generated DISCERN scores (≥50) demonstrating “good” quality of information except for Policy and Value by Copilot which were scored as “excellent” (≥70). The information provided by Copilot demonstrated greater reliability, with a JAMA benchmark criterion of 3 (of 4) as compared with Gemini (1) and ChatGPT (0). In terms of readability, the Flesch-Kincaid Reading Ease Score scores of all 3 sources were <30, apart from Fact by Copilot (31.9) demonstrating very complex answers. Similarly, all Flesch-Kincaid Grade Level scores were >13, indicating a minimum readability level of college level or college graduate. Finally, both Copilot and Gemini had a majority of references provided by journals (65.6% by Gemini and 75.4% by Copilot), followed by academic sources, whereas Copilot provided a greater number of overall citations (163) as compared with Gemini (64).</div></div><div><h3>Conclusions</h3><div>Microsoft Copilot was a better resource for patients to learn about ACL injuries and reconstruction compared with Google Gemini or OpenAI ChatGPT in terms of quality of information, reliability, and readability. The answers provided by LLMs are highly complex and no resource was overall the best.</div></div><div><h3>Clinical Relevance</h3><div>As artificial intelligence models continually evolve and demonstrate increased potential for answering complex surgical questions, it is important to investigate the quality and usefulness of the responses for patients. Although these resources may be helpful, they should not be used as a substitute for any discussions with health care providers.</div></div>","PeriodicalId":34631,"journal":{"name":"Arthroscopy Sports Medicine and Rehabilitation","volume":"7 2","pages":"Article 101043"},"PeriodicalIF":0.0000,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Microsoft Copilot Provides More Accurate and Reliable Information About Anterior Cruciate Ligament Injury and Repair Than ChatGPT and Google Gemini; However, No Resource Was Overall the Best\",\"authors\":\"Suhasini Gupta B.S. , Rae Tarapore M.D. , Brett Haislup M.D. , Allison Fillar M.D.\",\"doi\":\"10.1016/j.asmr.2024.101043\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Purpose</h3><div>To analyze and compare the quality, accuracy, and readability of information regarding anterior cruciate ligament (ACL) injury and reconstruction provided by various artificial intelligence AI interfaces (Google Gemini, Microsoft Copilot, and OpenAI ChatGPT).</div></div><div><h3>Methods</h3><div>Twenty questions regarding ACL reconstruction were inputted into ChatGPT 3.5, Gemini, and the more precise subinterface within Copilot and were categorized on the basis of the Rothwell criteria into Fact, Policy, and Value. The answers generated were analyzed using the DISCERN scale, JAMA benchmark criteria, and Flesch-Kincaid Reading Ease Score and Grade Level. The citations provided by Gemini and Copilot were further categorized by source of citation.</div></div><div><h3>Results</h3><div>All 3 AI interfaces generated DISCERN scores (≥50) demonstrating “good” quality of information except for Policy and Value by Copilot which were scored as “excellent” (≥70). The information provided by Copilot demonstrated greater reliability, with a JAMA benchmark criterion of 3 (of 4) as compared with Gemini (1) and ChatGPT (0). In terms of readability, the Flesch-Kincaid Reading Ease Score scores of all 3 sources were <30, apart from Fact by Copilot (31.9) demonstrating very complex answers. Similarly, all Flesch-Kincaid Grade Level scores were >13, indicating a minimum readability level of college level or college graduate. Finally, both Copilot and Gemini had a majority of references provided by journals (65.6% by Gemini and 75.4% by Copilot), followed by academic sources, whereas Copilot provided a greater number of overall citations (163) as compared with Gemini (64).</div></div><div><h3>Conclusions</h3><div>Microsoft Copilot was a better resource for patients to learn about ACL injuries and reconstruction compared with Google Gemini or OpenAI ChatGPT in terms of quality of information, reliability, and readability. The answers provided by LLMs are highly complex and no resource was overall the best.</div></div><div><h3>Clinical Relevance</h3><div>As artificial intelligence models continually evolve and demonstrate increased potential for answering complex surgical questions, it is important to investigate the quality and usefulness of the responses for patients. Although these resources may be helpful, they should not be used as a substitute for any discussions with health care providers.</div></div>\",\"PeriodicalId\":34631,\"journal\":{\"name\":\"Arthroscopy Sports Medicine and Rehabilitation\",\"volume\":\"7 2\",\"pages\":\"Article 101043\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-04-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Arthroscopy Sports Medicine and Rehabilitation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2666061X2400186X\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"Medicine\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Arthroscopy Sports Medicine and Rehabilitation","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2666061X2400186X","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Medicine","Score":null,"Total":0}

引用次数: 0

摘要

目的分析比较各种人工智能AI接口（谷歌Gemini、Microsoft Copilot、OpenAI ChatGPT）提供的前交叉韧带（ACL）损伤及重建信息的质量、准确性和可读性。方法将20个关于ACL重建的问题输入到ChatGPT 3.5、Gemini和Copilot中更精确的子界面中，并根据Rothwell标准分为Fact、Policy和Value。生成的答案使用DISCERN量表、JAMA基准标准和Flesch-Kincaid阅读轻松评分和年级水平进行分析。Gemini和Copilot提供的引文进一步按引文来源分类。结果除Copilot的Policy和Value得分为“优秀”（≥70）外，3个AI界面的DISCERN得分均为“良好”。与Gemini(1)和ChatGPT(0)相比，Copilot提供的信息显示出更高的可靠性，其JAMA基准标准为3分（满分4分）。在可读性方面，除了Fact by Copilot（31.9）显示非常复杂的答案外，所有3个来源的Flesch-Kincaid Reading Ease Score得分均为30分。同样，所有的Flesch-Kincaid Grade Level分数都是>；13，表示最低的可读性水平为大学水平或大学毕业生。最后，Copilot和Gemini都有期刊提供的大部分参考文献（Gemini占65.6%，Copilot占75.4%），其次是学术来源，而Copilot提供的总引用数（163）高于Gemini（64）。结论与谷歌Gemini或OpenAI ChatGPT相比，microsoft Copilot在信息质量、可靠性和可读性方面为患者了解ACL损伤及重建提供了更好的资源。法学硕士提供的答案非常复杂，没有任何资源是整体上最好的。随着人工智能模型的不断发展，并在回答复杂的外科问题方面显示出越来越大的潜力，研究患者反应的质量和有用性是很重要的。尽管这些资源可能有所帮助，但它们不应取代与卫生保健提供者的任何讨论。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Microsoft Copilot Provides More Accurate and Reliable Information About Anterior Cruciate Ligament Injury and Repair Than ChatGPT and Google Gemini; However, No Resource Was Overall the Best

Purpose

To analyze and compare the quality, accuracy, and readability of information regarding anterior cruciate ligament (ACL) injury and reconstruction provided by various artificial intelligence AI interfaces (Google Gemini, Microsoft Copilot, and OpenAI ChatGPT).

Methods

Twenty questions regarding ACL reconstruction were inputted into ChatGPT 3.5, Gemini, and the more precise subinterface within Copilot and were categorized on the basis of the Rothwell criteria into Fact, Policy, and Value. The answers generated were analyzed using the DISCERN scale, JAMA benchmark criteria, and Flesch-Kincaid Reading Ease Score and Grade Level. The citations provided by Gemini and Copilot were further categorized by source of citation.

Results

All 3 AI interfaces generated DISCERN scores (≥50) demonstrating “good” quality of information except for Policy and Value by Copilot which were scored as “excellent” (≥70). The information provided by Copilot demonstrated greater reliability, with a JAMA benchmark criterion of 3 (of 4) as compared with Gemini (1) and ChatGPT (0). In terms of readability, the Flesch-Kincaid Reading Ease Score scores of all 3 sources were <30, apart from Fact by Copilot (31.9) demonstrating very complex answers. Similarly, all Flesch-Kincaid Grade Level scores were >13, indicating a minimum readability level of college level or college graduate. Finally, both Copilot and Gemini had a majority of references provided by journals (65.6% by Gemini and 75.4% by Copilot), followed by academic sources, whereas Copilot provided a greater number of overall citations (163) as compared with Gemini (64).

Conclusions

Microsoft Copilot was a better resource for patients to learn about ACL injuries and reconstruction compared with Google Gemini or OpenAI ChatGPT in terms of quality of information, reliability, and readability. The answers provided by LLMs are highly complex and no resource was overall the best.

Clinical Relevance

As artificial intelligence models continually evolve and demonstrate increased potential for answering complex surgical questions, it is important to investigate the quality and usefulness of the responses for patients. Although these resources may be helpful, they should not be used as a substitute for any discussions with health care providers.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊