Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology.

IF 2.6 4区医学 Q2 OPHTHALMOLOGY

Ocular Immunology and Inflammation Pub Date : 2024-10-01 Epub Date: 2023-10-13 DOI:10.1080/09273948.2023.2266730

William Rojas-Carabali, Alok Sen, Aniruddha Agarwal, Gavin Tan, Carol Y Cheung, Andres Rousselot, Rajdeep Agrawal, Renee Liu, Carlos Cifuentes-González, Tobias Elze, John H Kempen, Lucia Sobrin, Quan Dong Nguyen, Alejandra de-la-Torre, Bernett Lee, Vishali Gupta, Rupesh Agrawal

{"title":"Chatbots Vs. Human Experts: Evaluating Diagnostic Performance of Chatbots in Uveitis and the Perspectives on AI Adoption in Ophthalmology.","authors":"William Rojas-Carabali, Alok Sen, Aniruddha Agarwal, Gavin Tan, Carol Y Cheung, Andres Rousselot, Rajdeep Agrawal, Renee Liu, Carlos Cifuentes-González, Tobias Elze, John H Kempen, Lucia Sobrin, Quan Dong Nguyen, Alejandra de-la-Torre, Bernett Lee, Vishali Gupta, Rupesh Agrawal","doi":"10.1080/09273948.2023.2266730","DOIUrl":null,"url":null,"abstract":"Purpose: To assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians' perception about utilizing artificial intelligence (AI) in ophthalmology practice.Methods: Six cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done.Results: Uveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools.Conclusions: ChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice.","PeriodicalId":19406,"journal":{"name":"Ocular Immunology and Inflammation","volume":" ","pages":"1591-1598"},"PeriodicalIF":2.6000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ocular Immunology and Inflammation","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1080/09273948.2023.2266730","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/10/13 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: To assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians' perception about utilizing artificial intelligence (AI) in ophthalmology practice.

Methods: Six cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done.

Results: Uveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools.

Conclusions: ChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice.

查看原文本刊更多论文

聊天机器人与人类专家：评估聊天机器人在葡萄膜炎中的诊断性能以及人工智能在眼科应用的前景。

目的：与著名的葡萄膜炎专家相比，评估两个聊天机器人ChatGPT和Glass在葡萄膜炎诊断中的诊断性能，并评估临床医生对在眼科实践中使用人工智能的看法。方法：向葡萄膜炎专家、ChatGPT（3.5版和4.0版）和Glass 1.0提交6例病例，并分析诊断准确性。此外，还对情绪、使用基于人工智能的工具的信心以及在临床实践中使用此类工具的可能性进行了调查。结果：葡萄膜炎专家准确诊断了所有病例（100%），而ChatGPT的诊断成功率为66%，Glass 1.0的诊断成功度为33%。大多数与会者对将人工智能应用于眼科实践感到兴奋或乐观。年龄较大和教育水平较高与采用基于人工智能的工具的倾向增加呈正相关。结论：ChatGPT在葡萄膜炎病例中表现出了良好的诊断能力，眼科医生对将人工智能融入临床实践表现出了热情。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Ocular Immunology and Inflammation 医学-眼科学

CiteScore

6.20

自引率

15.20%

发文量

285

审稿时长

6-12 weeks

期刊介绍： Ocular Immunology & Inflammation ranks 18 out of 59 in the Ophthalmology Category.Ocular Immunology and Inflammation is a peer-reviewed, scientific publication that welcomes the submission of original, previously unpublished manuscripts directed to ophthalmologists and vision scientists. Published bimonthly, the journal provides an international medium for basic and clinical research reports on the ocular inflammatory response and its control by the immune system. The journal publishes original research papers, case reports, reviews, letters to the editor, meeting abstracts, and invited editorials.