Assessing the diagnostic capacity of artificial intelligence chatbots for dysphonia types: Model development and validation.

IF 1.9 4区医学 Q2 OTORHINOLARYNGOLOGY

European Annals of Otorhinolaryngology-Head and Neck Diseases Pub Date : 2025-02-18 DOI:10.1016/j.anorl.2025.01.001

S Saeedi, M Aghajanzadeh

{"title":"Assessing the diagnostic capacity of artificial intelligence chatbots for dysphonia types: Model development and validation.","authors":"S Saeedi, M Aghajanzadeh","doi":"10.1016/j.anorl.2025.01.001","DOIUrl":null,"url":null,"abstract":"Purpose: User-friendly artificial intelligence (AI) chatbots are increasingly being explored to assist healthcare teams in their decision-making processes. As accurate diagnosis in all medical fields is vital in treatment planning, this research seeks to explore the function of two specific AI chatbots, ChatGPT and Perplexity AI, in distinguishing the various types of dysphonia (organic, functional, and neurological).Material and methods: In experiment 1, a script combining voice self-assessments plus the acoustic analysis, and in experiment 2, only the acoustic analysis of 37 dysphonic patients was fed into the ChatGPT and Perplexity AI chatbots specifying the type and asked to develop a complex AI-based model to determine dysphonia type. Then, the same process was redone with data from a sample of 27 other patients as a test.Results: Although ChatGPT could not analyze the data and only provided guidance, the Cohen's Kappa agreement between experts' diagnoses and Perplexity AI diagnoses in experiment 1 (P=0.773) and experiment 2 (P=0.067) lacked statistically significance.Conclusion: Regarding the preliminary poor performance of AI chatbots in differential diagnosis of dysphonia type, it is not currently recommended to use them in clinical settings. However, modifications in AI chatbots in the future might provide more promising results in determining the dysphonia type. Further research is needed to shed light on AI chatbots ability in voice clinics.","PeriodicalId":48834,"journal":{"name":"European Annals of Otorhinolaryngology-Head and Neck Diseases","volume":" ","pages":""},"PeriodicalIF":1.9000,"publicationDate":"2025-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Annals of Otorhinolaryngology-Head and Neck Diseases","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.anorl.2025.01.001","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"OTORHINOLARYNGOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: User-friendly artificial intelligence (AI) chatbots are increasingly being explored to assist healthcare teams in their decision-making processes. As accurate diagnosis in all medical fields is vital in treatment planning, this research seeks to explore the function of two specific AI chatbots, ChatGPT and Perplexity AI, in distinguishing the various types of dysphonia (organic, functional, and neurological).

Material and methods: In experiment 1, a script combining voice self-assessments plus the acoustic analysis, and in experiment 2, only the acoustic analysis of 37 dysphonic patients was fed into the ChatGPT and Perplexity AI chatbots specifying the type and asked to develop a complex AI-based model to determine dysphonia type. Then, the same process was redone with data from a sample of 27 other patients as a test.

Results: Although ChatGPT could not analyze the data and only provided guidance, the Cohen's Kappa agreement between experts' diagnoses and Perplexity AI diagnoses in experiment 1 (P=0.773) and experiment 2 (P=0.067) lacked statistically significance.

Conclusion: Regarding the preliminary poor performance of AI chatbots in differential diagnosis of dysphonia type, it is not currently recommended to use them in clinical settings. However, modifications in AI chatbots in the future might provide more promising results in determining the dysphonia type. Further research is needed to shed light on AI chatbots ability in voice clinics.

查看原文本刊更多论文

评估人工智能聊天机器人对发音障碍类型的诊断能力：模型开发与验证。

目的：用户友好的人工智能（AI）聊天机器人被越来越多地用于帮助医疗团队进行决策过程。由于所有医学领域的准确诊断对于治疗计划至关重要，因此本研究旨在探索两种特定的AI聊天机器人ChatGPT和Perplexity AI在区分各种类型的语音障碍（器质性，功能性和神经性）方面的功能。材料与方法：实验1将语音自评估与声学分析相结合的脚本，实验2仅将37例发声障碍患者的声学分析输入ChatGPT和Perplexity AI聊天机器人中指定类型，并要求开发基于人工智能的复杂模型来确定发声障碍类型。然后，从另外27名患者的样本数据中重新进行了同样的过程作为测试。结果：ChatGPT虽然不能对数据进行分析，只能提供指导，但在实验1 （P=0.773）和实验2 （P=0.067）中，专家诊断与Perplexity AI诊断之间的Cohen’s Kappa一致性缺乏统计学意义。结论：由于AI聊天机器人在语音障碍类型鉴别诊断中的初步表现较差，目前尚不建议在临床应用。然而，未来人工智能聊天机器人的改进可能会在确定语音障碍类型方面提供更有希望的结果。需要进一步研究人工智能聊天机器人在语音诊所的能力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

European Annals of Otorhinolaryngology-Head and Neck Diseases OTORHINOLARYNGOLOGY-

CiteScore

3.70

自引率

28.00%

发文量

审稿时长

12 days

期刊介绍： European Annals of Oto-rhino-laryngology, Head and Neck diseases heir of one of the oldest otorhinolaryngology journals in Europe is the official organ of the French Society of Otorhinolaryngology (SFORL) and the the International Francophone Society of Otorhinolaryngology (SIFORL). Today six annual issues provide original peer reviewed clinical and research articles, epidemiological studies, new methodological clinical approaches and review articles giving most up-to-date insights in all areas of otology, laryngology rhinology, head and neck surgery. The European Annals also publish the SFORL guidelines and recommendations.The journal is a unique two-armed publication: the European Annals (ANORL) is an English language well referenced online journal (e-only) whereas the Annales Françaises d’ORL (AFORL), mail-order paper and online edition in French language are aimed at the French-speaking community. French language teams must submit their articles in French to the AFORL site. Federating journal in its field, the European Annals has an Editorial board of experts with international reputation that allow to make an important contribution to communication on new research data and clinical practice by publishing high-quality articles.