A comprehensive evaluation of artificial intelligence-provided information on common ENT surgical procedures using the QAMAI tool.

IF 0.8 4区医学 Q3 OTORHINOLARYNGOLOGY

Journal of Laryngology and Otology Pub Date : 2025-09-01 DOI:10.1017/S0022215125103368

Mitat Selçuk Bozhöyük, Levent Yücel

{"title":"A comprehensive evaluation of artificial intelligence-provided information on common ENT surgical procedures using the QAMAI tool.","authors":"Mitat Selçuk Bozhöyük, Levent Yücel","doi":"10.1017/S0022215125103368","DOIUrl":null,"url":null,"abstract":"Objectives: This study aimed to evaluate the quality of information provided by artificial intelligence (AI) applications regarding ENT surgeries and usability for patients.Methods: ChatGPT 4.0, GEMINI 1.5 Flash, Copilot, Claude 3.5 Sonnet and DeepSeek-R1 were asked to provide detailed responses to patient-oriented questions about 15 ENT surgeries. Each AI application was queried three times, with a 3-day interval between each session. Two ENT specialists evaluated all responses using the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool.Results: Average QAMAI scores for each AI application were as follows: ChatGPT 4.0 (27.56 ± 1.20), GEMINI 1.5 Flash (26.24 ± 1.26), Copilot (26.84 ± 1.35), Claude 3.5 Sonnet (28.24 ± 0.77) and DeepSeek-R1 (28.13 ± 0.84). A statistically significant difference was found among the applications (p < 0.001). ICC analysis indicated high stability across evaluations conducted for all five AI applications (p < 0.001).Conclusion: AI has the potential to provide patients with accurate and consistent information about ENT surgeries, yet differences in QAMAI scores show that information quality varies between platforms.","PeriodicalId":16293,"journal":{"name":"Journal of Laryngology and Otology","volume":" ","pages":"1-6"},"PeriodicalIF":0.8000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Laryngology and Otology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1017/S0022215125103368","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"OTORHINOLARYNGOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Objectives: This study aimed to evaluate the quality of information provided by artificial intelligence (AI) applications regarding ENT surgeries and usability for patients.

Methods: ChatGPT 4.0, GEMINI 1.5 Flash, Copilot, Claude 3.5 Sonnet and DeepSeek-R1 were asked to provide detailed responses to patient-oriented questions about 15 ENT surgeries. Each AI application was queried three times, with a 3-day interval between each session. Two ENT specialists evaluated all responses using the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool.

Results: Average QAMAI scores for each AI application were as follows: ChatGPT 4.0 (27.56 ± 1.20), GEMINI 1.5 Flash (26.24 ± 1.26), Copilot (26.84 ± 1.35), Claude 3.5 Sonnet (28.24 ± 0.77) and DeepSeek-R1 (28.13 ± 0.84). A statistically significant difference was found among the applications (p < 0.001). ICC analysis indicated high stability across evaluations conducted for all five AI applications (p < 0.001).

Conclusion: AI has the potential to provide patients with accurate and consistent information about ENT surgeries, yet differences in QAMAI scores show that information quality varies between platforms.

查看原文本刊更多论文

使用QAMAI工具对人工智能提供的普通耳鼻喉外科手术信息进行综合评估。

目的：本研究旨在评估人工智能（AI）应用程序提供的有关耳鼻喉科手术的信息质量及其对患者的可用性。方法：采用ChatGPT 4.0、GEMINI 1.5 Flash、Copilot、Claude 3.5 Sonnet和DeepSeek-R1对15例耳鼻喉科手术的患者导向问题进行详细回答。每个AI应用程序被查询三次，每次会话之间间隔3天。两名耳鼻喉科专家使用医疗人工智能质量分析（QAMAI）工具对所有回复进行了评估。结果：各AI应用程序的QAMAI平均评分为ChatGPT 4.0(27.56±1.20),GEMINI 1.5 Flash(26.24±1.26),Copilot(26.84±1.35),Claude 3.5 Sonnet（28.24±0.77）和DeepSeek-R1（28.13±0.84）。应用程序之间的差异有统计学意义（p < 0.001）。ICC分析表明，对所有五种人工智能应用程序进行的评估具有很高的稳定性（p < 0.001）。结论：人工智能有可能为患者提供准确一致的耳鼻喉科手术信息，但QAMAI评分的差异表明，不同平台之间的信息质量存在差异。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Laryngology and Otology 医学-耳鼻喉科学

CiteScore

3.20

自引率

11.80%

发文量

593

审稿时长

3-6 weeks

期刊介绍： The Journal of Laryngology & Otology (JLO) is a leading, monthly journal containing original scientific articles and clinical records in otology, rhinology, laryngology and related specialties. Founded in 1887, JLO is absorbing reading for ENT specialists and trainees. The journal has an international outlook with contributions from around the world, relevant to all specialists in this area regardless of the country in which they practise. JLO contains main articles (original, review and historical), case reports and short reports as well as radiology, pathology or oncology in focus, a selection of abstracts, book reviews, letters to the editor, general notes and calendar, operative surgery techniques, and occasional supplements. It is fully illustrated and has become a definitive reference source in this fast-moving subject area. Published monthly an annual subscription is excellent value for money. Included in the subscription is access to the JLO interactive web site with searchable abstract database of the journal archive back to 1887.