{"title":"使用QAMAI工具对人工智能提供的普通耳鼻喉外科手术信息进行综合评估。","authors":"Mitat Selçuk Bozhöyük, Levent Yücel","doi":"10.1017/S0022215125103368","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>This study aimed to evaluate the quality of information provided by artificial intelligence (AI) applications regarding ENT surgeries and usability for patients.</p><p><strong>Methods: </strong>ChatGPT 4.0, GEMINI 1.5 Flash, Copilot, Claude 3.5 Sonnet and DeepSeek-R1 were asked to provide detailed responses to patient-oriented questions about 15 ENT surgeries. Each AI application was queried three times, with a 3-day interval between each session. Two ENT specialists evaluated all responses using the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool.</p><p><strong>Results: </strong>Average QAMAI scores for each AI application were as follows: ChatGPT 4.0 (27.56 ± 1.20), GEMINI 1.5 Flash (26.24 ± 1.26), Copilot (26.84 ± 1.35), Claude 3.5 Sonnet (28.24 ± 0.77) and DeepSeek-R1 (28.13 ± 0.84). A statistically significant difference was found among the applications (<i>p</i> < 0.001). ICC analysis indicated high stability across evaluations conducted for all five AI applications (<i>p</i> < 0.001).</p><p><strong>Conclusion: </strong>AI has the potential to provide patients with accurate and consistent information about ENT surgeries, yet differences in QAMAI scores show that information quality varies between platforms.</p>","PeriodicalId":16293,"journal":{"name":"Journal of Laryngology and Otology","volume":" ","pages":"1-6"},"PeriodicalIF":0.8000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A comprehensive evaluation of artificial intelligence-provided information on common ENT surgical procedures using the QAMAI tool.\",\"authors\":\"Mitat Selçuk Bozhöyük, Levent Yücel\",\"doi\":\"10.1017/S0022215125103368\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objectives: </strong>This study aimed to evaluate the quality of information provided by artificial intelligence (AI) applications regarding ENT surgeries and usability for patients.</p><p><strong>Methods: </strong>ChatGPT 4.0, GEMINI 1.5 Flash, Copilot, Claude 3.5 Sonnet and DeepSeek-R1 were asked to provide detailed responses to patient-oriented questions about 15 ENT surgeries. Each AI application was queried three times, with a 3-day interval between each session. Two ENT specialists evaluated all responses using the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool.</p><p><strong>Results: </strong>Average QAMAI scores for each AI application were as follows: ChatGPT 4.0 (27.56 ± 1.20), GEMINI 1.5 Flash (26.24 ± 1.26), Copilot (26.84 ± 1.35), Claude 3.5 Sonnet (28.24 ± 0.77) and DeepSeek-R1 (28.13 ± 0.84). A statistically significant difference was found among the applications (<i>p</i> < 0.001). ICC analysis indicated high stability across evaluations conducted for all five AI applications (<i>p</i> < 0.001).</p><p><strong>Conclusion: </strong>AI has the potential to provide patients with accurate and consistent information about ENT surgeries, yet differences in QAMAI scores show that information quality varies between platforms.</p>\",\"PeriodicalId\":16293,\"journal\":{\"name\":\"Journal of Laryngology and Otology\",\"volume\":\" \",\"pages\":\"1-6\"},\"PeriodicalIF\":0.8000,\"publicationDate\":\"2025-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Laryngology and Otology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1017/S0022215125103368\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"OTORHINOLARYNGOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Laryngology and Otology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1017/S0022215125103368","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"OTORHINOLARYNGOLOGY","Score":null,"Total":0}
A comprehensive evaluation of artificial intelligence-provided information on common ENT surgical procedures using the QAMAI tool.
Objectives: This study aimed to evaluate the quality of information provided by artificial intelligence (AI) applications regarding ENT surgeries and usability for patients.
Methods: ChatGPT 4.0, GEMINI 1.5 Flash, Copilot, Claude 3.5 Sonnet and DeepSeek-R1 were asked to provide detailed responses to patient-oriented questions about 15 ENT surgeries. Each AI application was queried three times, with a 3-day interval between each session. Two ENT specialists evaluated all responses using the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool.
Results: Average QAMAI scores for each AI application were as follows: ChatGPT 4.0 (27.56 ± 1.20), GEMINI 1.5 Flash (26.24 ± 1.26), Copilot (26.84 ± 1.35), Claude 3.5 Sonnet (28.24 ± 0.77) and DeepSeek-R1 (28.13 ± 0.84). A statistically significant difference was found among the applications (p < 0.001). ICC analysis indicated high stability across evaluations conducted for all five AI applications (p < 0.001).
Conclusion: AI has the potential to provide patients with accurate and consistent information about ENT surgeries, yet differences in QAMAI scores show that information quality varies between platforms.
期刊介绍：
The Journal of Laryngology & Otology (JLO) is a leading, monthly journal containing original scientific articles and clinical records in otology, rhinology, laryngology and related specialties. Founded in 1887, JLO is absorbing reading for ENT specialists and trainees. The journal has an international outlook with contributions from around the world, relevant to all specialists in this area regardless of the country in which they practise. JLO contains main articles (original, review and historical), case reports and short reports as well as radiology, pathology or oncology in focus, a selection of abstracts, book reviews, letters to the editor, general notes and calendar, operative surgery techniques, and occasional supplements. It is fully illustrated and has become a definitive reference source in this fast-moving subject area. Published monthly an annual subscription is excellent value for money. Included in the subscription is access to the JLO interactive web site with searchable abstract database of the journal archive back to 1887.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
