Artificial Intelligence Versus Professional Standards: A Cross-Sectional Comparative Study of GPT, Gemini, and ENT UK in Delivering Patient Information on ENT Conditions.

IF 3 Q2 MEDICINE, RESEARCH & EXPERIMENTAL

Diseases (Basel, Switzerland) Pub Date : 2025-09-01 DOI:10.3390/diseases13090286

Ali Alabdalhussein, Nehal Singhania, Shazaan Nadeem, Mohammed Talib, Derar Al-Domaidat, Ibrahim Jimoh, Waleed Khan, Manish Mair

{"title":"Artificial Intelligence Versus Professional Standards: A Cross-Sectional Comparative Study of GPT, Gemini, and ENT UK in Delivering Patient Information on ENT Conditions.","authors":"Ali Alabdalhussein, Nehal Singhania, Shazaan Nadeem, Mohammed Talib, Derar Al-Domaidat, Ibrahim Jimoh, Waleed Khan, Manish Mair","doi":"10.3390/diseases13090286","DOIUrl":null,"url":null,"abstract":"Objective: Patient information materials are sensitive and, if poorly written, can cause misunderstanding. This study evaluated and compared the readability, actionability, and quality of patient education materials on laryngology topics generated by ChatGPT, Google Gemini, and ENT UK.Methods: We obtained patient information from ENT UK and generated equivalent content with ChatGPT-4-turbo and Google Gemini 2.5 Pro for six laryngology conditions. We assessed readability (Flesch-Kincaid Grade Level, FKGL; Flesch Reading Ease, FRE), quality (DISCERN), and patient engagement (PEMAT-P for understandability and actionability). Statistical comparisons involved using ANOVA, Tukey's HSD, and Kruskal-Wallis tests.Results: ENT UK showed the highest readability (FRE: 64.6 ± 8.4) and lowest grade level (FKGL: 7.4 ± 1.5), significantly better than that of ChatGPT (FRE: 38.8 ± 10.5, FKGL: 11.0 ± 1.5) and Gemini (FRE: 38.3 ± 8.5, FKGL: 11.9 ± 1.2) (all p < 0.001). DISCERN scores did not differ significantly (ENT UK: 21.3 ± 7.5, GPT: 24.7 ± 9.1, Gemini: 29.5 ± 4.6; p > 0.05). PEMAT-P understandability results were similar (ENT UK: 72.7 ± 8.3%, GPT: 79.1 ± 5.8%, Gemini: 78.5 ± 13.1%), except for lower GPT scores on vocal cord paralysis (p < 0.05). Actionability was also comparable (ENT UK: 46.7 ± 16.3%, GPT: 41.1 ± 24.0%, Gemini: 36.7 ± 19.7%).Conclusion: GPT and Gemini produce patient information of comparable quality and engagement to ENT UK but require higher reading levels and fall short of recommended literacy standards.","PeriodicalId":72832,"journal":{"name":"Diseases (Basel, Switzerland)","volume":"13 9","pages":""},"PeriodicalIF":3.0000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12468877/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Diseases (Basel, Switzerland)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3390/diseases13090286","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MEDICINE, RESEARCH & EXPERIMENTAL","Score":null,"Total":0}

引用次数: 0

Abstract

Objective: Patient information materials are sensitive and, if poorly written, can cause misunderstanding. This study evaluated and compared the readability, actionability, and quality of patient education materials on laryngology topics generated by ChatGPT, Google Gemini, and ENT UK.

Methods: We obtained patient information from ENT UK and generated equivalent content with ChatGPT-4-turbo and Google Gemini 2.5 Pro for six laryngology conditions. We assessed readability (Flesch-Kincaid Grade Level, FKGL; Flesch Reading Ease, FRE), quality (DISCERN), and patient engagement (PEMAT-P for understandability and actionability). Statistical comparisons involved using ANOVA, Tukey's HSD, and Kruskal-Wallis tests.

Results: ENT UK showed the highest readability (FRE: 64.6 ± 8.4) and lowest grade level (FKGL: 7.4 ± 1.5), significantly better than that of ChatGPT (FRE: 38.8 ± 10.5, FKGL: 11.0 ± 1.5) and Gemini (FRE: 38.3 ± 8.5, FKGL: 11.9 ± 1.2) (all p < 0.001). DISCERN scores did not differ significantly (ENT UK: 21.3 ± 7.5, GPT: 24.7 ± 9.1, Gemini: 29.5 ± 4.6; p > 0.05). PEMAT-P understandability results were similar (ENT UK: 72.7 ± 8.3%, GPT: 79.1 ± 5.8%, Gemini: 78.5 ± 13.1%), except for lower GPT scores on vocal cord paralysis (p < 0.05). Actionability was also comparable (ENT UK: 46.7 ± 16.3%, GPT: 41.1 ± 24.0%, Gemini: 36.7 ± 19.7%).

Conclusion: GPT and Gemini produce patient information of comparable quality and engagement to ENT UK but require higher reading levels and fall short of recommended literacy standards.

查看原文本刊更多论文

人工智能与专业标准：GPT、Gemini和ENT UK在提供耳鼻喉疾病患者信息方面的横断面比较研究。

目的：患者信息材料是敏感的，如果写得不好，可能会引起误解。本研究评估并比较了ChatGPT、谷歌Gemini和ENT UK编制的喉科主题患者教育材料的可读性、可操作性和质量。方法：我们从ENT UK获得患者信息，并使用ChatGPT-4-turbo和谷歌Gemini 2.5 Pro对六种喉科疾病生成等效内容。我们评估了可读性（Flesch- kincaid Grade Level， FKGL; Flesch Reading Ease， FRE）、质量（DISCERN）和患者参与度（PEMAT-P，可理解性和可操作性）。统计比较采用方差分析、Tukey’s HSD和Kruskal-Wallis检验。结果：ENT UK的可读性最高（FRE: 64.6±8.4），分级最低（FKGL: 7.4±1.5），显著优于ChatGPT （FRE: 38.8±10.5,FKGL: 11.0±1.5）和Gemini (FRE: 38.3±8.5,FKGL: 11.9±1.2)（p均< 0.001）。两组间的差异无统计学意义（ENT UK: 21.3±7.5,GPT: 24.7±9.1,Gemini: 29.5±4.6;p < 0.05）。PEMAT-P可理解性结果相似（ENT UK: 72.7±8.3%,GPT: 79.1±5.8%,Gemini: 78.5±13.1%），但声带麻痹的GPT评分较低（p < 0.05）。可操作性也具有可比性（ENT UK: 46.7±16.3%,GPT: 41.1±24.0%,Gemini: 36.7±19.7%）。结论：GPT和Gemini提供的患者信息质量和参与度与ENT UK相当，但需要更高的阅读水平，低于推荐的读写标准。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊