大语言模型：开拓儿童近视教育新领域。

IF 2.6 3区医学 Q2 OPHTHALMOLOGY

Ophthalmology and Therapy Pub Date : 2025-06-01 Epub Date: 2025-04-21 DOI:10.1007/s40123-025-01142-x

Mohammad Delsoz, Amr Hassan, Amin Nabavi, Amir Rahdar, Brian Fowler, Natalie C Kerr, Lauren Claire Ditta, Mary E Hoehn, Margaret M DeAngelis, Andrzej Grzybowski, Yih-Chung Tham, Siamak Yousefi

{"title":"大语言模型：开拓儿童近视教育新领域。","authors":"Mohammad Delsoz, Amr Hassan, Amin Nabavi, Amir Rahdar, Brian Fowler, Natalie C Kerr, Lauren Claire Ditta, Mary E Hoehn, Margaret M DeAngelis, Andrzej Grzybowski, Yih-Chung Tham, Siamak Yousefi","doi":"10.1007/s40123-025-01142-x","DOIUrl":null,"url":null,"abstract":"Introduction: This study aimed to evaluate the performance of three large language models (LLMs), namely ChatGPT-3.5, ChatGPT-4o (o1 Preview), and Google Gemini, in producing patient education materials (PEMs) and improving the readability of online PEMs on childhood myopia.Methods: LLM-generated responses were assessed using three prompts. Prompt A requested to \"Write educational material on childhood myopia.\" Prompt B added a modifier specifying \"a sixth-grade reading level using the FKGL (Flesch-Kincaid Grade Level) readability formula.\" Prompt C aimed to rewrite existing PEMs to a sixth-grade level using FKGL. Reponses were assessed for quality (DISCERN tool), readability (FKGL, SMOG (Simple Measure of Gobbledygook)), Patient Education Materials Assessment Tool (PEMAT, understandability/actionability), and accuracy.Results: ChatGPT-4o (01) and ChatGPT-3.5 generated good-quality PEMs (DISCERN 52.8 and 52.7, respectively); however, quality declined from prompt A to prompt B (p = 0.001 and p = 0.013). Google Gemini produced fair-quality (DISCERN 43) but improved with prompt B (p = 0.02). All PEMs exceeded the 70% PEMAT understandability threshold but failed the 70% actionability threshold (40%). No misinformation was identified. Readability improved with prompt B; ChatGPT-4o (01) and ChatGPT-3.5 achieved a sixth-grade level or below (FGKL 6 ± 0.6 and 6.2 ± 0.3), while Google Gemini did not (FGKL 7 ± 0.6). ChatGPT-4o (01) outperformed Google Gemini in readability (p < 0.001) but was comparable to ChatGPT-3.5 (p = 0.846). Prompt C improved readability across all LLMs, with ChatGPT-4o (o1 Preview) showing the most significant gains (FKGL 5.8 ± 1.5; p < 0.001).Conclusions: ChatGPT-4o (o1 Preview) demonstrates potential in producing accurate, good-quality, understandable PEMs, and in improving online PEMs on childhood myopia.","PeriodicalId":19623,"journal":{"name":"Ophthalmology and Therapy","volume":"14 6","pages":"1281-1295"},"PeriodicalIF":2.6000,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12069199/pdf/","citationCount":"0","resultStr":"{\"title\":\"Large Language Models: Pioneering New Educational Frontiers in Childhood Myopia.\",\"authors\":\"Mohammad Delsoz, Amr Hassan, Amin Nabavi, Amir Rahdar, Brian Fowler, Natalie C Kerr, Lauren Claire Ditta, Mary E Hoehn, Margaret M DeAngelis, Andrzej Grzybowski, Yih-Chung Tham, Siamak Yousefi\",\"doi\":\"10.1007/s40123-025-01142-x\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Introduction: This study aimed to evaluate the performance of three large language models (LLMs), namely ChatGPT-3.5, ChatGPT-4o (o1 Preview), and Google Gemini, in producing patient education materials (PEMs) and improving the readability of online PEMs on childhood myopia.Methods: LLM-generated responses were assessed using three prompts. Prompt A requested to \\\"Write educational material on childhood myopia.\\\" Prompt B added a modifier specifying \\\"a sixth-grade reading level using the FKGL (Flesch-Kincaid Grade Level) readability formula.\\\" Prompt C aimed to rewrite existing PEMs to a sixth-grade level using FKGL. Reponses were assessed for quality (DISCERN tool), readability (FKGL, SMOG (Simple Measure of Gobbledygook)), Patient Education Materials Assessment Tool (PEMAT, understandability/actionability), and accuracy.Results: ChatGPT-4o (01) and ChatGPT-3.5 generated good-quality PEMs (DISCERN 52.8 and 52.7, respectively); however, quality declined from prompt A to prompt B (p = 0.001 and p = 0.013). Google Gemini produced fair-quality (DISCERN 43) but improved with prompt B (p = 0.02). All PEMs exceeded the 70% PEMAT understandability threshold but failed the 70% actionability threshold (40%). No misinformation was identified. Readability improved with prompt B; ChatGPT-4o (01) and ChatGPT-3.5 achieved a sixth-grade level or below (FGKL 6 ± 0.6 and 6.2 ± 0.3), while Google Gemini did not (FGKL 7 ± 0.6). ChatGPT-4o (01) outperformed Google Gemini in readability (p < 0.001) but was comparable to ChatGPT-3.5 (p = 0.846). Prompt C improved readability across all LLMs, with ChatGPT-4o (o1 Preview) showing the most significant gains (FKGL 5.8 ± 1.5; p < 0.001).Conclusions: ChatGPT-4o (o1 Preview) demonstrates potential in producing accurate, good-quality, understandable PEMs, and in improving online PEMs on childhood myopia.\",\"PeriodicalId\":19623,\"journal\":{\"name\":\"Ophthalmology and Therapy\",\"volume\":\"14 6\",\"pages\":\"1281-1295\"},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2025-06-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12069199/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Ophthalmology and Therapy\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1007/s40123-025-01142-x\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2025/4/21 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q2\",\"JCRName\":\"OPHTHALMOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ophthalmology and Therapy","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s40123-025-01142-x","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/4/21 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}

引用次数: 0

摘要

前言：本研究旨在评估ChatGPT-3.5、chatgpt - 40 （01 Preview）和谷歌Gemini三种大型语言模型（llm）在制作儿童近视患者教育材料（PEMs）和提高在线PEMs可读性方面的性能。方法：法学硕士生成的回答使用三个提示进行评估。提示A要求“编写儿童近视教材”。提示B添加了一个修饰符，指定“使用FKGL （Flesch-Kincaid Grade level）可读性公式的六年级阅读水平”。提示符C旨在使用FKGL将现有的PEMs重写为六年级水平。评估反馈的质量（DISCERN工具）、可读性（FKGL、SMOG（简单测量的Gobbledygook））、患者教育材料评估工具（PEMAT，可理解性/可操作性）和准确性。结果：chatgpt - 40（01）和ChatGPT-3.5生成质量良好的PEMs（分别为辨别52.8和52.7）；然而，从提示A到提示B，质量下降（p = 0.001和p = 0.013）。谷歌双子座产生了一般的质量（辨别43），但提高了提示B （p = 0.02）。所有的PEMs都超过了70%的PEMAT可理解性阈值，但未能达到70%的可操作性阈值（40%）。没有发现错误信息。提示符B提高了可读性；chatgpt - 40（01）和ChatGPT-3.5达到6级或以下水平（FGKL 6±0.6和6.2±0.3），而谷歌Gemini没有达到（FGKL 7±0.6）。chatgpt - 40（01）在可读性上优于谷歌Gemini (p)。结论：chatgpt - 40 （01 Preview）在制作准确、高质量、可理解的PEMs以及改善儿童近视在线PEMs方面具有潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Large Language Models: Pioneering New Educational Frontiers in Childhood Myopia.

Introduction: This study aimed to evaluate the performance of three large language models (LLMs), namely ChatGPT-3.5, ChatGPT-4o (o1 Preview), and Google Gemini, in producing patient education materials (PEMs) and improving the readability of online PEMs on childhood myopia.

Methods: LLM-generated responses were assessed using three prompts. Prompt A requested to "Write educational material on childhood myopia." Prompt B added a modifier specifying "a sixth-grade reading level using the FKGL (Flesch-Kincaid Grade Level) readability formula." Prompt C aimed to rewrite existing PEMs to a sixth-grade level using FKGL. Reponses were assessed for quality (DISCERN tool), readability (FKGL, SMOG (Simple Measure of Gobbledygook)), Patient Education Materials Assessment Tool (PEMAT, understandability/actionability), and accuracy.

Results: ChatGPT-4o (01) and ChatGPT-3.5 generated good-quality PEMs (DISCERN 52.8 and 52.7, respectively); however, quality declined from prompt A to prompt B (p = 0.001 and p = 0.013). Google Gemini produced fair-quality (DISCERN 43) but improved with prompt B (p = 0.02). All PEMs exceeded the 70% PEMAT understandability threshold but failed the 70% actionability threshold (40%). No misinformation was identified. Readability improved with prompt B; ChatGPT-4o (01) and ChatGPT-3.5 achieved a sixth-grade level or below (FGKL 6 ± 0.6 and 6.2 ± 0.3), while Google Gemini did not (FGKL 7 ± 0.6). ChatGPT-4o (01) outperformed Google Gemini in readability (p < 0.001) but was comparable to ChatGPT-3.5 (p = 0.846). Prompt C improved readability across all LLMs, with ChatGPT-4o (o1 Preview) showing the most significant gains (FKGL 5.8 ± 1.5; p < 0.001).

Conclusions: ChatGPT-4o (o1 Preview) demonstrates potential in producing accurate, good-quality, understandable PEMs, and in improving online PEMs on childhood myopia.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Ophthalmology and Therapy OPHTHALMOLOGY-

CiteScore

4.20

自引率

3.00%

发文量

157

审稿时长

6 weeks

期刊介绍： Aims and Scope Ophthalmology and Therapy is an international, open access, peer-reviewed (single-blind), and rapid publication journal. The scope of the journal is broad and will consider all scientifically sound research from preclinical, clinical (all phases), observational, real-world, and health outcomes research around the use of ophthalmological therapies, devices, and surgical techniques. The journal is of interest to a broad audience of pharmaceutical and healthcare professionals and publishes original research, reviews, case reports/series, trial protocols and short communications such as commentaries and editorials. Ophthalmology and Therapy will consider all scientifically sound research be it positive, confirmatory or negative data. Submissions are welcomed whether they relate to an international and/or a country-specific audience, something that is crucially important when researchers are trying to target more specific patient populations. This inclusive approach allows the journal to assist in the dissemination of quality research, which may be considered of insufficient interest by other journals. Rapid Publication The journal’s publication timelines aim for a rapid peer review of 2 weeks. If an article is accepted it will be published 3–4 weeks from acceptance. The rapid timelines are achieved through the combination of a dedicated in-house editorial team, who manage article workflow, and an extensive Editorial and Advisory Board who assist with peer review. This allows the journal to support the rapid dissemination of research, whilst still providing robust peer review. Combined with the journal’s open access model this allows for the rapid, efficient communication of the latest research and reviews, fostering the advancement of ophthalmic therapies. Open Access All articles published by Ophthalmology and Therapy are open access. Personal Service The journal’s dedicated in-house editorial team offer a personal “concierge service” meaning authors will always have an editorial contact able to update them on the status of their manuscript. The editorial team check all manuscripts to ensure that articles conform to the most recent COPE, GPP and ICMJE publishing guidelines. This supports the publication of ethically sound and transparent research. Digital Features and Plain Language Summaries Ophthalmology and Therapy offers a range of additional features designed to increase the visibility, readership and educational value of the journal’s content. Each article is accompanied by key summary points, giving a time-efficient overview of the content to a wide readership. Articles may be accompanied by plain language summaries to assist readers who have some knowledge of, but not in-depth expertise in, the area to understand the scientific content and overall implications of the article. The journal also provides the option to include various types of digital features including animated abstracts, video abstracts, slide decks, audio slides, instructional videos, infographics, podcasts and animations. All additional features are peer reviewed to the same high standard as the article itself. If you consider that your paper would benefit from the inclusion of a digital feature, please let us know. Our editorial team are able to create high-quality slide decks and infographics in-house, and video abstracts through our partner Research Square, and would be happy to assist in any way we can. For further information about digital features, please contact the journal editor (see ‘Contact the Journal’ for email address), and see the ‘Guidelines for digital features and plain language summaries’ document under ‘Submission guidelines’. For examples of digital features please visit our showcase page https://springerhealthcare.com/expertise/publishing-digital-features/ Publication Fees Upon acceptance of an article, authors will be required to pay the mandatory Rapid Service Fee of €5250/$6000/£4300. The journal will consider fee discounts and waivers for developing countries and this is decided on a case by case basis. Peer Review Process Upon submission, manuscripts are assessed by the editorial team to ensure they fit within the aims and scope of the journal and are also checked for plagiarism. All suitable submissions are then subject to a comprehensive single-blind peer review. Reviewers are selected based on their relevant expertise and publication history in the subject area. The journal has an extensive pool of editorial and advisory board members who have been selected to assist with peer review based on the afore-mentioned criteria. At least two extensive reviews are required to make the editorial decision, with the exception of some article types such as Commentaries, Editorials, and Letters which are generally reviewed by one member of the Editorial Board. Where reviewer recommendations are conflicted, the editorial board will be contacted for further advice and a presiding decision. Manuscripts are then either accepted, rejected or authors are required to make major or minor revisions (both reviewer comments and editorial comments may need to be addressed). Once a revised manuscript is re-submitted, it is assessed along with the responses to reviewer comments and if it has been adequately revised it will be accepted for publication. Accepted manuscripts are then copyedited and typeset by the production team before online publication. Appeals against decisions following peer review are considered on a case-by-case basis and should be sent to the journal editor. Preprints We encourage posting of preprints of primary research manuscripts on preprint servers, authors’ or institutional websites, and open communications between researchers whether on community preprint servers or preprint commenting platforms. Posting of preprints is not considered prior publication and will not jeopardize consideration in our journals. Authors should disclose details of preprint posting during the submission process or at any other point during consideration in one of our journals. Once the manuscript is published, it is the author’s responsibility to ensure that the preprint record is updated with a publication reference, including the DOI and a URL link to the published version of the article on the journal website. Please follow the link for further information on preprint sharing: https://www.springer.com/gp/authors-editors/journal-author/journal-author-helpdesk/submission/1302#c16721550 Copyright Ophthalmology and Therapy''s content is published open access under the Creative Commons Attribution-Noncommercial License, which allows users to read, copy, distribute, and make derivative works for non-commercial purposes from the material, as long as the author of the original work is cited. The author assigns the exclusive right to any commercial use of the article to Springer. For more information about the Creative Commons Attribution-Noncommercial License, click here: http://creativecommons.org/licenses/by-nc/4.0. Contact For more information about the journal, including pre-submission enquiries, please contact christopher.vautrinot@springer.com.