Zeynep Özer Özcan, Levent Doğan, Ibrahim Edhem Yilmaz
{"title":"Artificial Doctors: Performance of Chatbots as a Tool for Patient Education on Keratoconus.","authors":"Zeynep Özer Özcan, Levent Doğan, Ibrahim Edhem Yilmaz","doi":"10.1097/ICL.0000000000001160","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>We aimed to compare the answers given by ChatGPT, Bard, and Copilot and that obtained from the American Academy of Ophthalmology (AAO) website to patient-written questions related to keratoconus in terms of accuracy, understandability, actionability, and readability to find out whether chatbots can be used in patient education.</p><p><strong>Methods: </strong>Twenty patient-written questions obtained from the AAO website related to keratoconus were asked to ChatGPT, Bard, and Copilot. Two ophthalmologists independently assessed the answers obtained from chatbots and the AAO website in terms of accuracy, understandability, and actionability according to the Structure of Observed Learning Outcome taxonomy, Patient Education Materials Assessment Tool-Understandability, and Patient Education Materials Assessment Tool-Actionability tests, respectively. The answers were also compared for readability according to the Flesch Reading Ease scores obtained through the website.</p><p><strong>Results: </strong>Bard had significantly higher scores compared with ChatGPT-3.5, Copilot, and AAO website according to Structure of Observed Learning Outcome taxonomy and Patient Education Materials Assessment Tool-Understandability (P<0.001 for each), whereas there was no significant difference between the other groups. Bard and ChatGPT achieved significantly higher scores than the AAO website according to the Patient Education Materials Assessment Tool-Actionability scale (P=0.001). The AAO website achieved significantly higher scores than the Bard on the Flesch Reading Ease scale, whereas there was no significant difference between the other groups (P=0.017).</p><p><strong>Conclusion: </strong>Chatbots are promising to provide accurate, understandable, and actionable answers. Chatbots can be a valuable aid in the education of patients with keratoconus under clinician supervision. In this way, unnecessary hospital visits can be prevented, and the burden on the health care system can be alleviated, while patient awareness can be raised.</p>","PeriodicalId":50457,"journal":{"name":"Eye & Contact Lens-Science and Clinical Practice","volume":" ","pages":""},"PeriodicalIF":2.0000,"publicationDate":"2024-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Eye & Contact Lens-Science and Clinical Practice","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/ICL.0000000000001160","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Purpose: We aimed to compare the answers given by ChatGPT, Bard, and Copilot and that obtained from the American Academy of Ophthalmology (AAO) website to patient-written questions related to keratoconus in terms of accuracy, understandability, actionability, and readability to find out whether chatbots can be used in patient education.
Methods: Twenty patient-written questions obtained from the AAO website related to keratoconus were asked to ChatGPT, Bard, and Copilot. Two ophthalmologists independently assessed the answers obtained from chatbots and the AAO website in terms of accuracy, understandability, and actionability according to the Structure of Observed Learning Outcome taxonomy, Patient Education Materials Assessment Tool-Understandability, and Patient Education Materials Assessment Tool-Actionability tests, respectively. The answers were also compared for readability according to the Flesch Reading Ease scores obtained through the website.
Results: Bard had significantly higher scores compared with ChatGPT-3.5, Copilot, and AAO website according to Structure of Observed Learning Outcome taxonomy and Patient Education Materials Assessment Tool-Understandability (P<0.001 for each), whereas there was no significant difference between the other groups. Bard and ChatGPT achieved significantly higher scores than the AAO website according to the Patient Education Materials Assessment Tool-Actionability scale (P=0.001). The AAO website achieved significantly higher scores than the Bard on the Flesch Reading Ease scale, whereas there was no significant difference between the other groups (P=0.017).
Conclusion: Chatbots are promising to provide accurate, understandable, and actionable answers. Chatbots can be a valuable aid in the education of patients with keratoconus under clinician supervision. In this way, unnecessary hospital visits can be prevented, and the burden on the health care system can be alleviated, while patient awareness can be raised.
目的：我们将ChatGPT、Bard和Copilot从美国眼科学会（American Academy of Ophthalmology， AAO）网站上获得的答案与患者自述的圆锥角膜相关问题的准确性、可理解性、可操作性和可读性进行比较，以确定聊天机器人是否可以用于患者教育。方法：从AAO网站上获得的与圆锥角膜相关的患者书面问题20个，分别向ChatGPT、Bard和Copilot询问。两位眼科医生分别根据观察学习成果分类结构、患者教育材料评估工具-可理解性和患者教育材料评估工具-可操作性测试，独立评估从聊天机器人和AAO网站获得的答案的准确性、可理解性和可操作性。根据从网站获得的Flesch Reading Ease分数，对答案的可读性进行了比较。结果：根据观察学习成果分类结构和患者教育材料评估工具-可理解性(p)，巴德的得分明显高于ChatGPT-3.5， Copilot和AAO网站。结论：聊天机器人有望提供准确，可理解和可操作的答案。在临床医生的监督下，聊天机器人可以成为圆锥角膜患者教育的宝贵援助。通过这种方式，可以避免不必要的医院就诊，减轻卫生保健系统的负担，同时可以提高患者的意识。
期刊介绍：
Eye & Contact Lens: Science and Clinical Practice is the official journal of the Contact Lens Association of Ophthalmologists (CLAO), an international educational association for anterior segment research and clinical practice of interest to ophthalmologists, optometrists, and other vision care providers and researchers. Focusing especially on contact lenses, it also covers dry eye disease, MGD, infections, toxicity of drops and contact lens care solutions, topography, cornea surgery and post-operative care, optics, refractive surgery and corneal stability (eg, UV cross-linking). Peer-reviewed and published six times annually, it is a highly respected scientific journal in its field.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
