Utilizing AI-Generated Plain Language Summaries to Enhance Interdisciplinary Understanding of Ophthalmology Notes: A Randomized Trial

Abstract

Background Specialized terminology employed by ophthalmologists creates a comprehension barrier for non-ophthalmology providers, compromising interdisciplinary communication and patient care. Current solutions such as manual note simplification are impractical or inadequate. Large language models (LLMs) present a potential low-burden approach to translating ophthalmology documentation into accessible language. Methods This prospective, randomized trial evaluated the addition of LLM-generated plain language summaries (PLSs) to standard ophthalmology notes (SONs). Participants included non-ophthalmology providers and ophthalmologists. The study assessed: (1) non-ophthalmology providers' comprehension and satisfaction with either the SON (control) or SON+PLS (intervention), (2) ophthalmologists' evaluation of PLS accuracy, safety, and time burden, and (3) objective semantic and linguistic quality of PLSs. Results 85% of non-ophthalmology providers (n=362, 33% response rate) preferred the PLS to SON. Non-ophthalmology providers reported enhanced diagnostic understanding (p=0.012), increased note detail satisfaction (p<0.001), and improved explanation clarity (p<0.001) for notes containing a PLS. The addition of a PLS narrowed comprehension gaps between providers who were comfortable and uncomfortable with ophthalmology terminology at baseline (intergroup difference p<0.001 to p>0.05). PLS semantic analysis demonstrated high meaning preservation (BERTScore mean F1 score: 0.85) with greater readability (Flesch Reading Ease: 51.8 vs. 43.6, Flesch-Kincaid Grade Level: 10.7 vs. 11.9). Ophthalmologists (n=489, 84% response rate) reported high PLS accuracy (90% "a great deal") with minimal review time burden (94.9% ≤ 1 minute). PLS error rate on initial ophthalmologist review and editing was 26%, and 15% on independent ophthalmologist over-read of edited PLSs. 84.9% of identified errors were deemed low risk for patient harm and 0% had a risk of severe harm/death. Conclusions LLM-generated plain language summaries enhance accessibility and utility of ophthalmology notes for non-ophthalmology providers while maintaining high semantic fidelity and improving readability. PLS error rates underscore the need for careful implementation and ongoing safety monitoring in clinical practice.