Assessing the readability of dermatological patient information leaflets generated by ChatGPT-4 and its associated plugins.

Abstract

Background: In the UK, 43% of adults struggle to understand health information presented in standard formats. As a result, Health Education England recommends that patient information leaflets (PILs) be written at a readability level appropriate for an 11-year-old.

Objectives: To evaluate the ability of ChatGPT-4 and its three dermatology-specific plugins to generate PILs that meet readability recommendations and compare their readability with existing British Association of Dermatologists (BAD) PILs.

Methods: ChatGPT-4 and its three plugins were used to generate PILs for 10 preselected dermatological conditions. The readability of these PILs was assessed using three readability formulas Simple Measure of Gobbledygook (SMOG), Flesch Reading Ease Test (FRET) and Flesch-Kincaid Grade Level Test (FKGLT) and compared against the readability of BAD PILs. A one-way ANOVA was conducted to identify any significant differences.

Results: The readability scores of PILs generated by ChatGPT-4 and its plugins did not meet the recommended target range. However, some of these PILs demonstrated more favourable mean readability scores compared with those from the BAD, with certain plugins, such as Chat with a Dermatologist, showing significant differences in mean SMOG (P = 0.0005) and mean FKGLT (P = 0.002) scores. Nevertheless, the PILs generated by ChatGPT-4 were found to lack some of the content typically included in BAD PILs.

Conclusions: ChatGPT-4 can produce dermatological PILs free from misleading information, occasionally surpassing BAD PILs in terms of readability. However, these PILs still fall short of being easily understood by the general public, and the content requires rigorous verification by healthcare professionals to ensure reliability and quality.