Cholesteatoma: Conventional patient-focused versus AI-generated resources

Introduction

A diagnosis of cholesteatoma necessitates a collaborative effort between otolaryngologists and patients for effective treatment. While physicians play a central role in providing medical education, patients often seek additional information from external sources to enhance the comprehension of their diagnosis. This study compared patient-focused cholesteatoma literature from established sources to ChatGPT-generated material. Emphasizing health literacy's crucial influence on patient health outcomes, it evaluates both sources' accuracy, readability, understandability, and actionability to highlight potential differences in these frequently accessed resources.

Methods

A quantitative assessment was conducted by calculating the Patient Education Materials Assessment Tool (PEMAT) score, DISCERN score, Flesch-Kincaid Grade Level (FKGL), and Flesch Reading Ease Score (FRES) for each website and ChatGPT response. Raters determined accuracy by quantifying the number of errors in each resource.

Results

The patient-focused content was associated with better understandability compared to ChatGPT responses with a mean PEMAT-U score of 80.2 ± 10.6 and 60.0 ± 3.72 (P < .001), respectively. There was a significant difference in readability and quality demonstrated by average FKGL (P < .001), FRES (P < .001), and DISCERN scores in the individual ChatGPT responses (P < .001). There was no significant difference with regard to DISCERN combined responses (p = 0.224) or PEMAT-A scores (p = 0.567). An average of 2.6 ± 1.1 errors were found in the combined ChatGPT responses.

Conclusion

Patient-focused content from established sources on cholesteatoma was easier to read, more understandable, and more accurate when compared to responses from ChatGPT. k T Nickles, B.S.