Bridging the Health Literacy Gap for Patients With Oral Cancer: Readability Enhancement With AI Chatbots

Abstract

Background

Early diagnosis is crucial in improving oral cancer outcomes. Patient education materials support timely recognition and management. However, these resources are often written above recommended reading levels, beyond patients' health literacy and limiting accessibility.

Objectives

To assess the readability of available patient information on oral cancer by the NHS, to evaluate three large language models (LLMs; ChatGPT, Claude and Gemini) in simplifying texts while preserving their content, and to propose an improved leaflet based on UK materials, expert review and LLM adjustment to match average UK reading levels.

Methods

Materials were collected from NHS-affiliated websites. Original and LLM-simplified texts were assessed using validated readability tools (FRES, FKGL, GFI, CLI and SMOG). Content fidelity was assessed using character 3–5-g cosine, sentence-content retention and latent semantic analysis (LSA). An expert review was applied to the proposed leaflet.

Results

LLM-revisions significantly improved readability across all five indices (p < 0.0001). Mean FRES of original texts was 66.4 ± 7.7, while Claude (81.6 ± 6.2) was the only model to surpass the 80 benchmark. Semantic similarity to source text remained high (LSA means 0.97 ± 0.04, 0.94 ± 0.09 and 0.96 ± 0.08; character 3–5-g cosine 0.85 ± 0.05, 0.80 ± 0.08 and 0.82 ± 0.08 for respective models). Baseline readability of the proposed leaflet was comparable to NHS materials (FRES 65.7); Claude increased this to 81.2.

Conclusions

LLM-based simplification enhanced readability while preserving content fidelity. This approach can help enhance accessibility, particularly for populations disproportionately affected by oral cancer. With human oversight, it could be adopted at the policy level to standardise patient education and reduce health literacy disparities.