Improving Readability of Shoulder and Elbow Surgery Online Patient Education Material with Chat Generative Pretrained Transformer (ChatGPT) 4.

Abstract

Background: Health literacy is crucial for effective doctor-patient communication, particularly for surgical patients who need to comprehend complex procedures and care protocols. The American Medical Association and National Institutes of Health suggest patient education materials be at a sixth to eighth-grade reading level. Despite this, many online materials for orthopedic surgeries, including shoulder and elbow procedures are written above this level. ChatGPT-4, an AI language model, may help simplify these materials and improve readability given poor health literacy in patient populations.

Methods: Thirty excerpts of patient-facing information on shoulder and elbow surgeries were selected from academic and professional medical sources a variety of shoulder and elbow orthopedic surgical topics. Original readability was assessed using the SMOG (Simple Measure of Gobbledygook) Index. ChatGPT then analyzed readability and simplified the text to a sixth- to eighth-grade level. To simplify the text while maintaining medical accuracy, the following prompt was used: "Rewrite this text at a 6th to 8th grade level without losing information." ChatGPT achieved this by defining medical terminology, using common language equivalents, and restructuring information for easier readability. Simplified text was re-evaluated for readability by both SMOG Index and ChatGPT and accuracy by study authors.

Results: Original excerpts had an average SMOG readability score of 10.1, requiring about a tenth grade reading level. ChatGPT's initial analysis averaged slightly higher at 10.3 (p<0.001). Following simplification, the both the SMOG readability score and ChatGPT significantly dropped to 8.3, and 7.7, respectively, aligning closer to recommended readability levels (p<0.001). ChatGPT also provided targeted feedback on areas for readability improvement.

Conclusions: ChatGPT-4 demonstrated utility in analyzing and simplifying shoulder and elbow surgery patient education materials, lowering readability to near-recommended levels. By providing specific suggestions for simplification, ChatGPT streamlined the revision process, enhancing potential patient understanding and engagement. However, human review remains necessary to ensure clinical accuracy in AI-simplified materials.