Enhanced PROcedural Information READability for Patient-Centered Care in Interventional Radiology With Large Language Models (PRO-READ IR)

Abstract

Purpose

To evaluate the extent to which Generative Pre-trained Transformer 4 (GPT-4) can educate patients by generating easily understandable information about the most common interventional radiology (IR) procedures.

Materials and methods

We reviewed 10 IR procedures and prepared prompts for GPT-4 to provide patient educational instructions about each procedure in layman’s terms. The instructions were then evaluated by four clinical physicians and nine nonclinical assessors to determine their clinical appropriateness, understandability, and clarity using a survey. A grade-level readability assessment was performed using validated metrics to evaluate accessibility to a wide patient population. The same procedures were also evaluated from the patient instructions available at radiologyinfo.org and compared with GPT-generated instructions utilizing a paired t test.

Results

Evaluation by four clinical physicians shows that nine GPT-generated instructions were fully appropriate, whereas arterial embolization instructions was somewhat appropriate. Evaluation by nine nonclinical assessors shows that paracentesis, dialysis catheter placement, thrombectomy, ultrasound-guided biopsy, and nephrostomy-tube instructions were rated excellent by 57% and good by 43%. The arterial embolization and biliary-drain instructions were rated excellent by 28.6% and good by 71.4%. In contrast, thoracentesis, port placement, and CT-guided biopsy instructions received 43% excellent, 43% good, and 14% fair. The readability assessment across all procedural instructions showed a better Flesch-Kincaid mean grade of GPT-4 instructions compared with radiologyinfo.org (7.8 ± 0.87 versus 9.6 ± 0.83; P = .007) indicating excellent readability at 7th- to 8th-grade level compared with 9th to 10th grade. Additionally there was a lower Gunning Fog mean index (10.4 ± 1.2 versus 12.7 ± 0.93; P = .006), and higher Flesch Reading Ease mean score (69.4 ± 4.8 versus 51.3±3.9; P = .0001) indicating better readability.

Conclusion

IR procedural instructions generated by GPT-4 can aid in improving health literacy and patient-centered care in IR by generating easily understandable explanations.