Information about labor epidural analgesia: an updated evaluation on the readability, accuracy, and quality of ChatGPT responses incorporating patient preferences and complex clinical scenarios

Abstract

Background

Recent studies evaluating frequently asked questions (FAQs) on labor epidural analgesia (LEA) only used generic questions without incorporating detailed clinical information that reflects patient-specific inputs. We investigated the performance of ChatGPT in addressing these questions related to LEA with an emphasis on individual preferences and clinical conditions.

Methods

Twenty-nine questions for the AI chatbot were generated from the commonly asked questions relating to LEA based on clinical conditions. The generation of responses was performed in January 2025 with each question under individual sub-topics initiated as a “New chat” in ChatGPT-4o. Upon having the first questions answered, subsequent question(s) in the same sub-topic were continued in the same chat following the sequences as predefined. The readability of each response was graded using six readability indices, while the accuracy, Patient Education Materials Assessment Tool for Print (PEMAT) understandability and actionability was assessed by four obstetric anesthesiologists.

Results

The mean readability indices of the ChatGPT-4o responses to the questions were generally rated as fairly difficult to very difficult, which corresponded to a US grade level between 11th grade to college level entry. The mean (± standard deviation) accuracy of the responses was 97.7% ± 8.1%. The PEMAT understandability and actionability scores were 97.9% ± 0.9%) and 98.0% ± 1.4%), respectively.

Conclusions

ChatGPT can provide accurate and readable information about LEA even under different clinical contexts. However, improvement is needed to refine the responses with suitable prompts to simplify the outputs and improve readability. These approaches will thereby meet the need for the effective delivery of reliable patient education information.