A quality improvement project of patient perception of AI-generated discharge summaries: a comparison with doctor-written summaries.

Introduction: Every patient admitted to hospital should receive a discharge letter when they leave. Artificial intelligence (AI) has the capability to fulfil this task. Here, we investigate the use of AI to generate discharge letters compared with letters written by a doctor.

Methods: Using an AI tool, ChatGPT, we generated two discharge letters for hypothetical elective tonsillectomy patients. We asked the parents of paediatric tonsillectomy patients to blindly compare the AI letters with two anonymised real discharge letters for tonsillectomy patients, written by two ear, nose and throat (ENT) doctors. Participants were asked to rate the quality of medical information, the ease of reading and the length of each of the four discharge letters. They were also asked to deduce who they thought wrote each discharge letter (AI or a doctor).

Results: Forty-seven parents responded to the survey. Our results demonstrate that the AI letters were reported to contain significantly better medical information (p = 0.0059) and were significantly easier to read than the doctor-written letters (p < 0.0001). Respondents had a 50% sensitivity in correctly identifying the letters written by AI.

Conclusions: AI tools have the potential to write tonsillectomy discharge letters of comparable quality (as perceived by our participant population) to those written by ENT doctors. This study provides preliminary evidence to show that AI-generated discharge letters may be an interesting avenue of further investigation as an application for this tool.