Evaluating the use of artificial intelligence and human guidance to teach generic and specialist skills in breaking bad news: A mixed methods exploration.

Abstract

Objective: Breaking bad news (BBN) is specified in the General Medical Council's Generic Professional Capabilities framework, which sets out core skills to be developed alongside curricula-specified capabilities in postgraduate training. To date, a wide variety of approaches to teaching BBN have been used, however, the application of artificial intelligence (AI) to learning in this field remains in its early stages and requires further investigation.

Design, setting, participants and intervention: Two approaches were compared in response to the question What is the best way to break bad news?: a human-facilitated discussion versus an AI-generated response using ChatGPT-4. Both approaches produced high-level guidance condensed into short statements for online mixed methods evaluation by experienced medical educators.

Outcome measures and results: 41 doctors from 17 specialties and eight lay representatives provided free-text commentary as part of the assessment. Overall findings were compared with popular patient-focused BBN protocols, such as SPIKES, with qualitative analysis providing more details of skills requirements for generic and specialist levels of practice by adding more depth and nuance via three themes: patient experience, professionalism and access to support. Both approaches matched well with patient-focused protocols albeit they also addressed doctors' debriefing after BBN.

Conclusions: Overall, AI produced more comprehensive, highly rated guidance suitable for early careers, but qualitative evaluation highlighted the need for more specialised skills, their nature and application, plus differences in preferred style and content between approaches. The value of tailoring communication, via a personalised approach developed from real-life experience, was also confirmed.