I-SIRch: AI-powered concept annotation tool for equitable extraction and analysis of safety insights from maternity investigations.

Abstract

Background: Maternity care is a complex system involving treatments and interactions between patients, healthcare providers, and the care environment. To enhance patient safety and outcomes, it is crucial to understand the human factors (e.g. individuals' decisions, local facilities) influencing healthcare. However, most current tools for analysing healthcare data focus only on biomedical concepts (e.g. health conditions, procedures and tests), overlooking the importance of human factors.

Methods: We developed a new approach called I-SIRch, using artificial intelligence to automatically identify and label human factors concepts in maternity investigation reports describing adverse maternity incidents produced by England's Healthcare Safety Investigation Branch (HSIB). These incident investigation reports aim to identify opportunities for learning and improving maternal safety across the entire healthcare system. Unlike existing clinical annotation tools that extract solely biomedical insights, I-SIRch is uniquely designed to capture the socio-technical dimensions of patient safety incidents. This innovation enables a more comprehensive analysis of the complex systemic issues underlying adverse events in maternity care, providing insights that were previously difficult to obtain at scale. Importantly, I-SIRch employs a hybrid approach, incorporating human expertise to validate and refine the AI-generated annotations, ensuring the highest quality of analysis.

Findings: I-SIRch was trained using real data and tested on both real and synthetic data to evaluate its performance in identifying human factors concepts. When applied to real reports, the model achieved a high level of accuracy, correctly identifying relevant concepts in 90% of the sentences from 97 reports (Balanced Accuracy of 90% ± 18% (Recall 93% ± 18%, Precision 87% ± 34%, F-score 96% ± 10%). Applying I-SIRch to analyse these reports revealed that certain human factors disproportionately affected mothers from different ethnic groups. In particular, gaps in risk assessment were more prevalent for minority mothers, whilst communication issues were common across all groups but potentially more for minorities.

Interpretation: Our work demonstrates the potential of using automated tools to identify human factors concepts in maternity incident investigation reports, rather than focusing solely on biomedical concepts. This approach opens up new possibilities for understanding the complex interplay between social, technical and organisational factors influencing maternal safety and population health outcomes. By taking a more comprehensive view of maternal healthcare delivery, we can develop targeted interventions to address disparities and improve maternal outcomes. Targeted interventions to address these disparities could include culturally sensitive risk assessment protocols, enhanced language support, and specialised training for healthcare providers on recognising and mitigating biases. These findings highlight the need for tailored approaches to improve equitable care delivery and outcomes in maternity services. The I-SIRch framework thus represents a significant advancement in our ability to extract actionable intelligence from healthcare incident reports, moving beyond traditional clinical factors to encompass the broader systemic issues that impact patient safety.