Why implementing machine learning algorithms in the clinic is not a plug-and-play solution: a simulation study of a machine learning algorithm for acute leukaemia subtype diagnosis.

Abstract

Background: Artificial intelligence (AI) and machine learning (ML) algorithms have shown great promise in clinical medicine. Despite the increasing number of published algorithms, most remain unvalidated in real-world clinical settings. This study aims to simulate the practical implementation challenges of a recently developed ML algorithm, AI-PAL, designed for the diagnosis of acute leukaemia and report on its performance.

Methods: We conducted a detailed simulation of the AI-PAL algorithm's implementation at the University Hospital Essen. Cohort building was performed using our Fast Healthcare Interoperability Resources (FHIR) database, identifying all initially diagnosed patients with acute leukaemia and selected differential diagnoses. The algorithm's performance was assessed by reproducing the original study's results.

Findings: The AI-PAL algorithm demonstrated significantly lower performance in our simulated clinical implementation compared to prior published results. The area under the receiver operating characteristic curve for acute lymphoblastic leukaemia dropped to 0.67 (95% CI: 0.61-0.73) and for acute myeloid leukaemia to 0.71 (95% CI: 0.65-0.76). The recalibration of probability cutoffs determining confident diagnoses increased the number of confident positive diagnosis for acute leukaemia from 98 to 160, highlighting the necessity of local validation and adjustments.

Interpretation: The findings underscore the challenges of implementing ML algorithms in clinical practice. Despite robust development and validation in research settings, ML models like AI-PAL may require significant adjustments and recalibration to maintain performance in different clinical settings. Our results suggest that clinical decision support algorithms should undergo local performance validation before integration into routine care to ensure reliability and safety.

Funding: This study was supported by the DFG-cofounded UMEA Clinician Scientist Program and the Ministry of Culture and Science of the State of North Rhine-Westphalia.