Development and validation of a dynamic early warning system with time-varying machine learning models for predicting hemodynamic instability in critical care: a multicohort study

Abstract

Hemodynamic instability, a life-threatening condition marked by circulatory failure, presents a significant challenge in intensive care unit (ICU) settings, often leading to poor patient outcomes. Traditional monitoring methods that rely on single parameters may delay diagnosis. Machine learning models offer a solution by integrating multiple clinical parameters to more dynamically and accurately predict instability. We developed the Time-varying Hemodynamic Early Warning Score (TvHEWS), an AI-assisted model used to predict hemodynamic instability in intensive care unit (ICU) patients. The model was trained and internally validated via retrospective data from the VGHTPE 2010 cohort (2010–2021) at Taipei Veteran General Hospital. It was further validated with prospective data from the VGHTPE 2022 cohort and external data from the MIMIC IV cohort. TvHEWS includes hourly updating models, providing continuous risk assessments. TvHEWS showed strong predictive performance. In the VGHTPE 2010 cohort, the AUROC was 0.93, with a precision of 0.94 and a recall of 0.77. In the VGHTPE 2022 cohort, the AUROC was 0.92, with precision and recall balanced at 0.74 and 0.76, respectively. The MIMIC IV cohort had a slightly lower AUROC of 0.82, with a precision of 0.72 and a recall of 0.36. The calibration plots showed good alignment between the predicted and observed risks, with Brier scores of 0.082, 0.085, and 0.116 for the VGHTPE 2010, VGHTPE 2022, and MIMIC IV cohorts, respectively. TvHEWS predicted hemodynamic instability for up to 7 h before intervention in the VGHTPE 2010 cohort, 8.6 h in the VGHTPE 2022 cohort, and 21 h in the MIMIC IV cohort, with low false alarm rates. TvHEWS addresses the challenge of early detection of hemodynamic instability by integrating multiple clinical parameters and offering continuous, dynamic risk assessments. It enhances the ability to anticipate and manage critical circulatory issues, potentially improving patient outcomes through earlier interventions. Further prospective validation in other hospitals is needed to confirm its robustness across diverse settings.