Vital Signs–Only Machine Learning Model for Acute Inpatient Deterioration: A Retrospective Multicenter Study

Abstract

Objective

To develop predictive models that are compatible with vital signs monitoring devices to identify patients at risk of clinical deterioration, defined as requiring a rapid response team intervention or an unplanned intensive care unit transfer.

Patients and Methods

Targeted vital signs from 227,858 inpatients admitted to general care or telemetry beds at a multihospital health care institution between January 1, 2019, and July 31, 2023, were selected. After filtering for high-quality data, 30,118 patients were used to train a Light Gradient Boosting Machine, and 30,095 were reserved for blind validation. We developed a machine learning model designed to minimize false positives while maintaining clinical relevance in identifying low-prevalence clinical deterioration events.

Results

At a sensitivity of 73.4% (95% CI, 72.2%-74.4%), the model achieved a positive predictive value (PPV) of 30.4% (95% CI, 29.6%-31.3%), with a C-statistic of 0.874 (95% CI, 0.867-0.881), alert rate of 0.170 (95% CI, 0.167-0.173) per patient per day, and normalized alert rate of 2.41 (95% CI, 2.31-2.51). Stratified analysis by hospital revealed that PPV was highest at the Rochester site, reaching 54.9% (95% CI, 52.9%-57.0%) and outperforming the EPIC deterioration index by 46% or a factor of 6 (7.57%).

Conclusion

Achieving a high PPV is crucial because it ensures a larger proportion of alerts are true positives, reducing the burden of false alarms. The considerable improvement in results comes from the novel 2-window feature extraction method. This technique enables the model to capture both long-term trends and recent changes in patient status, enhancing predictive performance.