Predicting delirium in intensive care unit patients every 8 hours with machine learning: Model development and evaluation

Objectives

Delirium in the intensive care unit (ICU) is associated with poor short- and long-term outcomes, and identifying patients at risk of delirium during ICU stay remains difficult. This study aims to develop a machine learning–based prediction model for identifying delirium occurrence every 8 hours during ICU stay.

Methods

We retrospectively collected data from the electronic medical records in a single-centre mixed ICU. Adult patients who were admitted to the ICU between January 2023 and December 2023 and spent more than 24 h in the ICU were eligible for this study. The outcome was delirium defined as an Intensive Care Delirium Screening Checklist score of 4 or more. Four machine learning algorithms (XGBoost, LightGBM, CatBoost, and random forest) were used to develop prediction models using a holdout method.

Results

273 patients were included in the study, and 170 patients (62.3%) experienced delirium. The dataset consists of 2321 delirium assessments of which 822 (35.4%) were delirium positive. CatBoost demonstrated the best performance; area under the curve, mean precision, and brier score on the test dataset were 0.886 (95% confidence interval [CI]: 0.857–0.909), 0.804 (95% CI: 0.749–0.852), and 0.131 (95% CI: 0.115–0.149), respectively. The model achieved an accuracy of 0.816 and specificity of 0.900. Precision was 0.775 while maintaining recall at 0.668. Routinely collected nursing observational variables, including Intensive Care Delirium Screening Checklist subscores and the Glasgow Coma Scale score, played a significant role in predicting delirium.

Conclusions

Our machine learning–based prediction model demonstrated potential in identifying patients at risk of delirium. However, further research with a larger sample size and greater heterogeneity in the patient population would be needed to guide nursing interventions. Our prediction model would enable nursing professionals to identify patients at high risk of delirium using only routinely collected variables. Nurses would be able to implement timely preventive care and optimise staffing and the patients’ therapeutic environment to reduce risk factors of delirium.