Development and external validation of machine learning models for the early prediction of malnutrition in critically ill patients: a prospective observational study.

Abstract

Background: Early detection of malnutrition in critically ill patients is crucial for timely intervention and improved clinical outcomes. However, identifying individuals at risk remains challenging due to the complexity and variability of patient conditions. This study aimed to develop and externally validate machine learning models for predicting malnutrition within 24 h of intensive care unit (ICU) admission, culminating in a web-based malnutrition prediction tool for clinical decision support.

Methods: A total of 1006 critically ill adult patients (aged ≥ 18 years) were included in the model development group, and 300 adult patients comprised the external validation group. The development data were partitioned into training (80%) and testing (20%) sets. Hyperparameters were optimized via 5-fold cross-validation on the training set, eliminating the need for a separate validation set while ensuring internal validation. External validation was performed on an independent group to assess generalizability. Predictors were selected using random forest recursive feature elimination; seven machine learning models-Extreme Gradient Boosting (XGBoost), random forest, decision tree, support vector machine (SVM), Gaussian naive Bayes, k-nearest neighbor (k-NN), and logistic regression-were trained and evaluated for accuracy, precision, recall, F1 score, Area Under the Receiver Operating Characteristic Curve (AUC-ROC), Area Under the Precision-Recall Curve (AUC-PR). Model interpretability was analyzed using SHapley Additive exPlanations (SHAP) to quantify feature contributions.

Results: In the development phase, among 1006 patients, 34.0% had moderate malnutrition and 17.9% severe malnutrition. The XGBoost model achieved superior predictive accuracy with an accuracy of 0.90 (95% CI = 0.86-0.94), precision of 0.92 (95% CI = 0.88-0.95), recall of 0.92 (95% CI = 0.89-0.95), F1 score of 0.92 (95% CI = 0.89-0.95), AUC-ROC of 0.98 (95% CI = 0.96-0.99), and AUC-PR of 0.97 (95% CI = 0.95-0.99) on the testing set. External validation confirmed robust performance with an accuracy of 0.75 (95% CI: 0.70-0.79), precision of 0.79 (95% CI: 0.75-0.83), recall of 0.75 (95% CI: 0.70-0.79), F1 score of 0.74 (95% CI: 0.69-0.78), AUC-ROC of 0.88 (95% CI: 0.86-0.91), and AUC-PR of 0.77 (95% CI: 0.73-0.80).

Conclusions: Machine learning models, particularly XGBoost, demonstrated promising performance in early malnutrition prediction in ICU settings. The resultant web-based tool offers valuable resource for clinical decision support.

Trial registration: Chinese Clinical Trial Registry ChiCTR2200058286 ( https://www.chictr.org.cn/bin/project/edit? pid=248690 ). Registered 4th April 2022. Prospectively registered.