Development and external validation of an artificial intelligence model for predicting mortality and prolonged ICU stay in postoperative critically ill patients: a retrospective study

Abstract

Existing predictive models in critical care, specifically for postoperative critically ill patients, often struggle to accurately predict prolonged intensive care unit (ICU) stays, a key aspect of patient care. The integration of artificial intelligence (AI) offers a promising approach for bridging this gap. We aimed to develop an AI-based model to predict mortality and prolonged ICU stay in postoperative critically ill patients, enhance prognostic accuracy, and address the shortcomings of current models. This retrospective study included data from 6,029 postoperative critically ill patients from two medical centers, including a wide range of clinical, surgical, and laboratory variables. Multiple machine-learning models, including extreme gradient boosting, light gradient boosting, category boosting, random forest, and multilayer perceptron, were employed. A soft-voting ensemble model was developed to aggregate the strengths of individual models. The models underwent external validation, and the SHapley Additive exPlanations (SHAP) method was utilized to assess the impact of various features on predictions. In internal validation, the ensemble model demonstrated superior performance with an area under the receiver operating characteristic curve (AUROC) of 0.8812 for mortality and 0.7944 for prolonged ICU stay. It achieved 0.9095 accuracy and an F1 score of 0.7014 for mortality predictions. For prolonged ICU stay, it attained an accuracy of 0.9368 and an F1 score of 0.5762. During external validation, the model maintained high performance, with an AUROC of 0.8330 for mortality and 0.7376 for prolonged ICU stay. It showed 0.9200 accuracy and an F1 score of 0.6768 for mortality and 0.9028 accuracy with an F1 score of 0.5689 for prolonged ICU stay. SHAP analysis confirmed that key predictors, including emergency surgery, serum osmolality, lactate levels, and diastolic blood pressure, remained significant. This study represents a significant advancement in the application of AI in critical care, especially for postoperative critically ill patients. The developed AI model outperformed existing models in predicting mortality and prolonged ICU stay, demonstrating notable accuracy and reliability. Its ability to identify critical, under-emphasized clinical factors could enhance decision-making in critical care settings. Although promising, further validation in diverse clinical settings is essential to confirm the model’s efficacy and broader applicability.