Development and validation of a machine learning–based model for predicting intraoperative blood loss during burn surgery

Abstract

Background

Intraoperative blood loss is a critical issue in the care of patients with burns. The timely identification of patients at elevated risk for substantial blood loss during surgical procedures is imperative.

Methods

Demographic data, laboratory test results, and surgical factors of patients were collected. For predicting intraoperative blood loss >750 mL, the original cohort was randomly divided in an 8:2 ratio, with the larger group allocated for the model development and the smaller for internal validation. Six machine-learning algorithms, including logistic regression, decision tree, random forest, K-nearest neighbor, support vector machine, and extreme gradient boosting were used to develop the prediction models. The performance of the models was assessed by 8 metrics as well as density curve, calibration curve and decision curve. A scoring system was designed to assess the performance efficacy. Validation was conducted in another 2 cohorts. The optimal prediction model acquired. Ultimately, a web-based calculator to estimate the incidence of intraoperative blood loss >750 mL was created.

Results

A total of 395 burn surgeries from 2 hospitals were analyzed, with 245 surgeries for modeling, 89 surgeries for the internal-external validation, and 61 surgeries for the external validation. The model features consist of 8 clinical variables. The random forest model gained the greatest total metrics score of 36, followed by the support vector machine, extreme gradient boosting, K-nearest neighbor, logistic regression, and decision tree models with total scores of 33, 32, 28, 24, and 18, respectively. Specifically, the random forest model performed superior in most metrics compared to the other models, achieving greater accuracy (0.776), recall (0.930), F1 score (0.868), as well as the lowest log loss (0.423), and Brier score (0.142). Meanwhile, the random forest model demonstrated strong performance with an area under the curve of 0.784 (95% confidence interval, 0.779–0.789), ranking second only slightly behind the extreme gradient boosting model, which achieved the greatest area under the curve of 0.785 (95% confidence interval, 0.780–0.790). Other models showed comparatively lower area under the curve values. The density curve, calibration plot, histogram with mean predicted probabilities against counts and decision curve of the random forest model also performed well. In the internal-external validation cohort, the random forest reached the greatest total metrics score of 35. In the external validation cohort, the random forest again secured the greatest composite metrics score, amounting to 38. Overall, the random forest model was found to be the optimal model for predictive accuracy. In addition initial hemoglobin, time to start surgery after burn injury, initial platelets, % total body surface area excised and grafted, and duration of surgery were the first 5 features to predict intraoperative blood loss. Subsequently, the random forest model, along with the other 5 models, has been deployed online for broader accessibility. https://surgerybloodlosswebapp-m2jzz6hs87qu74vcfrhgz9.streamlit.app/.

Conclusion

The random forest model is a valuable artificial intelligence instrument for predicting the incidence of intraoperative blood loss >750 mL during burn surgery. Several principal predictors should receive attention. The 6 machine-learning algorithms have been used to develop a web-based app. This platform will facilitate subsequent data validation and system optimization.