Evaluating the generalization of machine learning models for predicting 14-day mortality in traumatic brain injury patients

Abstract

Traumatic Brain Injury (TBI) remains a leading cause of morbidity and mortality worldwide, with significant disparities in outcomes influenced by regional healthcare access and infrastructure. This study evaluates the performance and generalizability of machine learning models for predicting 14-day mortality in TBI patients using datasets from two distinct Brazilian regions: São Paulo, an urban center, and Manaus, an isolated urban center with unique logistical challenges. To our knowledge, this research represents the first cross-validation of predictive models across two datasets within the same country, underscoring the critical need for localized approaches in TBI research. Our findings indicate that while convolutional neural network (CNN)-based models achieved high performance, with an area under the curve (AUC) of 0.90 in São Paulo and 0.93 in Manaus, the best model from São Paulo exhibited a strikingly low AUC when applied to the Manaus dataset. The incorporation of context-specific features, such as pandemic-related variables and time from trauma to admission, significantly enhanced model accuracy, with the Manaus model reaching an impressive AUC of 0.98. Notably, the study highlights key regional differences in predictors of mortality, with hypoxia and hypotension being more critical in Manaus, emphasizing the importance of tailoring predictive models to local contexts. These regionally important variables identified in the ML prediction model may inform quality-improvement priorities and further research in these settings. Our results indicate that CNN-based models have the potential to enhance mortality predictions for patients with traumatic brain injury (TBI).