Clinically Guided Adaptive Machine Learning Update Strategies for Predicting Severe COVID-19 Outcomes.

Abstract

Background: Machine learning algorithms are essential for predicting severe outcomes during public health crises like COVID-19. However, the dynamic nature of diseases requires continual evaluation and updating of these algorithms. This study aims to compare three update strategies for predicting severe COVID-19 outcomes post-diagnosis: 'naive' (a single initial model), 'frequent' (periodic retraining), and 'context-driven' (retraining informed by clinical insights). The goal is to determine the most effective timing and approach for adapting algorithms to evolving disease dynamics and emerging data.

Methods: A dataset of 1.11 million COVID-19 patients from diverse U.S. regions was used to develop and validate an XGBoost algorithm for predicting severe outcomes upon diagnosis. Data included patient demographics, vital signs, comorbidities, and immunity-related factors (prior infection and vaccination status) from January 2007 to November 2021. The study analyzed the performance of the three update strategies from March 2020 to November 2021.

Results: Predictive features changed over the pandemic, with comorbidities and vitals being significant initially, and geography, demographics, and immunity-related variables gaining importance later. The 'naive' strategy had an average AUC of 0.77, the 'frequent' strategy-maintained stability with an average AUC of 0.81, and the 'context-driven' strategy averaged an AUC of 0.80, outperforming the 'naive' strategy and aligning closely with the 'frequent' strategy.

Conclusion: A context-driven approach, guided by clinical insights, can enhance predictive performance and offer cost-effective solutions for dynamic public health challenges. These findings have significant implications for efficiently managing healthcare resources during evolving disease outbreaks.