Leveraging near-real-time patient and population data to incorporate fluctuating risk of severe COVID-19: development and prospective validation of a personalised risk prediction tool.

Abstract

Background: Novel strategies that account for population-level changes in dominant variants, immunity, testing practices and changes in individual risk profiles are needed to identify patients who remain at high risk of severe COVID-19. The aim of this study was to develop and prospectively validate a tool to predict absolute risk of severe COVID-19 incorporating dynamic parameters at the patient and population levels that could be used to inform clinical care.

Methods: A retrospective cohort of vaccinated US Veterans with SARS-CoV-2 from July 1, 2021, through August 25, 2023 was created. Models were estimated using logistic-regression-based machine learning with backward selection and included a variable with fluctuating absolute risk of severe COVID-19 to account for temporal changes. Age, sex, vaccine type, fully boosted status, and prior infection before vaccination were included a priori. Variations in individual risk over time, e.g., due to receipt of immune suppressive medications, were also potentially included. The model was developed using data from July 1, 2021, through August 31, 2022 and prospectively validated on a subsequent second cohort (September 1, 2022, through August 25, 2023). Model performance was quantified by the area under the receiver operating characteristic curve (AUC) and calibration by Brier score. The final model was used to compare observed rates of severe disease to predicted rates among patients who received oral antivirals.

Findings: 216,890 SARS-CoV-2 infections in Veterans not treated with oral antivirals were included (median age, 65; 88% male). The development cohort included 165,303 patients (66,121 in the training set, 49,591 in the tuning set, and 49,591 in the testing set) and the prospective validation cohort included 51,587 patients. The percentage of severe infections ranged from 5% to 25%. Model performance improved until 24 clinical predictor variables including age, co-morbidities, and immune-suppressive medications plus a 30-day rolling risk window were included (AUC in development cohort, 0.88 (95% CI, 0.87-0.88), AUC in prospective validation, 0.85 (95% CI, 0.84-0.85), Brier Score, 0.13). The most important variables for predicting severe disease included age, chronic kidney disease, chronic obstructive pulmonary disease, Alzheimer's disease, heart failure, and anaemia. Glucocorticoid use during the one-month prior to COVID-19 diagnosis was the next most important predictor. Models that included a near-real time fluctuating population risk variable performed better than models stratified by circulating variant and models with dominant variant included as a predictor. Patients with predicted severe disease risk >3% who received oral antivirals had approximately 4-fold lower rates of severe COVID-19 untreated patients at a similar risk level.

Interpretation: Our novel risk prediction tool uses a simple method to adjust for temporal changes and can be implemented to facilitate uptake of evidence-based therapies. The study provides proof-of-concept for leveraging real-time data to support risk prediction that incorporates changing population-level trends and variation patient-level risk.

Funding: This work was supported by the VA Boston Cooperative Studies Programme. WBE was supported by VA HSR&D IIR 20-076; VA HSR&D IIR 20-101; VA National Artificial Intelligence Institute.