External validation of two clinical prediction models for mortality in COVID-19 patients (4C and NEWS2), in three centers in Medellín, Colombia: Assessing the impact of vaccination over time

Abstract

Objectives

External validation of the 4C and NEWS2 scores for the prediction of in-hospital mortality in COVID-19 patients, and evaluation of its operational performance in two time periods: before and after the start of the vaccination program in Colombia.

Methods

Retrospective cohort in three high complexity hospitals in the city of Medellín, Colombia, between June 2020 and April 2022.

Results

The areas under the ROC curve (AUC) for the 4C mortality risk score and the NEWS2 were 0.75 (95% CI 0.73–0.78) and 0.68 (95% CI 0.66–0.71), respectively. For the 4C score, the AUC for the first and second periods was 0.77 (95% CI 0.74–0.80) and 0.75 (95% CI 0.71–0.78); whilst for the NEWS2 score, it was 0.68 (95% CI 0.65–0.71) and 0.69 (95% CI 0.64–0.73). The calibration for both scores was adequate, albeit with reduced performance during the second period.

Conclusions

The 4C mortality risk score proved to be the more adequate predictor of in-hospital mortality in COVID-19 patients in this Latin American population. The operational performance during both time periods remained similar, which shows its utility notwithstanding major changes, including vaccination, as the pandemic evolved.