Severity scoring systems in intensive care: new models, or continuous recalibration?

Abstract

Purpose of review: To review how ICU illness severity scoring systems perform across settings and over time, and when recalibration is sufficient versus when new models are needed.

Findings: Illness severity scoring systems may consist of a numerical 'score' and/or an equation that translates that score or its components into the risk of an outcome such as mortality. The need for periodic updating of a scoring system depends on its primary purpose and on how its performance (discrimination, calibration and validity) varies over time and in different settings. Calibration is most vulnerable to temporal drift, case-mix change and health-system differences, making periodic recalibration essential for benchmarking. Descriptive scores are more stable and often retain good discrimination. They usually need redevelopment only when the relative importance of contributing variables changes or they no longer reflect contemporary practice. Examples including APACHE, ANZROD, SOFA-2, ICNARC, GOSSIS and machine-learning models show recalibration works when predictors remain relevant, whereas rebuilding is preferable when practice, data or modelling methods change substantially.

Summary: Regular recalibration with periodic rebuilding is generally required for most ICU illness severity scores and prediction models. This should be tailored to the scoring system and the context in which it is used.