Validity of different algorithmic methods to identify hospital readmissions from routinely coded medical data

Abstract

Background

Hospital readmission rates are used for quality and pay-for-performance initiatives. To identify readmissions from administrative data, two commonly employed methods are focusing either on unplanned readmissions (used by the Centers for Medicare & Medicaid Services, CMS) or potentially avoidable readmissions (used by commercial vendors such as SQLape or 3 M). However, it is not known which of these methods has higher criterion validity and can more accurately identify actually avoidable readmissions.

Objectives

A manual record review based on data from seven hospitals was used to compare the validity of the methods by CMS and SQLape.

Methods

Seven independent reviewers reviewed 738 single inpatient stays. The sensitivity, specificity, positive predictive value (PPV), and F1 score were examined to characterize the ability of an original CMS method, an adapted version of the CMS method, and the SQLape method to identify unplanned, potentially avoidable, and actually avoidable readmissions.

Results

Both versions of the CMS method had greater sensitivity (92/86% vs. 62%) and a higher PPV (84/91% vs. 71%) than the SQLape method, in terms of identifying their outcomes of interest (unplanned vs. potentially avoidable readmissions, respectively). To distinguish actually avoidable readmissions, the two versions of the CMS method again displayed higher sensitivity (90/85% vs. 66%), although the PPV did not differ significantly between the different methods.

Conclusions

Thus, the CMS method has both higher criterion validity and greater sensitivity for identifying actually avoidable readmissions, compared with the SQLape method. Consequently, the CMS method should primarily be used for quality initiatives.