Validating ICD-10 Algorithms for Identifying Patient Safety Indicators Through 10,655 Charts Review.

Abstract

Background: Patient Safety Indicators (PSIs) derived from administrative data are widely used for monitoring and improving hospital care quality. However, the validity of ICD-10-based PSI algorithms remains uncertain, particularly in terms of their sensitivity and specificity.

Objectives: To evaluate the diagnostic performance of ICD-10-CA-based algorithms for identifying fifteen PSIs using chart review as the reference standard.

Research design: Multicenter retrospective cohort validation study.

Subjects: A random sample of 10,665 adult patients admitted to 4 acute care hospitals in Calgary, Alberta, between January 1, 2017, and March 31, 2022.

Measures: Fifteen PSIs were identified using ICD-10-CA codes and validated against detailed chart reviews. Diagnostic performance was measured using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy. Analyses were stratified by diagnosis code type and relevant patient characteristics.

Results: Among 10,665 patients, 1688 had at least one PSI confirmed by chart review. ICD-10-CA coding detected any PSI with 67.0% sensitivity (95% CI, 64.7%-69.2%), 72.8% specificity (95% CI, 71.8%-73.7%), 31.6% PPV (95% CI, 30.1%-33.1%), 92.2% NPV (95% CI, 91.5%-92.8%), and 71.8% accuracy (95% CI, 71.0%-72.7%). Restricting PSIs to conditions that occurred after admission (limited diagnosis type II code) improved specificity (95.7%; 95% CI, 95.3%-96.1%) and PPV (56.5%; 95% CI, 53.2%-59.7%) but reduced sensitivity (29.6%; 95% CI, 27.4%-31.8%). Validity varied by PSI and patient characteristics, with higher sensitivity and PPV among older adults, males, and those with greater comorbidity, longer hospital and ICU stays, 30-day readmission, or in-hospital death.

Conclusions: ICD-10 coded administrative data demonstrate high specificity and NPV but varied sensitivity and PPV in identifying PSIs. Restricting to type II codes improves PPV but reduces sensitivity. Tailoring coding strategies to specific surveillance or quality improvement goals is critical.