Improved American College of Surgeons NSQIP Hospital Benchmarking with Risk Adjustment for Many CPT Codes Rather Than Just the Principal Code.

Abstract

Background: Because of technical limitations inherent to logistic regression, NSQIP benchmarking has historically risk adjusted for procedure using only one principal CPT code among other predictors. This has the potential to create bias (favorable or unfavorable) for hospitals depending on how many multiple-procedure operations they do. Hospital quality assessments using current statistical methods were compared to those using a new methodology that permits risk adjustment incorporating many recorded CPT codes (capped here at 21).

Study design: ACS-NSQIP data from 2023, composed of 994,332 patient cases from 676 hospitals were analyzed. Modeling included a preliminary logistic regression step where 5 years of historical data were used to generate a principal CPT-code-specific linear risk score (logit) for each of 14 outcomes. This score is then used as one of many risk-adjustment variables in follow-on models. For this re-analysis, the first step was replicated with a CatBoost machine learning algorithm that provides a logit risk score based on a set of up to 21 reported CPT codes. Changes in hospital assessments using the two approaches to CPT code-based risk were examined.

Results: Benchmarking results for the 14 outcomes were similar, but not identical, across the analytic methods. For 13 out of 14 outcomes studied, the greater the mean number of CPT codes reported for patients in a hospital, the greater their benchmarking advantage when the model considered all codes; hospitals that reported only the principal CPT code had a benchmarking advantage when the model considered only that code.

Conclusion: Because of differences between hospitals in the proportion of multiple-procedure operations performed, risk adjustment using many CPT codes provides more defensible benchmarking results.