Development and validation of an electronic quality measure for postoperative respiratory failure

Abstract

Background

Postoperatively, prolonged mechanical ventilation or unplanned intubation—collectively, “postoperative respiratory failure”—has high morbidity and mortality risk, but hospitals' automated ability to detect this complication is currently limited.

Study Design

We developed an electronic clinical quality measure for postoperative respiratory failure on the basis of input from clinical and quality experts. Using 2022 structured electronic health record data from 12 diverse hospitals encompassing 2 common electronic health record vendors, we retrospectively evaluated criteria for postoperative respiratory failure after an operation during an elective hospitalization. We sampled all denominator-eligible cases meeting postoperative respiratory failure criteria, and a subset that did not, from each center. Trained nurse abstractors reviewed medical records using a standard instrument. We assessed the positive and negative predictive value of the measure (with 95% confidence intervals), and the discrimination and calibration of its risk model.

Results

Among 95 records flagged by the measure and 310 denominator-eligible records not flagged, the postoperative respiratory failure electronic clinical quality measure numerator criteria had a positive predictive value of 88.7% (95% confidence intervals, 80.6–94.2%) and negative predictive value of 99.7% (95% confidence intervals, 98.2–100%). False-positive results frequently involved easily correctible respiratory therapist documentation errors. Risk-adjusted rates of postoperative respiratory failure across hospitals ranged from 0.0 to 16.8/1,000 hospitalizations. The risk model, which included 8 comorbidities, 6 laboratory tests, and American Society of Anesthesiologists physical status classification, had a c-statistic of 0.91.

Conclusion

Postoperative respiratory failure can be measured with high validity from readily available, structured electronic health record data. A postoperative respiratory failure electronic clinical quality measure would not be dependent on administrative claims data or collection by trained abstractors, offering the potential to inform quality improvement in elective perioperative care.