Predicting the Sustainability of Quality Improvement Initiatives in Healthcare: A Data-Driven Study

Abstract

Background

Sustaining the effects of healthcare quality improvement (QI) initiatives remains a persistent challenge. Although sustainability is widely recognized as an important goal of QI, empirical evidence on factors associated with post-project persistence remains limited. This study aimed to identify factors associated with post-project persistence in hospital-based QI initiatives and to examine the feasibility of using a random forest model to classify projects across persistence-duration categories.

Methods

This retrospective cross-sectional study analyzed 123 QI projects implemented at a tertiary medical center in Taiwan between 2015 and 2023. Sustainability was operationalized as the duration of observable post-project effects documented in project records and classified into three categories: < 1 year, 1–3 years, and > 3 years. Twenty candidate factors were assessed using a study-specific questionnaire with acceptable content validity (content validity index [CVI] 0.92). A random forest classifier was developed using 10-fold cross-validation and evaluated using class-specific area under the ROC (receiver operating characteristic) curve (AUC), F1-score, and confusion matrix analysis.

Results

The most influential predictors were alignment with clinical needs (17.67%), substantial benefits to patients (15.06%), team members’ knowledge and skills in QI (14.38%), degree of information technology intervention (10.31%), and improvements in work efficiency (10.31%). Model performance was strongest for the < 1-year and 1–3-year categories, with AUC values of 0.99 and 0.91, respectively. Performance for the > 3-year category was more limited (AUC 0.87, F1-score 0.29), with only one long-term case correctly classified in the test set, likely reflecting class imbalance and the small number of long-term projects. Five critical factors influencing sustainability were identified: alignment with clinical needs, substantial patient benefits, team competency in QI methods, information technology integration, and work efficiency improvements. The model demonstrated high predictive accuracy for short- and mid-term sustainability but lower accuracy for long-term impacts due to sample imbalances.

Conclusion

This study provides preliminary evidence that post-project persistence in healthcare QI initiatives is associated with clinical alignment, patient benefit, team QI competency, information technology support, and work efficiency improvement. The random forest model performed better in distinguishing short- and mid-term persistence than long-term persistence. These findings offer an initial data-driven basis for understanding conditions that may support the persistence of QI effects in practice and support the need for further multicenter validation.