Leverage machine learning to identify key measures in hospital operations management: a retrospective study to explore feasibility and performance of four common algorithms.

Abstract

Background: Measures in operations management are pivotal for monitoring and assessing various aspects of hospital performance. Existing literature highlights the importance of regularly updating key management measures to reflect changing trends and organizational goals. Advancements in machine learning (ML) have presented promising opportunities for enhancing the process of updating operations management measures. However, their specific application and performance remain relatively unexplored. We aimed to investigate the feasibility and effectiveness of using common ML techniques to identify and update key measures in hospital operations management.

Methods: Historical data on 43 measures on financial balance and quality of care under 4 categories were retrieved from the BI system of a regional health system in Central China. The dataset included 17 surgical and 15 non-surgical departments over 48 months. Four common ML techniques, linear models (LM), random forest (RF), partial least squares (PLS), and neural networks (NN), were used to identify the most important measures. Ordinary least square was employed to investigate the impact of the top 10 measures. A ground truth validation compared the ML-identified key measures against the humanly decided strategic measures from annual meeting minutes.

Results: For financial balancing, inpatient treatment revenue was an important measure in 3/4 years, followed by equipment depreciation costs. The measures identified using the same technique differed between years, though RF and PLS yielded relatively consistent results. For quality of care, none of the ML-identified measures repeated over the years. Those consistently important over four years differed almost entirely among four techniques. On ground truth validation, the 2016-2019 ML-identified measures were among the humanly identified measures, with the exception of equipment depreciation from the 2019 dataset. All the ML-identified measures for quality of care failed to coincide with the humanly decided measures.

Conclusions: Using ML to identify key hospital operational measures is viable but performance of ML techniques vary considerably. RF performs best among the four techniques in identifying key measures in financial balance. None of the ML techniques seem effective for identifying quality of care measures. ML is suggested as a decision support tool to remind and inspire decision-makers in certain aspects of hospital operations management.