Forecasting patient demand at urgent care clinics using explainable machine learning

Abstract

Urgent care clinics and emergency departments around the world periodically suffer from extended wait times beyond patient expectations due to surges in patient flows. The delays arising from inadequate staffing levels during these periods have been linked with adverse clinical outcomes. Previous research into forecasting patient flows has mostly used statistical techniques. These studies have also predominately focussed on short-term forecasts, which have limited practicality for the resourcing of medical personnel. This study joins an emerging body of work which seeks to explore the potential of machine learning algorithms to generate accurate forecasts of patient presentations. Our research uses datasets covering 10 years from two large urgent care clinics to develop long-term patient flow forecasts up to one quarter ahead using a range of state-of-the-art algorithms. A distinctive feature of this study is the use of eXplainable Artificial Intelligence (XAI) tools like Shapely and LIME that enable an in-depth analysis of the behaviour of the models, which would otherwise be uninterpretable. These analysis tools enabled us to explore the ability of the models to adapt to the volatility in patient demand during the COVID-19 pandemic lockdowns and to identify the most impactful variables, resulting in valuable insights into their performance. The results showed that a novel combination of advanced univariate models like Prophet as well as gradient boosting, into an ensemble, delivered the most accurate and consistent solutions on average. This approach generated improvements in the range of 16%–30% over the existing in-house methods for estimating the daily patient flows 90 days ahead.