An application of random forest regression for predicting healthcare costs using administrative databases.

Abstract

Background: Accurate healthcare cost predictions are essential for health economic decision-making. We propose a bottom-up approach that aggregates individual-level predictions to estimate expenditures for specific population segments without relying on a priori segmentation.

Methods: This retrospective population-based study used administrative healthcare data (2011-2023) from the Health Protection Agency of Bergamo (Italy). We traced 5-year healthcare resource utilization to predict individual's healthcare costs in the following year using random forest regression algorithms. Individual predictions were aggregated to derive mean per-capita cost estimates for the overall population and selected high-impact subgroups. Performance was assessed at the individual-level using error metrics, and at the population-level using prediction error (PE).

Results: At the individual-level, algorithms had a poor performance, systematically underestimating high-cost users. However, aggregated predictions were reliable. For the overall population, pre-pandemic PEs ranged from -2.5% to -0.4% and stabilized by 2023 (PE = -0.7%), after 2020-2021 volatility. In high-impact groups, dialysis and diabetic patients showed consistent PE temporal trends not exceeding -10%. Other segments exhibited more marked underestimations, with PE reaching -30%.

Conclusion: Machine learning-based bottom-up approach applied to administrative databases demonstrated potential for population-level public healthcare expenditure forecasting within the geographical context of this study. These findings support further evaluation of this approach for budget planning and health economic applications, while external validation across different areas and administrative settings will be necessary to assess its generalizability.