Evaluating the predictive performance of different data sources to forecast overdose deaths at the neighborhood level with machine learning in Rhode Island

Abstract

Objectives

To evaluate the predictive performance of different data sources to forecast fatal overdose in Rhode Island neighborhoods, with the goal of providing a template for other jurisdictions interested in predictive analytics to direct overdose prevention resources.

Methods

We evaluated seven combinations of data from six administrative data sources (American Community Survey (ACS) five-year estimates, built environment, emergency medical services non-fatal overdose response, prescription drug monitoring program, carceral release, and historical fatal overdose data). Fatal overdoses in Rhode Island census block groups (CBGs) were predicted using two machine learning approaches: linear regressions and random forests embedded in a nested cross-validation design. We evaluated performance using mean squared error and the percentage of statewide overdoses captured by CBGs forecast to be in top percentiles from 2019 to 2021.

Results

Linear models trained on ACS data combined with one other data source performed well, and comparably to models trained on all available data. Those including emergency medical service, prescription drug monitoring program, or carceral release data with ACS data achieved a priori goals for percentage of statewide overdoses captured by CBGs prioritized by models on average.

Conclusions

Prioritizing neighborhoods for overdose prevention with forecasting is feasible using a simple-to-implement model trained on publicly available ACS data combined with only one other administrative data source in Rhode Island, offering a starting point for other jurisdictions.