Identifying emergency department patients at high risk for opioid overdose using natural language processing and machine learning

Abstract

Introduction

Emergency departments (ED) are potential sites for identifying and treating individuals at high risk for opioid overdose. This study used machine learning (ML)-based models to predict opioid overdose death in the 12 months after an ED visit.

Methods

The study merged electronic health records (EHR), including clinical notes, of adult patients admitted to an urban safety net ED from 2011 to 2018 with opioid overdose-related mortality tables from 2012 to 2019. The sample includes all patients who experienced an opioid overdose related death (n = 729) and a subset of ED patients that did not (n = 4927). A mutual information classification algorithm was employed for feature selection. Predictive XGBoost, random forest, and regression models trained on 70 % of the sample with the reduced feature matrix and validated on a test set (30 % of sample).

Results

Feature selection reduced the feature matrix from 1336 to 50 features, with 37 originating from EHR clinical notes. Using a probability of >0.5 as a predictor of opioid overdose death, all models demonstrated satisfactory calibration and excellent accuracy, precision, and recall across all models (averaging 92 % accuracy, 75 % precision and 57 % recall).

Conclusion

ML algorithms based on structured and unstructured EHR can successfully classify patients at risk of fatal opioid overdose. Prospectively, these tools can be used to identify patients that may benefit from interventions to reduce their risk of opioid overdose death. The development of these predictive models may improve the timeliness and efficacy of clinical decision making and ED-initiated services for opioid use disorders.