KG-LIME: predicting individualized risk of adverse drug events for multiple sclerosis disease-modifying therapy.

Abstract

Objective: The aim of this project was to create time-aware, individual-level risk score models for adverse drug events related to multiple sclerosis disease-modifying therapy and to provide interpretable explanations for model prediction behavior.

Materials and methods: We used temporal sequences of observational medical outcomes partnership common data model (OMOP CDM) concepts derived from an electronic health record as model features. Each concept was assigned an embedding representation that was learned from a graph convolution network trained on a knowledge graph (KG) of OMOP concept relationships. Concept embeddings were fed into long short-term memory networks for 1-year adverse event prediction following drug exposure. Finally, we implemented a novel extension of the local interpretable model agnostic explanation (LIME) method, knowledge graph LIME (KG-LIME) to leverage the KG and explain individual predictions of each model.

Results: For a set of 4859 patients, we found that our model was effective at predicting 32 out of 56 adverse event types (P < .05) when compared to demographics and past diagnosis as variables. We also assessed discrimination in the form of area under the curve (AUC = 0.77 ± 0.15) and area under the precision-recall curve (AUC-PR = 0.31 ± 0.27) and assessed calibration in the form of Brier score (BS = 0.04 ± 0.04). Additionally, KG-LIME generated interpretable literature-validated lists of relevant medical concepts used for prediction.

Discussion and conclusion: Many of our risk models demonstrated high calibration and discrimination for adverse event prediction. Furthermore, our novel KG-LIME method was able to utilize the knowledge graph to highlight concepts that were important to prediction. Future work will be required to further explore the temporal window of adverse event occurrence beyond the generic 1-year window used here, particularly for short-term inpatient adverse events and long-term severe adverse events.