Predicting Prenatal Depression and Assessing Model Bias Using Machine Learning Models

Abstract

Background

Perinatal depression is one of the most common medical complications during pregnancy and postpartum period, affecting 10% to 20% of pregnant individuals, with higher rates among Black and Latina women who are also less likely to be diagnosed and treated. Machine learning (ML) models based on electronic medical records (EMRs) have effectively predicted postpartum depression in middle-class White women but have rarely included sufficient proportions of racial/ethnic minorities, which has contributed to biases in ML models. Our goal is to determine whether ML models could predict depression in early pregnancy in racial/ethnic minority women by leveraging EMR data.

Methods

We extracted EMRs from a large U.S. urban hospital serving mostly low-income Black and Hispanic women (n = 5875). Depressive symptom severity was assessed using the Patient Health Questionnaire-9 self-report questionnaire. We investigated multiple ML classifiers using Shapley additive explanations for model interpretation and determined prediction bias with 4 metrics: disparate impact, equal opportunity difference, and equalized odds (standard deviations of true positives and false positives).

Results

Although the best-performing ML model's (elastic net) performance was low (area under the receiver operating characteristic curve = 0.61), we identified known perinatal depression risk factors such as unplanned pregnancy and being single and underexplored factors such as self-reported pain, lower prenatal vitamin intake, asthma, carrying a male fetus, and lower platelet levels. Despite the sample comprising mostly low-income minority women (54% Black, 27% Latina), the model performed worse for these communities (area under the receiver operating characteristic curve: 57% Black, 59% Latina women vs. 64% White women).

Conclusions

EMR-based ML models could moderately predict early pregnancy depression but exhibited biased performance against low-income minority women.