Predictive models for low birth weight: a comparative analysis of algorithmic fairness-improving approaches.

Abstract

Objective: Evaluating whether common algorithmic fairness-improving approaches can improve low-birth-weight predictive model performance can provide important implications for population health management and health equity. This study aimed to evaluate alternative approaches for improving algorithmic fairness for low-birth-weight predictive models.

Study design: Retrospective, cross-sectional study of birth certificates linked with medical insurance claims.

Methods: Birth certificates (n = 191,943; 2014-2022) were linked with insurance claims (2013-2021) from the Arkansas All-Payer Claims Database to assess alternative approaches for algorithmic fairness in predictive models for low birth weight (< 2500 g). We fit an original model and compared 6 fairness-improving approaches using elastic net models trained and tested with 70/30 balanced random split samples and 10-fold cross validation.

Results: The original model had lower accuracy (percent predicted correctly) in predicting low birth weight among Black, Native Hawaiian/Other Pacific Islander, Asian, and unknown racial/ethnic populations relative to White individuals. For Black individuals, accuracy increased with all 6 fairness-improving approaches relative to the original model; however, sensitivity (true-positives correctly predicted as low birth weight) significantly declined, as much as 31% (from 0.824 to 0.565), in 5 of 6 approaches.

Conclusions: When developing and implementing decision-making algorithms, it is critical that model performance metrics align with management goals for the predictive tool. In our study, fairness-improving models improved accuracy and area under the curve scores for Black individuals but decreased sensitivity and negative predictive value, suggesting that the original model, although unfair, was not improved. Implementation of unfair models for allocating preventive services could perpetuate racial/ethnic inequities by failing to identify individuals most at risk for a low-birth-weight delivery.