Overlap Weights for Binary Outcomes: A Performance Assessment.

Abstract

Background: Inverse probability weighting (IPW) is a widely used method to estimate the causal effect of treatment from observational data. However, it can be unstable when extreme propensity score (PS) values lead to very large weights. Overlap weights (OW), which emphasize subjects in areas of covariate overlap, reduce the influence of extreme PS without excluding participants. While the OW method has shown strong performance in simulations with continuous outcomes, its utility in binary outcome settings-common in health research-has not been thoroughly evaluated.

Methods: We conducted simulation studies to evaluate the performance of OW in comparison to other PS weighting methods including IPW, trimmed IPW, and matching weights, in settings with extreme PS values and a binary outcome. Using simulated datasets with varying degrees of PS overlap and treatment prevalence, we assessed covariate balance and treatment effect estimation performance. The performance of the PS weighting methods was further illustrated through an application to data from a study on pancreatic ductal adenocarcinoma.

Results: In simulation studies, IPW's performance deteriorated markedly as the overlap in the covariate distribution decreased. In contrast, OW achieved exact covariate balance and consistently showed the highest efficiency among all methods evaluated. In the application to real-world data characterized by low treatment prevalence and substantial covariate imbalance, OW also outperformed the other methods in terms of both standard error and covariate balance.

Conclusion: These findings suggest superior performance of OW in terms of covariate balance and estimation efficiency in settings with extreme PS and a binary outcome.