Propensity Score Methods and Difference-in-Differences with an Exogenous Time-Varying Confounder: Evaluation of Methods

Abstract Quasi-experimental designs (QEDs) are used to estimate a treatment effect without randomization. Confounders have a causal relationship with the outcome and probability of treatment adoption and if unaccounted for can bias treatment effect estimates. A variable considered a confounder prior to treatment can change after treatment has occurred (i.e., a time-varying confounder) not as a result of treatment (what we call an exogenous time-varying confounder). If the post-treatment value causally affects the outcome to change and this post-treatment value of the exogenous time-varying confounder is unaccounted for, then the treatment effect may be biased. We review the Rubin Causal Model and QED assumptions and the effect an exogenous time-varying confounder has on the ability of QEDs to produce an appropriate counterfactual. We conduct a simulation study evaluating propensity score and difference-in-differences based methods for estimating a treatment effect with an exogenous time-varying confounder. Propensity score weighted two-way fixed effects, inverse probability weighted, or doubly robust difference-in-differences methods, each with propensity scores estimated using post-implementation values of the exogenous time-varying confounder, proved least biased when the exogenous time-varying confounder changed differentially for members of the treatment and control groups.