Nonparametric identification is not enough, but randomized controlled trials are.

Abstract

We argue that randomized controlled trials (RCTs) are special even among studies for which a nonparametric unconfoundedness assumption is credible. This claim follows from two results of Robins and Ritov (1997). First, in settings with at least one continuous confounder, there exists no estimator of the average treatment effect that is uniformly consistent unless the propensity score is known or additional assumptions are made on the complexity of the propensity score function. Second, with binary outcomes, knowledge of the propensity score yields a uniformly consistent estimator and finite-sample valid confidence intervals that shrink at a parametric rate, regardless of how complicated the propensity score function might be. We emphasize the latter point, and note that a successfully executed RCT provides knowledge of the propensity score to the researcher. We conclude that statistical estimation and inference tend to be fundamentally more difficult in observational settings than in RCTs, even when all confounders are observed and measured without error.