What randomization can and cannot guarantee.

Abstract

Aronow et al. (2024) provide a great service to the causal inference community by delineating the key results in Robins and Ritov (1997). They show that randomized controlled trials (RCTs) ensure much stronger statistical inference than unconfounded observational studies even though nonparametric identification is identical in both settings. These results are in sharp contrast to the claim in Pearl and Mackenzie (2018) that RCTs are not the gold standard of causal analysis. Pearl and Mackenzie's (2018) claim is false and misleading for empirical researchers who want to infer causal effects based on data with finite sample sizes. I will further review what randomization can and cannot guarantee more broadly. In particular, I will highlight the value of randomization-based inference in RCTs, the limit of randomization alone for more complicated causal inference questions, and the importance of sensitivity analysis in observational studies.