Does randomization assert the balance across trial arms? Revisiting Worrall's criticism.

Abstract

We revisit John Worrall's old but still prominent argument against the view that randomization balances the impact of both known and unknown confounders across the treatment and control arms. We argue that his argument involving indefinitely many possible confounders is at odds with statistical theory as it (1) presumes that the purpose of randomized studies is obtaining perfect point estimates for which perfect balance is needed; (2) mistakes equalizing each confounder with the overall (average) impact of all confounders, and (3) assumes that the joint effect of an infinite series of confounders cannot be bounded. We defend the role of randomization in balancing the impact of confounders across the treatment and control arms by putting forward the statistical sense of the balance claim. It involves the following three commitments: (1) randomization balances confounders in expectancy, (2) for RCTs to deliver unbiased estimates of the causal effect (true average treatment effect), the balance in the average effect of all confounders and not balancing each confounder is sufficient, and (3) randomization allows for calculating the probability of deviating from the balance. The paper includes a review of how the balance claim has been understood so far and discusses recent arguments supporting randomization balancing the impact of confounders in expectancy and the crucial role of the average impact of all actual confounders, and shows how statistical analysis of RCTs conducted both at the design and analysis stage makes possible estimating the probabilities of deviating from the balance.