A symposium on power in experiments – new practical insights and tools: preface

Abstract

The use of randomised control trials (RCTs) has become widespread in economics and other social sciences, and is likely to grow further as new digital tools and increasingly rich data facilitate the design and implementation of experiments. When rigorously designed and implemented, they have the potential to offer the most reliable empirical evidence on the causal impact of an intervention. But for that potential to be realised, RCTs need to be sufficiently powered to detect a meaningful effect, or to say confidently that the effect is negligible. This symposium offers practical insights for researchers on designing more powerful experiments and computing the required sample size, accompanied by tools that researchers can use in designing their own RCTs.

The first paper, by David McKenzie, discusses how to improve power at each stage of an RCT – design, implementation and analysis. While increasing sample size is the default option, McKenzie offers guidance on many other options available to researchers and why they work. The second paper, by Brendon McConnell and Marcos Vera-Hernández, dives into detailed aspects of implementing sample size calculations for different randomisation designs, and offers the formulae, tools and computer code necessary to implement them in practice. The final paper, by Brandon Hauser and Mauricio Olivares, studies hypothesis testing in randomised experiments, and its consequences for sample size calculations. The paper shows how small deviations from the most standard assumptions invalidate standard randomisation-based inference, and provides useful results and guidance for how to adapt power analysis to ensure that calculations remain valid.