On the Implementation of Approximate Randomization Tests in Linear Models with a Small Number of Clusters

Abstract This paper provides a user’s guide to the general theory of approximate randomization tests developed in Canay, Romano, and Shaikh (2017a. “Randomization Tests under an Approximate Symmetry Assumption.” Econometrica 85 (3): 1013–30) when specialized to linear regressions with clustered data. An important feature of the methodology is that it applies to settings in which the number of clusters is small – even as small as five. We provide a step-by-step algorithmic description of how to implement the test and construct confidence intervals for the parameter of interest. In doing so, we additionally present three novel results concerning the methodology: we show that the method admits an equivalent implementation based on weighted scores; we show the test and confidence intervals are invariant to whether the test statistic is studentized or not; and we prove convexity of the confidence intervals for scalar parameters. We also articulate the main requirements underlying the test, emphasizing in particular common pitfalls that researchers may encounter. Finally, we illustrate the use of the methodology with two applications that further illuminate these points: one to a linear regression with clustered data based on Meng, Qian, and Yared (2015. “The Institutional Causes of china’s Great Famine, 1959–1961.” The Review of Economic Studies 82 (4): 1568–611) and a second to a linear regression with temporally dependent data based on Munyo and Rossi (2015. “First-day Criminal Recidivism.” Journal of Public Economics 124: 81–90). The companion R and Stata packages facilitate the implementation of the methodology and the replication of the empirical exercises.