Rethinking Paired-Catchment Studies: Should We Be Replicating Our Controls?

Paired-catchment studies are widely used to examine the effects of land management practices (“treatments”) on hydrologic processes. Catchments are matched and a pretreatment calibration regression is used to identify the hydrological relationship between the reference and treated catchments. This method assumes that the calibration regression represents the actual relationship between the catchments (assumption of representativeness) and that the relationship will remain stable over time (assumption of stability). Errors are assumed to be small and similar between reference and treated catchments. Thus, observed differences between the catchments following treatment are assumed to result from that treatment alone. However, calibration periods are often short and it is impossible to know if the calibration period is representative. Further, because the study is unreplicated, it is impossible to determine if stability is maintained. Consequently, it is difficult to determine a minimum detectable effect sizes (MDES) below which estimates of changes in streamflow are statistically uncertain. Here, we use bootstrapped sampling from reference-by-reference (RxR) comparisons in a paired-catchment study design to evaluate the MDES. We generate frequency distributions of the potential changes in flow—changes that cannot be caused by treatment effects. From these, we estimate bootstrapped ±95% confidence intervals encompassing the non-treatment effects which we use as the MDES. We apply this method to long-term paired-catchment studies and reexamine changes in both annual water yields and late summer low flows at the HJA Experimental Forest. This bootstrapping method is widely transferable to any long-term paired catchment study sites where multiple reference catchments exist.