Substitution Bias and External Validity: Why an Innovative Anti-Poverty Program Showed No Net Impact

Abstract

The net impact of development interventions can depend on the availability of close substitutes to the intervention. We analyze a randomized trial of an innovative anti-poverty program in South India which provides “ultra-poor” households with inputs to create a new, sustainable livelihood. We find no statistically significant evidence of lasting net impact on consumption, income or asset accumulation. Instead, income from the new livelihood substituted for earnings from wage labor. A very similar intervention made a large difference elsewhere in South Asia, however, where wage labor alternatives were less compelling. The analysis highlights the roles of substitution bias and dropout bias in shaping evaluation results and delimiting external validity.