External Validity for Complex Interventions: Towards Pragmatic Strategies for Evaluators.

Abstract

Strategies for ensuring the generalizability of evaluation results often focus on building methodologically sound research designs. However, these strategies may fall short when the challenge lies as much in uncovering causal patterns than in applying them to uncertain contexts. This article proposes a set of pragmatic strategies to address generalizability issues in complex evaluations. These strategies address important questions such as: What needs to be generalized, for whom and for what purpose? What assumptions underlying the intervention could be generalized? How can approaches and methods be selected to support a generalization objective? How can potential users be involved to support the process of generalization? We illustrate these strategies using the example of a series of evaluations of bottom-up socio-ecological transition initiatives in France between 2013 and 2020. We suggest that these strategies can enhance policy learning and better equip policy stakeholders to navigate complex situations.