How can a research program influence public policy? Evaluating a decade of research impact using an evidence-based theory of change

Abstract

This study proposes the systematic empirical development of a theory of change for both planning and evaluating research impact on public policy. This use of theory of change is underdeveloped as a retrospective research impact evaluation method, capable of covering long timespans. Here, we assess impact at the level of a UK water research program with a 10-year timeframe. We developed a program-level theory of change from overlapping data sources. This involved integrating inside-out and outside-in perspectives on impact processes through survey, interview and focus group-style participatory workshops to triangulate a model of policy influence. The result was a triangulated theory of change, refined through an iterative process. This method offers an adaptable evaluation framework that can also provide a robust basis for planning future research impacts. The findings underscore the importance of considering multiple perspectives and evidence sources in understanding research impact pathways, contributing to more effective and impactful research strategies in policy domains. The study’s 10-year scope also shows the potential for evidence-based theories of change to explain some of the long-term, complex dynamics that enable research to influence policy.