Meta-evaluative practices of Clinical and Translational Research evaluators.

Abstract

Introduction: The National Institutes of Health (NIH) Institutional Development Award (IDeA) program was created to build capacity and enhance research in states with historically low levels of NIH funding. IDeA Clinical and Translational Research (CTR) networks are focused on building statewide and regional capacity to conduct biomedical research. The tracking and evaluation component of each CTR is tasked with collecting data to facilitate continuous improvement and measure impact.

Methods: This paper presents findings from a survey conducted with IDeA-CTR evaluators examining the following questions: 1) To what extent do evaluators use meta-evaluative practices and how does meta-evaluation inform their evaluation? and 2) What challenges evaluators face in their evaluation planning and implementation?

Results: Findings show that 50% of CTRs conducted some form of meta-evaluation. Further, quantitative and qualitative responses tell a compelling story of the challenges in translational research evaluation. The most prominent were the development of feasible and useful data management systems, the selection and endorsement of program-wide impact metrics, and the promulgation of realistic expectations regarding feasibility and utility for recipients of the evaluation, including expectations for project impacts that lead to systemic change.

Conclusions: Findings suggest the importance of internally adopting a participatory, collaborative approach to evaluation and externally sharing insights with and adopting strategies from fellow evaluators within a learning community. This study promotes the value of conducting meta-evaluation in CTR settings, demonstrates means for and results from doing so, and shares best practices for addressing challenges encountered by many CTR evaluators.