We can evaluate rapidly, but should we? Researchers' and research funders' perspectives on the uses, challenges and limitations of rapid health care evaluation.

Abstract

ObjectivesThere is increasing demand for rapid evaluation in health care to inform timely policy and practice decision-making. This qualitative study explored the perceived benefits, limitations and challenges of rapid evaluation, focusing on how considerations of timescale and research quality are balanced in study design and delivery in England.MethodsWe conducted fifteen semi-structured interviews with researchers and research funders involved in rapid evaluation, based in England. Data were thematically analysed using the Framework Method.ResultsResults are reported around five major themes: (i) rapid evaluations are purpose driven; (ii) 'good enough' evidence; (iii) trade-offs and limitations; (iv) mitigating the speed and rigour trade-off; and (v) deciding if and when to evaluate rapidly. Study participants agreed that rapid evaluation reflected a drive to better align evaluative processes and outcomes to the needs of service planners and policymakers. It was seen to generate quick data for short-term requirements, and information to justify the need for, and inform the design of, longer-term assessments. However, working rapidly could restrict or prohibit some research activities, and there were particular concerns about recruitment being limited to sites and participants that were easier to access in short timescales. Rapid evaluation was considered less suitable for 'high stakes' topics or decisions, where evidence robustness and generalisability was paramount. Several study participants had built an infrastructure to facilitate rapid working which, at least in part, reduced the need to make methodological compromises.ConclusionsRapid evaluation can support real-time learning for innovation and improvement and inform time-critical decisions, but timeliness is only one factor in the production of useful and usable evidence. It is a tool for specific circumstances and purposes, to be used alongside, rather than instead of, long-term and longitudinal designs.