On the same (evaluation) page: a novel approach to enhance mixed-methods implementation evaluation.

Abstract

Quality improvement and implementation science evaluations are often complex mixed methods approaches conducted by interdisciplinary teams. These efforts are challenging in normal circumstances; the COVID-19 pandemic and subsequent shift to remote work have posed additional challenges to this type of work. Novel approaches, tools, and processes may be needed to improve the rigor and cohesion of mixed methods evaluations, especially with a team working remotely.Quality improvement and implementation science evaluations are often complex mixed methods approaches conducted by interdisciplinary teams. These efforts are challenging in normal circumstances; the COVID-19 pandemic and subsequent shift to remote work have posed additional challenges to this type of work. Novel approaches, tools, and processes may be needed to improve the rigor and cohesion of mixed methods evaluations, especially with a team working remotely.Our aim was to create a rigorous evaluation plan for a large hybrid type III implementation-evaluation trial implementing new evidence-based processes at nine medical centers. Given the trial's complexity and a geographically-distributed remotely-working interdisciplinary team, we found that existing tools did not meet our needs. We thus created a novel process for developing a rigorous evaluation plan that others could replicate.This process has seven steps: 1) select a template and identify point person; 2) complete initial development; 3) obtain targeted asynchronous feedback; 4) identify and analyze gaps; 5) conduct targeted virtual synchronous discussion; 6) finalize working document; and 7) apply the plan and solicit ongoing feedback.Interdisciplinary quality improvement and implementation science project teams need tools and processes to ensure clear communication, well-ordered workflow, and rigorous operationalization of evaluation aims. The seven-step evaluation plan tool not only helped to enhance the rigor and execution of a large program evaluation, but the process also served an important convening function to enhance coordination between remote team members. Our work builds on existing processes for evaluation plan development while incorporating team science approaches.