A structured approach to modifying an implementation package while scaling up a complex evidence-based practice

Objective

To describe a structured, iterative, data-driven approach for modifying implementation strategies for a complex evidence-based practice during a nationwide scale-up initiative.

Data Sources and Study Setting

We scaled-up implementation of Critical Time Intervention (CTI)—an evidence-based case management model—across 32 diverse community-based Veterans Affairs (VA) “Grant and Per Diem” case management (GPD-CM) agencies that serve homeless-experienced Veterans transitioning to independent living. Primary data were collected using qualitative methods.

Study Design

We embarked on a scale-up initiative while conducting a pragmatic randomized evaluation using a roll-out design, comparing two versions of a CTI implementation package tailored to VA's GPD-CM program. We iteratively assessed contextual factors and implementation outcomes (e.g., acceptability); findings informed package modifications that were characterized using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies.

Data Collection Methods

We conducted semi-structured interviews with Veterans, GPD-CM staff, and liaising VA clinicians; periodic reflections with liaising VA clinicians and implementation team members; and drew upon detailed meeting notes. We used rapid qualitative methods and content analysis to integrate data and characterize modifications.

Principal Findings

After each scale-up wave—in response to variations in agency-level characteristics— we made iterative modifications to the implementation package to increase CTI adoption and fidelity across the diverse contexts of our scale-up sites. Modifications included adding, deleting, integrating, and altering the package; core package components were preserved.

Conclusions

Implementation packages for complex evidence-based practices undergoing scale-up in diverse contexts may benefit from iterative modifications to optimize practice adoption with fidelity. We offer a structured, pragmatic approach for iteratively identifying data-driven, midstream implementation package adjustments, for use in both VA and non-VA scale-up initiatives. Our project demonstrates the importance of assessing for and making modifications in a scale-up initiative, as well as the trade-offs of projects having simultaneous formative and summative evaluation aims.