A tool to evaluate the integration of implementation science methods in a research project: The Dissemination and Implementation Research Capability Self Survey (DIRC-SS).

Abstract

Introduction: Clinical and Translational Science Awards (CTSAs) are positioned to enhance the integration of rigorous implementation research methods into projects across their networks, but lack a systematic, standardized process to do so. This study introduces the Dissemination and Implementation Research Capability Self Survey (DIRC-SS), a pragmatic instrument to evaluate and integrate implementation science methods in traditional research activities.

Methods: We developed the 15-item DIRC-SS to assess researchers' use of implementation research methods across five key constructs. Its reliability (inter-rater agreement and internal consistency) and sensitivity (change over time) were examined in 10 NIH-funded research projects via ratings assigned by the research teams and by implementation science experts at baseline and one year later.

Results: The DIRC-SS total score demonstrated good internal consistency and inter-rater reliability increased over one year. Although the research team ratings did not change significantly over time, the expert ratings significantly increased, and effect sizes across research teams and expert raters were large in this small sample study.

Conclusions: The DIRC-SS demonstrated good internal consistency reliability and moderate inter-rater reliability. It effectively distinguished between different levels of implementation research methods integration. Unlike tools focused on grant proposals or final reports, the DIRC-SS can be used at any point in the research process by a research team as a self-survey, by implementation science experts in a consultation process, or across a CTSA program to characterize the implementation science methods employed across projects and highlight targeted areas for researcher education and training.