Modernizing CTSA hub evaluation: an integrated system for performance monitoring and translational science impact assessment.

Background: The Evaluation Unit at the Altman Clinical and Translational Research Institute (ACTRI) implemented a balanced scorecard model in conjunction with a project management tool to consolidate data collection for progress monitoring, strategic alignment, and impact assessment. This approach aims to streamline communication and enhance information accessibility for all partners. We developed an efficient system for collecting, analyzing, and reporting key information on unit progress, impact, and alignment with institutional goals. The Translational Science Benefits Model (TSBM) was proposed as a framework to evaluate the broader impact of our translational research, beyond immediate scientific advancements, across clinical, societal, economic, and policy domains.

Methods: The ACTRI Evaluation Unit initially adapted the balanced scorecard (BSC) to the research environment, substituting business perspectives with research grant aims. In its second iteration, the BSC was integrated into Monday.com, a project management platform, to create customized, real-time monitoring dashboards for each unit within the institute. The Evaluation Unit's 3.0 version further adapted the TSBM to assess the broader impacts of unit activities. Quarterly data collection was implemented, and partners were trained in impact assessment and dashboard usage. This process began in early 2023 and is ongoing.

Results: Eleven monitoring dashboards were developed and successfully implemented across the institute. The new system facilitated more efficient data collection and reporting, reducing communication overhead and increasing the frequency of updates. The data collected were utilized to draft annual reports as well as inform strategic planning and executive sessions.

Conclusions: Integrating the TSBM into our existing BSC framework, combined with a project management tool, effectively streamlined impact assessment and progress monitoring. This approach not only enhanced data collection and reporting efficiency but also encouraged units to align their goals and activities with desired impacts, thereby strengthening the institute's overall strategic focus.