Evaluation of the Centers for Disease Control and Prevention-Harvard T.H. Chan School of Public Health Program Evaluation Practicum.

Abstract

Objective: A Maternal and Child Health (MCH) workforce skilled in program evaluation is vital for program assessment and data-driven decision making. Collaborative evaluation capacity-building opportunities for state, tribal, local, and territorial (STLT) public health staff, partner organizations, and students are key strategies in building a skilled workforce, promoting program improvement, effectiveness, and sustainability to improve long-term MCH outcomes. We evaluated the Centers for Disease Control and Prevention and Harvard T.H. Chan School of Public Health Program Evaluation Practicum to determine if it contributed to STLT participants' ability to implement evaluation plans responsive to their MCH programs and gather recommendations for Practicum improvement.

Design: Qualitative one-on-one interviews; data collected July-August 2023.

Participants: Eighteen STLT public health staff, including Centers for Disease Control and Prevention Maternal and Child Health Epidemiology Program (MCHEP) field assignees, who participated in the Practicum between 2018 and 2022.

Main outcome measures: Experiences of STLT MCH program staff and Centers for Disease Control and Prevention MCHEP field assignees.

Results: Six facilitator themes and 6 barrier themes for evaluating MCH programs were identified. Facilitators: an appreciation for evaluation from program partners and recipients; data availability; resources to support evaluation tasks; funding; internal program evaluation capacity; and organizational support. Barriers: data or systems access issues; staff turnover; shifting priorities; lack of evaluation readiness; limited evaluation capacity; and data collection issues. Four lessons-learned themes were identified: need for improved data collection tools; identification of future training needs; facilitation of programmatic decisions; and facilitation of evaluation buy-in. Five themes for recommendations on improving the Practicum were identified: collaborative learning and real-world examples; more evaluation support; modifying the Practicum structure; establishing appropriate Practicum teams; and helping programs determine evaluation readiness.

Conclusions: Collaborative evaluation training like the Practicum can strengthen evaluation capacity within MCH programs and support program improvement. However, the Practicum may benefit from updated guidance, content, and structure to better assess participants' evaluation readiness and meet participants' needs.