What do we know about how the Program Evaluation Standards are used in public health?

Background: Released by the Centers for Disease Control and Prevention (CDC), Framework for Program Evaluation in Public Health prominently features the program evaluation standards (1999). The program evaluation standards (PES) include 30 statements in five domains: utility, feasibility, propriety, accuracy, and evaluation accountability. Despite decades of attention to the PES among framework users and others, how public health professionals apply these standards in their work is not well understood. Purpose: The study sought to identify notable commonalities in how the PES are used in public health. Setting: Application of the PES in evaluative work in public health and allied fields. Intervention: Not applicable. Research Design:  The study included a search of subscription and nonsubscription sources to identify documents that included explicit content concerning use of standards in evaluative work in public health. Documents identified were screened using predetermined criteria to include or exclude each item in the study. Items included were reviewed and coded using codes developed before examining all documents. For each code, reviewers discussed data from all documents to identify commonalities and variations in application of standards. Findings: The literature search returned 405 documents to be screened (179 from subscription and 226 from nonsubscription sources). Thirty-eight items were included in the study based on initial screening (11 from subscription and 27 from nonsubscription sources). The study revealed that authors discussed standards as a regular component of evaluation work, but precisely how standards were used was not always explained in detail. Also, authors did not always discuss standards statements but sometimes solely focused on general domains (e.g., feasibility or accuracy). When authors discussed specific statements, they were more descriptive in how they applied the PES (i.e., compared with articles that focused on general domains). Overall, authors placed far greater emphasis on Accuracy and Utility standards, compared with Propriety, Evaluation Accountability, or Feasibility. In many cases, authors used the PES in combination with other resources (e.g., checklists, guidelines, or other standards). Although program evaluation is crucial to public health practice, the mechanics of how professionals consider, integrate, or use evaluation standards is not fully understood. Keywords: program evaluation; program evaluation standards; public health