Evaluation and Program Planning最新文献

{"title":"Evaluating gendered fault lines: Nigeria’s COVID-19 stimulus through a feminist political economy and OECD-DAC lens","authors":"Chukwuemeka Onyebuchi Onyimadu ,&nbsp;Oluwaseun Blessing Adetutu","doi":"10.1016/j.evalprogplan.2025.102694","DOIUrl":"10.1016/j.evalprogplan.2025.102694","url":null,"abstract":"<div><div>This paper examines the gender responsiveness of Nigeria’s COVID-19 fiscal stimulus, the Economic Sustainability Plan (ESP), through a feminist political economy lens, using the UN Women–ILO gender assessment toolkit. By analyzing macroeconomic data, sector trends, and policies, it assesses whether the ESP effectively addressed the pandemic's uneven socio-economic impacts on women, particularly in the informal sector. The results show that although the ESP included interventions for vulnerable groups and women-led businesses, it lacked a comprehensive strategy for gender mainstreaming and failed to address underlying inequalities in care work, labour participation, and access to health and social services. The paper calls for policy reforms that prioritize women’s economic independence and social protection systems. Beyond its policy relevance, this study contributes to evaluation research by demonstrating how macroeconomic stimulus programs can be assessed using OECD-DAC evaluation criteria and a feminist political economy perspective. This demonstrates how gender-sensitive evaluation frameworks can yield valuable insights for national policy design and inform approach shows how gender-sensitive evaluation frameworks can produce insights for national policy design and the international evaluation community. The study adds to the discussion on gender-responsive recovery in Africa by providing evidence-based recommendations for future fiscal policy responses.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102694"},"PeriodicalIF":2.0,"publicationDate":"2025-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145049204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Systems exploration workshops: Using a virtual group model building approach to advance public health systems change","authors":"Jessica Simon ,&nbsp;Isabella Guynn ,&nbsp;Chantelle Dowsett ,&nbsp;Kristen Hassmiller Lich","doi":"10.1016/j.evalprogplan.2025.102698","DOIUrl":"10.1016/j.evalprogplan.2025.102698","url":null,"abstract":"<div><div>Creating well-functioning public health systems is complex and requires collaboration across often fragmented parts of the system. Group Model Building (GMB) is an evidence-based method that engages diverse partners in understanding complexity and identifying opportunities for systems change. Through a structured sequence of scripted activities, GMB supports the development of shared insights into the dynamics that shape outcomes in complex, change-resistant systems. The National Maternal and Child Health (MCH) Workforce Development Center supports state and jurisdictional MCH professionals through training, coaching, collaborative learning, and consultation. As part of a year-long cohort experience, the MCH Workforce Development Center facilitates a virtual two-day Systems Exploration Workshop to implement GMB activities with teams led by state MCH agencies to explore systemic challenges ranging from improving care coordination to advancing health equity. This paper presents a structured walkthrough of the virtual GMB Systems Exploration Workshop to encourage replication and adaptation in other settings. Evaluation results from 74 participants across 11 states indicate that the workshop fosters a shared understanding of focal systems. Teams reported an enhanced ability to collectively analyze their primary challenges, develop shared mental models of their key determinants, and prioritize impactful interventions. This virtual GMB Systems Exploration Workshop is a valuable approach to accelerate collaborative decision-making through a structured process that facilitates inclusion of diverse perspectives. This workshop offers a practical, systems-informed approach for public health teams seeking to build collective insight, strengthen early-stage strategic planning, and align evaluation efforts with the complexity of the systems in which they are undertaken.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102698"},"PeriodicalIF":2.0,"publicationDate":"2025-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145020943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A formal model for assessing the learning outcomes of academic programs","authors":"Manuel F. Caro ,&nbsp;Elviara P. Flórez ,&nbsp;Isabel C. Muñoz","doi":"10.1016/j.evalprogplan.2025.102644","DOIUrl":"10.1016/j.evalprogplan.2025.102644","url":null,"abstract":"<div><div>Many academic programs face persistent challenges in evaluating learning outcomes with consistency, traceability, and alignment to curricular goals. This study introduces a formal model for the systematic assessment of learning outcomes across an academic program. The model is grounded in set theory and matrix algebra, integrates the SOLO taxonomy to classify levels of cognitive performance, and structures evaluation across three defined assessment moments: early, mid, and final stages of the academic trajectory.</div><div>The model was applied to a five-year teacher training program in Computer Science and validated through empirical data collection and statistical analysis. A paired t-test confirmed a significant decline in learning outcome performance between the initial and final stages of the program, while ANOVA results indicated consistent performance across courses at each stage. Based on these findings, the model also supports the generation of outcome-specific improvement plans, recalibrating performance expectations iteratively.</div><div>The proposed model contributes to outcome-based education by offering a replicable framework for aligning course-level evidence with program-level competencies. Its integration of structured assessment, cognitive benchmarking, and iterative feedback enables academic institutions to monitor student achievement longitudinally and support continuous curriculum improvement.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102644"},"PeriodicalIF":2.0,"publicationDate":"2025-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145020944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Analysis of the degree of implementation of the HEALTHQUAL program","authors":"Joseph Adrien Emmanuel Demes ,&nbsp;Lambert Farand ,&nbsp;Marie-Pascale Pomey ,&nbsp;François Champagne","doi":"10.1016/j.evalprogplan.2025.102699","DOIUrl":"10.1016/j.evalprogplan.2025.102699","url":null,"abstract":"<div><h3>Introduction</h3><div>In evaluations of quality improvement programs, there is often a tendency to focus directly on outcomes or effects without first examining whether the program was fully implemented, only partially executed, or not implemented at all. In many cases, evaluators conclude that the program had little or no effect. However, it is difficult, if not impossible, to achieve meaningful results when the degree of implementation is low or nonexistent. This underscores the importance of assessing the degree of implementation as a prerequisite for explaining outcome gaps, analyzing the implementation process, and understanding the interaction between implementation level, context, and results. Generally, the higher the level of implementation, the more significant the potential outcomes.</div><div>Nevertheless, evaluating the degree of implementation remains a methodological challenge. The objective of this article is to rigorously assess the level of operationalization or implementation of a national quality improvement initiative, the HEALTHQUAL-Haiti program.</div><div>The HEALTHQUAL-Haiti program focuses on quality measurement, quality improvement activities, and the development of quality infrastructure, with the goal of enhancing patient and client satisfaction, strengthening health system performance, and improving the overall quality of care.</div></div><div><h3>Methodology</h3><div>We developed the logic model for the HEALTHQUAL-Haiti program. Building on this model, we selected twelve key dimensions or components for qualitative assessment of the program’s degree of implementation. These dimensions include: identification and measurement of patient needs and expectations; patient involvement; creation of partnerships; multidisciplinary teams/quality committees; use of scientific methods; process management; fact-based management; coaching and mentoring; staff involvement; leadership involvement; development and implementation of the quality plan; and quality program evaluation. Using explicit criteria, these components served as the basis to judge the level of implementation of the HEALTHQUAL program.</div></div><div><h3>Results</h3><div>The degree of implementation of the HEALTHQUAL program varies across health facilities: Some demonstrate a low level of implementation, while others have achieved a medium level. No health facility reached a high degree of implementation.</div></div><div><h3>Conclusion</h3><div>It is important to consider these findings when planning and implementing the HEALTHQUAL program. Identifying and addressing strategies to improve the degree of implementation, particularly by targeting the underlying factors that influence it, is essential. Further research is needed to better understand and clearly define both the degree of implementation and the factors that influence it.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102699"},"PeriodicalIF":2.0,"publicationDate":"2025-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144922477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Youth Alcohol, Tobacco, and Other Drug (ATOD) use rates in a municipal youth and family master plan","authors":"David Tataw","doi":"10.1016/j.evalprogplan.2025.102693","DOIUrl":"10.1016/j.evalprogplan.2025.102693","url":null,"abstract":"<div><h3>Background and objective</h3><div>The purpose of this study was to determine if community-wide activities in the Pomona Youth and Family Master Plan (PYFMP) might have influenced youth ATOD use changes in Pomona and in comparison to the US national jurisdiction.</div></div><div><h3>Methods</h3><div>An integrated framework encompassing social cognitive theory and the risk and protective factors approach guided the study using a pre-post quasi-experimental design and z-test analysis. Same-group and independent group comparisons within and across school grades, intervention years, and jurisdictions were conducted in a quasi-experimental design. The baseline (2005–06) and follow-up (2009–10) samples were as follows: Pomona sample was 3967(2005–06) and 2693(2009–10); and for the US, the samples were 122,243(2005–06) and 101,141(2009–10). Contemporary and context-sensitive evaluation paradigms shape the study's design, plan of analysis, and results interpretation.</div></div><div><h3>Results</h3><div>Within the five years under study, overall youth ATOD use trends in Pomona, particularly in comparison with US trends, showed a slow, but clear and positive shift.</div></div><div><h3>Discussion and conclusion</h3><div>Though PYFMP interventions might have positively impacted ATOD use among Pomona Youth, the pace of change was slow. Future studies should examine the following factors not investigated in this study: specific community-level factors that might have influenced individual youth ATOD use; the effect of underlying socioeconomic and resource factors on youth ATOD use; and the role of micro-level factors and interventions. Some evaluation lessons learned include involving evaluators in planning, being flexible with plans of analysis, and considering context when interpreting results.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102693"},"PeriodicalIF":2.0,"publicationDate":"2025-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144997784","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the hierarchical framework of the Kirkpatrick model in training evaluation from a developing country","authors":"Md. Faisal-E-Alam ,&nbsp;Abu Reza Md. Towfiqul Islam","doi":"10.1016/j.evalprogplan.2025.102696","DOIUrl":"10.1016/j.evalprogplan.2025.102696","url":null,"abstract":"<div><div>Training evaluation plays a vital role in determining the effectiveness of programs to enhance employee performance and achieve organizational goals. The Kirkpatrick Model, widely used across industries, offers a structured framework for evaluating training outcomes through its four levels: Reaction, Learning, Behavior, and Results. However, there is a significant lack of empirical validation of this model in developing economies, especially in the banking sector of Bangladesh. Furthermore, the existing literature often lacks evidence of causal relationships between the levels of the Kirkpatrick model. This study aims to examine the hierarchical relationships within the Kirkpatrick Model of training evaluation in the context of private commercial banks (PCBs) in Bangladesh. Using a quantitative research design and stratified random sampling, we tested three hypotheses concerning the interconnections between the four levels of the model. Data was collected through a questionnaire assessing trainee reactions, learning outcomes, behavioral changes, and results. The collected data was analyzed using Partial Least Squares (PLS)-based Structural Equation Modeling (SEM). The findings reveal that positive trainee reactions significantly influence learning outcomes, leading to behavioral changes in the workplace. These behavior changes ultimately lead to improved individual and organizational performance. The study highlights the importance of a hierarchical approach in training evaluation and validates the applicability of the Kirkpatrick Model in the banking sector. Overall, the findings provide both theoretical and practical implications, reinforcing the hierarchical relationships in the Kirkpatrick Training Evaluation Model and ensuring training effectiveness.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102696"},"PeriodicalIF":2.0,"publicationDate":"2025-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144911765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A participatory systems approach in community health promotion: Lessons learnt from an overall evaluation of a program to reduce health inequities","authors":"Annemarie Wagemakers ,&nbsp;Samantha Elkhuizen ,&nbsp;Anner Bindels ,&nbsp;Marte Wachter ,&nbsp;Dagmar Niewold ,&nbsp;Marthe Derkzen ,&nbsp;Kristina Thompson","doi":"10.1016/j.evalprogplan.2025.102695","DOIUrl":"10.1016/j.evalprogplan.2025.102695","url":null,"abstract":"<div><div>To address the complexity and multi-level dynamics in health promotion, participatory and systems approaches have recently gained attention. We combined these into a participatory systems approach, which actively involves stakeholders in the planning, implementation, and evaluation of health promotion initiatives. This approach promotes cross-sector collaboration, shared understanding, joint decision-making, empowerment, and acknowledges the dynamic, interconnected factors that influence health over time. However, practical guidance on how to implement and evaluate such complex programs remains limited, particularly on aligning implementation and evaluation. We applied a participatory systems approach to the planning and implementation of an overall evaluation of a program aimed at enhancing the health of families in vulnerable positions in the Netherlands, with the goal of sharing lessons learnt from this process. From the program, four projects in four municipalities were selected for this study. We studied the processes and results of the overall evaluation in four iterative steps: conducting participatory action research with the four projects, semi-structured interviews with the four case managers, three consortium reflection and learning sessions and document analysis. We identified five themes: flexibility; research and practice needs; research capacity; roles of stakeholders and defining boundaries. The main lessons from our participatory systems approach are: flexibility is needed to realise stakeholder participation; balance research with practice; ensure adequate research capacity; define the roles of stakeholders and explore system boundaries continuously. The implementation of these lessons is supported by a reflection and learning culture. These lessons are relevant for practitioners, researchers, policymakers and citizens in effectively planning, implementing and evaluating health promotion programs.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102695"},"PeriodicalIF":2.0,"publicationDate":"2025-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144916867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Predictive modeling and cohort data analytics for student success and retention","authors":"Shabnam Sodagari","doi":"10.1016/j.evalprogplan.2025.102689","DOIUrl":"10.1016/j.evalprogplan.2025.102689","url":null,"abstract":"<div><div>This study presents a data-driven analysis of academic performance, demographic disparities, and predictive modeling among more than 23,000 first-time freshmen at a US public University. We examine multiple factors influencing student outcomes, including GPA, credit accumulation, unit workload, Pell Grant eligibility, minority status, and parent education levels. Our analysis reveals several statistically significant disparities: non-minority students earn more units than minority students in their first two years, and Pell-eligible students accumulate fewer credits than their non-eligible peers. First-generation college students also exhibit lower credit accumulation compared to peers. GPA distributions show that minority students have a lower average GPA compared to non-minority students, with broader variation. Clustering analysis identifies three distinct academic engagement profiles based on GPA and unit load, highlighting heterogeneous performance patterns and the need for differentiated support. We develop and tune predictive models to forecast sophomore credit accumulation and GPA, achieving strong performance using deep learning. These models enable proactive risk identification and support strategic interventions. Our findings set the stage for actionable insights for institutional decision-makers aiming to enhance student retention, success, and academic momentum.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102689"},"PeriodicalIF":2.0,"publicationDate":"2025-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144922479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Elementary school teachers’ and principals’ views regarding components of optimal formative teacher assessment","authors":"Irit Levy-Feldman ,&nbsp;Barbara Fresko","doi":"10.1016/j.evalprogplan.2025.102697","DOIUrl":"10.1016/j.evalprogplan.2025.102697","url":null,"abstract":"<div><div>Formative teacher assessment models aim at identifying individual strengths and weaknesses as a basis for planning actions to stimulate professional development. School principals and teachers may perceive the process of teacher assessment differently which ultimately may influence its efficacy. In the present study the views of elementary school teachers and principals were compared concerning four major components of an optimal formative teacher assessment scheme: assessment criteria, sources of information, active partners, and actions following assessment. Data were collected by questionnaire from 1166 teachers and 177 principals, who rated the importance of including different aspects of the four components in such a scheme. Overall, teachers and principals held comparable visions that emphasize a comprehensive and varied assessment model encompassing a multiplicity of criteria, several active partners, various sources of information, and the use of teacher assessment for planning individual and staff development. However, consistent differences were found in the strength of their ratings, suggesting that principals tend to be more confident of their views than teachers. Identifying similarities and differences in what principals and teachers expect to be included in optimal formative teacher assessment can inform the design of assessment schemes to effectively support professional development. The findings of this study have broader relevance to formative dialogic assessment practices by emphasizing the importance of agreement between evaluators and those being evaluated regarding the process and purpose of assessment, as well as the need for linkage between assessment results and a concrete plan for their utilization to improve educational processes.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102697"},"PeriodicalIF":2.0,"publicationDate":"2025-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144907448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluability assessment of social protection services for people experiencing homelessness in Brazil","authors":"Wanessa Debôrtoli de Miranda ,&nbsp;Anelise Andrade de Souza ,&nbsp;Luísa da Matta Machado Fernades ,&nbsp;Daniela Corrêa ,&nbsp;Wanderson Costa Bomfim ,&nbsp;Helvécio Miranda Magalhães Júnior ,&nbsp;Alice de Rezende Brandão ,&nbsp;Rômulo Paes-Sousa","doi":"10.1016/j.evalprogplan.2025.102692","DOIUrl":"10.1016/j.evalprogplan.2025.102692","url":null,"abstract":"<div><div>This is an Evaluability Assessment (EA) of the Specialized Care for Vulnerable Populations (SEAS) and Specialized Reference Centers for People Experiencing Homelessness (Centro POP) in Belo Horizonte, Brazil. These are social protection services for people in vulnerable situations who use the streets for housing and/or survival. The Thurston and Ramaliu model was employed with the construction of theoretical and logical models and SWOT matrix, using the analysis of legal frameworks, technical documents, on-site visits, and discussion groups with local managers. Questions were developed to compose questionnaires validated using the Delphi technique. This validation enhanced the evaluative questions, increasing the analytical power of the tools. Significant lessons were identified regarding the object of investigation and the use of the EA. Regarding the object, the findings highlight how challenging it is to offer services tailored to this population, necessitating intensive intra- and intersectoral efforts. Conversely, significant potentials for the operation of these services were elucidated. Regarding the use of the EA, three key lessons stand out: i) EA supports the implementation of participatory evaluation, ii) employing various tools and data collection methods paves the way for applying multiple evaluation approaches, and iii) there are diverse stakeholders interested in the results of an EA. The feasibility of conducting a more extensive systematic evaluation was confirmed, making this EA a pioneering effort for services aimed at the people experiencing homelessness within the field of social assistance.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102692"},"PeriodicalIF":2.0,"publicationDate":"2025-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144916868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}