Evaluation and Program Planning最新文献

{"title":"Preparing resilient educators: A narrative review on CBT-based workshops for pre-service teachers","authors":"Carmit Gal","doi":"10.1016/j.evalprogplan.2025.102721","DOIUrl":"10.1016/j.evalprogplan.2025.102721","url":null,"abstract":"<div><div>Teacher stress and early career attrition represent global challenges in education, yet significant gaps exist in understanding how Cognitive Behavioral Therapy (CBT)-based interventions can build resilience in pre-service teachers. While mindfulness and social-emotional learning programs have received attention, few studies have systematically examined CBT workshops for teacher preparation or explored their implications for advancing evaluation methodology in educational contexts. This narrative review addresses these gaps by synthesizing evidence on CBT-based workshops for pre-service teachers while examining contributions to evaluation scholarship. Across diverse cultural and institutional contexts, CBT interventions consistently reduced stress, improved emotional regulation, and enhanced self-efficacy, with immediate benefits and variable long-term outcomes. The mechanistic clarity of CBT's cognitive restructuring approach enables precise measurement of causal pathways, making these interventions particularly suitable for theory-driven evaluation frameworks. The review reveals three critical contributions to evaluation methodology: validation of theory-driven approaches in complex educational settings, demonstration that cultural adaptation enhances program effectiveness without compromising evaluation rigor, and illumination of temporal complexity in preventive interventions. These findings provide lessons for the international evaluation community about implementing and assessing preventive interventions across diverse contexts. This study contributes a novel perspective demonstrating how CBT-based interventions can inform both teacher education program design and advancement of evaluation theory and practice.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102721"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145394055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Barriers and facilitators to recovery support interventions in US Rural Recovery Houses: Implementing the SMART life skills program","authors":"Madison Ashworth,&nbsp;David Johnson,&nbsp;Robin A. Thompson,&nbsp;Grace L. Clancy,&nbsp;Jennah Baier,&nbsp;Jean Spielman","doi":"10.1016/j.evalprogplan.2025.102717","DOIUrl":"10.1016/j.evalprogplan.2025.102717","url":null,"abstract":"<div><div>As substance use disorder continues to impact millions of Americans, various recovery support treatment and service modalities are being developed, evaluated, and refined for effectiveness. Recovery housing provides substance-free living environments for individuals in recovery and can offer a variety of services and structured programs for residents. One such program that has the potential to enhance resident outcomes is the Self-Management and Recovery Training (SMART) Successful Life Skills program (SLS), a 12-session curriculum that uses a mix of behavioral tools and aspects of the social model of recovery employed by standard mutual aid groups. In this paper, we discuss the implementation barriers and facilitators of a 3-year initiative that aimed to implement SLS in 100 rural recovery houses across the U.S. During this initiative, the study team faced significant barriers in recruitment and program implementation. Recruitment barriers included the unknown landscape of recovery housing, stigma and mistrust in research, inconsistent staffing models, and differing treatment philosophies. Further, access to technology, technical literacy, time constraints for both residents and recovery house staff, and staff turnover created a difficult environment to successfully implement and evaluate the impact of the SLS program. Numerous solutions were employed to overcome implementation and recruitment barriers in recovery housing such as multiple contact methods, collaboration, stakeholder engagement, financial/material support, evaluation flexibility and adaptability, and transparency.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102717"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145259587","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Systems exploration workshops: Using a virtual group model building approach to advance public health systems change","authors":"Jessica Simon ,&nbsp;Isabella Guynn ,&nbsp;Chantelle Dowsett ,&nbsp;Kristen Hassmiller Lich","doi":"10.1016/j.evalprogplan.2025.102698","DOIUrl":"10.1016/j.evalprogplan.2025.102698","url":null,"abstract":"<div><div>Creating well-functioning public health systems is complex and requires collaboration across often fragmented parts of the system. Group Model Building (GMB) is an evidence-based method that engages diverse partners in understanding complexity and identifying opportunities for systems change. Through a structured sequence of scripted activities, GMB supports the development of shared insights into the dynamics that shape outcomes in complex, change-resistant systems. The National Maternal and Child Health (MCH) Workforce Development Center supports state and jurisdictional MCH professionals through training, coaching, collaborative learning, and consultation. As part of a year-long cohort experience, the MCH Workforce Development Center facilitates a virtual two-day Systems Exploration Workshop to implement GMB activities with teams led by state MCH agencies to explore systemic challenges ranging from improving care coordination to advancing health equity. This paper presents a structured walkthrough of the virtual GMB Systems Exploration Workshop to encourage replication and adaptation in other settings. Evaluation results from 74 participants across 11 states indicate that the workshop fosters a shared understanding of focal systems. Teams reported an enhanced ability to collectively analyze their primary challenges, develop shared mental models of their key determinants, and prioritize impactful interventions. This virtual GMB Systems Exploration Workshop is a valuable approach to accelerate collaborative decision-making through a structured process that facilitates inclusion of diverse perspectives. This workshop offers a practical, systems-informed approach for public health teams seeking to build collective insight, strengthen early-stage strategic planning, and align evaluation efforts with the complexity of the systems in which they are undertaken.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102698"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145020943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A formal model for assessing the learning outcomes of academic programs","authors":"Manuel F. Caro ,&nbsp;Elviara P. Flórez ,&nbsp;Isabel C. Muñoz","doi":"10.1016/j.evalprogplan.2025.102644","DOIUrl":"10.1016/j.evalprogplan.2025.102644","url":null,"abstract":"<div><div>Many academic programs face persistent challenges in evaluating learning outcomes with consistency, traceability, and alignment to curricular goals. This study introduces a formal model for the systematic assessment of learning outcomes across an academic program. The model is grounded in set theory and matrix algebra, integrates the SOLO taxonomy to classify levels of cognitive performance, and structures evaluation across three defined assessment moments: early, mid, and final stages of the academic trajectory.</div><div>The model was applied to a five-year teacher training program in Computer Science and validated through empirical data collection and statistical analysis. A paired t-test confirmed a significant decline in learning outcome performance between the initial and final stages of the program, while ANOVA results indicated consistent performance across courses at each stage. Based on these findings, the model also supports the generation of outcome-specific improvement plans, recalibrating performance expectations iteratively.</div><div>The proposed model contributes to outcome-based education by offering a replicable framework for aligning course-level evidence with program-level competencies. Its integration of structured assessment, cognitive benchmarking, and iterative feedback enables academic institutions to monitor student achievement longitudinally and support continuous curriculum improvement.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102644"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145020944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Framing the shift to supported employment: Exploring the impacts of a person-centred programme evaluation approach through peer led participatory research","authors":"Fahimeh Malekinezhad,&nbsp;Paul Courtney","doi":"10.1016/j.evalprogplan.2025.102723","DOIUrl":"10.1016/j.evalprogplan.2025.102723","url":null,"abstract":"<div><div>Employment support programs play a crucial role in assisting the unemployed in overcoming challenges to achieve their employment goals. The experiences of participants with multiple challenges provide insights into programme effectiveness in achieving personalised goals of employability, social development, and inclusion. This study contributes to the evaluation of supported employment by presenting a peer-derived framework grounded in participants lived experiences. Interviews with twelve participants revealed themes around employment barriers, participation opportunities, one-to-one support, achievements, and recommendations. Findings highlight the value of flexible, personalised pathways that not only enhance employability but also build confidence, motivation and support social inclusion. The developed framework including personalised support, meaningful activities, and self-perceived progress offers guidance for designing inclusive employment programmes and their evaluation. A key recommendation is the need for ongoing support to sustain employment among disadvantaged individuals managing social, mental, and physical health challenges. From an evaluation perspective, the framework demonstrates how participant-informed mechanisms—such as confidence and empowerment—drive employability and social skills, consistent with the Context–Mechanism–Outcome logic of realist evaluation. By situating peer-led evaluation within realist, empowerment, and utilisation-focused traditions, this study refines programme evaluation and strengthens its practical relevance. It shows how outcome-focused fidelity models, which capture structural quality, can be complemented by peer-led approaches capturing experiential quality. Together, these perspectives provide a holistic and transferable evaluation model that speaks to both programme design and lived experience. Beyond the local context, lessons learned highlight the value of inclusive, participatory evaluation methods in generating credible, stakeholder-driven insights and advancing more effective employment support practices globally.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102723"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145394125","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing longitudinal impact evaluation of non-experimental interventions: A comparative analysis of generalized estimating equations and multilevel models based on empirical evaluation of temporal change in food security following an economic empowerment intervention","authors":"Amon Exavery ,&nbsp;Peter Josephat Kirigiti ,&nbsp;Ramkumar T. Balan ,&nbsp;John Charles","doi":"10.1016/j.evalprogplan.2025.102722","DOIUrl":"10.1016/j.evalprogplan.2025.102722","url":null,"abstract":"<div><h3>Background</h3><div>Inadequate impact evaluation methods such as percentage changes in outcomes between assessment points, persist as a significant challenge, undermining assessment accuracy. This study provides empirical evidence to enhance evaluation practices for non-experimental interventions by comparing Generalized Estimating Equations (GEE) and Multilevel Models (MLM), with the WORTH Yetu intervention’s impact on food security as a case in point.</div></div><div><h3>Methods</h3><div>Guided by the Realist Evaluation (RE) framework which focuses on exploring what works, for whom, in what circumstances, and how, this study employed a longitudinal design using a difference-in-differences (DID) estimator to assess the impact of the WORTH Yetu economic empowerment intervention on food security among families caring for orphaned and vulnerable children (OVC) in Tanzania. Both GEE and MLM were applied to compare their performance and determine which approach, within the RE framework, offers a simpler and more practical option for improving impact evaluation practices in non-experimental settings.</div></div><div><h3>Results</h3><div>The analysis included 497,293 observations from 249,655 caregivers (aged ≥18 years) at baseline, 247,638 of whom were reassessed at follow-up. The study found a higher likelihood of enhanced food security among WORTH Yetu participants at follow-up compared to non-participants: aOR = 1.551 (95 % CI 1.509–1.594, <em>p</em> &lt; 0.001) in GEE and aOR = 1.572 (95 % CI 1.528–1.616, <em>p</em> &lt; 0.001), ICC = 28.2 % in MLM. In the computation process, GEE was simpler, estimating one set of parameters, while MLM estimated both fixed- and random-effects, requiring additional considerations and longer runtime.</div></div><div><h3>Conclusion</h3><div>Both GEE and MLM produced comparable results in terms of the magnitude, direction, and statistical significance of the WORTH Yetu economic empowerment intervention’s impact on food security. However, while GEE was simpler and more efficient as it estimated one set of parameters, MLM required additional steps and considerations to estimate both fixed-effects and random-effects. As a result, this study found greater advantages in using GEE over MLM, an option likely to improve longitudinal impact evaluation practices, especially in resource-constrained non-experimental settings, by streamlining the analysis without compromising the statistical rigor.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102722"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145416868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Predictive factors for inter-agency partnership success","authors":"Nicola Drago ,&nbsp;Tullio Vardanega","doi":"10.1016/j.evalprogplan.2025.102719","DOIUrl":"10.1016/j.evalprogplan.2025.102719","url":null,"abstract":"<div><div>Building partnerships among development cooperation agencies strengthens their collective capacity. However, 2 partnerships out of 10 fail to achieve their intended objectives. Ex-ante evaluation of future partnerships should help attain a higher rate of success. To do this, agencies need a better understanding of the factors behind successful or unsuccessful partnerships. This paper draws from the first stage of a research project aimed at systematically identifying and classifying such factors by importance in a semi-automated manner, to streamline partnership planning and evaluation tasks. The principal result we report here is a dataset containing 750 factors of influence, visualized along three axes: operations and country context, governance and management, and project quality, and grouped into ten clusters. Evaluators and planners can readily use the factors and complete method to focus multi-stakeholder discussions and information gathering, as well as to learn to use Natural Language Processing and Machine Learning approaches on project design and evaluation document corpora. Future research can overcome the limitations of the method by standardizing the factor set through ontological and taxonomic work, as well as incorporating automated context and behavioral analyses.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102719"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145356431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Scenario-based planning for urban sustainability: Envisioning an ecological city in Shandiz, Iran by 2032","authors":"Forouzan Taheri ,&nbsp;Mahdi Kolahi","doi":"10.1016/j.evalprogplan.2025.102711","DOIUrl":"10.1016/j.evalprogplan.2025.102711","url":null,"abstract":"<div><div>Urban sustainability in developing countries is often hindered by infrastructural limitations and the complexity of environmental uncertainties. This study applies scenario-based planning to explore future pathways for the development of an ecological city in Shandiz, Razavi Khorasan Province, Iran, by 2032. Following a six-step scenario planning framework, the research employed a mixed-methods approach, including expert questionnaires to identify and prioritize key drivers, and semi-structured interviews using saturation theory to construct scenario logics. The analysis yielded three distinct future scenarios: the Ideal City (sustainable and inclusive development), the Command City (state-driven but fragile planning), and the Ruined City (unsustainable and fragmented growth). Strategies were formulated to guide urban stakeholders toward realizing the most desirable future. The study contributes to the field of urban futures by demonstrating how participatory scenario planning can support informed decision-making under uncertainty and promote sustainable urban transitions.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102711"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145082108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Does mentoring frequency matter? Effects on dropout rates and average grades among university students","authors":"Miguel A. Alonso García ,&nbsp;Aitana González Ortiz de Zárate ,&nbsp;Irene Aliagas ,&nbsp;Ma Ángeles Gómez-Flechoso","doi":"10.1016/j.evalprogplan.2025.102729","DOIUrl":"10.1016/j.evalprogplan.2025.102729","url":null,"abstract":"<div><div>Mentoring programs are essential in supporting first-year university students by reducing dropout rates and enhancing grades. However, the optimal number of meetings has not been extensively explored. This article examines how the number of meetings influences outcomes across disciplines and demographics. Using a quasi-experimental approach with 9063 students (4 + meetings, <em>n</em> = 3074; ≤3 meetings, <em>n</em> = 2958; 0 meetings, <em>n</em> = 3031), who belonged to different areas of study, we analyzed dropout rates and grades through chi-square tests, phi statistics, ANOVA, and Bonferroni adjustments. Results indicate that more meetings generally lead to lower dropout rates and higher grades. However, exceptions exist in Science and Engineering, ≤ 3 meetings were as effective as 4 + meetings at reducing dropouts; while in Arts, Humanities, Social, and Legal Sciences, they were equally effective at improving grades. These findings suggest institutions can optimize mentoring strategies by tailoring programs to specific student needs, improving educational outcomes and fostering success.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102729"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145460315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Impacting the substance abuse epidemic in America: A mixed methods examination of mentors of youth who experience substance use disorders within their families","authors":"Stephen Phillippi ,&nbsp;Ashley Fenton ,&nbsp;Kelsey Witmeier ,&nbsp;Sara Juneau ,&nbsp;Claudia Leonardi ,&nbsp;Edmond P. Bowers ,&nbsp;Brian Maus","doi":"10.1016/j.evalprogplan.2025.102715","DOIUrl":"10.1016/j.evalprogplan.2025.102715","url":null,"abstract":"<div><div>This study aimed to explore the experiences and needs of mentors in Camp Mariposa; a program designed to prevent substance use among youth who have a family member with a substance use disorder (SUD). Using a mixed-methods design, the study included a quantitative survey and focus groups. Of 280 eligible mentors, 116 (41 %) completed the survey, with 24 (21 %) participating in follow-up focus groups. Descriptive statistics illustrate survey data findings, and qualitative findings reflect themes that emerged from focus groups. Mentors reported positive experiences working with peers and youth but also highlighted challenges that could inform future mentoring efforts. Key factors influencing their experiences included the benefits and challenges of mentoring, program leadership, self-care needs, and strategies during camp weekends. Mentors expressed a desire for more training and emphasized the importance of pre-camp briefings and post-camp debriefings. Confidence in mentoring abilities and maintaining empathy and boundaries were also highlighted. This study provides valuable insights for improving mentor support and training in programs targeting youth facing adversities that come with having a family member with a SUD.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"114 ","pages":"Article 102715"},"PeriodicalIF":2.0,"publicationDate":"2026-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145266994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}