Krista R. Mehari , Phillip N. Smith , Angie Im , L. Taylor Stevens , Natalie A. Florez , Vishva Bhatt , Nicolette A. Louissaint
{"title":"Mobilization of community-based organizations during Covid-19: Perceptions of barriers and needed supports","authors":"Krista R. Mehari , Phillip N. Smith , Angie Im , L. Taylor Stevens , Natalie A. Florez , Vishva Bhatt , Nicolette A. Louissaint","doi":"10.1016/j.evalprogplan.2025.102665","DOIUrl":"10.1016/j.evalprogplan.2025.102665","url":null,"abstract":"<div><div>Human ecosystem disruptions such as Covid-19 and natural disasters increase existing health disparities among communities that have been historically underserved by healthcare systems, especially communities of color. Community-based organizations (CBOs) are well-placed to mobilize effective responses to local needs, which may mitigate the impact of human ecosystem disruptions on disparities. The purpose of this study was to explore CBO leaders’ perspectives on responding to community needs during human ecosystem disruptions, with the goal of informing entities that plan to support or partner with CBOs to maximize the effectiveness of disaster responding. Interview respondents were 31 adults with roles in human-serving organizations across two cities in disaster-prone regions and with populations that score highly across several indices for social vulnerability. Barriers that CBOs encountered included bureaucracy and procedural obstacles, lack of awareness of the work and resources of other organizations, and reduced operational capacity due to Covid-19. Strategies to support CBOs included increased funding and enhanced networking with other organizations. To mitigate the disparate impacts of human ecosystem disruptions, public and private funders should increase flexibility in funding to rapidly respond to emerging needs, and community agencies and institutions should proactively collaborate to build cross-agency infrastructure for communication, sharing resources, and referrals.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102665"},"PeriodicalIF":2.0,"publicationDate":"2025-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144757486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jessica Antunes de Oliveira , Marco Tulio Aniceto França
{"title":"Evaluating the impact of the Programa Saúde na Escola on adolescent sexual and reproductive health in Brazil: A quasi-experimental analysis using propensity score matching","authors":"Jessica Antunes de Oliveira , Marco Tulio Aniceto França","doi":"10.1016/j.evalprogplan.2025.102661","DOIUrl":"10.1016/j.evalprogplan.2025.102661","url":null,"abstract":"<div><div>Promoting access to knowledge about sexual and reproductive health in schools is important for adolescents' development, serving as a protective factor against risky sexual behaviors. Therefore, the objective of the study is to evaluate the impact of the <em>Programa Saúde na Escola</em> (PSE) on sexual- and reproductive-health indicators, such as condom use and knowledge on the topic, using data from <em>Pesquisa Nacional de Saúde Escolar</em> (PeNSE) from 2019. The study is designed to provide representative results of students regularly enrolled in public and private schools across the entire national territory, using a probabilistic sample of Brazilian students. After adjustments, the final sample totaled 6.663 observations of students from public schools. In a quasi-experimental approach, a propensity score matching model is used, to compare students attending schools that participate in the program (treatment group) with those attending non-participating schools (control group). The results show that students from schools participating in the PSE report receiving more guidance on preventing pregnancy, sexually transmitted infection, and how to obtain free condoms. Furthermore, students at schools participating in the program start their sexual life slightly earlier. Significant results are found for the total sample and for girls. The variables related to effective condom use are not significant in any specification. These findings suggest that although access to information has increased, no changes in behavior are perceived, which opens space to consider improvements in the implementation of the program, such as strengthening partnerships with basic health units. A sensitivity analysis using the Rosenbaum limits test indicates no bias in the sample.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102661"},"PeriodicalIF":1.5,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144714182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A university and community partnership that built capacity through program evaluation","authors":"Candace Lind , Beth Archer-Kuhn , Natalie Beltrano , Lisa Garrisen , Janet Hettler , Sandra M. Reilly , Leianne Vye-Rogers , Justine Reyes","doi":"10.1016/j.evalprogplan.2025.102669","DOIUrl":"10.1016/j.evalprogplan.2025.102669","url":null,"abstract":"<div><div>A community organization-university partnership was formed to complete program evaluation research to enhance the effectiveness of the organization in its efforts to improve the resiliency of families in crisis or in need of respite. Incorporating the voices of staff and former families who had the critical expertise and lived experiences with programs and evaluative tools undergirded this work. Six themes arose from focus groups and interview data that provided recommendations for leadership on the evaluative process and tools used and informed the literature review. The theoretical approach to this research highlighted program inequities and illuminated the need for more culturally safe program evaluation practices that respect diversity and inclusivity and focus on equity-building designs. Although time-consuming, front-line staff became familiar with their own program logic models, understood how they connected to their day-to-day work with children and families, and developed a sense of ownership through hands-on involvement. Importantly, logic model development was demystified. Recognizing the organization needed a less intimidating visual representation of their logic models, a one-page version was developed for each program along with fuller versions. A repository of measures developed for staff will ensure ongoing access to evidence-informed tools for updating the evaluative framework for their programs.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102669"},"PeriodicalIF":2.0,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144738120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Can programme budget preparation mechanisms improve the allocation of public resources? Examples from ministries in Cameroon","authors":"Onana Simon Pierre, Ousmane Diaby","doi":"10.1016/j.evalprogplan.2025.102668","DOIUrl":"10.1016/j.evalprogplan.2025.102668","url":null,"abstract":"<div><div>This article examines the impact of programme budget preparation mechanisms on the allocation of public resources in Cameroon. To achieve this, we first conducted an exploratory study of eight officials involved in preparing the state budget. This exploratory study enabled us to develop the variables that would lead to an analytical model. A survey was carried out with 475 actors in the budget design process in central services of 29 ministries in Cameroon. Findings from simple probit regressions indicate that programme budget preparation mechanisms improve the allocation of budgetary resources in Cameroon. According to estimates, multi-annuality, expenditure reviews, performance measurement indicators and the structuring of appropriations into programmes, actions, activities and tasks have a positive impact on the allocation of budget revenue. However, this positive impact is mitigated by the Parliament's limited role in budgetary authorisation. The study recommends that external budgetary controls should be stepped up.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102668"},"PeriodicalIF":1.5,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144702614","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Capturing the most significant change after a work-training program for community health workers","authors":"Sylvia Hoens , Suzannah D’Hooghe , Octavia Kint , Bert Quintiens , Dorien Brosens , Liesbeth De Donder , An-Sofie Smetcoren","doi":"10.1016/j.evalprogplan.2025.102667","DOIUrl":"10.1016/j.evalprogplan.2025.102667","url":null,"abstract":"<div><div>This paper examines the changes community health workers experienced after following a nine-month work-training program and the enabling/disabling factors leading to these changes. Using the Most Significant Change technique, stories from community health workers were collected through individual interviews (N = 9), after which other community health works discussed these stories in a focus group (N = 7). Results showed that Most Significant Changes were: increased knowledge, social skills and attitudes, career discovery and facilitated employability, expanded social networks and support systems, and gained social recognition. Mentoring, peer learning and peer support, and experience-based learning were indispensable in realizing those changes. Participants selected stories as significant when they were recognizable, inspirable, or admirable. Finally, our experience suggests that the Most Significant Change technique mostly identifies positive changes, which has the potential to move programs forward. In future evaluation research using the Most Significant Change, we suggest not only a horizontal (with peers) but also vertical story selection process (with coordinators) to have a deeper understanding of different perspectives and their reasoning behind selecting stories.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102667"},"PeriodicalIF":2.0,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144721940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Melissa Gonzales , Steven P. Verney , Jeremiah D. Simmons , Lisa Cacari Stone , Maia Scarpetta , Ryan Martin , Nina Wallerstein
{"title":"Participatory evaluation of a transdisciplinary behavioral health research center","authors":"Melissa Gonzales , Steven P. Verney , Jeremiah D. Simmons , Lisa Cacari Stone , Maia Scarpetta , Ryan Martin , Nina Wallerstein","doi":"10.1016/j.evalprogplan.2025.102664","DOIUrl":"10.1016/j.evalprogplan.2025.102664","url":null,"abstract":"<div><div>Health disparities interventions require the integration of culturally-aligned theories, concepts and methods from across multiple disciplines and socio-ecologic domains to create new models for addressing health inequities faced by vulnerable populations. The Transdisciplinary Research, Equity and Engagement Center (TREE) at the University of New Mexico expands community-based participatory research and translational team science principles to improve behavioral health equity among American Indian, Hispanic, rural, and other intersectional community partners. Consonant with this participatory emphasis, this case study summarizes the development and refinement of the Center‘s evaluation of its shared conceptual framework, based on understanding historic colonization as well as cultural resiliencies, for addressing behavioral health. We elaborate on the co-development of a participatory evaluation process, with transformational goals, including iterative dialogues with leadership, visioning retreats, adaptations to our logic model and refinement of survey instruments. Preliminary results are presented of our internal Center-wide evaluation survey on progress towards aims and new sub-aims as well as highlights of lessons learned on processes, innovative metrics, and equity implications for the Center and transdisciplinary research centers nationwide.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102664"},"PeriodicalIF":2.0,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144827905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Relationship between geographical factors and digital skills development among school administrators: A case study of Pathum Thani Province, Thailand","authors":"Pimolpun Phetsombat , Nawhath Thanwiset Thanvisitthpon","doi":"10.1016/j.evalprogplan.2025.102663","DOIUrl":"10.1016/j.evalprogplan.2025.102663","url":null,"abstract":"<div><div>This study investigates how geographical factors influence the digital skills of educational administrators in Thailand’s Pathum Thani province. A digital skills assessment framework, comprising three components: competency, knowledge, and experience, was developed and validated using confirmatory factor analysis. Data from 346 educational staff and 219 school teachers across seven districts were analyzed. The findings revealed significant district-level variation in digital skills, influenced by factors such as student population, student density, and budget support. The assessment framework demonstrated strong reliability and validity, with all model fit indices meeting accepted thresholds. This research contributes by providing a validated tool to assess digital proficiency in educational leadership and by identifying key geographical factors that shape digital skills development. It underscores the need for targeted digital training programs and equitable resource allocation to enhance digital proficiency among school administrators. These insights can inform national policies and guide the design of district-level digital literacy initiatives.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102663"},"PeriodicalIF":1.5,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144702615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jia Wang , Rolf Straubhaar , Seth Leon , David Kikoler , Elaine Rosales
{"title":"A quasi-experimental approach to evaluating magnet schools","authors":"Jia Wang , Rolf Straubhaar , Seth Leon , David Kikoler , Elaine Rosales","doi":"10.1016/j.evalprogplan.2025.102666","DOIUrl":"10.1016/j.evalprogplan.2025.102666","url":null,"abstract":"<div><div>The purpose of this article is to document an innovative and effective quasi-experimental, mixed methods approach developed for evaluating the effectiveness of magnet schools throughout the United States. Magnet schools continue to be one of the most common school choice mechanisms in the United States, and as a research team the authors have been evaluating the effectiveness of magnet schools in improving student achievement since 2010. In this paper, the authors share the evaluation research design that they have used through this work, with the intent of sharing best practices in effective magnet school evaluation with other scholars and practitioners engaged in work in similar school settings. Specifically, the authors here explain the benefits of various elements of their evaluation design: their selection of comparison schools, of comparison students within those schools, and their means of measuring magnet program implementation to evaluate the degree to which implementation affected student outcomes.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102666"},"PeriodicalIF":1.5,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144711104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating the English for General Purposes (EGP) program at a Taiwanese University: A CIPP (context, input, process, and product) model study","authors":"Shu-Hsiu Huang","doi":"10.1016/j.evalprogplan.2025.102662","DOIUrl":"10.1016/j.evalprogplan.2025.102662","url":null,"abstract":"<div><div>English has been the only compulsory foreign language taught in colleges in Taiwan since 2005. It was referred to as 'English for General Purposes (EGP)' and was taught not only for students' further major-related studies, but also for future job competitiveness. With the implementation of 'The Program on Bilingual Education for Students in College (BEST)' in the second half of 2021, the importance of English has been further strengthened, which has intensified the missions carried by EGP. Consequently, the subject needed to be examined for potential ingrained problems and ways to improve it. The present study collected and analyzed the perspectives of 470 non-English major students at a university in Taiwan on the EGP curriculum, using four factors (context, input, process, and product, or CIPP) as well as gender, grade level, and language proficiency as variables. The results showed that students generally reacted positively to the current EGP curriculum, especially with respect to the process factor. Gender had a significant effect on perceptions, but there were no differences in perceptions based on students' English proficiency levels and grade levels. It is suggested that EGP curriculum designers should consider these differences in their future development of EGP courses.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102662"},"PeriodicalIF":2.0,"publicationDate":"2025-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144748594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Adithy Adithy , Emma Emily de Wit , Gayitri Bhatt , J.G.F. Bunders-Aelen
{"title":"Evaluation of an online study group on trauma and dissociation for psychotherapists in India","authors":"Adithy Adithy , Emma Emily de Wit , Gayitri Bhatt , J.G.F. Bunders-Aelen","doi":"10.1016/j.evalprogplan.2025.102659","DOIUrl":"10.1016/j.evalprogplan.2025.102659","url":null,"abstract":"<div><h3>Background</h3><div>Mental health professionals (MHP<span><span><sup>1</sup></span></span>s) in India are generally not adequately educated in the treatment of trauma. Educational programs can help to disseminate knowledge and skills about effective treatment for trauma, to ensure mental health care for clients. This study aimed to evaluate the usefulness of an educational program on trauma and dissociation (T&D<span><span><sup>2</sup></span></span>) for psychotherapists in India.</div></div><div><h3>Methods</h3><div>An online study group for psychotherapists was led by a psychotherapist, experienced in trauma. The program included didactic presentations, discussions and case presentations, with topics including definitions and types of T&D, assessments, best practices, models and therapeutic methods. Three questionnaires, 14 semi-structured interviews, and a follow-up FGD, were used to evaluate the program.</div></div><div><h3>Findings</h3><div>The online study group was found to be beneficial for psychotherapists to learn about T&D. Evaluation using participants’ reports revealed that they learned to understand clients’ symptoms and conceptualize treatment with a lens of T&D, and they became more confident in working with clients. These reports also indicated that the clients were benefiting.</div></div><div><h3>Conclusions</h3><div>An online study group for psychotherapists, which also formed a supportive learning community, was effective for learning about T&D for participants from varied backgrounds. Such programs could bridge the knowledge gap about treatment for T&D among MHPs, especially in low- and middle-income countries (LMICs).</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"112 ","pages":"Article 102659"},"PeriodicalIF":1.5,"publicationDate":"2025-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144714183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}