{"title":"The criticality of language: Exploring STEM education evaluators conceptualizations of equity, diversity, and inclusion and the influence on their roles and practice","authors":"Cherie M. Avent","doi":"10.1016/j.evalprogplan.2024.102511","DOIUrl":"10.1016/j.evalprogplan.2024.102511","url":null,"abstract":"<div><div>Language is critical in evaluation. It influences understanding of goal attainment, judgments made about the quality of a program, interactions between stakeholder groups, interpretation of findings, and future actions. This paper reports research on language that unearths complexities related to the agenda of diversifying STEM and the utilization of evaluation in achieving its goal through an examination of STEM education evaluators’ conceptualizations of diversity, equity, and inclusion and the implications for their practice. Data was collected via semi-structured interviews. Findings revealed (1) the multifaceted nature of terms such as equity, diversity, and inclusion, (2) the role of an evaluator is not singular, and (3) the importance of surfacing the ways in which language is both value-laden and socially shaping for potentially disrupting one’s knowledge of status quo injustices.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102511"},"PeriodicalIF":1.5,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142394298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"“What is our actual impact?”: A mixed-method assessment of a Belgian shelter for homeless men","authors":"Coralie Helleputte , Marthe Nyssens , Anaïs Périlleux","doi":"10.1016/j.evalprogplan.2024.102508","DOIUrl":"10.1016/j.evalprogplan.2024.102508","url":null,"abstract":"<div><div>This paper presents an outcomes assessment carried out in an 84-bed long-term shelter for homeless men in Brussels. In the context of increasing Housing First studies, it investigates an instance of the traditional “treatment first” model and provides a new Belgian case study. Adopting a multidimensional approach, it aims to better understand what impact an average stay in a long-term shelter has on its residents. The assessment investigated several outcomes—income and housing, physical and mental health, life skills, social and assistance network—and relied on a participative mixed-method design. Although the shelter mission is broad and ambitious (i.e. autonomy, global well-being and reintegration into society), the assessment results show that the shelter struggles to have positive effects on the residents beyond the provision of basic care (a roof, food, administrative support) and that the stress felt by the residents even tends to increase during their stay. Several recommendations collectively emerged from the assessment: individualizing shelter support and making it evolve during the stay, reducing the size of the shelter while at the same time fostering community living, developing partnerships. At the public policy level, we would recommend revising the mission of long-term shelters in accordance with their means.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102508"},"PeriodicalIF":1.5,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142420394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vinícius Jaques Gerhardt , Julio Cezar Mairesse Siluk , Ismael Cristofer Baierle , João Francisco Pollo Gaspary , Janyel Trevisol , Claudia de Freitas Michelin , Norberto Arend Aviles
{"title":"A theoretical framework to companies value creation through a systematic review of intangibles’ management","authors":"Vinícius Jaques Gerhardt , Julio Cezar Mairesse Siluk , Ismael Cristofer Baierle , João Francisco Pollo Gaspary , Janyel Trevisol , Claudia de Freitas Michelin , Norberto Arend Aviles","doi":"10.1016/j.evalprogplan.2024.102506","DOIUrl":"10.1016/j.evalprogplan.2024.102506","url":null,"abstract":"<div><div>The purpose of this article is to develop a theoretical framework that identifies the intangible aspects that can be managed and contribute to the value creation for organizations. The theoretical framework was developed based on a systematic review performed according to a protocol that proposes steps to identify the intangible aspects present in the scientific literature. Mendeley software assisted in organizing and reading the 3152 articles identified by the systematic review. The results of the article propose a classification of intangible aspects identified in levels that collaborate with the value creation in organizations. The first and main group being called FPVs, encompassing: reputation, innovation, performance, legitimacy, and knowledge. Strategically, the FPVs are subdivided into 15 CSFs, that cover the other 35 intangibles perceived in the studies, called Indicators and that can be managed. The results of the article provide theoretical and managerial implications and can be used by the academic community and by managers of industrial organizations. The results present reflections on how intangible aspects are present in research and the management of organizations. Furthermore, the literature review proves the importance of analyzing and monitoring intangible aspects nowadays.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102506"},"PeriodicalIF":1.5,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142394297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tammy Rinehart Kochel , Matthew P. West , Marlon Mike Toro-Alvarez
{"title":"Modeling a needs assessment approach for policymakers to investigate, understand, and reduce gun violence","authors":"Tammy Rinehart Kochel , Matthew P. West , Marlon Mike Toro-Alvarez","doi":"10.1016/j.evalprogplan.2024.102505","DOIUrl":"10.1016/j.evalprogplan.2024.102505","url":null,"abstract":"<div><div>We conducted a needs assessment evaluation as part of a community effort to plan tailored responses to reduce gun violence in a midwestern town. Various data sources were used to analyze factors leading to gun violence. The results guided the creation of evidence-based response strategies. Study findings demonstrate that even in a small town, the nature and drivers of gun violence vary considerably at the micro level and thus should be examined at this level. In this paper, we present the recommendations that followed from our study, not as solutions for other jurisdictions to adopt on their face, but rather as an example of 1) the types of solutions that follow from a thorough analysis of a local problem, and 2) a plethora of potential responses by social institutions. The goal of the paper is to provide an example process for policymakers from jurisdictions of various sizes and locales of the application of a problem-oriented approach to understanding and preventing gun violence. Undertaking a comprehensive, micro-focus to analyze the gun violence problem and its drivers in a jurisdiction provides essential information to guide the formulation of tailored, evidence-based responses.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102505"},"PeriodicalIF":1.5,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142337015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Syka Iqbal, Macarena Chepo, Marc Hébert, Cecilia Vindrola-Padros
{"title":"To scope or not to scope? The benefits and challenges of integrating scoping studies in rapid qualitative research and evaluation","authors":"Syka Iqbal, Macarena Chepo, Marc Hébert, Cecilia Vindrola-Padros","doi":"10.1016/j.evalprogplan.2024.102495","DOIUrl":"10.1016/j.evalprogplan.2024.102495","url":null,"abstract":"<div><p>Creating sustainable change and fostering collaborative relationships between researchers and stakeholders is a recognized challenge in the field of evaluation. Identifying programme purpose, cultural context, potential challenges, and engaging stakeholders before an evaluation can produce responsive and impactful evaluations. This paper discusses implementing a targeted scoping study within the framework of rapid qualitative research and evaluation. A scoping study enables collaborative decision-making on evaluation priorities, and functions as an evaluability assessment in time-sensitive contexts. In our experience, a scoping study can be carried out in as little as five days or as long as six weeks. It is timely to revisit the question of what factors influence evaluation outcomes, a scoping study can be used to support an evaluation, address access to data and research, and strengthen communication channels. The methodological approach was used to co-produce an evaluation with an NGO that accurately reflected their needs, recognizing possible challenges and solutions.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102495"},"PeriodicalIF":1.5,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000971/pdfft?md5=e7b517fda6e66e5253068b449d13b057&pid=1-s2.0-S0149718924000971-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142172993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Arts-based evaluation of the Communities ChooseWell program","authors":"Karyn Hurlbut, Astrid Brousselle","doi":"10.1016/j.evalprogplan.2024.102496","DOIUrl":"10.1016/j.evalprogplan.2024.102496","url":null,"abstract":"<div><p>Arts-based evaluation is an effective and fun way to engage people and uncover meaningful, valid results. In this project, an arts-based approach was used to gain an understanding of the effects the Communities ChooseWell Program has had according to the Champions’ experiences. We wanted to identify what changes, if any, has Communities ChooseWell fostered through the past 10 years? This evaluation was completed using an arts-based approach which allowed us to explore varied long-term effects in different contexts. The creative process allowed for an open approach not predetermining the nature of potential effects. It also gave the participants space to identify what matters the most according to the ChooseWell Champions. This evaluation was in addition to evaluation requirements from the ChooseWell programs funder. In this article we will first present the context of evaluation and state our positionality. We will then present the methodology and methods. Finally, we will present and discuss the results and recommendations.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102496"},"PeriodicalIF":1.5,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000983/pdfft?md5=6c1c147bb27c304dd34c36a307a75f54&pid=1-s2.0-S0149718924000983-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142172994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning from educators: Implementation of a positive behavior support program targeting challenging behavior in children with autism","authors":"Zakaria Mestari , Mélina Rivard , Catherine Mello","doi":"10.1016/j.evalprogplan.2024.102491","DOIUrl":"10.1016/j.evalprogplan.2024.102491","url":null,"abstract":"<div><p>Challenging behaviors (CB) are a frequent co-occurring problem in children with autism spectrum disorder (ASD) and hinder their response to recommended interventions such as early intensive behavioral intervention (EIBI). The Prevent-Teach-Reinforce for young children (PTR-YC) program was implemented to meet community-based EIBI educators’ training and support needs in managing CB in their day-to-day work with families. Although this positive behavior support program has a strong empirical basis, its implementation by community-based educators has yet to be assessed from a systematic and structured program evaluation perspective. Using Chen’s (2015) theoretical framework for program evaluation, this study assessed the quality of implementation of PTR-YC as perceived by 17 educators who received training and supervision on applying PTR-YC among families of children with ASD receiving EIBI services. Educators’ post-intervention interviews and questionnaires were analyzed using the logical model for program evaluation to identify obstacles and facilitators to the implementation of PTR-YC.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102491"},"PeriodicalIF":1.5,"publicationDate":"2024-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000934/pdfft?md5=14c771ae742f7ab8c62e79a103ec3e00&pid=1-s2.0-S0149718924000934-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142146594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Training effectiveness evaluation: Advancing a Kirkpatrick model based composite framework","authors":"Sudipta Paul , Rajarshi Roy Burman , Rahul Singh","doi":"10.1016/j.evalprogplan.2024.102494","DOIUrl":"10.1016/j.evalprogplan.2024.102494","url":null,"abstract":"<div><p>The present study aims at ideating a quantitative protocol to evaluate effectiveness of training programmes especially meant for farmers, farm women, and rural youth. The specific objective was to develop a robust framework for measuring the effectiveness of rural training programmes organized by the Farmers Training Centres (FTCs) of India. Kirkpatrick’s training evaluation model provides us the foundation to design a four-dimensional composite framework based on range-based indicator normalization, principal component analysis based indicator weight estimation, and rank correlation based framework sensitivity testing. We used cross-sectional primary data generated through household survey and personal interviews with randomly selected one thousand trainees to test and validate our proposed protocol. Applying it on our evaluation target we find that the degree of effectiveness of the training programmes varies; one in every four training programmes may not be effective. Trainees’ reactions on various aspects of the training programmes may have positive and significant influence on learning. Training outcomes may be linked with the trainees’ post-training changes in behaviour. A sensitivity test confirms that the proposed framework is not susceptible to changes in weighting schemes, implying robustness of indicator selection. The findings offer dissection of individual training programmes guiding policy decisions for a training organization. The proposed framework enriches the Kirkpatrick’s training evaluation model by offering standardized indicators for training effectiveness evaluation.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102494"},"PeriodicalIF":1.5,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142128773","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Faruk Yilmaz , Anı Hande Mete , Buse Fidan Turkon , Canser Boz
{"title":"How enabling factors determine unmet healthcare needs? A panel data approach for countries","authors":"Faruk Yilmaz , Anı Hande Mete , Buse Fidan Turkon , Canser Boz","doi":"10.1016/j.evalprogplan.2024.102492","DOIUrl":"10.1016/j.evalprogplan.2024.102492","url":null,"abstract":"<div><p>Health service need refers to the essential care required to achieve optimal health outcomes within resource constraints. When necessary services to address identified health issues are not received, unmet needs arise. This research focuses on the determinants of unmet healthcare needs across the 34 countries within the European region from 2011 to 2019, focusing on Andersen's Behavioral Model's enabling factors. We employed a static and robust panel regression model using Stata 14.0 software. Key determinants analyzed include GDP per capita, urbanization rate, and physicians per capita. Findings reveal that lower GDP per capita and lower urbanization rates are significantly correlated with higher levels of unmet healthcare needs, highlighting income level and geographical accessibility as critical factors. Additionally, a higher number of physicians per capita is associated with reduced unmet healthcare needs, indicating the importance of healthcare resources in addressing healthcare access gaps. These findings underscore the importance of targeted healthcare policies that address income level, improve healthcare accessibility, and enhance healthcare resource allocation to reduce unmet healthcare needs effectively. These findings equip policymakers and administrators with empirically grounded insights to comprehend the factors contributing to unmet healthcare needs and to develop policies aimed at addressing this challenge.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102492"},"PeriodicalIF":1.5,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142128772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluation of the impact of Korea’s employment protection legislation on vocational training discrimination against non-regular workers","authors":"Ko Hyejin","doi":"10.1016/j.evalprogplan.2024.102493","DOIUrl":"10.1016/j.evalprogplan.2024.102493","url":null,"abstract":"<div><p>This longitudinal study evaluated the effectiveness of Korea’s 2007 employment protection legislation (EPL), aimed at mitigating labor discrimination against non-regular workers (NRW). Specifically, we use the Korean Workplace Panel Survey data from 2005 to 2013 and adopt a difference-in-difference methodology (DD) to explore whether establishment reduces the gap in training opportunities between NRW and regular workers (RW). Results show that compliance with government regulations depends on how business establishments react to the rules and the characteristics of these stakeholders. Overall, the 2007 reform partially improved the training opportunities in NRW. However, this positive outcome was mostly concentrated in companies with affluent resources or stronger bargaining power of NRW. Policymakers should consider in what context the policy works well and what factors may hinder its operation at the policy design stage.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102493"},"PeriodicalIF":1.5,"publicationDate":"2024-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142128774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}