Evaluation and Program Planning最新文献

Anticipatory evaluation. How to incorporate an anticipatory technique into a theory-driven evaluation process. Results of application in a case study. 预期评估。如何将预测技术纳入理论驱动的评价过程。案例研究中的应用结果。

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-16 DOI: 10.1016/j.evalprogplan.2024.102509

Mariangel Pacheco-Troisi, Mónica García-Melón, Fernando Jiménez-Sáez

{"title":"Anticipatory evaluation. How to incorporate an anticipatory technique into a theory-driven evaluation process. Results of application in a case study.","authors":"Mariangel Pacheco-Troisi, Mónica García-Melón, Fernando Jiménez-Sáez","doi":"10.1016/j.evalprogplan.2024.102509","DOIUrl":"https://doi.org/10.1016/j.evalprogplan.2024.102509","url":null,"abstract":"<p><p>In recent years, there has been increased focus on strategic learning from impact analysis, including in the field of science, technology, and innovation. In this paper, we propose combining techniques from two fields of study. Firstly, we adopt the approach of addressing impact through productive interactions between science and society, and secondly, we incorporate an anticipatory dimension by integrating game analysis involving key actors. Through a theory-driven evaluation design, we consider expected impacts as promises of the future. Within an anticipatory perspective, the future can be shaped by the interactions among different actors in the present. In this article, we apply this approach step-by-step to a research institute program in Uruguay. We demonstrate how the achieved results offer strategic insights to the program manager for anticipating and attaining the desired impacts. Additionally, we provide summative inputs for accountability using a flexible technique applicable at any stage of the program life cycle. The article concludes with a discussion of the advantages and disadvantages compared to other techniques, along with lessons learned that may benefit other evaluators seeking to replicate this approach. Furthermore, we explore potential extensions and opportunities for further improvement in this research.</p>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142477833","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

What evaluation criteria are used in policy evaluation research: A cross-field literature review 政策评估研究中使用哪些评估标准：跨领域文献综述

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-10 DOI: 10.1016/j.evalprogplan.2024.102512

{"title":"What evaluation criteria are used in policy evaluation research: A cross-field literature review","authors":"","doi":"10.1016/j.evalprogplan.2024.102512","DOIUrl":"10.1016/j.evalprogplan.2024.102512","url":null,"abstract":"<div><div>This literature review offers a comprehensive overview of the use of evaluation criteria across five policy fields: social services, land-use planning, teaching in higher education, vocational education, and the environment. Though it is a key part of the evaluation process, the question of how criteria are defined, chosen, and applied generates surprisingly little debate among the evaluation community. In evaluation practice, criteria are often taken for granted – and occasionally even used in ways that are neither explicit nor transparent. This cross-field literature review shows a strong presence of routinized evaluation criteria (relating to the specifics of each policy field), while some new sets of higher-degree criteria also emerge in the face of social challenges relating to sustainability, public acceptance, or social justice. Criteria development draws on both inductive bottom-up processes (which can include policy stakeholders) and top-down deductive processes (which derive criteria from the literature, as well as from national and international standards). A more profound reflection on evaluation criteria (that is, the dimensions used by societies to assess the success of policy interventions) might be required in the future of evaluation research and planning; a deeper cross-field dialogue could support this endeavor.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142444898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A program evaluation of the new choices workforce development program: An appreciative inquiry approach 新选择劳动力发展计划的项目评估：欣赏式探究方法

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-09 DOI: 10.1016/j.evalprogplan.2024.102507

{"title":"A program evaluation of the new choices workforce development program: An appreciative inquiry approach","authors":"","doi":"10.1016/j.evalprogplan.2024.102507","DOIUrl":"10.1016/j.evalprogplan.2024.102507","url":null,"abstract":"<div><div>A significant amount of money ($1.76B annually in the United States) is spent on workforce development programs, while there is limited research on the effectiveness of workforce development programs in meeting their program objectives and assisting program participants in attaining employment. This study evaluated the New Choices Program, a workforce development program offered by PA Women Work, to help its clients obtain employment and overcome personal and professional barriers. The program has historically been offered in a 30-hour in person format but was forced to be modified to a 10-hour virtual program when the COVID-19 pandemic restrictions began. This program evaluation included a comparative analysis of the perceptions of participants in the 30-hour in person program and the 10-hour virtual program. It was found that participants in both the 30-hour in person program and 10-hour virtual program perceived the program positively, experienced an increase in self-confidence and belonging, which led to either obtaining employment or being better prepared for the job search process. The data will help inform the New Choices program stakeholders on programmatic improvements and how best to structure the program in the post-pandemic employment world.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142438413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Problems (and possible solutions) of assessing risk, race and recidivism in long operating drug treatment courts 长期运作的戒毒法庭在评估风险、种族和累犯方面存在的问题（以及可能的解决方案

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-05 DOI: 10.1016/j.evalprogplan.2024.102510

引用次数: 0

The criticality of language: Exploring STEM education evaluators conceptualizations of equity, diversity, and inclusion and the influence on their roles and practice 语言的关键性：探索 STEM 教育评估员对公平、多样性和包容性的概念化及其对其角色和实践的影响。

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-01 DOI: 10.1016/j.evalprogplan.2024.102511

引用次数: 0

“What is our actual impact?”: A mixed-method assessment of a Belgian shelter for homeless men "我们的实际影响是什么？对比利时无家可归男子收容所的混合方法评估

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-01 DOI: 10.1016/j.evalprogplan.2024.102508

{"title":"“What is our actual impact?”: A mixed-method assessment of a Belgian shelter for homeless men","authors":"","doi":"10.1016/j.evalprogplan.2024.102508","DOIUrl":"10.1016/j.evalprogplan.2024.102508","url":null,"abstract":"<div><div>This paper presents an outcomes assessment carried out in an 84-bed long-term shelter for homeless men in Brussels. In the context of increasing Housing First studies, it investigates an instance of the traditional “treatment first” model and provides a new Belgian case study. Adopting a multidimensional approach, it aims to better understand what impact an average stay in a long-term shelter has on its residents. The assessment investigated several outcomes—income and housing, physical and mental health, life skills, social and assistance network—and relied on a participative mixed-method design. Although the shelter mission is broad and ambitious (i.e. autonomy, global well-being and reintegration into society), the assessment results show that the shelter struggles to have positive effects on the residents beyond the provision of basic care (a roof, food, administrative support) and that the stress felt by the residents even tends to increase during their stay. Several recommendations collectively emerged from the assessment: individualizing shelter support and making it evolve during the stay, reducing the size of the shelter while at the same time fostering community living, developing partnerships. At the public policy level, we would recommend revising the mission of long-term shelters in accordance with their means.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142420394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A theoretical framework to companies value creation through a systematic review of intangibles’ management 通过对无形资产管理进行系统审查，为公司创造价值提供理论框架。

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-10-01 DOI: 10.1016/j.evalprogplan.2024.102506

{"title":"A theoretical framework to companies value creation through a systematic review of intangibles’ management","authors":"","doi":"10.1016/j.evalprogplan.2024.102506","DOIUrl":"10.1016/j.evalprogplan.2024.102506","url":null,"abstract":"<div><div>The purpose of this article is to develop a theoretical framework that identifies the intangible aspects that can be managed and contribute to the value creation for organizations. The theoretical framework was developed based on a systematic review performed according to a protocol that proposes steps to identify the intangible aspects present in the scientific literature. Mendeley software assisted in organizing and reading the 3152 articles identified by the systematic review. The results of the article propose a classification of intangible aspects identified in levels that collaborate with the value creation in organizations. The first and main group being called FPVs, encompassing: reputation, innovation, performance, legitimacy, and knowledge. Strategically, the FPVs are subdivided into 15 CSFs, that cover the other 35 intangibles perceived in the studies, called Indicators and that can be managed. The results of the article provide theoretical and managerial implications and can be used by the academic community and by managers of industrial organizations. The results present reflections on how intangible aspects are present in research and the management of organizations. Furthermore, the literature review proves the importance of analyzing and monitoring intangible aspects nowadays.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142394297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Modeling a needs assessment approach for policymakers to investigate, understand, and reduce gun violence 为决策者提供需求评估方法模型，以调查、了解和减少枪支暴力。

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-09-24 DOI: 10.1016/j.evalprogplan.2024.102505

{"title":"Modeling a needs assessment approach for policymakers to investigate, understand, and reduce gun violence","authors":"","doi":"10.1016/j.evalprogplan.2024.102505","DOIUrl":"10.1016/j.evalprogplan.2024.102505","url":null,"abstract":"<div><div>We conducted a needs assessment evaluation as part of a community effort to plan tailored responses to reduce gun violence in a midwestern town. Various data sources were used to analyze factors leading to gun violence. The results guided the creation of evidence-based response strategies. Study findings demonstrate that even in a small town, the nature and drivers of gun violence vary considerably at the micro level and thus should be examined at this level. In this paper, we present the recommendations that followed from our study, not as solutions for other jurisdictions to adopt on their face, but rather as an example of 1) the types of solutions that follow from a thorough analysis of a local problem, and 2) a plethora of potential responses by social institutions. The goal of the paper is to provide an example process for policymakers from jurisdictions of various sizes and locales of the application of a problem-oriented approach to understanding and preventing gun violence. Undertaking a comprehensive, micro-focus to analyze the gun violence problem and its drivers in a jurisdiction provides essential information to guide the formulation of tailored, evidence-based responses.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142337015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

To scope or not to scope? The benefits and challenges of integrating scoping studies in rapid qualitative research and evaluation 确定范围还是不确定范围？在快速定性研究和评估中整合范围界定研究的益处和挑战

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-09-07 DOI: 10.1016/j.evalprogplan.2024.102495

{"title":"To scope or not to scope? The benefits and challenges of integrating scoping studies in rapid qualitative research and evaluation","authors":"","doi":"10.1016/j.evalprogplan.2024.102495","DOIUrl":"10.1016/j.evalprogplan.2024.102495","url":null,"abstract":"<div><p>Creating sustainable change and fostering collaborative relationships between researchers and stakeholders is a recognized challenge in the field of evaluation. Identifying programme purpose, cultural context, potential challenges, and engaging stakeholders before an evaluation can produce responsive and impactful evaluations. This paper discusses implementing a targeted scoping study within the framework of rapid qualitative research and evaluation. A scoping study enables collaborative decision-making on evaluation priorities, and functions as an evaluability assessment in time-sensitive contexts. In our experience, a scoping study can be carried out in as little as five days or as long as six weeks. It is timely to revisit the question of what factors influence evaluation outcomes, a scoping study can be used to support an evaluation, address access to data and research, and strengthen communication channels. The methodological approach was used to co-produce an evaluation with an NGO that accurately reflected their needs, recognizing possible challenges and solutions.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000971/pdfft?md5=e7b517fda6e66e5253068b449d13b057&pid=1-s2.0-S0149718924000971-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142172993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Arts-based evaluation of the Communities ChooseWell program 以艺术为基础的 "社区选择健康 "计划评估

IF 1.5 4区社会学

Evaluation and Program Planning Pub Date : 2024-09-05 DOI: 10.1016/j.evalprogplan.2024.102496

{"title":"Arts-based evaluation of the Communities ChooseWell program","authors":"","doi":"10.1016/j.evalprogplan.2024.102496","DOIUrl":"10.1016/j.evalprogplan.2024.102496","url":null,"abstract":"<div><p>Arts-based evaluation is an effective and fun way to engage people and uncover meaningful, valid results. In this project, an arts-based approach was used to gain an understanding of the effects the Communities ChooseWell Program has had according to the Champions’ experiences. We wanted to identify what changes, if any, has Communities ChooseWell fostered through the past 10 years? This evaluation was completed using an arts-based approach which allowed us to explore varied long-term effects in different contexts. The creative process allowed for an open approach not predetermining the nature of potential effects. It also gave the participants space to identify what matters the most according to the ChooseWell Champions. This evaluation was in addition to evaluation requirements from the ChooseWell programs funder. In this article we will first present the context of evaluation and state our positionality. We will then present the methodology and methods. Finally, we will present and discuss the results and recommendations.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000983/pdfft?md5=6c1c147bb27c304dd34c36a307a75f54&pid=1-s2.0-S0149718924000983-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142172994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0