Evaluation and Program Planning最新文献

{"title":"Effectiveness of the KiVa anti-bullying program in the Czech Republic: A cluster randomized control trial","authors":"Adam Klocek ,&nbsp;Lenka Kollerová ,&nbsp;Egle Havrdová ,&nbsp;Monika Kotrbová ,&nbsp;Jan Netík ,&nbsp;Marek Pour","doi":"10.1016/j.evalprogplan.2024.102459","DOIUrl":"10.1016/j.evalprogplan.2024.102459","url":null,"abstract":"<div><h3>Aims</h3><p>One of the most widely used evidence-based anti-bullying programs, KiVa, originates from Finland and aims to change students’ peer context, activate teachers, and inform parents with two main components (universal preventive actions and indicated actions when bullying occurs), complemented by monitoring. Because research documented somewhat varied KiVa outcomes in different countries and because there is a lack of research focusing solely on the effectiveness of the universal and indicated actions, this study aimed to evaluate the effectiveness of KiVa main components when implemented in a new country. This two-arm parallel cluster randomized control trial (RCT) evaluated the effectiveness of the KiVa anti-bullying program in elementary schools in the Czech Republic. It examined the effects of the universal and indicated actions on self-reported bullying and victimization as primary outcomes and well-being as a secondary outcome, while keeping monitoring constant across the intervention and control schools. The study also examined the role of implementation fidelity on the proposed outcomes.</p></div><div><h3>Methods</h3><p>Schools were allocated via stratified randomization into a KiVa intervention group (12 schools, 35 classes, <em>N</em> = 407 students) or a wait-list control group (12 schools, 32 classes, <em>N</em> = 400 students). The study employed data from baseline and post-measurement waves, which were 10 months apart.</p></div><div><h3>Results</h3><p>The data were analyzed using linear mixed effects models, which showed no significant intervention or fidelity effects for bullying, victimization, and well-being. However, promising trends (at α &lt; .10) were revealed, such as lower levels of bullying observed in the intervention group and in schools with high implementation fidelity. Additional analysis using Bayes factors supported these promising trends and provided moderate support for lower levels of victimization in the intervention group compared to the control schools.</p></div><div><h3>Conclusions</h3><p>Evaluation of effectiveness of anti-bullying programs could benefit from a more targeted fidelity assessment at the classroom or individual level and from distinguishing between the effects of the main components of the programs and the effects of monitoring. The promising yet non-significant intervention and fidelity effects suggest that schools may require enhanced support and longer implementation time frames than a single school year, especially when implementation faces structural obstacles, such as the Covid-19 pandemic.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102459"},"PeriodicalIF":1.5,"publicationDate":"2024-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141401629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A pilot study of the outcomes of human trafficking survivors assessment with girls who have experienced commercial sexual exploitation","authors":"Claire Helpingstine ,&nbsp;Maureen C. Kenny ,&nbsp;Ricardo Rubiales","doi":"10.1016/j.evalprogplan.2024.102462","DOIUrl":"10.1016/j.evalprogplan.2024.102462","url":null,"abstract":"<div><h3>Purpose</h3><p>This study utilized the Outcome of Human Trafficking Survivors (OHTS) to monitor the progress of female victims of CSE over a year while they were enrolled in a comprehensive treatment program<strong>.</strong></p></div><div><h3>Methods</h3><p>Sixty-seven girls (<em>M</em> age 17.70 years) who were identified as confirmed victims of CSE (46 %) or at risk for CSE (54 %) and presented for treatment were administered the OHTS upon intake into the program and again at 90 day intervals resulting in four data points. Ratings were provided by staff members who interact with the member in each area assessed. A repeated-measures ANOVA was conducted comparing mean outcome differences across all four timepoints.</p></div><div><h3>Results</h3><p>Results indicate that significant changes were found in the area of Housing and Education, with gains in Education over time. For this sample, Parenting and Immigration were domains of less relevance and data was not consistently obtained in these areas. Other categories assessed by the OHTS did not demonstrate significant changes over time.</p></div><div><h3>Conclusion</h3><p>The OHTS can be used to track progress of clients enrolled in treatment programming, but the goals of the program should align with the categories that are assessed in the OHTS. Repeated administration may be difficult due to high dropout rates in treatment and there may be rater bias.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102462"},"PeriodicalIF":1.6,"publicationDate":"2024-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141404641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Engaging evaluation champions: strategies to build evaluation capacity and promote data-informed decision making in youth development","authors":"Theresa N. Melton ,&nbsp;Ashlee L. Sjogren ,&nbsp;Jennifer P. Agans","doi":"10.1016/j.evalprogplan.2024.102458","DOIUrl":"10.1016/j.evalprogplan.2024.102458","url":null,"abstract":"<div><p>Youth development researchers and practitioners share an interest in ensuring that youth development programs contribute positively to youth outcomes. Engaging in evaluation and data-informed decision making (DIDM) has the potential to empower practitioners to improve and adapt programs, improving youth outcomes. Yet, not all practitioners are comfortable engaging in evaluation and/or DIDM. Engaging in partnerships and utilizing internal supporters of evaluations (or “champions”) have been identified as potential strategies to build evaluation capacity and strengthen DIDM within an organization. However, little research explores how to do so. This study engaged evaluation champions in four states to examine their experience as they partnered with practitioners within their organization. Results suggest that peer groups can be utilized to promote evaluation capacity, especially by utilizing peers that already have an interest in evaluation. Practitioner engagement can also be developed by using less academic jargon in communication, highlighting the practical value of evaluation, and building capacity slowly.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102458"},"PeriodicalIF":1.6,"publicationDate":"2024-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141327931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reimagining the university’s role in the health and well-being of the people it serves","authors":"Kathleen J. Quinn ,&nbsp;Julie M. Kapp ,&nbsp;Sawiyyah Chanay ,&nbsp;Thom Pancella ,&nbsp;Kimberly G. Hoffman","doi":"10.1016/j.evalprogplan.2024.102450","DOIUrl":"10.1016/j.evalprogplan.2024.102450","url":null,"abstract":"<div><p>Academic medical centers and university extension programs remain underdeveloped collaborators, despite the complementary objectives between translational science and extension. This case study details the creation of a nationally unique interprofessional organizational structure between the University of Missouri (MU) Office of Extension and Engagement (MU Extension) and the MU School of Medicine to accelerate statewide reach of research and education discoveries using high-touch community health approaches. This article describes specific strategies used to systematically plan for: 1) creation and operation of the new structure, 2) routinization and institutionalizing the work, and 3) sustainability. We further outline challenges and next steps. The development of the backbone organization office of Health Outreach Policy and Education (HOPE) brings together the interprofessional expertise of five units with a common agenda to advance mutually reinforcing activities. HOPE is poised to make significant contributions to amplify MU’s land grant mission, garner additional grant funding, and advance the health of Missourians.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102450"},"PeriodicalIF":1.5,"publicationDate":"2024-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000521/pdfft?md5=de9d110f343effce59cdd94a35136c02&pid=1-s2.0-S0149718924000521-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141411293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Icelandic Prevention Model Evaluation Framework and Implementation Integrity and Consistency Assessment","authors":"Michael J. Mann ,&nbsp;John P. Allegrante ,&nbsp;Megan L. Smith ,&nbsp;Inga Dora Sigfusdottir ,&nbsp;Alfgeir L. Kristjansson","doi":"10.1016/j.evalprogplan.2024.102451","DOIUrl":"https://doi.org/10.1016/j.evalprogplan.2024.102451","url":null,"abstract":"<div><p>The Icelandic Prevention Model (IPM) follows a systematic but flexible process of community capacity building, data collection, analysis, dissemination, and community-engaged decision-making to guide the data-informed selection, prioritization, and implementation of intervention strategies in preventing adolescent substance use. This paper describes two new evaluation tools intended to assess the: 1) integrity of IPM implementation, and 2) unique aspects of IPM implementation in different community contexts. These evaluation tools include a: 1) five-phase <em>IPM Evaluation Framework for Assessing Value Across Communities, Cultures, and Outcomes (IPM-EF)</em>; and 2) 10-Step <em>IPM Implementation Integrity and Consistency Assessment</em> (<em>IPM-IICA</em>) that utilizes both quantitative (scored) and qualitative (narrative) data elements to characterize implementation integrity and consistency at both community coalition and school community levels. The IPM-EF includes five phases. Phase 1: Describe the Intervention Context; Phase 2a: Document the Extent to Which the 10 Steps of the IPM were Implemented (using the IPM-IICA scored); Phase 2b: Document the Unique Community-Specific Methods Used within the 10 Steps of the IPM to Tailor Local Intervention Delivery (using the IPM-IICA narrative); Phase 3: Measure Changes in Community Risk and Protective Factors; Phase 4: Measure the Outcomes Associated with the IPM; and Phase 5: Investigate Multiple Full Cycles Over Time.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102451"},"PeriodicalIF":1.6,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000533/pdfft?md5=f4d4fedfa094d47cc0a82ca125168367&pid=1-s2.0-S0149718924000533-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141329054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Digital competence among pre-service teachers: A global perspective on curriculum change as viewed by experts from 33 countries","authors":"Łukasz Tomczyk","doi":"10.1016/j.evalprogplan.2024.102449","DOIUrl":"10.1016/j.evalprogplan.2024.102449","url":null,"abstract":"<div><p>In the intensely developing information society, the education of pre-service teachers presents an organisational, methodological, and conceptual challenge. Recent years have necessitated reflection on the content and modalities of the formation of digital competence in pre-service teachers. This article – considers which components the digital competences of the new generations of teachers should contain and how this area can be effectively developed within academic education. The article is the result of interviews conducted with 35 experts (recognized scholars in the field of media pedagogy) from North and South America, Africa, Australia and Oceania, Asia, and Europe. Qualitative interviews using the Delphi methodology were conducted between June and September 2022. Using grounded theory principles, it was noted that there is a need to modify pre-service teachers' curricula in the following areas: creating, retrieving and managing digital content; integrating ICT into subject content; developing soft skills; enhancing interactivity in the classroom; social aspects of ICT; distance learning; new socio-technical processes; advanced digital competencies; predictive learning; analysing the impact of ICT on human life; data protection; e-risks; low digital competencies; understanding media messaging; and supporting people with disabilities.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"105 ","pages":"Article 102449"},"PeriodicalIF":1.6,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S014971892400051X/pdfft?md5=9a75901808c2142c7901b1e6908a1742&pid=1-s2.0-S014971892400051X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141238797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Empowering underprivileged students beyond financial aid: Insights from a scholarship program's monitoring and evaluation","authors":"Miriam Mutevere ,&nbsp;Thulani Runyararo Dzinamarira ,&nbsp;Lorcadia Muzenda ,&nbsp;Stephen Nyoka ,&nbsp;Valentine Chokudinga ,&nbsp;Tawanda Mugoniwa ,&nbsp;Enos Moyo ,&nbsp;Fortunate Kakumura ,&nbsp;Tafadzwa Dzinamarira","doi":"10.1016/j.evalprogplan.2024.102448","DOIUrl":"10.1016/j.evalprogplan.2024.102448","url":null,"abstract":"<div><h3>Purpose</h3><p>This paper examines the impact of a scholarship program on underprivileged students, drawing on data from a two-year monitoring and evaluation (M&amp;E) process. The report identifies both enablers and barriers to academic success among scholarship beneficiaries.</p></div><div><h3>Methods</h3><p>Data on program impact was collected through interviews with parents, teachers, and school records over two academic years.</p></div><div><h3>Results</h3><p>Financial aid emerged as a crucial enabler, with scholarships allowing students to focus on their studies by alleviating pressure around basic necessities. However, the research also revealed the importance of a holistic support system. Beyond tuition, the high cost of essential learning materials, including stationery, and subject-specific resources, can create a significant barrier. The study also highlighted the importance of student well-being. Health concerns, limited access to nutritious food, and even unaddressed mental health issues can all negatively impact attendance and focus. Furthermore, a gender gap emerged, with girls facing additional challenges related to social pressures to prioritize chores and the cost of menstrual hygiene products.</p></div><div><h3>Conclusion</h3><p>This study highlights the importance of holistic scholarship programs that extend beyond tuition coverage. To maximize impact, policymakers and funders should prioritize initiatives that address the multifaceted needs of underprivileged students.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"105 ","pages":"Article 102448"},"PeriodicalIF":1.6,"publicationDate":"2024-05-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141181180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Case study in using value stream mapping in online graduate education","authors":"Candace D. Bloomquist ,&nbsp;Lydia Holtz ,&nbsp;Angela M. Lampe ,&nbsp;Christine T. Carmichael","doi":"10.1016/j.evalprogplan.2024.102447","DOIUrl":"10.1016/j.evalprogplan.2024.102447","url":null,"abstract":"<div><p>Given the rise in demand for online, graduate degree programs there is a need to guide program planning and evaluation professionals within institutions of higher education to utilize continuous quality improvement (CQI) strategies. Using principles of design thinking and CQI, the purpose of this case study was to describe a CQI project that sought to develop a better understanding of adult students’ experience progressing through their plan of study in an online doctoral program. As part of the CQI project, value stream mapping (VSM) was used to gain visibility and perspective on the actions required by faculty and staff to guide students through their plan of study. The VSM process provided information that led to conversations that furthered the CQI efforts and led to changes in the adult education program that would benefit all students. Improvement processes that lead to better quality and more positive experiences for users are valuable. This case study demonstrates the processes, challenges, lessons, and future directions in the use of VSM to better understand online graduate programs designed for adult learners.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"105 ","pages":"Article 102447"},"PeriodicalIF":1.6,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141135661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Examining the causes and effects of time overruns in construction projects promoted by rural municipalities in Nepal","authors":"Madhav Prasad Koirala ,&nbsp;Roshan Samrat Shahi","doi":"10.1016/j.evalprogplan.2024.102436","DOIUrl":"10.1016/j.evalprogplan.2024.102436","url":null,"abstract":"<div><p>Time overruns are the most common issue in construction projects, which have a significant negative impact on the project and the parties involved. This also applies to National Competitive Bidding (NCB) construction projects in municipalities, jeopardizing project timelines, costs, and quality standards. Poor performance by professionals in rural municipalities has resulted in inadequate infrastructure development in these areas. The goal of this study is to understand the causes and effects of time overruns by employing an exploratory and descriptive research methodology. The data from clients, consultants, and contractors consistently highlights key factors contributing to project delays, including adverse weather conditions, limited involvement from contractor head offices, insufficient consultant experience, inadequate site management, and delays in site mobilization. These delays incur substantial costs related to materials, land acquisition, labor, and market risks. The results included that 52.17% the projects experienced time overruns ranging from 24.4% to 514.71%. Furthermore, 21.73% of projects experienced cost overruns, with the highest at 19.92%. These findings provide critical insights for improving project management and addressing construction time overruns. It recommends conducting national-scale analyses to gain a comprehensive understanding of construction delays and to implement effective mitigation strategies.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"105 ","pages":"Article 102436"},"PeriodicalIF":1.6,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141055134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing evaluation measures for health departments sharing and linking HIV and STI surveillance data","authors":"Shannon Hitchcock,&nbsp;Leigh Evans,&nbsp;Jane Fox","doi":"10.1016/j.evalprogplan.2024.102435","DOIUrl":"10.1016/j.evalprogplan.2024.102435","url":null,"abstract":"<div><p>Enhancing data sharing, quality, and use across siloed HIV and STI programs is critical for national and global initiatives to reduce new HIV infections and improve the health of people with HIV. As part of the <em>Enhancing Linkage of STI and HIV Surveillance Data in the Ryan White HIV/AIDS Program</em> initiative, four health departments (HDs) in the U.S. received technical assistance to better share and link their HIV and STI surveillance data. The process used to develop evaluation measures assessing implementation and outcomes of linking HIV and STI data systems involved six steps: 1) measure selection and development, 2) review and refinement, 3) testing, 4) implementation and data collection, 5) data quality review and feedback, and 6) dissemination. Findings from pilot testing warranted slight adaptations, including starting with a core set of measures and progressively scaling up. Early findings showed improvements in data quality over time. Lessons learned included identifying and engaging key stakeholders early; developing resources to assist HDs; and considering measure development as iterative processes requiring periodic review and reassessment to ensure continued utility. These findings can guide programs and evaluations, especially those linking data across multiple systems, in developing measures to track implementation and outcomes over time.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"105 ","pages":"Article 102435"},"PeriodicalIF":1.6,"publicationDate":"2024-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141030267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}