Evaluation Review最新文献

Under What Conditions Does Evidence Travel? Assessing Context Sensitivity and Generalizability in Cash Transfer Program Effectiveness. 证据在什么条件下传播？评估现金转移计划有效性的情境敏感性和普遍性。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-12-01 Epub Date: 2026-07-14 DOI: 10.1177/0193841X261463320

Anis Ben Brik

{"title":"Under What Conditions Does Evidence Travel? Assessing Context Sensitivity and Generalizability in Cash Transfer Program Effectiveness.","authors":"Anis Ben Brik","doi":"10.1177/0193841X261463320","DOIUrl":"10.1177/0193841X261463320","url":null,"abstract":"The evidence-based policy movement assumes that rigorously evaluated interventions replicate across contexts to produce similar effects. This universalist assumption may be unwarranted for complex social programs, where context shapes effectiveness. This study examines generalizability using 15 cash transfer programs evaluated across 12 Middle Eastern and North African countries through four complementary approaches: leave-one-out cross-validation, permutation tests, equivalence testing, and fragility indices. Leave-one-out analysis demonstrates robust stability, with 100% of key relationships maintaining a magnitude within 25% of full-sample estimates. The correlation between transfer size and poverty reduction (r = 0.711, p < .001) shows strong consistency, and permutation tests yield p = .001, indicating genuine signals rather than chance patterns. Equivalence testing reveals that programs with nearly identical designs can produce statistically distinguishable outcomes, with confidence intervals excluding practical equivalence in 67% of matched comparisons. Fragility indices of 2 to 3 for most relationships indicate moderate robustness. These findings suggest that some design-outcome relationships show stability while context matters substantially for program effectiveness. This study offers methodological tools for assessing evidence robustness in small-sample comparative research and argues for epistemic humility in cross-context predictions.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1094-1139"},"PeriodicalIF":3.0,"publicationDate":"2026-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148449967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Green Policies for the Circular Economy and Entrepreneurship: International Evidence. 循环经济和创业的绿色政策：国际证据。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-12-01 Epub Date: 2026-05-31 DOI: 10.1177/0193841X261456358

Nga Phan Thi Hang

引用次数: 0

Peer-led Support Groups for Parents Following Child Removal: A Mixed-Methods Evaluation Study. 儿童移除后父母的同伴领导支持小组：一项混合方法评估研究。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-12-01 Epub Date: 2026-06-23 DOI: 10.1177/0193841X261462222

Linda M Callejas, Connie Walker-Egea, Svetlana Yampolskaya, Aaliyah Torres, Anna Davidson Abella

{"title":"Peer-led Support Groups for Parents Following Child Removal: A Mixed-Methods Evaluation Study.","authors":"Linda M Callejas, Connie Walker-Egea, Svetlana Yampolskaya, Aaliyah Torres, Anna Davidson Abella","doi":"10.1177/0193841X261462222","DOIUrl":"10.1177/0193841X261462222","url":null,"abstract":"The evidence base for parent support groups in child welfare systems is limited but growing. Evaluation of such programs requires flexible and context-specific designs that capture both outcomes and mechanisms of change. This exploratory mixed-methods evaluation examined a peer-led support group program for parents who experienced child removal in a large Florida county to better understand how these programs operate and benefit parents working toward reunification. A total of 22 parents participated in weekly sessions facilitated by peer specialists with prior child welfare involvement. Quantitative assessments measured changes in psychosocial functioning and engagement in child welfare services, while semi-structured interviews (n = 8) explored participant perceptions of group processes and benefits. Significant decreases were observed in depression, trauma, and loneliness, alongside increases in perceived emotional support. Thematic analysis identified four interrelated themes that informed an empirically grounded conceptual framework describing the functions and benefits of support groups.This evaluation study demonstrates how mixed-methods approaches can yield actionable evidence from small scale community-based interventions and offers insight into the mechanisms that may inform future implementation and evaluation of peer-led models.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1034-1059"},"PeriodicalIF":3.0,"publicationDate":"2026-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148309946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Brokerage, Gender, and Academic Performance in Interdisciplinary Co-Authorship Networks: A Study of Policy-Related Social Learning Publications. 跨学科合著网络中的经纪、性别与学术表现：政策相关社会学习出版物的研究。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-12-01 Epub Date: 2026-07-01 DOI: 10.1177/0193841X261464677

Elisa Espín-Gallardo, José Antonio Belso-Martínez, María José López-Sánchez

{"title":"Brokerage, Gender, and Academic Performance in Interdisciplinary Co-Authorship Networks: A Study of Policy-Related Social Learning Publications.","authors":"Elisa Espín-Gallardo, José Antonio Belso-Martínez, María José López-Sánchez","doi":"10.1177/0193841X261464677","DOIUrl":"10.1177/0193841X261464677","url":null,"abstract":"Brokers serve as key connectors linking academic researchers who might otherwise remain unconnected, to co-authorship networks. This study examines whether more complex interdisciplinary co-authorships yield greater scholarly impact than ties with authors from the same discipline, which are facilitated by cognitive similarity. It also tests the extent to which such collaborations influence the academic performance of female researchers. The study analyzed 594 authors and 271 papers in the field of social learning. Despite their complexity, regression analyses confirm the benefits of novel sources and combinations of interdisciplinary knowledge. The higher coordination and communication costs of interdisciplinary collaboration are offset by the potential of the new knowledge generated. Interdisciplinary collaboration is associated with higher performance among female scholars, contributing to increased recognition and reputation. This pattern suggests that collaboration strategies and institutional support should reduce coordination barriers in interdisciplinary co-authorship and facilitate women's access to high-impact brokerage opportunities.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1060-1093"},"PeriodicalIF":3.0,"publicationDate":"2026-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148362351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Insights From Developmental and Formative Evaluations: Developing and Implementing a Limited-Term Housing Mobility Program. 发展性和形成性评估的见解：制定和实施一项有期限的住房流动计划。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-12-01 Epub Date: 2026-07-16 DOI: 10.1177/0193841X261468394

Jee Young Lee, Jason Reece, Rachel Garshick Kleit

{"title":"Insights From Developmental and Formative Evaluations: Developing and Implementing a Limited-Term Housing Mobility Program.","authors":"Jee Young Lee, Jason Reece, Rachel Garshick Kleit","doi":"10.1177/0193841X261468394","DOIUrl":"10.1177/0193841X261468394","url":null,"abstract":"This paper aims to demonstrate how evaluation methods can inform the development of a program, using the example of a novel housing mobility program. The limited availability of Housing Choice Vouchers (HCVs) and other affordable housing assistance restricts the ability to address US national housing needs; millions more households qualify for HCVs than receive them. Those with HCVs encounter challenges in using vouchers in so-called high-opportunity areas. To address the dearth of HCVs and difficulty using them, a collaboration between faculty at Ohio State University and the community developed a new housing mobility program to increase the affordable housing supply, engaging stakeholders to understand the needs of low-income renters who are less likely to obtain an HCV. This paper outlines the Families Flourish program's development using both developmental and formative evaluations, examining the progress of the implementation and immediate outcomes using mixed methods. The developmental evaluation process required collaboration with a wide variety of stakeholders, partners, and the project team. According to the formative evaluation of the pilot program, participants' experiences suggested that enhanced housing security was associated with improvements in physical and mental health for parents and their children, and that a more stable and safe neighborhood environment was similarly associated with gains in employment, education, and income. Throughout the process of evaluation, particularly the developmental evaluation, the evaluator's communication and collaboration skills were essential to the methodology. The study concludes with lessons learned, as well as recommendations for policy and directions for future research.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1140-1190"},"PeriodicalIF":3.0,"publicationDate":"2026-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13672924/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148473117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Goldilocks Regression Discontinuity Design: A Dual-Threshold Framework for Evaluating Interventions Outside an Optimal Range. 金凤花回归不连续设计：评估最佳范围外干预的双阈值框架。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-11-06 DOI: 10.1177/0193841X261504243

Brian G Moss

{"title":"The Goldilocks Regression Discontinuity Design: A Dual-Threshold Framework for Evaluating Interventions Outside an Optimal Range.","authors":"Brian G Moss","doi":"10.1177/0193841X261504243","DOIUrl":"https://doi.org/10.1177/0193841X261504243","url":null,"abstract":"Regression discontinuity designs (RDDs) are widely used to estimate causal effects when treatment assignment is determined by a threshold on a continuous running variable. Many interventions, however, target individuals at both ends of a continuum, with treatment assigned when values fall either below a lower threshold or above an upper threshold. This paper introduces the Goldilocks Regression Discontinuity Design (GRDD), a dual-threshold framework for evaluating such interventions. GRDD estimates two locally identified treatment effects, one at each threshold, while allowing both effects to be interpreted within a common intervention framework. Separate local, simultaneous, and spline-based estimation strategies are presented. A 10-condition Monte Carlo simulation evaluates estimator performance across differences in sample size, functional form, heteroskedasticity, running-variable density, and treatment-effect magnitude. Results demonstrate an important tradeoff between precision and robustness. Simultaneous estimation provides substantial precision gains when the common functional form is correctly specified but can produce substantial bias and poor confidence interval coverage under misspecification. Separate local estimation is more consistently robust across conditions, while spline-based estimation was generally robust under nonlinear relationships. A semi-synthetic application using data from the National Health and Nutrition Examination Survey (NHANES) illustrates implementation of the framework at lower and upper body mass index thresholds. GRDD extends the application of RDD to dual-threshold interventions while preserving local identification at each cutoff and providing a unified framework for comparing intervention effects across distinct regions of need.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X261504243"},"PeriodicalIF":3.0,"publicationDate":"2026-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149371977","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Using Theory of Change Models to Evaluate the Implementation of Urinary Incontinence Management in Primary Care Initiatives. 用变化理论模型评价尿失禁管理在初级保健中的实施。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-11-02 DOI: 10.1177/0193841X261503769

Gabriela Alvarado, Shreya Huilgol, Nipher Malika, Peter Mendel, Elizabeth Cope, Jaime Adler, Brianna Bragg, Poonam Pardasaney

{"title":"Using Theory of Change Models to Evaluate the Implementation of Urinary Incontinence Management in Primary Care Initiatives.","authors":"Gabriela Alvarado, Shreya Huilgol, Nipher Malika, Peter Mendel, Elizabeth Cope, Jaime Adler, Brianna Bragg, Poonam Pardasaney","doi":"10.1177/0193841X261503769","DOIUrl":"https://doi.org/10.1177/0193841X261503769","url":null,"abstract":"Theory of change (ToC) models articulate how and why implementation interventions are expected to produce specific outcomes and can guide both program design and evaluation. As part of a theory-driven evaluation for a federal initiative to improve non-surgical management of urinary incontinence among women in primary care, we co-developed ToC models for five funded projects. Each model depicted intervention approaches, hypothesized mechanisms of change, and expected outcomes. Using a six-step process which included document review, consensus-based element identification, open coding, iterative refinement, and stakeholder validation, we created and refined a separate ToC for each grantee. The Technical Expert Panel, convened for the broader initiative, provided multidisciplinary feedback to ensure methodological rigor and contextual accuracy. Sharing the models in annual meetings supported cross-grantee learning and revealed common implementation themes: reliance on technology-based patient education and clinician training, challenges with practice recruitment, tailored facilitation to reduce clinician burden, and gaps in patient-level evaluation measures. The ToC models served four primary functions: orientation for evaluators, technical assistance for grantees, monitoring and evaluation of implementation processes, and cross-site collaboration. Key lessons included the importance of early stakeholder engagement, flexibility in model development, and the value of visual tools for communication and shared understanding. Although resource requirements and retrospective development were challenges, ToC models proved to be adaptable, dynamic tools that enhance evaluation and collaboration in complex, multi-site health initiatives. Their systematic application can strengthen future national and international implementation efforts.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X261503769"},"PeriodicalIF":3.0,"publicationDate":"2026-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149325979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reconceptualising Evaluation and Assessment in the Era of AI: From Human-Centred Assessment to Hybrid Intelligence. 人工智能时代评估与评估的重新定义：从以人为中心的评估到混合智能。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-10-27 DOI: 10.1177/0193841X261501223

Andy Nguyen, Luna Huynh, Lesley Gardner, Marios Kremantzis, Hiep-Hung Pham

{"title":"Reconceptualising Evaluation and Assessment in the Era of AI: From Human-Centred Assessment to Hybrid Intelligence.","authors":"Andy Nguyen, Luna Huynh, Lesley Gardner, Marios Kremantzis, Hiep-Hung Pham","doi":"10.1177/0193841X261501223","DOIUrl":"https://doi.org/10.1177/0193841X261501223","url":null,"abstract":"Artificial intelligence (AI) is reshaping the conceptual, methodological, and ethical foundations of educational evaluation and assessment. Contemporary assessment models are increasingly challenged by the erosion of clear boundaries between human and machine authorship, complicating assumptions about what constitutes learner-generated performance, knowledge, and competence. The classical assessment paradigm, grounded in the assumption that learner performance reflects individual and independently produced cognition, is increasingly inadequate in educational environments in which AI authors tasks, scores responses, generates feedback, converses with learners, and accompanies students as a partner in the learning processes. This editorial introduces five contributions to this special issue and uses them to advance a broader conceptual agenda for evaluation and assessment in the age of AI. We position the role of assessment in the age of AI across two interconnected areas: assessment and evaluation for AI-integrated learning, where learners collaborate with AI systems to co-construct knowledge and improve performance, and AI-assisted and AI-mediated assessment, in which AI supports, shapes, and increasingly participates in the processes of evaluation, feedback, and decision-making. Across both domains, we develop an intersection of the evidence generated by the five articles included in this special issue, extending beyond the individual contributions to identify broader conceptual, methodological, and practical implications for assessment in the age of AI. We argue that AI's integration into the learning-assessment ecosystem requires a fundamental reconceptualisation of assessment theory, validity, and learner agency under conditions of hybrid intelligence. We conclude by outlining a future research agenda for evaluation and assessment in AI-mediated educational environments.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X261501223"},"PeriodicalIF":3.0,"publicationDate":"2026-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149275585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Affirmative Action in Brazilian Universities: Evidence From Longitudinal Data. 巴西大学的平权行动：来自纵向数据的证据。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-10-01 Epub Date: 2026-03-18 DOI: 10.1177/0193841X261430056

Claudia Bueno Rocha Vidigal, Vinicius Gonçalves Vidigal

引用次数: 0

"I Can Make My Own Schedule": A Mixed Methods Analysis of the Impact of the Health Profession Opportunity Grant Program on Workplace Flexibility. “我可以制定我自己的时间表”：健康职业机会补助计划对工作场所灵活性影响的混合方法分析。

IF 3 4区社会学

Evaluation Review Pub Date : 2026-10-01 Epub Date: 2026-05-04 DOI: 10.1177/0193841X261446039

Muazzam Toshmatova, Andrea Hetling, Grace Maruska, Victoria Coty

{"title":"\"I Can Make My Own Schedule\": A Mixed Methods Analysis of the Impact of the Health Profession Opportunity Grant Program on Workplace Flexibility.","authors":"Muazzam Toshmatova, Andrea Hetling, Grace Maruska, Victoria Coty","doi":"10.1177/0193841X261446039","DOIUrl":"10.1177/0193841X261446039","url":null,"abstract":"Policymakers are increasingly focused on promoting job quality as a means to enhance financial stability and well-being through employment. Thus, evaluations of public workforce inventions are adding employment outcome measures that go beyond wages in order to assess job quality more broadly. This study employs a mixed methods design to understand what aspects of job quality, beyond wages and benefits, are valued by most participants of a federally funded sectoral job training program and then assess the impact of the program using these grounded insights. We use secondary data from a randomized control trial of the Health Profession Opportunity Grant (HPOG) for our analyses. Themes from a qualitative analysis of 153 participant interviews informed the selection of three flexibility-related outcome measures for quantitative analysis using experimental survey data. Intention-to-treat estimates were generated using linear regression and ordered logit models, with subgroup analyses examining race/ethnicity, nativity, parental status, and education. Workplace flexibility emerged as a widely valued aspect of job quality, with participants emphasizing scheduling choice as a strategy to achieve work-life balance. Statistical models demonstrate that the program facilitates access to jobs with greater workplace flexibility. Heterogeneity analysis reveals differential impacts by ethnicity and education level, and notable positive results for individuals with children, a group more likely to value and benefit from flexibility. HPOG improved access to jobs offering workplace flexibility-an aspect of job quality prioritized by participants. Findings underscore the importance of incorporating non-wage metrics, especially work-life balance, in workforce program evaluations.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"970-1008"},"PeriodicalIF":3.0,"publicationDate":"2026-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147844334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0