Evaluation Review最新文献

筛选
英文 中文
External Validity for Complex Interventions: Towards Pragmatic Strategies for Evaluators. 复杂干预的外部效度:面向评估者的实用策略。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-05-15 DOI: 10.1177/0193841X251343130
Thomas Delahais
{"title":"External Validity for Complex Interventions: Towards Pragmatic Strategies for Evaluators.","authors":"Thomas Delahais","doi":"10.1177/0193841X251343130","DOIUrl":"https://doi.org/10.1177/0193841X251343130","url":null,"abstract":"<p><p>Strategies for ensuring the generalizability of evaluation results often focus on building methodologically sound research designs. However, these strategies may fall short when the challenge lies as much in uncovering causal patterns than in applying them to uncertain contexts. This article proposes a set of pragmatic strategies to address generalizability issues in complex evaluations. These strategies address important questions such as: What needs to be generalized, for whom and for what purpose? What assumptions underlying the intervention could be generalized? How can approaches and methods be selected to support a generalization objective? How can potential users be involved to support the process of generalization? We illustrate these strategies using the example of a series of evaluations of bottom-up socio-ecological transition initiatives in France between 2013 and 2020. We suggest that these strategies can enhance policy learning and better equip policy stakeholders to navigate complex situations.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251343130"},"PeriodicalIF":3.0,"publicationDate":"2025-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144081408","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
External Validity in an Instrumental Variable Setting. 工具变量设置中的外部效度。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-05-15 DOI: 10.1177/0193841X251342619
Alexander Kwon, Kyungtae Lee
{"title":"External Validity in an Instrumental Variable Setting.","authors":"Alexander Kwon, Kyungtae Lee","doi":"10.1177/0193841X251342619","DOIUrl":"https://doi.org/10.1177/0193841X251342619","url":null,"abstract":"<p><p>We study the external validity of instrumental variable estimation. The key assumption we impose for external validity is conditional external unconfoundedness among compliers, which means that the treatment effect and target selection are independent among compliers conditional on covariates. We study this assumption with a case study about the impact of solid-fuel usage on women's average cooking time. Among the six countries examined, we find no statistical evidence that the assumptions required for external validity are violated for four countries (Ethiopia, Honduras, Kenya, and Zambia), but in Cambodia and Nepal, we find low external validity. These results provide suggestive evidence that the assumptions required for external validity are violated for these two countries.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251342619"},"PeriodicalIF":3.0,"publicationDate":"2025-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144081409","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Assessing the Impact of a University Transition Online Course on Student Continuation Using Statistical Matching Methods. 利用统计匹配方法评估大学转型在线课程对学生继续学习的影响。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-05-02 DOI: 10.1177/0193841X251339686
Billy Wong, Lydia Fletcher
{"title":"Assessing the Impact of a University Transition Online Course on Student Continuation Using Statistical Matching Methods.","authors":"Billy Wong, Lydia Fletcher","doi":"10.1177/0193841X251339686","DOIUrl":"https://doi.org/10.1177/0193841X251339686","url":null,"abstract":"<p><p>This study demonstrates how to evaluate a university-wide online course designed to support student transition into university by using Propensity Score Matching (PSM) and Doubly Robust Estimation (DRE). Using data from seven academic years, from 2016/17 to 2022/23, with more than 28,000 students, we examine whether enrolment in this optional pre-arrival course affects first-year pass rates. We also conducted additional analyses to compare outcomes from the year before and after the course's implementation, as well as to examine these patterns across recent cohorts to potentially account for contextual changes over time. Results indicate that enrolled students show a 6.2 percentage point increase in the likelihood of passing Year 1, controlling for factors including sex, domicile, age, ethnicity, disability and socioeconomic status. We demonstrate how utilising existing institutional data can potentially strengthen evidence of impact for centralised initiatives and conclude with reflections on the use of such institutional data and matching techniques and their viability for future evaluations.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251339686"},"PeriodicalIF":3.0,"publicationDate":"2025-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144039550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Systematic Review of the Methods Used in Rapid Approaches to Research and Evaluation. 快速研究和评估方法的系统综述。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-26 DOI: 10.1177/0193841X251338536
Sigrún Eyrúnardóttir Clark, Norha Vera San Juan, Thomas Moniz, Rebecca Appleton, Phoebe Barnett, Cecilia Vindrola-Padros
{"title":"A Systematic Review of the Methods Used in Rapid Approaches to Research and Evaluation.","authors":"Sigrún Eyrúnardóttir Clark, Norha Vera San Juan, Thomas Moniz, Rebecca Appleton, Phoebe Barnett, Cecilia Vindrola-Padros","doi":"10.1177/0193841X251338536","DOIUrl":"https://doi.org/10.1177/0193841X251338536","url":null,"abstract":"<p><p>Rapid approaches are essential when resources are limited and when findings are required in real-time to inform decisions. Limitations exist in their design and implementation, which can lead to a reduced level of trust in findings. This review sought to map the methods used across rapid evaluations and research to facilitate timeliness and support the rigour of studies. Four scientific databases and one search engine were searched between 11-16th August 2022. Screening led to the inclusion of 169 articles that provided a much-needed repository of methods that can be used during the design and implementation of rapid studies to improve their trustworthiness. No reporting guidelines specific to rapid research or evaluation were identified in the literature, we therefore suggest that this repository of methods informs the development of transparent reporting standards for future rapid research and evaluation.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251338536"},"PeriodicalIF":3.0,"publicationDate":"2025-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144040982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluating Bibliometrics Reviews: A Practical Guide for Peer Review and Critical Reading. 评价文献计量学评论:同行评议和批判性阅读的实用指南。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-18 DOI: 10.1177/0193841X251336839
Anh-Duc Hoang
{"title":"Evaluating Bibliometrics Reviews: A Practical Guide for Peer Review and Critical Reading.","authors":"Anh-Duc Hoang","doi":"10.1177/0193841X251336839","DOIUrl":"https://doi.org/10.1177/0193841X251336839","url":null,"abstract":"<p><p>Along with discussing bibliometric analyses' limitations and potential biases, this paper addresses the growing need for comprehensive guidelines in evaluating bibliometric research by providing systematic frameworks for both peer reviewers and readers. While numerous publications provide guidance on implementing bibliometric methods, there is a notable lack of frameworks for assessing such research, particularly regarding performance analysis and science mapping. Drawing from an extensive review of bibliometric practices and methodological literature, this paper develops structured evaluation frameworks that address the complexity of modern bibliometric analysis, introducing the VALOR framework (Verification, Alignment, Logging, Overview, Reproducibility) for assessing multi-source bibliometric studies. The paper's key contributions include comprehensive guidelines for evaluating data selection, cleaning, and analysis processes; specific criteria for assessing conceptual, intellectual, and social structure analyses; and practical guidance for integrating performance analysis with science mapping results. By providing structured frameworks for reviewers and practical guidelines for readers to interpret and apply bibliometric insights, this work enhances the rigor of bibliometric research evaluation while supporting more effective peer review processes and research planning. The paper also discusses potential areas for further development, including the integration of qualitative analysis with bibliometric data and the advancement of field-normalized metrics, ultimately aiming to support authors, reviewers, and readers in navigating the complexities of bibliometrics and enhancing the meaningfulness of bibliometric research.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251336839"},"PeriodicalIF":3.0,"publicationDate":"2025-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144054338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Optimizing an Omnichannel Retail Strategy Considering Customer Segmentation. 考虑顾客细分的全渠道零售策略优化。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-11 DOI: 10.1177/0193841X251328710
Shuangpeng Yang, Li Zhang
{"title":"Optimizing an Omnichannel Retail Strategy Considering Customer Segmentation.","authors":"Shuangpeng Yang, Li Zhang","doi":"10.1177/0193841X251328710","DOIUrl":"https://doi.org/10.1177/0193841X251328710","url":null,"abstract":"<p><p>Unlike previous studies on fixed logistics nodes, this research explored how consumer distribution impacts store selection and inventory balance, integrating the <i>ship-from-store</i> strategy to increase fulfillment within multiperiod sales plans. Specifically, omnichannel retailers (O-tailer) must sequentially decide on inventory replenishment from suppliers to the distribution center (DC), allocation from the DC to stores, and which department will fulfill online orders. We introduce a multiperiod stochastic optimization model and solve it with a robust two-stage approach (RTA). In Stage 1, we use the K-means algorithm and silhouette coefficients to determine the optimal number of stores. In Stage 2, linear decision rule (LDR) are employed to decide on replenishment, allocation, and order fulfillment quantities. Numerical experiments show that RTA outperforms existing methods, achieving solutions with efficiency gaps of less than 10%, even when assumptions are not fully met. Additionally, the sensitivity analysis shows that variations in product prices, fulfillment costs, market share, and customer distribution consistently lead to greater profits with the <i>ship-from-store</i> strategy.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251328710"},"PeriodicalIF":3.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143991817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Critical Reflection of Generalization in Mixed Methods Research. 混合方法研究中泛化问题的批判性反思。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-04 DOI: 10.1177/0193841X251331723
Quan Nha Hong, Sergi Fàbregues
{"title":"A Critical Reflection of Generalization in Mixed Methods Research.","authors":"Quan Nha Hong, Sergi Fàbregues","doi":"10.1177/0193841X251331723","DOIUrl":"https://doi.org/10.1177/0193841X251331723","url":null,"abstract":"<p><p>Mixed methods research, that is, research that integrates qualitative and quantitative methods, has become increasingly popular in program evaluation because of its potential for understanding complex interventions. Despite recent constructive and fruitful developments that have led to the consolidation of mixed methods as a distinctive methodology, fundamental methodological issues such as generalization have received little attention. The purpose of this paper is to provide a critical reflection on how the concept of generalization has been used in mixed methods research. The paper is structured into four main parts. First, we discuss the relevance of external validity and mixed methods research in impact evaluation. Second, we summarize how generalization is conceptualized in mixed methods research. Third, we present the results of a literature review on generalization practices in mixed methods research. Finally, we conclude with a discussion of threats to and strategies for enhancing generalization in mixed methods research.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251331723"},"PeriodicalIF":3.0,"publicationDate":"2025-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143784589","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluating the Effectiveness of Maternal, Neonatal, and Child Healthcare in Moroccan Hospitals and SDG 3: Using Two-Stage Data Envelopment Analysis and Tobit Regression. 评估摩洛哥医院中孕产妇、新生儿和儿童保健的有效性与可持续发展目标 3:使用两阶段数据包络分析和托比特回归。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-01 Epub Date: 2024-07-20 DOI: 10.1177/0193841X241264863
Youssef Er-Rays, Meriem M'dioud
{"title":"Evaluating the Effectiveness of Maternal, Neonatal, and Child Healthcare in Moroccan Hospitals and SDG 3: Using Two-Stage Data Envelopment Analysis and Tobit Regression.","authors":"Youssef Er-Rays, Meriem M'dioud","doi":"10.1177/0193841X241264863","DOIUrl":"10.1177/0193841X241264863","url":null,"abstract":"<p><p>Maternal, neonatal, and child health play crucial roles in achieving the objectives of Sustainable Development Goal (SDG) 2030, particularly in promoting health and wellbeing. However, maternal, neonatal, and child services in Moroccan public hospitals face challenges, particularly concerning mortality rates and inefficient resource allocation, which hinder optimal outcomes. This study aimed to evaluate the operational effectiveness of 76 neonatal and child health services networks (MNCSN) within Moroccan public hospitals. Using Data Envelopment Analysis (DEA), we assessed technical efficiency (TE) employing both Variable Returns to Scale for inputs (VRS-I) and outputs (VRS-O) orientation. Additionally, the Tobit method (TM) was utilized to explore factors influencing inefficiency, with hospital, doctor, and paramedical staff considered as inputs, and admissions, cesarean interventions, functional capacity, and hospitalization days as outputs. Our findings revealed that VRS-I exhibited a higher average TE score of 0.76 compared to VRS-O (0.23). Notably, the Casablanca-Anfa MNCSN received the highest referrals (30) under VRS-I, followed by the Khemisset MNCSN (24). In contrast, under VRS-O, Ben Msick, Rabat, and Mediouna MNCSN each had three peers, with 71, 22, and 17 references, respectively. Moreover, the average Malmquist Index under VRS-I indicated a 7.7% increase in productivity over the 9-year study period, while under VRS-O, the average Malmquist Index decreased by 8.7%. Furthermore, doctors and functional bed capacity received the highest Tobit model score of 0.01, followed by hospitalization days and cesarean sections. This study underscores the imperative for policymakers to strategically prioritize input factors to enhance efficiency and ensure optimal maternal, neonatal, and child healthcare outcomes.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"343-379"},"PeriodicalIF":3.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141731503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The Ripple Effect of Managerial Behavior: Exploring Post-experimental Impact of Leading by Example on Small Firms' Cooperation and Performance. 管理行为的涟漪效应:探索 "以身作则 "对小型企业合作与绩效的实验后影响。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-01 Epub Date: 2024-06-08 DOI: 10.1177/0193841X241260466
Quang Nguyen, Huong Trang Kim
{"title":"The Ripple Effect of Managerial Behavior: Exploring Post-experimental Impact of Leading by Example on Small Firms' Cooperation and Performance.","authors":"Quang Nguyen, Huong Trang Kim","doi":"10.1177/0193841X241260466","DOIUrl":"10.1177/0193841X241260466","url":null,"abstract":"<p><p>Cooperation between employees in a company is an important input to firm performance. This study examines how a manager's cooperative behavior and the visibility of this behavior affect the cooperation amongst employees, and subsequently firm performance. To do so, we conducted a field experiment with managers and their employees from 320 Vietnamese small and micro firms to determine the impact of a manager's leading by example (LBE) on employees' behavior, corporate culture, and firm performance. Both managers and employees participated in a Public Good experiment which aimed to elicit an individual cooperative behavior. Noteworthy is that the decision made by a manager in the experiment was given as an example to employees before they made decision in that same experiment. We considered that the example of cooperation by managers in the Public Good experiment communicated a powerful signal to the employees regarding the importance of fostering cooperation in the workplace. Such a signal by the manager, who is at the top in the organizational hierarchy, would impact their employees' behavior in the workplace and firm's outcomes beyond the experiment. Interestingly, we found that concealing a manager's identity from their employees enhances the impacts of LBE.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"270-303"},"PeriodicalIF":3.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141293836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Funding Innovation and Risk: A Grey-Based Startup Investment Decision. 资助创新与风险:基于灰色的初创企业投资决策。
IF 3 4区 社会学
Evaluation Review Pub Date : 2025-04-01 Epub Date: 2024-07-24 DOI: 10.1177/0193841X241262887
Manoj Kumar Srivastava, Ashutosh Dash, Imlak Shaikh
{"title":"Funding Innovation and Risk: A Grey-Based Startup Investment Decision.","authors":"Manoj Kumar Srivastava, Ashutosh Dash, Imlak Shaikh","doi":"10.1177/0193841X241262887","DOIUrl":"10.1177/0193841X241262887","url":null,"abstract":"<p><p>As found in behavioral decision theory, venture capitalists (VCs) rely on heuristics and bias, owing to their bounded rationality, either by limited alternatives or information and resources. India's booming startup scene challenges VCs in decision-making owing to information overload from numerous evolving ventures, which hinders informed judgment. VC investment behavior, due diligence, and cognitive factors related to decision-making have always drawn the attention of researchers. We provide an alternative approach for an optimal decision by VCs by identifying the attributes that influence investment or funding decisions at an early stage of a venture in tech-based industries. Through a literature review, we identify eight attributes, both on internal and external criteria, that venture investors consider when making investment decisions. Based on interviews with 20 experts, we further identify eight key tech-based sectors. Using grey system theory, we then determine the rankings of eight tech startups for investors' early-stage investment decisions. This study presents a linguistic variable-based approach of grey numbers to decide weights and ratings, the grey possibility degree to compare and rank different tech startups, and based on the results, suggests the ideal tech startup. We find that agritech ranks first; thus, investors should prefer venturing into such startups for early-stage investment. E-commerce and edutech ranked second and third, respectively, followed by electric vehicle infrastructure, insurtech, fintech, space tech, and software as a service.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"304-342"},"PeriodicalIF":3.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141761661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信