Evaluation ReviewPub Date : 2025-04-26DOI: 10.1177/0193841X251338536
Sigrún Eyrúnardóttir Clark, Norha Vera San Juan, Thomas Moniz, Rebecca Appleton, Phoebe Barnett, Cecilia Vindrola-Padros
{"title":"A Systematic Review of the Methods Used in Rapid Approaches to Research and Evaluation.","authors":"Sigrún Eyrúnardóttir Clark, Norha Vera San Juan, Thomas Moniz, Rebecca Appleton, Phoebe Barnett, Cecilia Vindrola-Padros","doi":"10.1177/0193841X251338536","DOIUrl":"https://doi.org/10.1177/0193841X251338536","url":null,"abstract":"<p><p>Rapid approaches are essential when resources are limited and when findings are required in real-time to inform decisions. Limitations exist in their design and implementation, which can lead to a reduced level of trust in findings. This review sought to map the methods used across rapid evaluations and research to facilitate timeliness and support the rigour of studies. Four scientific databases and one search engine were searched between 11-16th August 2022. Screening led to the inclusion of 169 articles that provided a much-needed repository of methods that can be used during the design and implementation of rapid studies to improve their trustworthiness. No reporting guidelines specific to rapid research or evaluation were identified in the literature, we therefore suggest that this repository of methods informs the development of transparent reporting standards for future rapid research and evaluation.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251338536"},"PeriodicalIF":3.0,"publicationDate":"2025-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144040982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-04-18DOI: 10.1177/0193841X251336839
Anh-Duc Hoang
{"title":"Evaluating Bibliometrics Reviews: A Practical Guide for Peer Review and Critical Reading.","authors":"Anh-Duc Hoang","doi":"10.1177/0193841X251336839","DOIUrl":"https://doi.org/10.1177/0193841X251336839","url":null,"abstract":"<p><p>Along with discussing bibliometric analyses' limitations and potential biases, this paper addresses the growing need for comprehensive guidelines in evaluating bibliometric research by providing systematic frameworks for both peer reviewers and readers. While numerous publications provide guidance on implementing bibliometric methods, there is a notable lack of frameworks for assessing such research, particularly regarding performance analysis and science mapping. Drawing from an extensive review of bibliometric practices and methodological literature, this paper develops structured evaluation frameworks that address the complexity of modern bibliometric analysis, introducing the VALOR framework (Verification, Alignment, Logging, Overview, Reproducibility) for assessing multi-source bibliometric studies. The paper's key contributions include comprehensive guidelines for evaluating data selection, cleaning, and analysis processes; specific criteria for assessing conceptual, intellectual, and social structure analyses; and practical guidance for integrating performance analysis with science mapping results. By providing structured frameworks for reviewers and practical guidelines for readers to interpret and apply bibliometric insights, this work enhances the rigor of bibliometric research evaluation while supporting more effective peer review processes and research planning. The paper also discusses potential areas for further development, including the integration of qualitative analysis with bibliometric data and the advancement of field-normalized metrics, ultimately aiming to support authors, reviewers, and readers in navigating the complexities of bibliometrics and enhancing the meaningfulness of bibliometric research.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251336839"},"PeriodicalIF":3.0,"publicationDate":"2025-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144054338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-04-11DOI: 10.1177/0193841X251328710
Shuangpeng Yang, Li Zhang
{"title":"Optimizing an Omnichannel Retail Strategy Considering Customer Segmentation.","authors":"Shuangpeng Yang, Li Zhang","doi":"10.1177/0193841X251328710","DOIUrl":"https://doi.org/10.1177/0193841X251328710","url":null,"abstract":"<p><p>Unlike previous studies on fixed logistics nodes, this research explored how consumer distribution impacts store selection and inventory balance, integrating the <i>ship-from-store</i> strategy to increase fulfillment within multiperiod sales plans. Specifically, omnichannel retailers (O-tailer) must sequentially decide on inventory replenishment from suppliers to the distribution center (DC), allocation from the DC to stores, and which department will fulfill online orders. We introduce a multiperiod stochastic optimization model and solve it with a robust two-stage approach (RTA). In Stage 1, we use the K-means algorithm and silhouette coefficients to determine the optimal number of stores. In Stage 2, linear decision rule (LDR) are employed to decide on replenishment, allocation, and order fulfillment quantities. Numerical experiments show that RTA outperforms existing methods, achieving solutions with efficiency gaps of less than 10%, even when assumptions are not fully met. Additionally, the sensitivity analysis shows that variations in product prices, fulfillment costs, market share, and customer distribution consistently lead to greater profits with the <i>ship-from-store</i> strategy.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251328710"},"PeriodicalIF":3.0,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143991817","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-04-04DOI: 10.1177/0193841X251331723
Quan Nha Hong, Sergi Fàbregues
{"title":"A Critical Reflection of Generalization in Mixed Methods Research.","authors":"Quan Nha Hong, Sergi Fàbregues","doi":"10.1177/0193841X251331723","DOIUrl":"https://doi.org/10.1177/0193841X251331723","url":null,"abstract":"<p><p>Mixed methods research, that is, research that integrates qualitative and quantitative methods, has become increasingly popular in program evaluation because of its potential for understanding complex interventions. Despite recent constructive and fruitful developments that have led to the consolidation of mixed methods as a distinctive methodology, fundamental methodological issues such as generalization have received little attention. The purpose of this paper is to provide a critical reflection on how the concept of generalization has been used in mixed methods research. The paper is structured into four main parts. First, we discuss the relevance of external validity and mixed methods research in impact evaluation. Second, we summarize how generalization is conceptualized in mixed methods research. Third, we present the results of a literature review on generalization practices in mixed methods research. Finally, we conclude with a discussion of threats to and strategies for enhancing generalization in mixed methods research.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251331723"},"PeriodicalIF":3.0,"publicationDate":"2025-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143784589","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-04-01Epub Date: 2024-07-20DOI: 10.1177/0193841X241264863
Youssef Er-Rays, Meriem M'dioud
{"title":"Evaluating the Effectiveness of Maternal, Neonatal, and Child Healthcare in Moroccan Hospitals and SDG 3: Using Two-Stage Data Envelopment Analysis and Tobit Regression.","authors":"Youssef Er-Rays, Meriem M'dioud","doi":"10.1177/0193841X241264863","DOIUrl":"10.1177/0193841X241264863","url":null,"abstract":"<p><p>Maternal, neonatal, and child health play crucial roles in achieving the objectives of Sustainable Development Goal (SDG) 2030, particularly in promoting health and wellbeing. However, maternal, neonatal, and child services in Moroccan public hospitals face challenges, particularly concerning mortality rates and inefficient resource allocation, which hinder optimal outcomes. This study aimed to evaluate the operational effectiveness of 76 neonatal and child health services networks (MNCSN) within Moroccan public hospitals. Using Data Envelopment Analysis (DEA), we assessed technical efficiency (TE) employing both Variable Returns to Scale for inputs (VRS-I) and outputs (VRS-O) orientation. Additionally, the Tobit method (TM) was utilized to explore factors influencing inefficiency, with hospital, doctor, and paramedical staff considered as inputs, and admissions, cesarean interventions, functional capacity, and hospitalization days as outputs. Our findings revealed that VRS-I exhibited a higher average TE score of 0.76 compared to VRS-O (0.23). Notably, the Casablanca-Anfa MNCSN received the highest referrals (30) under VRS-I, followed by the Khemisset MNCSN (24). In contrast, under VRS-O, Ben Msick, Rabat, and Mediouna MNCSN each had three peers, with 71, 22, and 17 references, respectively. Moreover, the average Malmquist Index under VRS-I indicated a 7.7% increase in productivity over the 9-year study period, while under VRS-O, the average Malmquist Index decreased by 8.7%. Furthermore, doctors and functional bed capacity received the highest Tobit model score of 0.01, followed by hospitalization days and cesarean sections. This study underscores the imperative for policymakers to strategically prioritize input factors to enhance efficiency and ensure optimal maternal, neonatal, and child healthcare outcomes.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"343-379"},"PeriodicalIF":3.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141731503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-04-01Epub Date: 2024-06-08DOI: 10.1177/0193841X241260466
Quang Nguyen, Huong Trang Kim
{"title":"The Ripple Effect of Managerial Behavior: Exploring Post-experimental Impact of Leading by Example on Small Firms' Cooperation and Performance.","authors":"Quang Nguyen, Huong Trang Kim","doi":"10.1177/0193841X241260466","DOIUrl":"10.1177/0193841X241260466","url":null,"abstract":"<p><p>Cooperation between employees in a company is an important input to firm performance. This study examines how a manager's cooperative behavior and the visibility of this behavior affect the cooperation amongst employees, and subsequently firm performance. To do so, we conducted a field experiment with managers and their employees from 320 Vietnamese small and micro firms to determine the impact of a manager's leading by example (LBE) on employees' behavior, corporate culture, and firm performance. Both managers and employees participated in a Public Good experiment which aimed to elicit an individual cooperative behavior. Noteworthy is that the decision made by a manager in the experiment was given as an example to employees before they made decision in that same experiment. We considered that the example of cooperation by managers in the Public Good experiment communicated a powerful signal to the employees regarding the importance of fostering cooperation in the workplace. Such a signal by the manager, who is at the top in the organizational hierarchy, would impact their employees' behavior in the workplace and firm's outcomes beyond the experiment. Interestingly, we found that concealing a manager's identity from their employees enhances the impacts of LBE.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"270-303"},"PeriodicalIF":3.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141293836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Funding Innovation and Risk: A Grey-Based Startup Investment Decision.","authors":"Manoj Kumar Srivastava, Ashutosh Dash, Imlak Shaikh","doi":"10.1177/0193841X241262887","DOIUrl":"10.1177/0193841X241262887","url":null,"abstract":"<p><p>As found in behavioral decision theory, venture capitalists (VCs) rely on heuristics and bias, owing to their bounded rationality, either by limited alternatives or information and resources. India's booming startup scene challenges VCs in decision-making owing to information overload from numerous evolving ventures, which hinders informed judgment. VC investment behavior, due diligence, and cognitive factors related to decision-making have always drawn the attention of researchers. We provide an alternative approach for an optimal decision by VCs by identifying the attributes that influence investment or funding decisions at an early stage of a venture in tech-based industries. Through a literature review, we identify eight attributes, both on internal and external criteria, that venture investors consider when making investment decisions. Based on interviews with 20 experts, we further identify eight key tech-based sectors. Using grey system theory, we then determine the rankings of eight tech startups for investors' early-stage investment decisions. This study presents a linguistic variable-based approach of grey numbers to decide weights and ratings, the grey possibility degree to compare and rank different tech startups, and based on the results, suggests the ideal tech startup. We find that agritech ranks first; thus, investors should prefer venturing into such startups for early-stage investment. E-commerce and edutech ranked second and third, respectively, followed by electric vehicle infrastructure, insurtech, fintech, space tech, and software as a service.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"304-342"},"PeriodicalIF":3.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141761661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-03-24DOI: 10.1177/0193841X251329459
Qiqi Cheng, Neil Humphrey
{"title":"Preliminary Evaluation of a Targeted, School-Based Social and Emotional Learning Intervention for at Risk Youth: Football Beyond Borders.","authors":"Qiqi Cheng, Neil Humphrey","doi":"10.1177/0193841X251329459","DOIUrl":"https://doi.org/10.1177/0193841X251329459","url":null,"abstract":"<p><p>This brief report provides preliminary independent evidence of the efficacy of Football Beyond Borders (FBB), a targeted, school-based social and emotional learning (SEL) intervention for at-risk youth. FBB includes weekly SEL classroom sessions, activities on the football pitch, 1:1 therapy sessions, holiday support, and rewards trips. Propensity score matching and difference-in-differences estimation were used in a pre-test/post-test control group design to assess the impact of FBB on the mental wellbeing (assessed via the Short Warwick-Edinburgh Mental Wellbeing Scale, SWEMWBS) of participants designated at-risk (<i>N</i> = 46 aged 12-14, 78.3% male), passive learners (<i>N</i> = 72, aged 12-14, 84.7% male), and role models (<i>N</i> = 35, aged 12-14, 85.7% male), with matched control samples derived from a subset of the #BeeWell cohort study (<i>N</i> = 8015). A significant intervention effect was observed for at-risk youth, with FBB leading to an increase of approximately 2.4 SWEMWBS points (<i>d</i> = 0.44). No significant intervention effects were observed for passive learners or role models. These results indicate that FBB can improve the mental wellbeing of at-risk youth. Accordingly, an explanatory trial is warranted.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251329459"},"PeriodicalIF":3.0,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143694098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-03-14DOI: 10.1177/0193841X251324594
Eugene Bardach
{"title":"Dubious Evidence, Valuable Information.","authors":"Eugene Bardach","doi":"10.1177/0193841X251324594","DOIUrl":"https://doi.org/10.1177/0193841X251324594","url":null,"abstract":"<p><p>I explore how \"low-quality evidence\" from program performance might still be useful in decision-making. Conceptually, a local government named \"Here\" is motivated to consider a program from \"Elsewhere\" that seems to show year-over-year exemplary performance. Here must manage five sources of uncertainty about whether and how to extrapolate from Elsewhere: chance in assessing Elsewhere's performance; illusion due to confounding variables; estimating the several powers of the program's components; substitutions in the design process made by Elsewhere and contemplated by Here; and estimating whether in the final analysis Here can meet its own breakeven criterion for going ahead. Here can begin with Elsewhere's experience, but it still must do much thinking and information-collecting on its own.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251324594"},"PeriodicalIF":3.0,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143630936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-03-12DOI: 10.1177/0193841X251326738
Thomas S Dee
{"title":"The Case for Preregistering Quasi-Experimental Program and Policy Evaluations.","authors":"Thomas S Dee","doi":"10.1177/0193841X251326738","DOIUrl":"https://doi.org/10.1177/0193841X251326738","url":null,"abstract":"<p><p>The recognition that researcher discretion coupled with unconscious biases and motivated reasoning sometimes leads to false findings (\"p-hacking\") led to the broad embrace of study preregistration and other open-science practices in experimental research. Paradoxically, the preregistration of quasi-experimental studies remains uncommon although such studies involve far more discretionary decisions and are the most prevalent approach to making causal claims in the social sciences. I discuss several forms of recent empirical evidence indicating that questionable research practices contribute to the comparative unreliability of quasi-experimental research and advocate for adopting the preregistration of such studies. The implementation of this recommendation would benefit from further consideration of key design details (e.g., how to balance data cleaning with credible preregistration) and a shift in research norms to allow for appropriately nuanced sensemaking across prespecified, confirmatory results and other exploratory findings.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X251326738"},"PeriodicalIF":3.0,"publicationDate":"2025-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143605535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}