Evaluation ReviewPub Date : 2025-02-01Epub Date: 2024-02-20DOI: 10.1177/0193841X241229106
Gregory Chernov
{"title":"The Alternative Factors Leading to Replication Crisis: Prediction and Evaluation.","authors":"Gregory Chernov","doi":"10.1177/0193841X241229106","DOIUrl":"10.1177/0193841X241229106","url":null,"abstract":"<p><p>Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most qualified expert. This mechanism can be seamlessly integrated into the existing replication prediction market framework with minimal implementation costs. It relies on an objective rather than subjective process and unstructured expert opinions to effectively identify various influences contributing to the replication crisis.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"147-164"},"PeriodicalIF":3.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139913696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating Sustainable Mobility: Motorized and Non-motorized Modes in Suburban Areas of Thailand.","authors":"Pawinee Iamtrakul, Jirawan Klaylee, I-Soon Raungratanaamporn","doi":"10.1177/0193841X241233669","DOIUrl":"10.1177/0193841X241233669","url":null,"abstract":"<p><p>The impact of sustainable mobility may be considered through the mode of non-motorization, which may vary by spatial environmental conditions, especially accessibility by active mobility in a necessarily safe, comfortable, and pleasant atmosphere. Sustainable mobility comprises willingness to allow commuting behaviors on non-motorized travel for safety and green ethics. These chiefly participate to create livable places, improve health and well-being, and promote quality of life and economic growth. By contrast, an impoverished street environment decreases the possibility of active mobility and relies on motorization. It is essential to explore the perception of road users through diverse environmental features of street networks and sidewalks directly affecting road user behavior and preferences about motorization and non-motorization. This study evaluated multi-indicators of sustainable urban mobility (SUM) for a group of road users, with data gathered by questionnaire-based survey. 1,998 Pathum Thani Province inhabitants in Thailand were asked to evaluate their neighborhood characteristics by SUM indicators. In addition to different social dimensions and socioeconomic backgrounds, transportation characteristics among diverse trip makers and different perceptions of mobility indicators were classified by factor analysis using principal components and spatial visualization. Results were that traffic management, including intersection controls and signage, as well as information, sidewalk, management, and personal mobility choices, is a key strategy to improve sustainable mobility. This is achieved by tailoring solutions to achieve a modal shift prioritized over other supportive facilities.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"36-60"},"PeriodicalIF":3.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139913695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-02-01Epub Date: 2024-02-24DOI: 10.1177/0193841X241234412
William Rhodes, Gerald Gaes, William Sabol
{"title":"Studying Parole Revocation Practices: Accounting for Dependency Between Competing Events.","authors":"William Rhodes, Gerald Gaes, William Sabol","doi":"10.1177/0193841X241234412","DOIUrl":"10.1177/0193841X241234412","url":null,"abstract":"<p><p>When individuals are released from prison, they typically enter a period of post confinement community supervision. While under community supervision, their behaviors are subject to special conditions requiring them to report to supervisors and prohibiting certain behaviors such as drug and alcohol use. Many supervisees are returned to prison because they violate those special conditions, or because they commit minor crimes that would not result in prison were they not being supervised. But others are returned to prison for serious new crimes. We distinguish the two as nuisance behaviors (the former) and pernicious behaviors (the latter). Our research applies competing events survival analysis to distinguish a structural model that accounts for nuisance behaviors from a structural model that accounts for pernicious behaviors. We demonstrate that returning offenders to prison for technical violations and minor crimes may reduce the incidence of major crimes because the occurrence of nuisance behaviors and pernicious behaviors are highly correlated. Our findings support the theory that nuisance behaviors signal the likelihood of pernicious behaviors.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"3-35"},"PeriodicalIF":3.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139944549","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-02-01Epub Date: 2024-03-29DOI: 10.1177/0193841X241241354
Steven Glazerman, Larissa Campuzano, Nancy Murray
{"title":"Education Experiments in Latin America: Empirical Evidence to Guide Evaluation Design.","authors":"Steven Glazerman, Larissa Campuzano, Nancy Murray","doi":"10.1177/0193841X241241354","DOIUrl":"10.1177/0193841X241241354","url":null,"abstract":"<p><p>Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also helpful to anticipate the benefits, in terms of statistical power, of collecting household data, testing students at baseline, or relying on administrative data on previous cohorts from the same school. We use data from multiple cluster-randomized trials in four Latin American countries to provide information on the intraclass correlations in early grade literacy outcomes. We also describe the proportion of variance explained by different types of covariates. These parameters will help future researchers conduct statistical power analysis, estimate the required sample size, and determine the necessity of collecting different types of baseline data such as child assessments, administrative data at the school level, or household surveys.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"115-146"},"PeriodicalIF":3.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140327263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-02-01Epub Date: 2024-03-20DOI: 10.1177/0193841X241239512
Luis Faundez, Robert Kaestner
{"title":"Estimating a Theoretically Consistent Human Capital Production Function With an Application to Head Start.","authors":"Luis Faundez, Robert Kaestner","doi":"10.1177/0193841X241239512","DOIUrl":"10.1177/0193841X241239512","url":null,"abstract":"<p><p>This article describes a conceptual and empirical approach for estimating a human capital production function of child development that incorporates mother- or child-fixed effects. The use of mother- or child-fixed effects is common in this applied economics literature, but its application is often inconsistent with human capital theory. We outline the problem and demonstrate its empirical importance with an analysis of the effect of Head Start and preschool on child and adult outcomes. The empirical specification we develop has broad implications for a variety of applied microeconomic analyses beyond our specific application. Results of our analysis indicate that attending Head Start or preschool had no economically or statistically significant effect on child or adult outcomes.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"61-114"},"PeriodicalIF":3.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140177092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2025-02-01Epub Date: 2024-03-20DOI: 10.1177/0193841X241240639
Thuan Van Pham, Loc My Thi Nguyen, Trung Tran, Hoang Yen Thi Duong, Hoan Huu Tran, Thanh Thi Nghiem
{"title":"Introducing a Competency Framework for Educational Researchers: The Case of Vietnam.","authors":"Thuan Van Pham, Loc My Thi Nguyen, Trung Tran, Hoang Yen Thi Duong, Hoan Huu Tran, Thanh Thi Nghiem","doi":"10.1177/0193841X241240639","DOIUrl":"10.1177/0193841X241240639","url":null,"abstract":"<p><p>In this paper, based on the established Vitae Researcher Development Framework (RDF), we introduce a new framework, tailor-made specifically for Vietnamese educational researchers, namely, Vietnam's Framework for Educational Researchers (VFER). VFER is expected to serve as a tool for Vietnamese educational researchers to self-evaluate their skills and support them in developing their career qualities from junior to senior career ladders. The framework includes four domains with ten subdomains and 28 indicators. To date, VFER has been implemented in some Vietnamese universities of pedagogical education. We expect that other research fields will look to VFER as a reference to build their own research capacity framework.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"165-174"},"PeriodicalIF":3.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140177093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2024-12-19DOI: 10.1177/0193841X241304293
Tansif Ur Rehman
{"title":"Workplace Cyberbullying: Nature, Characteristics, and Implications.","authors":"Tansif Ur Rehman","doi":"10.1177/0193841X241304293","DOIUrl":"https://doi.org/10.1177/0193841X241304293","url":null,"abstract":"<p><p>The information technology revolution has fundamentally altered company operations around the world. The Internet has significantly enhanced employee connectedness in the workplace, eclipsing the antiquated brick-and-mortar model. Nonetheless, as information technology advances, cyberbullying has grown in popularity in the professional environment. Cyberbullying is not geographically, temporally, or milieu-specific. Arguments concerning this ubiquitous danger have plagued scholars and professionals for several decades as they debated its conception, prevalence, and implications. The current research digs into many facets of cyberbullying to facilitate the creation of constructive policies and effectively manage the labor environment. Additionally, a few ideas and remedies are offered, as well as the future course of action for effectively addressing the crucial issue of cyberbullying.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241304293"},"PeriodicalIF":3.0,"publicationDate":"2024-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142856024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2024-12-01Epub Date: 2023-12-14DOI: 10.1177/0193841X231221303
Rebecca Walcott, Isabelle Cohen, Denise Ferris
{"title":"When Who Matters: Interviewer Effects and Survey Modality.","authors":"Rebecca Walcott, Isabelle Cohen, Denise Ferris","doi":"10.1177/0193841X231221303","DOIUrl":"10.1177/0193841X231221303","url":null,"abstract":"<p><p>When and how to survey potential respondents is often determined by budgetary and external constraints, but choice of survey modality may have enormous implications for data quality. Different survey modalities may be differentially susceptible to measurement error attributable to interviewer assignment, known as interviewer effects. In this paper, we leverage highly similar surveys, one conducted face-to-face (FTF) and the other via phone, to examine variation in interviewer effects across survey modality and question type. We find that while there are no cross-modality differences for simple questions, interviewer effects are markedly higher for sensitive questions asked over the phone. These findings are likely explained by the enhanced ability of in-person interviewers to foster rapport and engagement with respondents. We conclude with a thought experiment that illustrates the potential implications for power calculations, namely, that using FTF data to inform phone surveys may substantially underestimate the necessary sample size for sensitive questions.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1024-1049"},"PeriodicalIF":3.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138809521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2024-12-01Epub Date: 2023-08-11DOI: 10.1177/0193841X231194565
Emmanuel Orkoh, Uchenna Efobi
{"title":"Effects of Behaviour Change Communication on Knowledge and Prevention of Malaria Among Women in Ghana.","authors":"Emmanuel Orkoh, Uchenna Efobi","doi":"10.1177/0193841X231194565","DOIUrl":"10.1177/0193841X231194565","url":null,"abstract":"<p><p>Behaviour change communication (BCC) remains a central component of the interventions used in the fight against malaria in Ghana. However, there is limited evidence of its effectiveness. This study evaluated the effects of BCC strategies on knowledge (symptoms, causes and prevention) and overall knowledge of malaria among Ghanaian women aged 15-49 years. The propensity score matching (PSM) approach and logistic regression were used to analyse data from the 2016 edition of the Malaria Indicator Survey (MIS). Women who participated in community-level education or heard/saw media messages on malaria, or both, had significantly more knowledge of the disease than women who lacked access to any of these mediums of communication. The effect of these strategies on women's overall knowledge of malaria is about 2% to 4% and is higher on their knowledge of the symptoms (3% to 6%) and prevention (2% to 4%) than the causes (2%). The combined effects of both mediums of communication are relatively higher than the effect of either of them as a single medium of communication. Further analysis showed that improved knowledge of the disease is associated with higher preventive measures taken by women for themselves and for their children. The results are more significant in rural and poor households than in urban and non-poor households. These findings underscore the need for the Ministry of Health and its partner institutions to adopt an innovative approach which combines the two strategies in intensively educating Ghanaians, and women in particular, on the symptoms and prevention of malaria, giving due cognisance to households' socioeconomic status and geographical location.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1050-1087"},"PeriodicalIF":3.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11447998/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10033653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evaluation ReviewPub Date : 2024-12-01Epub Date: 2023-12-25DOI: 10.1177/0193841X231223374
Eirini M Mitropoulou, Leonidas A Zampetakis, Ioannis Tsaousis
{"title":"Calibrating Items Using an Unfolding Model of Item Response Theory: The Case of the Trait Personality Questionnaire 5 (TPQue5).","authors":"Eirini M Mitropoulou, Leonidas A Zampetakis, Ioannis Tsaousis","doi":"10.1177/0193841X231223374","DOIUrl":"10.1177/0193841X231223374","url":null,"abstract":"<p><p>Unfolding item response theory (IRT) models are important alternatives to dominance IRT models in describing the response processes on self-report tests. Their usage is common in personality measures, since they indicate potential differentiations in test score interpretation. This paper aims to gain a better insight into the structure of trait personality, by investigating whether the dominance or alternatively the unfolding IRT model are better descriptors of the response processes on a personality measure constructed under the dominance response theorem. For the assessment of the dominant model, the Graded Response Model (GRM) is used; while for the unfolding model, the Generalized Graded Unfolding Model (GGUM) was examined. All analyses are conducted with the freely available R. A sample of 1340 Greek adults, employed in private and public organizations, fulfilled the Trait Personality Questionnaire 5 short-form (TPQue5). Findings contradict previous research on trait personality. In accordance to the construction method employed, the TPQue5 items are best understood by monotonically increasing item response functions (IRFs). Individuals responding to the TPQue5 increase their probability of endorsing its items as their trait level increases; this stands for all personality dimensions, although Openness to Experience exhibited mixed type of item response patterns. Further research directions, implications and limitations are also discussed.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"1146-1159"},"PeriodicalIF":3.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139038099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}