Evaluation Review最新文献

{"title":"Evaluating the Stroke Risk of Patients using Machine Learning: A New Perspective from Sichuan and Chongqing.","authors":"Jin Zheng, Yao Xiong, Yimei Zheng, Haitao Zhang, Rui Wu","doi":"10.1177/0193841X231193468","DOIUrl":"10.1177/0193841X231193468","url":null,"abstract":"<p><p>Stroke is the leading cause of death and disability among people in China, and it leads to heavy burdens for patients, their families and society. An accurate prediction of the risk of stroke has important implications for early intervention and treatment. In light of recent advances in machine learning, the application of this technique in stroke prediction has achieved plentiful promising results. To detect the relationship between potential factors and the risk of stroke and examine which machine learning method significantly can enhance the prediction accuracy of stroke. We employed six machine learning methods including logistic regression, naive Bayes, decision tree, random forest, K-nearest neighbor and support vector machine, to model and predict the risk of stroke. Participants were 233 patients from Sichuan and Chongqing. Four indicators (accuracy, precision, recall and F1 metric) were examined to evaluate the predictive performance of the different models. The empirical results indicate that random forest yields the best accuracy, recall and F1 in predicting the risk of stroke, with an accuracy of .7548, precision of .7805, recall of .7619 and F1 of .7711. Additionally, the findings show that age, cerebral infarction, PM 8 (an anti-atrial fibrillation drug), and drinking are independent risk factors for stroke. Further studies should adopt a broader assortment of machine learning methods to analyze the risk of stroke, by which better accuracy can be expected. In particular, RF can successfully enhance the forecasting accuracy for stroke.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"346-369"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10302426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using Ecometric Data to Explore Sources of Cross-Site Impact Variance in Multi-Site Trials.","authors":"David R Judkins, Gabriel Durham","doi":"10.1177/0193841X231175549","DOIUrl":"10.1177/0193841X231175549","url":null,"abstract":"<p><p>In 2003, Bloom, Hill, and Riccio (BHR) published an influential paper introducing novel methods for explaining the variation in local impacts observed in multi-site randomized control trials of socio-economic interventions in terms of site-level mediators. This paper seeks to improve upon this previous work by using student-level data to measure site-level mediators and confounders. Development of asymptotic behavior backed up with simulations and an empirical example. Students and training providers. Two simulations and an empirical application to data from an evaluation of the Health Professions Opportunity Grants (HPOG) Program. This empirical analysis involved roughly 6600 participants across 37 local sites. We examine bias and mean square error of estimates of mediation coefficients as well as the true coverage of nominal 95-percent confidence intervals on the mediation coefficients. Simulations suggest that the new methods generally improve the quality of inferences even when there is no confounding. Applying this methodology to the HPOG study shows that program-average FTE months of study by month six was a significant mediator of both career progress and long-term degree/credential receipt. Evaluators can robustify their BHR-style analyses by the use of the methods proposed here.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"274-311"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9969498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Novel Indicator to Correct for Individual Reported Heterogeneity. An Application to Self-Evaluation of Later-Life Depression.","authors":"Serena Berretta, Sara Garbin, Maria Iannario, Omar Paccagnella","doi":"10.1177/0193841X231171965","DOIUrl":"10.1177/0193841X231171965","url":null,"abstract":"<p><p>Program evaluations often investigate complex or multi-dimensional constructs, such as individual opinions or attitudes, by means of ratings. A different interpretation of the same question may affect cross-country comparability, leading to the Differential Item Functioning problem. Anchoring vignettes were introduced in the literature as a way to adjust self-evaluations from this interpersonal incomparability. In this paper, we first introduce a new nonparametric solution to analyse anchoring vignette data, recoding a variable based on a rating scale to a <i>new corrected-</i>variable that guarantees comparability in any cross-country analysis. Then, we exploit the flexibility of a mixture model introduced to account for uncertainty in the response process (the CUP model) to test if the proposed solution is effectively able to remove this reported heterogeneity. This solution is easy to construct and has important advantages compared with the original nonparametric solution adopted with anchoring vignette data. The novel indicator is applied to investigate self-reported depression in an old population. Data that will be analysed come from the second wave of the Survey of Health, Ageing and Retirement in Europe, collected in 2006/2007. Results highlight the need of correcting for reported heterogeneity comparing individual self-evaluations. Once interpersonal incomparability resulting from the different uses of response scales is removed from the self-assessments, some estimates are reversed in magnitude and signs with respect to the analysis of the collected data.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"221-250"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9433536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Impact of Economic Corridor and Tourism on Local Community's Quality of Life under One Belt One Road Context.","authors":"Shiying Wang, Jaffar Abbas, Khalid Ibrahim Al-Sulati, Syed Ale Raza Shah","doi":"10.1177/0193841X231182749","DOIUrl":"10.1177/0193841X231182749","url":null,"abstract":"<p><p>Economic corridors unlock new economic opportunities and tourism development in the region to achieve sustainable development goals. Green economic growth is conducive to environmental sustainability. Economic mega-projects of CPEC promote tourism that leads to communities' well-being and better quality of life. Modern infrastructure development contributes significantly to economic growth and tourism activities. This study's objectives emphasize exploring tourism and sustainable development pursuits under OBOR economic projects that open doors to improving residents' quality of life. The growing world is an eyewitness to a continuous rise in emissions and its severe consequences for humankind. It is necessary to show off the leading factors that result in tourism and economic activities causing environmental pollution rather than blame policymakers. Undoubtedly, many studies previously focused on demonstrating the influence of socio-economic factors that lead to better environmental quality. However, the empirical literature on tourism, social well-being, foreign direct investment, and the Environment in Belt and Road developed economies needed improvement. This research applied a series of advanced estimators that help demonstrate the study's probable results. This study explores the role of Social well-being (HDI), tourism development, FDI, renewable energy, information & communication technology (ICT), and urbanization on CO2 emissions in Belt and Road (BRI) developed economies.Estimated results exhibited the significant contribution of ICT and renewable energy to sustainability. Besides, FDI contributes to emissions reduction after its threshold level. Conversely, urbanization and tourism activities contribute to environmental pollution. The study outcomes stated inverted/EKC U-shaped hypotheses related to specified economies. Finally, the analysis based on the D-H panel causality test constructs exciting results.The present study concludes that economic corridor plays a vital role in tourism development, the community's well-being, and SDGs goals (sustainable development) impact on environmental safety. The findings suggest essential and applicable policies to attain the desired sustainability level. Findings contribute to the literature on tourism, well-being, and sustainability. Further studies can use insights using this methodology.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"312-345"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10050838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Education Experiments in Latin America: Empirical Evidence to Guide Evaluation Design.","authors":"Steven Glazerman, Larissa Campuzano, Nancy Murray","doi":"10.1177/0193841X241241354","DOIUrl":"https://doi.org/10.1177/0193841X241241354","url":null,"abstract":"<p><p>Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also helpful to anticipate the benefits, in terms of statistical power, of collecting household data, testing students at baseline, or relying on administrative data on previous cohorts from the same school. We use data from multiple cluster-randomized trials in four Latin American countries to provide information on the intraclass correlations in early grade literacy outcomes. We also describe the proportion of variance explained by different types of covariates. These parameters will help future researchers conduct statistical power analysis, estimate the required sample size, and determine the necessity of collecting different types of baseline data such as child assessments, administrative data at the school level, or household surveys.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241241354"},"PeriodicalIF":0.9,"publicationDate":"2024-03-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140327263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Estimating a Theoretically Consistent Human Capital Production Function With an Application to Head Start.","authors":"Luis Faundez, Robert Kaestner","doi":"10.1177/0193841X241239512","DOIUrl":"https://doi.org/10.1177/0193841X241239512","url":null,"abstract":"<p><p>This article describes a conceptual and empirical approach for estimating a human capital production function of child development that incorporates mother- or child-fixed effects. The use of mother- or child-fixed effects is common in this applied economics literature, but its application is often inconsistent with human capital theory. We outline the problem and demonstrate its empirical importance with an analysis of the effect of Head Start and preschool on child and adult outcomes. The empirical specification we develop has broad implications for a variety of applied microeconomic analyses beyond our specific application. Results of our analysis indicate that attending Head Start or preschool had no economically or statistically significant effect on child or adult outcomes.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241239512"},"PeriodicalIF":0.9,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140177092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Introducing a Competency Framework for Educational Researchers: The Case of Vietnam.","authors":"Thuan Van Pham, Loc My Thi Nguyen, Trung Tran, Hoang Yen Thi Duong, Hoan Huu Tran, Thanh Thi Nghiem","doi":"10.1177/0193841X241240639","DOIUrl":"https://doi.org/10.1177/0193841X241240639","url":null,"abstract":"<p><p>In this paper, based on the established Vitae Researcher Development Framework (RDF), we introduce a new framework, tailor-made specifically for Vietnamese educational researchers, namely, Vietnam's Framework for Educational Researchers (VFER). VFER is expected to serve as a tool for Vietnamese educational researchers to self-evaluate their skills and support them in developing their career qualities from junior to senior career ladders. The framework includes four domains with ten subdomains and 28 indicators. To date, VFER has been implemented in some Vietnamese universities of pedagogical education. We expect that other research fields will look to VFER as a reference to build their own research capacity framework.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241240639"},"PeriodicalIF":0.9,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140177093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Program Evaluation’s Path to Greater Policy Relevance: Learning From Rossi’s Iron Laws","authors":"Douglas J. Besharov","doi":"10.1177/0193841x241238031","DOIUrl":"https://doi.org/10.1177/0193841x241238031","url":null,"abstract":"In a 1987 article, Peter R. Rossi promulgated “The Iron Law of Evaluation and Other Metallic Rules.” The Metallic Laws were meant as an informal (and humorous) overstatement of the weakness of contemporary evaluations of social programs. Rossi’ s underlying worry was not so much about the state of evaluation technology in the abstract, but, rather, in its inability to advance our broad understanding of social problems and what to do about them---in other words, to make evaluation policy relevant. Rossi attributed the continuing failure to develop successful “large-scale social programs” to the failure to build a strong knowledge base for this kind of “social engineering.” The qualities of studies that enable such accumulated learning are variously labeled “external validity,” “generalizability,” “applicability,” or “transferability.” This Special Issue includes five papers that seek to explore and apply this understanding.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":"12 1","pages":""},"PeriodicalIF":0.9,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140036990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Studying Parole Revocation Practices: Accounting for Dependency Between Competing Events.","authors":"William Rhodes, Gerald Gaes, William Sabol","doi":"10.1177/0193841X241234412","DOIUrl":"https://doi.org/10.1177/0193841X241234412","url":null,"abstract":"<p><p>When individuals are released from prison, they typically enter a period of post confinement community supervision. While under community supervision, their behaviors are subject to special conditions requiring them to report to supervisors and prohibiting certain behaviors such as drug and alcohol use. Many supervisees are returned to prison because they violate those special conditions, or because they commit minor crimes that would not result in prison were they not being supervised. But others are returned to prison for serious new crimes. We distinguish the two as nuisance behaviors (the former) and pernicious behaviors (the latter). Our research applies competing events survival analysis to distinguish a structural model that accounts for nuisance behaviors from a structural model that accounts for pernicious behaviors. We demonstrate that returning offenders to prison for technical violations and minor crimes may reduce the incidence of major crimes because the occurrence of nuisance behaviors and pernicious behaviors are highly correlated. Our findings support the theory that nuisance behaviors signal the likelihood of pernicious behaviors.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241234412"},"PeriodicalIF":0.9,"publicationDate":"2024-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139944549","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Alternative Factors Leading to Replication Crisis: Prediction and Evaluation.","authors":"Gregory Chernov","doi":"10.1177/0193841X241229106","DOIUrl":"https://doi.org/10.1177/0193841X241229106","url":null,"abstract":"<p><p>Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most qualified expert. This mechanism can be seamlessly integrated into the existing replication prediction market framework with minimal implementation costs. It relies on an objective rather than subjective process and unstructured expert opinions to effectively identify various influences contributing to the replication crisis.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"193841X241229106"},"PeriodicalIF":0.9,"publicationDate":"2024-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139913696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}