Evaluation Review最新文献

{"title":"Conditioning on the Pre-Test versus Gain Score Modelling: Revisiting the Controversy in a Multilevel Setting","authors":"Bruno Arpino, Silvia Bacci, Leonardo Grilli, Raffaele Guetto, Carla Rampichini","doi":"10.1177/0193841x241246833","DOIUrl":"https://doi.org/10.1177/0193841x241246833","url":null,"abstract":"We consider estimating the effect of a treatment on a given outcome measured on subjects tested both before and after treatment assignment in observational studies. A vast literature compares the competing approaches of modelling the post-test score conditionally on the pre-test score versus modelling the difference, namely, the gain score. Our contribution lies in analyzing the merits and drawbacks of two approaches in a multilevel setting. This is relevant in many fields, such as education, where students are nested within schools. The multilevel structure raises peculiar issues related to contextual effects and the distinction between individual-level and cluster-level treatments. We compare the two approaches through a simulation study. For individual-level treatments, our findings align with existing literature. However, for cluster-level treatments, the scenario is more complex, as the cluster mean of the pre-test score plays a key role. Its reliability crucially depends on the cluster size, leading to potentially unsatisfactory estimators with small clusters.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":"2012 1","pages":""},"PeriodicalIF":0.9,"publicationDate":"2024-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140612307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Do Trade and Financial Cooperation Improve Environmentally Sustainable Development: A Distinction Between <i>de facto</i> and <i>de jure</i> Globalization.","authors":"Mehmet Akif Destek, İbrahim Halil Oğuz, Nuh Okumuş","doi":"10.1177/0193841X231181747","DOIUrl":"10.1177/0193841X231181747","url":null,"abstract":"<p><p>The adoption of growth strategies based on foreign trade, especially in the previous century when liberal policies began to dominate, is one of the main reasons for the increase in output and indirectly for environmental concerns. On the other hand, there are complex claims about the environmental effects of liberal policies and thus of globalization. This study intends to analyze the effects of global collaborations involving 11 transition economies that have completed the transition process on the environmentally sustainable development of these nations. In this direction, the effects of financial and commercial globalization indices on carbon emissions are investigated. The distinctions of globalization are used to distinguish the consequences of the two types of globalization. In doing so, the de facto and de jure indicator distinctions of globalization are used to differentiate the consequences of two types of globalization. In addition, the effects of real GDP, energy efficiency, and use of renewable energy on environmental pollution are dissected. For the main purpose of the study, the CS-ARDL estimation technique that allows cross-sectional dependency among observed countries is used to separate the short and long-run influences of explanatory variables. In addition, CCE-MG estimator is used for robustness check. According to the empirical findings, the economic growth and increasing energy intensity increases carbon emissions, but the increase in renewable energy consumption improves environmental quality. Furthermore, trade globalization does not have a significant impact on the environment in the context of globalization. On the other hand, the increase in de facto and de jure financial globalization indices results in an increase in carbon emissions, but de jure financial globalization causes more environmental damage. The harmful impact of de jure financial globalization on environmental quality suggests that the decreasing investment restrictions and international investment agreements of transition countries have been implemented in a manner that facilitates the relocation of investments from pollution-intensive industries to these countries.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"251-273"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9957122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Promoting Pro-Environmental Behavior in Policymaking: A Text-Mining Approach for Literature Review.","authors":"Md Abul Kalam Azad, Mehedi Hasan Ifti, Chowdhury Noushin Novera, Anh Ngoc Quang Huynh, Esra Sipahi Döngül","doi":"10.1177/0193841X231176869","DOIUrl":"10.1177/0193841X231176869","url":null,"abstract":"<p><p>The impact of pro-environmental behavior on policymaking has been an exciting area of research. While the relationship between pro-environmental behavior and policymaking has been explored in numerous studies, there needs to be more synthesis on this topic. This is the first text-mining study of pro-environmental effects in which policymaking is a significant factor. In response, this study, for the first time, takes a novel approach by using text mining in R programming to analyze 30 publications from the Scopus database on pro-environmental behavior in policymaking, highlighting major research themes and prospective research areas for future investigation. Results from text mining yielded 10 topic models, which are presented with a synopsis of the published research and a list of the primary authors, as well as a posterior probability via latent Dirichlet allocation (LDA). Additionally, the study conducts a trend analysis of the top 10 journals with the highest impact factor, considering the influence of each journal's mean citation. The study offers an overview of the impacts of pro-environmental behavior in policymaking, showing the most relevant and frequently discussed themes, introduces the scientific visualization of papers published in the Scopus database, and proposes future study directions. These findings can help researchers and environmental specialists better understand how pro-environmental behavior can be fostered more effectively through policymaking.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"370-398"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9543356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating the Stroke Risk of Patients using Machine Learning: A New Perspective from Sichuan and Chongqing.","authors":"Jin Zheng, Yao Xiong, Yimei Zheng, Haitao Zhang, Rui Wu","doi":"10.1177/0193841X231193468","DOIUrl":"10.1177/0193841X231193468","url":null,"abstract":"<p><p>Stroke is the leading cause of death and disability among people in China, and it leads to heavy burdens for patients, their families and society. An accurate prediction of the risk of stroke has important implications for early intervention and treatment. In light of recent advances in machine learning, the application of this technique in stroke prediction has achieved plentiful promising results. To detect the relationship between potential factors and the risk of stroke and examine which machine learning method significantly can enhance the prediction accuracy of stroke. We employed six machine learning methods including logistic regression, naive Bayes, decision tree, random forest, K-nearest neighbor and support vector machine, to model and predict the risk of stroke. Participants were 233 patients from Sichuan and Chongqing. Four indicators (accuracy, precision, recall and F1 metric) were examined to evaluate the predictive performance of the different models. The empirical results indicate that random forest yields the best accuracy, recall and F1 in predicting the risk of stroke, with an accuracy of .7548, precision of .7805, recall of .7619 and F1 of .7711. Additionally, the findings show that age, cerebral infarction, PM 8 (an anti-atrial fibrillation drug), and drinking are independent risk factors for stroke. Further studies should adopt a broader assortment of machine learning methods to analyze the risk of stroke, by which better accuracy can be expected. In particular, RF can successfully enhance the forecasting accuracy for stroke.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"346-369"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10302426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using Ecometric Data to Explore Sources of Cross-Site Impact Variance in Multi-Site Trials.","authors":"David R Judkins, Gabriel Durham","doi":"10.1177/0193841X231175549","DOIUrl":"10.1177/0193841X231175549","url":null,"abstract":"<p><p>In 2003, Bloom, Hill, and Riccio (BHR) published an influential paper introducing novel methods for explaining the variation in local impacts observed in multi-site randomized control trials of socio-economic interventions in terms of site-level mediators. This paper seeks to improve upon this previous work by using student-level data to measure site-level mediators and confounders. Development of asymptotic behavior backed up with simulations and an empirical example. Students and training providers. Two simulations and an empirical application to data from an evaluation of the Health Professions Opportunity Grants (HPOG) Program. This empirical analysis involved roughly 6600 participants across 37 local sites. We examine bias and mean square error of estimates of mediation coefficients as well as the true coverage of nominal 95-percent confidence intervals on the mediation coefficients. Simulations suggest that the new methods generally improve the quality of inferences even when there is no confounding. Applying this methodology to the HPOG study shows that program-average FTE months of study by month six was a significant mediator of both career progress and long-term degree/credential receipt. Evaluators can robustify their BHR-style analyses by the use of the methods proposed here.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"274-311"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9969498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Novel Indicator to Correct for Individual Reported Heterogeneity. An Application to Self-Evaluation of Later-Life Depression.","authors":"Serena Berretta, Sara Garbin, Maria Iannario, Omar Paccagnella","doi":"10.1177/0193841X231171965","DOIUrl":"10.1177/0193841X231171965","url":null,"abstract":"<p><p>Program evaluations often investigate complex or multi-dimensional constructs, such as individual opinions or attitudes, by means of ratings. A different interpretation of the same question may affect cross-country comparability, leading to the Differential Item Functioning problem. Anchoring vignettes were introduced in the literature as a way to adjust self-evaluations from this interpersonal incomparability. In this paper, we first introduce a new nonparametric solution to analyse anchoring vignette data, recoding a variable based on a rating scale to a <i>new corrected-</i>variable that guarantees comparability in any cross-country analysis. Then, we exploit the flexibility of a mixture model introduced to account for uncertainty in the response process (the CUP model) to test if the proposed solution is effectively able to remove this reported heterogeneity. This solution is easy to construct and has important advantages compared with the original nonparametric solution adopted with anchoring vignette data. The novel indicator is applied to investigate self-reported depression in an old population. Data that will be analysed come from the second wave of the Survey of Health, Ageing and Retirement in Europe, collected in 2006/2007. Results highlight the need of correcting for reported heterogeneity comparing individual self-evaluations. Once interpersonal incomparability resulting from the different uses of response scales is removed from the self-assessments, some estimates are reversed in magnitude and signs with respect to the analysis of the collected data.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"221-250"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9433536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Impact of Economic Corridor and Tourism on Local Community's Quality of Life under One Belt One Road Context.","authors":"Shiying Wang, Jaffar Abbas, Khalid Ibrahim Al-Sulati, Syed Ale Raza Shah","doi":"10.1177/0193841X231182749","DOIUrl":"10.1177/0193841X231182749","url":null,"abstract":"<p><p>Economic corridors unlock new economic opportunities and tourism development in the region to achieve sustainable development goals. Green economic growth is conducive to environmental sustainability. Economic mega-projects of CPEC promote tourism that leads to communities' well-being and better quality of life. Modern infrastructure development contributes significantly to economic growth and tourism activities. This study's objectives emphasize exploring tourism and sustainable development pursuits under OBOR economic projects that open doors to improving residents' quality of life. The growing world is an eyewitness to a continuous rise in emissions and its severe consequences for humankind. It is necessary to show off the leading factors that result in tourism and economic activities causing environmental pollution rather than blame policymakers. Undoubtedly, many studies previously focused on demonstrating the influence of socio-economic factors that lead to better environmental quality. However, the empirical literature on tourism, social well-being, foreign direct investment, and the Environment in Belt and Road developed economies needed improvement. This research applied a series of advanced estimators that help demonstrate the study's probable results. This study explores the role of Social well-being (HDI), tourism development, FDI, renewable energy, information & communication technology (ICT), and urbanization on CO2 emissions in Belt and Road (BRI) developed economies.Estimated results exhibited the significant contribution of ICT and renewable energy to sustainability. Besides, FDI contributes to emissions reduction after its threshold level. Conversely, urbanization and tourism activities contribute to environmental pollution. The study outcomes stated inverted/EKC U-shaped hypotheses related to specified economies. Finally, the analysis based on the D-H panel causality test constructs exciting results.The present study concludes that economic corridor plays a vital role in tourism development, the community's well-being, and SDGs goals (sustainable development) impact on environmental safety. The findings suggest essential and applicable policies to attain the desired sustainability level. Findings contribute to the literature on tourism, well-being, and sustainability. Further studies can use insights using this methodology.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"312-345"},"PeriodicalIF":0.9,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10050838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Program Evaluation’s Path to Greater Policy Relevance: Learning From Rossi’s Iron Laws","authors":"Douglas J. Besharov","doi":"10.1177/0193841x241238031","DOIUrl":"https://doi.org/10.1177/0193841x241238031","url":null,"abstract":"In a 1987 article, Peter R. Rossi promulgated “The Iron Law of Evaluation and Other Metallic Rules.” The Metallic Laws were meant as an informal (and humorous) overstatement of the weakness of contemporary evaluations of social programs. Rossi’ s underlying worry was not so much about the state of evaluation technology in the abstract, but, rather, in its inability to advance our broad understanding of social problems and what to do about them---in other words, to make evaluation policy relevant. Rossi attributed the continuing failure to develop successful “large-scale social programs” to the failure to build a strong knowledge base for this kind of “social engineering.” The qualities of studies that enable such accumulated learning are variously labeled “external validity,” “generalizability,” “applicability,” or “transferability.” This Special Issue includes five papers that seek to explore and apply this understanding.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":"12 1","pages":""},"PeriodicalIF":0.9,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140036990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Go for Economic Transformation and Development in China: Financial Development, Higher Education, and Green Technology Evolution.","authors":"Xiaoxiao Zhou, Xinyue Hu, Mei Duan, Licheng Peng, Xin Zhao","doi":"10.1177/0193841X231166741","DOIUrl":"10.1177/0193841X231166741","url":null,"abstract":"<p><p>Technology innovation is the key driving force in achieving economic transformation and development. Financial development and the expansion of higher education can promote technological progress primarily by easing financing constraints and improving the level of human capital. This study examines the impact of financial development and higher education expansion on green technology innovation. It conducts an empirical analysis by constructing a linear panel model and a nonlinear threshold model. The present study sample is based on the urban panel data of China from 2003-2019. (1) Financial development can significantly promote the expansion of higher education. (2) The expansion of higher education can improve energy and environment-based technological progress. (3) Financial development can both directly and indirectly promote green technology evolution by expanding higher education. The joint financial development and higher education expansion can significantly empower green technology innovation. (4) In the process of promoting green technology innovation, financial development has a non-linear influence on it, with higher education as the threshold. The effect of financial development on green technology innovation varies according to the degree of higher education. Based on these findings, we put forward policy proposals for green technology innovation to promote economic transformation and development in China.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"32-62"},"PeriodicalIF":0.9,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9319907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Digitalization, new business Startups, information and Communication Technologies and product innovation: Evidence From China in the lens of sustainability.","authors":"Jiawen Bai, Tianyu Bai, Chengyun Zhang","doi":"10.1177/0193841X231169419","DOIUrl":"10.1177/0193841X231169419","url":null,"abstract":"<p><p>The global economies and international organizations are inclined towards sustainable growth, technological advancements and product innovations. China is the leading economy in information and communication technologies and among the major industrially expanded economies covering a substantial share of the global market in exports. The prime objective of this study is to explore the role of digitalization and Information and communication technologies (ICT) for product innovation (PIN). In doing so, the study also attempts to draw some novel implications regarding business, entrepreneurship, and product innovation in the lens of sustainability. This current study use the annual data of China from 1990-2020. The empirical analysis was conducted using the stationarity testing and the Johansen cointegration test. In addition, due to the data's asymmetrical distribution, the non-parametric \"quantile regression\" is used. For robustness, this study employs the Fully Modified Ordinary Least Square, Canonical Cointegration, and Dynamic Ordinary Least Square methods. The empirical results reveal that economic progress and financial development are substantial factors of product innovation. The robust analysis reveals that medium and high-tech industries and information and communication technology adversely affect product innovation. Further, the presence of financial development transforms the negative influence of information and communication technology into a positive. The current study concludes more investments in the technological industry are required to encourage product innovation in China. The study discusses some policy-related implications in the context of business sustainability and product innovation.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":" ","pages":"90-118"},"PeriodicalIF":0.9,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9382855","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}