Evaluation Review最新文献_第9页

Can Non-Randomised Studies of Interventions Provide Unbiased Effect Estimates? A Systematic Review of Internal Replication Studies. 干预措施的非随机研究能否提供无偏效果估计?内部复制研究的系统回顾。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-06-01 DOI: 10.1177/0193841X221116721

Hugh Sharma Waddington, Paul Fenton Villar, Jeffrey C Valentine

{"title":"Can Non-Randomised Studies of Interventions Provide Unbiased Effect Estimates? A Systematic Review of Internal Replication Studies.","authors":"Hugh Sharma Waddington, Paul Fenton Villar, Jeffrey C Valentine","doi":"10.1177/0193841X221116721","DOIUrl":"https://doi.org/10.1177/0193841X221116721","url":null,"abstract":"Non-randomized studies of intervention effects (NRS), also called quasi-experiments, provide useful decision support about development impacts. However, the assumptions underpinning them are usually untestable, their verification resting on empirical replication. The internal replication study aims to do this by comparing results from a causal benchmark study, usually a randomized controlled trial (RCT), with those from an NRS conducted at the same time in the sampled population. We aimed to determine the credibility and generalizability of findings in internal replication studies in development economics, through a systematic review and meta-analysis. We systematically searched for internal replication studies of RCTs conducted on socioeconomic interventions in low- and middle-income countries. We critically appraised the benchmark randomized studies, using an adapted tool. We extracted and statistically synthesized empirical measures of bias. We included 600 estimates of correspondence between NRS and benchmark RCTs. All internal replication studies were found to have at least \"some concerns\" about bias and some had high risk of bias. We found that study designs with selection on unobservables, in particular regression discontinuity, on average produced absolute standardized bias estimates that were approximately zero, that is, equivalent to the estimates produced by RCTs. But study conduct also mattered. For example, matching using pre-tests and nearest neighbor algorithms corresponded more closely to the benchmarks. The findings from this systematic review confirm that NRS can produce unbiased estimates. Authors of internal replication studies should publish pre-analysis protocols to enhance their credibility.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/e0/ef/10.1177_0193841X221116721.PMC10186563.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9486562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Linking Trait Affectivity, Cognitive Ability, and Preferences Among Top Managers: Insights From a Lab-In-The-Field Experiment. 将高层管理人员的特质情感、认知能力和偏好联系起来：实验室现场实验的启示》。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-06-01 Epub Date: 2022-12-05 DOI: 10.1177/0193841X221143829

Huong Trang Kim

{"title":"Linking Trait Affectivity, Cognitive Ability, and Preferences Among Top Managers: Insights From a Lab-In-The-Field Experiment.","authors":"Huong Trang Kim","doi":"10.1177/0193841X221143829","DOIUrl":"10.1177/0193841X221143829","url":null,"abstract":"Even though there has been increasing interest in the role of cognition in leadership and in identifying the personality traits of effective leaders, there is a paucity of studies that investigate the unique influence of managers' trait affectivity and cognitive ability on their different risk and time preferences. This paper investigates the role of managers' trait affectivity and cognitive ability in their loss aversion and present bias among 623 top managers at textile and garment firms in Vietnam. We combine data on preferences elicited through a lab-in-the-field experiment with survey data. We find that managers with high positive affectivity (PA) or cognitive ability are less subject to loss aversion and present bias. In contrast, a manager with high negative affectivity (NA) is more likely to be impatient and loss averse. Furthermore, heterogeneity of trait affectivity and cognitive ability determines different loss aversion and present bias levels of managers in SMEs vis-à-vis their counterparts in large firms. Remarkably, we observe striking evidence that trait affectivity and cognitive ability significantly affect loss aversion and present bias levels of managers who were born during the Vietnam War. Still, it is not the story of their counterparts born after the Vietnam War. The results of our study are expected to provide valuable information regarding the role played by trait affectivity and cognitive ability in determining managers' loss aversion and present bias in different pathways.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9476531","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Government Bonds and COVID-19. An International Evaluation Under Different Market States. 政府债券和COVID-19。不同市场状态下的国际评价。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-06-01 DOI: 10.1177/0193841X221143680

Francisco Jareño, María-Isabel Martínez-Serna, María Chicharro

{"title":"Government Bonds and COVID-19. An International Evaluation Under Different Market States.","authors":"Francisco Jareño, María-Isabel Martínez-Serna, María Chicharro","doi":"10.1177/0193841X221143680","DOIUrl":"https://doi.org/10.1177/0193841X221143680","url":null,"abstract":"This study evaluates the sensitivity of government bond yields from the countries most affected by the COVID-19 pandemic to variations in some international risk factors during the period between January 2020 and April 2021. This sample period allows us to focus the study on the first, and the subsequent waves of the COVID-19 pandemic. Specifically, we propose an extended risk factor model estimated using the quantile regression approach. In addition, this study compares the COVID-19 pandemic period with a pre-pandemic and a post-vaccination period. Interesting differences among them are observed, remarking that gold is the key risk factor during the pandemic, whereas VIX and crude oil play that role in the pre-pandemic and the post-vaccination periods, respectively, mainly for bearish states. As expected, the explanatory power of the model is better at extreme quantiles, showing relevant differences between sensitivities, because the found effects are quantile-, country- and risk factor-dependent. The results during the pandemic are robust to the inclusion of a country-specific factor and a factor accounting for the mutual influence of the government bonds.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9720420/pdf/10.1177_0193841X221143680.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10095150","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Evaluating the Safe-Haven Abilities of Bitcoin and Gold for Crude Oil Market: Evidence During the COVID-19 Pandemic. 评估比特币和黄金对原油市场的避险能力：COVID-19 大流行期间的证据。

IF 3 4区社会学

Evaluation Review Pub Date : 2023-06-01 Epub Date: 2022-12-01 DOI: 10.1177/0193841X221141812

Qian Wang, Yu Wei, Yifeng Zhang, Yuntong Liu

{"title":"Evaluating the Safe-Haven Abilities of Bitcoin and Gold for Crude Oil Market: Evidence During the COVID-19 Pandemic.","authors":"Qian Wang, Yu Wei, Yifeng Zhang, Yuntong Liu","doi":"10.1177/0193841X221141812","DOIUrl":"10.1177/0193841X221141812","url":null,"abstract":"The COVID-19 pandemic poses a serious threat to investors in the crude oil market. Furthermore, investors have an increasing need to find a safe haven in their investment portfolios when facing unprecedented risks in crude oil markets during the COVID-19 pandemic. According to a review of the literature, there are contradictory findings on which investment is the safer haven for the oil market. Therefore, this paper aims to evaluate whether bitcoin is a safer haven for the crude oil market than the commonly used gold during the COVID-19 pandemic. Three spillover measurements based on the time, and frequency domains, and a network framework are employed to quantify the return spillover effects among bitcoin, gold and three major crude oil futures markets. We divide the sample into two periods, pre-COVID-19 and post-COVID-19. The results show that bitcoin has a weak safe-haven effect on the crude oil market only over a short period, while gold maintains a good safe-haven ability for crude oil futures across various time horizons (frequencies), both before and after the outbreak of the COVID-19 pandemic. The findings of this study have important implications for policy-makers, crude oil producers and global investors. In particularly, investors cannot ignore the importance of bitcoin and gold in selecting more profitable portfolio policies when searching for safe-haven assets.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9720065/pdf/10.1177_0193841X221141812.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9481915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating Research Efficiency in Indonesian Higher Education Institution. 评价印尼高等教育机构的研究效率。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-04-01 DOI: 10.1177/0193841X221118181

Muhammad Dimyati, Adhi Indra Hermanu

{"title":"Evaluating Research Efficiency in Indonesian Higher Education Institution.","authors":"Muhammad Dimyati, Adhi Indra Hermanu","doi":"10.1177/0193841X221118181","DOIUrl":"https://doi.org/10.1177/0193841X221118181","url":null,"abstract":"The Indonesian government has evaluated the research performance of universities, whose measurement process is projected into resources, management, outputs, and revenues to determine the provision of incentives, grants, and program funding to universities. However, efficiency calculations have shown that the outputs and competition-based incentives that drive scientific productivity are more complex. The most competitive systems must also be the most productive when considering resources. This study aimed to analyze the research efficiency in the Indonesian higher education system. The efficiency was analyzed by maximizing the 13 product outputs from the research budget and university staff. The result was then compared with the existing performance measurement analysis. Data Envelopment Analysis (DEA) was used to evaluate the efficiency based on the data of 47 universities in the Mandiri cluster and 144 in the Utama cluster for the 2014-2018 period. These findings showed that about 68% of universities have an efficiency value of 1 for the Mandiri group, almost 40% in the Utama group, and 41% for the two groups combined. Additionally, this study compared the efficiency analysis and the impact of the performance evaluation. The comparison showed that adding efficiency or productivity factors in the performance evaluation assessment produced a more accurate result.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9075550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

2020 Rossi Award Lecture: The Evolving Art of Program Evaluation. 2020 年罗西奖讲座：不断发展的计划评估艺术。

IF 3 4区社会学

Evaluation Review Pub Date : 2023-04-01 Epub Date: 2022-08-29 DOI: 10.1177/0193841X221121241

Randall S Brown

{"title":"2020 Rossi Award Lecture: The Evolving Art of Program Evaluation.","authors":"Randall S Brown","doi":"10.1177/0193841X221121241","DOIUrl":"10.1177/0193841X221121241","url":null,"abstract":"Evaluation of public programs has undergone many changes over the past four decades since Peter Rossi coined his \"Iron Law\" of program evaluation: \"The expected value of any net impact assessment of any large-scale social program is zero.\" While that assessment may be somewhat overstated, the essence still holds. The failures far outnumber the successes, and the estimated favorable effects are rarely sizeable. Despite this grim assessment, much can be learned from \"failed\" experiments, and from ones that are successful in only some sites or subgroups. Advances in study design, statistical models, data, and how inferences are drawn from estimates have substantially improved our analyses and will continue to do so. However, the most actual learning about \"what works\" (and why, when, and where) is likely to come from gathering more detailed and comprehensive data on how the intervention was implemented and attempting to link that data to estimated impacts. Researchers need detailed data on the target population served, the content of the intervention, and the process by which it is delivered to participating service providers and individuals. Two examples presented here illustrate how researchers drew useful broader lessons from impact estimates for a set of related programs. Rossi posited three reasons most interventions fail-wrong question, wrong intervention, poor implementation. Speeding the accumulation of wisdom about how social programs can best help vulnerable populations will require that researchers work closely with program funders, developers, operators, and participants to gather and interpret these detailed data about program implementation.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":3.0,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9095736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Regression Discontinuity for Binary Response and Local Maximum Likelihood Estimator to Extrapolate Treatment. 二元响应的回归不连续及外推处理的局部极大似然估计。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-04-01 DOI: 10.1177/0193841X221105968

Goeun Lee, Myoung-Jae Lee

引用次数: 1

Renewable Energy Consumption-Growth Nexus in European Countries: A Sectoral Approach. 欧洲国家的可再生能源消费-增长关系:一个部门方法。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-04-01 DOI: 10.1177/0193841X221125982

Mihaela Simionescu, Magdalena Rădulescu, Javier Cifuentes-Faura

{"title":"Renewable Energy Consumption-Growth Nexus in European Countries: A Sectoral Approach.","authors":"Mihaela Simionescu, Magdalena Rădulescu, Javier Cifuentes-Faura","doi":"10.1177/0193841X221125982","DOIUrl":"https://doi.org/10.1177/0193841X221125982","url":null,"abstract":"The renewable energy consumption plays a significant role in achieving sustainable development, but a sectoral approach is necessary to design the better recommendations for each sector. In this context, the main objective of this paper is to assess the impact of the use of this type of energy on economic growth in 23 European Union (EU) member states in the period 1990-2020. Besides overall renewable energy consumption, different utilisations of this energy are considered: in industry, transport, in commercial and public services, and for residential purposes. The methodological background is built around panel data models that start from a Cobb-Douglas function. The renewable energy consumption is considered an important factor that should generate economic growth. The panel data approach based on causality analysis and Augmented Mean Group and Common Correlated Effects Mean Group estimators suggests that renewable energy use in industry does not determine economic growth, but economic development is a cause for more utilisation of this energy in industry. In addition, more renewable energy consumption in transport enhances economic growth. A high level of economic development can promote the consumption of renewable energies in industrial sectors. In this way, industrial companies can allocate more financial funds to research in the field of renewable energies and can afford to adopt renewable energy sources. Investment in biofuels can contribute to achieving sustainable transport in the EU.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10821866","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

Evaluating the Importance of Monetary Policy Uncertainty: The Long- and Short-Term Effects and Responses. 评估货币政策不确定性的重要性:长期和短期影响及对策。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-04-01 DOI: 10.1177/0193841X221124434

Yang Hu, Yanran Hong, Kai Feng, Jikai Wang

{"title":"Evaluating the Importance of Monetary Policy Uncertainty: The Long- and Short-Term Effects and Responses.","authors":"Yang Hu, Yanran Hong, Kai Feng, Jikai Wang","doi":"10.1177/0193841X221124434","DOIUrl":"https://doi.org/10.1177/0193841X221124434","url":null,"abstract":"Monetary policy changes have an irreplaceable impact on economic activity. Considering the close linkage among economic policies, we employ a bi-directional Granger causality test to investigate the potential linkages between monetary policy uncertainty (MPU) and other categorical economic policy uncertainty (CEPU) in the time and frequency domains. We consider all news-based U.S. categorical economic policy uncertainty indices (CEPU). All monthly CEPU indicators, covering January 1986 to January 2022, can be obtained from the website of Economic Policy Uncertainty. On an average, causality running from each CEPU to MPU is not apparent, while MPU can significantly affect six policy-related uncertainties: taxes, government spending, health care, national security, entitlement programs and regulation. A further frequency-domain study showed the dynamic changes in the relationship between them. For instance, we capture mid- and long-run causality running from tax uncertainty to MPU, while MPU has an impact on taxes in the medium run. Our findings provide policymakers with a better understanding of the nexus between MPU and other CEPU for formulating appropriate economic policies. Particularly, if a sectional government considers the long- and short-term effects of different policies when formulating strategies, risk transmission may be curbed to some extent.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10821867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Estimating the Impact of Emergency Assistance on Educational Progress for Low-Income Adults: Experimental and Nonexperimental Evidence. 估计紧急援助对低收入成人教育进步的影响:实验和非实验证据。

IF 0.9 4区社会学

Evaluation Review Pub Date : 2023-04-01 DOI: 10.1177/0193841X221118454

Daniel Litwok

{"title":"Estimating the Impact of Emergency Assistance on Educational Progress for Low-Income Adults: Experimental and Nonexperimental Evidence.","authors":"Daniel Litwok","doi":"10.1177/0193841X221118454","DOIUrl":"https://doi.org/10.1177/0193841X221118454","url":null,"abstract":"Methods for estimating causal impact aim to either remove or reduce bias. This study estimates the degree of bias reduction obtained from regression adjustment and propensity score methods when only a weak set of predictors are available. The study uses an experimental test of providing emergency financial assistance to participants in a job training program to estimate an experimental benchmark and compares it to nonexperimental estimates of the impact of receiving assistance. When estimating the impact of receiving assistance, those who received it constitute the treatment group. The study explores two different comparison groups: those who could have (because they were assigned to the experimental treatment group) but did not receive emergency assistance; and those who could not receive emergency assistance because they were randomly assigned to the experimental control group. It uses these groups to estimate impacts by applying three estimation strategies: unadjusted mean comparison, regression adjustment, and inverse propensity weighting. It then compares these estimates to the experimental benchmark using statistical tests recommended by the within-study comparison literature. The nonexperimental approaches to addressing selection bias suggest large positive impacts. These are statistically different from the experimental benchmark, which shows that receipt of emergency assistance does not improve educational progress. Further, over 90% of the bias from a simple comparison of means remains. Unless a stronger set of predictors are available, future evaluations of such interventions should be wary of relying on these methods for either unbiased estimation of impacts or bias reduction.","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9075559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0