Estimating the Impact of Emergency Assistance on Educational Progress for Low-Income Adults: Experimental and Nonexperimental Evidence.

IF 3 4区 社会学 Q1 SOCIAL SCIENCES, INTERDISCIPLINARY
Daniel Litwok
{"title":"Estimating the Impact of Emergency Assistance on Educational Progress for Low-Income Adults: Experimental and Nonexperimental Evidence.","authors":"Daniel Litwok","doi":"10.1177/0193841X221118454","DOIUrl":null,"url":null,"abstract":"<p><p>Methods for estimating causal impact aim to either remove or reduce bias. This study estimates the degree of bias reduction obtained from regression adjustment and propensity score methods when only a weak set of predictors are available. The study uses an experimental test of providing emergency financial assistance to participants in a job training program to estimate an experimental benchmark and compares it to nonexperimental estimates of the impact of receiving assistance. When estimating the impact of receiving assistance, those who received it constitute the treatment group. The study explores two different comparison groups: those who could have (because they were assigned to the experimental treatment group) but did not receive emergency assistance; and those who could not receive emergency assistance because they were randomly assigned to the experimental control group. It uses these groups to estimate impacts by applying three estimation strategies: unadjusted mean comparison, regression adjustment, and inverse propensity weighting. It then compares these estimates to the experimental benchmark using statistical tests recommended by the within-study comparison literature. The nonexperimental approaches to addressing selection bias suggest large positive impacts. These are statistically different from the experimental benchmark, which shows that receipt of emergency assistance does not improve educational progress. Further, over 90% of the bias from a simple comparison of means remains. Unless a stronger set of predictors are available, future evaluations of such interventions should be wary of relying on these methods for either unbiased estimation of impacts or bias reduction.</p>","PeriodicalId":47533,"journal":{"name":"Evaluation Review","volume":"47 2","pages":"231-263"},"PeriodicalIF":3.0000,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Evaluation Review","FirstCategoryId":"90","ListUrlMain":"https://doi.org/10.1177/0193841X221118454","RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SOCIAL SCIENCES, INTERDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

Methods for estimating causal impact aim to either remove or reduce bias. This study estimates the degree of bias reduction obtained from regression adjustment and propensity score methods when only a weak set of predictors are available. The study uses an experimental test of providing emergency financial assistance to participants in a job training program to estimate an experimental benchmark and compares it to nonexperimental estimates of the impact of receiving assistance. When estimating the impact of receiving assistance, those who received it constitute the treatment group. The study explores two different comparison groups: those who could have (because they were assigned to the experimental treatment group) but did not receive emergency assistance; and those who could not receive emergency assistance because they were randomly assigned to the experimental control group. It uses these groups to estimate impacts by applying three estimation strategies: unadjusted mean comparison, regression adjustment, and inverse propensity weighting. It then compares these estimates to the experimental benchmark using statistical tests recommended by the within-study comparison literature. The nonexperimental approaches to addressing selection bias suggest large positive impacts. These are statistically different from the experimental benchmark, which shows that receipt of emergency assistance does not improve educational progress. Further, over 90% of the bias from a simple comparison of means remains. Unless a stronger set of predictors are available, future evaluations of such interventions should be wary of relying on these methods for either unbiased estimation of impacts or bias reduction.

估计紧急援助对低收入成人教育进步的影响:实验和非实验证据。
估计因果影响的方法旨在消除或减少偏倚。当只有一组弱预测因子可用时,本研究估计了通过回归调整和倾向评分方法获得的偏倚减少程度。这项研究使用了一项为职业培训项目的参与者提供紧急财政援助的实验测试来估计一个实验基准,并将其与接受援助影响的非实验估计进行比较。在评估接受援助的影响时,接受援助的人构成治疗组。该研究探讨了两个不同的对照组:那些本可以(因为他们被分配到实验治疗组)但没有得到紧急援助的人;而那些不能得到紧急援助的人因为他们被随机分配到实验对照组。它利用这些群体通过应用三种估计策略来估计影响:未经调整的平均值比较,回归调整和逆倾向加权。然后将这些估计值与实验基准进行比较,使用研究内比较文献推荐的统计测试。解决选择偏差的非实验方法显示出很大的积极影响。这些数据在统计上与实验基准不同,实验基准表明,接受紧急援助并没有改善教育进展。此外,从简单的均值比较中得出的偏差超过90%仍然存在。除非有一组更有力的预测因子可用,否则未来对此类干预措施的评估应谨慎,不要依赖这些方法对影响进行无偏估计或减少偏倚。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Evaluation Review
Evaluation Review SOCIAL SCIENCES, INTERDISCIPLINARY-
CiteScore
2.90
自引率
11.10%
发文量
80
期刊介绍: Evaluation Review is the forum for researchers, planners, and policy makers engaged in the development, implementation, and utilization of studies aimed at the betterment of the human condition. The Editors invite submission of papers reporting the findings of evaluation studies in such fields as child development, health, education, income security, manpower, mental health, criminal justice, and the physical and social environments. In addition, Evaluation Review will contain articles on methodological developments, discussions of the state of the art, and commentaries on issues related to the application of research results. Special features will include periodic review essays, "research briefs", and "craft reports".
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信