{"title":"Children Learning and Parents Earning: Exploring the Average and Heterogeneous Effects of Head Start on Parental Earnings","authors":"Owen N. Schochet, Christina M. Padilla","doi":"10.1080/19345747.2021.2005202","DOIUrl":"https://doi.org/10.1080/19345747.2021.2005202","url":null,"abstract":"Abstract Head Start (HS) is our nation’s largest two-generation program that provides early education services to children and a variety of family support services that may promote economic wellbeing. Yet, no prior research has documented or described the effects of HS on parental earnings. We explore whether the program promotes parental earnings on average, investigate for whom these effects are greatest, evaluate the extent to which earnings impacts vary across HS sites, and identify which characteristics of centers associate with cross-site variation. We find that HS does not improve earnings overall. However, the program does increase parental earnings in a younger program cohort two and three years after random assignment. These effects are larger for single parents and those who are initially employed or in school. Earnings effects are typically homogenous across sites, although we do observe increasing variation over time that reaches statistical significance four years after random assignment. We are generally unable to explain this variation using measures of what HS sites do or provide apart from the economic wellbeing of the families they serve.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46109874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jonté A. Myers, Elizabeth M. Hughes, Bradley Witzel, Rubia D. Anderson, Jennifer Owens
{"title":"A Meta-Analysis of Mathematical Interventions for Increasing the Word Problem Solving Performance of Upper Elementary and Secondary Students with Mathematics Difficulties","authors":"Jonté A. Myers, Elizabeth M. Hughes, Bradley Witzel, Rubia D. Anderson, Jennifer Owens","doi":"10.1080/19345747.2022.2080131","DOIUrl":"https://doi.org/10.1080/19345747.2022.2080131","url":null,"abstract":"Abstract This meta-analysis synthesized the findings of experimental and quasi-experimental studies to determine the efficacy of math interventions for enhancing the word problem-solving accuracy of students with mathematics difficulties in Grades 4 to 12. Further, we identified intervention and design characteristics that moderate the effectiveness of interventions. We appraised study quality using Council for Exceptional Children (CEC's) evidence standards. We also used random-effects meta-regression models with robust variance estimation (RVE) to compute the overall weighted treatment effect and investigate the moderating impact of 15 variables. Results from 36 intervention studies published between 1989 and 2020 reveal a relatively large and positive mean effect size (g = 0.71; SE = 0.11, 95% CI [0.49, 0.92]) across 88 outcomes, suggesting the examined interventions were effective. The moderator analyses' findings showed the treatment efficacy varied as a function of five intervention characteristics, including intervention model, interventionist, grade level, math topic, and intervention duration. None of the design features we examined led to significant moderator effects. We discuss findings and limitations and provide directions for future research and practice.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"60072032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Framework for Addressing Instrumentation Biases When Using Observation Systems as Outcome Measures in Instructional Interventions","authors":"Mark White, Bridget Maher, Brian Rowan","doi":"10.1080/19345747.2022.2081275","DOIUrl":"https://doi.org/10.1080/19345747.2022.2081275","url":null,"abstract":"Abstract Many educational interventions seek to change teachers’ instructional practice. Standards-based observation systems are a common and useful tool to understand these interventions’ impact, but the process of measuring instructional change with observation systems is highly complex. This paper introduces a framework for examining and understanding potential instrumentation biases that arise when evaluations use observation systems to understand instructional change. The framework systematizes two processes that all studies must undertake: (1) the process of operationalizing the construct of teaching quality, and (2) the process of data collection. A study that engages in these processes generates observation scores that capture their own raters’ perspectives on specific segments of instruction. These scores must be generalized to draw conclusions about the intended constructs and settings. Systematizing these two processes highlights the necessary steps of a validity argument supporting evaluation conclusions and the instrumentation biases that threaten such conclusions. The framework is illustrated with an example from our recent work, which sought to understand instructional change since the adoption of the Common Core State Standards (CCSS).","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44842822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Why Do We Find These Effects? An Examination of Mediating Pathways Explaining the Effects of School Turnaround","authors":"L. D. Pham","doi":"10.1080/19345747.2022.2081276","DOIUrl":"https://doi.org/10.1080/19345747.2022.2081276","url":null,"abstract":"Abstract Mixed results from evaluations of school reform suggest a need for evidence to explain why some models succeed while others fail. Addressing that need, this study uses structural equation modeling to estimate difference-in-differences models that examine mediating mechanisms for positive effects produced by Innovation Zone (iZone) reforms in Memphis, Tennessee. I find that iZone schools increased peer collaboration between teachers, which resulted in improved student achievement. Also, recruiting effective teachers led to a more positive learning environment and ultimately to improved student achievement. These results highlight peer collaboration, a positive learning environment, and the recruitment of effective educators as important practices that will likely facilitate improved school performance under future school reform plans.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43902685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Recipe for Disappointment: Policy, Effect Size, and the Winner’s Curse","authors":"A. Simpson","doi":"10.1080/19345747.2022.2066588","DOIUrl":"https://doi.org/10.1080/19345747.2022.2066588","url":null,"abstract":"Standardized effect size estimates are commonly used by the ‘evidence-based education’ community as a key metric for judging relative importance, effectiveness, or practical significance of interventions across a set of studies: larger effect sizes indicate more effective interventions. However, this argument applies rarely; only when linearly equatable outcomes, identical comparison treatments and equally representative samples are used in every study.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43865769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Does Principal Professional Development Improve Schooling Outcomes? Evidence from Pennsylvania’s Inspired Leadership Induction Program","authors":"Matthew P. Steinberg, Haisheng Yang","doi":"10.1080/19345747.2022.2052386","DOIUrl":"https://doi.org/10.1080/19345747.2022.2052386","url":null,"abstract":"Abstract Principals shape the academic setting of schools. Yet, there is limited evidence on whether principal professional development improves schooling outcomes. Beginning in 2008–2009, Pennsylvania’s Inspired Leadership (PIL) induction program required that newly hired principals complete targeted in-service professional development tied to newly established state leadership standards within 5 years of employment. Using panel data on all Pennsylvania students, teachers, and principals, we leverage variation in the timing of PIL induction across principal-school cells and employ difference-in-differences and event study strategies to estimate the impact of PIL induction on teacher and student outcomes. We find that PIL induction increased student math achievement through improvements in teacher effectiveness, and that the effects of PIL induction on teacher effectiveness were concentrated among the most economically disadvantaged and urban schools in Pennsylvania. Principal professional development had the greatest impact on teacher effectiveness when principals completed PIL induction during their first 2 years in the principalship. We also find evidence that teacher turnover declined in the years following the completion of PIL induction. We discuss the implications of our findings for principal induction efforts.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43243535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ira Nichols-Barrer, Marilyn J. Bartlett, Thomas Coen, Phil Gleason
{"title":"The Impacts on College Enrollment of a Charter Network Serving Disadvantaged Students: Evidence From KIPP Middle School Lotteries","authors":"Ira Nichols-Barrer, Marilyn J. Bartlett, Thomas Coen, Phil Gleason","doi":"10.1080/19345747.2022.2049406","DOIUrl":"https://doi.org/10.1080/19345747.2022.2049406","url":null,"abstract":"Abstract Recent studies of charter school effectiveness have questioned whether charter school networks can produce a lasting impact on students’ long-term outcomes. Our study is the first to examine this issue at the network of Knowledge is Power Program (KIPP) schools, which primarily serve disadvantaged students of color and constitute the nation’s largest charter school network. Using admission lotteries as a random assignment instrument, we estimate the impacts of 13 KIPP middle schools on college enrollment and persistence in college over the first two postsecondary years. Building on prior studies of KIPP that show KIPP middle schools have strong positive effects on middle school achievement, we find that KIPP middle schools also positively affect rates of enrollment in 4-year college programs. The magnitude of the estimated impact of a middle school admission offer (6.9%) and enrollment at a KIPP school (12.9%) is substantial relative to nationwide disparities in college enrollment across racial groups.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49170808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"College or Bust… or Both: The Effects of the Great Recession on College Enrollment for Black and Latinx Young Adults","authors":"Tolani A. Britton","doi":"10.1080/19345747.2022.2054885","DOIUrl":"https://doi.org/10.1080/19345747.2022.2054885","url":null,"abstract":"Abstract This article looks at whether the Great Recession led to changes in two-year and four-year college enrollment patterns for students aged 18–24. In particular, I examine how the probability of enrollment changed for Black and Latinx students. It is not initially clear whether the Great Recession would increase or decrease college enrollment for these populations. On the one hand, higher unemployment rates could prompt people to enroll in college, but on the other, reduced credit availability could decrease an individual’s ability to cover costs. I use the severity of the recession in different states to compare how enrollment changed in states that had relatively larger increases in the unemployment rates during the recession using the Current Population Survey (CPS) October Education supplements from 2002 to 2012 as the data source. I find that the probability of college enrollment increased after the onset of the Great Recession for Black students. There is some weak evidence that enrollment might have increased for Latinx students. Enrollment was not more likely in states with higher than average unemployment rates.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47787912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. French, A. Sariaslan, H. Larsson, D. Kneale, G. Leckie
{"title":"Estimating the Importance of Families in Modeling Educational Achievement Using Linked Swedish Administrative Data","authors":"R. French, A. Sariaslan, H. Larsson, D. Kneale, G. Leckie","doi":"10.1080/19345747.2022.2054480","DOIUrl":"https://doi.org/10.1080/19345747.2022.2054480","url":null,"abstract":"Abstract While the family is a critical determinant of educational achievement, methodological difficulties and the availability of data limit estimation of the family contribution in school effectiveness models. This study uses multilevel modeling to estimate the proportion of variation in student educational achievement between families, family-level intraclass correlation coefficients, and specific family structure effects (family size, birth order, birth spacing, sibling sex ratio). We use cross-classified random effects to account for school and neighborhood variation. We analyze Swedish administrative education records linked with birth records for four academic cohorts of students, with siblings identified from a wider pool of 21 cohorts. We show that almost half of the variation in student achievement described as “between students” in traditional school effectiveness studies would be better described as variation “between families,” suggesting effectiveness research might give greater consideration to family-based interventions in tandem with existing student- and school-based approaches to raising low achievement.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48717626","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Diego Catalán Molina, Tenelle Porter, Catherine Oberle, Misha D. Haghighat, Afiya C. Fredericks, Kristen Budd, Sylvia Roberts, L. Blackwell, Kali Trzesniewski
{"title":"How to Measure Quality of Delivery: Focus on Teaching Practices That Help Students to Develop Proximal Outcomes","authors":"Diego Catalán Molina, Tenelle Porter, Catherine Oberle, Misha D. Haghighat, Afiya C. Fredericks, Kristen Budd, Sylvia Roberts, L. Blackwell, Kali Trzesniewski","doi":"10.1080/19345747.2022.2054481","DOIUrl":"https://doi.org/10.1080/19345747.2022.2054481","url":null,"abstract":"Abstract Recent studies have suggested that quality of delivery matters to achieve better student outcomes in the context of school interventions. However, studies rarely measure quality of delivery and test its association with students’ outcomes, perhaps due to lack of clarity regarding how to measure it. Here, we offer recommendations on how to select or design measures of quality of delivery. These recommendations focus on identifying teaching practices that help students to develop proximal outcomes during the delivery of an intervention. Additionally, we illustrate an application of these recommendations to the study of quality of delivery in a cluster-randomized efficacy study of Brainology, a program that promotes students’ motivation and learning. We found that, although teachers fluctuated in their quality of delivery across lessons, students who received the intervention with higher quality of delivery on average increased more in targeted proximal outcomes (effort beliefs and learning goals) than students exposed to low quality. We discuss these results in terms of their implications for measuring quality of delivery, supporting teachers, and studying the conditions that make school interventions successful.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2022-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47574631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}