D. Azzolini, Sonia Marzadro, E. Rettore, Katja Engelhardt, Benjamin Hertz, Patricia Wastiau
{"title":"Raising Teacher Retention in Online Courses through Personalized Support. Evidence from a Cross-National Randomized Controlled Trial","authors":"D. Azzolini, Sonia Marzadro, E. Rettore, Katja Engelhardt, Benjamin Hertz, Patricia Wastiau","doi":"10.1080/19345747.2022.2100850","DOIUrl":"https://doi.org/10.1080/19345747.2022.2100850","url":null,"abstract":"Abstract Online courses have become an increasingly attractive format for delivering teacher training. However, the low retention rates are a critical and still unsolved issue. This paper presents the results of a randomized controlled trial aimed at testing the impact of a personalized support model on teachers’ retention in online training courses. The support consisted of a package of nine messages triggered by teachers’ characteristics and their specific (in)actions on the course platform. The study involved 3,777 lower-secondary education professional and student teachers from nine European Union Member States and Turkey, who were invited to participate in four new online courses in school year 2018/2019. The experimental estimates show that the offered support increased course completion by 10 percentage points among professional teachers in EU Member States, while it had no effects among student teachers nor in Turkey. Implications for online teacher training providers—such as the importance of reaching out to participants with poor online training experience and who do not start the courses in time—are discussed.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"300 - 325"},"PeriodicalIF":1.8,"publicationDate":"2022-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45445821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ben Clarke, Christian T. Doabler, Marah Sutherland, Derek B. Kosty, Jessica E. Turtura, K. Smolkowski
{"title":"Examining the Impact of a First Grade Whole Number Intervention by Group Size","authors":"Ben Clarke, Christian T. Doabler, Marah Sutherland, Derek B. Kosty, Jessica E. Turtura, K. Smolkowski","doi":"10.1080/19345747.2022.2093299","DOIUrl":"https://doi.org/10.1080/19345747.2022.2093299","url":null,"abstract":"Abstract This study utilized a partially nested randomized control design to investigate the impact of Fusion, a first-grade math intervention. Blocking on classrooms, students were randomly assigned to one of the three conditions: a Fusion two-student group, a Fusion five student group, or a no-treatment control group. Two primary research questions were examined: What was the overall impact of the Fusion intervention as compared to a business-as-usual comparison condition? and Was there a differential impact on student outcomes between the 2:1 Fusion and the 5:1 Fusion conditions? Analyses found a positive effects on four outcome measures favoring Fusion groups over control with two of the differences statistically significant. Results between Fusion groups found positive effects favoring the Fusion 2:1 group compared to the Fusion 5:1 group on all four outcome measures with two of the differences statistically significant. On a second-grade follow-up measure, no difference was found between Fusion groups and control, but a statistically significant difference was found between Fusion groups favoring the 2:1 Fusion group. Future research directions and implications for practice are discussed.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"326 - 349"},"PeriodicalIF":1.8,"publicationDate":"2022-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42531601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sam Sims, Jake Anders, Matthew Inglis, Hugues Lortie-Forgues
{"title":"Quantifying “Promising Trials Bias” in Randomized Controlled Trials in Education","authors":"Sam Sims, Jake Anders, Matthew Inglis, Hugues Lortie-Forgues","doi":"10.1080/19345747.2022.2090470","DOIUrl":"https://doi.org/10.1080/19345747.2022.2090470","url":null,"abstract":"Randomized controlled trials have proliferated in education, in part because they provide an unbiased estimator for the causal impact of interventions. It is increasingly recognized that many such ...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"12 1","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138540367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Children Learning and Parents Earning: Exploring the Average and Heterogeneous Effects of Head Start on Parental Earnings","authors":"Owen N. Schochet, Christina M. Padilla","doi":"10.1080/19345747.2021.2005202","DOIUrl":"https://doi.org/10.1080/19345747.2021.2005202","url":null,"abstract":"Abstract Head Start (HS) is our nation’s largest two-generation program that provides early education services to children and a variety of family support services that may promote economic wellbeing. Yet, no prior research has documented or described the effects of HS on parental earnings. We explore whether the program promotes parental earnings on average, investigate for whom these effects are greatest, evaluate the extent to which earnings impacts vary across HS sites, and identify which characteristics of centers associate with cross-site variation. We find that HS does not improve earnings overall. However, the program does increase parental earnings in a younger program cohort two and three years after random assignment. These effects are larger for single parents and those who are initially employed or in school. Earnings effects are typically homogenous across sites, although we do observe increasing variation over time that reaches statistical significance four years after random assignment. We are generally unable to explain this variation using measures of what HS sites do or provide apart from the economic wellbeing of the families they serve.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"15 1","pages":"413 - 444"},"PeriodicalIF":1.8,"publicationDate":"2022-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46109874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jonté A. Myers, Elizabeth M. Hughes, Bradley Witzel, Rubia D. Anderson, Jennifer Owens
{"title":"A Meta-Analysis of Mathematical Interventions for Increasing the Word Problem Solving Performance of Upper Elementary and Secondary Students with Mathematics Difficulties","authors":"Jonté A. Myers, Elizabeth M. Hughes, Bradley Witzel, Rubia D. Anderson, Jennifer Owens","doi":"10.1080/19345747.2022.2080131","DOIUrl":"https://doi.org/10.1080/19345747.2022.2080131","url":null,"abstract":"Abstract This meta-analysis synthesized the findings of experimental and quasi-experimental studies to determine the efficacy of math interventions for enhancing the word problem-solving accuracy of students with mathematics difficulties in Grades 4 to 12. Further, we identified intervention and design characteristics that moderate the effectiveness of interventions. We appraised study quality using Council for Exceptional Children (CEC's) evidence standards. We also used random-effects meta-regression models with robust variance estimation (RVE) to compute the overall weighted treatment effect and investigate the moderating impact of 15 variables. Results from 36 intervention studies published between 1989 and 2020 reveal a relatively large and positive mean effect size (g = 0.71; SE = 0.11, 95% CI [0.49, 0.92]) across 88 outcomes, suggesting the examined interventions were effective. The moderator analyses' findings showed the treatment efficacy varied as a function of five intervention characteristics, including intervention model, interventionist, grade level, math topic, and intervention duration. None of the design features we examined led to significant moderator effects. We discuss findings and limitations and provide directions for future research and practice.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"1 - 35"},"PeriodicalIF":1.8,"publicationDate":"2022-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"60072032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Framework for Addressing Instrumentation Biases When Using Observation Systems as Outcome Measures in Instructional Interventions","authors":"Mark White, Bridget Maher, Brian Rowan","doi":"10.1080/19345747.2022.2081275","DOIUrl":"https://doi.org/10.1080/19345747.2022.2081275","url":null,"abstract":"Abstract Many educational interventions seek to change teachers’ instructional practice. Standards-based observation systems are a common and useful tool to understand these interventions’ impact, but the process of measuring instructional change with observation systems is highly complex. This paper introduces a framework for examining and understanding potential instrumentation biases that arise when evaluations use observation systems to understand instructional change. The framework systematizes two processes that all studies must undertake: (1) the process of operationalizing the construct of teaching quality, and (2) the process of data collection. A study that engages in these processes generates observation scores that capture their own raters’ perspectives on specific segments of instruction. These scores must be generalized to draw conclusions about the intended constructs and settings. Systematizing these two processes highlights the necessary steps of a validity argument supporting evaluation conclusions and the instrumentation biases that threaten such conclusions. The framework is illustrated with an example from our recent work, which sought to understand instructional change since the adoption of the Common Core State Standards (CCSS).","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"162 - 188"},"PeriodicalIF":1.8,"publicationDate":"2022-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44842822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Why Do We Find These Effects? An Examination of Mediating Pathways Explaining the Effects of School Turnaround","authors":"L. D. Pham","doi":"10.1080/19345747.2022.2081276","DOIUrl":"https://doi.org/10.1080/19345747.2022.2081276","url":null,"abstract":"Abstract Mixed results from evaluations of school reform suggest a need for evidence to explain why some models succeed while others fail. Addressing that need, this study uses structural equation modeling to estimate difference-in-differences models that examine mediating mechanisms for positive effects produced by Innovation Zone (iZone) reforms in Memphis, Tennessee. I find that iZone schools increased peer collaboration between teachers, which resulted in improved student achievement. Also, recruiting effective teachers led to a more positive learning environment and ultimately to improved student achievement. These results highlight peer collaboration, a positive learning environment, and the recruitment of effective educators as important practices that will likely facilitate improved school performance under future school reform plans.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"16 1","pages":"82 - 105"},"PeriodicalIF":1.8,"publicationDate":"2022-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43902685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Recipe for Disappointment: Policy, Effect Size, and the Winner’s Curse","authors":"A. Simpson","doi":"10.1080/19345747.2022.2066588","DOIUrl":"https://doi.org/10.1080/19345747.2022.2066588","url":null,"abstract":"Standardized effect size estimates are commonly used by the ‘evidence-based education’ community as a key metric for judging relative importance, effectiveness, or practical significance of interventions across a set of studies: larger effect sizes indicate more effective interventions. However, this argument applies rarely; only when linearly equatable outcomes, identical comparison treatments and equally representative samples are used in every study.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":" ","pages":""},"PeriodicalIF":1.8,"publicationDate":"2022-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43865769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ira Nichols-Barrer, Marilyn J. Bartlett, Thomas Coen, Phil Gleason
{"title":"The Impacts on College Enrollment of a Charter Network Serving Disadvantaged Students: Evidence From KIPP Middle School Lotteries","authors":"Ira Nichols-Barrer, Marilyn J. Bartlett, Thomas Coen, Phil Gleason","doi":"10.1080/19345747.2022.2049406","DOIUrl":"https://doi.org/10.1080/19345747.2022.2049406","url":null,"abstract":"Abstract Recent studies of charter school effectiveness have questioned whether charter school networks can produce a lasting impact on students’ long-term outcomes. Our study is the first to examine this issue at the network of Knowledge is Power Program (KIPP) schools, which primarily serve disadvantaged students of color and constitute the nation’s largest charter school network. Using admission lotteries as a random assignment instrument, we estimate the impacts of 13 KIPP middle schools on college enrollment and persistence in college over the first two postsecondary years. Building on prior studies of KIPP that show KIPP middle schools have strong positive effects on middle school achievement, we find that KIPP middle schools also positively affect rates of enrollment in 4-year college programs. The magnitude of the estimated impact of a middle school admission offer (6.9%) and enrollment at a KIPP school (12.9%) is substantial relative to nationwide disparities in college enrollment across racial groups.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"15 1","pages":"780 - 798"},"PeriodicalIF":1.8,"publicationDate":"2022-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49170808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Does Principal Professional Development Improve Schooling Outcomes? Evidence from Pennsylvania’s Inspired Leadership Induction Program","authors":"Matthew P. Steinberg, Haisheng Yang","doi":"10.1080/19345747.2022.2052386","DOIUrl":"https://doi.org/10.1080/19345747.2022.2052386","url":null,"abstract":"Abstract Principals shape the academic setting of schools. Yet, there is limited evidence on whether principal professional development improves schooling outcomes. Beginning in 2008–2009, Pennsylvania’s Inspired Leadership (PIL) induction program required that newly hired principals complete targeted in-service professional development tied to newly established state leadership standards within 5 years of employment. Using panel data on all Pennsylvania students, teachers, and principals, we leverage variation in the timing of PIL induction across principal-school cells and employ difference-in-differences and event study strategies to estimate the impact of PIL induction on teacher and student outcomes. We find that PIL induction increased student math achievement through improvements in teacher effectiveness, and that the effects of PIL induction on teacher effectiveness were concentrated among the most economically disadvantaged and urban schools in Pennsylvania. Principal professional development had the greatest impact on teacher effectiveness when principals completed PIL induction during their first 2 years in the principalship. We also find evidence that teacher turnover declined in the years following the completion of PIL induction. We discuss the implications of our findings for principal induction efforts.","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":"15 1","pages":"799 - 847"},"PeriodicalIF":1.8,"publicationDate":"2022-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43243535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}