{"title":"Quantifying and Predicting Variation in the Medium-Term Effects of an Oversubscribed Prekindergarten Program","authors":"Rebecca Unterman, Christina Weiland","doi":"10.1080/19345747.2023.2286280","DOIUrl":"https://doi.org/10.1080/19345747.2023.2286280","url":null,"abstract":"While there is a consensus that attending prekindergarten better prepares children for kindergarten and that effects can last into elementary school, there is little evidence on variation in these ...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140147974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Matthew Ronfeldt, Emanuele Bardelli, Matthew Truwit, Kevin Schaaf, Julie Baker
{"title":"Mentors Matter Recruitment Replication & Extension: Investigating Effects Across Implementation Years","authors":"Matthew Ronfeldt, Emanuele Bardelli, Matthew Truwit, Kevin Schaaf, Julie Baker","doi":"10.1080/19345747.2023.2287615","DOIUrl":"https://doi.org/10.1080/19345747.2023.2287615","url":null,"abstract":"The Mentors Matter Recruitment initiative leveraged state administrative data to recommend and successfully recruit more instructionally effective and experienced teachers to serve as clinical ment...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139470068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Statistical Power Analysis for Univariate Meta-Analysis: A Three-Level Model","authors":"Bixi Zhang, Spyros Konstantopoulos","doi":"10.1080/19345747.2023.2290544","DOIUrl":"https://doi.org/10.1080/19345747.2023.2290544","url":null,"abstract":"This study extends prior work on power analysis in two-level meta-analysis and provides methods on power analysis for univariate three-level meta-analysis. In a three-level hierarchical structure e...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139374043","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Steven L. Kramer, Michael A. Posner, Alexander S. Browman, Nancy R. Lawrence, Jennifer Roem, Kathleen Krier
{"title":"The Impacts of a Standards-Based Grading System Emphasizing Formative Assessment, Feedback, and Re-Assessment: A Mixed Methods, Cluster Randomized Control Trial in Ninth Grade Mathematics Classrooms","authors":"Steven L. Kramer, Michael A. Posner, Alexander S. Browman, Nancy R. Lawrence, Jennifer Roem, Kathleen Krier","doi":"10.1080/19345747.2023.2287594","DOIUrl":"https://doi.org/10.1080/19345747.2023.2287594","url":null,"abstract":"We investigated the impact in ninth-grade mathematics classrooms of Proficiency-based Assessment and Reassessment of Learning Outcomes (PARLO), a standards-based grading system. Key components of P...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139374002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reading Aloud to Children, Social Inequalities and Vocabulary Development: Evidence from a Randomized Controlled Trial","authors":"Carlo Barone, Denis Fougère, Karine Martel","doi":"10.1080/19345747.2023.2283475","DOIUrl":"https://doi.org/10.1080/19345747.2023.2283475","url":null,"abstract":"This article presents the results of an RCT assessing the impact of a Shared Book Reading (SBR) intervention that targeted children aged 4 attending kindergarten. We selected a large, random sample...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139376707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Intervention Effect or Measurement Artifact? Using Invariance Models to Reveal Response-Shift Bias in Experimental Studies","authors":"Margarita Olivera-Aguilar, Samuel H. Rikoon","doi":"10.1080/19345747.2023.2284768","DOIUrl":"https://doi.org/10.1080/19345747.2023.2284768","url":null,"abstract":"One aspect usually overlooked in studies evaluating the efficacy of educational interventions, is the extent to which posttest responses reflect actual changes in targeted latent constructs versus ...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139374037","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Redefining Populations of Inference for Generalizations from Small Studies","authors":"Wendy Chan, Jimin Oh, Katherine J. Wilson","doi":"10.1080/19345747.2023.2290546","DOIUrl":"https://doi.org/10.1080/19345747.2023.2290546","url":null,"abstract":"Over the past decade, propensity score-based methods have made an important contribution to the improvement of generalizations from educational studies. However, an important limitation is that man...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139374072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Estimating Treatment Effects with the Explanatory Item Response Model","authors":"Joshua B. Gilbert","doi":"10.1080/19345747.2023.2287601","DOIUrl":"https://doi.org/10.1080/19345747.2023.2287601","url":null,"abstract":"This simulation study examines the characteristics of the Explanatory Item Response Model (EIRM) when estimating treatment effects when compared to classical test theory (CTT) sum and mean scores a...","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139373885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wilhelmina van Dijk, Christopher Schatschneider, Stephanie Al Otaiba, Sara A Hart
{"title":"Student Behavior Ratings and Response to Tier 1 Reading Intervention: Which Students Do Not Benefit?","authors":"Wilhelmina van Dijk, Christopher Schatschneider, Stephanie Al Otaiba, Sara A Hart","doi":"10.1080/19345747.2023.2194894","DOIUrl":"10.1080/19345747.2023.2194894","url":null,"abstract":"<p><p>Core reading instruction and interventions have differential effects based on student characteristics such as cognitive ability and pre-intervention skill level. Evidence for differential effect based on affective characteristics is scant and ambiguous; however, students with problem behavior are more often non-responsive to core reading instruction and intensive reading interventions. In this study, we estimated the range of students' behavior ratings in which a core reading instruction intervention was effective using a data set including 3,024 students in K-3. Data came from seven independent studies evaluating the Individualized Student Instruction (ISI) Tier 1 reading intervention and were pooled using integrative data analysis. We estimated Johnson-Neyman intervals of student behavior ratings that showed a treatment effect both at the within and between classroom level. ISI was effective in improving reading scores (<i>b</i>=0.51, <i>p</i>=.020, <i>d</i> = 0.08). However, students with very low or very high behavior ratings did not benefit from the approaches (range of behavior rating factor scores: -0.95 - 2.87). At the classroom level, students in classrooms with a higher average of problem behaviors did not benefit from ISI (average classroom behavior rating factor score: 0.05 - 4.25). Results suggest differentiating instruction alone is not enough for students with behavior problems to grow in reading ability.</p>","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11309152/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45518336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robert B Olsen, Larry L Orr, Stephen H Bell, Elizabeth Petraglia, Elena Badillo-Goicoechea, Atsushi Miyaoka, Elizabeth A Stuart
{"title":"Using a Multi-Site RCT to Predict Impacts for a Single Site: Do Better Data and Methods Yield More Accurate Predictions?","authors":"Robert B Olsen, Larry L Orr, Stephen H Bell, Elizabeth Petraglia, Elena Badillo-Goicoechea, Atsushi Miyaoka, Elizabeth A Stuart","doi":"10.1080/19345747.2023.2180464","DOIUrl":"10.1080/19345747.2023.2180464","url":null,"abstract":"<p><p>Multi-site randomized controlled trials (RCTs) provide unbiased estimates of the average impact in the study sample. However, their ability to accurately predict the impact for individual sites outside the study sample, to inform local policy decisions, is largely unknown. To extend prior research on this question, we analyzed six multi-site RCTs and tested modern prediction methods-lasso regression and Bayesian Additive Regression Trees (BART)-using a wide range of moderator variables. The main study findings are that: (1) all of the methods yielded accurate impact predictions when the variation in impacts across sites was close to zero (as expected); (2) none of the methods yielded accurate impact predictions when the variation in impacts across sites was substantial; and (3) BART typically produced \"less inaccurate\" predictions than lasso regression or than the Sample Average Treatment Effect. These results raise concerns that when the impact of an intervention varies considerably across sites, statistical modelling using the data commonly collected by multi-site RCTs will be insufficient to explain the variation in impacts across sites and accurately predict impacts for individual sites.</p>","PeriodicalId":47260,"journal":{"name":"Journal of Research on Educational Effectiveness","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10914338/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42133979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}