Promoting conceptual replications in educational research

Abstract

Despite the abundant and frequent calls for replication studies from research communities (e.g., Shavelson & Towne, 2002) and funding agencies (e.g., Institute of Education Sciences [IES] & National Science Foundation [NSF], 2013), the number of such studies remains stubbornly small. For example, in an analysis of all articles published since 1900 in the top-10 psychological journals, Makel et al. (2012) found that less than 1% were replication studies. Moreover, from the top100 education journals, as ranked by a 5-year impact factor, Makel and Plucker (2014) found that only 0.13% of articles were replication studies, with most successful replications being authored by the same individuals who had carried out the initial studies. Among all research articles published in the Journal for Research in Mathematics Education (JRME) from its inception in 1970 through 2016, only about 3% clearly intended to replicate prior studies (Cai et al., 2018). And, at the IES in the United States, the majority of funded grant applications have not explicitly stated an intent to conduct a replication (Chhin et al., 2018). There are many reasons for the limited number of published replication studies. One major reason is a lack of clarity with respect to the nature of replications and their significance. There has, however, been increasing recognition of the importance of replication studies. Replication studies provide new knowledge and can help researchers, practitioners, and policymakers gain insights about which interventions improve (or do not improve) education outcomes, for whom, and under what conditions (Cai et al., 2018; NSF & IES, 2018). Perry et al. (2022) found that despite their small number, the rate of replication studies in education has gradually increased from 2011 to 2020. Fortunately, some journals have published special issues on replication studies, such as this special issue of Educational Research and Evaluation (ERE) and that in JRME (Cai et al., 2018). In addition, funding agencies such as the NSF and IES in the United States have explicitly called for grant proposals for replication studies (NSF & IES, 2018).