Streamlining the self-correction process: a review of the use of replication research by organizational scholars

Abstract

Purpose

Replication is a primary self-correction device in science. In this paper, we have two aims: to examine how and when the results of replications are used in management and organization research and to use the results of this examination to offer guidelines for improving the self-correction process.

Design/methodology/approach

Study 1 analyzes co-citation patterns for 135 original-replication pairs to assess the direct impact of replications, specifically examining how often and when a replication study is co-cited with its original. In Study 2, a similar design is employed to measure the indirect impact of replications by assessing how often and when a meta-analysis that includes a replication of the original study is co-cited with the original study.

Findings

Study 1 reveals, among other things, that a huge majority (92%) of sources that cite the original study fail to co-cite a replication study, thus calling into question the impact of replications in our field. Study 2 shows that the indirect impact of replications through meta-analyses is likewise minimal. However, our analyses also show that replications published in the same journal that carried the original study and authored by teams including the authors of the original study are more likely to be co-cited, and that articles in higher-ranking journals are more likely to co-cite replications.

Originality/value

We use our results to formulate recommendations that would streamline the self-correction process in management research at the author-, reviewer- and journal-level. Our recommendations would create incentives to make replication attempts more common, while also increasing the likelihood that these attempts are targeted at the most relevant original studies.