Policy and Scientific Implications of Compromised Replications

ABSTRACT Replications and the sharing of data with other researchers have become an essential part of the scientific process, and it is an aspect of the social sciences that has grown exponentially over the last few decades and is now required for most federally funded studies. Unfortunately, there are few guidelines or narratives for unexpected complications between replicators and those providing the data sets. The current article is a case study regarding a replication of a study of 270 criminally-justice involved individuals with substance use disorders, who were randomized either to self-help recovery homes, professionally led therapeutic communities or a control condition. The replicators and the original investigators arrived at different conclusions regarding critical post-incarceration outcomes. These types of discrepancies can have critical policy implications particularly as there are so few randomized studies examining post-incarceration outcomes for those released from jails and prisons. The contrasting conclusions were, in part, due to differences in a variety of conceptual and methodological issues. To avoid these types of difficulties, we propose several types of recommendations designed to enhance the scientific process for replication studies and secondary use data sets.