ORMA: A strategy to reduce Psychology's replication problems

Abstract

Because of their historical reliance upon null hypothesis statistical tests (NHST), the human sciences have developed a number of potentially problematic research literatures. While aware of the file drawer effect since the 1970s, scientists have been largely unsuccessful at addressing its pernicious effects. Because significant results have a greater likelihood of being published than do nonsignificant effects, many of our research literatures might currently be constructed upon a series of Type I errors and inflated effect sizes. A method (called Original Replication of Meta-Analyses or ORMA) has recently been developed for identifying problematic research literatures and offering a method to address the problems due to publication bias. Philosophers of science have long argued that a chief reason for science's preeminence as a source of knowledge rests in its ability to self-correct. Researchers in the human sciences are now able to empirically test their research literatures to ascertain which are in need of repair. The use of ORMA serves to lessen the problems that led to the recent calls for bans on significant/nonsignificant statistics in human science research. ORMA will also improve psychology's ability to successfully replicate its research findings.