Nonsignificance misinterpreted as an effect's absence in psychology: prevalence and temporal analyses.

Abstract

Nonsignificant findings in psychological research are frequently misinterpreted as reflecting the effect's absence. However, this issue's exact prevalence remains unclear, as does whether this issue is getting better or worse. In this pre-registered study, we sought to answer these questions by examining the discussion sections of 599 articles published across 10 psychology journals and three time points (2009, 2015 and 2021), and coding whether a nonsignificant finding was interpreted in such a way as to suggest the effect does not exist. Our models indicate that between 76% and 85% of psychology articles published between 2009 and 2021 that discussed a nonsignificant finding misinterpreted nonsignificance as reflecting no effect. It is likely between 54% and 62% of articles over this time period claimed explicitly that this meant no effect on the population of interest. Our findings also indicate that only between 4% and 8% of articles explicitly discussed the possibility that the nonsignificant effect may exist but could not be found. Differences in prevalence rates over time were nonsignificant. Collectively, our findings indicate this interpretative error is a major problem in psychology. We call on stakeholders with an interest in improving psychological science to prioritize tackling it.