Novelty, Consistency, Transparency: The Trilemma of Psychological Sciences and its Consequences on Open Science Practices.

Abstract

The past decade has emphasised the importance of transparency for robust psychological research. However, transparent research has a cost, and it is hardly compatible with both conceptual novelty and statistical consistency across multiple studies. We propose that these three criteria can be conceptualized as a trilemma: fulfilling two of them considerably reduces the probability of satisfying the third one. An article testing a novel idea and transparently reporting evidence is likely to include empirical failure that impede consistency. An article transparently reporting consistent findings probably will acknowledge a replication effort that does not seek theoretical advances. Finally, an article presenting consistent evidence through multiple studies for a novel idea is not likely to be transparent. At a practical level, we argue that the pressure of the trilemma poses a threat for transparency, which is less tangible and historically important in the evaluation of research articles than the two other criteria. While the open science movement grows in importance, the pressure of the trilemma may encourage an opportunistic use of open science practices as a form of virtue signalling compensating for low transparency. Stakeholders, from editors to reviewers, should be aware of the constraints posed by transparency to continue improving the robustness of psychological science and avoiding a deleterious use of open science practices. We review potential solutions to break the pressure of the trilemma.