Replication of experiments and the canonisation of incorrect conclusions

Abstract

We are in the middle of a revolution over how to collect, analyse and interpret results in the human behavioural sciences. A key issue is to determine the extent to which experimental results are repeatable. However, if the experiment being replicated lacks the appropriate controls or null hypotheses, then replication of experiments can lead to the canonisation of incorrect conclusions. Experiments are required that allow a strong possibility to falsify hypotheses or allow competing hypotheses to be tested in an unbiased manner. We illustrate this problem, and how it can be solved, with examples from the literature on public goods games, which has been used to test hypotheses about the evolution of cooperation in humans.