Accountability in evidence syntheses: On the need for rigorous peer review and reporting guidelines.

Abstract

It is an unfortunate reality that metascience and metaresearch are not immune to methodological or reporting issues. Best practices for conducting and reporting evidence syntheses are well-established, and we expect them to be enforced by this journal. Thus, we were surprised to see a recent "systematic review" published in Accountability in Research state that the first article published about retractions was from 1998. As we know of several potentially relevant articles published prior to 1998, we looked deeper into the review and its methodology. Because of the methodological and reporting flaws throughout the review, which we describe in this commentary, the conclusions it presents overstate the certainty and completeness of the findings, and the review includes inaccuracies. That these issues were seemingly not addressed prior to publication highlights the importance of involving individuals with methodological expertise, including search expertise, in the review process, and the necessity of adhering to the appropriate reporting guidelines to ensure accuracy, credibility, and reproducibility. We call on Accountability in Research to ensure that evidence syntheses are peer reviewed by individuals with appropriate methodological expertise; to ensure that authors are adhering to appropriate reporting guidelines; and to consider requiring study preregistration and presubmission inquiries for evidence syntheses.