Systematic reviews on the same topic are common but often fail to meet key methodological standards: a research-on-research study.

Abstract

Objectives: To assess 1)the frequency of overlapping systematic reviews (SRs) on the same topic including overlap in outcomes, 2)whether SRs meet key methodological characteristics and 3)to describe discrepancies in results.

Study design and setting: For this research-on-research study, we gathered a random sample of SRs with meta-analysis (MA) published in 2022, identified the questions they addressed and, for each question, searched all SRs with MA published from 2018 to 2023 to assess the frequency of overlap. We assessed whether SRs met a minimum set of 6 key methodological characteristics: protocol registration, search of major electronic databases, search of trial registries, double selection and extraction, use of the Cochrane Risk-of-Bias tool and GRADE assessment.

Results: From a sample of 107 SRs with MA published in 2022, we extracted 105 different questions and identified 123 other SRs with MA published from 2018 to 2023. There were overlapping SRs for 33 questions (31.4%, 95% CI: 22.9-41.3), with a median of three overlapping SRs per question (interquartile range 2-6; range 2-19). Of the 230 SRs, 15 (6.5%) met the minimum set of 6 key methodological characteristics, and 12 (11.4%) questions had at least one SR meeting this criterion. Among the 33 questions with overlapping SRs, for 7 (21.2%), the SRs had discrepant results.

Conclusions: One-third of the SRs published in 2022 had at least one overlapping SR published from 2018 to 2023, and most did not meet a minimum set of methodological standards. For one-fifth of the questions, overlapping SRs provided discrepant results.