Assessing the process reproducibility of meta-analyses published in the top 20 pathology journals: A cross-sectional study.

Abstract

Objective: The objective of this study is to investigate the rigor of reporting and the potential for process reproducibility of meta-analyses published within top pathology journals.

Methods: This cross-sectional, meta-research study assessed eligible systematic reviews with meta-analysis indexed in MEDLINE through PubMed. We included those studies that were published within the top 20 pathology journals (h-5 index) from inception to March 21, 2024. We extracted proper reporting variables across 4 key quantitative synthesis domains: (1) primary study eligibility, (2) search strategy, (3) screening and extraction methods, and (4) quantitative synthesis approach.

Results: We found 282 studies eligible for masked duplicate data extraction. Less than half of studies (40.8% ± 2.9%) reported whether unpublished literature was eligible for inclusion, while less than 20% reported the date of their database search (18.8% ± 2.3%). Similarly, less than 20% reported a full, reproducible search strategy (19.1% ± 2.3%). Not all studies reported primary study effects (92.9% ± 1.5%). The reported use or mention of a relevant synthesis reporting guideline was associated with significant improvement in reporting of search factors (P < .001) and screening factors (P < .001). Nine meta-analyses (9 of 282; 3.2%) were deemed potentially process-reproducible.

Conclusions: Fewer than 10 meta-analyses from top pathology journals were potentially process-reproducible without reasonable effort. Most individual summary estimates were reproducible due to the presence of forest plots. Nevertheless, reproducibility factors related to search strategies are the single largest hindrance to reproducible meta-analyses published within our sample.