Meta-analysis of Nonrandomized Controlled Trials Is Rarely Justified: Systematic Reviews Must Avoid Improper Pooling

Abstract

Systematic reviews of the literature, as compared with original scientific articles, are the easiest types of studies to perform, and using contemporary meta-analysis software, the press of a button yields a “pooled weighted mean” (averaging the outcomes of the included articles and adjusting for sample size). The results seem conclusive. However, if included studies are not homogeneous and/or are of lower quality (high risk of bias), which is typical of nonrandomized trials, synthesis in a meta-analysis is not recommended, and quantitative pooling of nonrandomized studies is improper. In addition, by exploring clinical and methodologic differences (heterogeneity) between studies included in a systematic review, we discover and reveal reasons for the differences in outcomes among studies. This allows us to more accurately inform individual patient care and future research.