An umbrella review reveals that control variables are rarely considered as a source of heterogeneity in systematic reviews of observational studies.

Abstract

Objectives: The effect estimates in systematic reviews of observational studies often exhibit high heterogeneity. A potentially important source of heterogeneity are differences in the control variables across observational studies. However, it remains unclear how often this source of heterogeneity is considered in practice. The objective of this umbrella review is to determine how often control for different sets of variables across primary studies was considered as a source of heterogeneity in published systematic reviews of observational epidemiologic studies.

Study design and setting: We systematically searched for systematic reviews of observational studies published in a quartile 1 Web of Science or Scopus indexed epidemiology journal between January 1, 2023, and December 31, 2023. Eligibility screening, data extraction, and quality appraisal were performed by two independent reviewers. Data was summarized using descriptive statistics.

Results: Eligibility criteria were met by 297 systematic reviews, of which a random sample of 50 systematic reviews were included in this umbrella review. Differences in confounder sets were mentioned as a potential source of heterogeneity in 5/50 reviews (10.0%), differences in covariate sets in 4/50 reviews (8.0%), control for mediators in 0/50 reviews (0.0%), and control for colliders in 0/50 reviews (0.0%).

Conclusion: While differences in control for confounders, mediators, and colliders may explain heterogeneity in systematic reviews of observational studies, these sources of heterogeneity are rarely considered in practice. To avoid invalid pooled effect estimates, it is important that future systematic reviews of observational studies assess these potential sources of heterogeneity.