Application of methodological strategies to address unmeasured confounding in real-world vaccine safety and effectiveness study: a systematic review

Abstract

Objectives

Uses of real-world data to evaluate vaccine safety and effectiveness are often challenged by unmeasured confounding. The study aimed to review the application of methods to address unmeasured confounding in observational vaccine safety and effectiveness research.

Study Design and Setting

We conducted a systematic review (PROSPERO: CRD42024519882), and searched PubMed, Web of Science, Embase, and Scopus for epidemiological studies investigating influenza and COVID-19 vaccines as exposures, and respiratory and cardiovascular diseases as outcomes, published between January 1, 2017, and December 31, 2023. Data on study design and statistical analyses were extracted from eligible articles.

Results

A total of 913 studies were included, of which 42 (4.6%, 42/913) accounted for unmeasured confounding through statistical correction (31.0%, 13/42) or confounding detection or quantification (78.6%, 33/42). Negative control was employed in 24 (57.1%, 24/42) studies—2 (8.3%, 2/24) for confounding correction and 22 (91.7%, 22/24) for confounding detection or quantification—followed by E-value (31.0%, 13/42), prior event rate ratio (11.9%, 5/42), regression discontinuity design (7.1%, 3/42), instrumental variable (4.8%, 2/42), and difference-in-differences (2.4%, 1/42). A total of 871 (95.4%, 871/913) studies did not address unmeasured confounding, but 38.9% (355/913) reported it as study limitation.

Conclusion

Unmeasured confounding in real-world vaccine safety and effectiveness studies remains underexplored. Current research primarily employed confounding detection or quantification, notably negative control and E-value, which did not yield adjusted effect estimates. While some studies used correction methods like instrumental variable, regression discontinuity design, and negative control, challenges arise from the stringent assumptions. Future efforts should prioritize developing valid methodologies to mitigate unmeasured confounding.