Characteristics of Vaccine Safety Observational Studies and Authors' Attitudes: A Systematic Review.

Background: Post-licensure observational studies are the mainstay of vaccine safety evaluation. However, these studies have well-known methodological limitations, rendering them particularly vulnerable to unmeasured confounding. We sought to describe high-impact observational studies of vaccine safety, investigate the authors' attitudes towards their study's findings and limitations, and report on spin practices.

Methods: We conducted a Pubmed systematic review of comparative observational studies of vaccine safety published in the six top medical journals from inception to March 2024.

Results: Thirty-seven studies were included, spanning publications from 1995 to 2024. Most studies focused on COVID19 and influenza vaccines (n=11, 30%, and n=10, 27%, respectively). Study designs and methodologies varied. Electronic health records (54%), passive surveillance databases (32%) and national registries (27%) were the most common data sources. Negative control outcomes were used in a single study. Residual confounding was conceded in 54% of studies, and an additional 24% did so implicitly. Spin was noted in 48.6% of the studies. This systematic review found that authors of observational vaccine safety studies in high-impact medical journals often acknowledge residual confounding, but rarely use methods like negative control outcomes to better detect unmeasured confounding. Furthermore, spin is common, occurring in approximately 50% of the studies.

Conclusions: Although our findings are somewhat limited by subjectivity in study assessments, they suggest that editors and reviewers of high-impact journals should ensure the language used in reporting observational studies accurately reflects the findings and their limitations.