Assessment and mitigation of bias in influenza and COVID-19 vaccine effectiveness analyses — IVY Network, September 1, 2022–March 30, 2023

Abstract

Background

In test-negative studies of vaccine effectiveness (VE), including patients with co-circulating, vaccine-preventable, respiratory pathogens in the control group for the pathogen of interest can introduce a downward bias on VE estimates.

Methods

A multicenter sentinel surveillance network in the US prospectively enrolled adults hospitalized with acute respiratory illness from September 1, 2022–March 31, 2023. We evaluated bias in estimates of VE against influenza–associated and COVID-19–associated hospitalization based on: inclusion vs exclusion of patients with a co-circulating virus among VE controls; observance of VE against the co-circulating virus (rather than the virus of interest), unadjusted and adjusted for vaccination against the virus of interest; and observance of influenza or COVID-19 against a sham outcome of respiratory syncytial virus (RSV).

Results

Overall VE against influenza–associated hospitalizations was 6 percentage points lower when patients with COVID-19 were included in the control group, and overall VE against COVID-19–associated hospitalizations was 2 percentage points lower when patients with influenza were included in the control group. Analyses of VE against the co-circulating virus and against the sham outcome of RSV showed that downward bias was largely attributable the correlation of vaccination status across pathogens, but also potentially attributable to other sources of residual confounding in VE models.

Conclusion

Excluding cases of confounding respiratory pathogens from the control group in VE analysis for a pathogen of interest can reduce downward bias. This real-world analysis demonstrates that such exclusion is a helpful bias mitigation strategy, especially for measuring influenza VE, which included a high proportion of COVID-19 cases among controls.