Analysing Adverse Event Databases: Principles, Challenges, and Examples

Abstract

Background

Databases of reported adverse events after vaccination are used to detect alarming signals by qualitative methods (case series) and quantitative methods (the proportional reporting ratio).

Objective

This methodological-empirical paper explores several key questions: How useful are these databases for detecting alarming signals? To which study design do they correspond? Which measure of association should be computed? Which key biases might operate, and what can be done to avoid or reduce them?

Methods

A contemporary methodological tool—causal diagrams—was used to answer these questions. The analytical approach was demonstrated for three possible outcomes of Covid vaccines: Thrombosis with Thrombocytopenia Syndrome, Guillain-Barré Syndrome, and reported death.

Results

A database of reported adverse events corresponds to a case-control study and should be analysed accordingly. The preferred measure of association is the odds ratio, not the proportional reporting ratio. Reporting bias operates to overestimate the true odds ratio, whereas control selection bias operates in the opposite direction (underestimation). As illustrated by three examples of reported death, the magnitude of the biases depends on the choice of the reference vaccine. However, extended methodological and empirical work is needed.

Conclusions

Databases of reported adverse events after vaccination are a rich source for quantitative research, provided that several methodological guidelines are followed. These databases should be analysed according to the principles of a case-control study, and the inference should be drawn on a case-by-case basis. It is crucial to estimate the reporting accuracy of a particular event by the type of vaccine, and only a clinical evaluation of a sample of records can provide this information.