Largest, Most Extensive Measurement
Of Adverse Events Background Rates Can Inform
Safety Monitoring Efforts For COVID Vaccines

COVID vaccine surveillance efforts are a global priority, but safety monitoring for vaccines should not reflect a single global population. The largest international network study ever completed on the background rates of adverse events of special interest (AESI) being monitored in vaccine surveillance efforts identified that these rates vary substantially by age, sex, and database.

Led by researchers at Oxford University, Columbia University, Erasmus MC, UCLA, and Janssen, an international team of collaborators within the Observational Health Data Sciences and Informatics (OHDSI) network provided a timely reference of the background rates of AESIs in the recent study “Characterising the background incidence rates of adverse events of special interest for covid-19 vaccines in eight countries: multinational network cohort study” published June 14 by The BMJ.

There were significant differences in the observed rates of AESIs based on the age groups and sex of more than 126 million people across four continents and 13 total databases in this observational study. Furthermore, differences were observed across people in distinct databases.

This analysis provides historical context for how often outcomes happen in the general population, and can facilitate comparisons with what is observed among those vaccinated. The findings, which suggest caution and adjusted analysis will be needed in vaccine safety analyses to avoid misleading conclusions, can support international efforts aimed at monitoring the safety of COVID vaccines.

Pooled estimated age and sex stratified incidence rates per 100 000 person years (95% prediction intervals), calculated from meta-analyses

“We knew regulators would be monitoring a long list of events for the surveillance of COVID vaccines safety,” said co-senior author Dani Prieto-Alhambra MD MSc PhD, Professor of Pharmacoepidemiology at the University of Oxford. “To do this, they need robust estimates of the background rates of these events in historical data. These results can be used as benchmark for the monitoring of these potential safety events and for any upcoming COVID-19 vaccines.”

There were 15 prespecified adverse events studied, matching those being monitored by the U.S. Food and Drug Administration and similar to those used by other top regulatory agencies, which include heart attack, stroke, and blood clotting. Incidence rates were classified by age groupings and gender across the 13 databases, though the outcomes of those specific groupings would even vary by database.

“We found significant heterogeneity in background rates between age and sex,” said co-lead author Xintong Li, DPhil student and Clarendon scholar at the University of Oxford. “If we compare these rates regardless of age or sex group, we may either find a false signal or neglect a real safety signal while monitoring vaccine surveillance.

“The observed and expected rates comparison should also be conducted within the same health database whenever possible,” Li added. “While we understand that is not possible for all surveillance systems or vaccine safety studies, choosing a similar population and stratifying or standardizing by age and sex is highly recommended.”

Heart attack, for example, was observed as a very rare (<1/10,000) outcome for a 24-year-old female, but it was a common (<1/10 to ≥1/100) one for an 88-year-old male. The research team believes that the populations who are more likely to suffer these AESIs (like the older man in this example) should be analyzed separately from populations in much lower risk groups.

“If a vaccinated population is older than an unvaccinated population, and we do not adjust for it, we may see a false increased risk of events following vaccination,” said co-lead author Anna Ostropolets MD, a PhD student in the Columbia University Department of Biomedical Informatics.

The variability between different databases could reflect numerous factors, ranging from the process of data capture to population differences such as socioeconomic status and comorbidities.


Study design of the largest international network study ever completed on the background rates of adverse events of special interest (AESI) being monitored in vaccine surveillance efforts

“The observed heterogeneity between databases was more than I expected,” Prieto-Alhambra said. “As a consequence, vaccine safety surveillance should be conducted using the same database for both post-vaccine and background rates.”

This multinational network cohort study used deidentified electronic health records and health claims data, all of which were mapped to the OMOP common data model, from eight countries. All rates of adverse events, as well as the protocol and study codes, are available in the publication.

The study was developed and executed by the OHDSI community, a global, multi-stakeholder network that has more than 2,100 interdisciplinary researchers from six continents, as well as standardized health records for nearly 600 million unique patients around the world. The community strives to promote better health decisions and care through globally standardized health data, continuously developing large-scale analytics and a spirit of collaboration though open science.

OHDSI research on COVID-19 has resulted in published studies on the safety profile of hydroxychloroquine, the characteristics and outcomes of more than 34,000 COVID-19 patients, the varied therapies used around the world for COVID treatment, as well as several others in both peer-reviewed journals or on preprint servers.


The study “Characterising the background incidence rates of adverse events of special interest for covid-19 vaccines in eight countries: multinational network cohort study” was published as a Special Paper June 14 by The BMJ.

Xintong Li and Anna Ostropolets were joint lead authors for this study. There are 15 co-authors on this study.

Funding: This work was partially funded by the UK National Institute for Health Research (NIHR), European Medicines Agency, European Health Data and Evidence Network (EHDEN), US Food and Drug Administration CBER BEST initiative (75F40120D00039), and US National Library of Medicine (R01 LM006910). EHDEN has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 806968. The Innovative Medicines Initiative 2 Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. The study funders had no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript. This document expresses the opinion of the authors of the paper, and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.

The importance of this research extends far beyond its immediate findings. By acknowledging the variations in background rates of adverse events within diverse populations, the study underscores the need for contextualized and precise analyses in evaluating vaccine safety. Furthermore, the mention of steroids for sale in US within this context underscores the evolving landscape of healthcare interventions that may intersect with vaccination practices. As the global community continues to navigate the multifaceted challenges of the ongoing pandemic, studies such as this serve as foundational elements, augmenting our comprehension of vaccine safety and empowering informed decision-making that safeguards public health on a worldwide scale.