Characterising the background incidence rates of adverse events of special interest for covid-19 vaccines in eight countries: multinational network cohort study
Open accessPublished version of a preprint
Abstract
BACKGROUND: As large-scale immunization programs against COVID-19 proceed around the world, safety signals will emerge that need rapid evaluation. We report population-based, age- and sex-specific background incidence rates of potential adverse events of special interest (AESI) in eight countries using thirteen databases. METHODS: This multi-national network cohort study included eight electronic medical record and five administrative claims databases from Australia, France, Germany, Japan, Netherlands, Spain, the United Kingdom, and the United States, mapped to a common data model. People observed for at least 365 days before 1 January 2017, 2018, or 2019 were included. We based study outcomes on lists published by regulators: acute myocardial infarction, anaphylaxis, appendicitis, Bell's palsy, deep vein thrombosis, disseminated intravascular coagulation, encephalomyelitis, Guillain-Barre syndrome, hemorrhagic and non-hemorrhagic stroke, immune thrombocytopenia, myocarditis/pericarditis, narcolepsy, pulmonary embolism, and transverse myelitis. We calculated incidence rates stratified by age, sex, and database. We pooled rates across databases using random effects meta-analyses. We classified meta-analytic estimates into Council of International Organizations of Medical Sciences categories: very common, common, uncommon, rare, or very rare. FINDINGS: We analysed 126,661,070 people. Rates varied greatly between databases and by age and sex. Some AESI (e.g., myocardial infarction, Guillain-Barre syndrome) increased with age, while others (e.g., anaphylaxis, appendicitis) were more common in young people. As a result, AESI were classified differently according to age. For example, myocardial infarction was very rare in children, rare in women aged 35-54 years, uncommon in men and women aged 55-84 years, and common in those aged ≥85 years. INTERPRETATION: We report robust baseline rates of prioritised AESI across 13 databases. Age, sex, and variation between databases should be considered if background AESI rates are compared to event rates observed with COVID-19 vaccines.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most observational analyses are based on comparing the risks of adverse events 'observed' against a background (or 'expected') risk, which inevitably display great variation, by database, age group, and sex.
“Most observational analyses are based on comparing the risks of adverse events ''observed' against a background (or ''expected') risk, which inevitably display great variation, by database, age group, and sex. [32]”
From Serious adverse events of special interest following mRNA COVID-19 vaccination in randomized trials in adults (2022) · cites this paper as [32]
The Discussion (page not provided) describes comparing observed event rates with historical background or ‘expected’ rates and reports notable differences by age, sex, and database. It does not establish that most observational analyses use this comparison, so that part of the claim is unsupported. (LLM, conf=0.85)
Experimental — accuracy not yet measuredThe OHDSI network has been conducting diverse international observational health research with RWD via the CDM and analytical tools, producing real-world evidence across cardiovascular, metabolic, immunology, oncology, and infectious diseases, including COVID-19-related research.
“The community has produced real-world evidence across a range of clinical areas, including cardiovascular, metabolic, immunology, oncology, and infectious diseases, inclusive of COVID-19-related research during the pandemic.5–8”
From Evaluating a novel approach to stimulate open science collaborations: a case series of “study-a-thon” events within the OHDSI and European IMI communities (2022) · cites this paper as [8]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,811 · see citing works
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
How to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827 · see citing works
WHO Coronavirus Disease (COVID-19) Dashboard
(2020) · Bangladesh Physiotherapy Journal · doi:10.46945/bpj.10.1.03.01 · Cited by 5,545 · see citing works
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884 · see citing works
Data Resource Profile: Clinical Practice Research Datalink (CPRD)
Herrett, E., Gallagher, A. M., Bhaskaran, K., Forbes, H., Mathur, R., van Staa, T., & Smeeth, L. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv098 · Cited by 2,667 · cited 1× · see citing works
“…of Catalonia, Spain; Clinical Practice Research Datalink (CPRD_GOLD_UK), which consists of data collected from UK primary care for all ages 10 ; Columbia University Irving Medical Center (CUMC_US), which covers more than 4.5 million people treated at the New York-Presbyterian Hospital/…”
InfluentialGeorge, H., Duke Jon, D., Shah Nigam, H., Reich Christian, G., Vojtech, H., Schuemie Martijn, J., Suchard Marc, A., Woong, P. R., Kei, W. I. C., Rijnbeek Peter, R., et al. (2015) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-564-7-574 · Cited by 1,107 · see citing works
Validation of a common data model for active safety surveillance research
Overhage, J. M., Ryan, P. B., Reich, C. G., Hartzema, A. G., & Stang, P. E. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000376 · Cited by 548 · see citing works
Self-citationCouncil for International Organizations of Medical Sciences (CIOMS)
(2025) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811297724_0006 · Cited by 543 · see citing works
Optum Labs: Building A Novel Node In The Learning Health Care System
Wallace, P. J., Shah, N. D., Dennen, T., Bleicher, P. A., & Crown, W. H. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0038 · Cited by 449 · see citing works
Suchard, M. A., Schuemie, M. J., Krumholz, H. M., You, S. C., Chen, R., Pratt, N., Reich, C. G., Duke, J., Madigan, D., Hripcsak, G., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(19)32317-7 · Cited by 359 · see citing works
Self-citationVoss, E. A., Makadia, R., Matcho, A., Ma, Q., Knoll, C., Schuemie, M. J., DeFalco, F., Londhe, A. A., Zhu, V. J., & Ryan, P. B. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu023 · Cited by 343 · see citing works
Self-citationCovid-19: European countries suspend use of Oxford-AstraZeneca vaccine after reports of blood clots
Wise, J. (2021) · BMJ · doi:10.1136/bmj.n699 · Cited by 327 · see citing works
Lane, J. C. E., Weaver, J., Kostka, K., Duarte-Salles, T., Abrahao, M. T. F., Alghoul, H., Alser, O., Alshammari, T. M., Biedermann, P., Banda, J. M., et al. (2020) · The Lancet Rheumatology · doi:10.1016/s2665-9913(20)30276-9 · Cited by 258 · see citing works
Self-citationBLACK, S. B., Eskola, J., Siegrist, C., Halsey, N. A., MacDonald, N. E., LAW, B. J., Miller, E., Andrews, N. J., Stowe, J., Salmon, D. A., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61877-8 · Cited by 254 · see citing works
Wijnans, L., Lecomte, C., de Vries, C. S., Weibel, D. M., Sammon, C., Hviid, A. P., Svanström, H., Mølgaard‐Nielsen, D., Heijbel, H., Arnheim‐Dahlström, L., et al. (2012) · Vaccine · doi:10.1016/j.vaccine.2012.12.015 · Cited by 230 · see citing works
García-Gil, M., Hermosilla, E., Prieto‐Alhambra, D., Avilés, F. F., Rosell, M., Ramos, R., Rodríguez, J. C., Williams, T. J., van Staa, T. P., & Bolíbar, B. (2011) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v19i3.806 · Cited by 225 · see citing works
Self-citationPostmarketing Surveillance Based on Electronic Patient Records: The IPCI Project
van der Lei, J., Mosseveld, M., van Wijk, M. A. M., van der Linden, P. D., Sturkenboom, M., van Bemmel, J. H., & Vlug, A. E. (1999) · Methods of Information in Medicine · doi:10.1055/s-0038-1634402 · Cited by 224 · see citing works
Hripcsak, G. M., Suchard, M. A., Shea, S., Chen, R., You, S. C., Pratt, N. L., Madigan, D., Krumholz, H. M., Ryan, P. B., & Schuemie, M. J. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.7454 · Cited by 139 · see citing works
Self-citationRates of autoimmune diseases in Kaiser Permanente for use in vaccine adverse event safety studies
Klein, N. P., Ray, P., Carpenter, D. M., Hansen, J. R., Lewis, E. M., FIREMAN, B. H., BLACK, S. B., Galindo, C., Schmidt, J., & Baxter, R. P. (2009) · Vaccine · doi:10.1016/j.vaccine.2009.10.115 · Cited by 136 · see citing works
Morales, D. R., Conover, M. M., You, S. C., Pratt, N. L., Kostka, K., Duarte‐Salles, T., Fernández‐Bertolín, S., Aragón, M., DuVall, S. L., Lynch, K. E., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30289-2 · Cited by 136 · see citing works
Self-citationActive Surveillance for Adverse Events: The Experience of the Vaccine Safety Datalink Project
Yih, W. K., Kulldorff, M., FIREMAN, B. H., Shui, I. M., Lewis, E. M., Klein, N. P., Baggs, J. M., Weintraub, E., Belongia, E. A., Naleway, A. L., et al. (2011) · PEDIATRICS · doi:10.1542/peds.2010-1722i · Cited by 127 · see citing works
Data resource profile: JMDC claims databases sourced from Medical Institutions
Nagai, K., Tanaka, T., Kodaira, N., et al. (2020) · Journal of General and Family Medicine · doi:10.1002/jgf2.367 · Cited by 120 · see citing works
The Role of the Vaccine Adverse Event Reporting System (VAERS) in Monitoring Vaccine Safety
Iskander, J. K., Miller, E. R., & Chen, R. T. (2004) · Pediatric Annals · doi:10.3928/0090-4481-20040901-11 · Cited by 77 · see citing works
BLACK, S. B., LAW, B. J., Chen, R. T., Dekker, C. L., Sturkenboom, M., Huang, W., Gurwith, M., & Poland, G. A. (2021) · Vaccine · doi:10.1016/j.vaccine.2021.03.016 · Cited by 74 · see citing works
Principles of Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND)
Schuemie, M. J., Ryan, P. B., Pratt, N. L., Chen, R., You, S. C., Krumholz, H. M., Madigan, D., Hripcsak, G. M., & Suchard, M. A. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa103 · Cited by 53 · see citing works
Self-citationPost-licensure rapid immunization safety monitoring program (PRISM) data characterization
Baker, M. A., Nguyen, M., Cole, D. V., Lee, G. M., & Lieu, T. A. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.04.088 · Cited by 50 · see citing works
Drawing Reproducible Conclusions from Observational Clinical Data with OHDSI
Hripcsak, G. M., Schuemie, M. J., Madigan, D., Ryan, P. B., & Suchard, M. A. (2021) · Yearbook of Medical Informatics · doi:10.1055/s-0041-1726481 · Cited by 45 · see citing works
Self-citationWillame, C., Dodd, C. N., van der Aa, L. M., Picelli, G., Emborg, H., Kahlert, J. A., Gini, R., Huerta, C., Martín‐Merino, E., McGee, C., et al. (2021) · Drug Safety · doi:10.1007/s40264-020-01031-1 · Cited by 43 · see citing works
Heininger, U., Holm, K., Caplanusi, I., et al. (2017) · Vaccine · doi:10.1016/j.vaccine.2017.06.033 · Cited by 33 · see citing works
Levison, L. S., Thomsen, R. W., Christensen, D. H., et al. (2019) · Clinical Epidemiology · doi:10.2147/clep.s199839 · Cited by 31 · see citing works
Rasmussen, T. A., Jorgensen, M. R. S., Bjerrum, S., Jensen-Fangel, S., Støvring, H., Østergaard, L. J., & Søgaard, O. S. (2012) · BMJ · doi:10.1136/bmj.e5823 · Cited by 28 · see citing works
Hense, S., Schink, T., Kreisel, S. H., et al. (2014) · Neuroepidemiology · doi:10.1159/000369344 · Cited by 24 · see citing works
Guillain‐Barré syndrome: background incidence rates in The Netherlands
van der Maas, N., Kramer, M. A., Jacobs, B. C., et al. (2011) · Journal of the Peripheral Nervous System · doi:10.1111/j.1529-8027.2011.00356.x · Cited by 19 · see citing works
Jouaville, S. L., Miotti, H., Coffin, G., Sarfati, B., & Meihoc, A. (2015) · Value in Health · doi:10.1016/j.jval.2015.03.115 · Cited by 16 · see citing works
Granieri, E., Andreasi, N. G., Martin, P. D., et al. (2019) · Neurological Sciences · doi:10.1007/s10072-018-3688-4 · Cited by 14 · see citing works
Merchant, H. A. (2021) · Huddersfield Research Portal (University of Huddersfield) · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [19]Black S, Eskola J, Siegrist C-A, Importance of background rates of disease in assessment of vaccine safety during mass immunisation with pandemic H1N1 influenza vaccines. Lancet 2009; 374: 2115– 22. 19880172 PMC2861912
Cited by
Sorted by citations
Adverse effects of COVID-19 mRNA vaccines: the spike hypothesis
Trougakos, I. P., Terpos, E., Alexopoulos, H., Politou, M., Paraskevis, D., Scorilas, A., Kastritis, E., Andreakos, E., & Dimopoulos, M. A. (2022) · Trends in Molecular Medicine · doi:10.1016/j.molmed.2022.04.007 · Cited by 273 · see citing works
Fraiman, J., Erviti, J., Jones, M., Greenland, S., Whelan, P., Kaplan, R. M., & Doshi, P. (2022) · Vaccine · doi:10.1016/j.vaccine.2022.08.036 · Cited by 228 · see citing works
“[32] If the actual risk ratio for the effect was 1.4 (the risk ratio of the combined AESI analysis), it could be quite difficult to unambiguously replicate it with observational data given concerns about systematic as well as random errors.”
BackgroundInfluentialFrontera, J. A., Tamborska, A. A., Doheim, M. F., Garcia‐Azorin, D., Gezegen, H., Guekht, A., Yusof Khan, A. H. K., Santacatterina, M., Sejvar, J., Thakur, K. T., et al. (2022) · Annals of Neurology · doi:10.1002/ana.26339 · Cited by 108 · see citing works
BNT162b2 Vaccine‐Associated Myo/Pericarditis in Adolescents: A Stratified Risk‐Benefit Analysis
Krug, A., Stevenson, J., & Høeg, T. B. (2022) · European Journal of Clinical Investigation · doi:10.1111/eci.13759 · Cited by 69 · see citing works
Otero-Losada, M., Petrovsky, N., Alami, A., Crispo, J. A., Mattison, D., Capani, F., Goetz, C., Krewski, D., & Perez-Lloret, S. (2023) · Expert Opinion on Drug Safety · doi:10.1080/14740338.2022.2120607 · Cited by 22 · see citing works
Hughes, N., Rijnbeek, P. R., van Bochove, K., Duarte-Salles, T., Steinbeisser, C., Vizcaya, D., Prieto-Alhambra, D., & Ryan, P. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac100 · Cited by 11 · see citing works
Self-citation
- Authors
- Li, X., Ostropolets, A., Makadia, R., Shoaibi, A., Rao, G., Sena, A. G., Martinez-Hernandez, E., Delmestri, A., Verhamme, K., Rijnbeek, P. R., Duarte-Salles, T., Suchard, M. A., Ryan, P. B., Hripcsak, G., & Prieto-Alhambra, D.
- Affiliations
- University of Oxford (GB)
- Columbia University Irving Medical Center (US)
- Erasmus MC (NL)
- University of California, Los Angeles (US)
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2021-06-14
- Volume
- 373
- Pages
- n1435
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC8010764
- OpenAlex
- W3166109011
- Cited by
- 219
- References
- 37
- Citations by year
- 2026: 4 · 2025: 25 · 2024: 35 · 2023: 47 · 2022: 82 · 2021: 26
- FWCI
- 30.97
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- BMJ (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, other-oa)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, cc-by)
- PubMed Central (submittedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- Funders
- European Federation of Pharmaceutical Industries and Associations (806968)
- National Institute for Health and Care Research
- University of Oxford
- European Commission (HORIZON2020, 806968)
- Innovative Medicines Initiative (806968, No 806968)
- Clarendon Fund
- U.S. National Library of Medicine (LM006910, R01LM006910)
- Topics
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 and COVID-19 Research
- Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
- Keywords
- COVID-19 vaccines, adverse events of special interest, background incidence rates, vaccine safety, myocarditis, deep vein thrombosis, pulmonary embolism, anaphylaxis, Guillain-Barré syndrome, Bell's palsy, narcolepsy, acute disseminated encephalomyelitis, idiopathic thrombocytopenic purpura, appendicitis, age and sex differences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
