Observed‐over‐Expected analysis as additional method for pharmacovigilance signal detection in large‐scaled spontaneous adverse event reporting
Open accessAbstract
Background: The large‐scale COVID‐19 vaccination campaigns in 2021 and 2022 led to a rapid increase in numbers of received adverse event reports in spontaneous reporting systems. As background incidences of naturally occurring medical events became increasingly relevant for causality assessment of potential associations with the vaccines, a novel approach for signal detection was warranted. Objectives: This article illustrates the Observed‐over‐Expected (O/E) analysis as an additional method for signal detection and risk assessment in large‐scaled spontaneous reporting systems. Methods: All individual case safety reports (ICSRs) concerning idiopathic peripheral facial paralysis or Bell's palsy following administration of the COVID‐19 vaccines (n = 291) manufactured by Pfizer/BioNTech (Comirnaty), Moderna (Spikevax), AstraZeneca (Vaxzevria) and Janssen (JCOVDEN) received by the National Pharmacovigilance Centre Lareb until 24th March 2022 were included in the O/E analysis, within a risk window of 7 and 14 days following immunisation. Dutch background incidence rates from 2019 and exposure of the Dutch population to the vaccines were obtained from the PHARMO institute and RIVM. The data was stratified in age groups, gender and administered dose in order to differentiate between population subgroups. Results: Bell's palsy was reported more frequently than expected in several population subgroups following administration of all four COVID‐19 vaccines, including children and adolescents. Among children, a high O/E ratio was found for boys aged 5–14 years after receiving the Pfizer/BioNTech vaccine. Regarding adolescents and young adults, women aged 15–24 years receiving Pfizer/BioNTech and Moderna, and men aged 15–24 years receiving Janssen developed Bell's palsy more often than expected. Furthermore, O/E ratios were high for individuals aged 25–64, regarding females receiving Pfizer, Moderna and AstraZeneca and males receiving Janssen. As facial paralysis was not labelled as an adverse event for the Janssen vaccine, this analysis contributed to signalling the association and warranting further regulatory action. Conclusions: The O/E method is a useful approach for signal detection of potential adverse reactions when handling large numbers of ICSRs. Further research is needed to attest to the causality on a clinical basis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Such comparisons can be executed rapidly and can play a key role in early detection of potential vaccine safety signals or when regulatory and public health agencies need rapid assessment of an emerging safety signal.
“Such comparisons can be executed rapidly and can play a key role in early detection of potential vaccine safety signals or when regulatory and public health agencies need rapid assessment of an emerging safety signal [4,6].”
From COVID-19 vaccines and adverse events of special interest: A multinational Global Vaccine Data Network (GVDN) cohort study of 99 million vaccinated individuals (2024) · cites this paper as [4]
The Introduction describes observed-over-expected comparisons as an additional method for detecting vaccine safety signals, and the Discussion (section 5) says they can contribute to regulatory decisions during a pandemic vaccination campaign. The paper describes timely surveillance, but the cited passages do not establish that the comparisons themselves can be executed rapidly or specifically address rapid assessment by public health agencies. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine
Polack, F. P., Thomas, S. J., Kitchin, N., Absalon, J., Gurtman, A., Lockhart, S., Perez, J. L., Pérez Marc, G., Moreira, E. D., Zerbini, C., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa2034577 · Cited by 14,592 · see citing works
Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine
Baden, L. R., El Sahly, H. M., Essink, B., Kotloff, K., Frey, S., Novak, R., Diemert, D., Spector, S. A., Rouphael, N., Creech, C. B., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2035389 · Cited by 10,119 · see citing works
Voysey, M., Costa Clemens, S. A., Madhi, S. A., Weckx, L. Y., Folegatti, P. M., Aley, P. K., Angus, B., Baillie, V. L., Barnabas, S. L., Bhorat, Q. E., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00432-3 · Cited by 1,119 · see citing works
Olsen, S. J., Azziz‐Baumgartner, E., Budd, A., Brammer, L., Sullivan, S. G., Fasce, R., Cohen, C., & Fry, A. M. (2020) · MMWR Morbidity and Mortality Weekly Report · doi:10.15585/mmwr.mm6937a6 · Cited by 580 · see citing works
Covid-19: European countries suspend use of Oxford-AstraZeneca vaccine after reports of blood clots
Wise, J. (2021) · BMJ · doi:10.1136/bmj.n699 · Cited by 327 · see citing works
Wan, E. Y. F., Chui, C. S. L., Lai, F. T. T., Chan, E. W., Li, X., Yan, V. K. C., Gao, L., Yu, Q., Lam, I. C. H., Chun, R. K. C., et al. (2021) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(21)00451-5 · Cited by 241 · see citing works
Bell's palsy and SARS-CoV-2 vaccines
Ozonoff, A., Nanishi, E., & Levy, O. (2021) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(21)00076-1 · Cited by 158 · see citing works
Association of COVID-19 Vaccination and Facial Nerve Palsy
Shemer, A., Pras, E., Lifshitz, A. E., Dubinsky‐Pertzov, B., & Hecht, I. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2021.1259 · Cited by 121 · see citing works
Li, X., Raventós, B., Roel, E., Pistillo, A., Martinez-Hernandez, E., Delmestri, A., Reyes, C., Strauss, V., Prieto-Alhambra, D., Burn, E., et al. (2022) · BMJ · doi:10.1136/bmj-2021-068373 · Cited by 118 · see citing works
Bell’s palsy following COVID-19 vaccination
Colella, G., Orlandi, M., & Cirillo, N. (2021) · Journal of Neurology · doi:10.1007/s00415-021-10462-4 · Cited by 105 · see citing works
Incidence of Bell Palsy in Patients With COVID-19
Tamaki, A., Cabrera, C. I., Li, S., Rabbani, C. C., Thuener, J. E., Rezaee, R. P., & Fowler, N. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2021.1266 · Cited by 98 · see citing works
Zhou, W., Pool, V., Destefano, F. R., Iskander, J. K., Haber, P., & Chen, R. T. (2004) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.998 · Cited by 85 · see citing works
Rath, B. A., Linder, T. E., Cornblath, D., et al. (2007) · Vaccine · doi:10.1016/j.vaccine.2007.10.043 · Cited by 67 · see citing works
Willame, C., Dodd, C. N., Durán, C. E., Elbers, R., Gini, R., Bartolini, C., Paoletti, O., Wang, L., Ehrenstein, V., Kahlert, J. A., et al. (2022) · Vaccine · doi:10.1016/j.vaccine.2022.11.031 · Cited by 66 · see citing works
Sato, K., Mano, T., Niimi, Y., et al. (2021) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2021.08.071 · Cited by 64 · see citing works
Pharmacoepidemiological considerations in observed‐to‐expected analyses for vaccines
Mahaux, O., Bauchau, V., & Holle, L. V. (2015) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3918 · Cited by 62 · see citing works
Sequential contralateral facial nerve palsies following COVID-19 vaccination first and second doses
Burrows, A., Bartholomew, T., Rudd, J., & Walker, D. (2021) · BMJ Case Reports · doi:10.1136/bcr-2021-243829 · Cited by 60 · see citing works
Real-world safety data for the Pfizer BNT162b2 SARS-CoV-2 vaccine: historical cohort study
Shasha, D., Bareket, R., Sikron, F. H., Gertel, O., Tsamir, J., Dvir, D., Mossinson, D., Heymann, A. D., & Zacay, G. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2021.09.018 · Cited by 60 · see citing works
Kant, A. C., Jansen, J. R. G., van Balveren, L., & van Hunsel, F. P. A. M. (2022) · Drug Safety · doi:10.1007/s40264-022-01151-w · Cited by 57 · see citing works
Vakharia, K. T., Vakharia, K., Vakharia, K., et al. (2015) · Facial Plastic Surgery Clinics of North America · doi:10.1016/j.fsc.2015.08.001 · Cited by 56 · see citing works
Shibli, R., Barnett, O., Abu‐Ful, Z., Gronich, N., Najjar‐Debbiny, R., Doweck, I., Rennert, G., & Saliba, W. (2021) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2021.100236 · Cited by 52 · see citing works
Rath, B. A., Gidudu, J., Anyoti, H., Bollweg, B. C., Caubel, P., Chen, Y.-H., Cornblath, D., Fernandopulle, R., Fries, L. F., Galama, J. M. D., et al. (2016) · Vaccine · doi:10.1016/j.vaccine.2016.05.023 · Cited by 49 · see citing works
Wong, C. K. H., Lau, K. T. K., Xiong, X., Au, I. C. H., Lai, F. T. T., Wan, E. Y. F., Chui, C. S. L., Li, X., Chan, E. W., Gao, L., et al. (2022) · PLoS Medicine · doi:10.1371/journal.pmed.1004018 · Cited by 45 · see citing works
Global safety monitoring of COVID-19 vaccines: how pharmacovigilance rose to the challenge
Rudolph, A., Mitchell, J., Barrett, J., Sköld, H., Taavola, H., Erlanson, N., Melgarejo-González, C., & Yue, Q. (2022) · Therapeutic Advances in Drug Safety · doi:10.1177/20420986221118972 · Cited by 44 · see citing works
Bell’s Palsy After 24 Hours of mRNA-1273 SARS-CoV-2 Vaccine
Iftikhar, H., Noor, S. M. U., Masood, M., & Bashir, K. (2021) · Cureus · doi:10.7759/cureus.15935 · Cited by 34 · see citing works
Bell’s palsy following a single dose of mRNA SARS-CoV-2 vaccine: a case report
Martín-Villares, C., Vazquez-Feito, A., Gonzalez-Gimeno, M. J., et al. (2021) · Journal of Neurology · doi:10.1007/s00415-021-10617-3 · Cited by 32 · see citing works
Signaling COVID-19 Vaccine Adverse Events
Harpaz, R., DuMouchel, W., van Manen, R. P., Nip, A., Bright, S., Szarfman, A., Tonning, J. M., & Lerch, M. (2022) · Drug Safety · doi:10.1007/s40264-022-01186-z · Cited by 31 · see citing works
Takeuchi, Y., Iwagami, M., Ono, S., et al. (2022) · Vaccine · doi:10.1016/j.vaccine.2022.10.088 · Cited by 28 · see citing works
Type I interferons as the potential mechanism linking mRNA COVID-19 vaccines to Bell's palsy
Soeiro, T., Salvo, F., Pariente, A. E., et al. (2021) · Therapies · doi:10.1016/j.therap.2021.03.005 · Cited by 27 · see citing works
Bell’s palsy following the Ad26.COV2.S COVID-19 vaccination
Nishizawa, Y., Hoshina, Y., & Baker, V. (2021) · QJM · doi:10.1093/qjmed/hcab143 · Cited by 27 · see citing works
Global covid-19 vaccine rollout and safety surveillance—how to keep pace
Re, V. L., Klungel, O. H., Chan, K. A., Panozzo, C. A., Zhou, W., & Winterstein, A. G. (2021) · BMJ · doi:10.1136/bmj.n1416 · Cited by 26 · see citing works
Safety surveillance and challenges in accelerated COVID-19 vaccine development
Cole, A., Webster, P. S., Liew, D. V., Salas, M., Aimer, O., & Malikova, M. A. (2022) · Therapeutic Advances in Drug Safety · doi:10.1177/20420986221116452 · Cited by 24 · see citing works
Scholl, J. H. G., van Hunsel, F. P. A. M., Hak, E., et al. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4364 · Cited by 22 · see citing works
Bell’s palsy following COVID-19 vaccination with high CSF antibody response
Obermann, M., Krasniqi, M., Ewers, N., et al. (2021) · Neurological Sciences · doi:10.1007/s10072-021-05496-5 · Cited by 21 · see citing works
Oosterhuis, I., Scholl, J. H. G., van Puijenbroek, E., Kant, A. C., & van Hunsel, F. P. A. M. (2022) · Drug Safety · doi:10.1007/s40264-022-01253-5 · Cited by 19 · see citing works
Bell’s palsy following COVID-19 vaccination: a case report
Caro, G. G. D. T., Diaz, S., Alé, M. P., et al. (2021) · Neurología (English Edition) · doi:10.1016/j.nrleng.2021.04.002 · Cited by 17 · see citing works
The validity and reliability of a signal impact assessment tool
Rolfes, L., Kolfschoten, J., van Hunsel, F. P. A. M., et al. (2016) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4038 · Cited by 14 · see citing works
Vaccines and Bell's palsy: A narrative review
Bertin, B., Grenet, G., Pizzoglio-Billaudaz, V., et al. (2022) · Therapies · doi:10.1016/j.therap.2022.07.009 · Cited by 14 · see citing works
Strengthening international surveillance of vaccine safety
Klungel, O. H., & Pottegård, A. (2021) · BMJ · doi:10.1136/bmj.n1994 · Cited by 10 · see citing works
Willame, C., Dodd, C. N., Carlos, D., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4036376 · Cited by 10 · see citing works
Hamid, R., Korkmazer, B., Kızılkılıç, E. K., et al. (2022) · Ear Nose & Throat Journal · doi:10.1177/01455613221113818 · Cited by 6 · see citing works
Eine akute Fazialisparese als mögliche Impfkomplikation bei einer Impfung gegen SARS-CoV-2
Volk, G. F., Kuttenreich, A., Geitner, M., et al. (2021) · Laryngo-Rhino-Otologie · doi:10.1055/a-1501-0470 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Holland NJ, Bernstein JM. Bell's palsy. BMJ: Clin Evid. 2014;2014: 1204.
Cited by
Sorted by citations
Faksova, K., Walsh, D., Jiang, Y., Griffin, J., Phillips, A., Gentile, A., Kwong, J. C., Macartney, K., Naus, M., Grange, Z., et al. (2024) · Vaccine · doi:10.1016/j.vaccine.2024.01.100 · Cited by 212 · see citing works
“One flexible approach for assessing AESI is the comparison of observed AESI rates following the introduction of a vaccine program with the expected (or background) rates based on historical periods pre-vaccine roll out [4,5].”
BackgroundInfluential
- Authors
- van der Boom, M. D. X., van Eekeren, R., & van Hunsel, F. P. A. M.
- Affiliations
- Netherlands Pharmacovigilance Centre Lareb 's‐Hertogenbosch The Netherlands
- Venue
- Pharmacoepidemiology and Drug Safety
- Publisher
- Wiley
- Published
- 2023-03-21
- Volume
- 32
- Issue
- 7
- Pages
- 783-794
- ISSN
- 1053-8569 (ISSN-L)
- 1099-1557
- OpenAlex
- W4324309765
- Cited by
- 11
- References
- 42
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 2 · 2023: 1
- FWCI
- 1.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Pharmacoepidemiology and Drug Safety (publishedVersion)
- PubMed (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion)
- Topics
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Health Sciences › Medicine › Neurology › Facial Nerve Paralysis Treatment and Research
- Keywords
- COVID-19 vaccines, Bell's palsy, spontaneous adverse event reporting, individual case safety reports, background incidence rates, vaccine safety, risk assessment, subpopulations, BNT162b2, Moderna vaccine, ChAdOx1 nCoV-19 vaccine
- MeSH terms
- COVID-19, COVID-19 Vaccines, Adolescent, Child, Female, Humans, Male, Vaccines, Bell Palsy, Young Adult, Pharmacovigilance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
