COVID-19 vaccine efficacy and effectiveness—the elephant (not) in the room
Open accessThe DOI matches its Crossref record; 7 of the 8 references it cites were found in a registry or an index.
Abstract
Approximately 96 COVID-19 vaccines are at various stages of clinical development.1Zimmer C Corum J Wee S-L Covid-19 Vaccine Tracker.https://www.nytimes.com/interactive/2020/science/coronavirus-vaccine-tracker.htmlDate accessed: March 10, 2021Google Scholar At present, we have the interim results of four studies published in scientific journals (on the Pfizer–BioNTech BNT162b2 mRNA vaccine,2Polack FP Thomas SJ Kitchin N et al.Safety and efficacy of the BNT162b2 mRNA COVID-19 Vaccine.N Engl J Med. 2020; 383: 2603-2615Crossref PubMed Scopus (9742) Google Scholar the Moderna–US National Institutes of Health [NIH] mRNA-1273 vaccine,3Baden LR El Sahly HM Essink B et al.Efficacy and safety of the mRNA-1273 SARS-CoV-2 Vaccine.N Engl J Med. 2021; 384: 403-416Crossref PubMed Scopus (6728) Google Scholar the AstraZeneca–Oxford ChAdOx1 nCov-19 vaccine,4Voysey M Clemens SAC Madhi SA et al.Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK.Lancet. 2021; 397: 99-111Summary Full Text Full Text PDF PubMed Scopus (3331) Google Scholar and the Gamaleya GamCovidVac [Sputnik V] vaccine)5Logunov DY Dolzhikova IV Shcheblyakov DV et al.Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia.Lancet. 2021; 397: 671-681Summary Full Text Full Text PDF PubMed Scopus (1226) Google Scholar and three studies through the US Food and Drug Administration (FDA) briefing documents (on the Pfizer–BioNTech,6US Food and Drug AdministrationVaccines and Related Biological Products Advisory Committee meeting: FDA briefing document.https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-december-10-2020-meeting-announcementDate: Dec 10, 2020Date accessed: March 10, 2021Google Scholar Moderna–NIH,7US Food and Drug AdministrationVaccines and Related Biological Products Advisory Committee meeting: FDA briefing document.https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-december-17-2020-meeting-announcementDate: Dec 17, 2020Date accessed: March 10, 2021Google Scholar and Johnson & Johnson [J&J] Ad26.COV2.S vaccines).8US Food and Drug AdministrationVaccines and Related Biological Products Advisory Committee meeting: FDA briefing document.https://www.fda.gov/advisory-committees/advisory-committee-calendar/vaccines-and-related-biological-products-advisory-committee-february-26-2021-meeting-announcementDate: Feb 26, 2021Date accessed: March 10, 2021Google Scholar Furthermore, excerpts of these results have been widely communicated and debated through press releases and media, sometimes in misleading ways.9Olliaro P What does 95% COVID-19 vaccine efficacy really mean?.Lancet Infect Dis. 2021; (published online Feb 17.)https://doi.org/10.1016/S1473-3099(21)00075-XSummary Full Text Full Text PDF Scopus (33) Google Scholar Although attention has focused on vaccine efficacy and comparing the reduction of the number of symptomatic cases, fully understanding the efficacy and effectiveness of vaccines is less straightforward than it might seem. Depending on how the effect size is expressed, a quite different picture might emerge (figure; appendix). Vaccine efficacy is generally reported as a relative risk reduction (RRR). It uses the relative risk (RR)—ie, the ratio of attack rates with and without a vaccine—which is expressed as 1–RR. Ranking by reported efficacy gives relative risk reductions of 95% for the Pfizer–BioNTech, 94% for the Moderna–NIH, 91% for the Gamaleya, 67% for the J&J, and 67% for the AstraZeneca–Oxford vaccines. However, RRR should be seen against the background risk of being infected and becoming ill with COVID-19, which varies between populations and over time. Although the RRR considers only participants who could benefit from the vaccine, the absolute risk reduction (ARR), which is the difference between attack rates with and without a vaccine, considers the whole population. ARRs tend to be ignored because they give a much less impressive effect size than RRRs: 1·3% for the AstraZeneca–Oxford, 1·2% for the Moderna–NIH, 1·2% for the J&J, 0·93% for the Gamaleya, and 0·84% for the Pfizer–BioNTech vaccines. ARR is also used to derive an estimate of vaccine effectiveness, which is the number needed to vaccinate (NNV) to prevent one more case of COVID-19 as 1/ARR. NNVs bring a different perspective: 81 for the Moderna–NIH, 78 for the AstraZeneca–Oxford, 108 for the Gamaleya, 84 for the J&J, and 119 for the Pfizer–BioNTech vaccines. The explanation lies in the combination of vaccine efficacy and different background risks of COVID-19 across studies: 0·9% for the Pfizer–BioNTech, 1% for the Gamaleya, 1·4% for the Moderna–NIH, 1·8% for the J&J, and 1·9% for the AstraZeneca
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Yet, ARRs “tend to be ignored because they give a much less impressive effect size than RRRs” [14].”
From Relative risk reduction: Misinformative measure in clinical trials and COVID-19 vaccine efficacy (2022) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine
Polack, F. P., Thomas, S. J., Kitchin, N., Absalon, J., Gurtman, A., Lockhart, S., Perez, J. L., Pérez Marc, G., Moreira, E. D., Zerbini, C., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa2034577 · Cited by 14,592
Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine
Baden, L. R., El Sahly, H. M., Essink, B., Kotloff, K., Frey, S., Novak, R., Diemert, D., Spector, S. A., Rouphael, N., Creech, C. B., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2035389 · Cited by 10,119
Voysey, M., Clemens, S. A. C., Madhi, S. A., Weckx, L. Y., Folegatti, P. M., Aley, P. K., Angus, B., Baillie, V. L., Barnabas, S. L., Bhorat, Q. E., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(20)32661-1 · Cited by 4,616
BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting
Dagan, N., Barda, N., Kepten, E., Miron, O., Perchik, S., Katz, M. A., Hernán, M. A., Lipsitch, M., Reis, B., & Balicer, R. D. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2101765 · Cited by 2,527
Logunov, D. Y., Dolzhikova, I. V., Shcheblyakov, D. V., Tukhvatulin, A. I., Zubkova, O. V., Dzharullaeva, A. S., Kovyrshina, A. V., Lubenets, N. L., Grousova, D. M., Erokhova, A. S., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00234-8 · Cited by 1,747
What does 95% COVID-19 vaccine efficacy really mean?
Olliaro, P. (2021) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(21)00075-x · Cited by 49
Self-citationOutcome Reporting Bias in COVID-19 mRNA Vaccine Clinical Trials
Brown, R. B. (2021) · Medicina · doi:10.3390/medicina57030199 · Cited by 44
Cited by
Mohammed, I., Nauman, A., Paul, P., Ganesan, S., Chen, K.-H., Jalil, S. M. S., Jaouni, S. H., Kawas, H., Khan, W. A., Vattoth, A. L., et al. (2022) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2022.2027160 · Cited by 406
“ARRs are often overlooked since they provide a considerably smaller effect magnitude than RRRs.(71)”
BackgroundMethodInfluentialReview of COVID-19 vaccine subtypes, efficacy and geographical distributions
Francis, A. I., Ghany, S., Gilkes, T., & Umakanthan, S. (2022) · Postgraduate Medical Journal · doi:10.1136/postgradmedj-2021-140654 · Cited by 215
Bravo, L., Smolenov, I., Han, H. H., Li, P., Hosain, R., Rockhold, F., Clemens, S. A. C., Roa, C., Borja-Tabora, C., Quinsaat, A., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(22)00055-1 · Cited by 106
COVID-19 Pandemic and Vaccines Update on Challenges and Resolutions
Khan, W. H., Hashmi, Z., Goel, A., Ahmad, R., Gupta, K., Khan, N., Alam, I., Ahmed, F., & Ansari, M. A. (2021) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2021.690621 · Cited by 115
“…468 centrally confirmed cases of symptomatic COVID-19 with an onset at least 14 days after administration were observed, of which 464 were moderate to severe–critical (116 cases in the vaccine group vs. 348 in the placebo group), which indicates a vaccine efficacy of 66.9% (Olliaro et al., 2021).”
BackgroundInfluentialA comprehensive review of various categories of face masks resistant to Covid-19
Das, S., Sarkar, S., Das, A., Das, S., Chakraborty, P., & Sarkar, J. (2021) · Clinical Epidemiology and Global Health · doi:10.1016/j.cegh.2021.100835 · Cited by 99
Metadata
- Authors
- Olliaro, P., Torreele, E., & Vaillant, M.
- Affiliations
- University of Oxford (GB)
- University College London (GB)
- Luxembourg Institute of Health (LU)
- Corresponding author
- Olliaro, Piero
- Venue
- The Lancet Microbe
- Publisher
- Elsevier BV
- Published
- 2021-07
- Volume
- 2
- Issue
- 7
- Pages
- e279-e280
- ISSN
- 2666-5247 (ISSN-L)
- PMC
- PMC8057721
- OpenAlex
- W3152691333
- Cited by
- 214
- References
- 8
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 19 · 2023: 43 · 2022: 92 · 2021: 49
- FWCI
- 16.06
- Type
- journal article
- Language
- en
- Update notices
- erratum (2021-07-01)
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
- Locations
- The Lancet Microbe (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 and COVID-19 Research
- Social Sciences › Health › Vaccine Coverage and Hesitancy
- Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
- Keywords
- COVID-19 vaccine efficacy, vaccine effectiveness, SARS-CoV-2, mRNA vaccines, BNT162b2, mRNA-1273, AZD1222, Sputnik V, vaccine safety, vaccine hesitancy
- MeSH terms
- COVID-19, COVID-19 Vaccines, Vaccine Efficacy, Animals, Humans, Vaccination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: COVID-19 vaccine efficacy and effectiveness—the elephant (not) in the room. https://app.cochat.ai/papers/10.1016/s2666-5247(21)00069-0
- The paper
- Olliaro, P., Torreele, E., & Vaillant, M. (2021). COVID-19 vaccine efficacy and effectiveness—the elephant (not) in the room. The Lancet Microbe. https://doi.org/10.1016/s2666-5247(21)00069-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
