The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Covid-19 Vaccine Effectiveness and the Test-Negative Design
Open accessAbstract
Observational studies are emerging as fundamental sources of information about vaccine effectiveness outside the controlled environment of randomized trials, and they are being used to generate evi...
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The test-negative design, which does not assume that a lack of a positive test equals a negative test and somewhat adjusts for testing volume, attempts to better mitigate this bias and is the reason we chose this method for this analysis”
From Waning effectiveness of the third dose of the BNT162b2 mRNA COVID-19 vaccine (2022) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The test-negative design for estimating influenza vaccine effectiveness
Jackson, M. L., & Nelson, J. C. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.02.053 · Cited by 510
Prevention and Attenuation of Covid-19 with the BNT162b2 and mRNA-1273 Vaccines
Thompson, M. G., Burgess, J. L., Naleway, A. L., Tyner, H. L., Yoon, S. K., Meece, J. K., Olsho, L. E. W., Caban-Martinez, A. J., Fowlkes, A., Lutrick, K., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2107058 · Cited by 482
Effectiveness of Covid-19 Vaccines in Ambulatory and Inpatient Care Settings
Thompson, M. G., Stenehjem, E. A., Grannis, S. J., Ball, S. W., Naleway, A. L., Ong, T. C., DeSilva, M. B., Natarajan, K., Bozio, C. H., Lewis, N., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2110362 · Cited by 467
Sullivan, S. G., Tchetgen, E. J. T., & Cowling, B. J. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kww064 · Cited by 337
Understanding COVID-19 vaccine efficacy
Lipsitch, M., & Dean, N. E. (2020) · Science · doi:10.1126/science.abe5938 · Cited by 230
Self-citationThe Use of Test-negative Controls to Monitor Vaccine Effectiveness
Chua, H., Feng, S., Lewnard, J. A., Sullivan, S. G., Blyth, C. C., Lipsitch, M., & Cowling, B. J. (2019) · Epidemiology · doi:10.1097/ede.0000000000001116 · Cited by 165
Theoretical Framework for Retrospective Studies of the Effectiveness of SARS-CoV-2 Vaccines
Lewnard, J. A., Patel, M. M., Jewell, N. P., Verani, J. R., Kobayashi, M., Tenforde, M. W., Dean, N. E., Cowling, B. J., & Lopman, B. A. (2021) · Epidemiology · doi:10.1097/ede.0000000000001366 · Cited by 103
Self-citationVandenbroucke, J. P., & Pearce, N. E. (2019) · Epidemiology · doi:10.1097/ede.0000000000001088 · Cited by 78
Vaccine effectiveness studies in the field
Evans, S. J. W., & Jewell, N. P. (2021) · New England Journal of Medicine · doi:10.1056/nejme2110605 · Cited by 65
Schnitzer, M. E., Harel, D., Ho, V., Koushik, A., & Merckx, J. (2021) · Epidemiology · doi:10.1097/ede.0000000000001385 · Cited by 9
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Robins JM. Comment on “choice as an alternative to control in observational studies.” Stat Sci 1999;14:281-293 (https://www.jstor.org/stable/2676763).
Cited by
Sorted by citations
Items 101–170 of 170
State-of-the-art learning COVID-19 vaccine effectiveness using LSTM
Shen, C., Lin, M., Lee, R., Dong, M., & Zhao, L. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101561 · Cited by 3
van den Berg, J. M., Blom, M. T., Overbeek, J. A., Remmelzwaal, S., Herings, R. M. C., & Elders, P. J. M. (2025) · Vaccines · doi:10.3390/vaccines13060564 · Cited by 3
Fusco, E., Loiodice, M., Romero, A., L’assainato, S., Banfi, A., Chiaffoni, G., Porro, C., Fabiano, V., Izzo, F., Giorgis, V. D., et al. (2025) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000004968 · Cited by 3
Jiang, C., Talbot, D. C., Carazo, S., & Schnitzer, M. E. (2025) · Statistics in Medicine · doi:10.1002/sim.70025 · Cited by 5
Self-citationBhattacharyya, A., Anand, S., & Rai Shesh, N. (2023) · Annals of Public Health Reports · doi:10.36959/856/540 · Cited by 2
To boost or not to boost: navigating post-pandemic COVID-19 vaccination
Goldberg, Y., & Huppert, A. (2023) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(23)00350-8 · Cited by 3
Hypothesis testing and sample size considerations for the test-negative design
Huo, Y., Yang, Y., Halloran, M. E., Longini, I. M., & Dean, N. E. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02277-4 · Cited by 3
Self-citationTschiderer, L., Seekircher, L., Richter, L., von Laer, D., Lass‐Flörl, C., Forer, L., Schönherr, S., Krammer, F., Embacher-Aichhorn, S., Tilg, H., et al. (2022) · iScience · doi:10.1016/j.isci.2022.105380 · Cited by 3
COVID-19 vaccination is associated with reduced non-COVID in-hospital mortality
Benbassat, J., Froom, P., & Shimoni, Z. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107326 · Cited by 3
Yassi, A., Barker, S. J., Lockhart, K., Grant, J., Okpani, A. I., Sprague, S., Zungu, M., Lubin, S., & Sing, C. K. (2021) · medRxiv · doi:10.1101/2021.12.02.21267190 · Cited by 3
Black, B. S., & Thaw, D. B. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4155493 · Cited by 3
Qasmieh, S. A., Ferdinands, J. M., Chung, J. R., Wiegand, R. E., Flannery, B. L., Rane, M. S., & Nash, D. (2024) · medRxiv · doi:10.1101/2024.12.30.24319700 · Cited by 3
Early-Life Exposure to Acid-Suppressive Therapy and the Development of Celiac Disease Autoimmunity
Achler, T., Patalon, T., Gazit, S., Cohen, S., Shaoul, R. R., & Ben‐Tov, A. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.3376 · Cited by 2
The effect of SARS-CoV-2 testing on healthcare-seeking behaviour at primary care level
van Hagen, C. C. E., Vos, E. R. A., Delaunay, C. L., de Melker, H. E., Kissling, E., & Knol, M. J. (2026) · International Journal of Epidemiology · doi:10.1093/ije/dyag031 · Cited by 2
Association of public health and social measures with vaccine effectiveness
Tsang, T. K., Sullivan, S. G., Huang, X., Wang, C., Li-ping, P., Yang, B., & Cowling, B. J. (2026) · Nature Health · doi:10.1038/s44360-026-00127-9 · Cited by 2
Influenza Vaccine Effectiveness and Test-Negative Study Design Within the Department of Defense
Sayers, D. R., & Iskander, J. K. (2023) · Military Medicine · doi:10.1093/milmed/usac436 · Cited by 2
Tamada, Y., Takeuchi, K., Kusama, T., Maeda, M., Murata, F., Osaka, K., & Fukuda, H. (2023) · Vaccine · doi:10.1016/j.vaccine.2023.07.035 · Cited by 2
Boyer, C., Li, K. Q., Shi, X., & Tchetgen, E. J. T. (2025) · Epidemiology · doi:10.1097/ede.0000000000001926 · Cited by 4
Hsieh, M.-J., Tsai, P., Chiang, P.-H., Kao, Z.-K., Zhuang, Z.-Q., Hsieh, A., Ho, H., Chiou, S., Liang, K., & Chen, Y. (2024) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2024.2399382 · Cited by 2
Castillo, M. S., Khaoua, H., & Courtejoie, N. (2022) · medRxiv · doi:10.1101/2022.02.17.22270791 · Cited by 2
Impact of Vaccination, Prior Infection, and Therapy on Delta and Omicron VariantsPreprint
Wang, X., Zein, J., Ji, X., & Lin, D.-Y. (2022) · medRxiv · doi:10.1101/2022.03.24.22272901 · Cited by 2
BNT162b2 effectiveness against Delta & Omicron variants in teens by dosing interval and durationPreprint
Ionescu, I. G., Skowronski, D. M., Sauvageau, C., Chuang, E., Ouakki, M., Kim, S. H., & Serres, G. D. (2022) · medRxiv · doi:10.1101/2022.06.27.22276790 · Cited by 2
Lind, M. L., Copin, R., McCarthy, S., Coppi, A. C., Warner, F., Ferguson, D. J. P., Duckwall, C., Borg, R., Muenker, M. C., Overton, J. D., et al. (2022) · medRxiv · doi:10.1101/2022.08.25.22278443 · Cited by 2
van Hagen, C. C. E., Vos, E. R. A., Delaunay, C. L., de Melker, H. E., Kissling, E., & Knol, M. J. (2025) · medRxiv · doi:10.1101/2025.03.31.25324928 · Cited by 2
Dal‐Ré, R., Porcher, R., Rosendaal, F. R., & Schwarzer‐Daum, B. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.11.007 · Cited by 1
Aglipay, M., Maguire, J. L., Swayze, S., Tuite, A. R., Mamdani, M., Keown‐Stoneman, C. D. G., Birken, C. S., & Kwong, J. C. (2024) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofae718 · Cited by 1
Timely Personalized Exposure Risk Estimation for Air Travelers
Chan, C., & Wen, T. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3523336 · Cited by 1
Ashby, E., Janes, H., Follmann, D. A., Gilbert, P. B., Zhou, H., Wang, X., Girard, B., Priddy, F. H., Kublin, J. G., Corey, L., et al. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf176 · Cited by 4
Follmann, D. A., Dang, L., Chu, E., Fintzi, J., Janes, H., Gilbert, P. B., Andrews, L. I. B., Serebryannyy, L., Carroll, R., Lin, B. C., et al. (2025) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiaf572 · Cited by 1
Middleton, C. E., & Larremore, D. B. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf280 · Cited by 1
2024-2025 COVID-19 Vaccine and Major Adverse Cardiovascular Events Among US Veterans
Cai, M., Xie, Y., & Al‐Aly, Z. (2026) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2026.1929 · Cited by 3
COVID-19 vaccine effectiveness studies in Nigeria: Quo vadis?
Akande, O. W., Igumbor, E. U., Elimian, K., Ohonsi, C., Nwozor, L., Oden, O., Ekpenyong, E. N., Ndodo, N. D., Ike, I. F., Egede, M., et al. (2022) · Journal of Global Health · doi:10.7189/jogh.12.03055 · Cited by 1
Evaluating COVID-19 Vaccines in the Era of Endemicity—Recency vs Reformulation
Mohan, G. S., Mina, M. J., & Ankomah, P. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.1637 · Cited by 1
Cossrow, N., Joshi, R., Klinker, K. P., & Buchwald, U. K. (2022) · Microorganisms · doi:10.3390/microorganisms10101987 · Cited by 1
Mandegari, Z., & Soodejani, M. T. (2023) · Research Square · doi:10.21203/rs.3.rs-3278248/v1 · Cited by 1
Some principles for using epidemiologic study results to parameterize transmission modelsPreprint
Joshi, K., Kahn, R. J., Boyer, C., & Lipsitch, M. (2023) · medRxiv · doi:10.1101/2023.10.03.23296455 · Cited by 1
Meakin, S. R., Camacho, A., Nsio, J. M., Kitenge, R., Coulborn, R. M., Gignoux, E., Johnson, J. K., Sterk, E., Musenga, E. M., Mustafa, S. H. B., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4812493 · Cited by 1
Middleton, C. E., & Larremore, D. B. (2025) · medRxiv · doi:10.1101/2025.04.05.25325304 · Cited by 1
Subramanian, R. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3987311
Natural Capital Exploration and Coronavirus Disease Spread in 2020
Piao, X. (2022) · Theoretical Economics Letters · doi:10.4236/tel.2022.122022
Solante, R., Álvarez, C., Burhan, E., Chariyalertsak, S., Chiu, N., Chuenkitmongkol, S., Dung, D. V., Hwang, K., Ibarra, J. O., Kiertiburanakul, S., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-2015733/v1
Subramanian, R. (2022) · Research Square · doi:10.21203/rs.3.rs-2073308/v1
Fukui, N., Li, S. S., DeGuzman, J. L., Myers, J. F., Openshaw, J. J., Sharma, A., Watt, J., Lewnard, J. A., Jain, S., Andrejko, K. L., et al. (2023) · medRxiv · doi:10.1101/2023.02.27.23286454
Aglipay, M., Maguire, J. L., Swayze, S., Tuite, A. R., Mamdani, M., Keown‐Stoneman, C. D. G., Birken, C. S., & Kwong, J. C. (2023) · medRxiv · doi:10.1101/2023.06.27.23291933
Fukui, N., Li, S. S., DeGuzman, J. L., Myers, J. F., Openshaw, J. J., Sharma, A., Watt, J., Lewnard, J. A., Jain, S., Andrejko, K. L., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301070
Caveats regarding the test-negative design to study mpox risk factors and vaccine effectiveness
Núñez, I. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae284
Shahzad, A., Ghani, E., Mansoor, E., Rathore, M. A., Niazi, S. K., & Ahmed, F. (2024) · Pakistan Armed Forces Medical Journal · doi:10.51253/pafmj.v74i4.10199
Forero-Peña, D. A., Leyva, J. L., Valenzuela, M. V., Omaña‐Ávila, Ó. D., Regalado-Gutiérrez, O. A., Mendoza-Millán, D. L., Sánchez-Ytriago, E. A., Hachem, A. C. L.-E., Farro, K. R., Maita, A. K., et al. (2024) · Discover Public Health · doi:10.1186/s12982-024-00259-x
Brambilla, R., Gili, R., Vigna‐Taglianti, F., Lenzi, J., Riccò, M., Burioni, R., Scarvaglieri, M., Rocco, R., Buttafuoco, V., Cristaudo, R. M. T. A., et al. (2024) · Vaccines · doi:10.3390/vaccines12111245
Makadzange, T., Gundidza, P., Konono, K. C. C., Gurumani, M., & Ndhlovu, C. E. (2024) · Vaccines · doi:10.3390/vaccines12121303
Jiang, C., Fang, F., Talbot, D. C., & Schnitzer, M. E. (2025) · medRxiv · doi:10.1101/2025.02.24.25322826
Self-citationEnhancing health security in Africa: Collaboration between public health institutes and academia
Kambugu, A. D., Ayebare, R. R., Kakooza, F., Kiragga, A. N., Tsegaye, A., Yeboah, G., Mbabazi, L., Muwonge, T., Shaweno, T., Dereje, N., et al. (2025) · Journal of Public Health in Africa · doi:10.4102/jphia.v16i1.780
Wilton, J., García, H. A. V., Naveed, Z., Crabtree, A., Buxton, J. A., Wong, J., Krajden, M., Sbihi, H., & Janjua, N. Z. (2025) · Vaccine · doi:10.1016/j.vaccine.2025.127423
Agrupis, K. A., Ylade, M. C., Yang, Q., Crisostomo, M. V., Daag, J. V., Magsakay, G. L., Lo, J. R., Florendo, K. L., Lopez, M. H. J., Enriquez, J. M., et al. (2025) · Expert Review of Vaccines · doi:10.1080/14760584.2025.2539889
Observational Studies: Common and Useful but with Several Important Limitations
Aggarwal, R., & Sahni, P. (2025) · The National Medical Journal of India · doi:10.25259/nmji_1086_2025
Evidence, Opinion, and Uncertainty About COVID-19 Vaccines
Califf, R. M. (2025) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2025.5462
Wiratama, B. S., Oktaria, V., Indrihutami, K., Hardhantyo, M., Maula, A. W., Djasri, H., Dinari, R., Syairaji, M., Suwarti, S., Pradipta, A., et al. (2025) · The Lancet Regional Health - Southeast Asia · doi:10.1016/j.lansea.2025.100689
de Paula, R. R. O., Cintra, P. R., Andrade, E. F., Pala, L. O. D. O., Murata, R. M., & Pereira, L. J. (2025) · medRxiv · doi:10.1101/2025.11.20.25340681
Song, S., Hitchings, M. D. T., Yang, Y., & Longini, I. M. (2026) · Epidemiology · doi:10.1097/ede.0000000000001972
COVID-19 Vaccination and Lethality Reduction: A Prospective Cohort Study in VenezuelaPreprint
Forero-Peña, D. A., Leyva, J. L., Valenzuela, M. V., Omaña-Ávila, Ó. D., Mendoza-Millán, D. L., Sánchez-Ytriago, E. A., Hachem, A. C. L.-E., Farro, K. R., Maita, A. K., González, R. D. C., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-3813947/v1
Additional file 1 of Hypothesis testing and sample size considerations for the test-negative design
Huo, Y., Yang, Y., Halloran, M. E., M, L. I., & Dean, N. E. (2024) · Figshare · doi:10.6084/m9.figshare.26749925
Self-citationEusebi, P., Speybroeck, N., Hartnack, S., Stærk‐Østergaard, J., Denwood, M. J., & Kostoulas, P. (2024) · Figshare · doi:10.6084/m9.figshare.26571748
Green, M. A., Hungerford, D., Hughes, D. M., Garcia-Fiñana, M., Turtle, L., Cheyne, C. P., Ashton, M., Leeming, G., Semple, M. G., Singleton, A. D., et al. (2022) · Figshare · doi:10.6084/m9.figshare.21629220
Tamada, Y., Takeuchi, K., Kusama, T., Maeda, M., Murata, F., Osaka, K., & Fukuda, H. (2024) · Figshare · doi:10.6084/m9.figshare.26673882
Ess, G., Lew, A. S., Tippett, A., Salazar, L. W., Choi, C., Castro, K. D., Taylor, E. G., Reese, O., Momin, H., Ciric, C. R., et al. (2025) · Vaccines · doi:10.3390/vaccines14010045
Ovalle-Marroquín, D. F., Pablos-Verdugo, A., Wong-Valenzuela, E. I., Zazueta, O. E., Bojórquez, I., Texcalac-Sangrador, J. L., & Schilmann, A. (2026) · Discover Public Health · doi:10.1186/s12982-026-01411-5
Gentile, A., Juárez, M. D. V., Lución, M. F., Ensinck, G., Gregorio, G., Lopez, O., Pejito, M. N., Zuzel, P. Y., Lobertti, S., Gioiosa, A., et al. (2025) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0000000000005045
Why the Test-Negative Design Is Used for Routine Vaccine Monitoring
Dean, N. E. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.25157
Self-citationFlannery, B. L., Chung, J. R., Holiday, C., Jefferson, S. N., Gaglani, M., Murthy, K., Zimmerman, R. K., Nowalk, M. P., Jackson, M. L., Wernli, K. J., et al. (2026) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiag371 · Cited by 2
Metadata
- Authors
- Dean, N. E., Hogan, J. W., & Schnitzer, M. E.
- Affiliations
- Emory University (US)
- Corresponding author
- Hogan, Joseph W.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2021-10-07
- Volume
- 385
- Issue
- 15
- Pages
- 1431-1433
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- PMC
- PMC8451180
- OpenAlex
- W3196352038
- Cited by
- 168
- References
- 10
- Citations by year
- 2026: 8 · 2025: 24 · 2024: 29 · 2023: 39 · 2022: 61 · 2021: 6
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- New England Journal of Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Allergy and Infectious Diseases (R01AI139761, P30AI042853)
- Topics
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 and COVID-19 Research
- Social Sciences › Health › Vaccine Coverage and Hesitancy
- Physical Sciences › Mathematics › Modeling and Simulation › COVID-19 epidemiological studies
- Keywords
- COVID-19 vaccines, vaccine effectiveness, test-negative design, observational studies, SARS-CoV-2
- MeSH terms
- COVID-19, SARS-CoV-2, COVID-19 Vaccines, Hospitalization, Humans, Case-Control Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review