The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Theoretical Basis of the Test-Negative Study Design for Assessment of Influenza Vaccine Effectiveness
Open accessAbstract
Influenza viruses undergo frequent antigenic changes. As a result, the viruses circulating change within and between seasons, and the composition of the influenza vaccine is updated annually. Thus, estimation of the vaccine's effectiveness is not constant across seasons. In order to provide annual estimates of the influenza vaccine's effectiveness, health departments have increasingly adopted the "test-negative design," using enhanced data from routine surveillance systems. In this design, patients presenting to participating general practitioners with influenza-like illness are swabbed for laboratory testing; those testing positive for influenza virus are defined as cases, and those testing negative form the comparison group. Data on patients' vaccination histories and confounder profiles are also collected. Vaccine effectiveness is estimated from the odds ratio comparing the odds of testing positive for influenza among vaccinated patients and unvaccinated patients, adjusting for confounders. The test-negative design is purported to reduce bias associated with confounding by health-care-seeking behavior and misclassification of cases. In this paper, we use directed acyclic graphs to characterize potential biases in studies of influenza vaccine effectiveness using the test-negative design. We show how studies using this design can avoid or minimize bias and where bias may be introduced with particular study design variations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Sontrop, J. M., Dixon, S. N., Garg, A. X., Buendia-Jimenez, I., Dohein, O., Huang, S.-H. S., & Clark, W. F. (2013) · American Journal of Nephrology · doi:10.1159/000350377 · Cited by 5,871
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
Air Pollution and Hospital Admissions for Heart Disease in Eight U.S. Counties
Schwartz, J. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00005 · Cited by 3,038
Simulation Study of Confounder-Selection Strategies
Maldonado, G., & Greenland, S. (1993) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116813 · Cited by 2,491
Overadjustment bias and unnecessary adjustment in epidemiologic studies.
Schisterman, E. F., Cole, S. R., & Platt, R. W. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181a819a1 · Cited by 2,096
Hernan, M. A. (2002) · American Journal of Epidemiology · doi:10.1093/aje/155.2.176 · Cited by 1,445
Cross-Reactive Antibody Responses to the 2009 Pandemic H1N1 Influenza Virus
Hancock, K., Veguilla, V., Lu, X., Zhong, W., Butler, E. N., Sun, H., Liu, F., Dong, L., DeVos, J. R., Gargiullo, P. M., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0906453 · Cited by 1,259
Time Lines of Infection and Disease in Human Influenza: A Review of Volunteer Challenge Studies
Carrat, F., Vergu, E., Ferguson, N. M., Lemaitre, M., Cauchemez, S., Leach, S., & Valleron, A.-J. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwm375 · Cited by 1,133
Selection of Controls in Case-Control Studies
Wacholder, S., McLaughlin, J. K., Silverman, D. T., & Mandel, J. S. (1992) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116396 · Cited by 998
Book Review: ‘The Australian Immunisation Handbook 8 th Edition’
Shaban, R. Z., & Forster, E. (2005) · Australasian Journal of Paramedicine · doi:10.33151/ajp.3.1.314 · Cited by 712
Miller, E., Hoschler, K., Hardelid, P., Stanford, E., Andrews, N., & Zambon, M. (2010) · The Lancet · doi:10.1016/s0140-6736(09)62126-7 · Cited by 708
(1951) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/11.6.1269 · Cited by 694
Basic Methods for Sensitivity Analysis of Biases
GREENLAND, S. (1996) · International Journal of Epidemiology · doi:10.1093/ije/25.6.1107 · Cited by 586
The test-negative design for estimating influenza vaccine effectiveness
Jackson, M. L., & Nelson, J. C. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.02.053 · Cited by 510
Causal Inference from Complex Longitudinal Data
Robins, J. M. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-1842-5_4 · Cited by 510
Evidence of bias in estimates of influenza vaccine effectiveness in seniors
Jackson, L. A., Jackson, M. L., Nelson, J. C., Neuzil, K. M., & Weiss, N. S. (2006) · International Journal of Epidemiology · doi:10.1093/ije/dyi274 · Cited by 493
Confounding by Indication: An Example of Variation in the Use of Epidemiologic Terminology
Salas, M., Hotman, A., & Stricker, B. H. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009758 · Cited by 471
Validation of self-report of influenza and pneumococcal vaccination status in elderly outpatients
Mac Donald, R., Baken, L., Nelson, A., & Nichol, K. L. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(98)00159-7 · Cited by 410
Variable efficacy of repeated annual influenza vaccination
Smith, D. J., Forrest, S., Ackley, D. H., & Perelson, A. S. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.24.14001 · Cited by 395
Broome, C. V., Facklam, R. R., & Fraser, D. W. (1980) · New England Journal of Medicine · doi:10.1056/nejm198009043031003 · Cited by 338
Berkson's Bias, Selection Bias, and Missing Data
Westreich, D. (2012) · Epidemiology · doi:10.1097/ede.0b013e31823b6296 · Cited by 322
He, X.-S., Holmes, T. H., Zhang, C., Mahmood, K., Kemble, G. W., Lewis, D. B., Dekker, C. L., Greenberg, H. B., & Arvin, A. M. (2006) · Journal of Virology · doi:10.1128/jvi.01460-06 · Cited by 321
McLean, H. Q., Thompson, M. G., Sundaram, M. E., Meece, J. K., McClure, D. L., Friedrich, T. C., & Belongia, E. A. (2014) · Clinical Infectious Diseases · doi:10.1093/cid/ciu680 · Cited by 281
On the definition of a confounder
VanderWeele, T. J., & Shpitser, I. (2013) · The Annals of Statistics · doi:10.1214/12-aos1058 · Cited by 268 · cited 1×
“However, while controlling for a variable that represents an imperfect measure of a confounder may not eliminate bias, it can reduce it (41, 42).”
BackgroundThe case test-negative design for studies of the effectiveness of influenza vaccine
Foppa, I. M., Haber, M., Ferdinands, J. M., & Shay, D. K. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.04.026 · Cited by 266
Risk Factors, Confounding, and the Illusion of Statistical Control
Christenfeld, N. J. S., Sloan, R. P., Carroll, D., & Greenland, S. (2004) · Psychosomatic Medicine · doi:10.1097/01.psy.0000140008.70959.41 · Cited by 260 · cited 1×
“For confounder adjustment, causal diagrams have advantages over statistical approaches, such as stepwise regression or change-in-estimate criteria, which may select nonconfounders or exclude important confounders and therefore introduce bias (70, 71).”
BackgroundImmunosenescence: influenza vaccination and the elderly
Haq, K., & McElhaney, J. E. (2014) · Current Opinion in Immunology · doi:10.1016/j.coi.2014.03.008 · Cited by 256 · cited 1×
“Age is a confounder because young children and the elderly are more likely to be infected due to immaturity of the immune system in children (21) or immunosenescence in the elderly (22); they may also be more or less likely to be vaccinated, depending on public health strategies and other factors (23).”
BackgroundOrenstein, E. W., De Serres, G., Haber, M. J., Shay, D. K., Bridges, C. B., Gargiullo, P., & Orenstein, W. A. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym021 · Cited by 254
Patterns of Shedding of Myxoviruses and Paramyxoviruses in Children
Frank, A. L., Taber, L. H., Wells, C. R., Wells, J. M., Glezen, W. P., & Paredes, A. (1981) · Journal of Infectious Diseases · doi:10.1093/infdis/144.5.433 · Cited by 249
Preliminary Assessment of the Efficacy of a T-Cell–Based Influenza Vaccine, MVA-NP+M1, in Humans
Lillie, P. J., Berthoud, T. K., Powell, T. J., Lambe, T., Mullarkey, C., Spencer, A. J., Hamill, M., Peng, Y., Blais, M.-E., Duncan, C. J. A., et al. (2012) · Clinical Infectious Diseases · doi:10.1093/cid/cis327 · Cited by 248
De Serres, G., Skowronski, D. M., Wu, X. W., & Ambrose, C. S. (2013) · Eurosurveillance · doi:10.2807/1560-7917.es2013.18.37.20585 · Cited by 245
Invited Commentary: Causal Diagrams and Measurement Bias
Hernan, M. A., & Cole, S. R. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp293 · Cited by 220
Kissling, E., Valenciano, M., Larrauri, A., Oroszi, B., Cohen, J. M., Nunes, B., Pitigoi, D., Rizzo, C., Rebolledo, J., Paradowska-Stankiewicz, I., et al. (2013) · Eurosurveillance · doi:10.2807/ese.18.05.20390-en · Cited by 195
Influenza Vaccine Effectiveness in the Community and the Household
Ohmit, S. E., Petrie, J. G., Malosh, R. E., Cowling, B. J., Thompson, M. G., Shay, D. K., & Monto, A. S. (2013) · Clinical Infectious Diseases · doi:10.1093/cid/cit060 · Cited by 188
Self-citationSullivan, S. G., Feng, S., & Cowling, B. J. (2014) · Expert Review of Vaccines · doi:10.1586/14760584.2014.966695 · Cited by 183
Self-citationBarr, I. G., McCauley, J., Cox, N., Daniels, R., Engelhardt, O. G., Fukuda, K., Grohmann, G., Hay, A., Kelso, A., Klimov, A., et al. (2010) · Vaccine · doi:10.1016/j.vaccine.2009.11.043 · Cited by 179
Validation of influenza and pneumococcal vaccine status in adults based on self-report
MANGTANI, P., SHAH, A., & ROBERTS, J. A. (2007) · Epidemiology and Infection · doi:10.1017/s0950268806006479 · Cited by 169
The concept of residual confounding in regression models and some applications
Becher, H. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111308 · Cited by 168
Evaluation of self-reported and registry-based influenza vaccination status in a Wisconsin cohort
Irving, S. A., Donahue, J. G., Shay, D. K., Ellis-Coyle, T. L., & Belongia, E. A. (2009) · Vaccine · doi:10.1016/j.vaccine.2009.08.050 · Cited by 165
Using causal diagrams to guide analysis in missing data problems
Daniel, R. M., Kenward, M. G., Cousens, S. N., & De Stavola, B. L. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280210394469 · Cited by 155 · cited 1×
“DAGs, being nonparametric in nature, cannot convey these properties of the odds ratio (19).”
BackgroundJurek, A. M., Greenland, S., & Maldonado, G. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dym291 · Cited by 152
WONG, V. W. Y., COWLING, B. J., & AIELLO, A. E. (2014) · Epidemiology and Infection · doi:10.1017/s095026881400003x · Cited by 146
Self-citationLi, C., Wang, L., Eng, H., You, H., Chang, L., Tang, K.-S., Lin, Y.-J., Kuo, H.-C., Lee, I., Liu, J.-W., et al. (2010) · Emerging infectious diseases · doi:10.3201/eid1608.091918 · Cited by 144
Druce, J. D., Tran, T., Kelly, H. A., Kaye, M., Chibo, D., Kostecki, R., Amiri, A., Catton, M. C., & Birch, C. J. (2004) · Journal of Medical Virology · doi:10.1002/jmv.20246 · Cited by 122
Viral Shedding Patterns of Children with Influenza B Infection
Hall, C. B., Douglas, R. G., Geiman, J. M., & Meagher, M. P. (1979) · The Journal of Infectious Diseases · doi:10.1093/infdis/140.4.610 · Cited by 97
Jackson, M. L., & Rothman, K. J. (2015) · Vaccine · doi:10.1016/j.vaccine.2015.01.069 · Cited by 88
Small-sample bias and corrections for conditional maximum-likelihood odds-ratio estimators
Greenland, S. (2000) · Biostatistics · doi:10.1093/biostatistics/1.1.113 · Cited by 88
Graphical Models for Inference Under Outcome-Dependent Sampling
Didelez, V., Kreiner, S., & Keiding, N. (2010) · Statistical Science · doi:10.1214/10-sts340 · Cited by 84
Kelly, H. A., Jacoby, P. A., Dixon, G. A., Carcione, D., Williams, S. H., Moore, H. C., Smith, D. W., Keil, A. D., Buynder, P. G. V., & Richmond, P. (2010) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0b013e318201811c · Cited by 74
Potential Confounding by Exposure History and Prior Outcomes: An Example From Perinatal Epidemiology
Howards, P. P., Schisterman, E. F., & Heagerty, P. J. (2007) · Epidemiology · doi:10.1097/ede.0b013e31812001e6 · Cited by 70
Greenland, S., Lanes, S., & Jara, M. (2008) · Clinical Trials · doi:10.1177/1740774507087703 · Cited by 68
On the Nondifferential Misclassification of a Binary Confounder
Ogburn, E. L., & VanderWeele, T. J. (2012) · Epidemiology · doi:10.1097/ede.0b013e31824d1f63 · Cited by 63
Beck, C., McKenzie, B. C., Hashim, A. B., Harris, R. C., Zanuzdana, A., Agboado, G., Orton, E., Béchard‐Evans, L., Morgan, G. S., Stevenson, C. N., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0029249 · Cited by 62
Valenciano, M., Ciancio, B. C., Moren, A., & Collective the influenza vaccine effectiveness working group (2008) · Eurosurveillance · doi:10.2807/ese.13.43.19015-en · Cited by 52
The Case-Control Study as Data Missing by Design: Estimating Risk Differences
Wacholder, S. (1996) · Epidemiology · doi:10.1097/00001648-199603000-00007 · Cited by 41
Pascalis, H., Temmam, S., Turpin, M., Rollot, O., Flahault, A., Carrat, F., de Lamballerie, X. N., Gérardin, P., & Dellagi, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0044755 · Cited by 38
Levy, A., Sullivan, S. G., Tempone, S., Wong, K. L., Regan, A. K., Dowse, G. K., Effler, P. V., & Smith, D. W. (2014) · Vaccine · doi:10.1016/j.vaccine.2014.08.066 · Cited by 28
Self-citationRothman, K. J., Greenland, S., & Lash, T. L. (2007) · Handbook of statistics · doi:10.1016/s0169-7161(07)27003-8 · Cited by 23
Howard, P. F., McCaw, J. M., Richmond, P., Nissen, M. D., Sloots, T. P., Lambert, S. B., Lai, M. H., Greenberg, M. E., Nolan, T. M., & McVernon, J. (2012) · Influenza and Other Respiratory Viruses · doi:10.1111/j.1750-2659.2012.00395.x · Cited by 22
Widdowson, M., & Monto, A. S. (2013) · Wiley eBooks · doi:10.1002/9781118636817.ch16 · Cited by 14
Cited by
Lauring, A. S., Tenforde, M. W., Chappell, J. D., Gaglani, M., Ginde, A. A., McNeal, T., Ghamande, S., Douin, D. J., Talbot, H. K., Casey, J. D., et al. (2022) · BMJ · doi:10.1136/bmj-2021-069761 · Cited by 656
Accorsi, E. K., Britton, A., Fleming-Dutra, K. E., Smith, Z. R., Shang, N., Derado, G., Miller, J., Schrag, S. J., & Verani, J. R. (2022) · JAMA · doi:10.1001/jama.2022.0470 · Cited by 628
Effectiveness of mRNA-1273 against SARS-CoV-2 omicron and delta variants
Tseng, H. F., Ackerson, B. K., Luo, Y., Sy, L. S., Talarico, C. A., Tian, Y., Bruxvoort, K. J., Tubert, J. E., Florea, A., Ku, J. H., et al. (2022) · doi:10.1101/2022.01.07.22268919 · Cited by 541
Ranzani, O. T., Hitchings, M. D. T., Dorion, M., D’Agostini, T. L., de Paula, R. C., de Paula, O. F. P., Villela, E. F. D. M., Torres, M. S. S., de Oliveira, S. B., Schulz, W., et al. (2021) · BMJ · doi:10.1136/bmj.n2015 · Cited by 348
SARS-CoV-2 breakthrough infections in vaccinated individuals: measurement, causes and impact
Lipsitch, M., Krammer, F., Regev-Yochay, G., Lustig, Y., & Balicer, R. D. (2022) · Nature Reviews Immunology · doi:10.1038/s41577-021-00662-4 · Cited by 336
Metadata
- Authors
- Sullivan, S. G., Tchetgen Tchetgen, E. J., & Cowling, B. J.
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-08-31
- Volume
- 184
- Issue
- 5
- Pages
- 345-353
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2507826825
- Cited by
- 319
- References
- 62
- FWCI
- 18.26
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.