Depletion-of-susceptibles bias in influenza vaccine waning studies: how to ensure robust results
Open accessPublished version of a preprint
Abstract
Vaccine effectiveness studies are subject to biases due to depletion-of-persons at risk of infection, or at especially high risk of infection, at different rates from different groups (depletion-of-susceptibles bias), a problem that can also lead to biased estimates of waning effectiveness, including spurious inference of waning when none exists. An alternative study design to identify waning is to study only vaccinated persons, and compare for each day the incidence in persons with earlier or later dates of vaccination to assess waning in vaccine protection as a function of vaccination time (namely whether earlier vaccination would result in lower subsequent protection compared to later vaccination). Prior studies suggested under what conditions this alternative would yield correct estimates of waning. Here we define the depletion-of-susceptibles process formally and show mathematically that for influenza vaccine waning studies, a randomised trial or corresponding observational study that compares incidence at a specific calendar time among individuals vaccinated at different times before the influenza season begins will not be vulnerable to depletion-of-susceptibles bias in its inference of waning as a function of vaccination time under the null hypothesis that none exists, and will – if waning does actually occur – underestimate the extent of waning. Such a design is thus robust in the sense that a finding of waning in that inference framework reflects actual waning of vaccine-induced immunity. We recommend such a design for future studies of waning, whether observational or randomised.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Depletion of susceptibles could lead to spuriously lower vaccine effectiveness estimates during the BA.4/5 period.
“This so-called ''depletion of susceptibles' could lead to spuriously lower VE estimates during the BA.4/5 period [38].”
From Assessing COVID-19 vaccine effectiveness against Omicron subvariants: Report from a meeting of the World Health Organization (2023) · cites this paper as [38]
The Abstract explains that depletion of susceptibles can make vaccine protection appear to wane, supporting the proposed direction of bias. The paper analyzes influenza vaccination, however, and does not establish that this bias affected vaccine-effectiveness estimates during the BA.4/5 period. The supplied passages do not give page numbers. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1× · see citing works
“We note that this analysis considers only the biases that result from susceptible depletion (which can be seen as a form of selection bias [14]).”
BackgroundHernán, M. Á. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181c1ea43 · Cited by 1,367 · see citing works
Heterogeneity's Ruses: Some Surprising Effects of Selection on Population Dynamics
Vaupel, J. W., & Yashin, A. I. (1985) · The American Statistician · doi:10.2307/2683925 · Cited by 796 · see citing works
Sullivan, S. G., Tchetgen, E. J. T., & Cowling, B. J. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kww064 · Cited by 316 · see citing works
Ferdinands, J. M., Fry, A. M., Reynolds, S. B., Petrie, J. G., Flannery, B. L., Jackson, M. L., & Belongia, E. A. (2016) · Clinical Infectious Diseases · doi:10.1093/cid/ciw816 · Cited by 197 · see citing works
Lipsitch, M., Jha, A., & Simonsen, L. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw124 · Cited by 124 · see citing works
Self-citationMeasurement of Vaccine Direct Effects Under the Test-Negative Design
Lewnard, J. A., Tedijanto, C., Cowling, B. J., & Lipsitch, M. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwy163 · Cited by 122 · see citing works
Self-citationEstimability and Interpretation of Vaccine Efficacy Using Frailty Mixing Models
Halloran, M. E., Longini, I. M., & Struchiner, C. J. (1996) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a008858 · Cited by 121 · see citing works
Intraseason Waning of Influenza Vaccine Effectiveness
Ray, G. T., Lewis, N., Klein, N. P., et al. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/ciy770 · Cited by 118 · see citing works
Self-citationWaning protection of influenza vaccination during four influenza seasons, 2011/2012 to 2014/2015
Puig‐Barberà, J., Mira‐Iglesias, A., Tortajada‐Girbés, M., López‐Labrador, F. X., Librero, J., Díez‐Domingo, J., Carballido‐Fernández, M., Carratalá‐Munuera, C., Medina, P. C., Gil-Guillén, V. F., et al. (2017) · Vaccine · doi:10.1016/j.vaccine.2017.09.035 · Cited by 59 · see citing works
Challenges of Vaccine Effectiveness and Waning Studies
Lipsitch, M. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/ciy773 · Cited by 55 · see citing works
Self-citationHow host heterogeneity governs tuberculosis reinfection?
Gomes, M. G. M., Aguas, R. J., Lopes, J. S., Nunes, M. C., Rebelo, C., Rodrigues, P., & Struchiner, C. J. (2012) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2011.2712 · Cited by 54 · see citing works
Depletion-of-susceptibles Bias in Analyses of Intra-season Waning of Influenza Vaccine Effectiveness
Ray, G. T., Lewis, N., Klein, N. P., Daley, M. F., Lipsitch, M., & Fireman, B. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciz706 · Cited by 48 · see citing works
Self-citationAnalyzing Vaccine Trials in Epidemics With Mild and Asymptomatic Infection
Kahn, R. J., Hitchings, M. D. T., Wang, R., et al. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwy239 · Cited by 23 · see citing works
If Influenza Vaccines Wane, Can We Delay Vaccination Without Compromising Coverage?
Klein, N. P., & FIREMAN, B. H. (2019) · Clinical Infectious Diseases · doi:10.1093/cid/ciz459 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [7]O'Hagan JJ (2012) Apparent declining efficacy in randomized trials: examples of the Thai RV144 HIV vaccine and South African CAPRISA 004 microbicide trials. AIDS (London, England) 26, 123– 126. PMC3319457 22045345
Cited by
Sorted by citations
BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting
Dagan, N., Barda, N., Kepten, E., Miron, O., Perchik, S., Katz, M. A., Hernán, M. A., Lipsitch, M., Reis, B., & Balicer, R. D. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2101765 · Cited by 2,527 · see citing works
Self-citationKatikireddi, S. V., Cerqueira-Silva, T., Vasileiou, E., Robertson, C., Amele, S., Pan, J., Taylor, B., Boaventura, V., Werneck, G. L., Flores-Ortiz, R., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)02754-9 · Cited by 124 · see citing works
Feikin, D. R., Higdon, M. M., Andrews, N., Collie, S., Deloria Knoll, M., Kwong, J. C., Link-Gelles, R., Pilishvili, T., & Patel, M. K. (2023) · Vaccine · doi:10.1016/j.vaccine.2023.02.020 · Cited by 36 · see citing works
“Additionally, a larger percentage of hospitalizations during the Omicron period likely had incidental, rather than causal, infections with SARS-CoV-2, and infections would be more affected by true waning of VE.[38], [42]”
Background
- Authors
- Lipsitch, M., Goldstein, E., Ray, G. T., & Fireman, B.
- Affiliations
- Harvard University (US)
- Kaiser Permanente (US)
- Corresponding author
- Lipsitch, M.
- Venue
- Epidemiology and Infection
- Publisher
- Cambridge University Press (CUP)
- Published
- 2019
- Volume
- 147
- Pages
- e306-e306
- ISSN
- 0950-2688 (ISSN-L)
- 1469-4409
- PMC
- PMC7003633
- OpenAlex
- W2990987718
- Cited by
- 56
- References
- 15
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 11 · 2023: 9 · 2022: 14 · 2021: 7 · 2020: 1
- FWCI
- 4.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Epidemiology and Infection (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- Government of the United Kingdom
- National Institutes of Health
- National Institute of General Medical Sciences (U54GM088558)
- National Institute of Allergy and Infectious Diseases (R01AI107721, T32AI007535)
- Topics
- Health Sciences › Medicine › Epidemiology › Influenza Virus Research Studies
- Social Sciences › Health › Vaccine Coverage and Hesitancy
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- influenza vaccination, vaccine waning, vaccine effectiveness, vaccine-induced immunity, research design, vaccination timing
- MeSH terms
- Disease Susceptibility, Humans, Influenza, Human, Influenza Vaccines, Models, Biological, Research Design, Bias, Randomized Controlled Trials as Topic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
