CoChat
Sign Up
Cited by 1,031SARS-CoV-2 and COVID-19 ResearchVaccine Coverage and HesitancyCOVID-19 Clinical Research Studies

Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study

Open access
Menni, Cristina, Klaser, Kerstin, May, Anna, Polidori, Lorenzo, Capdevila, Joan, Louca, Panayiotis, Sudre, Carole H, Nguyen, Long H +20 more·2021·The Lancet Infectious Diseases
DOI 10.1016/s1473-3099(21)00224-3PMC PMC8078878
Open this paper in CoChat — free accountOpen PDF

Abstract

Background The Pfizer-BioNTech (BNT162b2) and the Oxford-AstraZeneca (ChAdOx1 nCoV-19) COVID-19 vaccines have shown excellent safety and efficacy in phase 3 trials. We aimed to investigate the safety and effectiveness of these vaccines in a UK community setting. Methods In this prospective observational study, we examined the proportion and probability of self-reported systemic and local side-effects within 8 days of vaccination in individuals using the COVID Symptom Study app who received one or two doses of the BNT162b2 vaccine or one dose of the ChAdOx1 nCoV-19 vaccine. We also compared infection rates in a subset of vaccinated individuals subsequently tested for SARS-CoV-2 with PCR or lateral flow tests with infection rates in unvaccinated controls. All analyses were adjusted by age (≤55 years vs >55 years), sex, health-care worker status (binary variable), obesity (BMI <30 kg/m 2 vs ≥30 kg/m 2 ), and comorbidities (binary variable, with or without comorbidities). Findings Between Dec 8, and March 10, 2021, 627 383 individuals reported being vaccinated with 655 590 doses: 282 103 received one dose of BNT162b2, of whom 28 207 received a second dose, and 345 280 received one dose of ChAdOx1 nCoV-19. Systemic side-effects were reported by 13·5% (38 155 of 282 103) of individuals after the first dose of BNT162b2, by 22·0% (6216 of 28 207) after the second dose of BNT162b2, and by 33·7% (116 473 of 345 280) after the first dose of ChAdOx1 nCoV-19. Local side-effects were reported by 71·9% (150 023 of 208 767) of individuals after the first dose of BNT162b2, by 68·5% (9025 of 13 179) after the second dose of BNT162b2, and by 58·7% (104 282 of 177 655) after the first dose of ChAdOx1 nCoV-19. Systemic side-effects were more common (1·6 times after the first dose of ChAdOx1 nCoV-19 and 2·9 times after the first dose of BNT162b2) among individuals with previous SARS-CoV-2 infection than among those without known past infection. Local effects were similarly higher in individuals previously infected than in those without known past infection (1·4 times after the first dose of ChAdOx1 nCoV-19 and 1·2 times after the first dose of BNT162b2). 3106 of 103 622 vaccinated individuals and 50 340 of 464 356 unvaccinated controls tested positive for SARS-CoV-2 infection. Significant reductions in infection risk were seen starting at 12 days after the first dose, reaching 60% (95% CI 49–68) for ChAdOx1 nCoV-19 and 69% (66–72) for BNT162b2 at 21–44 days and 72% (63–79) for BNT162b2 after 45–59 days. Interpretation Systemic and local side-effects after BNT162b2 and ChAdOx1 nCoV-19 vaccination occur at frequencies lower than reported in phase 3 trials. Both vaccines decrease the risk of SARS-CoV-2 infection after 12 days. Funding ZOE Global, National Institute for Health Research, Chronic Disease Research Foundation, National Institutes of Health, UK Medical Research Council, Wellcome Trust, UK Research and Innovation, American Gastroenterological Association.

AI snapshot

Written from the paper’s full text.

TL;DR

In UK users of the COVID Symptom Study app, self-reported side-effects after BNT162b2 and ChAdOx1 nCoV-19 vaccination were less frequent than in phase 3 trials and were mostly mild and short-lived. Both vaccines were associated with reduced SARS-CoV-2 infection risk starting 12 days after the first dose.

Key findings

  • Among 627,383 vaccinated individuals reporting 655,590 doses, systemic side-effects occurred in 13.5% after first-dose BNT162b2, 22.0% after second-dose BNT162b2, and 33.7% after first-dose ChAdOx1 nCoV-19; local side-effects occurred in 71.9%, 68.5%, and 58.7%, respectively.
  • Side-effects were more common in people with previous SARS-CoV-2 infection: systemic effects were 1.6 times higher after first-dose ChAdOx1 and 2.9 times higher after first-dose BNT162b2; local effects were 1.4 times and 1.2 times higher, respectively.
  • Infection risk reductions versus unvaccinated controls began at 12 days: BNT162b2 reduced risk by 58% (95% CI 54–62) at 12–20 days, 69% (66–72) at 21–44 days, and 72% (63–79) at 45–59 days; ChAdOx1 reduced risk by 39% (21–53) at 12–20 days and 60% (49–68) at 21–44 days.
  • Overall, 3106 of 103,622 vaccinated individuals and 50,340 of 464,356 unvaccinated controls tested positive for SARS-CoV-2.

Methods

  • Prospective observational study using self-reported data from the UK COVID Symptom Study app between Dec 8, 2020, and March 10, 2021; participants reported vaccination and side-effects within 8 days, and a subset reported PCR or lateral flow test results after vaccination.
  • Side-effect analyses adjusted for age (≤55 vs >55 years), sex, health-care worker status, obesity (BMI <30 vs ≥30 kg/m²), and comorbidities using Pearl’s back-door adjustment; infection analyses used Poisson regressions adjusted for testing and incidence.
  • Infection rates in vaccinated individuals were compared with unvaccinated app users who reported a COVID-19 test in the same week as a vaccinated user.

Limitations

  • The paper states that self-reported data can introduce information bias, misclassification, or exposure bias; app users were a self-selected group not representative of the general population; collider bias is possible; ChAdOx1 recipients had not received two doses at the time; reporting completeness was higher for systemic than local effects; severe side-effects might have been missed if users could not report them; testing selection bias is possible; the observational design does not allow causal inference; only short-term adverse effects were evaluated; side-effects within 1 week may not be vaccine related; and power was insufficient to assess differential rates by ethnic group.
  • The text also makes evident that previous infection status and test results relied on self-report, and stratified infection models pooled vaccine types because some strata had relatively small sample sizes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/s1473-3099(21)00224-3

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 30 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Systemic side-effects were reported by 33·7% (116 473 of 345 280) after the first dose of ChAdOx1 nCoV-19.

    “by 33·7% (116 473 of 345 280) after the first dose of ChAdOx1 nCoV-19”
    Quote in the paperNot a substantial claim
  • Findingown data

    3106 of 103 622 vaccinated individuals and 50 340 of 464 356 unvaccinated controls tested positive for SARS-CoV-2 infection.

    “3106 of 103 622 vaccinated individuals and 50 340 of 464 356 unvaccinated controls tested positive for SARS-CoV-2 infection”
    Quote in the paperSubstantial claim
  • Findingown data

    Systemic side-effects were 1·6 times more common among individuals with previous SARS-CoV-2 infection than among those without known past infection after the first dose of ChAdOx1 nCoV-19.

    “Systemic side-effects were more common (1·6 times after the first dose of ChAdOx1 nCoV-19”
    Quote in the paperSubstantial claim
  • Findingown data

    Local effects were 1·2 times higher in individuals previously infected than in those without known past infection after the first dose of BNT162b2.

    “and 1·2 times after the first dose of BNT162b2)”
    Quote in the paperNot a substantial claim
  • Findingown data

    BNT162b2 vaccination was associated with a 72% reduction in infection risk at 45-59 days after the first dose compared with unvaccinated controls.

    “and 72% (63–79) after 45–59 days following BNT162b2”
    Quote in the paperSubstantial claim
  • May combine two claims. 30
  • Classified as probably not a substantial claim. 15

  • May combine two claims.

    Findingown data

    Systemic side-effects were most frequently reported within the first 24 h after vaccination and lasted a mean of 1·01 days (SD 0·1).

    “These were most frequently reported within the first 24 h after vaccination and lasted a mean of 1·01 days (SD 0·1; figure 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    There was a modest association between symptom reporting and socioeconomic status measured as index of multiple deprivation (r=0·021 [95% CI 0·019-0·025]), corresponding to 0·04% of the variance in symptoms reporting.

    “we found a modest association (r=0·021 [95% CI 0·019–0·025]), corresponding to 0·04% of the variance in symptoms reporting.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] Ministry of Housing, Communities, and Local Government English indices of deprivation 2019 https://www.gov.uk/government/statistics/english-indices-of-deprivation-2019 Sept 26, 2019

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Users who reported receiving a BNT162b2 vaccine were slightly younger, had more comorbidities, and were more frequently female than users who reported receiving a ChAdOx1 nCoV-19 inoculation.

    “Users who reported receiving a BNT162b2 vaccine were slightly younger, had more comorbidities, and were more frequently female than users who reported receiving a ChAdOx1 nCoV-19 inoculation”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Reported side-effects were minor in severity and of short duration.

    “Reported side-effects were minor in severity and of short duration.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Tenderness and local pain around the injection site were the most frequently reported local effects, occurring most often on the day after injection and lasting a mean of 1·02 days (SD 0·15).

    “Tenderness and local pain around the injection site were the most frequently reported local effects (table), occurring most often on the day after injection and lasting a mean of 1·02 days (SD 0·15; figure 1).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The phase 2-3 trial of the ChAdOx1 nCoV-19 vaccine reported systemic adverse effects in 88% of participants aged 18-55 years who received the first injection, whereas this study found a lower rate of 33.7% after the first dose in the overall sample and 46.9% in individuals aged 18-55 years.

    “The phase 2–3 trial of the ChAdOx1 nCoV-19 vaccine7 reported systemic adverse effects in 88% of participants aged 18–55 years who received the first injection, whereas we found a lower rate of 33·7% after the first dose in the overall sample and 46·9% in individuals aged 18–55 years (data not shown).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] Ramasamy MN Minassian AM Ewer KJ Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): a single-blind, randomised, controlled, phase 2/3 trial Lancet 396 2021 1979 1993 33220855 PMC7674972

      Not checked in the free run.

    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • May combine two claims.

    Findingown data

    Borderline differences were observed for BMI (RR for BMI <30 kg/m2 −69% [95% CI −71 to −67]; RR for BMI ≥30 kg/m2 −63% [−67 to −59]) and sex (RR for female −69% [95% CI −71 to −67]; RR for male −61% [−66 to −57]).

    “Borderline differences were observed for BMI (RR for BMI <30 kg/m2 −69% [95% CI −71 to −67]; RR for BMI ≥30 kg/m2 −63% [−67 to −59]) and sex (RR for female −69% [95% CI −71 to −67]; RR for male −61% [–66 to −57]; appendix p 5).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the community analysis, less than 30% of users complained of injection-site pain and less than 25% of fatigue and headache after the first dose.

    “However, in our community analysis, less than 30% of users complained of injection-site pain and less than 25% of fatigue and headache after the first dose.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • May combine two claims.

    Findingown data

    Systemic adverse effects, including headache and fatigue, affected fewer than one in four people and were less common in the community than expected from clinical trials.

    “We found that systemic adverse effects, including headache and fatigue, affected fewer than one in four people and were less common in the community than expected from clinical trials.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Polack FP Thomas SJ Kitchin N Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine N Engl J Med 383 2020 2603 2615 33301246 PMC7745181

      Not checked in the free run.

    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • May combine two claims.

    Findingreasoning

    The overall mean age of the vaccinated app users was higher than that of the general population (40.3 years) yet was lower than those of the samples within other UK COVID-19 effectiveness studies.

    “The overall mean age of the vaccinated app users was higher than that of the general population (40·3 years)24 yet was lower than those of the samples within other UK COVID-19 effectiveness studies,11, 12 largely because of the presence of a small proportion of health-care workers among the participants of this study.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [24] Office for National Statistics Population estimates for the UK, England and Wales, Scotland and Northern Ireland, provisional: mid-2019 https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/bulletins/annualmidyearpopulationestimates/mid2019 May 6, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [11] Lopez Bernal J Andrews N Gower C Early effectiveness of COVID-19 vaccination with BNT162b2 mRNA vaccine and ChAdOx1 adenovirus vector vaccine on symptomatic disease, hospitalisations and mortality in older adults in England medRxiv 2021 published online March 2. 10.1101/2021.03.01.21252652 (preprint). source

      Not checked in the free run.

    • cites [12] Vasileiou E Simpson CR Shi T Interim findings from first-dose mass COVID-19 vaccination roll-out and COVID-19 hospital admissions in Scotland: a national prospective cohort study Lancet 2021 published online April 23. 10.1016/S0140-6736(21)00677-2 PMC8064669 33901420 source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Side-effects were significantly more prevalent in women than in men, in people aged 55 years or younger than in those older than 55 years, and after the second than after the first dose.

    “Although side-effects were significantly more prevalent in women than in men, in people aged 55 years or younger than in those older than 55 years, and after the second than after the first dose, they occurred at much lower frequencies than expected from the published literature.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • May combine two claims.

    Findingown data

    Whereas 51-59% of participants reported fatigue after the second BNT162b2 dose in the phase 3 trial, fatigue was reported by less than 15% of participants after the second dose in this study.

    “For instance, whereas 51–59% of participants reported fatigue after the second BNT162b2 dose in the phase 3 trial of that vaccine,3 fatigue was reported by less than 15% of participants after the second dose in our study.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Polack FP Thomas SJ Kitchin N Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine N Engl J Med 383 2020 2603 2615 33301246 PMC7745181

      Not checked in the free run.

    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • May combine two claims.

    Findingown data

    There was a further reduction in infection risk after one dose of BNT162b2 vaccine compared with unvaccinated controls at 21-44 days after vaccination (RR −69% [95% CI −72 to −66]) and at 45-59 days after vaccination (−72% [−79 to −63]).

    “We observed a further reduction in infection risk after one dose of the BNT162b2 vaccine when compared with unvaccinated controls at 21–44 days after vaccination (RR −69% [95% CI −72 to −66]) and at 45–59 days after vaccination (−72% [–79 to −63]; figure 4).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In this study, 89% of respondents who logged at least one systemic effect after the ChAdOx1 nCoV-19 vaccine did not report any systemic effects after 3 days, and 98.3% did not report any after 1 week.

    “but in our study 89% of respondents who logged at least one systemic effect after the ChAdOx1 nCoV-19 vaccine did not report any systemic effects after 3 days, and 98·3% did not report any after 1 week.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Phase 3 trials reported the BNT162b2 vaccine to have an efficacy of 52% at 12 days after the first dose and of 95% after the second dose if administered 3-4 weeks apart in participants without previous SARS-CoV-2 infection.

    “Phase 3 trials reported the BNT162b2 vaccine to have an efficacy of 52% at 12 days after the first dose and of 95% after the second dose if administered 3–4 weeks apart in participants without previous SARS-CoV-2 infection.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] Polack FP Thomas SJ Kitchin N Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine N Engl J Med 383 2020 2603 2615 33301246 PMC7745181

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    We did not collect information on why individuals were tested, so not all tested individuals were necessarily experiencing COVID-19-associated symptoms at testing, and some individuals might have been routinely tested while being asymptomatic.

    “We did not collect information on why individuals were tested, so not all tested individuals were necessarily experiencing COVID-19-associated symptoms at testing, and some individuals might have been routinely tested while being asymptomatic.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The Poisson regression model allowed us to control for the number of follow-up days for each group.

    “This model allowed us to control for the number of follow-up days for each group.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The Poisson regression model correctly specifies the relationship and the included covariates (including incidence) adequately control for temporal and other confounders.

  • Classified as probably not a substantial claim.

    Premisecitation

    The BNT162b2 phase 3 trial showed a decrease in infections from 12 days after vaccination.

    “Since the BNT162b2 phase 3 trial showed a decrease in infections from 12 days after vaccination”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Polack FP Thomas SJ Kitchin N Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine N Engl J Med 383 2020 2603 2615 33301246 PMC7745181

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    All analyses were adjusted by age (≤55 years vs >55 years), sex, health-care worker status, obesity (BMI <30 kg/m2 vs ≥30 kg/m2), and comorbidities (binary variable, with or without comorbidities).

    “All analyses were adjusted by age (≤55 years vs >55 years), sex, health-care worker status (binary variable), obesity (BMI <30 kg/m2vs ≥30 kg/m2), and comorbidities (binary variable, with or without comorbidities).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The adjustment set (age, sex, health-care worker status, obesity, and comorbidities) is sufficient to control for confounding in the observational comparisons.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    We included incidence as a covariate in the Poisson regression model.

    “we also included incidence as a covariate in the Poisson regression model”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The Poisson regression model correctly specifies the relationship and the included covariates (including incidence) adequately control for temporal and other confounders.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Due to a relatively small sample size in some of the strata, we did not differentiate between vaccine types but pooled all vaccinated contributors.

    “Due to a relatively small sample size in some of the strata, we did not differentiate between vaccine types but pooled all vaccinated contributors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The ChAdOx1 nCoV-19 trial found efficacy against symptomatic disease of 76% at 22-90 days after at least one standard dose.

    “The ChAdOx1 nCoV-19 trial found efficacy against symptomatic disease of 76% at 22–90 days after at least one standard dose.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [6] Voysey M Clemens SAC Madhi SA Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK Lancet 397 2021 99 111 33306989 PMC7723445 source

      Not checked in the free run.

    • cites [7] Ramasamy MN Minassian AM Ewer KJ Safety and immunogenicity of ChAdOx1 nCoV-19 vaccine administered in a prime-boost regimen in young and old adults (COV002): a single-blind, randomised, controlled, phase 2/3 trial Lancet 396 2021 1979 1993 33220855 PMC7674972 source

      Not checked in the free run.

    • cites [8] Voysey M Clemens SAC Madhi SA Single-dose administration and the influence of the timing of the booster dose on immunogenicity and efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine: a pooled analysis of four randomised trials Lancet 397 2021 881 891 33617777 PMC7894131 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Adverse effects are more frequently reported in younger individuals, women, and among those who previously had COVID-19.

    “Adverse effects are more frequently reported in younger individuals, women, and among those who previously had COVID-19.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported previous positive PCR or lateral flow result accurately identifies past SARS-CoV-2 infection.
    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

  • May combine two claims.

    Study design and conductnone given

    Strengths of the study include its large sample size; capture of data on SARS-CoV-2 RT-PCR or lateral flow test results, regardless of symptoms; the prospective real-time capture of information on symptoms; and the availability of both BNT162b2 and ChAdOx1 nCoV-19 vaccines in the UK, which allowed cross-vaccine comparison.

    “Strengths of our study include its large sample size; capture of data on SARS-CoV-2 RT-PCR or lateral flow test results, regardless of symptoms; the prospective real-time capture of information on symptoms; and the availability of both BNT162b2 and ChAdOx1 nCoV-19 vaccines in the UK, which allowed cross-vaccine comparison.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Recipients of the ChAdOx1 nCoV-19 vaccine might differ from recipients of the BNT162b2 vaccine by age or dependency.

    “Recipients of the ChAdOx1 nCoV-19 vaccine might differ from recipients of the BNT162b2 vaccine by age or dependency.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The observed decrease of infection over time seems to be in line with efficacy reported in the BNT162b2 phase 3 trial and supports the UK Government's decision to delay the timing of the second injection to 12 weeks.

    “Although our data, due to their observational nature, does not allow us to comment directly on effectiveness, the observed decrease of infection over time seems to be in line with efficacy reported in the BNT162b2 phase 3 trial3 and supports the UK Government's decision to delay the timing of the second injection to 12 weeks to maximise the number of people receiving at least one dose.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Polack FP Thomas SJ Kitchin N Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine N Engl J Med 383 2020 2603 2615 33301246 PMC7745181

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    Because the ChAdOx1 nCoV-19 vaccine started being rolled out in January, 2021, and the second dose is to be administered at 12 weeks, no app users had received two doses of ChAdOx1 nCoV-19 at the time of this report.

    “Furthermore, because the ChAdOx1 nCoV-19 vaccine started being rolled out in January, 2021, and the second dose is to be administered at 12 weeks, no app users had received two doses of ChAdOx1 nCoV-19 at the time of this report.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    The systemic side-effects were collected from daily reports within 1 week from the injection date, so the study cannot rule out that these effects might not be vaccine related.

    “Finally, the systemic side-effects were collected from daily reports within 1 week from the injection date, so we cannot rule out that these effects might not be vaccine related.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Previous work has shown that data from the app is able to produce estimates of population-level disease prevalence that agree well with surveys with random, representative designs, suggesting that behavioural issues are not substantially biasing the app population.

    “Previous work has shown that data from our app is able to produce estimates of population-level disease prevalence that agree well with surveys with random, representative designs,18, 28 suggesting that behavioural issues are not substantially biasing our app population.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [18] Varsavsky T Graham MS Canas LS Detecting COVID-19 infection hotspots in England using large-scale self-reported data from a mobile application: a prospective, observational study Lancet Public Health 6 2021 e21 e29 33278917 PMC7785969

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [28] Bowyer RCE Varsavsky T Thompson EJ Geo-social gradients in predicted COVID-19 prevalence in Great Britain: results from 1 960 242 users of the COVID-19 Symptoms Study app Thorax 2020 published online Dec 29. 10.1136/thoraxjnl-2020-215119 PMC8223682 33376145 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    Participants using the app were a self-selected group and not representative of the general population.

    “Participants using the app were a self-selected group and not representative of the general population, as has been observed in other digital platform studies.27”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] Baltrusaitis K Santillana M Crawley AW Chunara R Smolinski M Brownstein JS Determinants of participants' follow-up and characterization of representativeness in flu near you, a participatory disease surveillance system JMIR Public Health Surveill 3 2017 e18 28389417 PMC5400887

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    This is an observational study, with data captured during a specific timeframe, and the study design does not allow an inference of causality.

    “This is an observational study, with data captured during a specific timeframe, and our study design does not allow an inference of causality.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The data support results from randomised controlled trials in a large community-based scenario showing evidence of reduction in infection after 12 days and substantial protection after 3 weeks.

    “Furthermore, our data support results from randomised controlled trials in a large community-based scenario showing evidence of reduction in infection after 12 days and substantial protection after 3 weeks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The completeness of reporting was higher for systemic effects than for local effects, which might have introduced some bias.

    “The completeness of reporting was higher for systemic effects than for local effects, which might have introduced some bias (appendix p 12).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Vaccines have increased immunogenicity in individuals with past infection and these people have higher antibody titres than those without previous infection.

    “It has been shown that vaccines have increased immunogenicity in individuals with past infection and these people have higher antibody titres than those without previous infection.22, 23, 25”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [22] Krammer F Srivastava K Simon V Robust spike antibody responses and increased reactogenicity in seropositive individuals after a single dose of SARS-CoV-2 mRNA vaccine medRxiv 2021 published online Feb 1. 10.1101/2021.01.29.21250653 (preprint). source

      Not checked in the free run.

    • cites [23] Wise J COVID-19: people who have had infection might only need one dose of mRNA vaccine BMJ 372 2021 n308 33531333 source

      Not checked in the free run.

    • cites [25] Manisty C Otter AD Treibel TA Antibody response to first BNT162b2 dose in previously SARS-CoV-2-infected individuals Lancet 397 2021 1057 1058 33640038 PMC7972310 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The study evaluated only short-term adverse effects, and long-term surveillance in the general population will be required to investigate possible future effects.

    “Also, we evaluated only short-term adverse effects, and long-term surveillance in the general population will be required to investigate possible future effects.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The study used self-reported data, which can introduce information bias, including misclassification, or effect bias exposure.

    “We used self-reported data, which can introduce information bias, including misclassification, or effect bias exposure.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    Users of a participatory platform are likely to be more interested in health, and might behave differently to the general population as a result.

    “Users of a participatory platform (as well as participants in all voluntary studies, including clinical trials) are likely to be more interested in health, and might behave differently to the general population as a result.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Some severe side-effects might have been missed if app users experiencing them were unable to use the app to report side-effects.

    “Some severe side-effects might have been missed if app users experiencing them were unable to use the app to report side-effects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Some participants might be more likely to report symptoms than others, and there is the potential for users to drop out of reporting in the app.

    “Also, some participants might be more likely to report symptoms than others, and there is the potential for users to drop out of reporting in the app.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Short-term adverse effects of both vaccines are moderate in frequency, mild in severity, and short-lived.

    “In conclusion, short-term adverse effects of both vaccines are moderate in frequency, mild in severity, and short-lived.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported adverse effects are accurate enough to estimate their prevalence and associations.
40 of 40

  • The paper relies on this without support.

    COVID Symptom Study app users are representative of the UK general population.

    “The aim of this study was to investigate the adverse effects and infection rate of vaccinated people in a community (general population app users) scenario.”

  • The paper relies on this without support.

    Self-reported vaccination status, side-effects, and SARS-CoV-2 test results are accurate.

    “The app enables self-reported information related to SARS-CoV-2 infection to be captured.”

  • The paper relies on this without support.

    The adjustment set (age, sex, health-care worker status, obesity, and comorbidities) is sufficient to control for confounding in the observational comparisons.

    “we used Pearl's back-door adjustment16 to account for differences within the populations.”

  • The paper relies on this without support.

    The phase 3 trial populations and side-effect definitions are comparable to the app population, so frequencies can be compared.

    “Systemic and local side-effects after BNT162b2 and ChAdOx1 nCoV-19 vaccination occur at frequencies lower than reported in phase 3 trials.”

  • The paper relies on this without support.

    The Poisson regression model correctly specifies the relationship and the included covariates (including incidence) adequately control for temporal and other confounders.

    “we also included incidence as a covariate in the Poisson regression model”

  • The paper relies on this without support.

    Self-reported previous positive PCR or lateral flow result accurately identifies past SARS-CoV-2 infection.

    “previous SARS-CoV-2 infection (based on self-reported previous positive PCR or lateral flow result)”

  • The paper relies on this without support.

    Self-reported adverse effects are accurate enough to estimate their prevalence and associations.

    “We used self-reported data, which can introduce information bias, including misclassification, or effect bias exposure.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • These findings are in line with real-world evidence on COVID-19 mRNA vaccines (BNT162b2, mRNA-1273) which describe higher reactogenicity in females compared with males.

    “These findings are in line with the real-world evidence on COVID-19 mRNA vaccines (BNT162b2, mRNA-1273) which describe higher reactogenicity in females compared with males38,39.”

    From An Omicron-specific, self-amplifying mRNA booster vaccine for COVID-19: a phase 2/3 randomized trial (2024) · cites this paper as [39]

  • High-quality real-world safety data on the messenger RNA-based Covid-19 vaccines remain relatively sparse in the literature.

    “but high-quality real-world safety data on the messenger RNA (mRNA)–based Covid-19 vaccines remain relatively sparse in the literature.”

    From Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting (2021) · cites this paper as [12]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Systemic side-effects were not reported by fewer than one in four users

    Minor

    The discussion says systemic adverse effects affected fewer than one in four people, but the results report 25·4%-more than one in four.

1 of 1

Cites

Flagged first, then by citations

4 of the 31 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. MacKenna, 2021, Trends, regional variation, and clinical characteristics of COVID-19 vaccine recipients: a retrospective cohort study in 23·4 million patients using OpenSAFELY, medRxiv

Cited by

Sorted by citations

Related

Authors
Menni, C., Klaser, K., May, A., Polidori, L., Capdevila, J., Louca, P., Sudre, C. H., Nguyen, L. H., Drew, D. A., Merino, J., Hu, C., Selvachandran, S., Antonelli, M., Murray, B., Canas, L. S., Molteni, E., Graham, M. S., Modat, M., Joshi, A. D., Mangino, M., Hammers, A., Goodman, A. L., Chan, A. T., Wolf, J., Steves, C. J., Valdes, A. M., Ourselin, S., & Spector, T. D.
Affiliations
  • King's College London (GB)
  • ZOE (United Kingdom) (GB)
  • Harvard University (US)
  • Broad Institute (US)
  • St Thomas' Hospital (GB)
  • University of Nottingham (GB)
Corresponding author
Menni, Cristina
ORCID
Venue
The Lancet Infectious Diseases
Publisher
Elsevier BV
Published
2021-07
Volume
21
Issue
7
Pages
939-949
ISSN
1473-3099 (ISSN-L)
DOI
10.1016/s1473-3099(21)00224-3
PMC
PMC8078878
OpenAlex
W3158373340
Semantic Scholar
417264cadae077b05eac39e243c53085734f5548
Cited by
1,031
References
31
Citations by year
2026: 18 · 2025: 62 · 2024: 115 · 2023: 243 · 2022: 391 · 2021: 198 · 2020: 1 · 2013: 1
FWCI
74.92
Type
journal article
Language
en
Open access
Licence
Locations
  • The Lancet Infectious Diseases (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Portal (King's College London) (publishedVersion, cc-by)
  • Repository@Nottingham (University of Nottingham) (publishedVersion, cc-by)
  • Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
  • Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano) (submittedVersion, other-oa)
  • UCL Discovery (University College London) (submittedVersion)
  • IRIS eCampus Telematic University (Università degli Studi eCampus) (submittedVersion)
Funders
  • European Commission
  • Crohn's and Colitis Foundation (AGA2021-5102)
  • Newcastle upon Tyne Hospitals NHS Foundation Trust
  • Chronic Disease Research Foundation
  • British Heart Foundation (MR/M016560/1)
  • American Gastroenterological Association
  • Department of Health and Social Care
  • Wellcome Trust (212904/Z/18/Z, WT203148/Z/16/Z, WT213038/Z/18/Z)
  • Alzheimer's Society
  • King's College London
  • King's College Hospital NHS Foundation Trust
  • American Diabetes Association (7-21-JDFM-005)
  • Engineering and Physical Sciences Research Council
  • National Institute of Diabetes and Digestive and Kidney Diseases (DK40561, K01DK120742)
  • National Institute for Health Research
  • Massachusetts General Hospital
  • UK Research and Innovation
  • NIHR Biomedical Research Centre, Royal Marsden NHS Foundation Trust/Institute of Cancer Research
  • National Institutes of Health (K23DK125838)
  • Medical Research Council (AS-JF-17-011, MR/V027883/1)
Topics
  • Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 and COVID-19 Research
  • Social Sciences › Health › Vaccine Coverage and Hesitancy
  • Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
Keywords
COVID-19 vaccines, BNT162b2, ChAdOx1 nCoV-19 vaccine, vaccine side effects, vaccine effectiveness, community settings, self-reported symptoms, injection site reactions, systemic side effects, comorbidities, obesity, healthcare workers, lateral flow tests
MeSH terms
COVID-19, SARS-CoV-2, COVID-19 Vaccines, Female, United Kingdom, Humans, Male, Middle Aged, Prospective Studies, Safety, Vaccination, Self Report, Drug-Related Side Effects and Adverse Reactions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review