Cited by 445Education during COVID-19 pandemicCOVID-19 Pandemic ImpactsE-Learning and COVID-19

Learning loss during Covid-19: An early systematic review

Open access
Donnelly, Robin, Patrinos, Harry Anthony·2022·PROSPECTS

Abstract

With Covid-19 having caused significant disruption to the global education system, researchers are beginning to become concerned with the impact that this has had on student learning progress and, in particular, whether learning loss has been experienced. To evaluate this, the authors conducted a thorough analysis of recorded learning loss evidence documented between March 2020 and March 2021. This systematic review aims to consolidate available data and to document what has been reported in the literature. Given the novelty of the subject, eight studies were identified; seven of these found evidence of student learning loss among at least some of the participants while one of the seven also found instances of learning gains in a particular subgroup. The remaining study found increased learning gains in their participants. Additionally, four of the studies observed increases in inequality where certain demographics of students experienced learning losses more significant than others. It is determined that further research is needed to increase the quantity of studies produced, their geographical focus, and the numbers of students they observe.

Integrity signals

Registry

Crossref · 10.1007/s11125-021-09582-6

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 27 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    Tomasik, Helbling, and Moser (2020) reported that primary-school students in Switzerland made more than twice as much progress during in-person learning as during the eight-week school closure.

    “Tomasik, Helbling, and Moser (2020) found learning progress of primary-school students in Switzerland during in-person learning to be more than twice as high compared to the progress made during the eight-week school closure.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tomasik, Helbling, & Moser (2020) source

      Not checked in the free run.

  • Findingcitation

    Engzell et al. (2021) found that learning losses were up to 60 percent larger among students from uneducated homes.

    “Engzell et al. (2021) determined that losses were up to 60 percent larger among students from uneducated homes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Engzell et al. (2021)

      Not checked in the free run.

  • Findingown data

    The review identified eight studies that met its inclusion criteria.

    “After this screening process, eight articles remained (Table 1).”
    • Also stated in: § Abstract · plain
      “Given the novelty of the subject, eight studies were identified”
    Quote in the paperSubstantial claim
  • Findingcitation

    Maldonado and De Witte (2020) reported that Grade 6 students in Belgium lost 0.29 SD in Dutch.

    “Maldonado and De Witte (2020) found Grade 6 students in Belgium experienced losses of 0.19 SD in math and 0.29 SD in Dutch.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Maldonado & De Witte (2020)

      Not checked in the free run.

  • Findingown data

    The eighth included study reported increased learning gains among its participants.

    “The remaining study found increased learning gains in their participants.”
    • Also stated in: § Conclusion · plain
      “The remaining study observed learning gains among university students.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 11
  • The quote only partly states this claim. 2
  • May combine two claims. 6
  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingcitation

    Maldonado and De Witte (2020) reported insignificant impacts in social science for Grade 6 students in Belgium.

    “both Maldonado and De Witte (2020) as well as Kuhfeld et al. (2020) found learning losses in certain subjects but insignificant impacts in others.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Maldonado & De Witte (2020)

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Four of the eight included studies reported increased inequality, with certain demographic groups experiencing larger learning losses than others.

    “Additionally, four of the studies observed increases in inequality where certain demographics of students experienced learning losses more significant than others.”
    • Also stated in: § Discussion · plain
      “Of the eight studies, four found instances of inequality”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingcitation

    Gore et al. (2021) found both instances of increased inequality and instances of no change.

    “Gore et al. (2021) found instances of increased inequality as well as instances of no change.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gore et al. (2021) source

      Not checked in the free run.

  • The quote only partly states this claim.

    Findingcitation

    Kuhfeld et al. (2020) found insignificant impacts in reading for Grade 3-8 students in the USA.

    “both Maldonado and De Witte (2020) as well as Kuhfeld et al. (2020) found learning losses in certain subjects but insignificant impacts in others.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kuhfeld et al. (2020)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The majority of articles identified in the search pertained to hypothesized or predicted learning loss.

    “When conducting the search analysis, thousands of articles were identified; however, the majority of these pertained to hypothesized or predicted learning loss.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    The literature on Covid-19's impact on student learning progress is limited in the number of studies, geographic regions analyzed, and participating students.

    “In general, the literature representing the impact that Covid-19 has had on student learning progress is limited in the quantity of studies available, geographical regions analyzed, and number of participating students.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Currie and Thomas (2001) observed that a 0.20 SD decrease in standardized test scores could decrease future employment probability by 0.86 percent.

    “Currie and Thomas (2001) put this into perspective because they observe that a 0.20 standard deviation (SD) decrease in standardized test scores could decrease future employment probability by 0.86 percent.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Currie and Thomas (2001)

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The search used English-language articles published between March 1, 2020, and March 18, 2021, across ECONLIT, Google Scholar, PubMed, ERIC, and Cochrane Library.

    “The initial search was performed using English-language articles published between March 1, 2020, and March 18, 2021.”
    • Also stated in: § Methodology · plain
      “To ensure a comprehensive, multidisciplinary search, the electronic databases included were ECONLIT, Google Scholar, PubMed, Education Resources Information Center, and Cochrane Library.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Psacharopoulos and Patrinos (2018) found that another year of schooling is associated with an 8-9 percent gain in future earnings.

    “Likewise, another year of schooling is, on average, associated with an 8–9 percent gain in future earnings (Psacharopoulos & Patrinos, 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Psacharopoulos & Patrinos (2018) source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The search combined keywords for Covid-19 with keywords for learning loss, learning slide, education gap, or achievement gap.

    “To conduct the search, the keywords “Covid-19”, “coronavirus”, “2019-ncov”, “sars-cov-2”, or “cov-19”, were used in combination with “learning loss”, “learning slide”, “education gap”, or “achievement gap”.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    To the authors' knowledge, this is the first systematic review of its kind.

    “To the best of our knowledge, this is the first systematic review of its kind.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Article abstracts were screened to narrow the results to studies with recorded outcomes.

    “To narrow this down to studies with recorded results, article abstracts were then screened.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Studies were included if they conducted student analyses and reported impacts on learning progress, whether positive, negative, or insignificant, due to Covid-19 school disruptions.

    “Studies that conducted student analyses and reported impacts on learning progress (either positive, negative, or insignificant) as a result of Covid-19 school disruptions were included.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chetty et al. (2014) observed that increasing student achievement by 0.20 SD results on average in a 2.6 percent increase in annual lifetime earnings.

    “Additionally, Chetty et al. (2014) observe that increasing student achievement by 0.20 SD results on average in a 2.6 percent increase in annual lifetime earnings (Maldonado & De Witte, 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chetty et al. (2014) source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Studies were excluded if they lacked student analyses or recorded impacts, occurred before Covid-19, or reported hypothesized results.

    “Factors for rejecting studies from our review included the absence of student analyses or recorded impacts on learning progress, analyses occurring before the onset of Covid-19, or hypothesized results.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    Although robust empirical research on Covid-19-related learning loss is limited, learning loss itself may not be limited.

    “This signals that although robust and empirical research on Covid-19-related student learning loss is limited, learning loss itself may not be.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The learning losses reported in the included studies are attributable to Covid-19 school disruptions rather than other factors.
    • Rests on an assumption: The findings of the included studies are valid and reliable enough to support the review's conclusions.
    • Rests on an assumption: The included studies' measures of learning loss are comparable enough to be synthesized.
16 of 16
  • The paper relies on this without support.

    The learning losses reported in the included studies are attributable to Covid-19 school disruptions rather than other factors.

    “Covid-19-related student learning loss”
  • The paper relies on this without support.

    The findings of the included studies are valid and reliable enough to support the review's conclusions.

    “the early findings of seven studies provide evidence of experienced learning losses”
  • The paper relies on this without support.

    The included studies' measures of learning loss are comparable enough to be synthesized.

    “These observed losses are occurring across a range of subjects, grade levels, and geographical regions.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Systematic reviews and meta-analyses suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation, at least one-third of a year's worth of learning.

    “Nevertheless, there are several systematic reviews and meta-analyses, and these suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation—that is, at least one-third of a year’s worth of learning16–20.”

    From COVID-19, school closures, and student learning outcomes. New global evidence from PISA (2025) · cites this paper as [17]

    The Analytical review (page unavailable) reports learning losses in seven of eight studies, supporting the general claim that COVID-19 school closures were associated with learning loss. It gives results for individual studies, but does not report an overall average of 0.17 standard deviations or equate that figure to at least one-third of a year’s learning. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • The study's results are similar to what systematic reviews are finding.

    “Interestingly, our results are similar to what systematic reviews are finding learning16–20,44,45.”
  • There is already a relatively large consensus in the literature that student learning suffered a setback due to Covid-19.

    “While in the literature there is already a relatively large consensus that student learning suffered a setback due to Covid-19, as pointed out by several researchers (e.g., Donnelly & Patrinos, 2022; Patrinos et al., 2022)”

    From The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis (2023) · cites this paper as Donnelly & Patrinos, 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Germany is omitted from the list of countries studied

    Minor

    The discussion says the available studies cover only six named countries, but Table 3 also includes a study from Germany.

1 of 1

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Donnelly, R., & Patrinos, H. A.
Affiliations
  • University of Ottawa (CA)
  • World Bank (US)
Corresponding author
Patrinos, Harry Anthony
Venue
PROSPECTS
Published
2021-11-10
Volume
51
Issue
4
Pages
601-609
ISSN
  • 0033-1538 (ISSN-L)
  • 1573-9090
OpenAlex
W3160694446
Cited by
445
References
16
Citations by year
2026: 44 · 2025: 69 · 2024: 107 · 2023: 135 · 2022: 80 · 2021: 9 · 2020: 1
FWCI
20.40
Type
journal article
Language
en
Open access
Locations
  • Prospects (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Education › Education during COVID-19 pandemic
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
  • Social Sciences › Education › E-Learning and COVID-19
Keywords
learning loss, COVID-19 pandemic, student learning progress, educational disruption, learning gains, educational inequality, student demographics, school closures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review