CoChat
Sign Up
Cited by 173Education during COVID-19 pandemicCOVID-19 Pandemic ImpactsSchool Choice and Performance

The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis

Open access
Di Pietro, Giorgio·2023·Educational Research Review
DOI 10.1016/j.edurev.2023.100530PMC PMC10028259
Open this paper in CoChat — free account

Abstract

This work attempts to synthetize existing research about the impact of Covid-19 school closure on student achievement. It extends previous systematic reviews and meta-analyses by (a) using a more balanced sample in terms of country composition, (b) considering new moderators (type of data and research design), and (c) including studies on tertiary education students in addition to primary and secondary education students. Our meta-analysis findings show that the pandemic had, on average, a detrimental effect on learning. The magnitude of this learning deficit (about 0.19 standard deviations of student achievement) appears to be roughly comparable to that suffered by students who have experienced a significant disruption in their schooling due to a major natural disaster (e.g., Hurricane Katrina). Students are also found to have lost more ground in math/science than in other subjects. Additionally, one year or more after the first lockdown, students seem to have been unable to catch up on unfinished learning from the pandemic. This result suggests that more efforts should be made to ensure students recover their missed learning in order to avoid negative long-term consequences for them and society.

Verified 2026-10-07 · 1 reference not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

This meta-analysis of 239 estimates from 39 studies across 19 countries finds that Covid-19 school closures reduced student achievement by about 0.19 standard deviations on average. Learning deficits were larger in math/science and showed no statistically significant recovery by 2021.

Key findings

  • The random-effects robust variance estimation mean effect size was −0.186 SD (SE = 0.0646, p = 0.0065, 95% CI [−0.316, −0.055]), described as roughly comparable to losses after major natural disasters and equivalent to just over half a school year.
  • Math/science achievement was 0.17 SD lower than humanities/subject mix, other things equal.
  • There was no statistically significant difference between assessments in 2020 and 2021, suggesting students had not caught up more than one year after the first lockdown.
  • Learning deficits were smaller in Europe than non-Europe, and studies using quasi-experimental/experimental designs or cross-sectional data tended to find larger deficits than descriptive/correlational or longitudinal studies.

Methods

  • PRISMA-guided meta-analysis; searched six electronic databases through 15 July 2022, yielding 6,075 hits and a final sample of 39 studies, 239 effect sizes, and 19 countries.
  • Effect sizes were converted to Cohen’s d; the main model used random-effects meta-analysis with robust variance estimation to account for multiple dependent estimates per study, plus weighted least squares meta-regression with general-to-specific selection and Bayesian Model Averaging.
  • Moderators included subject, education level, timing of assessment, geographical area, data type, research design, publication year/type, and standard error; publication bias was assessed with a Doi plot/LFK index and Egger’s test.

Limitations

  • Stated in the paper: the analysis was not conducted at grade level, so it cannot rule out disproportionate effects on very young pupils (e.g., grade 1) or heterogeneity within primary school children.
  • Stated in the paper: ROBINS-I assessment found confounding, participant selection, and missing data were common sources of potential bias; vulnerable students were under-represented in some Covid-19 samples, likely underestimating learning delays, while the direction of bias was unclear in convenience samples.
  • Evident from the text: the evidence base is limited to 39 studies and 19 countries, the search ended in July 2022, and tertiary education is represented by only 10 effect sizes; publication bias tests found no asymmetry (LFK index = 0; Egger’s test p = 0.380).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.edurev.2023.100530

Cites

112 of 113 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 38 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The meta-analysis findings show that the pandemic had, on average, a detrimental effect on learning.

    “Our meta-analysis findings show that the pandemic had, on average, a detrimental effect on learning.”
    Quote in the paperSubstantial claim
  • Findingown data

    The magnitude of the learning deficit is about 0.19 standard deviations of student achievement.

    “The magnitude of this learning deficit (about 0.19 standard deviations of student achievement)”
    Quote in the paperSubstantial claim
  • Findingown data

    The study covers 19 countries.

    “First, compared to previous meta-analyses, this study covers a larger number of countries (i.e., 19).”
    • Also stated in: § Sample characteristics · plain
      “The studies cover a total of 19 countries.”
    Quote in the paperSubstantial claim
  • Findingown data

    The mean effect size of the pandemic on student achievement is approximately −0.19 standard deviations (RVE RE mean = −0.18613, SE = 0.0646, p = 0.0065, 95% CI [-0.316, −0.055]).

    “The RVE RE mean effect size turns out to be −0.18613 (SE = 0.0646, p-value = 0.0065, 95% CI [-0.316, −0.055]).”
    • Also stated in: § Discussion of the main results · plain
      “we find that, on average, the pandemic depressed student achievement by around 0.19 standard deviations.”
    Quote in the paperSubstantial claim
  • Findingown data

    Students lost more ground in math/science than in other subjects.

    “Students are also found to have lost more ground in math/science than in other subjects.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 24
  • May combine two claims. 16
  • The quote only partly states this claim. 2

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The largest source countries are the US (71 estimates), Germany (39 estimates) and Belgium (33 estimates).

    “The largest source countries are the US (71 estimates), Germany (39 estimates) and Belgium (33 estimates).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Each study included in the dataset contains a number of estimates that vary from 1 to 32.

    “Each study included in the dataset contains a number of estimates that vary from 1 to 32.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The meta-analysis uses 239 separate estimates from 39 studies.

    “A total of 239 separate estimates from 39 studies are considered.”
    • Also stated in: § Sample characteristics · plain
      “The dataset used for the meta-analysis includes a total of 239 different impact estimates extracted from 39 separate studies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In this paper, approximately 33.9% of the effect sizes are derived from US and UK studies, compared to about 71.1% in Betthäuser et al. (2023).

    “For instance, while in Betthäuser et al. (2023) about 71.1% of the effect sizes are derived from these studies, in our paper the corresponding figure is approximately 33.9%.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Betthäuser et al. (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Most studies (79%) reported multiple estimates.

    “Several reasons explain why most studies (i.e., 79%) reported multiple estimates.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The I2 statistic is 100%, indicating that all variability in the effect-size estimates is due to heterogeneity rather than sampling error.

    “The appropriateness of the RE model is confirmed as I2 has a value of 100%.14 This suggests that all the variability in the effect-size estimates is due to heterogeneity as opposed to sampling error.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    The between-study variance τ2 is 1.74, corroborating substantial heterogeneity of the effect sizes.

    “Its large value of 1.74 further corroborates the hypothesis of substantial heterogeneity of the effect sizes (Takase & Yoshida, 2021).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Takase & Yoshida, 2021

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Students experienced larger learning deficits in math/science compared to humanities/subject mix; achievement in math/science is on average 0.17 standard deviations smaller.

    “students experienced larger learning deficits in math/science. More precisely, other things being equal, student achievement in math/science is on average found to be 0.17 standard deviations smaller than in humanities/subject mix.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In 2021, students were not able to recover from the learning deficits caused by Covid-19 school closures in 2020; there is no statistically significant difference between assessments in 2020 and 2021.

    “our results indicate that in 2021 students were not able to recover from the learning deficits caused by Covid-19 school closures in 2020. There is no statistically significant difference in student performance between assessments that have taken place several months or more than one year after the outbreak of the coronavirus and those that have occurred in the early stages of the pandemic.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingnone given

    Studies employing cross-sectional data tend to suggest greater learning deficits than longitudinal studies.

    “studies employing cross-sectional data as well as those focusing on non-European countries tend to suggest greater learning deficits.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The meta-analysis was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA).

    “To perform this meta-analysis, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (Moher et al., 2009).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moher et al., 2009 source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    The relevance of type of data and research design in explaining heterogeneity of results across studies is well-known in the meta-analysis literature.

    “The relevance of these factors in explaining the heterogeneity of results across studies is well-known in the meta-analysis literature.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Havránek et al., 2020

      Not checked in the free run.

    • cites Stanley and Jarrell, 1989

      Not checked in the free run.

    • cites Judd, 2015

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Inter-rater agreement for study selection was 97%, and disagreements were discussed until consensus was reached.

    “While the inter-rater agreement was very high (i.e., 97%), studies on which there was disagreement were discussed in depth until consensus was reached.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The next stages of the study selection process were carried out by the author and another researcher who independently classified studies as relevant or irrelevant based on predefined inclusion criteria.

    “While all the titles and abstracts were screened only by the author, the next stages of the study selection process were carried out by the author and by another researcher who independently classified the studies as relevant and irrelevant based on the predefined inclusion criteria.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Several methods are used to account for within-study dependence.

    “As discussed later in the paper, several methods are used to account for within-study dependence.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Study coding was performed following the same procedure used for the final stages of the study selection process.

    “Study coding was performed following the same procedure used for the final stages of the study selection process.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    In most countries, educational institutions had to close for several weeks or months in an attempt to reduce the spread of the virus.

    “In most countries, educational institutions had to close for several weeks or months in an attempt to reduce the spread of the virus (UNESCO, 2020a).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites UNESCO, 2020a

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The final grade awarded to students successfully completing university is an important predictor of their labour market prospects.

    “Several studies from various countries (e.g., Bratti et al., 2004; Dabalen et al., 2001; Koda & Yuki, 2013) show that the final grade awarded to students successfully completing university is an important predictor of their labour market prospects.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Bratti et al., 2004

      Not checked in the free run.

    • cites Dabalen et al., 2001

      Not checked in the free run.

    • cites Koda & Yuki, 2013

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Inter-rater agreement for study coding was 93%.

    “The inter-rater agreement was again high (i.e., 93%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Non-journal articles are more likely to contain typos in their regression tables.

    “Additionally, non-journal articles are more likely to contain typos in their regression tables (Cazachevici et al., 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Cazachevici et al., 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisereasoning

    Articles published in journals are expected to be of higher scientific rigour because they are more likely to have gone through a review process.

    “Articles published in journals are expected to be of higher scientific rigour since they are more likely to have gone through a review process.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    There is a risk that the disruption in learning caused by Covid-19 may persist over time, having long-term consequences on students' knowledge and skills as well as on their labour market prospects.

    “As argued in several papers (e.g., Hanushek & Woessmann, 2020; Psacharopoulos et al., 2021), there is the risk that the disruption in learning caused by Covid-19 may persist over time, having long-term consequences on students’ knowledge and skills as well as on their labour market prospects.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Hanushek & Woessmann, 2020

      Not checked in the free run.

    • cites Psacharopoulos et al., 2021 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    During school closures, less advantaged students are less likely to have had access to a computer, an internet connection, and a space conducive to learning.

    “During school closures, these students are less likely to have had access to a computer, an internet connection, and a space conducive to learning (Blaskó et al., 2022; Di Pietro et al., 2020).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Blaskó et al., 2022

      Not checked in the free run.

    • cites Di Pietro et al., 2020

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The BMA methodology identifies Math/Science, Europe, and Journal article as relevant variables, and these are also included in the specific model.

    “The results, which are reported in Table 5, show that all the variables that are consistently identified by the BMA methodology as relevant (i.e., Math/Science, Europe and Journal article) are also included in the specification whose estimates are reported in Columns (2), (3) and (4) of Table 4.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    High-income countries are likely to have experienced smaller learning deficits due to higher technological capability and lower share of households below the poverty line.

    “As suggested by Donnelly and Patrinos (2020), high-income countries are likely to have experienced smaller learning deficits as a result of Covid-19 because of their higher technological capability and the lower share of households living below the poverty line.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Donnelly and Patrinos (2020)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationown data

    The analysis was not conducted at grade level, so one cannot rule out that the pandemic disproportionately affected very young pupils (e.g., grade 1).

    “given that our analysis was not conducted at grade level, one cannot rule out the possibility that the pandemic has disproportionately affected the achievement of very young pupils (e.g., grade 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Children of less educated parents are expected to have received less parental support while learning at home than children of more educated parents.

    “one would expect children of less educated parents to have received less parental support while learning at home than children of more educated parents.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ariyo et al. (2022)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Promoting the development of metacognition skills is a powerful way to enhance student learning and performance.

    “Promoting the development of metacognition skills is, for instance, a powerful way to enhance student learning and performance (Stanton et al., 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stanton et al., 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The underestimation of pandemic-related learning delays may have important policy implications.

    “The underestimation of pandemic-related learning delays may have important policy implications as it could result in under-provision of remedial support to students who are falling behind due to Covid-19.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The risk of bias in nonrandomized studies was assessed using the ROBINS-I tool.

    “the risk of bias in nonrandomized studies was assessed in 3811 studies using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al., 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sterne et al., 2016

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The Doi plot shows no asymmetry (LFK index = 0), indicating that no publication bias is detected.

    “As shown in Fig. 2, the Doi plot shows no asymmetry (LFK index = 0), indicating that no publication bias is detected.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Less attention has been paid to a number of time- and cost-effective pedagogical practices.

    “less attention has been paid to a number of time- and cost-effective pedagogical practices (Carrasco et al., 2021)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Carrasco et al., 2021

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    European higher education institutions were better prepared to respond to the pandemic than counterparts in other parts of the world, as shown by a survey (higher percentage replaced classroom teaching with distance teaching in Europe).

    “there is evidence, but only available at higher education level, that European educational institutions were better prepared to respond to the challenges posed by the pandemic than their counterparts in other parts of the world. A survey carried out by the International Association of Universities immediately after the outbreak of the coronavirus shows that the percentage of higher education”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Marinoni et al., 2020

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The robustness test using inverse variance weighting yields coefficients broadly in line with the main model, except for longitudinal data.

    “As a robustness test, the model depicted in Column (3) of Table 4 is re-estimated but this time each effect size is weighted by its inverse variance. As shown in Column (4) of Table 4, with the exception of the estimate on longitudinal data, the sign and the magnitude of the other coefficients are broadly in line with those depicted in Column (3).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Non-causal methods fail to account for trends in student progress prior to the outbreak, and by assuming a counterfactual where achievement has stayed flat, they generate estimates of learning deficits that are biased downwards.

    “As pointed out by Engzell et al. (2021), non-causal methods fail to account for trends in student progress prior to the outbreak of Covid-19 and, hence, by assuming a counterfactual where achievement has stayed flat, they generate estimates of learning deficits that are biased downwards.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Engzell et al. (2021) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Each study was independently evaluated by the author and another researcher, and disagreements were resolved through discussion to reach consensus.

    “Each study was independently evaluated by the author and another researcher, and any disagreements were resolved through discussion to reach a consensus.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Interpretationreasoning

    The finding of a mean effect of −0.19 SD is more generalizable than previous meta-analyses because the sample is more balanced in terms of country composition.

    “This would suggest that our finding is more generalizable than that of previous studies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The sample used in this meta-analysis is more balanced in terms of country composition than previous meta-analyses.

  • May combine two claims.

    Interpretationreasoning

    In several studies, vulnerable students tend to be under-represented in the Covid-19 sample, which may lead to an underestimation of pandemic-related learning delays.

    “In several studies, vulnerable students, who have been among the most hardly hit by the pandemic, tend to be under-represented in the Covid-19 sample. This may lead to an underestimation of the pandemic-related learning delays.”
    Quote in the paperSubstantial claim
38 of 38

  • The paper relies on this without support.

    The European countries included in the study have, on average, a higher GDP per capita than most non-European countries included (except US and Australia).

    “the European countries considered in this study have, on average, a higher gross domestic product per capita than most of the non-European countries included in the analysis (this is not true for the US and Australia).”

  • The paper relies on this without support.

    The sample used in this meta-analysis is more balanced in terms of country composition than previous meta-analyses.

    “it should be taken into account that we use a more balanced sample in terms of country composition.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

9 of the 122 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. Borgonovi, 2022, A longitudinal perspective of the effects of Covid-19 on students' resilience, The effect of the pandemic on the reading and mathematics achievement of 5th and 8th graders in Italy
  2. Burgess, 2020, How we should deal with the lockdown learning loss in England's schools, VOX CEPR Policy Portal
  3. Marinoni, 2020, The impact of Covid-19 on higher education around the world, IAU Global Survey Report

Cited by

Sorted by citations

Related

Authors
Di Pietro, G.
ORCID
Di Pietro, Giorgio
Venue
Educational Research Review
Publisher
Elsevier BV
Published
2023-05
Volume
39
Pages
100530
ISSN
1747-938X (ISSN-L)
DOI
10.1016/j.edurev.2023.100530
PMC
PMC10028259
OpenAlex
W4328053575
Semantic Scholar
19437454e7960020d7e2f1fb49ca5ca3c49e72d9
Cited by
173
References
122
Citations by year
2026: 49 · 2025: 59 · 2024: 42 · 2023: 22 · 2022: 1
FWCI
127.24
Type
journal article
Language
en
Open access
Licence
Locations
  • Educational Research Review (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
European Commission Joint Research Centre
Topics
  • Social Sciences › Education › Education during COVID-19 pandemic
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
  • Social Sciences › Education › School Choice and Performance
Keywords
COVID-19 pandemic, school closures, academic performance, learning loss, learning recovery, tertiary students, K-12 education, math and science achievement, Hurricane Katrina, lockdown duration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review