CoChat
Sign Up
PreprintCited by 1,739Education during COVID-19 pandemicSchool Choice and PerformanceCOVID-19 Pandemic Impacts

The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.

Learning Loss Due to School Closures During the COVID-19 Pandemic

Open access
Engzell, Per, Frey, Arun, Verhagen, Mark D.·2020·Proceedings of the National Academy of Sciences
DOI 10.31235/osf.io/ve4z7PMC PMC8092566
Open this paper in CoChat — free account

Abstract

Suspension of face-to-face instruction in schools during the COVID-19 pandemic has led to concerns about consequences for student learning. So far, data to study this question have been limited. Here we evaluate the effect of school closures on primary school performance using exceptionally rich data from the Netherlands (n≈350,000). We use the fact that national exams took place before and after lockdown, and compare progress during this period to the same period in the three previous years. The Netherlands underwent only a relatively short lockdown (8 weeks), and features an equitable system of school funding and the world's highest rate of broadband access. Still, our results reveal a learning loss of about 3 percentile points or 0.08 standard deviations. The effect is equivalent to a fifth of a school year, the same period that schools remained closed. Losses are up to 60% larger among students from less-educated homes, confirming worries about the uneven toll of the pandemic on children and families. Investigating mechanisms, we find that most of the effect reflects the cumulative impact of knowledge learned rather than transitory influences on the day of testing. Results remain robust when balancing on the estimated propensity of treatment and using maximum entropy weights, or with fixed-effects specifications that compare students within the same school and family. The findings imply that students made little or no progress whilst learning from home, and suggest losses even larger in countries with weaker infrastructure or longer school closures.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.31235/osf.io/ve4z7

Cites

91 of 91 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. Distribution of testing dates 2017 to 2020 and timeline of 2020 school closures. Density curves show the distribution of testing dates for national standardized assessments in 2020 and the three comparison years 2017 to 2019. Vertical lines show the beginning and end of nationwide school closures in 2020. Schools closed nationally on March 16 and reopened on May 11, after 8 wk of remote learning. Our difference-in-differences design compares learning progress between the two testing dates in 2020 to that in the 3 previous years.
Fig. 1. Distribution of testing dates 2017 to 2020 and timeline of 2020 school closures. Density curves show the distribution of testing dates for national standardized assessments in 2020 and the three comparison years 2017 to 2019. Vertical lines show the beginning and end of nationwide school closures in 2020. Schools closed nationally on March 16 and reopened on May 11, after 8 wk of remote learning. Our difference-in-differences design compares learning progress between the two testing dates in 2020 to that in the 3 previous years.CC BY · PubMed Central
Fig. 2. Difference in test scores 2017 to 2020. Density curves show the difference between students’ percentile placement between the first and the second test in each of the years 2017 to 2020. Note that this graph does not adjust for confounding due to trends, testing date, or sample composition, which we address in subsequent analyses using a variety of techniques.
Fig. 2. Difference in test scores 2017 to 2020. Density curves show the difference between students’ percentile placement between the first and the second test in each of the years 2017 to 2020. Note that this graph does not adjust for confounding due to trends, testing date, or sample composition, which we address in subsequent analyses using a variety of techniques.CC BY · PubMed Central
Fig. 3. Estimates of learning loss for the whole sample and by subgroup and test. The graph shows estimates of learning loss from a difference-in-differences specification that compares learning progress between the two testing dates in 2020 to that in the 3 previous years. Statistical controls include time elapsed between testing dates and a linear trend in year. Point estimates are with 95% confidence intervals, with robust standard errors accounting for clustering at the school level. One percentile point corresponds to ∼0.025 SD. Where not otherwise noted, effects refer to a composite score of math, spelling, and reading. Regression tables underlying these results can be found in SI Appendix, section 7.1.
Fig. 3. Estimates of learning loss for the whole sample and by subgroup and test. The graph shows estimates of learning loss from a difference-in-differences specification that compares learning progress between the two testing dates in 2020 to that in the 3 previous years. Statistical controls include time elapsed between testing dates and a linear trend in year. Point estimates are with 95% confidence intervals, with robust standard errors accounting for clustering at the school level. One percentile point corresponds to ∼0.025 SD. Where not otherwise noted, effects refer to a composite score of math, spelling, and reading. Regression tables underlying these results can be found in SI Appendix, section 7.1.CC BY · PubMed Central
Fig. 4. Robustness to specification. The graph shows estimates of learning loss for the whole sample and separately by parental education, using a variety of adjustments for loss to follow-up. Point estimates are with 95% confidence intervals, with robust standard errors accounting for clustering at the school level. For details, see Materials and Methods and SI Appendix, sections 4.2 and 7.4–7.8.
Fig. 4. Robustness to specification. The graph shows estimates of learning loss for the whole sample and separately by parental education, using a variety of adjustments for loss to follow-up. Point estimates are with 95% confidence intervals, with robust standard errors accounting for clustering at the school level. For details, see Materials and Methods and SI Appendix, sections 4.2 and 7.4–7.8.CC BY · PubMed Central
Fig. 5. Knowledge learned vs. transitory influences. The graph compares estimates for the composite achievement score in our main analysis (light color) with test not designed to assess curricular content (dark color). Both sets of estimates refer to our baseline specification reported in Fig. 3. Point estimates are with 95% confidence intervals, with robust standard errors accounting for clustering at the school level. For details, see Materials and Methods and SI Appendix, sections 3.1 and 7.8.
Fig. 5. Knowledge learned vs. transitory influences. The graph compares estimates for the composite achievement score in our main analysis (light color) with test not designed to assess curricular content (dark color). Both sets of estimates refer to our baseline specification reported in Fig. 3. Point estimates are with 95% confidence intervals, with robust standard errors accounting for clustering at the school level. For details, see Materials and Methods and SI Appendix, sections 3.1 and 7.8.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 82 references

Research works printed but not linked: 9

  1. Angrist, J. D., and Pischke, J., Mostly Harmless Econometrics: An empiricist's companion; Princeton University Press: 2008.
  2. Aigner, D. J. (1973). Regression with a binary independent variable subject to errors of observation. Journal of Econometrics 1, 49-59.
  3. Bollinger, C. R. (1996). Bounding mean regressions when a binary regressor is mismeasured. Journal of Econometrics 73, 387-399.
  4. Jerrim, J., and Micklewright, J. (2014). Socio-economic gradients in children's cognitive skills: Are cross-country comparisons robust to who reports family back- ground? European Sociological Review 30, 766-781.
  5. Black, D., Sanders, S., and Taylor, L. (2003). Measurement of higher education in the census and current population survey. Journal of the American Statistical Association 98, 545-554.
  6. Kapteyn, A., and Ypma, J. Y. (2007). Measurement error and misclassification: A comparison of survey and administrative data. Journal of Labor Economics 25, 513-551.
  7. Abowd, J. M., and Stinson, M. H. (2013). Estimating measurement error in an- nual job earnings: A comparison of survey and administrative data. Review of Economics and Statistics 95, 1451-1467.
  8. Buchmann, C., and DiPrete, T. A. (2006). The growing female advantage in college completion: The role of family background and academic achievement. American Sociological Review 71, 515-541.
  9. Autor, D., Figlio, D., Karbownik, K., Roth, J., and Wasserman, M. (2019). Fam- ily disadvantage and the gender gap in behavioral and educational outcomes. American Economic Journal: Applied Economics 11, 338-81.

Cited by

Sorted by citations

See all 1,739

Related

Metadata

Authors
Engzell, P., Frey, A., & Verhagen, M. D.
Affiliations
  • Stockholm University (SE)
  • University of Oxford (GB)
Corresponding author
  • Engzell, Per
  • Frey, Arun
  • Verhagen, Mark D.
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
Center for Open Science
Published
2020-10-29
Volume
118
Issue
17
ISSN
0027-8424 (ISSN-L)
DOI
10.31235/osf.io/ve4z7
PMC
PMC8092566
OpenAlex
W3143968054
Cited by
1,739
References
82
Citations by year
2026: 110 · 2025: 170 · 2024: 293 · 2023: 463 · 2022: 464 · 2021: 235 · 2020: 3
FWCI
639.03
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • SocArXiv (OSF Preprints) (submittedVersion, cc-by-nc-nd)
  • SocArXiv (OSF Preprints) (submittedVersion)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
Funders
  • Leverhulme Trust
  • University of Oxford
  • Vetenskapsrådet (2016-07099)
  • Forskningsrådet om Hälsa, Arbetsliv och Välfärd (2016-07099)
  • Organisation de Coopération et de Développement Économiques
  • Economic and Social Research Council
Topics
  • Social Sciences › Education › Education during COVID-19 pandemic
  • Social Sciences › Education › School Choice and Performance
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
Keywords
learning loss, school closures, COVID-19 pandemic, lockdown, national examinations, academic performance, home background, parental education, home-based learning, educational inequality
MeSH terms
COVID-19, SARS-CoV-2, Child, Female, Humans, Learning, Male, Netherlands, Quarantine, Schools, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review