The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index; 14 of the 59 sources its claims cite were checked for support.
Learning loss due to school closures during the COVID-19 pandemic
Open accessAbstract
Significance: School closures have been a common tool in the battle against COVID-19. Yet, their costs and benefits remain insufficiently known. We use a natural experiment that occurred as national examinations in The Netherlands took place before and after lockdown to evaluate the impact of school closures on students’ learning. The Netherlands is interesting as a “best-case” scenario, with a short lockdown, equitable school funding, and world-leading rates of broadband access. Despite favorable conditions, we find that students made little or no progress while learning from home. Learning loss was most pronounced among students from disadvantaged homes.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Using Dutch national assessments taken before and after an 8-week COVID-19 school closure, this study finds that primary students made little or no progress while learning from home. The Netherlands is a “best-case” setting, yet average learning loss was 3.16 percentile points (0.08 SD), with larger losses among students from disadvantaged homes.
Key findings
- In the baseline difference-in-differences estimate for a composite of math, spelling, and reading, students lost on average 3.16 percentile points, equivalent to 0.08 SD; raw differences ranged from −0.76 percentiles in spelling to −2.15 percentiles in math.
- Losses were concentrated among students from less-educated homes: the two lowest parental-education categories, together 8% of the population, suffered losses 40% larger than average (high, −3.07; low, −4.34; lowest, −4.25); within-family estimates showed a gap of 60% (high, −3.25; low, −4.67; lowest, −5.20).
- There was little evidence that effects differed by sex, school grade, subject, or prior performance, but school-level variation was considerable, with some schools seeing losses of 10 percentile points or more and others no losses or small gains.
- On a learning-readiness test not tied to curricular content, effects shrank by nearly two-thirds (main effect −1.19 vs. −3.16), suggesting most of the drop reflects knowledge learned; 3.16 percentiles translates to 7.9 to 10.5 weeks of lost learning depending on the benchmark.
Methods
- Natural experiment/difference-in-differences design using biannual national standardized tests in The Netherlands from 2017 to 2020, covering 15% of Dutch primary schools (n ≈ 350,000) and students aged 8 to 11 years (grades 4 to 7); 2020 tests occurred just before and after 8 weeks of nationwide closures starting March 16.
- Main outcomes were math, spelling, reading, and a composite score, plus an oral reading fluency/learning-readiness test; scores were transformed to percentiles, with baseline models adjusting for a linear year trend and days between tests and clustering standard errors at the school level.
- Robustness included placebo analyses, year exclusions, loss-to-follow-up adjustments via regression, propensity-score and maximum-entropy weighting, school and family fixed effects, and a specification curve analysis of more than 2,000 alternative models.
Limitations
- The paper states that only time will tell whether students rebound, remain stable, or fall farther behind; test scores do not capture psychosocial development or societal and parental costs; and results may underestimate the full costs of school closures.
- The paper notes that The Netherlands is a best-case scenario due to short closures, high technological preparedness, and equitable school funding, making the estimates a likely lower bound; the short closure also gave little time to adapt, and remote learning might improve with time.
- The text makes evident that the sample covers 15% of primary schools, loss to follow-up is addressed but only a subset of students took tests after lockdown, and the within-family fixed-effects specification reduces the sample by approximately 60% and shifts it toward larger families.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
23 Own result · 16 Own method · 12 Own interpretation · 10 No full text of the sources · 9 Sources not found · 4 Partly supported · 4 Not checked yet · 3 No basis given · 2 Supported so far · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In their projections of learning loss due to the pandemic, the World Bank assumes a yearly progress of 0.40 SD (46).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Partly supportedcites [46] J. P. Azevedo, A. Hasan, D. Goldemberg, S. A. Iqbal, K. Geven (2020) Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. World Bank Policy Research Working Paper (9284). https://openknowledge.worldbank.org/handle/10986/33945. Accessed 9 February 2021. sourceExperimental — accuracy not yet measured
The Empirical methodology (p. 15) says the projections assume learning gains of 0.2–0.5 SD from one grade to the next, depending on country income. Annex A.1 (p. 44) specifies 40 PISA points, equivalent to 0.40 SD, as the baseline for upper-middle-income countries—not as a universal yearly rate. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our results show that learning loss was particularly pronounced for students from disadvantaged homes, confirming the fears held by many that school closures would cause socioeconomic gaps to widen (51 –55).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Supportedcites [51] E. A. Hanushek, L. Woessmann, The Economic Impacts of Learning Losses (Organisation for Economic Co-operation and Development OECD, 2020). source
The Summary of closure experiences says that school closures have widened gaps in skill development and warns that they may increase inequality. Section 4, “Distributional issues,” says the negative impact was greater for students from disadvantaged households. The citing authors’ claim about what their own results show is not attributed to this source. (LLM, conf=0.92)
- Supportedcites [52] F. Agostinelli, M. Doepke, G. Sorrenti, F. Zilibotti, When the Great Equalizer Shuts Down: Schools, Peers, and Parents in Pandemic Times (National Bureau of Economic Research, 2020). PMC8735857 35017763 source
The Conclusions state that the paper’s analysis finds higher educational inequality and that learning gaps grow during the pandemic. This supports the cited concern that school closures would widen socioeconomic gaps; the citing authors’ own learning-loss results are not attributed to this source. (LLM, conf=0.93)
- Partly supportedcites [53] A. Bacher-Hicks, J. Goodman, C. Mulhern, Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time. J. Publ. Econ. 193, 104345 (2020). PMC8486492 34629567 sourceExperimental — accuracy not yet measured
The Introduction reports that school closures widened socioeconomic gaps in searches for online learning resources. The Discussion says this pattern suggests academic gaps will be wider in future school years, but the paper does not establish that closures had already widened learning-loss gaps; its measured outcome is primarily search engagement. (LLM, conf=0.86)
- Supportedcites [54] Z. Parolin, E. K. Lee, Large socio-economic, geographic and demographic disparities exist in exposure to school closures. Nat. Hum. Beh., 10.1038/s41562-021-01087-8. (2021). PMC8060162 33737734 source
The Discussion says that disadvantaged students’ greater exposure to school closures and distance learning may exacerbate socio-economic gaps in learning outcomes. This supports the cited concern that closures could widen those gaps; the citing authors attribute the observed learning loss to their own results, not to this source. (LLM, conf=0.90)
- Supportedcites [55] D. H. Bailey, G. J. Duncan, R. J. Murnane, N. A. Yeung (2021) Achievement gaps in the wake of COVID-19 (EdWorkingPaper No. 21-346, Annenberg Institute at Brown University). https://www.edworkingpapers.com/ai21-346. Accessed 9 February 2021. source
The Discussion reports researchers’ prediction that COVID-19 would markedly increase achievement gaps between children from low- and higher-income families. The Abstract identifies these as forecasts, supporting the cited ‘fears’; the claim that results confirm them refers to the citing authors’ own results, not to this source. (LLM, conf=0.90)
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Nevertheless, the magnitude of our findings appears to validate scenarios projected by bodies such as the European Commission (34) and the World Bank (46).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Not matchedcites [34] G. Di Pietro, F. Biagi, P. Costa, Z. Karpinski, J. Mazza, The Likely Impact of COVID-19 on Education: Reflections Based on the Existing Literature and Recent International Datasets (Publications Office of the European Union, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [46] J. P. Azevedo, A. Hasan, D. Goldemberg, S. A. Iqbal, K. Geven (2020) Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. World Bank Policy Research Working Paper (9284). https://openknowledge.worldbank.org/handle/10986/33945. Accessed 9 February 2021. source
The citation to this paper is attached to the World Bank scenarios, not to the European Commission or the citing authors’ judgment that their findings validate them. The Conclusion (p. 35) presents optimistic, intermediate, and pessimistic global scenarios for the effects of COVID-19 school closures on schooling and learning. (LLM, conf=0.91)
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Moreover, we match treatment and control groups on a wider range of individual- and school-level characteristics using reweighting on the propensity of treatment (68) and maximum-entropy balancing (69).”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > Propensity Score and Entropy Weighting.
What the sources say
- Supportedcites [68] G. W. Imbens, J. M. Wooldridge, Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47, 5– 86 (2009). source
Section 5.4 explains that adjusting for the propensity score accounts for differences in covariates between treated and control units. Section 5.6 describes weighting observations by their propensity of treatment or non-treatment. Citation (68) supports the propensity-reweighting method; the sentence cites a different source for maximum-entropy balancing. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [69] J. Hainmueller, Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Polit. Anal., 20, 25– 46 (2012). source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The largest gap in effect sizes between educational backgrounds is found in our within-family analysis, estimated at 60% (parental education: high, −3.25; low, −4.67; lowest, −5.20).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Those in the two lowest categories of parental education—together accounting for 8% of the population ( SI Appendix, section 5.1)—suffered losses 40% larger than the average student (estimates by parental education: high, −3.07; low, −4.34; lowest, −4.25).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Baseline Specification.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Disregarding the trend and instead assuming a counterfactual where achievement had stayed flat between 2019 and 2020, the estimated treatment effect shrinks by 21% to −2.51 percentiles ( SI Appendix, section 7.11).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Specification Curve Analysis.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, we find little evidence that the effect differs by sex, school grade, subject, or prior performance.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Baseline Specification.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The placebo estimate closest to zero is found in the version of our preferred specification that includes family fixed effects.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Specification Curve Analysis.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, adjusting for these covariates makes little difference to our results ( SI Appendix, section 7.1).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 19
- It may state two separate claims as one. 30
- The quote only partly states this claim. 1
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Dynamic models of learning stress how small losses can accumulate into large disadvantages with time (56 –58).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Not checked — no full text of the sourcecites [56] T. A. DiPrete, G. M. Eirich, Cumulative advantage as a mechanism for inequality: A review of theoretical and empirical developments. Annu. Rev. Sociol. 32, 271– 297 (2006). source
- Supportedcites [57] M. Kaffenberger, Modelling the long-run learning impact of the Covid-19 learning shock: Actions to (more than) mitigate loss. Int. J. Educ. Dev. 81, 102326 (2021). 33716394 PMC7923185 source
The Introduction reports that the model projects a one-third reduction in grade 3 learning to become a full-year deficit by grade 10, and a one-half reduction to become a 1.5-year deficit. These projections support the claim that an initial learning loss can lead to a larger disadvantage over time. (LLM, conf=0.92)
- Supportedcites [58] N. Fuchs-Schündeln, D. Krueger, A. Ludwig, I. Popova, The Long-Term Distributional and Welfare Effects of COVID-19 School Closures (National Bureau of Economic Research, 2020). source
The Introduction (page unavailable) explains that an initial shortfall in children’s human capital reduces the productivity of later investments, leading to less investment and worse educational and earnings outcomes. Section 4.1.1 (page unavailable) likewise says that reduced accumulation at younger ages translates into lower human capital and investment in the future. (LLM, conf=0.90)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Test scores do not consider children’s psychosocial development (63, 64), either societal costs due to productivity decline or heightened pressure among parents (65, 66).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Partly supportedcites [63] A. Gassman-Pines, C. M. Gibson-Davis, E. O. Ananat, How economic downturns affect children’s development: An interdisciplinary perspective on pathways of influence. Child Dev. Perspect. 9, 233– 238 (2015). 31327980 PMC6641565 sourceExperimental — accuracy not yet measured
The section on economic and psychological effects on continuously employed individuals reports increased financial and psychological stress within families and greater parental distress during downturns, supporting the point about pressure on parents. The paper also discusses children’s emotional well-being separately from test scores, but it does not say that test scores omit societal costs from declining productivity or explicitly assess what test scores capture. No page number was available in our copy. (LLM, conf=0.82)
- Could not verifycites [64] L. Shanahan, Emotional distress in young adults during the COVID-19 pandemic: Evidence of risk and resilience from a longitudinal cohort study. Psychol. Med., 10.1017/S003329172000241X (2020). PMC7338432 32571438 sourceExperimental — accuracy not yet measured
The Methods (Measures; page unavailable) assess economic and lifestyle disruptions among young adults, and the Discussion (page unavailable) links those disruptions to emotional distress. Neither passage addresses what test scores omit, societal costs from declining productivity, or heightened pressure on parents. Searches for those specific points found no supporting passage in our copy. (LLM, conf=0.73)
- Partly supportedcites [65] C. Collins, L. C. Landivar, L. Ruppanner, W. J. Scarborough, COVID-19 and the gender gap in work hours. Gender Work. Organ. 28, 101– 12. (2020). PMC7361447 32837019 sourceExperimental — accuracy not yet measured
The Introduction describes acute time and financial pressures on working parents, and the Results (section 4.1; page unavailable) report that mothers reduced paid work hours amid childcare and homeschooling demands. These passages support heightened parental pressure and reduced work time, but the paper does not assess test scores or establish societal costs from a decline in productivity. (LLM, conf=0.84)
- Partly supportedcites [66] P. Biroli (2020) “ Family life in lockdown” (IZA Discussion Paper 13398, Institute of Labor Economics, Bonn, Germany). sourceExperimental — accuracy not yet measured
The Results section (Family Cooperation and Tensions; page unavailable) reports increased household tension associated with changes in childcare and other household tasks during lockdown, supporting the point about pressure on parents. The paper does not establish societal costs from declining productivity or state that test scores omit these effects. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Doing so identifies the control for pretreatment trends as the most influential, followed by the control for test timing and the inclusion of school and family fixed effects.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Specification Curve Analysis.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the fixed-effects specification shifts the sample toward larger families, and effects in this subsample are similar using our baseline specification ( SI Appendix, section 7.7).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Excluding adjustment for testing date decreases the effect size by 12%, while including fixed effects increases it by 1.6% (school level) or 6.3% (family level).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Specification Curve Analysis.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We validate these benchmarks in our data by exploiting variation in testing dates during comparison years and show that test scores improve by 0.30 to 0.40 percentiles per week, equivalent to 0.31 to 0.41 SD annually ( SI Appendix, section 4.3).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Students lost on average 3.16 percentile points in the national distribution, equivalent to 0.08 standard deviations (SD) ( SI Appendix, section 4.3).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Baseline Specification.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In SI Appendix, Table S3, we show that the treatment sample is not skewed on sex, parental education, or prior performance.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This graph reveals a raw difference ranging from −0.76 percentiles in spelling to −2.15 percentiles in math.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These losses are evident throughout the age range we study and across all of the three subject areas: math, spelling, and reading.”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also reestimate our main specification dropping comparison years one at a time ( SI Appendix, section 7.3). These results are estimated with less precision but otherwise in line with those of our main analysis.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Placebo Analysis and Year Exclusions.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Our estimated treatment effect translates into 3.16/35 = 0.09 percentiles or 0.002 SD per school day and is thus on the lower end of that range.*”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“These groups make up, respectively, 92, 4, and 4% of the student body and our sample ( SI Appendix, section 5.1).”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > Parental Education.
What the sources say
- Not matchedcites SI Appendix, section 5.1
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“This literature reports reductions in achievement ranging from 0.001 to 0.010 SD per school day lost (10).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Not checked — no full text of the sourcecites [10] M. Kuhfeld, Projecting the potential impacts of COVID-19 school closures on academic achievement. Educ. Res. 49, 549– 565 (2020). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the smaller benchmark, learning loss exceeds the period of school closures (3.16/0.30 = 10.5 wk), implying that students regressed during this time.”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the larger benchmark, a treatment effect of 3.16 percentiles would translate into 3.16/0.40 = 7.9 wk of lost learning—nearly exactly the same period that schools in The Netherlands remained closed.”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, failure to adjust for pretreatment trends generates placebo estimates that are biased in a positive direction and is thus likely to underestimate treatment effects.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Specification Curve Analysis.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Hypotheses and analysis protocols for this study were preregistered ( SI Appendix, section 4.1).”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The Netherlands is interesting as a “best-case” scenario, with a short lockdown, equitable school funding, and world-leading rates of broadband access.”
- Also stated in: § Study Setting · hedged
“For these reasons, The Netherlands presents a “best-case” scenario, providing a likely lower bound on learning loss elsewhere in Europe and the world.”
§ Learning loss due to school closures during the COVID-19 pandemic > Abstract
What the sources say
- Not matchedcites [30] H. A. Patrinos, “ School choice in the Netherlands” (CESifo DICE Report 2/2011, ifo Institute for Economic Research, Munich, Germany, 2011).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [31] H. F. Ladd, E. B. Fiske, Weighted student funding in The Netherlands: A model for the US? J. Pol. Anal. Manag. 30, 470– 498 (2011). source
- Not matchedcites [33] Statistics Netherlands (CBS), The Netherlands leads Europe in internet access. https://www.cbs.nl/en-gb/news/2018/05/the-Netherlands-leads-europe-in-internet-access. Accessed 9 February 2021.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [34] G. Di Pietro, F. Biagi, P. Costa, Z. Karpinski, J. Mazza, The Likely Impact of COVID-19 on Education: Reflections Based on the Existing Literature and Recent International Datasets (Publications Office of the European Union, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [36] Johns Hopkins Coronavirus Resource Center, COVID-19 case tracker. https://coronavirus.jhu.edu/data. Accessed 9 February 2021.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [37] P. Tullis, Dutch cooperation made an ‘intelligent lockdown’ a success. Bloomberg Businessweek (4 June 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Study Setting · hedged
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which would indicate that remote learning operated at 50% efficiency.”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In our baseline specification, we adjust for a linear trend in year and the time elapsed between testing dates and cluster standard errors at the school level.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, we restrict analysis to schools where at least 75% of students took their tests after lockdown ( SI Appendix, section 7.5).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Although early influential studies also found that summer is a time when socioeconomic learning gaps widen, this finding has failed to replicate in more recent studies (8, 9) or in European samples (48, 49).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Not checked — no full text of the sourcecites [8] P. T. von Hippel, C. Hamrock, Do test score gaps grow before, during, or between the school years?: Measurement artifacts and what we can know in spite of them. Soc. Sci. 6, 43 (2019). source
- Not checked — no full text of the sourcecites [9] M. Kuhfeld, Surprising new evidence on summer learning loss. Phi Delta Kappan 101, 25– 29 (2019). source
- Not checked — no full text of the sourcecites [48] P. Verachtert, J. Van Damme, P. Onghena, P. Ghesquière, A seasonal perspective on school effectiveness: Evidence from a Flemish longitudinal study in kindergarten and first grade. Sch. Effect. Sch. Improv. 20, 215– 233 (2009). source
- Not checked — no full text of the sourcecites [49] F. Meyer, K. Meissel, S. McNaughton, Patterns of literacy learning in German primary schools over the summer and the influence of home literacy practices. J. Res. Read. 40, 233– 253 (2017). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our study represents one of the first attempts to quantify learning loss from COVID-19 using externally validated tests, a representative sample, and techniques that allow for causal inference.”
§ Learning loss due to school closures during the COVID-19 pandemic > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The country is close to the Organization for Economic Cooperation and Development (OECD) average in school spending and reading performance, but among its top performers in math (32).”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
What the sources say
- Not matchedcites [32] A. Schleicher, PISA 2018: Insights and Interpretations (OECD Publishing, 2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The short duration of school closures gave students, educators, and parents little time to adapt.”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“To identify the model components that exert the most influence on the magnitude of estimates we assessed more than 2,000 alternative models in a specification curve analysis (44) ( SI Appendix, section 7.13).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Specification Curve Analysis.
What the sources say
- Not checked — no full text of the sourcecites [44] U. Simonsohn, J. P. Simmons, L. D. Nelson, Specification curve analysis. Nat. Hum. Behav. 4, 1208– 1214 (2020). 32719546 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Despite favorable conditions, survey evidence from lockdown indicates high levels of dissatisfaction with remote learning (38) and considerable disparities in help with schoolwork and learning resources (39).”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
What the sources say
- Not checked — no full text of the sourcecites [38] M. de Haas, R. Faber, M. Hamersma, How COVID-19 and the Dutch ‘intelligent lockdown’ changed activities, work and travel behaviour: Evidence from longitudinal data in the Netherlands. Transport. Res. Interdisciplinary Perspect. 6. 100150 (2020). PMC7284275 34171019 source
- Not checked — no full text of the sourcecites [39] T Bol, Inequality in homeschooling during the corona crisis in the Netherlands. First results from the LISS Panel. SocArXiv [Preprint] (2020). 10.31235/osf.io/hf32q (Accessed 9 February 2021). source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The high degree of school autonomy in The Netherlands is also likely to have created considerable variation in the pandemic response, possibly explaining the wide school-level variation in estimated learning loss ( SI Appendix, section 7.9).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our difference-in-differences design discards those students who did not take the tests, which might lead to bias if their performance trajectories differ from those we observe.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Early estimates suggest that grades 3 to 8 students more than 6 mo into the pandemic underperformed by 7.5 percentile points in math but saw no loss in reading achievement (28).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Not matchedcites [28] M. Kuhfeld, B. Tarasawa, A. Johnson, E. Ruzek, K. Lewis, “ Learning during COVID-19: Initial findings on students’ reading and math achievement and growth” (NWEA Brief, Portland, OR, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We do so using a difference-in-differences design ( SI Appendix, section 4.2) and address loss to follow-up using various techniques: regression adjustment, rebalancing on propensity scores and maximum-entropy weights, and fixed-effects designs that compare students within the same schools and families.”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Following school reopenings, tests were taken in person under normal conditions and with minimal social distancing.”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Knowledge Learned vs. Transitory Influences.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Studies of school days lost due to other causes are mixed—some find durable effects and spillovers to adult earnings (59, 60), while others report a fadeout of effects over time (61, 62).”
§ Learning loss due to school closures during the COVID-19 pandemic > Discussion
What the sources say
- Not checked — no full text of the sourcecites [59] A. Ichino, R. Winter-Ebmer, The long-run educational cost of World War II. J. Labor Econ. 22, 57– 87 (2004). source
- Not checked — no full text of the sourcecites [60] D. Jaume, A. Willén, The long-run effects of teacher strikes: Evidence from Argentina. J. Labor Econ. 37, 1097– 1139 (2019). source
- Not checked — no full text of the sourcecites [61] S. Cattan, D. Kamhöfer, M. Karlsson, T. Nilsson “ The short- and long-term effects of student absence: Evidence from Sweden” (IZA Discussion Paper 10995, Institute of Labor Economics, Bonn, Germany, 2017). source
- Not checked — no full text of the sourcecites [62] B. Sacerdote, When the saints go marching out: Long-term outcomes for student evacuees from hurricanes Katrina and Rita. Am. Econ. J. Appl. Econ. 4, 109– 135 (2012). source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“School closures were short in comparative perspective ( SI Appendix, section 1), and the first wave of the pandemic had less of an impact than in other European countries (36, 37).”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
What the sources say
- Not matchedcites [36] Johns Hopkins Coronavirus Resource Center, COVID-19 case tracker. https://coronavirus.jhu.edu/data. Accessed 9 February 2021.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [37] P. Tullis, Dutch cooperation made an ‘intelligent lockdown’ a success. Bloomberg Businessweek (4 June 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Social distancing measures may have altered factors such as seating arrangements or indoor climate that in turn can influence student performance (41 –43).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Knowledge Learned vs. Transitory Influences.
What the sources say
- Not checked — no full text of the sourcecites [41] M. J. Mendell, G. A. Heath, Do indoor pollutants and thermal conditions in schools influence student performance?: A critical review of the literature. Indoor Air 15, 27– 52 (2005). 15660567 source
- Not checked — no full text of the sourcecites [42] P. D. Marshall, M. Losonczy-Marshall, Classroom ecology: Relations between seating location, performance, and attendance. Psychol. Rep. 107, 567– 577 (2010). 21117484 source
- Not checked — no full text of the sourcecites [43] R. J. Park, J. Goodman, A. P. Behrer, Learning is inhibited by heat exposure, both internationally and within the United States. Nat. Hum. Behav. 5, 19– 27 (2020). 33020588 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“No other country has higher rates of broadband penetration (33, 34), and efforts were made early in the pandemic to ensure access to home learning devices (35).”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
What the sources say
- Not matchedcites [33] Statistics Netherlands (CBS), The Netherlands leads Europe in internet access. https://www.cbs.nl/en-gb/news/2018/05/the-Netherlands-leads-europe-in-internet-access. Accessed 9 February 2021.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [34] G. Di Pietro, F. Biagi, P. Costa, Z. Karpinski, J. Mazza, The Likely Impact of COVID-19 on Education: Reflections Based on the Existing Literature and Recent International Datasets (Publications Office of the European Union, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [35] F. M. Reimers, A. Schleicher, A Framework to Guide an Education Response to the COVID-19 Pandemic of 2020 (OECD Publishing, 2020).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“The Dutch school system combines centralized and equitable school funding with a high degree of autonomy in school management (30, 31).”
§ Learning loss due to school closures during the COVID-19 pandemic > Study Setting
What the sources say
- Not matchedcites [30] H. A. Patrinos, “ School choice in the Netherlands” (CESifo DICE Report 2/2011, ifo Institute for Economic Research, Munich, Germany, 2011).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [31] H. F. Ladd, E. B. Fiske, Weighted student funding in The Netherlands: A model for the US? J. Pol. Anal. Manag. 30, 470– 498 (2011). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Next, we balance treatment and control groups on a wider set of covariates, including at the school level, using maximum-entropy weights and the estimated propensity of treatment ( SI Appendix, section 7.4).”
§ Learning loss due to school closures during the COVID-19 pandemic > Results > Adjusting for Loss to Follow-up.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This step reduces the size of our sample by approximately 60%, as not every student has a sibling attending a sampled school within the years that we are able to observe.”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > School and Family Fixed Effects.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We interpret it as a test of learning readiness.”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > Test Scores.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The first one is the student monitoring system, which provides our test score data (40). This system comprises a series of mandatory tests that are taken twice a year throughout a child’s primary school education (ages 6 to 12 y).”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods
What the sources say
- Not checked — no full text of the sourcecites [40] K. F. Vlug, Because every pupil counts: The success of the pupil monitoring system in The Netherlands. Educ. Inf. Technol. 2, 287– 306 (1997). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Throughout our analyses, we adjust confidence intervals for clustering on schools using robust standard errors.”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > Difference-in-Differences Analysis.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Nationally standardized tests are taken across three main subjects: math, spelling, and reading ( SI Appendix, section 3.1).”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > Test Scores.
What the sources say
- Not matchedcites SI Appendix, section 3.1
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“The sample covers 15% of all primary schools and is broadly representative of the national student body ( SI Appendix, section 5.1).”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods
What the sources say
- Not matchedcites SI Appendix, section 5.1
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the terminology of the test producer, its goal is to assess “technical reading ability”—likely a mix of reading ability, cognitive processing, and verbal fluency.”
§ Learning loss due to school closures during the COVID-19 pandemic > Materials and Methods > Test Scores.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
Students across The Netherlands take the same examination within a given year.
“Students across The Netherlands take the same examination within a given year.”
The paper relies on this without support.
The norm for transforming test results to percentile scores is the same across years, so absolute changes in performance over time are preserved.
“Test results are transformed to percentile scores, but the norm for transformation is the same across years so absolute changes in performance over time are preserved.”
The paper relies on this without support.
The oral reading fluency assessment does not test for the retention of curricular content.
“As this part of the assessment does not test for the retention of curricular content, we would expect it to be less affected by school closures, which is indeed what we find.”
The authors state this as an assumption.
Percentiles were drawn from an underlying normal distribution.
“Assuming that percentiles were drawn from an underlying normal distribution, we use the following formula to convert between one and the other: d=Φ−10.50+δ100, [3] where δ is the treatment effect on the percentile scale, and Φ−1 is the inverse cumulative standard normal distribution.”
The paper relies on this without support.
In the absence of the pandemic, the rate of progress in 2020 would have followed the same trend as in prior years.
“We analyze the rate of progress in 2020 to that in previous years using a difference-in-differences design.”
The paper relies on this without support.
Comprehension of the words is not needed and students and parents are discouraged to prepare for the oral reading fluency test.
“Crucially, comprehension of the words is not needed and students and parents are discouraged to prepare for it.”
The paper relies on this without support.
The test producer's translation keys are based on smaller samples than the study's sample.
“However, as these keys are actually based on smaller samples than that at our disposal, we further impose a uniform distribution in our sample within cells defined by subject, grade, and testing occasion: midyear vs. end of year.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“School attendance is foundational to young people’s educational attainment, social development, and future life opportunities (Engzell et al., 2021; McDonald et al., 2022).”
From Beyond the Classroom: Strengthening Pupil Engagement, Attendance and Belonging through Alternative Education and Community Partnerships (2025) · cites this paper as Engzell et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The impact is 40 percent larger among those in the least educated homes, suggesting that the pandemic not only increased educational inequality, but that disadvantaged children's skills actually deteriorated.”
From Educational Inequality* (2022) · cites this paper as Engzell, Frey, and Verhagen (2021)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Results from Engzell, Frey, and Verhagen (2021) for an eight-week school closure imply that this gap would further increase by around 0.05 of a standard deviation.”
From Educational Inequality* (2022) · cites this paper as Engzell, Frey, and Verhagen (2021)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Engzell et al. [6] found that the time-measured learning loss in relation to the progress that would be achieved in a normal school year corresponds to a period of 7.9 to 10.5 weeks of non-learning [6] (p. 10).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Engzell, Frey, and Verhagen (2021) find that in the Netherlands, eight weeks of online rather than in-person learning led to 0.08 of a standard deviation lower test scores for students aged eight to eleven.”
From Educational Inequality* (2022) · cites this paper as Engzell, Frey, and Verhagen (2021)
The cited source says what the claim attributes to it.
In the citing paper
“However, the learning loss was up to 60% greater among students from disadvantaged backgrounds (i.e., students from less-educated households), indicating the disproportionate impact of the pandemic on student learning.”
From Impacts of the COVID-19 Pandemic on Student Learning and Opportunity Gaps Across the 2020–2021 School Year: A National Survey of Teachers (2022) · cites this paper as Engzell et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“Several studies have demonstrated that student achievement has been lower during the pandemic compared to prior to the pandemic, with estimates ranging from three percentile points in the Netherlands (Engzell et al., 2021) to more than half of a school year in the U.S. state of Illinois (Streich et al., 2021).”
From Impacts of the COVID-19 Pandemic on Student Learning and Opportunity Gaps Across the 2020–2021 School Year: A National Survey of Teachers (2022) · cites this paper as Engzell et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Results indicated a learning loss equivalent to 3 percentile points despite the relatively short lockdown.”
From Impacts of the COVID-19 Pandemic on Student Learning and Opportunity Gaps Across the 2020–2021 School Year: A National Survey of Teachers (2022) · cites this paper as Engzell et al. (2021)
In the citing paper
“The authors also noted that access to technology alone is not enough to guarantee quality distance learning [6] (p. 11).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
In the citing paper
“Overall, a learning loss of 3 percentile points or 0.08 standard deviations was determined, whereby the losses in the case of pupils from less educated parents turned out to be 60% greater [6] (p. 1).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
In the citing paper
“With a sample size of n = 286,515, a significant difference in mean values was found when comparing the control group and the test group.The group of pupils that was tested after the school closures showed significantly worse test values than a comparison group that did not experience any school closures [6] (p. 85).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
In the citing paper
“Engzell and colleagues quantified the learning loss, taking into account all the differences measured by them, with an overall negative effect of -0.08 standard deviations [6] (p. 1).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
In the citing paper
“The test values of the group of pupils that were tested after the school closures (test group, M = -1.28,SD = 11.83), were on average significantly lower than those of the comparison group (control group, M = 0.28, SD = 10.99) that had not experienced any school closures [6] (p. 85).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
In the citing paper
“Also, with regard to the area of spelling, a significantly lower test mean value was reported for the test group (M = 0.71, SD = 17.07, n = 58,627) compared to the control group (M = 0.05, SD = 17.27, n = 284,499) (p.85).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [6]
In the citing paper
“As pointed out by Engzell et al. (2021), non-causal methods fail to account for trends in student progress prior to the outbreak of Covid-19 and, hence, by assuming a counterfactual where achievement has stayed flat, they generate estimates of learning deficits that are biased downwards.”
From The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis (2023) · cites this paper as Engzell et al. (2021)
In the citing paper
“there is a large consensus that it did not affect students equally.”
From The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis (2023) · cites this paper as Engzell et al., 2021
In the citing paper
“several studies (see, for example, Engzell et al., 2021; Hevia et al., 2022) showed that Covid-19 had a detrimental effect especially on the achievement of students from less advantaged backgrounds.”
From The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis (2023) · cites this paper as Engzell et al., 2021
In the citing paper
“The pandemic has also interrupted education and has adversely affected children’s social and emotional development and mental health.16-19”
From Evaluation of the BNT162b2 Covid-19 Vaccine in Children 5 to 11 Years of Age (2022) · cites this paper as [17]
In the citing paper
“Covid-19–associated school closures and quarantines also have social and economic costs for families and caregivers.16-19”
From Evaluation of the BNT162b2 Covid-19 Vaccine in Children 5 to 11 Years of Age (2022) · cites this paper as [17]
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
The Psychological Impact of Quarantine and How to Reduce It: Rapid Review of the Evidence
Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020) · doi:10.2139/ssrn.3532534 · Cited by 16,394
The psychological impact of quarantine and how to reduce it: Rapid review of the evidence
Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30460-8 · Cited by 16,378
Multilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 6,830
Hainmueller, J. (2012) · Political Analysis · doi:10.1093/pan/mpr025 · Cited by 6,026
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926
Viner, R. M., Russell, S. J., Croker, H., Packer, J., Ward, J., Stansfield, C., Mytton, O., Bonell, C., & Booy, R. (2020) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(20)30095-x · Cited by 2,213
Cited by
Sorted by citations
Items 201–300 of 1,570
SARS-CoV-2 Circulation in the School Setting: A Systematic Review and Meta-Analysis
Caini, S., Martinoli, C., Vecchia, C. L., Raimondi, S., Bellerba, F., D’Ecclesiis, O., Sasso, C., Basso, A., Cammarata, G., & Gandini, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095384 · Cited by 19
“Numerous studies have shown that this decision had a negative impact not only on the learning loss [66] in students but also on their mental health [67,68].”
BackgroundLyu, W., & Wehby, G. L. (2023) · Academic Pediatrics · doi:10.1016/j.acap.2022.12.016 · Cited by 21
“Other evidence suggests that children had poorer academic performance during the COVID-19 pandemic, and that children from disadvantaged family backgrounds or with special care needs were more adversely affected.(39) Continuing to monitor these outcomes over time is important given the pandemic continuation in 2021 an…”
BackgroundInfants’ and toddlers’ language development during the pandemic: Socioeconomic status mattered
Fung, P., Pierre, T. S., Raja, M., & Johnson, E. K. (2023) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2023.105744 · Cited by 19
Ryan, B. J., Kako, M., Garner, S. L., Fink, R., Tayfur, İ., Abrahams, J., Bhatia, S., Campelo, A., Fendt, M. W., Fontenot, A., et al. (2024) · International Journal of Disaster Risk Science · doi:10.1007/s13753-024-00537-x · Cited by 19
“For example, school closures were implemented in many locations as a measure to slow the spread of COVID-19, but as a result, students missed out on significant education opportunities with many losing one to two years of competencies (Engzell et al. 2021).”
The COVID ‐19 pandemic and young people's civic engagement: A scoping review
Schoon, I., Shukla, S., Verma, S., Terol, E. H., & da Cunha, J. M. (2024) · Journal of Research on Adolescence · doi:10.1111/jora.13039 · Cited by 24
Fiscal Federalism and K–12 Education Funding: Policy Lessons from Two Educational Crises
Shores, K., & Steinberg, M. P. (2022) · Educational Researcher · doi:10.3102/0013189x221125764 · Cited by 24
“…constant other pandemic-related disruptions to learning) shows losses of 0.48 SD for low-income students and 0.35 SD for non-low-income students (Engzell et al., 2021).5 At the low end, Kuhfeld and colleagues provide estimates of lost learning ranging from 0.25 SD for non-low-income students to…”
BackgroundInfluentialThe Impacts of Remote Learning in Secondary Education: Evidence from Brazil during the PandemicPreprint
Lichand, G. F., Dória, C. A., Neto, O. L., et al. (2021) · Research Square · doi:10.21203/rs.3.rs-568605/v1 · Cited by 19
COVID-19 Pandemic and the Developmental Health of Kindergarteners
Perrigo, J. L., Morales, J., JACKSON, N. J., Janus, M., Stanley, L., Wong, M. D., & Halfon, N. (2025) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2024.7057 · Cited by 20
Zartler, U., Dafert, V., & Dirnberger, P. (2021) · Journal of Family Research · doi:10.20377/jfr-713 · Cited by 18
“Moreover, there is growing evidence that school closures promote social exclusion and reinforce unequal educational opportunities for children (Champeaux et al. 2020; Engzell et al. 2021; Huber 2020; OECD 2020a, 2020b; SORA 2020a).”
BackgroundRaimondi, S., Gandini, S., Horemheb-Rubio, G., Abecasis, A. B., Lopalco, P. L., D’Ecclesiis, O., Chiocca, S., Tomezzoli, E., Cutica, I., Mazzoni, D., et al. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-022-07947-6 · Cited by 16
“Importantly, school closure, supported or not by remote teaching, has been proven to cause harms to children and society in terms of health [21], psychological consequences [22, 23] and learning deprivation [24] with consequent long-term learning losses [25, 26], that are higher in socio-economically disadvantaged ind…”
BackgroundWho Lost Most Wages and Household Income during the COVID‐19 Pandemic in Poor Rural China?
Long, W., Zeng, J., & Sun, T. (2021) · China & World Economy · doi:10.1111/cwe.12396 · Cited by 19
“, 2020; 2021), and education (Engzell et al., 2021) over the past decade.”
BackgroundNo Evidence of a Major Learning Slide 14 Months into the COVID-19 Pandemic in DenmarkPreprint
Birkelund, J. F., & Karlson, K. B. (2021) · doi:10.31235/osf.io/md5zn · Cited by 18
Learning loss and recovery from the COVID-19 pandemic: A systematic review of evidence
Cruz, N. A. D., Adona, A. J., Molato-Gayares, R., & Park, A. F. (2025) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2025.103271 · Cited by 21
Pan, Q., Reichert, F., Liang, Q., TORRE, J. D. L., & Law, N. W. Y. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13592-8 · Cited by 28
“Comprehensive DL, encompassing various competences required to use information and communication technology (ICT) effectively, is essential for mitigating educational inequalities (Engzell et al., 2021).”
Rhamdan, D., Kule, A., & Wahid, S. M. A. (2021) · JURNAL PENDIDIKAN DAN KEWIRAUSAHAAN · doi:10.47668/pkwu.v9i2.263 · Cited by 20
Goodrich, J. M., Hebert, M., & Namkung, J. M. (2022) · Frontiers in Education · doi:10.3389/feduc.2021.793285 · Cited by 24
“Even in countries with relatively short lockdown periods, remote instruction resulted in minimal progress, and students frommarginalized backgrounds were impacted the most by school closures (Engzell et al., 2021).”
BackgroundLichand, G. F., Dória, C. A., Cossi, J., & Neto, O. L. (2022) · JAMA Health Forum · doi:10.1001/jamahealthforum.2021.5032 · Cited by 18
Turkey's transition to face-to-face schooling during the COVID-19 pandemic
Özer, M., Eren, S. U. N. A., Perc, M., Şensoy, S., & İlikhan, S. U. (2022) · TURKISH JOURNAL OF MEDICAL SCIENCES · doi:10.55730/1300-0144.5343 · Cited by 20
Bayrakdar, S., & Güveli, A. (2022) · Sociology · doi:10.1177/00380385221122444 · Cited by 18
“Emerging research suggests far-reaching educational consequences, especially for children in working-class families (Andrew et al., 2020; Bol, 2020; Engzell et al., 2020).”
BackgroundInfluentialSpitzer, M., Moeller, K., & Musslick, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0284868 · Cited by 18
“In line with the two studies mentioned earlier [9,10], their findings revealed significant learning losses in mathematics, specifically for students from low socioeconomic backgrounds.”
ResultHeikonen, L., Ahtiainen, R., Hotulainen, R., Oinas, S., Rimpelä, A., & Koivuhovi, S. (2023) · Teaching and Teacher Education · doi:10.1016/j.tate.2023.104399 · Cited by 21
“There is a growing amount of research about the effects of COVID-19 pandemic on students ’ learning and wellbeing (e.g., Engzell et al., 2021; Hossain et al., 2022; Maldonado & De Witte, 2022), but the pandemic has also put pressure on teachers in terms of needing to re-design teaching routines and…”
Background“Let’s transform!”: A systematic literature review of science learning in COVID-19 pandemic era
Nurwidodo, N., Ibrohim, I., Sueb, S., & Husamah, H. (2023) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/12875 · Cited by 20
A meta-analysis of the impact of COVID-19 on student achievement
Storey, N., & Zhang, Q. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100624 · Cited by 18
Distance Learning Effects Among Italian Children and Parents During COVID-19 Related School Lockdown
Crisci, G., Mammarella, I. C., Moscardino, U. M. M., Roch, M., & Thorell, L. B. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.782353 · Cited by 17
“Regarding positive effects of DL the Global test indicated that the assumptions of the regression model could not be rejected, χ2(4)= 7.”
BackgroundMethodInfluentialPatzina, A., & Dietrich, H. W. (2022) · SSM - Population Health · doi:10.1016/j.ssmph.2022.101054 · Cited by 17
“Research indicates that COVID-19 policies (e.g., school closures or mandatory mask wearing) have different effects on individuals from different parts of the social strata (e.g., Engzell et al., 2021; Hearne & Ni ˜ no, 2021; Papageorge et al., 2021).”
Test-to-Stay After Exposure to SARS-CoV-2 in K–12 Schools
Campbell, M. M., Benjamin, D. K., Mann, T. K., Fist, A., Kim, H., Edwards, L., Rak, Z., Brookhart, M. A., Anstrom, K. J., Moore, Z., et al. (2022) · PEDIATRICS · doi:10.1542/peds.2021-056045 · Cited by 17
Bloem, J., Flunger, B., Stroet, K., & Hornstra, L. (2023) · Social Psychology of Education · doi:10.1007/s11218-023-09831-w · Cited by 22
Hačatrjana, L. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10010007 · Cited by 20
“…are crucial for students to maintain their academic performance and well-being as much as possible during this time (Rosen et al. 2021; Hacatrjana 2021a), and some recent data have indicated a decrease in students’ academic performance that is probably due to the pandemic (Engzell et al. 2021).”
BackgroundPattnaik, J., Nath, N., & Nath, S. (2022) · Early Childhood Education Journal · doi:10.1007/s10643-022-01331-4 · Cited by 28
Davis, T. J., Merchant, Z., & Kwok, O. (2022) · Education Sciences · doi:10.3390/educsci12040262 · Cited by 21
Doghonadze, N., Dolidze, T., & Vasadze, N. (2021) · Journal of Education in Black Sea Region · doi:10.31578/jebs.v7i1.254 · Cited by 17
Aufholen nach Corona? Maßnahmen der Länder im Kontext des Aktionsprogramms von Bund und Ländern
Helbig, M., Edelstein, B., Fickermann, D., & Zink, C. (2022) · Waxmann Verlag GmbH eBooks · doi:10.31244/9783830996033 · Cited by 17
Samsen‐Bronsveld, H. E., van der Ven, S. H. G., Speetjens, P., & Bakx, A. W. E. A. (2022) · Journal of Research in Special Educational Needs · doi:10.1111/1471-3802.12583 · Cited by 19
Connell, A. M., & Stormshak, E. A. (2023) · Journal of Prevention · doi:10.1007/s10935-023-00727-1 · Cited by 17
“Children and adolescents were also heavily impacted by the pandemic, and increases in youth emotional, behavioral, and academic difficulties (e.g. Engzell et al., 2021; Thorisdottir et al., 2021; Waller et al., 2021) represented another significant stressor for parents.”
BackgroundTime use of youth during a pandemic: Evidence from Mexico
Boruchowicz, C., Parker, S. W., & Robbins, L. (2021) · World Development · doi:10.1016/j.worlddev.2021.105687 · Cited by 19
“Moreover, existing evidence from the pandemic in other contexts shows a significant reduction in learning, as measured by standardized tests (Kuhfeld, Tarasawa, Johnson, Ruzek, & Lewis, 2020; Engzell et al., 2021).”
BackgroundInfluentialRotevatn, T. A., Larsen, V. B., Johansen, T. B., Astrup, E., Surén, P., Greve‐Isdahl, M., & Telle, K. E. (2022) · BMJ Medicine · doi:10.1136/bmjmed-2021-000026 · Cited by 17
Sattler, S., Taflinger, S., Ernst, A., & Hasselhorn, F. A. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.842368 · Cited by 13
“The social and economic costs of such measures include, but are not limited to, negative impacts on student learning (5) and a sharp decrease in working hours and job losses (6).”
BackgroundEnhancing statistics education through Project‐Based Learning ( PBL ) and the emergence of ChatGPT
Al‐Labadi, L., & Ly, A. (2025) · Teaching Statistics · doi:10.1111/test.12405 · Cited by 25
Computer-Based Intervention Closes Learning Gap in Maths Accumulated in Remote Learning
Ökördi, R., & Molnár, G. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10030058 · Cited by 16
“Similar trends have been published for the overall performance of Flemish primary students (Maldonado and De Witte 2020), for Dutch students (Engzell et al. 2021) and for US students in mathematics (Kuhfeld et al.”
BackgroundInfluentialReading fluency during the COVID-19 pandemic: a longitudinal and cross-sectional analysis
Alves, L. M., Carvalho, I. M., dos Santos, L. F., Ribeiro, G. D. L., Freire, L. D. S. C., Martins-Reis, V. D. O., Labanca, L., & Celeste, L. C. (2022) · Arquivos de Neuro-Psiquiatria · doi:10.1055/s-0042-1758446 · Cited by 17
“11,12 Based on the data that has already been published on the effects of the pandemic on the educational systems of other countries, 7,8,11,12 it has become evident that there is a need to monitor the reading skills of Brazilian students to be able to assess the effect of e-learning on the…”
BackgroundResultInfluentialBertoletti, A., Biagi, F., Pietro, G. D., & Karpiński, Z. (2023) · Large-scale Assessments in Education · doi:10.1186/s40536-023-00154-y · Cited by 19
“There is a large body of evidence suggesting that, following the COVID-19 pandemic, the sudden shift from physical presence at school to on line learning has led to a significant learning loss (see, for instance, Maldonado & De Witte, 2022; Engzell et al., 2021).”
BackgroundNazareno, L. D. A., & Galvao, J. D. C. (2023) · Population Research and Policy Review · doi:10.1007/s11113-023-09749-3 · Cited by 16
“Recent studies show that, while conditional cash transfer programs implemented during the COVID-19 pandemic were important to curb poverty increases, they were not enough, for example, to address schooling inequalities as a result of prolonged school closures (Engzell et al., 2021; Lustig et al., 2021; Zoido et al., 2…”
BackgroundRodríguez‐Rivas, M. E., Alfaro, J., Benavente, M., Varela, J. J., Roberto Melipillán, A., & Reyes, F. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e15001 · Cited by 15
Mendoza, N. B., & Dizon, J. I. W. T. (2023) · Social Psychology of Education · doi:10.1007/s11218-023-09834-7 · Cited by 22
Techniques to enhance the public policy impact of qualitative system dynamics models
Veldhuis, G. A., Smits-Clijsen, E., & van Waas, R. P. M. (2023) · System Dynamics Review · doi:10.1002/sdr.1758 · Cited by 17
Spitzer, M., & Moeller, K. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100162 · Cited by 16
COVID-19 disruptions and education in South Africa: Two years of evidence
Wills, G., & van der Berg, S. (2024) · Development Southern Africa · doi:10.1080/0376835x.2024.2311711 · Cited by 22
“Even in high-income countries like the Netherlands, Belgium, Germany and Italy, where distance learning was feasible, learning outcomes fell short of their usual standards (Contini et al., 2021; Engzell et al., 2021; Maldonado & De Witte, 2022; Schult et al., 2022).”
BackgroundDurrani, N., & Ozawa, V. (2024) · SAGE Open · doi:10.1177/21582440241233402 · Cited by 16
“Parental education and household income have been reported as key markers of disadvantage and privilege (Engzell et al., 2021; Haelermans et al., 2022; Haser et al., 2022).”
PERMASALAHAN BELAJAR DARI RUMAH BAGI GURU LEMBAGA PENDIDIKAN ANAK USIA DINI DI DAERAH TERPENCIL
Pratiwi, H. P., & -, A. (2021) · Jurnal Pendidikan dan Kebudayaan · doi:10.24832/jpnk.v6i2.1928 · Cited by 27
Crisis‐ready educational design: The case of mathematics
Foster, C. F., Burkhardt, H., & Schoenfeld, A. H. (2022) · The Curriculum Journal · doi:10.1002/curj.159 · Cited by 16
Health promoting schools and COVID-19: preparing for the future
Sormunen, M., Lattke, L. S., Leksy, K., Dadaczynski, K., Sakellari, Ε., Velasco, V., Borzucka‐Sitkiewicz, K., Miranda-Velasco, M. J., Rabaglietti, E., & Rosario, R. V. (2022) · Scandinavian Journal of Public Health · doi:10.1177/14034948221091155 · Cited by 16
“[6] Engzell P, Frey A and Verhagen MD.”
BackgroundGrit and Academic Resilience During the Covid-19 PandemicPreprint
Chen, D. L., Ertaç, S., Evgeniou, T., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4001431 · Cited by 16
Grit and academic resilience during the COVID-19 pandemic
Chen, D. L., Ertaç, S., Evgeniou, T., Miao, X., Nadaf, A., & Yilmaz, E. (2024) · npj Science of Learning · doi:10.1038/s41539-024-00265-3 · Cited by 30
“Indeed, recent evidence shows that even in “ best-case scenario ” countries with favorable conditions, students have experienced learning losses with remote instruction during the pandemic 7 .”
BackgroundInternational Inequality and the COVID-19 Pandemic
Dauderstädt, M. (2022) · Intereconomics · doi:10.1007/s10272-022-1026-9 · Cited by 15
“The number of life years lost per 100,000 people in 2020 appears to be higher in richer than in poor countries (Ferreira, 2021).”
BackgroundMayers, T., Mathis, B. J., Ho, C. K., Morikawa, K., Maki, N., & Hisatake, K. (2022) · Education Sciences · doi:10.3390/educsci12090628 · Cited by 15
“As seen in some European studies of school students and undergraduates experiencing a similar lockdown period, motivation seems to have decreased due to many factors, including poor internet access, socioeconomic factors (loss of part-time work), perception of online studies as being less useful, and so on [2,17].”
BackgroundBertogg, A., & Koos, S. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.837968 · Cited by 15
“A number of studies warn that the pandemic may have reinforced existing social inequalities (Ohlbrecht and Jellen, 2020; Engzell et al., 2021; Gauthier et al., 2021).”
BackgroundMueed, A., Aliani, R., Abdullah, M., Kazmi, T., Sultan, F., & Khan, A. A. (2022) · PLOS Global Public Health · doi:10.1371/journal.pgph.0000266 · Cited by 15
Cloud Computing-based Higher Education Platforms during the COVID-19 Pandemic
Han, H., & Trimi, S. (2022) · International Conference on e-Education, e-Business, e-Management and e-Learning · doi:10.1145/3514262.3514307 · Cited by 25
“One of the long-lasting impact areas of COVID-19 has been education where millions of students had to learn remotely online or through Zoom [7].”
BackgroundOostdam, R., van Diepen, M., Zijlstra, B. J. H., & Fukkink, R. G. (2023) · European Journal of Psychology of Education · doi:10.1007/s10212-023-00679-4 · Cited by 15
“The Hague is an urban region in the Netherlands with large differences between school populations in various neighborhoods; the number of households with children with a low income varies from low (< 7.5”
InfluentialBranch‐Elliman, W., Elwy, A. R., & Chambers, D. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1207679 · Cited by 16
Dantchev, S., & Zemp, M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.761276 · Cited by 15
Hofer, S. I., Reinhold, F., & Koch, M. (2022) · European Journal of Psychology of Education · doi:10.1007/s10212-021-00590-w · Cited by 26
“These students hence lack at least some of the protective resources that a supportive home learning environment provides (Engzell et al., 2021).”
BackgroundBergen, P. V., & Daniel, E. (2022) · The Australian Educational Researcher · doi:10.1007/s13384-022-00565-w · Cited by 16
COVID-19 Projections for K12 Schools in Fall 2021: Significant Transmission without InterventionsPreprint
Zhang, Y., Johnson, K., Yu, Z., Fujimoto, A. B., Lich, K. H., Ivy, J. S., Keskinocak, P., Mayorga, M. E., & Swann, J. (2021) · medRxiv · doi:10.1101/2021.08.10.21261726 · Cited by 14
Controlling SARS-CoV-2 in schools using repetitive testing strategies
Torneri, A., Willem, L., Vittoria Colizza, A., Kremer, C., Meuris, C., Darcis, G., Hens, N., & Libin, P. (2022) · eLife · doi:10.7554/elife.75593 · Cited by 14
Nabung, A., Ni, L., & Edu, A. L. (2022) · Jurnal Basicedu · doi:10.31004/basicedu.v6i3.2799 · Cited by 14
Review of higher education policy during the pandemic: A Spanish perspective
Mula-Falcón, J., González, C. C., Segovia, J. D., & Rodríguez, C. L. (2022) · Policy Futures in Education · doi:10.1177/14782103221134188 · Cited by 14
“As a measure to contain the pandemic, school closures led to the accelerated deployment of distance learning solutions to ensure pedagogical continuity in higher education (Darras et al., 2021; Engzell et al., 2021).”
BackgroundCOVID-19 school closures and mental health of adolescent students: Evidence from rural Mozambique
Chimbutane, F., Herrera‐Almanza, C., Karachiwalla, N., Lauchande, C., & Leight, J. (2023) · SSM - Mental Health · doi:10.1016/j.ssmmh.2023.100203 · Cited by 14
“Particularly in low-income countries, where schools are a crucial path out of poverty (and the technology for learning at a distance is almost entirely absent), the disruption in schooling over a prolonged period of time had meaningful implications for children's learning, health, and emotional well-being (Engzell et…”
BackgroundLo, N., & to, B. K. H. (2023) · SN Computer Science · doi:10.1007/s42979-023-01779-0 · Cited by 16
Chatzoglou, E., Fishstrom, S., Payne, S. B., Andress, T. T., & Vaughn, S. R. (2023) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2023.104585 · Cited by 14
Hsieh, H.-M., & Maritz, A. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i20.43907 · Cited by 14
The politicization of PISA in evidence-based policy discourses
Volante, L., & Mattei, P. (2024) · Policy Futures in Education · doi:10.1177/14782103241227309 · Cited by 14
“…troubling learning losses associated with the pandemic (Bailey et al., 2021; Blazko et al., 2021; Donnelly and Patrinos, 2021; Dorn et al., 2020; Engzell et al., 2021; Kaffenberger, 2021; Maldonato and De Witte, 2022; Milner et al., 2021; Schnepf et al., in press; Volante et al., 2022), further…”
Ghanamah, R., Eghbaria-Ghanamah, H., & Adi‐Japha, E. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101898 · Cited by 18
Bellibaş, M. Ş., Ryskulueva, F., Levin, J., & Pietsch, M. (2024) · Educational Management Administration & Leadership · doi:10.1177/17411432241264696 · Cited by 32
“…cant over the past decade, with educators worldwide facing a rapidly diversifying student body with increasingly varied needs due to wars, con fl icts, and natural disasters leading to increased immigration (Schachner et al., 2019), alongside post-pandemic learning challenges (Engzell et al., 2021).”
Pattnaik, J., & Jalongo, M. R. (2021) · Early Childhood Education Journal · doi:10.1007/s10643-021-01220-2 · Cited by 14
Considerations for the Safe Operation of Schools During the Coronavirus Pandemic
Lordan, R., Prior, S., Hennessy, E. J., Naik, A. S., Ghosh, S., Paschos, G. K., Skarke, C., Barekat, K., Hollingsworth, T., Juska, S. A., et al. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.751451 · Cited by 16
“The impact of the extended school closures during the pandemic remains to be fully appreciated, but students in the Netherlands showed little progress while learning from home during the first lockdown (96).”
BackgroundDEVELOPING COMMUNICATION COMPETENCE IN DIGITAL LEARNING ENVIRONMENTS FOR PRIMARY SCIENCE STUDENTS
Legvart, P., Aberšek, M. K., & Kerneža, M. (2022) · Journal of Baltic Science Education · doi:10.33225/jbse/22.21.836 · Cited by 14
In-person schooling is essential even during periods of high transmission of COVID-19
Munro, A. P. S., Buonsenso, D., González‐Dambrauskas, S., Hughes, R. C., Bhopal, S., Vásquez‐Hoyos, P., Çevik, M., Rubio, M. L. M., & Roland, D. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112277 · Cited by 14
School closures and effective in-person learning during COVID-19
Kurmann, A., & Lalé, É. (2023) · Economics of Education Review · doi:10.1016/j.econedurev.2023.102422 · Cited by 18
“…to estimate the consequences of reduced in-person instruction on student enrollment and academic achievement (Dee et al., 2021; Dorn et al., 2021; Engzell et al., 2021; Jack et al., 2022; Kogan and Lavertu, 2021; Lewis et al., 2021; Goldhaber et al., 2022), COVID infection and death rates…”
Zhang, Q., & Storey, N. (2022) · Theory Into Practice · doi:10.1080/00405841.2022.2096380 · Cited by 14
“Since standardized assessment scores only reflect the performance of the students who attend scheduled examinations, vulnerable students who were absent from exam days during the pandemic are more likely to be ignored (Engzell et al., 2021).”
BackgroundChernozhukov, V., Kasahara, H., & Schrimpf, P. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2103420118 · Cited by 63
Welter, V. D. E., Becker, L. B., & Großschedl, J. (2022) · Education Sciences · doi:10.3390/educsci12050325 · Cited by 18
Weber, C., Helm, C., & Kemethofer, D. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.737064 · Cited by 15
“Our results are in line with the growing research that highlights that COVID-19-related school closures have affected educationally disadvantaged students more strongly than their more advantaged peers, resulting in a widening achievement gap (Maldonado and De Witte, 2020; Engzell et al., 2021; Pier et al., 2021).”
BackgroundResultHast, M. A., Swanson, M. R., Scott, C., Oraka, E., Espinosa, C. C., Burnett, E., Kukielka, E. A., Rice, M. E., Mehari, L., McCloud, J., et al. (2022) · BMC Public Health · doi:10.1186/s12889-021-12347-7 · Cited by 13
Chaban, T., Rameeva, R., Denisov, I., Дмитриевна, К. Ю., & Zvyagintsev, R. (2022) · Voprosy Obrazovaniya/ Educational Studies Moscow · doi:10.17323/1814-9545-2022-1-160-188 · Cited by 13
The Role of Preschool Dosage and Quality in Children’s Self-Regulation Development
Melo, C., Pianta, R. C., LoCasale‐Crouch, J., Romo, F., & Ayala, M. C. (2022) · Early Childhood Education Journal · doi:10.1007/s10643-022-01399-y · Cited by 19
Designing Immersive, Narrative-Based Interfaces to Guide Outdoor Learning
Cheng, A. Y., Ritchie, J., Agrawal, N., Childs, E., DeVeaux, C., Jee, Y., Leon, T., Maples, B., Cuadra, A., & Landay, J. A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581365 · Cited by 14
“Many children are behind in spelling [16], which affects their writing skills, whether typing or handwriting.”
BackgroundChallenges in Implementing Kurikulum Merdeka for EFL Teachers
Reza, F., Rohmah, Z., & Abdullah, N. N. (2023) · JEELS (Journal of English Education and Linguistics Studies) · doi:10.30762/jeels.v10i2.1899 · Cited by 13
“This condition occurs when students encounter impediments or delays in the acquisition of new knowledge or skills (Kuhfeld et al., 2020; Engzell et al., 2021).”
BackgroundTrends in Mental Wellbeing of US Children, 2019–2022: Erosion of Mental Health Continued in 2022
Bhandari, N., & Gupta, S. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21020132 · Cited by 15
“A growing body of literature suggests prolonged school closures impeded academic progress and contributed to “learning loss” in children [29,30].”
BackgroundCopeland, K. A., Porter, L., Gorecki, M. C., Reyner, A., White, C., & Kahn, R. S. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.6458 · Cited by 13
Projecting the long-term effects of the COVID-19 pandemic on U.S. population structure
Tilstra, A. M., Polizzi, A., Wagner, S., & Akimova, E. T. (2024) · Nature Communications · doi:10.1038/s41467-024-46582-4 · Cited by 13
“The halting of in-migrationunderTitle 42 likely contributed to a reduction in working-aged individuals, of which the biggest reduction was among reproductive-aged individuals (660 thousand people; 0.41”
BackgroundInfluentialGermany’s education policy during the COVID-19 crisis
Freundl, V., Lergetporer, P., & Zierow, L. (2021) · Zeitschrift für Politikwissenschaft · doi:10.1007/s41358-021-00262-7 · Cited by 13
Closing lower secondary schools had no impact on COVID-19 incidence in 13–15-year-olds in Finland
Juutinen, A., Sarvikivi, E., Laukkanen‐Nevala, P., & Helve, O. (2021) · Epidemiology and Infection · doi:10.1017/s0950268821002351 · Cited by 11
Schools under mandatory testing can mitigate the spread of SARS-CoV-2
Diederichs, M., van Ewijk, R. J. G., Isphording, I. E., & Pestel, N. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201724119 · Cited by 13
“School closures produced substantial direct and indirect costs in learning losses (Engzell et al., 2021), children’s health and mental well-being (Viner et al., 2021), parental labor market outcomes (Heggeness, 2020) and domestic violence (Leslie and Wilson, 2020).”
Chzhen, Y., Symonds, J. E., Devine, D., Mikolai, J., Harkness, S., Sloan, S., & Sainz, G. M. (2022) · Child Indicators Research · doi:10.1007/s12187-022-09922-8 · Cited by 13
Haelermans, C., Jacobs, M., van der Velden, R., van Vugt, L., & van Wetten, S. (2022) · AEA Papers and Proceedings · doi:10.1257/pandp.20221107 · Cited by 13
Lifelong learning processes in professional development for online teachers during the Covid era
Reychav, I., Elyakim, N., & McHaney, R. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1041800 · Cited by 13
“, 2021) causing anxiety, concerns about instructional quality, and educators’ teaching success (Engzell et al., 2021; Paliwal and Singh, 2021).”
BackgroundNkambule, B. I. (2023) · Research in Social Sciences and Technology · doi:10.46303/ressat.2023.10 · Cited by 13
“RESSAT 2023, 8(2): 37-55 technology applications (Engzell et al., 2021).”
BackgroundInfluentialInequalities in late adolescents’ educational experiences and wellbeing during the Covid-19 pandemic
Anders, J., Macmillan, L., Sturgis, P., & Wyness, G. (2022) · Oxford Review of Education · doi:10.1080/03054985.2022.2124964 · Cited by 12
“These studies reveal stark differences in home learning experiences by family income (Andrew et al., 2020), higher learning loss for those from lower socio-economic backgrounds (Engzell et al., 2021), and larger impacts on mental health and wellbeing for women (Banks & Xu, 2020).”
Metadata
- Authors
- Engzell, P., Frey, A., & Verhagen, M. D.
- Affiliations
- Stockholm University (SE)
- University of Oxford (GB)
- Corresponding author
- Engzell, Per
- Frey, Arun
- Verhagen, Mark D.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-07
- Volume
- 118
- Issue
- 17
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8092566
- OpenAlex
- W3143968054
- Cited by
- 1,848
- References
- 75
- Citations by year
- 2026: 154 · 2025: 211 · 2024: 312 · 2023: 476 · 2022: 456 · 2021: 231
- FWCI
- 562.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- SocArXiv (OSF Preprints) (submittedVersion, cc-by-nc-nd)
- SocArXiv (OSF Preprints) (submittedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- Funders
- Leverhulme Trust
- University of Oxford
- Vetenskapsrådet (2016-07099)
- Forskningsrådet om Hälsa, Arbetsliv och Välfärd (2016-07099)
- Organisation de Coopération et de Développement Économiques
- Economic and Social Research Council
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Keywords
- learning loss, school closures, COVID-19 pandemic, lockdown, national examinations, academic performance, home background, parental education, home-based learning, educational inequality
- MeSH terms
- COVID-19, SARS-CoV-2, Child, Female, Humans, Learning, Male, Netherlands, Quarantine, Schools, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
