The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index; 15 of the 39 sources its claims cite were checked for support.
Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review
Open accessPublished version of a preprint
Abstract
The COVID-19 pandemic led to numerous governments deciding to close schools for several weeks in spring 2020. Empirical evidence on the impact of COVID-19-related school closures on academic achievement is only just emerging. The present work aimed to provide a first systematic overview of evidence-based studies on general and differential effects of COVID-19-related school closures in spring 2020 on student achievement in primary and secondary education. Results indicate a negative effect of school closures on student achievement, specifically in younger students and students from families with low socioeconomic status. Moreover, certain measures can be identified that might mitigate these negative effects. The findings are discussed in the context of their possible consequences for national educational policies when facing future school closures.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review synthesizes 11 studies on COVID-19-related school closures in spring 2020 and finds mostly negative effects on student achievement, with a median effect of −0.08 SD. Negative effects were especially pronounced for younger students and students from low socioeconomic status families, while some online-learning software studies reported positive effects.
Key findings
- Across included studies, effects ranged from −0.37 SD to +0.25 SD, with a median of −0.08 SD; median effects were −0.10 SD for mathematics and −0.09 SD for reading.
- Seven studies reported negative effects on mathematics, five on reading, and two on other subjects such as science.
- Differential effects: younger children were more negatively affected than older children (−0.37 SD vs. −0.10 SD; Tomasik et al., 2020), and low-SES students were more affected, with one study reporting +60% learning loss in low-SES students (Engzell et al., 2021).
- Three studies using online-learning software reported positive effects (+0.20 SD in Meeter, +0.25 SD for correct solutions to open questions in van der Velde et al., and improved mathematics performance in Spitzer and Musslick).
Methods
- Systematic review following PRISMA; searched Web of Science from March 1, 2020 to April 30, 2021, plus preprint servers PsyArXiv, EdArXiv, and SocArXiv, yielding 601 potentially relevant studies.
- Inclusion required a focus on COVID-19-related school closures, primary/secondary education, student achievement/test scores as dependent variable, actual data collected prior to and during/after closures, statistical analyses, and reported effect sizes; final inclusion was 9 studies plus 2 from backward reference searching (11 total).
- Risk of bias assessed with ROBINS-I; synthesis extracted country, closure duration, sample, subjects, statistical method, general effects, and differential effects, converting effects to SD where possible and calculating medians; meta-analytic models were not feasible due to the small number of studies.
Limitations
- The paper states that evidence is available for only a small number of countries, clustered around developed countries, with no studies from developing countries yet.
- The paper states that the original studies were not designed to identify reasons for the differential effects, so additional studies are required to explain them exhaustively.
- The paper states that the highest risk of bias was due to lack of inclusion of potential confounding variables (ROBINS-I Domain 1), and that the small number of studies prevented random- or mixed-effects meta-analytic models.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
7 Supported · 7 Partly supported · 7 Sources not found · 7 Own method · 7 Own interpretation · 5 Not checked yet · 4 No full text of the sources · 4 Own result · accuracy not yet measured
Every source the paper cites for this claim was checked and supports it.
In the paper
“Similarly, van der Velde et al. (2021) reported an increase in correct solutions on open questions within a French learning program.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Supportedcites van der Velde et al., 2021 source
The Trial-level performance section (§3.2.2) reports that accuracy on open-answer questions rose from 77.47% before lockdown to 83.64% during lockdown. It also explains that only multiple-choice performance was reported for English, so the open-answer result concerns the French course. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Finally, with the possibility to adapt the assignments in online programs individually to the students, low-performing children might have been addressed more thoroughly according to their needs (Spitzer and Musslick, 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Supportedcites Spitzer and Musslick, 2021 source
The Discussion (section 4; page not supplied) reports greater performance improvements among low-performing students and offers individualized problem-set assignments as one possible explanation. The citing sentence preserves that uncertainty with “might.” (LLM, conf=0.93)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Additionally, low-performing students might be less distracted in comparison to learning in a classroom setting (Spitzer and Musslick, 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Supportedcites Spitzer and Musslick, 2021 source
The Discussion proposes that students, especially low-performing students, may have been less distracted by other students, teachers, or stressful classroom settings while learning at home. The citing sentence preserves the paper’s tentative wording; the paper offers this as a possible explanation, not a measured finding. (LLM, conf=0.93)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Second, children from families with a low socioeconomic status (SES) were more affected than children from families with a high SES were (Maldonado and De Witte, 2020; Engzell et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Not matchedcites Maldonado and De Witte, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Engzell et al., 2021 source
Section 7.10 reports greater learning loss among students from less-educated homes. The Materials and Methods section identifies parental education as the paper’s measure of family disadvantage, supporting the citing sentence’s low-versus-high-SES paraphrase. Page numbers were not provided in our copy. (LLM, conf=0.90)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Finally, low-performing students benefited more from systematic online-learning methods (Clark et al., 2020; Spitzer and Musslick, 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Not matchedcites Clark et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Spitzer and Musslick, 2021 source
The Discussion reports that low-performing students showed greater performance improvements than high-performing students in the online mathematics-learning environment. Its conclusion specifies that the greatest improvements were among students who had performed below average in 2019. The finding concerns this software during the 2020 school shutdown, not a comparison with traditional instruction. (LLM, conf=0.89)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Meeter (2021) and Spitzer and Musslick (2021) showed students to improve their mathematics achievement when learning with an online-learning software during the COVID-related school closures.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Not matchedcites Meeter, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Spitzer and Musslick, 2021 source
The Abstract reports that German K-12 students’ performance on mathematical problem sets in online learning software increased during the 2020 school shutdown relative to the year before. The Discussion reiterates the improvement. This supports the part of the sentence attributed to Spitzer and Musslick; the verdict does not assess Meeter (2021). (LLM, conf=0.92)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“five studies on reading (Clark et al., 2020; Maldonado and De Witte, 2020; Tomasik et al., 2020; Engzell et al., 2021; Schult et al., 2021)”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“two studies on other subjects, such as science (Maldonado and De Witte, 2020; Engzell et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
plaindirectn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Seven studies reported a negative effect on mathematics (Clark et al., 2020; Kuhfeld et al., 2020b; Maldonado and De Witte, 2020; Tomasik et al., 2020; Depping et al., 2021; Engzell et al., 2021; Schult et al., 2021)”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“First, the main finding was that younger children were more negatively affected in their learning than older children were (-0.37 SD vs.−0.10 SD; Tomasik et al., 2020).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Not checked — no full text of the sourcecites Tomasik et al., 2020 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 15
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Moreover, students increased the time using the online-learning software at home, were less distracted or experienced less time pressure in a home-schooling rather than classroom setting, or were presented with individualized assignments within the online program (see also Meeter, 2021; Spitzer and Musslick, 2021; van der Velde et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Not matchedcites Meeter, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Spitzer and Musslick, 2021 sourceExperimental — accuracy not yet measured
The Introduction reports increased software use during school closures, but measures this as three times more students using it, not more time spent per student. The Discussion proposes reduced distraction at home and individualized problem sets as possible explanations, rather than established findings. It does not identify reduced time pressure as a factor. (LLM, conf=0.86)
- Partly supportedcites van der Velde et al., 2021 sourceExperimental — accuracy not yet measured
The Discussion reports increased use of the online retrieval-practice tool during lockdown. It says higher accuracy and response times on open-answer questions *perhaps* indicated less time pressure or better focus at home; it does not establish that students were less distracted. The Data Set section describes an adaptive tool, but does not say students were given individualized assignments. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Additionally, two studies found positive effects on student achievement in mathematics and reading (Gore et al., 2021), or in reading only (Depping et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Partly supportedcites Gore et al., 2021 sourceExperimental — accuracy not yet measured
The Results sections on student achievement and location report additional mathematics growth for Year 3 students in mid-ICSEA schools and additional reading growth in specific location subgroups. But neither Year 3 nor Year 4 showed an overall difference from the 2019 cohort, and low-ICSEA Year 3 students had less mathematics growth. The claim omits these subgroup limits and implies broadly positive effects in both subjects. (LLM, conf=0.89)
- Not checked — no full text of the sourcecites Depping et al., 2021 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In this context, one study reported an interaction between grade and SES, that is, for younger children from schools with low school-level SES, learning losses of 0.16 SD were found, while younger children from schools with medium school-level SES experienced learning gains of 0.15 SD (Gore et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Partly supportedcites Gore et al., 2021 sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The review is organized following the reporting guidelines of the PRISMA statement (Page et al., 2021)”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Introduction
What the sources say
- Supportedcites Page et al., 2021 source
The cited paper is the PRISMA statement, an updated reporting guideline for systematic reviews. Its Discussion describes the development of its reporting recommendations. The source therefore supports identifying PRISMA as the guideline the citing authors say they followed, though it cannot independently verify their compliance. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Moreover, Haeck and Lefebvre (2020) estimated that socioeconomic achievement gaps would increase by up to 30%.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Introduction
What the sources say
- Supportedcites Haeck and Lefebvre, 2020 source
The Concluding Remarks state that the authors’ back-of-the-envelope calculation found the socioeconomic-status score gap could increase by as much as 30 percent. No page number is provided in our copy. (LLM, conf=0.94)
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Third, low-performing students were more affected by COVID-19-related school closures in mathematics, while high-performing students were more affected by COVID-19-related school closures in reading (Schult et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Supportedcites Schult et al., 2021 source
The ‘Comparisons for High-, Average-, and Low-Achieving Groups (RQ2)’ section states that differences in reading comprehension were larger for high-achieving students, whereas differences in mathematics were larger for low-achieving students. This comparison concerns incoming fifth-graders’ 2020 scores against those of pre-pandemic cohorts. (LLM, conf=0.94)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On the other hand, reasons for finding larger learning losses in mathematics than in reading might be that students spent more time on reading during school closures and that supporting children in their reading skills might have been easier to realize for parents than supporting children in improving their competencies in mathematics (e.g., Depping et al., 2021; Schult et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Not checked — no full text of the sourcecites Depping et al., 2021 source
- Partly supportedcites Schult et al., 2021 sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
Every source the paper cites for this claim was checked and supports it.
In the paper
“The Cochrane Risk Assessment of the included studies was conducted independently by the first and second author using the “Risk Of Bias in Non-Randomized Studies of Interventions” tool (ROBINS-I; Sterne et al., 2016).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Method > Risk of Bias Assessment
What the sources say
- Supportedcites Sterne et al., 2016 source
The Abstract identifies ROBINS-I as a tool for evaluating risk of bias in non-randomised studies of interventions, and the Discussion describes its use in systematic reviews. The citation identifies the tool used; the statement that the citing paper’s first and second authors conducted the assessment independently describes their own procedure, not a finding attributed to Sterne et al. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even though the median for the effect on mathematics and reading is comparable when averaging above all studies (d = −0.10 SD and−0.09 SD for mathematics and reading, respectively)”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For instance, Woessmann (2020) estimated a negative effect of 0.10 SD on student achievement due to COVID-19-related school closures.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Introduction
What the sources say
- Not matchedcites Woessmann, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“As another explanation, many students might not speak the language in which they are tested in at home, hence, not benefitting much in their language skills during school closures (e.g., Maldonado and De Witte, 2020).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Not matchedcites Maldonado and De Witte, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Lastly, bias due to missing data, measurement of outcomes, or selection of reported results is unlikely, as most studies exhibited very small proportions of missing data (except Depping et al., 2021), and were highly transparent in their reporting of the results (except, partially, van der Velde et al., 2021, who only report significant mean differences between groups in sufficient detail).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Method > Risk of Bias Assessment
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Nevertheless, the proportion of students enrolled in primary or secondary education is considerably smaller in poorer countries (Ward, 2020).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
What the sources say
- Not checked — no full text of the sourcecites Ward, 2020 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To determine the degree of rater agreement on the selection of the studies, a randomly selected subset of 20 studies was evaluated by both the authors and the research assistants.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Method > Selection of Studies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The differential effect on low-performing and high-performing students may be due to high-performing students being capable of improving their performance regardless of the learning environment, while low-performing students specifically benefit from systematic online learning (Clark et al., 2020).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Not matchedcites Clark et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“The reported effects are comparable in size to findings of research on summer losses (d = −0.005 SD to −0.05 SD per week; see also Kuhfeld et al., 2020a) and comparable to Woessmann's initial estimate.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
What the sources say
- Not matchedcites Kuhfeld et al., 2020a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Woessmann, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
In the paper
“The reported effects are comparable in size to findings of research on summer losses (d = −0.005 SD to −0.05 SD per week; see also Kuhfeld et al., 2020a)”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
What the sources say
- Not matchedcites Kuhfeld et al., 2020a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It has to be noted that the number of countries for which evidence of these effects are available is still small, and clustered around developed countries. Especially studies from developing countries are not available yet.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to the cumulative nature of learning processes and student achievement, additional learning losses are likely.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
- Rests on an assumption: The first COVID-19-related school closures in spring 2020 were followed by similar measures in the fall and winter of 2020/2021.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Studies were required (1) to have collected actual data prior to and during/after COVID-19-related school closures, and (2) to have applied statistical analyses and to report an effect size.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Method > Selection of Studies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This indicates that most remote learning measures implemented during the first school closures in spring 2020 were not effective for student learning; there was no difference between them and the absence of systematic teaching during summer vacation.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The reasons for students from low SES families being more affected relate to access to remote learning, their learning behavior, and the support provided from families and schools.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“A learning loss of up to 38 points on the Programme for International Student Assessment (PISA1) scale is estimated, which corresponds to an effect size (Cohen's d) of 0.38 or 0.9 school years (Azevedo et al., 2020; Kuhfeld et al., 2020a; Wyse et al., 2020; Kaffenberger, 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Introduction
What the sources say
- Not matchedcites Azevedo et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Kuhfeld et al., 2020a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Wyse et al., 2020 source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Children from families with a low SES are less likely to have access to remote learning (UNESCO, 2021), are less often provided with active learning assistance from their schools (Tomasik et al., 2020), and spend less time on learning (Meeter, 2021) than children from families with a high SES.”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > Differential Effects on Groups of Students
What the sources say
- Not matchedcites UNESCO, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Tomasik et al., 2020 source
- Not matchedcites Meeter, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
In the paper
“This result might be accounted for by the achievement measurement being timed some months after school closures in both studies and the possibility of effective compensatory measures being implemented by teachers, schools, and local policy makers during this time to counteract learning losses, such as offering learning groups during summer vacation in parts of Germany (Depping et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Not checked — no full text of the sourcecites Depping et al., 2021 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“On the one hand, reasons for finding larger learning losses in reading than in mathematics might be that “mathematics is easier to teach in distance learning, as it is simple to provide exercises and tests digitally or as worksheets” (Maldonado and De Witte, 2020, p. 13).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
What the sources say
- Not matchedcites Maldonado and De Witte, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“We know, however, that the reduction in in-person learning was smaller for low-income countries than for medium-income countries (UNESCO, 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Discussion
What the sources say
- Not matchedcites UNESCO, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
The authors state this as an assumption.
In spring 2020, the ad hoc implementation of online teaching gave students, teachers, schools, and parents little time to prepare for or adapt to measures of remote learning.
“the assumption that, in spring 2020, the ad hoc implementation of online teaching gave students, teachers, schools, and parents little time to prepare for or adapt to measures of remote learning.”
The paper relies on this without support.
COVID-19 related school closures were not intended interventions.
“This is because the COVID-19 related school closures were not intended interventions.”
The paper relies on this without support.
The first COVID-19-related school closures in spring 2020 were followed by similar measures in the fall and winter of 2020/2021.
“The first COVID-19-related school closures in spring 2020 were followed by similar measures in the fall and winter of 2020/2021.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Nevertheless, there are several systematic reviews and meta-analyses, and these suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation—that is, at least one-third of a year’s worth of learning16–20.”
From COVID-19, school closures, and student learning outcomes. New global evidence from PISA (2025) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The results confirm the core conclusion of prior synthesizing research examining the first phase school closures.”
From The Impact of COVID‐19‐Related School Closures on Student Achievement—A Meta‐Analysis (2022) · cites this paper as Hammerstein et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Interestingly, our results are similar to what systematic reviews are finding learning16–20,44,45.”
From COVID-19, school closures, and student learning outcomes. New global evidence from PISA (2025) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Results > General Effects of COVID-19-Related School Closures on Student Achievement
“two studies on other subjects, such as science (Maldonado and De Witte, 2020; Engzell et al., 2021).”
§ Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review > Method > Synthesis
“| Engzell et al.(Netherlands) | 8 weeks | Elementary and Secondary | 350,000 | Reading, Mathematics | DiD Regression | −0.09 SD (reading)−0.14 SD (mathematics) | +60% learning loss in low-SES students |”
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal
Ndiaye, M., Lespez, L., Tribolo, C., Rassé, M., Hadjas, I., Davidoux, S., Huysecom, É., & Douze, K. (2024) · Monash University Research Portal (Monash University) · doi:10.3929/ethz-b-000667478 · Cited by 24,214
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2 · Cited by 5,177
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2022376118 · Cited by 1,848
Self-citationLearning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2020) · doi:10.31235/osf.io/ve4z7 · Cited by 1,739
Ranking the effectiveness of worldwide COVID-19 government interventions
Haug, N., Geyrhofer, L., Londei, A., Dervic, E., Desvars-Larrive, A., Loreto, V., Pinior, B., Thurner, S., & Klimek, P. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-01009-0 · Cited by 1,412
Xie, X., Xue, Q., Zhou, Y., Zhu, K., Liu, Q., Zhang, J., & Song, R. (2020) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2020.1619 · Cited by 1,001
Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement
Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020) · Educational Researcher · doi:10.3102/0013189x20965918 · Cited by 997
PISA 2018 Results (Volume II): Where All Students Can Succeed
OECD (2019) · PISA · doi:10.1787/b5fd1b8f-en · Cited by 807
Cited by
Sorted by citations
A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic
Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01506-4 · Cited by 568
Schult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2022) · School Effectiveness and School Improvement · doi:10.1080/09243453.2022.2061014 · Cited by 262
“Meta-analytic and systematic reviews, which included a preprint of the present study, estimate average learning losses between d = –0.09 (in reading; Hammerstein et al., 2021) and d = –0.17 (in mathematics; Zierer, 2021) during the first wave of the COVID-19 pandemic in the year 2020.”
BackgroundInfluentialThe impact of Covid-19 on student achievement: Evidence from a recent meta-analysis
Di Pietro, G. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100530 · Cited by 173
“(2023) and Hammerstein et al. (2021), the risk of bias in nonrandomized studies was assessed in 3811 studies using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al.”
The Impact of COVID‐19‐Related School Closures on Student Achievement—A Meta‐Analysis
König, C., & Frey, A. (2022) · Educational Measurement: Issues and Practice · doi:10.1111/emip.12495 · Cited by 147
Self-citationSchool Performance among Children and Adolescents during COVID-19 Pandemic: A Systematic Review
Panagouli, E. D., Stavridou, A., Savvidi, C., Kourti, A., Ψαλτοπούλου, Θ., Sergentanis, T. Ν., & Τσίτσικα, Ά. (2021) · Children · doi:10.3390/children8121134 · Cited by 148
“Although this is not the first systematic review that examined the performance of students during the COVID-19 pandemic [60], we researched a variety of factors that contributed to the evaluation of academic performance.”
BackgroundSchult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2021) · doi:10.31234/osf.io/pqtgf · Cited by 262
Pandemic Schooling Mode and Student Test Scores: Evidence from US States
Halloran, C., Jack, R., Okun, J., & Oster, E. (2021) · doi:10.3386/w29497 · Cited by 110
The legacy of Covid-19 in education
Werner, K., & Woessmann, L. (2023) · Economic Policy · doi:10.1093/epolic/eiad016 · Cited by 118
Students’ Perceptions of Online Learning during COVID-19 Pandemic: A Qualitative Approach
Curelaru, M., Curelaru, V., & Cristea, M. (2022) · Sustainability · doi:10.3390/su14138138 · Cited by 131
K−12 teachers' stress and burnout during the COVID-19 pandemic: A systematic review
Westphal, A., Kalinowski, E., Hoferichter, C. J., & Vock, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.920326 · Cited by 162
“, 2020) and achievement, especially for students from families with low socioeconomic statuses (Hammerstein et al., 2021), thus widening existing social disparities (e.”
BackgroundKuhfeld, M. R., Lewis, K., & Peltier, T. K. (2022) · Reading and Writing · doi:10.1007/s11145-022-10345-8 · Cited by 91
“Given the magnitude of the disruptions due to the COVID19 pandemic, it is not surprising that numerous studies have now documented impacts of COVID-19 on student test scores (Betthäuser et al., 2022; Hammerstein et al., 2021; Thorn & Vincent-Horn, 2021; West & Lake, 2021; Zierer, 2021).”
BackgroundMazrekaj, D., & Witte, K. D. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231181108 · Cited by 128
“Hammerstein et al. (2021) reviewed 11 observational studies (five cross-sectional and six longitudinal) between March 2020 and April 2021 in four European countries, China, Australia, and the United States (only one of which was judged to have a low risk of bias) and found that school closures…”
BackgroundMeeter, M. (2021) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2021.100163 · Cited by 90
COVID-19 Pandemic and Student Reading Achievement: Findings From a School Panel Study
Ludewig, U., Kleinkorres, R., Schaufelberger, R., Schlitter, T., Lorenz, R., König, C., Frey, A., & McElvany, N. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.876485 · Cited by 112
Self-citationThe effectiveness of a school-based, universal mental health programme in six European countries
Cefai, C., Camilleri, L., Bartolo, P. A., Grazzani, I., Cavioni, V., Conte, E., Ornaghi, V., Agliati, A., Gandellini, S., Vorkapić, S. T., et al. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.925614 · Cited by 99
“Furthermore, the PROMEHS curriculum has been implemented during the COVID pandemic when the schools were closed for several weeks or months (Hammerstein et al., 2021).”
MethodNo learning loss in Sweden during the pandemic: Evidence from primary school reading assessments
Hallin, A. E., Danielsson, H., Nordström, T., & Fälth, L. (2022) · International Journal of Educational Research · doi:10.1016/j.ijer.2022.102011 · Cited by 68
Fütterer, T., Hoch, E., Lachner, A., Scheiter, K., & Stürmer, K. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104788 · Cited by 86
“…in and how advanced digitization was in that country (Betth ¨ auser et al., 2023), or if the sample of a study had already worked with the learning program before the change to digital distance teaching, it was assumed that they were more familiar with the program (Hammerstein et al., 2021).”
InfluentialA global analysis of the effectiveness of policy responses to COVID-19
Agyapon-Ntra, K., & McSharry, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-31709-2 · Cited by 62
Association Between the COVID-19 Pandemic and Early Childhood Development
Sato, K., Fukai, T., Fujisawa, K. K., & Nakamuro, M. (2023) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.2096 · Cited by 68
Molnár, G., & Hermann, Z. (2022) · Learning and Instruction · doi:10.1016/j.learninstruc.2022.101706 · Cited by 73
No evidence of a major learning slide 14 months into the COVID-19 pandemic in Denmark
Birkelund, J. F., & Karlson, K. B. (2022) · European Societies · doi:10.1080/14616696.2022.2129085 · Cited by 70
“Although research shows evidence of a short-term learning loss (Hammerstein et al .”
BackgroundSchool closures during COVID-19: an overview of systematic reviews
Hume, S., Brown, S. R., & Mahtani, K. R. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112085 · Cited by 53
Rubiyatno, R., Perdana, R. P., Fallo, I. S., et al. (2023) · Pedagogy of Physical Culture and Sports · doi:10.15561/26649837.2023.0304 · Cited by 48
“Coronavirus Disease pandemic or commonly referred to as Covid- 19 has had a considerable impact on people’s lives throughout the world, such as closing office areas, places of worship [1], schools [2], universities and sports venues [3, 4, 5, 6].”
BackgroundSchult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.867316 · Cited by 63
“The corresponding learning loss during the first pandemic wave has been summarized by several systematic reviews and metaanalyses (De Witte and Smet, 2021; Donnelly and Patrinos, 2021; Hammerstein et al., 2021; Panagouli et al., 2021; Zierer, 2021).”
BackgroundLerkkanen, M., Pakarinen, E., Salminen, J., & Torppa, M. (2022) · Reading and Writing · doi:10.1007/s11145-022-10358-3 · Cited by 77
COVID-19, school closures, and student learning outcomes. New global evidence from PISA
Jakubowski, M., Gajderowicz, T., & Patrinos, H. A. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00297-3 · Cited by 65
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Online teaching, procrastination and student achievement
De Paola, M., Gioia, F., & Scoppa, V. (2023) · Economics of Education Review · doi:10.1016/j.econedurev.2023.102378 · Cited by 38
Early Childhood Learning Losses during COVID-19: Systematic Review
Uğraş, M., ZENGİN, E., Papadakis, S., & Kalogiannakis, M. (2023) · Sustainability · doi:10.3390/su15076199 · Cited by 57
What Kind of Learning Media do You Want? Need Analysis On Elementary School Online Learning
Utomo, G. M., Setiawan, B., Rachmadtullah, R., et al. (2021) · Jurnal Basicedu · doi:10.31004/basicedu.v5i5.1468 · Cited by 36
Covid-19 distance and online learning: a systematic literature review in pharmacy education
Muhaimin, M., Habibi, A., Riady, Y., Alqahtani, T. M., Chaerunisaa, A. Y., Wijaya, T. T., Milanda, T., Yusop, F. D., & Albelbisi, N. A. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04346-6 · Cited by 51
School closure policies and student reading achievement: evidence across countries
Kennedy, A. I., & Strietholt, R. (2023) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-023-09415-4 · Cited by 45
:null (2022) · Waxmann Verlag GmbH eBooks · doi:10.31244/9783830995449 · Cited by 35
Rethinking education in the context of climate change
Nusche, D., Rabella, M. F., & Lauterbach, S. (2024) · OECD education working papers · doi:10.1787/f14c8a81-en · Cited by 59
How Did COVID-19 Affect Education and What Can Be Learned Moving Forward?
Daumiller, M. H., Rinas, R., Schoon, I., & Lüftenegger, M. (2023) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000527 · Cited by 37
“Alarmingly, school closures due to COVID-19 specifically impacted younger children and families with low socio-economic status (see Hammerstein et al., 2021, regarding student achievement).”
BackgroundDevelopmental Milestone Attainment in US Children Before and During the COVID-19 Pandemic
Johnson, S. B., Kuehn, M., Lambert, J. O., Spin, J. P., Klein, L. M., Howard, B. J., STURNER, R. A., & Perrin, E. M. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2024.0683 · Cited by 28
The crisis you can’t plan for: K-12 leader responses and organisational preparedness during COVID-19
Voto, C. D., & Superfine, B. M. (2023) · School Leadership and Management · doi:10.1080/13632434.2023.2171003 · Cited by 28
“…most Kindergarten-12th-grade (K-12) schools have now resumed in-person instruction, the pandemic ’ s academic and emotional toll remains a lasting reminder (Hammerstein et al. 2021), especially for leaders who have had to navigate these uncertain times (De Voto et al. in press; DeMatthews 2021).”
BackgroundA Mental Health Pandemic? Assessing the Impact of COVID-19 on Young People’s Mental Health
Lloyd, K., Schubotz, D., Roche, R. R., Manzi, J., & McKnight, M. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20166550 · Cited by 31
“With the disruption of these routines caused by the series of lockdowns, the emotional distress expressed by the participants is a logical sequela of these events [65].”
BackgroundBaten, E., Vlaeminck, F., Muès, M., Valcke, M., Desoete, A., & Warreyn, P. (2022) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-021-05383-0 · Cited by 33
“Furthermore, a systematic review on the effect of COVID-19 related school closures showed negative effects on student achievement, especially for younger students and students from lower SES families (Hammerstein et al., 2021).”
BackgroundDatu, J. A. D., Buenconsejo, J. U., Shek, C. Y. C., Choy, Y. L. E., & Sou, K. L. E. (2023) · School Psychology International · doi:10.1177/01430343221147273 · Cited by 27
“The pandemic has also affected students’ academic achievement, particularly those from lower grade levels (i.e., primary school students) and families of low socio-economic status (Hammerstein et al., 2021).”
BackgroundThe Long-Term Effects of the COVID-19 Pandemic on Children’s Writing: a Follow-up Replication Study
Skar, G. B. U., Graham, S., & Huebner, A. R. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09729-1 · Cited by 32
“Most of the research examining the effects of COVID-19 on students’ academic outcomes has focused on mathematics and reading (see De Witte & Smet, 2021; Hammerstein et al., 2021; König & Frey, 2022; OECD, 2021).”
BackgroundFlegr, S., Kühn, J., & Scheiter, K. (2023) · Learning and Instruction · doi:10.1016/j.learninstruc.2023.101778 · Cited by 29
Wang, C., Yang, Y., Lu, X., Yang, L., Du, J., Zhang, Z., Lian, X., Cui, Y., Li, H., Zhang, W., et al. (2022) · Journal of Medical Virology · doi:10.1002/jmv.27803 · Cited by 18
A Systematic Review on Teachers’ Well-Being in the COVID-19 Era
Katsarou, E. C., Chatzipanagiotou, P., & Sougari, A.-M. (2023) · Education Sciences · doi:10.3390/educsci13090927 · Cited by 38
Gunčaga, J., Lopúchová, J., Ferdiánová, V., Žáček, M., & Ashimov, Y. (2022) · Education Sciences · doi:10.3390/educsci12070458 · Cited by 28
Effects of the COVID-19 pandemic on reading performance of second grade children in Germany
Förster, N., Forthmann, B., Back, M. D., & Souvignier, E. (2022) · Reading and Writing · doi:10.1007/s11145-022-10379-y · Cited by 37
“They also contradict the assumption and finding that particularly young students’ academic learning was impaired by the school closures (e.g., Georgiou, 2021; Hammerstein et al., 2021; Tomasik et al., 2021).”
BackgroundResultInfluentialCOVID-19 and inequalities in educational achievement in Italy
Borgonovi, F., & Ferrara, A. (2023) · Research in Social Stratification and Mobility · doi:10.1016/j.rssm.2023.100760 · Cited by 24
“We contribute to the emerging literature on the educational consequences of the pandemic [see (Thorn & Vincent-Lancrin, 2021) and (Hammerstein et al., 2021) for reviews and (Betthäuser et al.”
Spitzer, M., & Moeller, K. (2023) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2023.100203 · Cited by 28
(2024) · doi:10.2147/prbm.s447485 · Cited by 22
“12 Many educators and parents worry about K-12 students returning to school with lower academic performance when emergencies end.”
Was bedeutet Systemrelevanz in Zeiten der Pandemie?
Kaldewey, D. (2022) · Berliner Journal für Soziologie · doi:10.1007/s11609-022-00464-y · Cited by 18
“Distanzunterricht, so das Resultat der Metastudie von Hammerstein et al. (2021), erwies sich als in etwa so effektiv wie Sommerferien.”
BackgroundTapia‐Serrano, M. Á., Sánchez‐Oliva, D., Sevil‐Serrano, J., Marques, A., & Miguel, P. A. S. (2022) · Scientific Reports · doi:10.1038/s41598-022-21096-5 · Cited by 20
The existence of environmental education in the COVID-19 pandemic: A systematic literature review
Husamah, H., Suwono, H., Nur, H., Dharmawan, A., & Chang, C. (2023) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/13668 · Cited by 21
“at the primary and secondary school level as well as students at tertiary institutions were affected (Engzell et al., 2021; Hammerstein et al., 2021; Russell, 2022).”
BackgroundSARS-CoV-2 Circulation in the School Setting: A Systematic Review and Meta-Analysis
Caini, S., Martinoli, C., Vecchia, C. L., Raimondi, S., Bellerba, F., D’Ecclesiis, O., Sasso, C., Basso, A., Cammarata, G., & Gandini, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095384 · Cited by 19
Assessment of the Long-Term Mental Health Effects on Austrian Students after COVID-19 Restrictions
Kaltschik, S., Pieh, C., Dale, R., Probst, T., Pammer, B., & Humer, E. C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192013110 · Cited by 24
Money Matters: Time for Prevention and Early Intervention to Address Family Economic Circumstances
Axford, N., & Berry, V. (2023) · Journal of Prevention · doi:10.1007/s10935-022-00717-9 · Cited by 23
“, 2022), reduced student achievement (Hammerstein et al., 2021) and increased risk of child maltreatment (Marmor et al.”
Lalin, S. A. A., Ahmed, M. N. Q., & Haq, S. M. A. (2024) · Regional Science Policy & Practice · doi:10.1016/j.rspp.2024.100046 · Cited by 27
Tomaszewski, W., Zając, T., Rudling, E. S., Riele, K. T., McDaid, L. M., & Western, M. (2022) · Australian Journal of Social Issues · doi:10.1002/ajs4.219 · Cited by 29
Spitzer, M., Moeller, K., & Musslick, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0284868 · Cited by 18
“These closures required teachers to adopt different approaches to home schooling and distance learning —with varying degrees of success [2,3].”
BackgroundA meta-analysis of the impact of COVID-19 on student achievement
Storey, N., & Zhang, Q. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100624 · Cited by 18
Schooling upheaval during COVID-19: troubling consequences for students’ return to school
Fray, L., Jaremus, F., Gore, J. M., & Harris, J. (2022) · The Australian Educational Researcher · doi:10.1007/s13384-022-00572-x · Cited by 17
“Substantial research has been conducted on student learning, with most studies reporting negative effects, particularly for younger children and those from low socio-economic backgrounds (Hammerstein et al., 2021; Patrinos et al., 2022).”
Background“Let’s transform!”: A systematic literature review of science learning in COVID-19 pandemic era
Nurwidodo, N., Ibrohim, I., Sueb, S., & Husamah, H. (2023) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/12875 · Cited by 20
Koirala, A., Winkler, N., Quinn, H., Gardiner, E. R. K., Liu, B. C., Forbes, J., Sharpe, C., van Tussenbroek, T., Wood, N., & Macartney, K. K. (2022) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2023.100736 · Cited by 17
Quansah, F., Frimpong, J. B., Sambah, F., Oduro, P., Anin, S. K., Srem‐Sai, M., Hagan, J. E., & Schack, T. (2022) · Healthcare · doi:10.3390/healthcare10050920 · Cited by 17
Bertoletti, A., Biagi, F., Pietro, G. D., & Karpiński, Z. (2023) · Large-scale Assessments in Education · doi:10.1186/s40536-023-00154-y · Cited by 19
“Along with the access to the digital devices and the ability to use them efficiently, additional factors related to students’ sense of social isolation, psychological distress, emotional support, subjective well-being, and relationships with peers, teachers, or family, might have affected their academic performance an…”
BackgroundSpitzer, M., & Moeller, K. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100162 · Cited by 16
Bracco, J., Ciaschi, M., Gasparini, L., et al. (2024) · Review of Income and Wealth · doi:10.1111/roiw.12687 · Cited by 16
Oostdam, R., van Diepen, M., Zijlstra, B. J. H., & Fukkink, R. G. (2023) · European Journal of Psychology of Education · doi:10.1007/s10212-023-00679-4 · Cited by 15
“This finding is in line with previous studies, concerning the effects of the recent school lockdowns (see Engzell et al., 2020; Hammerstein et al., 2021).”
InfluentialTeachers’ well-being and their teaching quality during the COVID-19 pandemic: a retrospective study
Sacré, M., Ries, N., Wolf, K., & Kunter, M. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1136940 · Cited by 31
Branch‐Elliman, W., Elwy, A. R., & Chambers, D. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1207679 · Cited by 16
Peters, S. J., Langi, M., Kuhfeld, M., & Lewis, K. (2025) · Journal for the Education of the Gifted · doi:10.1177/01623532241301079 · Cited by 27
“Given the well-established effects of school closures on student learning (Goldhaber et al., 2022; Hammerstein et al., 2021) it is not surprising that student groups who were in virtual learning environments for longer would see more un fi nished learning.”
Welter, V. D. E., Becker, L. B., & Großschedl, J. (2022) · Education Sciences · doi:10.3390/educsci12050325 · Cited by 18
Miller, A., Fray, L., & Gore, J. M. (2023) · The Australian Educational Researcher · doi:10.1007/s13384-023-00614-y · Cited by 14
“Learning loss was reported, for instance, in nine of the 11 studies included in Hammerstein’s (2021) systematic review (effect size − 0.37 SD to − 0.03 SD ) (Hammerstein et al., 2021), and Patrinos et al. (2022) reported learning loss of 0.17 standard deviations on average from 36 robust studies.”
InfluentialLo, N., & to, B. K. H. (2023) · SN Computer Science · doi:10.1007/s42979-023-01779-0 · Cited by 16
Paczos, W., Sawulski, J., & Leśniewicz, F. (2023) · International Review of Economics & Finance · doi:10.1016/j.iref.2023.07.010 · Cited by 14
Ghanamah, R., Eghbaria-Ghanamah, H., & Adi‐Japha, E. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101898 · Cited by 18
Weber, C., Helm, C., & Kemethofer, D. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.737064 · Cited by 15
“However, although there is growing research confirming the expected learning losses and growth of disparities, the number of studies is still small and largely limited to English-speaking countries (see Helm et al., 2021b; Hammerstein et al., 2021).”
BackgroundResultInfluentialYuda, A. K., Resita, C., Sumarsono, R. N., Gani, R. A., Németh, Z., & Setiawan, E. (2022) · Physical Education Theory and Methodology · doi:10.17309/tmfv.2022.2.05 · Cited by 24
Assessing the Nexus Between Education, Economic Growth, and Innovation: An Empirical Analysis
Boţoroga, C.-A., Horobeț, A. L., Belaşcu, L., et al. (2022) · Studies in Business and Economics · doi:10.2478/sbe-2022-0043 · Cited by 14
Muneer, S., Santhosh, M., Parangusan, H., & Bhadra, J. (2026) · International Journal of Technology and Design Education · doi:10.1007/s10798-025-10005-2 · Cited by 14
Chaban, T., Rameeva, R., Denisov, I., Дмитриевна, К. Ю., & Zvyagintsev, R. (2022) · Voprosy Obrazovaniya/ Educational Studies Moscow · doi:10.17323/1814-9545-2022-1-160-188 · Cited by 13
Copeland, K. A., Porter, L., Gorecki, M. C., Reyner, A., White, C., & Kahn, R. S. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.6458 · Cited by 13
Maurer, J., Becker, A. B. C., Hilkenmeier, J., & Daseking, M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.733865 · Cited by 13
School Closure Policies and Student Reading Achievement: Evidence Across CountriesPreprint
Kennedy, A., & Strietholt, R. (2023) · doi:10.31219/osf.io/93rgz · Cited by 13
Estrela, M., Leitão, C., Silva, T. M., Figueiras, A., Roque, F., & Herdeiro, M. T. (2022) · Vaccines · doi:10.3390/vaccines10121984 · Cited by 10
Nutley, S. B., Burén, J., & Thorell, L. B. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20166583 · Cited by 13
“Considering that previous studies have demonstrated a significant decline in academic progression due to remote learning [51,52], it is possible that the increase in stress observed in our study is at least partially a result of the cumulative strain of struggling academically during remote learning.”
Tanri, A., Benny Aprial, M., Mashud, M., et al. (2023) · Retos · doi:10.47197/retos.v50.100587 · Cited by 12
Buda, A., & Czékmán, B. (2021) · Central European Journal of Educational Research · doi:10.37441/cejer/2021/3/3/10391 · Cited by 19
Kirsch, C., & Vaıoulı, P. (2023) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2023.100433 · Cited by 13
Social disparities in students’ intention to enter higher education during the COVID-19 pandemic
Stark, J., Daniel, A., & Twardawski, M. (2022) · PLoS ONE · doi:10.1371/journal.pone.0267978 · Cited by 15
“In line with recently published studies, students from non-academic households reported a lower academic performance than their peers from academic households [3, 25, 26].”
ResultPaola, M. D., Gioia, F., & Scoppa, V. (2022) · Social Science Research Network · doi:10.2139/ssrn.4114561 · Cited by 15
Brankston, G., Merkley, E., Loewen, P. J., Avery, B. P., Carson, C. A., Dougherty, B. P., Fisman, D. N., Tuite, A. R., Poljak, Z., & Greer, A. L. (2022) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2022.101993 · Cited by 11
“…2020; Ł aszewska et al., 2021) and may be due to a variety of factors including perceptions of negative impacts on learning (Engzell et al., 2021; Hammerstein et al., 2021), psychosocial impacts (Meherali et al., 2021; Pfefferbaum, 2021), and declines in physical well-being (Maximum City, 2020;…”
Kappus-Kron, H. R., Chatila, D., MacLachlan, A. M., Pulido, N., Yang, N., & Larsen, D. A. (2024) · PLOS Global Public Health · doi:10.1371/journal.pgph.0001803 · Cited by 10
Karadağ, E., & Sertel, G. (2025) · Educational Management Administration & Leadership · doi:10.1177/17411432251327645 · Cited by 30
“The relationship between leadership practices and student achievement during the pandemic appears to be complex and multifaceted, in fl uenced by factors such as subject area, grade level, and socioeconomic status (Asakawa et al., 2024; Hammerstein et al., 2021; Ludewig et al., 2022).”
Coles, L., Johnstone, M., Pattinson, C. L., Thorpe, K. J., Halen, O. V., Zheng, Z., Bayliss, O., Lakeman, N., Western, M., Povey, J., et al. (2022) · Australian Journal of Social Issues · doi:10.1002/ajs4.236 · Cited by 11
The ability of low- and High-SES schools to inhibit learning losses during the COVID-19 pandemic
Vit, E. (2022) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2022.100393 · Cited by 12
“closures harmed students’ achievement [30,31,32,33].”
Emergency remote assessment practices in higher education in sub-Saharan Africa during COVID-19
Baidoo-Anu, D., Asamoah, D., Quainoo, E. A., et al. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1221115 · Cited by 11
Sarker, M. F., & Ullah, M. S. (2023) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2023.100740 · Cited by 11
Theorising post-truth in the COVID era
Nally, D. (2022) · Journal of Educational Change · doi:10.1007/s10833-022-09457-3 · Cited by 11
COVID-19 Pandemic and Student Reading Achievement – Findings from a School Panel StudyPreprint
Ludewig, U., Kleinkorres, R., Schaufelberger, R., et al. (2022) · doi:10.31234/osf.io/hrzae · Cited by 11
Self-citationCampos, D., Fütterer, T., Gfrörer, T., Lavelle-Hill, R. E., Murayama, K., König, L., Hecht, M., Zitzmann, S., & Scherer, R. (2023) · doi:10.31234/osf.io/fpwc2 · Cited by 11
Policy responses to extreme heat and its impact on education: The Philippine experience
Preña, E. M., & Labayo, C. P. (2024) · Policy Futures in Education · doi:10.1177/14782103241288276 · Cited by 15
“…affect students from disadvantaged backgrounds, particularly those lacking access to essential learning tools such as home computers, thereby exacerbating educational inequalities (Engzell et al., 2021; Hammerstein et al., 2021; Kennedy and Strietholt, 2023; Maldonado and De Witte, 2022).”
Metadata
- Authors
- Hammerstein, S., König, C., Dreisörner, T., & Frey, A.
- Affiliations
- Goethe University Frankfurt (DE)
- Corresponding author
- Hammerstein, Svenja
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2021-09-16
- Volume
- 12
- Pages
- 746289-746289
- ISSN
- 1664-1078 (ISSN-L)
- PMC
- PMC8481663
- OpenAlex
- W3172046965
- Semantic Scholar
- 650bd6b2e0b8178e55d72195ab3ccad79308a63f
- Cited by
- 588
- References
- 38
- Citations by year
- 2026: 60 · 2025: 92 · 2024: 129 · 2023: 172 · 2022: 120 · 2021: 15
- FWCI
- 19.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Psychology (publishedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- OSF Preprints (OSF Preprints) (acceptedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) (submittedVersion, cc-by)
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Social Sciences › Education › School Choice and Performance
- Keywords
- COVID-19 pandemic, school closures, academic performance, primary education, secondary education, socioeconomic status, low-income students, younger students, education policy, mitigation measures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
