Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review
Open accessPublished version of a preprint
Abstract
The COVID-19 pandemic led to numerous governments deciding to close schools for several weeks in spring 2020. Empirical evidence on the impact of COVID-19-related school closures on academic achievement is only just emerging. The present work aimed to provide a first systematic overview of evidence-based studies on general and differential effects of COVID-19-related school closures in spring 2020 on student achievement in primary and secondary education. Results indicate a negative effect of school closures on student achievement, specifically in younger students and students from families with low socioeconomic status. Moreover, certain measures can be identified that might mitigate these negative effects. The findings are discussed in the context of their possible consequences for national educational policies when facing future school closures.
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review synthesizes 11 studies on COVID-19-related school closures in spring 2020 and finds mostly negative effects on student achievement, with a median effect of −0.08 SD. Negative effects were especially pronounced for younger students and students from low socioeconomic status families, while some online-learning software studies reported positive effects.
Key findings
- Across included studies, effects ranged from −0.37 SD to +0.25 SD, with a median of −0.08 SD; median effects were −0.10 SD for mathematics and −0.09 SD for reading.
- Seven studies reported negative effects on mathematics, five on reading, and two on other subjects such as science.
- Differential effects: younger children were more negatively affected than older children (−0.37 SD vs. −0.10 SD; Tomasik et al., 2020), and low-SES students were more affected, with one study reporting +60% learning loss in low-SES students (Engzell et al., 2021).
- Three studies using online-learning software reported positive effects (+0.20 SD in Meeter, +0.25 SD for correct solutions to open questions in van der Velde et al., and improved mathematics performance in Spitzer and Musslick).
Methods
- Systematic review following PRISMA; searched Web of Science from March 1, 2020 to April 30, 2021, plus preprint servers PsyArXiv, EdArXiv, and SocArXiv, yielding 601 potentially relevant studies.
- Inclusion required a focus on COVID-19-related school closures, primary/secondary education, student achievement/test scores as dependent variable, actual data collected prior to and during/after closures, statistical analyses, and reported effect sizes; final inclusion was 9 studies plus 2 from backward reference searching (11 total).
- Risk of bias assessed with ROBINS-I; synthesis extracted country, closure duration, sample, subjects, statistical method, general effects, and differential effects, converting effects to SD where possible and calculating medians; meta-analytic models were not feasible due to the small number of studies.
Limitations
- The paper states that evidence is available for only a small number of countries, clustered around developed countries, with no studies from developing countries yet.
- The paper states that the original studies were not designed to identify reasons for the differential effects, so additional studies are required to explain them exhaustively.
- The paper states that the highest risk of bias was due to lack of inclusion of potential confounding variables (ROBINS-I Domain 1), and that the small number of studies prevented random- or mixed-effects meta-analytic models.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
15 of 39 cited sources checked · 10 supported · 0 flagged · accuracy not yet measured
- Findingcitation
van der Velde et al. (2021) reported an increase in correct solutions on open questions within a French learning program.
“Similarly, van der Velde et al. (2021) reported an increase in correct solutions on open questions within a French learning program.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites van der Velde et al., 2021 source
- Interpretationcitation
With the possibility to adapt assignments in online programs individually, low-performing children might have been addressed more thoroughly according to their needs.
“Finally, with the possibility to adapt the assignments in online programs individually to the students, low-performing children might have been addressed more thoroughly according to their needs (Spitzer and Musslick, 2021).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Spitzer and Musslick, 2021 source
- Interpretationcitation
Low-performing students might be less distracted in comparison to learning in a classroom setting.
“Additionally, low-performing students might be less distracted in comparison to learning in a classroom setting (Spitzer and Musslick, 2021).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Spitzer and Musslick, 2021 source
- Findingcitation
Children from families with low SES were more affected than children from families with high SES (Maldonado and De Witte, 2020; Engzell et al., 2021).
“Second, children from families with a low socioeconomic status (SES) were more affected than children from families with a high SES were (Maldonado and De Witte, 2020; Engzell et al., 2021).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Maldonado and De Witte, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Engzell et al., 2021 source
- Findingcitation
Low-performing students benefited more from systematic online-learning methods (Clark et al., 2020; Spitzer and Musslick, 2021).
“Finally, low-performing students benefited more from systematic online-learning methods (Clark et al., 2020; Spitzer and Musslick, 2021).”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites Clark et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Spitzer and Musslick, 2021 source
- Findingcitation
Meeter (2021) and Spitzer and Musslick (2021) showed students improved their mathematics achievement when learning with online-learning software during COVID-related school closures.
“Meeter (2021) and Spitzer and Musslick (2021) showed students to improve their mathematics achievement when learning with an online-learning software during the COVID-related school closures.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Meeter, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Spitzer and Musslick, 2021 source
- Findingown data
Five studies reported a negative effect on reading.
“five studies on reading (Clark et al., 2020; Maldonado and De Witte, 2020; Tomasik et al., 2020; Engzell et al., 2021; Schult et al., 2021)”
Quote in the paperNot a substantial claim - Findingcitation
Two studies reported a negative effect on other subjects, such as science.
“two studies on other subjects, such as science (Maldonado and De Witte, 2020; Engzell et al., 2021).”
Quote in the paperNot a substantial claim - Findingcitation
Seven studies reported a negative effect on mathematics.
“Seven studies reported a negative effect on mathematics (Clark et al., 2020; Kuhfeld et al., 2020b; Maldonado and De Witte, 2020; Tomasik et al., 2020; Depping et al., 2021; Engzell et al., 2021; Schult et al., 2021)”
Quote in the paperNot a substantial claim - Findingcitation
Younger children were more negatively affected in their learning than older children were (−0.37 SD vs. −0.10 SD; Tomasik et al., 2020).
“First, the main finding was that younger children were more negatively affected in their learning than older children were (-0.37 SD vs.−0.10 SD; Tomasik et al., 2020).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Tomasik et al., 2020 source
- Findingown data
COVID-19-related school closures in spring 2020 had a negative effect on student achievement in primary and secondary education.
“Results indicate a negative effect of school closures on student achievement, specifically in younger students and students from families with low socioeconomic status.”
- Also stated in: § Discussion · boosted
“In sum, there is clear evidence for a negative effect of COVID-19-related school closures on student achievement.”
- Also stated in: § General Effects of COVID-19-Related School Closures on Student Achievement · plain
“Most studies found negative effects of COVID-19 related school closures on student achievement.”
Quote in the paperSubstantial claim - Also stated in: § Discussion · boosted
- Findingcitation
Three studies reported positive effects of COVID-19-related school closures on student achievement.
“Three studies reported positive effects of COVID-19 related school closures on student achievement.”
Quote in the paperNot a substantial claim - Findingown data
The general effects of COVID-19-related school closures on student achievement ranged from −0.37 SD to +0.25 SD, with a median of −0.08 SD.
“The studies on the effect of COVID-19-related school closures on student achievement selected for our review reported mixed findings, with effects ranging from−0.37 SD to +0.25 SD (Mdn = −0.08 SD).”
Quote in the paperSubstantial claim - Study design and conductown data
Most included studies included at least a few relevant controls.
“Most studies, however, included at least a few relevant controls.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The positive effects may be explained by students being familiar with online-learning software prior to school closures, so they did not have to adapt to a new learning environment.
“Thus, the positive effects may be explained by the students under investigation being familiar working with the corresponding online-learning software prior to school closures. Hence, they did not have to adapt to a new learning environment when in-person teaching was interrupted due to COVID-19.”
Quote in the paperSubstantial claim - Interpretationcitation
Systematic online material and software have been found to compensate for learning losses, specifically in high-risk children.
“In this regard, systematic online material and software have been found to compensate for learning losses, specifically in high-risk children.”
Quote in the paperSubstantial claim - Interpretationreasoning
The positive effects of school closures on performance in online-learning programs may have occurred due to increased use of software during school closures and students' familiarity with online-learning programs.
“Hence, the positive effects of COVID-19-related school closures on performance in such online-learning programs may have occurred due to the increased use of software during school closures and the fact that students from these studies were familiar working with online-learning programs, hence, did not have to adapt to a new learning environment during COVID-19-related school closures.”
Quote in the paperSubstantial claim - Study design and conductown data
The highest risk of bias in included studies is due to the lack of inclusion of potential confounding variables (Domain 1).
“Taken together, the highest risk of bias is due to the lack of inclusion of potential confounding variables (Domain 1).”
Quote in the paperNot a substantial claim - Study design and conductreasoning
Bias due to selection of participants was unlikely as the groups were formed naturally.
“Bias due to selection of participants was unlikely as the groups were formed naturally.”
Quote in the paperNot a substantial claim - Study design and conductreasoning
Random- or mixed-effects meta-analytic models were not feasible due to the relatively small number of studies.
“In light of the relatively small number of studies, random- or even mixed-effects meta-analytic models were not feasible.”
Quote in the paperSubstantial claim - Premisecitation
Parents with high SES are more likely to provide greater psychological support for their children.
“Moreover, parents with a high SES are more likely to provide greater psychological support for their children (OECD, 2019), which seems to be specifically relevant in a situation such as the COVID-19 pandemic.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites OECD, 2019 source
- May combine two claims. 17
- Classified as probably not a substantial claim. 15
May combine two claims.
InterpretationcitationStudents increased time using online-learning software at home, were less distracted or experienced less time pressure in home-schooling, or were presented with individualized assignments.
“Moreover, students increased the time using the online-learning software at home, were less distracted or experienced less time pressure in a home-schooling rather than classroom setting, or were presented with individualized assignments within the online program (see also Meeter, 2021; Spitzer and Musslick, 2021; van der Velde et al., 2021).”
Quote in the paperSubstantial claim2 of 3 sources checked- cites Meeter, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Spitzer and Musslick, 2021 source
The Introduction reports increased software use during school closures, but measures this as three times more students using it, not more time spent per student. The Discussion proposes reduced distraction at home and individualized problem sets as possible explanations, rather than established findings. It does not identify reduced time pressure as a factor. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites van der Velde et al., 2021 source
The Discussion reports increased use of the online retrieval-practice tool during lockdown. It says higher accuracy and response times on open-answer questions *perhaps* indicated less time pressure or better focus at home; it does not establish that students were less distracted. The Data Set section describes an adaptive tool, but does not say students were given individualized assignments. (LLM, conf=0.86)
Experimental — accuracy not yet measured
May combine two claims.
FindingcitationTwo studies found positive effects on student achievement in mathematics and reading (Gore et al., 2021), or in reading only (Depping et al., 2021).
“Additionally, two studies found positive effects on student achievement in mathematics and reading (Gore et al., 2021), or in reading only (Depping et al., 2021).”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites Gore et al., 2021 source
The Results sections on student achievement and location report additional mathematics growth for Year 3 students in mid-ICSEA schools and additional reading growth in specific location subgroups. But neither Year 3 nor Year 4 showed an overall difference from the 2019 cohort, and low-ICSEA Year 3 students had less mathematics growth. The claim omits these subgroup limits and implies broadly positive effects in both subjects. (LLM, conf=0.89)
Experimental — accuracy not yet measured - cites Depping et al., 2021 source
May combine two claims.
FindingcitationOne study reported an interaction between grade and SES: for younger children from schools with low school-level SES, learning losses of 0.16 SD were found, while younger children from schools with medium school-level SES experienced learning gains of 0.15 SD (Gore et al., 2021).
“In this context, one study reported an interaction between grade and SES, that is, for younger children from schools with low school-level SES, learning losses of 0.16 SD were found, while younger children from schools with medium school-level SES experienced learning gains of 0.15 SD (Gore et al., 2021).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Gore et al., 2021 source
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe review was organized following the PRISMA reporting guidelines.
“The review is organized following the reporting guidelines of the PRISMA statement (Page et al., 2021)”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Page et al., 2021 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationHaeck and Lefebvre (2020) estimated that socioeconomic achievement gaps would increase by up to 30%.
“Moreover, Haeck and Lefebvre (2020) estimated that socioeconomic achievement gaps would increase by up to 30%.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Haeck and Lefebvre, 2020 source
May combine two claims.
FindingcitationLow-performing students were more affected by COVID-19-related school closures in mathematics, while high-performing students were more affected in reading (Schult et al., 2021).
“Third, low-performing students were more affected by COVID-19-related school closures in mathematics, while high-performing students were more affected by COVID-19-related school closures in reading (Schult et al., 2021).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Schult et al., 2021 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationReasons for finding larger learning losses in mathematics than in reading might be that students spent more time on reading during school closures and that supporting children in reading skills might have been easier for parents than supporting mathematics.
“On the other hand, reasons for finding larger learning losses in mathematics than in reading might be that students spent more time on reading during school closures and that supporting children in their reading skills might have been easier to realize for parents than supporting children in improving their competencies in mathematics (e.g., Depping et al., 2021; Schult et al., 2021).”
Quote in the paperNot a substantial claim1 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe Cochrane Risk Assessment was conducted independently by the first and second author using ROBINS-I.
“The Cochrane Risk Assessment of the included studies was conducted independently by the first and second author using the “Risk Of Bias in Non-Randomized Studies of Interventions” tool (ROBINS-I; Sterne et al., 2016).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Sterne et al., 2016 source
May combine two claims.
Findingown dataThe median effect on mathematics was d = −0.10 SD and on reading was −0.09 SD.
“Even though the median for the effect on mathematics and reading is comparable when averaging above all studies (d = −0.10 SD and−0.09 SD for mathematics and reading, respectively)”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationWoessmann (2020) estimated a negative effect of 0.10 SD on student achievement due to COVID-19-related school closures.
“For instance, Woessmann (2020) estimated a negative effect of 0.10 SD on student achievement due to COVID-19-related school closures.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Woessmann, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationMany students might not speak the language in which they are tested at home, hence not benefitting much in their language skills during school closures.
“As another explanation, many students might not speak the language in which they are tested in at home, hence, not benefitting much in their language skills during school closures (e.g., Maldonado and De Witte, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Maldonado and De Witte, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Study design and conductown dataBias due to missing data, measurement of outcomes, or selection of reported results is unlikely, as most studies exhibited very small proportions of missing data (except Depping et al., 2021), and were highly transparent in their reporting of the results (except, partially, van der Velde et al., 2021).
“Lastly, bias due to missing data, measurement of outcomes, or selection of reported results is unlikely, as most studies exhibited very small proportions of missing data (except Depping et al., 2021), and were highly transparent in their reporting of the results (except, partially, van der Velde et al., 2021, who only report significant mean differences between groups in sufficient detail).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe proportion of students enrolled in primary or secondary education is considerably smaller in poorer countries.
“Nevertheless, the proportion of students enrolled in primary or secondary education is considerably smaller in poorer countries (Ward, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Ward, 2020 source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataA randomly selected subset of 20 studies was evaluated by both the authors and the research assistants to determine rater agreement.
“To determine the degree of rater agreement on the selection of the studies, a randomly selected subset of 20 studies was evaluated by both the authors and the research assistants.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationThe differential effect on low-performing and high-performing students may be due to high-performing students being capable of improving their performance regardless of the learning environment, while low-performing students specifically benefit from systematic online learning.
“The differential effect on low-performing and high-performing students may be due to high-performing students being capable of improving their performance regardless of the learning environment, while low-performing students specifically benefit from systematic online learning (Clark et al., 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Clark et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationcitationThe reported effects are comparable in size to findings of research on summer losses (d = −0.005 SD to −0.05 SD per week) and comparable to Woessmann's initial estimate.
“The reported effects are comparable in size to findings of research on summer losses (d = −0.005 SD to −0.05 SD per week; see also Kuhfeld et al., 2020a) and comparable to Woessmann's initial estimate.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Kuhfeld et al., 2020a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Woessmann, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationSummer losses are d = −0.005 SD to −0.05 SD per week.
“The reported effects are comparable in size to findings of research on summer losses (d = −0.005 SD to −0.05 SD per week; see also Kuhfeld et al., 2020a)”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kuhfeld et al., 2020a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationnone givenThe number of countries for which evidence of these effects is available is still small and clustered around developed countries; studies from developing countries are not available yet.
“It has to be noted that the number of countries for which evidence of these effects are available is still small, and clustered around developed countries. Especially studies from developing countries are not available yet.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningAdditional learning losses are likely due to the cumulative nature of learning processes and student achievement.
“Due to the cumulative nature of learning processes and student achievement, additional learning losses are likely.”
Quote in the paperNot a substantial claim- Rests on an assumption: The first COVID-19-related school closures in spring 2020 were followed by similar measures in the fall and winter of 2020/2021.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningStudies were required to have collected actual data prior to and during/after COVID-19-related school closures and to have applied statistical analyses and report an effect size.
“Studies were required (1) to have collected actual data prior to and during/after COVID-19-related school closures, and (2) to have applied statistical analyses and to report an effect size.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningMost remote learning measures implemented during the first school closures in spring 2020 were not effective for student learning; there was no difference between them and the absence of systematic teaching during summer vacation.
“This indicates that most remote learning measures implemented during the first school closures in spring 2020 were not effective for student learning; there was no difference between them and the absence of systematic teaching during summer vacation.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationcitationThe reasons for students from low SES families being more affected relate to access to remote learning, their learning behavior, and the support provided from families and schools.
“The reasons for students from low SES families being more affected relate to access to remote learning, their learning behavior, and the support provided from families and schools.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationA learning loss of up to 38 points on the PISA scale is estimated, corresponding to an effect size of 0.38 or 0.9 school years.
“A learning loss of up to 38 points on the Programme for International Student Assessment (PISA1) scale is estimated, which corresponds to an effect size (Cohen's d) of 0.38 or 0.9 school years (Azevedo et al., 2020; Kuhfeld et al., 2020a; Wyse et al., 2020; Kaffenberger, 2021).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Azevedo et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Kuhfeld et al., 2020a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Wyse et al., 2020 source
May combine two claims.
PremisecitationChildren from families with low SES are less likely to have access to remote learning, are less often provided with active learning assistance from their schools, and spend less time on learning than children from families with high SES.
“Children from families with a low SES are less likely to have access to remote learning (UNESCO, 2021), are less often provided with active learning assistance from their schools (Tomasik et al., 2020), and spend less time on learning (Meeter, 2021) than children from families with a high SES.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites UNESCO, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Tomasik et al., 2020 source
- cites Meeter, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationcitationThe positive effects on mathematics and reading in two studies might be accounted for by achievement measurement being timed some months after school closures and effective compensatory measures being implemented.
“This result might be accounted for by the achievement measurement being timed some months after school closures in both studies and the possibility of effective compensatory measures being implemented by teachers, schools, and local policy makers during this time to counteract learning losses, such as offering learning groups during summer vacation in parts of Germany (Depping et al., 2021).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Depping et al., 2021 source
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationcitationReasons for finding larger learning losses in reading than in mathematics might be that mathematics is easier to teach in distance learning, as it is simple to provide exercises and tests digitally or as worksheets.
“On the one hand, reasons for finding larger learning losses in reading than in mathematics might be that “mathematics is easier to teach in distance learning, as it is simple to provide exercises and tests digitally or as worksheets” (Maldonado and De Witte, 2020, p. 13).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Maldonado and De Witte, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe reduction in in-person learning was smaller for low-income countries than for medium-income countries.
“We know, however, that the reduction in in-person learning was smaller for low-income countries than for medium-income countries (UNESCO, 2021).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites UNESCO, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
The authors state this as an assumption.
In spring 2020, the ad hoc implementation of online teaching gave students, teachers, schools, and parents little time to prepare for or adapt to measures of remote learning.
“the assumption that, in spring 2020, the ad hoc implementation of online teaching gave students, teachers, schools, and parents little time to prepare for or adapt to measures of remote learning.”
The paper relies on this without support.
COVID-19 related school closures were not intended interventions.
“This is because the COVID-19 related school closures were not intended interventions.”
The paper relies on this without support.
The first COVID-19-related school closures in spring 2020 were followed by similar measures in the fall and winter of 2020/2021.
“The first COVID-19-related school closures in spring 2020 were followed by similar measures in the fall and winter of 2020/2021.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Systematic reviews and meta-analyses suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation, at least one-third of a year's worth of learning.
“Nevertheless, there are several systematic reviews and meta-analyses, and these suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation—that is, at least one-third of a year’s worth of learning16–20.”
From COVID-19, school closures, and student learning outcomes. New global evidence from PISA (2025) · cites this paper as [18]
The Results (General Effects; page not provided) report that most included studies found learning losses after COVID-19 school closures. But the review reports a median effect of −0.08 SD, not an average of −0.17 SD, and does not equate its findings to at least one-third of a year's learning. The Method (Synthesis; page not provided) also says a meta-analysis was not feasible. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe results confirm the core conclusion of prior synthesizing research examining the first phase of school closures.
“The results confirm the core conclusion of prior synthesizing research examining the first phase school closures.”
From The Impact of COVID‐19‐Related School Closures on Student Achievement—A Meta‐Analysis (2022) · cites this paper as Hammerstein et al., 2021
The study's results are similar to what systematic reviews are finding.
“Interestingly, our results are similar to what systematic reviews are finding learning16–20,44,45.”
From COVID-19, school closures, and student learning outcomes. New global evidence from PISA (2025) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Engzell study credited with a result in a subject not listed in Table 1
MinorThe results section counts Engzell et al. among studies reporting a negative effect in other subjects, such as science, but Table 1 lists only reading and mathematics for that study.
“two studies on other subjects, such as science (Maldonado and De Witte, 2020; Engzell et al., 2021).”
“| Engzell et al.(Netherlands) | 8 weeks | Elementary and Secondary | 350,000 | Reading, Mathematics | DiD Regression | −0.09 SD (reading)−0.14 SD (mathematics) | +60% learning loss in low-SES students |”
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283 · see citing works
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2 · Cited by 5,177 · see citing works
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2022376118 · Cited by 1,848 · see citing works
Self-citationLearning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2020) · doi:10.31235/osf.io/ve4z7 · Cited by 1,739 · see citing works
Ranking the effectiveness of worldwide COVID-19 government interventions
Haug, N., Geyrhofer, L., Londei, A., Dervic, E., Desvars-Larrive, A., Loreto, V., Pinior, B., Thurner, S., & Klimek, P. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-01009-0 · Cited by 1,412 · see citing works
Xie, X., Xue, Q., Zhou, Y., Zhu, K., Liu, Q., Zhang, J., & Song, R. (2020) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2020.1619 · Cited by 1,001 · see citing works
Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement
Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020) · Educational Researcher · doi:10.3102/0013189x20965918 · Cited by 997 · see citing works
PISA 2018 Results (Volume II): Where All Students Can Succeed
OECD (2019) · PISA · doi:10.1787/b5fd1b8f-en · Cited by 807 · see citing works
The effect of school closures on standardised student test outcomes
Maldonado, J. E., & De Witte, K. (2022) · British Educational Research Journal · doi:10.1002/berj.3754 · Cited by 499 · see citing works
Gadermann, A. C., Thomson, K. C., Richardson, C. G., Gagné, M., McAuliffe, C., Hirani, S., & Jenkins, E. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-042871 · Cited by 479 · see citing works
Goldemberg, D., Azevedo, J. P., Iqbal, S. A., Hasan, A., & Geven, K. (2020) · doi:10.1596/1813-9450-9284 · Cited by 476 · see citing works
Inequalities in Children's Experiences of Home Learning during the COVID‐19 Lockdown in England*
Andrew, A., Cattan, S., Dias, M. C., Farquharson, C., Kraftman, L., Krutikova, S., Phimister, A., & Sevilla, A. (2020) · Fiscal Studies · doi:10.1111/1475-5890.12240 · Cited by 451 · see citing works
Tomasik, M. J., Helbling, L. A., & Moser, U. (2021) · International Journal of Psychology · doi:10.1002/ijop.12728 · Cited by 378 · see citing works
Kaffenberger, M. (2021) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2020.102326 · Cited by 308 · see citing works
A Qualitative Study of Child and Adolescent Mental Health during the COVID-19 Pandemic in Ireland
O’Sullivan, K., Clark, S., McGrane, A., Rock, N., Burke, L., Boyle, N., Joksimovic, N., & Marshall, K. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18031062 · Cited by 283 · see citing works
Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic
Clark, A. E., Nong, H., Zhu, H., & Zhu, R. (2021) · China Economic Review · doi:10.1016/j.chieco.2021.101629 · Cited by 271 · see citing works
Schult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2021) · doi:10.31234/osf.io/pqtgf · Cited by 262 · see citing works
Schult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2022) · School Effectiveness and School Improvement · doi:10.1080/09243453.2022.2061014 · Cited by 262 · see citing works
COVID-19 and educational inequality: How school closures affect low- and high-achieving students
Grewenig, E., Lergetporer, P., Werner, K., Woessmann, L., & Zierow, L. (2021) · European Economic Review · doi:10.1016/j.euroecorev.2021.103920 · Cited by 260 · see citing works
Zaccoletti, S., Camacho, A., Correia, N., Aguiar, C., Mason, L., Alves, R. A., & Daniel, J. R. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.592670 · Cited by 209 · see citing works
The impact of COVID-19 on student learning in New South Wales primary schools: an empirical study
Gore, J., Fray, L., Miller, A., Harris, J., & Taggart, W. (2021) · The Australian Educational Researcher · doi:10.1007/s13384-021-00436-w · Cited by 201 · see citing works
Spitzer, M. W. H., & Musslick, S. (2021) · PLOS ONE · doi:10.1371/journal.pone.0255629 · Cited by 138 · see citing works
The potential impact of COVID-19 on student learning and how schools can respond
Wyse, A. E., Stickney, E. M., Butz, D., Beckler, A., & Close, C. N. (2020) · Educational Measurement Issues and Practice · doi:10.1111/emip.12357 · Cited by 115 · see citing works
Panoramic view of digital education in COVID‐19: A new explored avenue
Maity, S., Sahu, T. N., & Sen, N. (2020) · Review of Education · doi:10.1002/rev3.3250 · Cited by 103 · see citing works
Pandemic school closures may increase inequality in test scores
Haeck, C., & Lefebvre, P. (2020) · Canadian Public Policy · doi:10.3138/cpp.2020-055 · Cited by 95 · see citing works
Azevedo, J. P., Hasan, A., Goldemberg, D., et al. (2021) · The World Bank Research Observer · doi:10.1093/wbro/lkab003 · Cited by 86 · see citing works
Meeter, M. (2021) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2021.100163 · Cited by 85 · see citing works
Smith, J., Guimond, F., Bergeron, J., St‐Amand, J., Fitzpatrick, C., & Gagnon, M. (2021) · Education Sciences · doi:10.3390/educsci11010030 · Cited by 81 · see citing works
Learning inequalities during the Covid-19 pandemic: how families cope with home-schooling
Pensiero, N., Kelly, A. E., & Bokhove, C. (2020) · ePrints Soton (University of Southampton) · doi:10.5258/soton/p0025 · Cited by 73 · see citing works
van der Velde, M., Sense, F., Spijkers, R., Meeter, M., & van Rijn, H. (2021) · doi:10.31234/osf.io/fr2v8 · Cited by 57 · see citing works
Kompetenzstände Hamburger Schüler*innen vor und während der Corona-Pandemie
Depping, D., Lücken, M., Musekamp, F., & Thonke, F. (2021) · Schule während der Corona-Pandemie. Neue Ergebnisse und Überblick über ein dynamisches Forschungsfeld · doi:10.31244/9783830993315.03 · Cited by 46 · see citing works
Blume, F., Schmidt, A., Kramer, A. C., Schmiedek, F., & Neubauer, A. B. (2021) · Zeitschrift für Erziehungswissenschaft · doi:10.1007/s11618-021-01011-w · Cited by 44 · see citing works
Impact of the COVID‐19 pandemic on children with and without risk for neurodevelopmental impairments
Ehrler, M., Werninger, I., Schnider, B., Eichelberger, D. A., Naef, N., Disselhoff, V., Kretschmar, O., Hagmann, C. F., Latal, B., & Wehrle, F. M. (2021) · Acta Paediatrica · doi:10.1111/apa.15775 · Cited by 42 · see citing works
COVID-19 and Educational Inequality: How School Closures Affect Low- and High-Achieving StudentsPreprint
Grewenig, E., Lergetporer, P., Werner, K., Woessmann, L., & Zierow, L. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3720405 · Cited by 40 · see citing works
Chadwick, R., & McLoughlin, E. (2021) · Irish Educational Studies · doi:10.1080/03323315.2021.1915838 · Cited by 25 · see citing works
O’Connor, M. (2020) · Pastoral Care in Education · doi:10.1080/02643944.2020.1855674 · Cited by 16 · see citing works
Meeter, M. (2021) · doi:10.31234/osf.io/8un6x · Cited by 16 · cited 5× · see citing works
“Children from families with a low SES are less likely to have access to remote learning (UNESCO, 2021), are less often provided with active learning assistance from their schools (Tomasik et al., 2020), and spend less time on learning (Meeter, 2021) than children from families with a high SES.”
InfluentialBlume, F., Schmidt), A. I. (., Kramer, A. C., et al. (2020) · doi:10.31234/osf.io/tnrdj · Cited by 10 · see citing works
Maity, S., Sahu, T. N., & Sen, N. (2020) · Review of Education · doi:10.1002/rev3.3249 · Cited by 5 · see citing works
PISA for Development: Out-of-school assessment: Results in Focus
(2020) · PISA in Focus · doi:10.1787/491fb74a-en · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- 2021, What's Next? Lessons on Education Recovery
Cited by
Sorted by citations
A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic
Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01506-4 · Cited by 568 · see citing works
Schult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2022) · School Effectiveness and School Improvement · doi:10.1080/09243453.2022.2061014 · Cited by 262 · see citing works
“Meta-analytic and systematic reviews, which included a preprint of the present study, estimate average learning losses between d = –0.09 (in reading; Hammerstein et al., 2021) and d = –0.17 (in mathematics; Zierer, 2021) during the first wave of the COVID-19 pandemic in the year 2020.”
BackgroundInfluentialThe impact of Covid-19 on student achievement: Evidence from a recent meta-analysis
Di Pietro, G. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100530 · Cited by 173 · see citing works
“(2023) and Hammerstein et al. (2021), the risk of bias in nonrandomized studies was assessed in 3811 studies using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al.”
The Impact of COVID‐19‐Related School Closures on Student Achievement—A Meta‐Analysis
König, C., & Frey, A. (2022) · Educational Measurement: Issues and Practice · doi:10.1111/emip.12495 · Cited by 125 · see citing works
Self-citationSchool Performance among Children and Adolescents during COVID-19 Pandemic: A Systematic Review
Panagouli, E. D., Stavridou, A., Savvidi, C., Kourti, A., Ψαλτοπούλου, Θ., Sergentanis, T. Ν., & Τσίτσικα, Ά. (2021) · Children · doi:10.3390/children8121134 · Cited by 121 · see citing works
“Although this is not the first systematic review that examined the performance of students during the COVID-19 pandemic [60], we researched a variety of factors that contributed to the evaluation of academic performance.”
BackgroundSchult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2021) · doi:10.31234/osf.io/pqtgf · Cited by 262 · see citing works
Pandemic Schooling Mode and Student Test Scores: Evidence from US States
Halloran, C., Jack, R., Okun, J., & Oster, E. (2021) · doi:10.3386/w29497 · Cited by 110 · see citing works
The legacy of Covid-19 in education
Werner, K., & Woessmann, L. (2023) · Economic Policy · doi:10.1093/epolic/eiad016 · Cited by 111 · see citing works
Students’ Perceptions of Online Learning during COVID-19 Pandemic: A Qualitative Approach
Curelaru, M., Curelaru, V., & Cristea, M. (2022) · Sustainability · doi:10.3390/su14138138 · Cited by 115 · see citing works
K−12 teachers' stress and burnout during the COVID-19 pandemic: A systematic review
Westphal, A., Kalinowski, E., Hoferichter, C. J., & Vock, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.920326 · Cited by 132 · see citing works
“, 2020) and achievement, especially for students from families with low socioeconomic statuses (Hammerstein et al., 2021), thus widening existing social disparities (e.”
BackgroundKuhfeld, M. R., Lewis, K., & Peltier, T. K. (2022) · Reading and Writing · doi:10.1007/s11145-022-10345-8 · Cited by 91 · see citing works
“Given the magnitude of the disruptions due to the COVID19 pandemic, it is not surprising that numerous studies have now documented impacts of COVID-19 on student test scores (Betthäuser et al., 2022; Hammerstein et al., 2021; Thorn & Vincent-Horn, 2021; West & Lake, 2021; Zierer, 2021).”
BackgroundMazrekaj, D., & Witte, K. D. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231181108 · Cited by 105 · see citing works
“Hammerstein et al. (2021) reviewed 11 observational studies (five cross-sectional and six longitudinal) between March 2020 and April 2021 in four European countries, China, Australia, and the United States (only one of which was judged to have a low risk of bias) and found that school closures…”
BackgroundMeeter, M. (2021) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2021.100163 · Cited by 85 · see citing works
COVID-19 Pandemic and Student Reading Achievement: Findings From a School Panel Study
Ludewig, U., Kleinkorres, R., Schaufelberger, R., Schlitter, T., Lorenz, R., König, C., Frey, A., & McElvany, N. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.876485 · Cited by 95 · see citing works
Self-citationThe effectiveness of a school-based, universal mental health programme in six European countries
Cefai, C., Camilleri, L., Bartolo, P. A., Grazzani, I., Cavioni, V., Conte, E., Ornaghi, V., Agliati, A., Gandellini, S., Vorkapić, S. T., et al. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.925614 · Cited by 76 · see citing works
“Furthermore, the PROMEHS curriculum has been implemented during the COVID pandemic when the schools were closed for several weeks or months (Hammerstein et al., 2021).”
MethodNo learning loss in Sweden during the pandemic: Evidence from primary school reading assessments
Hallin, A. E., Danielsson, H., Nordström, T., & Fälth, L. (2022) · International Journal of Educational Research · doi:10.1016/j.ijer.2022.102011 · Cited by 68 · see citing works
Fütterer, T., Hoch, E., Lachner, A., Scheiter, K., & Stürmer, K. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104788 · Cited by 77 · see citing works
“…in and how advanced digitization was in that country (Betth ¨ auser et al., 2023), or if the sample of a study had already worked with the learning program before the change to digital distance teaching, it was assumed that they were more familiar with the program (Hammerstein et al., 2021).”
InfluentialA global analysis of the effectiveness of policy responses to COVID-19
Agyapon-Ntra, K., & McSharry, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-31709-2 · Cited by 62 · see citing works
Association Between the COVID-19 Pandemic and Early Childhood Development
Sato, K., Fukai, T., Fujisawa, K. K., et al. (2023) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.2096 · Cited by 58 · see citing works
Molnár, G., & Hermann, Z. (2023) · Learning and Instruction · doi:10.1016/j.learninstruc.2022.101706 · Cited by 54 · see citing works
No evidence of a major learning slide 14 months into the COVID-19 pandemic in Denmark
Birkelund, J. F., & Karlson, K. B. (2022) · European Societies · doi:10.1080/14616696.2022.2129085 · Cited by 64 · see citing works
“Although research shows evidence of a short-term learning loss (Hammerstein et al .”
BackgroundSchool closures during COVID-19: an overview of systematic reviews
Hume, S., Brown, S. R., & Mahtani, K. R. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112085 · Cited by 50 · see citing works
Rubiyatno, R., Perdana, R. P., Fallo, I. S., et al. (2023) · Pedagogy of Physical Culture and Sports · doi:10.15561/26649837.2023.0304 · Cited by 48 · see citing works
“Coronavirus Disease pandemic or commonly referred to as Covid- 19 has had a considerable impact on people’s lives throughout the world, such as closing office areas, places of worship [1], schools [2], universities and sports venues [3, 4, 5, 6].”
BackgroundSchult, J., Mahler, N., Fauth, B., & Lindner, M. A. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.867316 · Cited by 50 · see citing works
“The corresponding learning loss during the first pandemic wave has been summarized by several systematic reviews and metaanalyses (De Witte and Smet, 2021; Donnelly and Patrinos, 2021; Hammerstein et al., 2021; Panagouli et al., 2021; Zierer, 2021).”
BackgroundLerkkanen, M., Pakarinen, E., Salminen, J., & Torppa, M. (2022) · Reading and Writing · doi:10.1007/s11145-022-10358-3 · Cited by 48 · see citing works
COVID-19, school closures, and student learning outcomes. New global evidence from PISA
Jakubowski, M., Gajderowicz, T., & Patrinos, H. A. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00297-3 · Cited by 39 · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Online teaching, procrastination and student achievement
De Paola, M., Gioia, F., & Scoppa, V. (2023) · Economics of Education Review · doi:10.1016/j.econedurev.2023.102378 · Cited by 38 · see citing works
Early Childhood Learning Losses during COVID-19: Systematic Review
Uğraş, M., ZENGİN, E., Papadakis, S., & Kalogiannakis, M. (2023) · Sustainability · doi:10.3390/su15076199 · Cited by 37 · see citing works
What Kind of Learning Media do You Want? Need Analysis On Elementary School Online Learning
Utomo, G. M., Setiawan, B., Rachmadtullah, R., et al. (2021) · Jurnal Basicedu · doi:10.31004/basicedu.v5i5.1468 · Cited by 36 · see citing works
Covid-19 distance and online learning: a systematic literature review in pharmacy education
Muhaimin, M., Habibi, A., Riady, Y., Alqahtani, T. M., Chaerunisaa, A. Y., Wijaya, T. T., Milanda, T., Yusop, F. D., & Albelbisi, N. A. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04346-6 · Cited by 46 · see citing works
School closure policies and student reading achievement: evidence across countries
Kennedy, A. I., & Strietholt, R. (2023) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-023-09415-4 · Cited by 45 · see citing works
:null (2022) · Waxmann Verlag GmbH eBooks · doi:10.31244/9783830995449 · Cited by 35 · see citing works
Rethinking education in the context of climate change
Nusche, D., Rabella, M. F., & Lauterbach, S. (2024) · OECD education working papers · doi:10.1787/f14c8a81-en · Cited by 50 · see citing works
How Did COVID-19 Affect Education and What Can Be Learned Moving Forward?
Daumiller, M. H., Rinas, R., Schoon, I., & Lüftenegger, M. (2023) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000527 · Cited by 30 · see citing works
“Alarmingly, school closures due to COVID-19 specifically impacted younger children and families with low socio-economic status (see Hammerstein et al., 2021, regarding student achievement).”
BackgroundDevelopmental Milestone Attainment in US Children Before and During the COVID-19 Pandemic
Johnson, S. B., Kuehn, M., Lambert, J. O., Spin, J. P., Klein, L. M., Howard, B. J., STURNER, R. A., & Perrin, E. M. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2024.0683 · Cited by 28 · see citing works
The crisis you can’t plan for: K-12 leader responses and organisational preparedness during COVID-19
Voto, C. D., & Superfine, B. M. (2023) · School Leadership and Management · doi:10.1080/13632434.2023.2171003 · Cited by 28 · see citing works
“…most Kindergarten-12th-grade (K-12) schools have now resumed in-person instruction, the pandemic ’ s academic and emotional toll remains a lasting reminder (Hammerstein et al. 2021), especially for leaders who have had to navigate these uncertain times (De Voto et al. in press; DeMatthews 2021).”
BackgroundA Mental Health Pandemic? Assessing the Impact of COVID-19 on Young People’s Mental Health
Lloyd, K., Schubotz, D., Roche, R. R., Manzi, J., & McKnight, M. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20166550 · Cited by 27 · see citing works
“With the disruption of these routines caused by the series of lockdowns, the emotional distress expressed by the participants is a logical sequela of these events [65].”
BackgroundBaten, E., Vlaeminck, F., Muès, M., Valcke, M., Desoete, A., & Warreyn, P. (2022) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-021-05383-0 · Cited by 31 · see citing works
“Furthermore, a systematic review on the effect of COVID-19 related school closures showed negative effects on student achievement, especially for younger students and students from lower SES families (Hammerstein et al., 2021).”
BackgroundDatu, J. A. D., Buenconsejo, J. U., Shek, C. Y. C., Choy, Y. L. E., & Sou, K. L. E. (2023) · School Psychology International · doi:10.1177/01430343221147273 · Cited by 27 · see citing works
“The pandemic has also affected students’ academic achievement, particularly those from lower grade levels (i.e., primary school students) and families of low socio-economic status (Hammerstein et al., 2021).”
BackgroundThe Long-Term Effects of the COVID-19 Pandemic on Children’s Writing: a Follow-up Replication Study
Skar, G. B. U., Graham, S., & Huebner, A. R. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09729-1 · Cited by 25 · see citing works
“Most of the research examining the effects of COVID-19 on students’ academic outcomes has focused on mathematics and reading (see De Witte & Smet, 2021; Hammerstein et al., 2021; König & Frey, 2022; OECD, 2021).”
BackgroundFlegr, S., Kühn, J., & Scheiter, K. (2023) · Learning and Instruction · doi:10.1016/j.learninstruc.2023.101778 · Cited by 29 · see citing works
Wang, C., Yang, Y., Lu, X., Yang, L., Du, J., Zhang, Z., Lian, X., Cui, Y., Li, H., Zhang, W., et al. (2022) · Journal of Medical Virology · doi:10.1002/jmv.27803 · Cited by 18 · see citing works
A Systematic Review on Teachers’ Well-Being in the COVID-19 Era
Katsarou, E. C., Chatzipanagiotou, P., & Sougari, A.-M. (2023) · Education Sciences · doi:10.3390/educsci13090927 · Cited by 31 · see citing works
Gunčaga, J., Lopúchová, J., Ferdiánová, V., et al. (2022) · Education Sciences · doi:10.3390/educsci12070458 · Cited by 24 · see citing works
Effects of the COVID-19 pandemic on reading performance of second grade children in Germany
Förster, N., Forthmann, B., Back, M. D., & Souvignier, E. (2022) · Reading and Writing · doi:10.1007/s11145-022-10379-y · Cited by 25 · see citing works
“They also contradict the assumption and finding that particularly young students’ academic learning was impaired by the school closures (e.g., Georgiou, 2021; Hammerstein et al., 2021; Tomasik et al., 2021).”
BackgroundResultInfluentialCOVID-19 and inequalities in educational achievement in Italy
Borgonovi, F., & Ferrara, A. (2023) · Research in Social Stratification and Mobility · doi:10.1016/j.rssm.2023.100760 · Cited by 24 · see citing works
“We contribute to the emerging literature on the educational consequences of the pandemic [see (Thorn & Vincent-Lancrin, 2021) and (Hammerstein et al., 2021) for reviews and (Betthäuser et al.”
Spitzer, M., & Moeller, K. (2023) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2023.100203 · Cited by 23 · see citing works
(2024) · doi:10.2147/prbm.s447485 · Cited by 22 · see citing works
“12 Many educators and parents worry about K-12 students returning to school with lower academic performance when emergencies end.”
Was bedeutet Systemrelevanz in Zeiten der Pandemie?
Kaldewey, D. (2022) · Berliner Journal für Soziologie · doi:10.1007/s11609-022-00464-y · Cited by 20 · see citing works
“Distanzunterricht, so das Resultat der Metastudie von Hammerstein et al. (2021), erwies sich als in etwa so effektiv wie Sommerferien.”
BackgroundTapia‐Serrano, M. Á., Sánchez‐Oliva, D., Sevil‐Serrano, J., Marques, A., & Miguel, P. A. S. (2022) · Scientific Reports · doi:10.1038/s41598-022-21096-5 · Cited by 20 · see citing works
The existence of environmental education in the COVID-19 pandemic: A systematic literature review
Husamah, H., Suwono, H., Nur, H., Dharmawan, A., & Chang, C. (2023) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/13668 · Cited by 21 · see citing works
“at the primary and secondary school level as well as students at tertiary institutions were affected (Engzell et al., 2021; Hammerstein et al., 2021; Russell, 2022).”
BackgroundSARS-CoV-2 Circulation in the School Setting: A Systematic Review and Meta-Analysis
Caini, S., Martinoli, C., Vecchia, C. L., Raimondi, S., Bellerba, F., D’Ecclesiis, O., Sasso, C., Basso, A., Cammarata, G., & Gandini, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095384 · Cited by 19 · see citing works
Assessment of the Long-Term Mental Health Effects on Austrian Students after COVID-19 Restrictions
Kaltschik, S., Pieh, C., Dale, R., Probst, T., Pammer, B., & Humer, E. C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192013110 · Cited by 22 · see citing works
Money Matters: Time for Prevention and Early Intervention to Address Family Economic Circumstances
Axford, N., & Berry, V. (2023) · Journal of Prevention · doi:10.1007/s10935-022-00717-9 · Cited by 23 · see citing works
“, 2022), reduced student achievement (Hammerstein et al., 2021) and increased risk of child maltreatment (Marmor et al.”
Lalin, S. A. A., Ahmed, M. N. Q., & Haq, S. M. A. (2024) · Regional Science Policy & Practice · doi:10.1016/j.rspp.2024.100046 · Cited by 23 · see citing works
Tomaszewski, W., Zając, T., Rudling, E. S., Riele, K. T., McDaid, L. M., & Western, M. (2022) · Australian Journal of Social Issues · doi:10.1002/ajs4.219 · Cited by 23 · see citing works
Spitzer, M., Moeller, K., & Musslick, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0284868 · Cited by 18 · see citing works
“These closures required teachers to adopt different approaches to home schooling and distance learning —with varying degrees of success [2,3].”
BackgroundA meta-analysis of the impact of COVID-19 on student achievement
Storey, N., & Zhang, Q. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100624 · Cited by 18 · see citing works
Schooling upheaval during COVID-19: troubling consequences for students’ return to school
Fray, L., Jaremus, F., Gore, J. M., & Harris, J. (2022) · The Australian Educational Researcher · doi:10.1007/s13384-022-00572-x · Cited by 17 · see citing works
“Substantial research has been conducted on student learning, with most studies reporting negative effects, particularly for younger children and those from low socio-economic backgrounds (Hammerstein et al., 2021; Patrinos et al., 2022).”
Background“Let’s transform!”: A systematic literature review of science learning in COVID-19 pandemic era
Nurwidodo, N., Ibrohim, I., Sueb, S., & Husamah, H. (2023) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/12875 · Cited by 20 · see citing works
Koirala, A., Winkler, N., Quinn, H., Gardiner, E. R. K., Liu, B. C., Forbes, J., Sharpe, C., van Tussenbroek, T., Wood, N., & Macartney, K. K. (2022) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2023.100736 · Cited by 17 · see citing works
Quansah, F., Frimpong, J. B., Sambah, F., Oduro, P., Anin, S. K., Srem‐Sai, M., Hagan, J. E., & Schack, T. (2022) · Healthcare · doi:10.3390/healthcare10050920 · Cited by 17 · see citing works
Bertoletti, A., Biagi, F., Pietro, G. D., & Karpiński, Z. (2023) · Large-scale Assessments in Education · doi:10.1186/s40536-023-00154-y · Cited by 19 · see citing works
“Along with the access to the digital devices and the ability to use them efficiently, additional factors related to students’ sense of social isolation, psychological distress, emotional support, subjective well-being, and relationships with peers, teachers, or family, might have affected their academic performance an…”
BackgroundSpitzer, M., & Moeller, K. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100162 · Cited by 16 · see citing works
Bracco, J., Ciaschi, M., Gasparini, L., et al. (2024) · Review of Income and Wealth · doi:10.1111/roiw.12687 · Cited by 16 · see citing works
Oostdam, R., van Diepen, M., Zijlstra, B. J. H., & Fukkink, R. G. (2023) · European Journal of Psychology of Education · doi:10.1007/s10212-023-00679-4 · Cited by 15 · see citing works
“This finding is in line with previous studies, concerning the effects of the recent school lockdowns (see Engzell et al., 2020; Hammerstein et al., 2021).”
InfluentialTeachers’ well-being and their teaching quality during the COVID-19 pandemic: a retrospective study
Sacré, M., Ries, N., Wolf, K., & Kunter, M. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1136940 · Cited by 26 · see citing works
Branch‐Elliman, W., Elwy, A. R., & Chambers, D. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1207679 · Cited by 15 · see citing works
Peters, S. J., Langi, M., Kuhfeld, M., & Lewis, K. (2025) · Journal for the Education of the Gifted · doi:10.1177/01623532241301079 · Cited by 15 · see citing works
“Given the well-established effects of school closures on student learning (Goldhaber et al., 2022; Hammerstein et al., 2021) it is not surprising that student groups who were in virtual learning environments for longer would see more un fi nished learning.”
Welter, V. D. E., Becker, L. B., & Großschedl, J. (2022) · Education Sciences · doi:10.3390/educsci12050325 · Cited by 17 · see citing works
Miller, A., Fray, L., & Gore, J. M. (2023) · The Australian Educational Researcher · doi:10.1007/s13384-023-00614-y · Cited by 14 · see citing works
“Learning loss was reported, for instance, in nine of the 11 studies included in Hammerstein’s (2021) systematic review (effect size − 0.37 SD to − 0.03 SD ) (Hammerstein et al., 2021), and Patrinos et al. (2022) reported learning loss of 0.17 standard deviations on average from 36 robust studies.”
InfluentialLo, N., & to, B. K. H. (2023) · SN Computer Science · doi:10.1007/s42979-023-01779-0 · Cited by 16 · see citing works
Paczos, W., Sawulski, J., & Leśniewicz, F. (2023) · International Review of Economics & Finance · doi:10.1016/j.iref.2023.07.010 · Cited by 14 · see citing works
Ghanamah, R., Eghbaria-Ghanamah, H., & Adi‐Japha, E. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101898 · Cited by 16 · see citing works
Weber, C., Helm, C., & Kemethofer, D. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.737064 · Cited by 13 · see citing works
“However, although there is growing research confirming the expected learning losses and growth of disparities, the number of studies is still small and largely limited to English-speaking countries (see Helm et al., 2021b; Hammerstein et al., 2021).”
BackgroundResultInfluentialYuda, A. K., Resita, C., Sumarsono, R. N., Gani, R. A., Németh, Z., & Setiawan, E. (2022) · Physical Education Theory and Methodology · doi:10.17309/tmfv.2022.2.05 · Cited by 14 · see citing works
Assessing the Nexus Between Education, Economic Growth, and Innovation: An Empirical Analysis
Boţoroga, C.-A., Horobeț, A. L., Belaşcu, L., et al. (2022) · Studies in Business and Economics · doi:10.2478/sbe-2022-0043 · Cited by 14 · see citing works
Muneer, S., Santhosh, M., Parangusan, H., & Bhadra, J. (2026) · International Journal of Technology and Design Education · doi:10.1007/s10798-025-10005-2 · Cited by 14 · see citing works
Chaban, T., Rameeva, R., Denisov, I., Дмитриевна, К. Ю., & Zvyagintsev, R. (2022) · Voprosy Obrazovaniya/ Educational Studies Moscow · doi:10.17323/1814-9545-2022-1-160-188 · Cited by 13 · see citing works
Copeland, K. A., Porter, L., Gorecki, M. C., Reyner, A., White, C., & Kahn, R. S. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.6458 · Cited by 13 · see citing works
Maurer, J., Becker, A. B. C., Hilkenmeier, J., & Daseking, M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.733865 · Cited by 13 · see citing works
School Closure Policies and Student Reading Achievement: Evidence Across CountriesPreprint
Kennedy, A., & Strietholt, R. (2023) · doi:10.31219/osf.io/93rgz · Cited by 13 · see citing works
Estrela, M., Leitão, C., Silva, T. M., Figueiras, A., Roque, F., & Herdeiro, M. T. (2022) · Vaccines · doi:10.3390/vaccines10121984 · Cited by 10 · see citing works
Nutley, S. B., Burén, J., & Thorell, L. B. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20166583 · Cited by 13 · see citing works
“Considering that previous studies have demonstrated a significant decline in academic progression due to remote learning [51,52], it is possible that the increase in stress observed in our study is at least partially a result of the cumulative strain of struggling academically during remote learning.”
Tanri, A., Benny Aprial, M., Mashud, M., et al. (2023) · Retos · doi:10.47197/retos.v50.100587 · Cited by 12 · see citing works
Buda, A., & Czékmán, B. (2021) · Central European Journal of Educational Research · doi:10.37441/cejer/2021/3/3/10391 · Cited by 12 · see citing works
Kirsch, C., & Vaıoulı, P. (2023) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2023.100433 · Cited by 13 · see citing works
Social disparities in students’ intention to enter higher education during the COVID-19 pandemic
Stark, J., Daniel, A., & Twardawski, M. (2022) · PLoS ONE · doi:10.1371/journal.pone.0267978 · Cited by 13 · see citing works
“In line with recently published studies, students from non-academic households reported a lower academic performance than their peers from academic households [3, 25, 26].”
ResultPaola, M. D., Gioia, F., & Scoppa, V. (2022) · Social Science Research Network · doi:10.2139/ssrn.4114561 · Cited by 15 · see citing works
Brankston, G., Merkley, E., Loewen, P. J., Avery, B. P., Carson, C. A., Dougherty, B. P., Fisman, D. N., Tuite, A. R., Poljak, Z., & Greer, A. L. (2022) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2022.101993 · Cited by 11 · see citing works
“…2020; Ł aszewska et al., 2021) and may be due to a variety of factors including perceptions of negative impacts on learning (Engzell et al., 2021; Hammerstein et al., 2021), psychosocial impacts (Meherali et al., 2021; Pfefferbaum, 2021), and declines in physical well-being (Maximum City, 2020;…”
Kappus-Kron, H. R., Chatila, D., MacLachlan, A. M., Pulido, N., Yang, N., & Larsen, D. A. (2024) · PLOS Global Public Health · doi:10.1371/journal.pgph.0001803 · Cited by 10 · see citing works
Karadağ, E., & Sertel, G. (2025) · Educational Management Administration & Leadership · doi:10.1177/17411432251327645 · Cited by 24 · see citing works
“The relationship between leadership practices and student achievement during the pandemic appears to be complex and multifaceted, in fl uenced by factors such as subject area, grade level, and socioeconomic status (Asakawa et al., 2024; Hammerstein et al., 2021; Ludewig et al., 2022).”
Coles, L., Johnstone, M., Pattinson, C. L., Thorpe, K. J., Halen, O. V., Zheng, Z., Bayliss, O., Lakeman, N., Western, M., Povey, J., et al. (2022) · Australian Journal of Social Issues · doi:10.1002/ajs4.236 · Cited by 11 · see citing works
The ability of low- and High-SES schools to inhibit learning losses during the COVID-19 pandemic
Vit, E. (2022) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2022.100393 · Cited by 12 · see citing works
“closures harmed students’ achievement [30,31,32,33].”
Emergency remote assessment practices in higher education in sub-Saharan Africa during COVID-19
Baidoo-Anu, D., Asamoah, D., Quainoo, E. A., et al. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1221115 · Cited by 11 · see citing works
Sarker, M. F., & Ullah, M. S. (2023) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2023.100740 · Cited by 11 · see citing works
Theorising post-truth in the COVID era
Nally, D. (2022) · Journal of Educational Change · doi:10.1007/s10833-022-09457-3 · Cited by 11 · see citing works
COVID-19 Pandemic and Student Reading Achievement – Findings from a School Panel StudyPreprint
Ludewig, U., Kleinkorres, R., Schaufelberger, R., et al. (2022) · doi:10.31234/osf.io/hrzae · Cited by 11 · see citing works
Self-citationCampos, D., Fütterer, T., Gfrörer, T., Lavelle-Hill, R. E., Murayama, K., König, L., Hecht, M., Zitzmann, S., & Scherer, R. (2023) · doi:10.31234/osf.io/fpwc2 · Cited by 11 · see citing works
Policy responses to extreme heat and its impact on education: The Philippine experience
Preña, E. M., & Labayo, C. P. (2024) · Policy Futures in Education · doi:10.1177/14782103241288276 · Cited by 10 · see citing works
“…affect students from disadvantaged backgrounds, particularly those lacking access to essential learning tools such as home computers, thereby exacerbating educational inequalities (Engzell et al., 2021; Hammerstein et al., 2021; Kennedy and Strietholt, 2023; Maldonado and De Witte, 2022).”
- Authors
- Hammerstein, S., König, C., Dreisörner, T., & Frey, A.
- Affiliations
- Goethe University Frankfurt (DE)
- Corresponding author
- Hammerstein, Svenja
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2021-09-16
- Volume
- 12
- Pages
- 746289-746289
- ISSN
- 1664-1078 (ISSN-L)
- PMC
- PMC8481663
- OpenAlex
- W3172046965
- Semantic Scholar
- 650bd6b2e0b8178e55d72195ab3ccad79308a63f
- Cited by
- 461
- References
- 38
- Citations by year
- 2026: 32 · 2025: 66 · 2024: 109 · 2023: 155 · 2022: 114 · 2021: 14
- FWCI
- 26.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Psychology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- OSF Preprints (OSF Preprints) (acceptedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Europe PMC (PubMed Central) (submittedVersion, other-oa)
- PubMed Central (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Social Sciences › Education › School Choice and Performance
- Keywords
- COVID-19 pandemic, school closures, academic performance, primary education, secondary education, socioeconomic status, low-income students, younger students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review