The DOI matches its Crossref record; 112 of the 122 references it cites were found in a registry or an index, and 1 are not checked yet; none of the 38 sources its claims cite is checked for support yet.
The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis
Open accessAbstract
This work attempts to synthetize existing research about the impact of Covid-19 school closure on student achievement. It extends previous systematic reviews and meta-analyses by (a) using a more balanced sample in terms of country composition, (b) considering new moderators (type of data and research design), and (c) including studies on tertiary education students in addition to primary and secondary education students. Our meta-analysis findings show that the pandemic had, on average, a detrimental effect on learning. The magnitude of this learning deficit (about 0.19 standard deviations of student achievement) appears to be roughly comparable to that suffered by students who have experienced a significant disruption in their schooling due to a major natural disaster (e.g., Hurricane Katrina). Students are also found to have lost more ground in math/science than in other subjects. Additionally, one year or more after the first lockdown, students seem to have been unable to catch up on unfinished learning from the pandemic. This result suggests that more efforts should be made to ensure students recover their missed learning in order to avoid negative long-term consequences for them and society.
Checked 2026-10-07 · 1 reference not checked yet
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This meta-analysis of 239 estimates from 39 studies across 19 countries finds that Covid-19 school closures reduced student achievement by about 0.19 standard deviations on average. Learning deficits were larger in math/science and showed no statistically significant recovery by 2021.
Key findings
- The random-effects robust variance estimation mean effect size was −0.186 SD (SE = 0.0646, p = 0.0065, 95% CI [−0.316, −0.055]), described as roughly comparable to losses after major natural disasters and equivalent to just over half a school year.
- Math/science achievement was 0.17 SD lower than humanities/subject mix, other things equal.
- There was no statistically significant difference between assessments in 2020 and 2021, suggesting students had not caught up more than one year after the first lockdown.
- Learning deficits were smaller in Europe than non-Europe, and studies using quasi-experimental/experimental designs or cross-sectional data tended to find larger deficits than descriptive/correlational or longitudinal studies.
Methods
- PRISMA-guided meta-analysis; searched six electronic databases through 15 July 2022, yielding 6,075 hits and a final sample of 39 studies, 239 effect sizes, and 19 countries.
- Effect sizes were converted to Cohen’s d; the main model used random-effects meta-analysis with robust variance estimation to account for multiple dependent estimates per study, plus weighted least squares meta-regression with general-to-specific selection and Bayesian Model Averaging.
- Moderators included subject, education level, timing of assessment, geographical area, data type, research design, publication year/type, and standard error; publication bias was assessed with a Doi plot/LFK index and Egger’s test.
Limitations
- Stated in the paper: the analysis was not conducted at grade level, so it cannot rule out disproportionate effects on very young pupils (e.g., grade 1) or heterogeneity within primary school children.
- Stated in the paper: ROBINS-I assessment found confounding, participant selection, and missing data were common sources of potential bias; vulnerable students were under-represented in some Covid-19 samples, likely underestimating learning delays, while the direction of bias was unclear in convenience samples.
- Evident from the text: the evidence base is limited to 39 studies and 19 countries, the search ended in July 2022, and tertiary education is represented by only 10 effect sizes; publication bias tests found no asymmetry (LFK index = 0; Egger’s test p = 0.380).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
27 Not checked yet · 18 Own result · 10 Own method · 6 Own interpretation · 4 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our meta-analysis findings show that the pandemic had, on average, a detrimental effect on learning.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The magnitude of this learning deficit (about 0.19 standard deviations of student achievement)”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Abstract
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, compared to previous meta-analyses, this study covers a larger number of countries (i.e., 19).”
- Also stated in: § Sample characteristics · plain
“The studies cover a total of 19 countries.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Sample characteristics · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The RVE RE mean effect size turns out to be −0.18613 (SE = 0.0646, p-value = 0.0065, 95% CI [-0.316, −0.055]).”
- Also stated in: § Discussion of the main results · plain
“we find that, on average, the pandemic depressed student achievement by around 0.19 standard deviations.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Summary effect size
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Discussion of the main results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Students are also found to have lost more ground in math/science than in other subjects.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Abstract
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, one year or more after the first lockdown, students seem to have been unable to catch up on unfinished learning from the pandemic.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Abstract
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, the current meta-analysis adds to previous meta-analyses by including also studies looking at the impact of Covid-19 among tertiary education students in addition to primary and secondary education students.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The sample includes 239 estimates from 39 studies covering 19 countries.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“studies employing cross-sectional data as well as those focusing on non-European countries tend to suggest greater learning deficits.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Heterogeneity
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the negative effect of Covid-19 on student achievement appears to be more pronounced when using experimental/quasi experimental techniques than when using descriptive or correlational research designs.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Heterogeneity
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 24
- It may state two separate claims as one. 16
- The quote only partly states this claim. 2
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The largest source countries are the US (71 estimates), Germany (39 estimates) and Belgium (33 estimates).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Sample characteristics
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each study included in the dataset contains a number of estimates that vary from 1 to 32.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Sample characteristics
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A total of 239 separate estimates from 39 studies are considered.”
- Also stated in: § Sample characteristics · plain
“The dataset used for the meta-analysis includes a total of 239 different impact estimates extracted from 39 separate studies.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Also stated in: § Sample characteristics · plain
In the paper
“For instance, while in Betthäuser et al. (2023) about 71.1% of the effect sizes are derived from these studies, in our paper the corresponding figure is approximately 33.9%.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
What the sources say
- Not checked yetcites Betthäuser et al. (2023) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Several reasons explain why most studies (i.e., 79%) reported multiple estimates.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Sample characteristics
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The appropriateness of the RE model is confirmed as I2 has a value of 100%.14 This suggests that all the variability in the effect-size estimates is due to heterogeneity as opposed to sampling error.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Summary effect size
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Its large value of 1.74 further corroborates the hypothesis of substantial heterogeneity of the effect sizes (Takase & Yoshida, 2021).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Summary effect size
What the sources say
- Not checked yetcites Takase & Yoshida, 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“students experienced larger learning deficits in math/science. More precisely, other things being equal, student achievement in math/science is on average found to be 0.17 standard deviations smaller than in humanities/subject mix.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Heterogeneity
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“our results indicate that in 2021 students were not able to recover from the learning deficits caused by Covid-19 school closures in 2020. There is no statistically significant difference in student performance between assessments that have taken place several months or more than one year after the outbreak of the coronavirus and those that have occurred in the early stages of the pandemic.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“studies employing cross-sectional data as well as those focusing on non-European countries tend to suggest greater learning deficits.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Heterogeneity
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“To perform this meta-analysis, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (Moher et al., 2009).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method
What the sources say
- Not checked yetcites Moher et al., 2009 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The relevance of these factors in explaining the heterogeneity of results across studies is well-known in the meta-analysis literature.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
What the sources say
- Not checked yetcites Havránek et al., 2020
- Not checked yetcites Stanley and Jarrell, 1989
- Not checked yetcites Judd, 2015
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“While the inter-rater agreement was very high (i.e., 97%), studies on which there was disagreement were discussed in depth until consensus was reached.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Search ad screening procedures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“While all the titles and abstracts were screened only by the author, the next stages of the study selection process were carried out by the author and by another researcher who independently classified the studies as relevant and irrelevant based on the predefined inclusion criteria.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Search ad screening procedures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As discussed later in the paper, several methods are used to account for within-study dependence.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Study coding
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Study coding was performed following the same procedure used for the final stages of the study selection process.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Study coding
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In most countries, educational institutions had to close for several weeks or months in an attempt to reduce the spread of the virus (UNESCO, 2020a).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
What the sources say
- Not checked yetcites UNESCO, 2020a
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Several studies from various countries (e.g., Bratti et al., 2004; Dabalen et al., 2001; Koda & Yuki, 2013) show that the final grade awarded to students successfully completing university is an important predictor of their labour market prospects.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
What the sources say
- Not checked yetcites Bratti et al., 2004
- Not checked yetcites Dabalen et al., 2001
- Not checked yetcites Koda & Yuki, 2013
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The inter-rater agreement was again high (i.e., 93%).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Study coding
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Additionally, non-journal articles are more likely to contain typos in their regression tables (Cazachevici et al., 2020).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Study coding
What the sources say
- Not checked yetcites Cazachevici et al., 2020
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Articles published in journals are expected to be of higher scientific rigour since they are more likely to have gone through a review process.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Study coding
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“As argued in several papers (e.g., Hanushek & Woessmann, 2020; Psacharopoulos et al., 2021), there is the risk that the disruption in learning caused by Covid-19 may persist over time, having long-term consequences on students’ knowledge and skills as well as on their labour market prospects.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Introduction
What the sources say
- Not checked yetcites Hanushek & Woessmann, 2020
- Not checked yetcites Psacharopoulos et al., 2021 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“During school closures, these students are less likely to have had access to a computer, an internet connection, and a space conducive to learning (Blaskó et al., 2022; Di Pietro et al., 2020).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Summary effect size
What the sources say
- Not checked yetcites Blaskó et al., 2022
- Not checked yetcites Di Pietro et al., 2020
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The results, which are reported in Table 5, show that all the variables that are consistently identified by the BMA methodology as relevant (i.e., Math/Science, Europe and Journal article) are also included in the specification whose estimates are reported in Columns (2), (3) and (4) of Table 4.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Heterogeneity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As suggested by Donnelly and Patrinos (2020), high-income countries are likely to have experienced smaller learning deficits as a result of Covid-19 because of their higher technological capability and the lower share of households living below the poverty line.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
What the sources say
- Not checked yetcites Donnelly and Patrinos (2020)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“given that our analysis was not conducted at grade level, one cannot rule out the possibility that the pandemic has disproportionately affected the achievement of very young pupils (e.g., grade 1).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“one would expect children of less educated parents to have received less parental support while learning at home than children of more educated parents.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Summary effect size
What the sources say
- Not checked yetcites Ariyo et al. (2022)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Promoting the development of metacognition skills is, for instance, a powerful way to enhance student learning and performance (Stanton et al., 2021).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Conclusions
What the sources say
- Not checked yetcites Stanton et al., 2021
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The underestimation of pandemic-related learning delays may have important policy implications as it could result in under-provision of remedial support to students who are falling behind due to Covid-19.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“the risk of bias in nonrandomized studies was assessed in 3811 studies using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al., 2016).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Risk of bias assessment
What the sources say
- Not checked yetcites Sterne et al., 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Fig. 2, the Doi plot shows no asymmetry (LFK index = 0), indicating that no publication bias is detected.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Publication bias
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“less attention has been paid to a number of time- and cost-effective pedagogical practices (Carrasco et al., 2021)”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Conclusions
What the sources say
- Not checked yetcites Carrasco et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“there is evidence, but only available at higher education level, that European educational institutions were better prepared to respond to the challenges posed by the pandemic than their counterparts in other parts of the world. A survey carried out by the International Association of Universities immediately after the outbreak of the coronavirus shows that the percentage of higher education”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
What the sources say
- Not checked yetcites Marinoni et al., 2020
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As a robustness test, the model depicted in Column (3) of Table 4 is re-estimated but this time each effect size is weighted by its inverse variance. As shown in Column (4) of Table 4, with the exception of the estimate on longitudinal data, the sign and the magnitude of the other coefficients are broadly in line with those depicted in Column (3).”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Heterogeneity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As pointed out by Engzell et al. (2021), non-causal methods fail to account for trends in student progress prior to the outbreak of Covid-19 and, hence, by assuming a counterfactual where achievement has stayed flat, they generate estimates of learning deficits that are biased downwards.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
What the sources say
- Not checked yetcites Engzell et al. (2021) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each study was independently evaluated by the author and another researcher, and any disagreements were resolved through discussion to reach a consensus.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Risk of bias assessment
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This would suggest that our finding is more generalizable than that of previous studies.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Results and discussion > Discussion of the main results
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
- Rests on an assumption: The sample used in this meta-analysis is more balanced in terms of country composition than previous meta-analyses.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In several studies, vulnerable students, who have been among the most hardly hit by the pandemic, tend to be under-represented in the Covid-19 sample. This may lead to an underestimation of the pandemic-related learning delays.”
§ The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis☆ > Method > Risk of bias assessment
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
The European countries included in the study have, on average, a higher GDP per capita than most non-European countries included (except US and Australia).
“the European countries considered in this study have, on average, a higher gross domestic product per capita than most of the non-European countries included in the analysis (this is not true for the US and Australia).”
The paper relies on this without support.
The sample used in this meta-analysis is more balanced in terms of country composition than previous meta-analyses.
“it should be taken into account that we use a more balanced sample in terms of country composition.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Interestingly, our results are similar to what systematic reviews are finding learning16–20,44,45.”
From COVID-19, school closures, and student learning outcomes. New global evidence from PISA (2025) · cites this paper as [44]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 2×
“Next, we compute the I 2 statistic to assess the heterogeneity of the results across studies (Higgins et al., 2003).”
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · cited 3×
“2, one would have expected Covid-19 school closures to have had a more negative impact on the achievement of younger students than older students, this effect could have been offset by the greater support in terms of parental involvement received by the former group of students during online learning. Bubb and Jones (…”
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · cited 2×
“(2021), the risk of bias in nonrandomized studies was assessed in 38(10) studies using the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al., 2016).”
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,762
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Cochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,232
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037 · cited 3×
“G. Di Pietro clusters (i.e., 39), following Cameron and Miller (2015) we apply the correction for small of clusters by employing wild score bootstrapping (Kline & Santos, 2012).”
InfluentialA Literature Review on Impact of COVID-19 Pandemic on Teaching and Learning
Pokhrel, S., & Chhetri, R. (2021) · Higher Education for the Future · doi:10.1177/2347631120983481 · Cited by 2,400 · cited 1×
“In addition to problems in accessing appropriate technology (computers, reliable internet connection, etc.), not all students had a home environment free of disturbances and distractions, hence conducive to learning (Pokhrel & Chhetri, 2021).”
Research works printed but not linked: 3
- Borgonovi, 2022, A longitudinal perspective of the effects of Covid-19 on students' resilience, The effect of the pandemic on the reading and mathematics achievement of 5th and 8th graders in Italy
- Burgess, 2020, How we should deal with the lockdown learning loss in England's schools, VOX CEPR Policy Portal
- Marinoni, 2020, The impact of Covid-19 on higher education around the world, IAU Global Survey Report
Cited by
Sorted by citations
Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014) · American Economic Review · doi:10.1257/aer.104.9.2633 · Cited by 1,717
Mazrekaj, D., & Witte, K. D. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231181108 · Cited by 128
“Di Pietro (2023) conducted a meta-analysis of 38 observational studies (26 cross-sectional and 12 longitudinal) and one experimental study between 2020 and 2022 in 12 European countries, Australia, Brazil, China, Egypt, Mexico, South Africa, and the United States (six of which were rated to have a…”
BackgroundCOVID-19, school closures, and student learning outcomes. New global evidence from PISA
Jakubowski, M., Gajderowicz, T., & Patrinos, H. A. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00297-3 · Cited by 65
School closure policies and student reading achievement: evidence across countries
Kennedy, A. I., & Strietholt, R. (2023) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-023-09415-4 · Cited by 45
“1 Substantial variation in learning deficits can be observed across individual studies and countries, as demonstrated by two recent meta-analyses (Betthäuser et al., 2023; Di Pietro, 2023).”
InfluentialLiu, X., Hansen, K. Y., Valcke, M., & Neve, J. D. (2024) · ZDM: Mathematics Education · doi:10.1007/s11858-024-01630-7 · Cited by 38
“This fits with the recent literature showing that social policies and economic conditions have been favourable to increased access and resources for education in these countries (Di Pietro, 2023; Tadesse & Muluye, 2020).”
Metadata
- Authors
- Di Pietro, G.
- ORCID
- Di Pietro, Giorgio
- Venue
- Educational Research Review
- Publisher
- Elsevier BV
- Published
- 2023-05
- Volume
- 39
- Pages
- 100530
- ISSN
- 1747-938X (ISSN-L)
- PMC
- PMC10028259
- OpenAlex
- W4328053575
- Semantic Scholar
- 19437454e7960020d7e2f1fb49ca5ca3c49e72d9
- Cited by
- 173
- References
- 122
- Citations by year
- 2026: 49 · 2025: 59 · 2024: 42 · 2023: 22
- FWCI
- 127.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Educational Research Review (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- European Commission Joint Research Centre
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Social Sciences › Education › School Choice and Performance
- Keywords
- COVID-19 pandemic, school closures, academic performance, learning loss, learning recovery, tertiary students, K-12 education, math and science achievement, Hurricane Katrina, lockdown duration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
