COVID-19, school closures, and student learning outcomes. New global evidence from PISA
Open accessAbstract
The COVID-19 pandemic resulted in significant disruption in schooling worldwide. Global test score data is used to estimate learning losses by modeling the effect of school closures on achievement by predicting the deviation of the most recent results from a linear trend using data from all rounds of PISA. Mathematics scores declined an average of 14 percent of a standard deviation, roughly equal to seven months of learning. Losses are greater for students in schools that faced relatively longer closures, for boys, immigrants, and disadvantaged students. Educational losses may translate into significant national income losses over time.
AI snapshot
Written from the paper’s full text.
TL;DR
Using all PISA rounds through 2022, the paper estimates that COVID-19 school closures were associated with a global decline in mathematics achievement of 12 PISA points, or 14.2% of a standard deviation—roughly seven months of learning. Losses were larger where schools were closed longer and were concentrated among lower-achieving students, with potential large economic costs.
Key findings
- Mathematics scores declined by an average of 12 points, equal to 14.2% of a within-country SD and more than 7 months of learning; one week of full school closure was associated with an additional decline of 0.44 points.
- Learning losses increased with closure length: countries at the shortest closures lost 8–10 points (9–12% SD), average closures about 12 points (13–14% SD), and the longest closures 15 points or more (17–23% SD), or 9–12 months of learning.
- Boys and immigrant students had lower overall losses than girls and natives, but longer closures affected them more: each week of full closure was associated with 0.15–0.17 additional points for boys and 0.33 points for immigrant students.
- The socioeconomic achievement slope increased by 2.62 points, about 7% relative to the pre-pandemic OECD average, and quantile results show larger losses among average and lowest-achieving students in countries with prolonged closures; reading results were similar to mathematics, while science effects were insignificant.
- The paper estimates average global economic losses of $17 trillion, with average GDP growth losses of 0.15 percentage points, ranging from zero to 0.57 points.
Methods
- Uses OECD PISA data from all rounds through 2022, covering 72 countries with at least 2022 and one pre-pandemic round, more than 3 million students, and representing 175 million 15-year-olds; mathematics is the focus because it was the main PISA domain in 2022 and has comparable trends.
- Estimates each country’s linear achievement trend separately with country fixed effects and controls for student gender, socioeconomic status, immigrant background, and country averages of these indicators; the main estimate is the 2022 departure from that trend.
- Uses four school-closure measures—UNESCO full closures, UNESCO full and partial closures, PISA student-reported closures, and PISA principal-reported closures—and runs robustness checks including placebo tests, quadratic trends, alternative samples, different weights, and unconditional quantile regressions.
Limitations
- The paper states that reading and science results should be interpreted cautiously because their pre-pandemic trends were not linear, so the identification strategy cannot clearly separate pandemic effects from long-term declines; both were also minor PISA 2022 domains with less precise measurement.
- The paper states that all four school-closure measures are imperfect: UNESCO full closures cover only whole-country all-school closures and do not differentiate education levels; UNESCO full-plus-partial closures do not capture closure intensity; PISA student and principal reports are self-reported and rely on memory and interpretation, and students may have attended different schools during the pandemic.
- The text makes evident an inconsistency between the abstract, which says losses were greater for boys and immigrants, and the results/discussion, which report that boys and immigrant students had lower overall learning loss than girls and natives but were more negatively affected by longer closures.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
19 of 35 cited sources checked · 10 supported · 2 flagged · accuracy not yet measured
- Premisecitation
For PIRLS, losses are about a year's worth of learning.
“For PIRLS, losses are about a year’s worth of learning.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [24] Jakubowski, M., Gajderowicz, T. & Patrinos, H. A. Global learning loss in student achievement: first estimates using comparable reading scores. Econ. Lett. 232, 111313 (2023). source
- cites [25] Kennedy, A. I. & Strietholt, R. School closure policies and student reading achievement: evidence across countries. Educ. Assess. Eval. Account. 35, 475–501 (2023). source
Section 6.1 estimates that a full year of school closures would entail a PIRLS reading loss of 57% of a school year's learning, not about a full year's worth. The paper notes that typical annual progress may differ across countries. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
- Premisecitation
A large-scale randomized trial testing SMS messages and phone calls with parents in Botswana improved learning by 0.12 standard deviations.
“A large-scale randomized trial testing low-technology interventions—SMS messages and phone calls—with parents to support their child in Botswana improved learning by 0.12 standard deviations35 but this would not be enough to reverse the total losses and it has only been tested in a few low- and middle-income environments36, and the effects are limited when not implemented well and, in any case, w”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [35] Angrist, N., Bergman, P. & Matsheng, M. Experimental evidence on learning using low-tech when school is out. Nat. Hum. Behav. 6, 941–950 (2022). source
- Findingown data
Students in countries with the shortest school closures, at the 10th percentile, lost around 8-10 points, or 9-12% of a standard deviation, equivalent to 4-6 months of learning.
“students in countries with the shortest closures (10th percentile of all participating countries) lost around 8–10 points or 9–12% SD.”
- Also stated in: § Learning loss in mathematics · plain
“Countries with the shortest closures experienced relatively small losses, 9–12% of a SD, or 4–6 months of learning.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Learning loss in mathematics · plain
- Findingreasoning
The overall decline in mathematics associated with pandemic-era school closures is 12 points, equal to 14.2% of a standard deviation.
“Mathematics scores declined an average of 14 percent of a standard deviation, roughly equal to seven months of learning.”
- Also stated in: § Learning loss in mathematics · plain
“The overall decline in mathematics associated with pandemic-era school closures is 12 points. In 2022, the average within-country SD of mathematics is 86.2. Thus, the decline in achievement is equal to 14.2% of a SD.”
- Also stated in: § Discussion · plain
“Math scores declined from 2018 to 2022 by an average of 12 points, or 14% of a standard deviation (SD), roughly equivalent to 7 months of learning.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Learning loss in mathematics · plain
- Findingown data
Longer school closures had a more negative effect on boys.
“nevertheless, longer school closures had a more negative effect on both groups”
- Also stated in: § Learning loss by gender, immigrant background, and achievement level · plain
“Depending on the closure length measure, each week of school closures was associated with 0.17 or 0.15 points of additional decline in achievement for boys.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Learning loss by gender, immigrant background, and achievement level · plain
- Findingown data
Every week of full school closures diminished the increase in the ESCS slope by 0.12 points.
“On the other hand, every week of full school closures diminished this increase by 0.12 points.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
Immigrant students experienced a lower learning loss compared to natives.
“While boys and immigrant students experienced a lower learning loss compared to girls and natives, nevertheless, longer school closures had a more negative effect on both groups (see Supplementary Table 4).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
Boys experienced a lower learning loss compared to girls.
“While boys and immigrant students experienced a lower learning loss compared to girls and natives, nevertheless, longer school closures had a more negative effect on both groups (see Supplementary Table 4).”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
Depending on the closure length measure, each week of school closures was associated with 0.17 or 0.15 points of additional decline in achievement for boys.
“Depending on the closure length measure, each week of school closures was associated with 0.17 or 0.15 points of additional decline in achievement for boys.”
- Also stated in: § Discussion · plain
“Depending on the closure length measure, each week of school closures was associated with 0.15–0.17 points or around 0.2% of a SD of additional decline in achievement for boys.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Discussion · plain
- Findingown data
Students with an immigrant background lost 0.33 points with every week of full school closures.
“Students with an immigrant background lost 0.33 points with every week of full school closures and while in countries with shorter closures their learning loss was smaller, for countries with around 25 weeks of closures their learning loss is equal to that for natives, but it increases for longer closures.”
- Also stated in: § Discussion · plain
“Students with an immigrant background lost 0.33 points or 0.4% of a SD with every week of full school closures and while in countries with shorter closures their learning loss was smaller, for countries with around 25 weeks of closures their learning loss is like the learning loss for natives, and it increases for longer closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Discussion · plain
- Findingown data
One week of full school closures, as defined by UNESCO, is associated with an additional decline of 0.44 points in mathematics achievement.
“The clearest definition is the full closure defined by UNESCO; thus, this parameter (−0.44) will be used further to estimate the global welfare losses.”
- Also stated in: § Global economic impact and the need for learning recovery · plain
“Our model predicts that every week of full school shutdown will result in a 0.44-point achievement decrease.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Global economic impact and the need for learning recovery · plain
- Findingown data
Declines are similar across the mathematics achievement spectrum only for countries with relatively short closures.
“Our quantile regression estimates in Supplementary Fig. 1 show this is true only for countries with relatively short closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
Learning losses increase with the length of school closures.
“First, the learning losses increase with the length of school closures.”
- Also stated in: § Learning loss in mathematics · plain
“The length of school closures is associated with the estimated learning loss.”
- Also stated in: § Discussion · plain
“Larger declines are recorded for students in schools that faced relatively longer closures and for lower-achieving students.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Learning loss in mathematics · plain
- Findingown data
In countries with lengthy school closures, learning loss is even more considerable among the lowest-achieving students than among average-achieving students.
“However, in countries with lengthy closures, the learning loss is more extensive among the average students and even more considerable among the lowest achievers.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The coefficient for the 2022 dummy captures the departure of achievement in 2022 from country-specific time trends assuming no school closures.
- Findingown data
In countries with below-average lengths of school closures, learning loss is similar for low-, average-, and high-achieving students.
“For countries with below-average lengths of school closures, the learning loss is similar for low-, average-, and high-achieving students.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The coefficient for the 2022 dummy captures the departure of achievement in 2022 from country-specific time trends assuming no school closures.
- Findingown data
Using PISA, the study finds seven months' worth of losses.
“But using PISA, we find seven months’ worth of losses.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
For the average student, the loss ranges from around 14 points in countries with the shortest closures to 24 points in countries with the most prolonged closures.
“For the average student, the loss ranges from around 14 points in the countries with the shortest closures but increases to 24 in countries with the most prolonged closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
In countries with the most prolonged school closures, learning losses differ significantly between students at different achievement levels.
“Distributional analyses reveal significant differences in the learning losses between students at different achievement levels in countries with the most prolonged school closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The coefficient for the 2022 dummy captures the departure of achievement in 2022 from country-specific time trends assuming no school closures.
- Findingown data
For the lowest-achieving students, the loss ranges from 8 points in countries with the shortest closures to 33 points in countries with the most prolonged closures.
“For the lowest achieving students, the loss ranges from 8 points in the countries with the shortest closures to 33 points in countries with the most prolonged closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
In countries with lengthy school closures, learning loss is more extensive among average-achieving students than among high-achieving students.
“However, in countries with lengthy closures, the learning loss is more extensive among the average students and even more considerable among the lowest achievers.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The coefficient for the 2022 dummy captures the departure of achievement in 2022 from country-specific time trends assuming no school closures.
- Findingown data
For countries with more prolonged closures, declines are more significant among the lowest-achieving students.
“For countries with more prolonged closures, the declines are more significant among the lowest-achieving students, while in all countries, the best-achieving students lost around 8–9 points.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingnone given
In science, the overall effect of the pandemic is insignificant.
“In science, the overall effect of the pandemic is insignificant.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
In all countries, the best-achieving students lost around 8-9 points.
“For countries with more prolonged closures, the declines are more significant among the lowest-achieving students, while in all countries, the best-achieving students lost around 8–9 points.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
In reading, the effects associated with school closures tend to be similar to those for mathematics.
“Also, the effects associated with school closures tend to be similar.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Findingown data
In reading, the overall impact of the pandemic is similar to that for mathematics.
“In reading, the overall impact of the pandemic is similar to that for mathematics (Supplementary Table 5).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Interpretationreasoning
For countries with longer closures, the pandemic decreased the relationship between socioeconomic background and mathematics achievement.
“It means that socioeconomic disparities increased only in countries with relatively short closures, below 22 weeks, and for countries with longer closures the pandemic decreased the relationship between socioeconomic background and mathematics achievement.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Interpretationreasoning
The pandemic increased socioeconomic inequality as measured by the ESCS regression slope by around 7%.
“Thus, the pandemic increased socioeconomic inequality as measured by the ESCS regression slope by around 7%.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Interpretationreasoning
On average across around 70 countries, students lost the equivalent of more than 7 months of learning.
“Considering that one year of learning is equivalent to roughly 20 points on the PISA scale26, this means that on average across around 70 countries students lost an equivalent of more than 7 months of learning.”
- Also stated in: § Discussion · hedged
“roughly equivalent to 7 months of learning.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [26] Avvisati, F. & Givord, P. The learning gain over one school year among 15-year-olds: an international comparison based on PISA. Labour Econ. 84, 102365 (2023). source
- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Discussion · hedged
- Interpretationreasoning
Socioeconomic disparities increased only in countries with relatively short closures, below 22 weeks.
“It means that socioeconomic disparities increased only in countries with relatively short closures, below 22 weeks, and for countries with longer closures the pandemic decreased the relationship between socioeconomic background and mathematics achievement.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Study design and conductown data
Different regression specifications, controlling or not for individual and country-average effects, produced nearly identical results.
“To further test the robustness of our results, we use different regression specifications, controlling or not for individual and country-average effects. The results were nearly identical, demonstrating that the control variables included help to explain achievement variation but are not driving our results.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The GDP loss in nominal values using World Bank GDP 2021 data equals $17 trillion.
“This enormous GDP loss in nominal values using World Bank GDP 2021 data equals $17 trillion of economic loss.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Rests on an assumption: The length of school closures is not confounded with other pandemic-related factors affecting student achievement, so differences by closure length identify the effect of closures.
- Study design and conductown data
PISA mathematics results were relatively stable before the pandemic.
“This confirms two things. First, PISA results in mathematics were relatively stable before the pandemic.”
- Also stated in: § Robustness checks · hedged
“This indicates that PISA mathematics results were stable before the pandemic and that the decline in 2022 was not accidental.”
Quote in the paperSubstantial claim - Also stated in: § Robustness checks · hedged
- Premisecitation
Online tutoring programs were shown to reduce learning loss significantly in several randomized controlled trials.
“During the school closures, online tutoring programs were shown to reduce learning loss significantly in several randomized controlled trials33,34 but the evidence base is limited.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [33] Carlana, M. & La Ferrara, E. Apart but connected: Online tutoring and student outcomes during the COVID-19 pandemic. IZA Discussion Papers, 14094 (2021). source
- cites [34] Gortazar, L., Hupkau, C. & Roldán, A. Online tutoring works: Experimental evidence from a program with vulnerable children. ESADE Center for Economic Policy and School of Public Policy Working Paper (2022). source
- Premisecitation
The pandemic had a larger impact on math scores for students residing in high poverty areas.
“Moreover, the pandemic had a larger impact on math scores for students residing in high poverty areas46.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [46] Goldhaber, D. et al. The educational consequences of remote and hybrid instruction during the Pandemic. Am. Econ. Rev. 5, 377–392 (2023). source
- Study design and conductown data
Models with quadratic time trends and with different years dropped did not change the mathematics results substantially.
“Moreover, we estimated models with quadratic time trends, allowing for non-linear trends in achievement, and with different years dropped from the analyses. In both cases the results for mathematics did not change substantially.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The average learning loss across all educational systems is approximately 9 points.
“Using the UNESCO database, we estimate that the average learning loss across all educational systems is approximately 9 points.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Interpretationown data
The decline in 2022 was not accidental.
“Second, the results in 2022 are distinct as they do not follow the average results from previous rounds.”
- Also stated in: § Robustness checks · hedged
“This indicates that PISA mathematics results were stable before the pandemic and that the decline in 2022 was not accidental.”
Quote in the paperSubstantial claim - Also stated in: § Robustness checks · hedged
- Study design and conductown data
Placebo tests show that only the departure in 2022 is substantial.
“Figure 2 shows that although estimates for different years vary, representing small variation in results of different PISA rounds, only the departure in 2022 is substantial.”
- Also stated in: § Robustness checks · plain
“Only the departure in 2022 is substantial.”
Quote in the paperSubstantial claim - Also stated in: § Robustness checks · plain
- Premisecitation
High-dosage and one-to-one tutoring was shown to be effective before the pandemic.
“High-dosage and one-to-one tutoring was shown to be effective before the pandemic38; online tutoring variants might be effective and much less costly39 but more research is needed.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] Slavin, R. E., Lake, C., Davis, S. & Madden, N. A. Effective programs for struggling readers: a best-evidence synthesis. Educ. Res. Rev. 6, 1–26 (2011). source
- May combine two claims. 20
- Classified as probably not a substantial claim. 16
May combine two claims.
PremisecitationDistance learning during school closures does not seem to have helped much; only the duration of school closures led to variations.
“Distance learning during the school closures does not seem to have helped much21,22; only the duration of school closures led to variations.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [21] Haelermans, C., Jacobs, M., Van Der Velden, R., Van Vugt, L. & Van Wetten, S. Inequality in the effects of primary school closures due to the COVID-19 Pandemic. Am. Econ. Assoc. Pap. Proc. 112, 303–307 (2022). source
- cites [22] Singh, A., Romero, M. & Muralidharan, K. COVID-19 learning loss and recovery: panel data evidence from India. J. Hum. Resour. https://doi.org/10.3368/jhr.0723-13025R2 (2024). source
The cited paper does not say that distance learning made little difference. In Section III.A (page unavailable), it reports that most educational inputs available during closures were significantly predictive of learning changes; the Notes table also shows positive associations for video classes and several remote-learning resources. Those findings are contrary to the claim that only closure duration accounted for variation. (LLM, conf=0.82)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationLosses are greater for disadvantaged students, those struggling before the pandemic, and by duration.
“Losses are greater for disadvantaged students, those struggling before the pandemic, and by duration.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [24] Jakubowski, M., Gajderowicz, T. & Patrinos, H. A. Global learning loss in student achievement: first estimates using comparable reading scores. Econ. Lett. 232, 111313 (2023). source
- cites [25] Kennedy, A. I. & Strietholt, R. School closure policies and student reading achievement: evidence across countries. Educ. Assess. Eval. Account. 35, 475–501 (2023). source
Section 6.1 reports larger reading-achievement losses for socioeconomically disadvantaged students, and section 5.1 links longer school closures to larger declines. Section 6.1 suggests that students who usually trail their peers may have fallen further behind, but it does not establish greater losses for students identified as struggling before the pandemic independently of socioeconomic disadvantage or computer access. The paper's duration finding concerns school closures, not time elapsed during the pandemic generally. (LLM, conf=0.86)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationSystematic reviews and meta-analyses suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation, at least one-third of a year's worth of learning.
“Nevertheless, there are several systematic reviews and meta-analyses, and these suggest an average learning loss attributable to COVID-19 school closures of 0.17 of a standard deviation—that is, at least one-third of a year’s worth of learning16–20.”
Quote in the paperNot a substantial claim5 of 5 sources checked- cites [16] Betthäuser, B. A., Bach-Mortensen, A. M. & Engzell, P. A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nat. Hum. Behav. 7, 375–385 (2023). source
The Results section (page not specified) reports an average deficit of 0.14 standard deviations, equivalent to about 35% of a school year. The claimed 0.17 is the bound of the reported confidence interval, not the average. The paper measures learning deficits during the pandemic and identifies school closures as a major contributor, but does not attribute the entire pooled deficit specifically to closures. (LLM, conf=0.90)
Experimental — accuracy not yet measured - cites [17] Donnelly, R. & Patrinos, H. A. Learning loss during COVID-19: an early systematic review. Prospects 51, 601–609 (2021). source
The Analytical review (page unavailable) reports learning losses in seven of eight studies, supporting the general claim that COVID-19 school closures were associated with learning loss. It gives results for individual studies, but does not report an overall average of 0.17 standard deviations or equate that figure to at least one-third of a year’s learning. (LLM, conf=0.87)
Experimental — accuracy not yet measured - cites [18] Hammerstein, S., König, C., Dreisörner, T. & Frey, A. Effects of COVID-19-related school closures on student achievement: a systematic review. Front. Psychol. 12, 746289 (2021). source
The Results (General Effects; page not provided) report that most included studies found learning losses after COVID-19 school closures. But the review reports a median effect of −0.08 SD, not an average of −0.17 SD, and does not equate its findings to at least one-third of a year's learning. The Method (Synthesis; page not provided) also says a meta-analysis was not feasible. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [19] Zierer, K. Effects of pandemic-related school closures on pupils’ performance and learning in selected countries: a rapid review. Educ. Sci. 11, 252 (2021). source
Section 10 (page not provided) reports learning loss after school closures, but its mean effect is −0.14 standard deviations, not −0.17. It estimates lost learning time at 23%–35% of a school year, so the loss is not consistently at least one-third. The paper describes itself as a rapid review and says a comprehensive meta-analysis is still needed. (LLM, conf=0.90)
Experimental — accuracy not yet measured - cites [20] Patrinos, H., Vegas, E. & Carter-Rau, R. An analysis of COVID-19 student learning loss. Oxford Research Encyclopedia of Economics and Finance (Oxford, 2023). source
May combine two claims.
PremisecitationUntil now, there has only been one international student assessment, covering only fourth grade students in just 55 countries.
“However, until now, there has only been one international student assessment. It covered only fourth grade students in just 55 countries.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [24] Jakubowski, M., Gajderowicz, T. & Patrinos, H. A. Global learning loss in student achievement: first estimates using comparable reading scores. Econ. Lett. 232, 111313 (2023). source
- cites [25] Kennedy, A. I. & Strietholt, R. School closure policies and student reading achievement: evidence across countries. Educ. Assess. Eval. Account. 35, 475–501 (2023). source
Section 3.2 (Sample; page unavailable) identifies PIRLS as an international assessment focused on fourth-grade students, supporting that part of the claim. But the paper analyzes 29 countries, not a stated total of 55, and Section 6 (Conclusion; page unavailable) calls PIRLS 2021 “one of the first” international assessments after the pandemic, not the only one. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationcitationThe study's results are similar to what systematic reviews are finding.
“Interestingly, our results are similar to what systematic reviews are finding learning16–20,44,45.”
Quote in the paperNot a substantial claim6 of 7 sources checked- cites [16] Betthäuser, B. A., Bach-Mortensen, A. M. & Engzell, P. A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nat. Hum. Behav. 7, 375–385 (2023). source
- cites [17] Donnelly, R. & Patrinos, H. A. Learning loss during COVID-19: an early systematic review. Prospects 51, 601–609 (2021). source
- cites [18] Hammerstein, S., König, C., Dreisörner, T. & Frey, A. Effects of COVID-19-related school closures on student achievement: a systematic review. Front. Psychol. 12, 746289 (2021). source
- cites [19] Zierer, K. Effects of pandemic-related school closures on pupils’ performance and learning in selected countries: a rapid review. Educ. Sci. 11, 252 (2021). source
- cites [20] Patrinos, H., Vegas, E. & Carter-Rau, R. An analysis of COVID-19 student learning loss. Oxford Research Encyclopedia of Economics and Finance (Oxford, 2023). source
- cites [44] Di Pietro, G. The impact of Covid-19 on student achievement: evidence from a recent meta-analysis. Educ. Res. Rev. 39, 100530 (2023). source
- cites [45] Storey, N. & Zhang, Q. A meta-analysis of the impact of COVID-19 on student achievement. Educ. Res. Rev 44, 100624 (2024). source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThese economic estimates are similar to other studies.
“These estimates are similar to other studies23,30,31.”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [23] Psacharopoulos, G., Collis, V., Patrinos, H. A. & Vegas, E. The COVID-19 cost of school closures in earnings and income across the world. Comp. Educ. Rev. 65, 271–287 (2021). source
- cites [30] Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: a set of global estimates. World Bank Res. Observer 36, 1–40 (2021).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [31] Doty, E., Kane, T. J., Patterson, T. & Staiger, D. O. What do changes in state test scores imply for later life outcomes? Natl Bureau Econ. Res. Work. Paper w30701 (2022). source
May combine two claims.
PremisecitationGlobal studies document significant learning losses; international reading scores declined by an average of 33 percent of a standard deviation, equivalent to more than a year of schooling.
“Also, global studies—that is, studies using data from international assessments where students in many countries take the same tests and under the same conditions, thus, making them comparable across country and time—document significant learning losses24,25. International reading scores declined by an average of 33 percent of a standard deviation, equivalent to more than a year of schooling.”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [24] Jakubowski, M., Gajderowicz, T. & Patrinos, H. A. Global learning loss in student achievement: first estimates using comparable reading scores. Econ. Lett. 232, 111313 (2023). source
- cites [25] Kennedy, A. I. & Strietholt, R. School closure policies and student reading achievement: evidence across countries. Educ. Assess. Eval. Account. 35, 475–501 (2023). source
The Introduction and §6.1 support the claim that international assessment data show significant reading declines associated with school closures. But §6.1 estimates an average effect of 0.15 standard deviations for the sampled countries, not 0.33, and estimates that even a full year of closures would correspond to about 57% of a school year’s learning—not more than a year. The cited passages provide no page numbers. (LLM, conf=0.90)
Experimental — accuracy not yet measured
May combine two claims.
Findingown dataAfter the pandemic, the relationship between achievement and student socioeconomic background became slightly stronger; the slope of the ESCS index increased by 2.62 points.
“After the pandemic, the relationship between achievement and student socioeconomic background became slightly stronger. The slope of the ESCS index measuring socioeconomic status in PISA increased by 2.62 points.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
May combine two claims.
Findingown dataStudents in countries with the longest school closures, at the 90th percentile, lost 15 points or more, or 17-23% of a standard deviation, equivalent to 9-12 months of learning.
“students in countries with the longest closures (90th percentile) lost 15 points or more (17–23% SD).”
- Also stated in: § Learning loss in mathematics · plain
“Countries with the longest closures, experienced losses of 17–23% SD, or 9–12 months of learning.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Learning loss in mathematics · plain
May combine two claims.
Findingown dataStudents in countries with average length of school closures lost around 12 points, or 13-14% of a standard deviation, equivalent to 7 months of learning.
“Students in countries with average closures lost around 12 points (13–14% SD)”
- Also stated in: § Learning loss in mathematics · plain
“Countries with average length of school closures experienced losses of 13–14% SD, or 7 months of learning.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Also stated in: § Learning loss in mathematics · plain
May combine two claims.
Findingown dataAll four measures of the length of school closures are correlated, with correlations from 0.6 to 0.8.
“All measures are correlated (from 0.6 to 0.8), but as the definitions and data collection modes vary, the actual numbers are different.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningAt around 40 weeks of full school closures, the overall learning loss for boys is the same as for girls and increases for longer closures.
“At around 40 weeks of full school closures, the overall learning loss for boys is the same as for girls and increases for longer closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
May combine two claims.
InterpretationreasoningThe achievement decline is associated with school closures and not only the overall impact of the pandemic on students.
“This shows that the achievement decline is associated with school closures and not only the overall impact of the pandemic on students.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Rests on an assumption: The length of school closures is not confounded with other pandemic-related factors affecting student achievement, so differences by closure length identify the effect of closures.
May combine two claims.
Interpretationown dataFor countries with around 25 weeks of closures, immigrant students' learning loss equals that for natives, but it increases for longer closures.
“Students with an immigrant background lost 0.33 points with every week of full school closures and while in countries with shorter closures their learning loss was smaller, for countries with around 25 weeks of closures their learning loss is equal to that for natives, but it increases for longer closures.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
May combine two claims.
InterpretationreasoningThe average loss of 9 points for 199 education systems translates to an average of 0.15 percentage points of GDP growth losses, ranging from zero to 0.57 points.
“The average loss of 9 points (9% of a SD) for 199 education systems (recognized by the UNESCO school closure database) translates to an average of 0.15 percentage points of GDP growth losses, ranging from zero (if schools remained open) up to 0.57 points.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [29] Hanushek, E. A., Woessmann, L. The High Cost of Low Educational Performance (OECD Publishing, Paris, 2010).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
- Rests on an assumption: The length of school closures is not confounded with other pandemic-related factors affecting student achievement, so differences by closure length identify the effect of closures.
May combine two claims.
Study design and conductown dataIn reading and science, the time trends are not linear, and one cannot distinguish between the effects of the pandemic and the long-term decline in achievement.
“Our robustness checks for reading and science results confirm that indeed in these two domains the time trends are not linear, and one cannot distinguish between the effects of the pandemic and the long-term decline in achievement.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThese learning losses could translate to earnings losses and could cost this generation of students trillions of dollars.
“These learning losses could translate to earnings losses and could cost this generation of students trillions of dollars23.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23] Psacharopoulos, G., Collis, V., Patrinos, H. A. & Vegas, E. The COVID-19 cost of school closures in earnings and income across the world. Comp. Educ. Rev. 65, 271–287 (2021). source
Classified as probably not a substantial claim.
Study design and conductreasoningThe study controls for individual student gender, socioeconomic status, immigrant background, and country-averages of these indicators to exclude potential biases from changing student samples.
“In addition, we control for individual student gender, socioeconomic status, immigrant background, and we also control for country-averages of these indicators to exclude any potential biases arising from changing student samples.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA causal estimate of the impact of duration of school closures finds that for every week that schools were closed, learning levels declined by almost 1% of a standard deviation.
“A causal estimate of the impact of duration of school closures finds that for every week that schools were closed, learning levels declined by almost 1% of a standard deviation1.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [1] Patrinos, H. A. The longer students were out of school, the less they learned. J. Sch. Choice 17, 161–175 (2023). source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningThe study compares results on four measures of school closure length to check robustness of the findings to the definition of school closure and data source.
“We compare results on these four measures to check robustness of the findings to the definition of school closure and data source.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenReading and science were minor domains in PISA 2022, meaning the measurement of student achievement was less precise than in mathematics.
“We can only note that both reading and science were minor domains in PISA 2022, meaning the measurement of student achievement was less precise than in mathematics.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataLearning loss increases with the length of school closures for average- and lowest-achieving students but is similar across countries for the best students.
“In other words, the learning loss increases with the length of school closures for the average- and the lowest-achieving students but is similar across countries for the best students.”
Quote in the paperSubstantial claim- The authors state this as an assumption: In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe study estimates each country's linear trend separately and includes country-level fixed effects to control for unobserved time-invariant country characteristics.
“We estimate each country’s linear trend separately and include country-level fixed effects to control for unobserved time-invariant country characteristics.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe study performs multiple robustness checks and placebo tests.
“We perform multiple robustness checks to see if our results hold depending on regression specifications and data selection. We also run placebo tests to see if our regression model identifies similar effects assuming false-treatment in other years.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe study performs multiple robustness checks to test whether the main estimates of learning loss are robust to different regression and sample specifications.
“Thus, we perform multiple robustness checks.”
- Also stated in: § Empirical strategy · plain
“We compare these results to see if the main estimates of the learning loss are robust to different regression and sample specifications.”
Quote in the paperSubstantial claim- Also stated in: § Empirical strategy · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningMany students taking PISA in 2022 were in different schools during the pandemic, for example, in primary schools that could experience different closures.
“Finally, while principals report closures for their schools, many students taking PISA in 2022 where in different schools during the pandemic, for example, in primary schools that could experience different closures.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductnone givenPISA-provided data on school closures are self-reported, relying on people's memory and their understanding of what school closure means.
“However, these data are self-reported relying on people’s memory and their understanding of what school closure means.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study estimates measurement errors by estimating variation among point estimates obtained for different plausible values.
“In addition, we estimate measurement errors by estimating variation among point estimates obtained for different plausible values.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductown dataThe study uses country-level fixed effects to control for unobserved time-invariant country characteristics.
“We estimate each country’s linear trend separately and include country-level fixed effects to control for unobserved time-invariant country characteristics.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe final standard errors are obtained by Rubin's formula, the same way as in the official OECD publication.
“The final standard errors are obtained by the so-called Rubin’s formula, the same way as in the official OECD publication27.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [27] OECD. PISA 2022 Results (Volume I): The State of Learning and Equity in Education. (OECD Publishing, Paris, 2023). source
Classified as probably not a substantial claim.
Study design and conductreasoningComparing estimates across school closure measures assures that the results are not driven by data sources or definitions.
“Comparing estimates across school closure measures assures that our results are not driven by data sources or definitions (Table 1).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe study uses replicate balanced repeated replication (BRR) weights to obtain sampling errors.
“We use the sets of replicate balanced repeated replication (BRR) weights provided in the data to obtain sampling errors.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
In the absence of the COVID-19 pandemic and school closures, student achievement trends in mathematics would have remained stable and linear, so the 2022 deviation from the pre-pandemic linear trend can be attributed to the pandemic and school closures.
“assuming no school closures”
“if our assumption of stable trends over time is valid”
The paper relies on this without support.
The length of school closures is not confounded with other pandemic-related factors affecting student achievement, so differences by closure length identify the effect of closures.
“This shows that the achievement decline is associated with school closures and not only the overall impact of the pandemic on students.”
The authors state this as an assumption.
The coefficient for the 2022 dummy captures the departure of achievement in 2022 from country-specific time trends assuming no school closures.
“In this case, captures the departure of achievement in 2022 from country-specific time trends assuming no school closures.”
The paper relies on this without support.
Some country studies reported more significant losses among low-achieving students, while others showed similar losses across the achievement spectrum.
“Some country studies reported more significant losses among low-achieving students, while others showed similar losses across the achievement spectrum.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Abstract reverses the reported comparison for boys and immigrant students
MajorThe abstract says losses were greater for boys and immigrants, but the results say both groups had lower learning losses than girls and natives overall. The results distinguish this overall comparison from the more negative effect of longer closures on those groups.
“Losses are greater for students in schools that faced relatively longer closures, for boys, immigrants, and disadvantaged students.”
“While boys and immigrant students experienced a lower learning loss compared to girls and natives, nevertheless, longer school closures had a more negative effect on both groups (see Supplementary Table 4).”
Cites
Unconditional Quantile Regressions
FIRPO, S. P., Fortin, N. M., & Lemieux, T. (2009) · Econometrica · doi:10.3982/ecta6822 · Cited by 2,256 · see citing works
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2022376118 · Cited by 1,848 · cited 1× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2020) · doi:10.31235/osf.io/ve4z7 · Cited by 1,739 · see citing works
(2022) · doi:10.1787/53f23881-en · Cited by 1,507 · see citing works
The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review
Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996) · Review of Educational Research · doi:10.3102/00346543066003227 · Cited by 991 · see citing works
A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic
Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01506-4 · Cited by 568 · cited 2× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
InfluentialThe effect of school closures on standardised student test outcomes
Maldonado, J. E., & De Witte, K. (2022) · British Educational Research Journal · doi:10.1002/berj.3754 · Cited by 499 · see citing works
Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.746289 · Cited by 461 · cited 1× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Learning loss during Covid-19: An early systematic review
Donnelly, R., & Patrinos, H. A. (2022) · PROSPECTS · doi:10.1007/s11125-021-09582-6 · Cited by 445 · cited 1× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Self-citationEffective programs for struggling readers: A best-evidence synthesis
Slavin, R. E., Lake, C., Davis, S., & Madden, N. A. (2011) · Educational Research Review · doi:10.1016/j.edurev.2010.07.002 · Cited by 353 · see citing works
Measuring human capital using global learning data
Angrist, N., Djankov, S., Goldberg, P. K., & Patrinos, H. A. (2021) · Nature · doi:10.1038/s41586-021-03323-7 · Cited by 328 · cited 1× · see citing works
“Early on, the school closures were expected to contribute to what was already described as a learning crisis (Angrist et al. 2021; World Bank 2017).”
Self-citationThe impact of Covid-19 on student achievement: Evidence from a recent meta-analysis
Di Pietro, G. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100530 · Cited by 173 · see citing works
von Hippel, P., & Hamrock, C. (2019) · Sociological Science · doi:10.15195/v6.a3 · Cited by 149 · see citing works
Zierer, K. (2021) · Education Sciences · doi:10.3390/educsci11060252 · Cited by 118 · cited 1× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Higher chronic absenteeism threatens academic recovery from the COVID-19 pandemic
Dee, T. S. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2312249121 · Cited by 110 · see citing works
Experimental evidence on learning using low-tech when school is out
Angrist, N., Bergman, P., & Matsheng, M. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01381-z · Cited by 105 · cited 1× · see citing works
“A large-scale randomized trial testing low-technology interventions – SMS messages and phone calls – with parents to support their child in Botswana improved learning by 0.12 standard deviation (Angrist et al. 2022).”
InfluentialAn Analysis of COVID-19 Student Learning Loss
Patrinos, H. A., Vegas, E., & Carter-Rau, R. (2022) · World Bank policy research working paper · doi:10.1596/1813-9450-10033 · Cited by 105 · see citing works
Self-citationThe COVID-19 Cost of School Closures in Earnings and Income across the World
Psacharopoulos, G., Collis, V., Patrinos, H. A., & Vegas, E. (2021) · Comparative Education Review · doi:10.1086/713540 · Cited by 98 · cited 2× · see citing works
“These learning losses could translate to earnings losses and could cost this generation of students trillions of dollars (Psacharopoulos et al. 2021).”
Self-citationSurprising new evidence on summer learning loss
Kuhfeld, M. (2019) · Phi Delta Kappan · doi:10.1177/0031721719871560 · Cited by 93 · see citing works
Azevedo, J. P., Hasan, A., Goldemberg, D., et al. (2021) · The World Bank Research Observer · doi:10.1093/wbro/lkab003 · Cited by 86 · see citing works
Apart But Connected: Online Tutoring and Student Outcomes During the Covid-19 Pandemic
Carlana, M., & La Ferrara, E. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3785058 · Cited by 72 · see citing works
Schuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S., & Hornstra, L. (2021) · Scandinavian Journal of Educational Research · doi:10.1080/00313831.2021.2006307 · Cited by 67 · see citing works
COVID-19, school closures, and outcomes.
Jack, R., & Oster, E. (2023) · Journal of Economic Perspectives · doi:10.1257/jep.37.4.51 · Cited by 57 · cited 1× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Global learning loss in student achievement: First estimates using comparable reading scores
Jakubowski, M., Gajderowicz, T., & Patrinos, H. A. (2023) · Economics Letters · doi:10.1016/j.econlet.2023.111313 · Cited by 51 · cited 5× · see citing works
“This could be because more time has passed since the pandemic: PIRLS compared results over time to 2021, while PISA looked at results up to 2022.”
InfluentialSelf-citationThe Educational Consequences of Remote and Hybrid Instruction during the Pandemic
Goldhaber, D., Kane, T. J., McEachin, A., Morton, E., Patterson, T., & Staiger, D. O. (2023) · American Economic Review: Insights · doi:10.1257/aeri.20220180 · Cited by 49 · see citing works
School closure policies and student reading achievement: evidence across countries
Kennedy, A. I., & Strietholt, R. (2023) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-023-09415-4 · Cited by 45 · cited 2× · see citing works
“There are only two other international comparisons using global achievement data, both using PIRLS (Jakubowski et al. 2023; Kennedy and Strietholt 2023).”
An Analysis of COVID-19 Student Learning Loss
Patrinos, H., Vegas, E., & Carter-Rau, R. (2023) · Oxford Research Encyclopedia of Economics and Finance · doi:10.1093/acrefore/9780190625979.013.893 · Cited by 42 · see citing works
Self-citationApart but Connected: Online Tutoring and Student Outcomes during the COVID-19 PandemicPreprint
Carlana, M., & Ferrara, E. L. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3777556 · Cited by 38 · see citing works
Building Resilient Education Systems: Evidence from Large-Scale Randomized Trials in Five Countries
Angrist, N., Ainomugisha, M., Bathena, S. P., Bergman, P., Crossley, C., Cullen, C., Letsomo, T., Matsheng, M., Panti, R. M., Sabarwal, S., et al. (2023) · National Bureau of Economic Research · doi:10.3386/w31208 · Cited by 37 · cited 1× · see citing works
“Even low-cost, low-technology interventions can recover learning losses in some cases (Angrist et al. 2023), but more intense instruction will be needed in others (Crawfurd et al. 2023).”
The Longer Students Were Out of School, the Less They Learned
Patrinos, H. A. (2023) · Journal of School Choice · doi:10.1080/15582159.2023.2210941 · Cited by 35 · cited 1× · see citing works
“In some cases, they quickly reopened in 2021, but in some countries, they remained closed longer (Patrinos 2023).”
Self-citationThe learning gain over one school year among 15-year-olds: An international comparison based on PISA
Avvisati, F., & Givord, P. (2023) · Labour Economics · doi:10.1016/j.labeco.2023.102365 · Cited by 32 · cited 1× · see citing works
“Considering that one year of learning is equivalent to roughly 20 points on the PISA scale (Avvisati and Givord 2023), then this means that on average across around 70 countries students lost an equivalent of more than 7 months of learning.”
Online Tutoring Works: Experimental Evidence from a Program with Vulnerable Children
Gortazar, L., Hupkau, C., & Roldán, A. (2023) · doi:10.2139/ssrn.4390248 · Cited by 32 · see citing works
Learning losses during the COVID-19 pandemic: Evidence from Mexico
Alasino, E., Ramírez, M. J., Romero, M., Schady, N., & Uribe, D. (2023) · Economics of Education Review · doi:10.1016/j.econedurev.2023.102492 · Cited by 30 · cited 1× · see citing works
“Country studies show large losses (Alasino et al. 2024; Betthäuser et al. 2023; Carlana and La Ferrara 2023; Donnelly and Patrinos 2021; Engzell et al. 2021; Hammerstein et al. 2021; Jack and Oster 2023; Jack et al 2022; Maldonado and De Witte 2022; Zierer 2021).”
Exacerbated Inequalities: The Learning Loss from COVID-19 in Italy
Carlana, M., Ferrara, E. L., & Lopez, C. (2023) · AEA Papers and Proceedings · doi:10.1257/pandp.20231010 · Cited by 21 · see citing works
Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone
Crawfurd, L., Evans, D. K., Hares, S., & Sandefur, J. (2023) · Journal of Development Economics · doi:10.1016/j.jdeveco.2023.103114 · Cited by 19 · cited 1× · see citing works
“Even low-cost, low-technology interventions can recover learning losses in some cases (Angrist et al. 2023), but more intense instruction will be needed in others (Crawfurd et al. 2023).”
Post COVID-19 Test Score Recovery: Initial Evidence from State Testing Data
Halloran, C., Hug, C., Jack, R. S., & Oster, E. F. (2023) · National Bureau of Economic Research · doi:10.3386/w31113 · Cited by 19 · see citing works
A meta-analysis of the impact of COVID-19 on student achievement
Storey, N., & Zhang, Q. (2024) · Educational Research Review · doi:10.1016/j.edurev.2024.100624 · Cited by 18 · see citing works
COVID-19 Learning Loss and Recovery: Panel Data Evidence from India
Singh, A., Romero, M., & Muralidharan, K. (2026) · Journal of Human Resources · doi:10.3368/jhr.0723-13025r2 · Cited by 15 · see citing works
Haelermans, C., Jacobs, M., van der Velden, R., van Vugt, L., & van Wetten, S. (2022) · AEA Papers and Proceedings · doi:10.1257/pandp.20221107 · Cited by 13 · see citing works
Overview: Learning to realize education’s promise
Bank, W. (2017) · World Bank, Washington, DC eBooks · doi:10.1596/978-1-4648-1096-1_ov · Cited by 13 · see citing works
What Do Changes in State Test Scores Imply for Later Life Outcomes?
Doty, E., Kane, T., Patterson, T., & Staiger, D. (2022) · doi:10.3386/w30701 · Cited by 11 · cited 1× · see citing works
“These estimates are similar to other studies (see, for example, Azevedo et al. 2021; Doty et al. 2022; Psacharopoulos et al. 2021).”
Wise, S. L., Kuhfeld, M. R., & Cronin, J. B. (2022) · Educational Assessment · doi:10.1080/10627197.2022.2087621 · Cited by 9 · see citing works
Reading achievement progress across countries
Jakubowski, M., & Pokropek, A. (2015) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2015.09.011 · Cited by 8 · see citing works
Self-citationCapturing the Educational and Economic Impacts of School Closures in PolandPreprint
Gajderowicz, T. J., Jakubowski, M., Patrinos, H. A., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4298822 · Cited by 3 · see citing works
Self-citationcapturing the Educational and Economic Impacts of School Closures in Poland
Gajderowicz, T. J., Jakubowski, M., Patrinos, H. A., & Wrona, S. (2022) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-10253 · Cited by 1 · see citing works
Self-citationAssessment in the Time of COVID-19
Wise, S. L., Kuhfeld, M., & Cronin, J. (2023) · Twin Pandemics · doi:10.4324/9781003431145-4
Building Resilient Education Systems: Evidence from Large-Scale Randomized Trials in Five Countries
Angrist, N., Ainomugisha, M., Bathena, S. P., et al. (2023) · doi:10.2139/ssrn.4440703
Flagged first, then by citations
Research works printed but not linked: 3
- Jack, R., Halloran, C., Okun, J. & Oster, E. Pandemic schooling mode and student test scores. Am. Econ. Rev. 5, 173–190 (2022).
- Von Hippel, P. T. Is summer learning loss real? How I lost faith in one of education research’s classic results. Education 19, 8–15 (2019).
- Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: a set of global estimates. World Bank Res. Observer 36, 1–40 (2021).
Cited by
Sorted by citations
Le, Z., Lou, Y., Tao, Y., Wang, H., & Xu, N. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1599602 · Cited by 14 · see citing works
“Various factors, including social isolation, heightened academic pressure, family conflicts, and limited access to mental health services, have collectively contributed to the increase in depressive disorders during this period (22, 23).”
The impact of the pandemic and school closures on non-cognitive characteristics: Evidence from PISA
Pena, J. W. Q., Volante, L., & Witte, K. D. (2025) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2025.100489 · Cited by 15 · see citing works
Repo, J., Reimer, D., & Kilpi‐Jakonen, E. (2025) · Large-scale Assessments in Education · doi:10.1186/s40536-025-00251-0 · Cited by 7 · see citing works
“During distance education, family resources have been shown to play a significant role in adolescents’ ability to thrive and maintain academic progress (Jakubowski et al., 2025; Reim et al., 2024).”
Felix, E. D., & Green, J. G. (2025) · Current Psychiatry Reports · doi:10.1007/s11920-025-01642-4 · Cited by 8 · see citing works
Newson, N. K., Begeny, J. C., Wang, J., Jordán, Y. P., Codding, R. S., & Kromminga, K. R. (2025) · Psychology in the Schools · doi:10.1002/pits.70034 · Cited by 2 · see citing works
“…examining student performance both nationally (The Annie E. Casey Foundation 2024; Donnelly and Patrinos 2022; Locke et al. 2021) and internationally (Jakubowski et al. 2025) — suggest that in the initial years of the pandemic, students did not follow learning trajectories commensurate with what…”
Courtney, M. G. R., Cespedes, A. A., Kalemeneva, D., de Crombrugghe, D., Zhakibayeva, A., & Baikenova, A. (2025) · Prospects · doi:10.1007/s11125-025-09746-8 · Cited by 2 · see citing works
“However, it was not unusual for performance on standardized tests to fluctuate in a statistically significant manner during the pre-pandemic period (see international PISA math scores, Jakubowski et al., 2025, p. 3, Figure 2).”
Diagnosing specific learning disorder in adults Part II: bias, base rates, and a path forward
Harrison, A. G., & sparks, R. L. (2026) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2026.2654023 · Cited by 2 · see citing works
Huang, H., Luo, Y., Huang, Q. R., Chen, Z., Yang, X., Tang, J., Tang, W., & Tian, X. (2026) · BMC Nursing · doi:10.1186/s12912-025-04261-9 · Cited by 1 · see citing works
“Thus, understanding the global mechanisms of nursing students’ adaptation has emerged as an internationally urgent priority in post-pandemic educational reconstruction [8].”
Steinmann, I., Medgyesi, M., & Symeonaki, M. (2026) · Large-scale Assessments in Education · doi:10.1186/s40536-025-00274-7 · Cited by 1 · see citing works
“In light of previous research, educational disruptions due to the Covid pandemic that exacerbated educational inequalities might explain this increase in 2021 (Jakubowski et al., 2025; Kennedy & Strietholt, 2023).”
González, M., Tórtora, G., & Vásquez‐Echeverría, A. (2026) · Child Development · doi:10.1093/chidev/aacag067 · Cited by 1 · see citing works
Nusair, L. (2026) · Compare A Journal of Comparative and International Education · doi:10.1080/03057925.2026.2657825 · Cited by 1 · see citing works
Gilleece, L., Karakolidis, A., & Clerkin, A. (2026) · Large-scale Assessments in Education · doi:10.1186/s40536-026-00305-x · Cited by 1 · see citing works
“Findings from PISA show that the strength of the association between ESCS and achievement was impacted internationally, with socioeconomic disparities increasing in countries with shorter school closures and decreasing in countries with longer closures (Jakubowski et al., 2025).”
Golle, J., Lösche, P., Beißert, H. M., Wieland, A.-K., Rieber, R., Schult, J., Hasselhorn, M., Nagengast, B., & Trautwein, U. (2026) · npj Science of Learning · doi:10.1038/s41539-026-00439-1
Atuhurra, J., & Nishimura, M. (2025) · International Journal of Educational Research · doi:10.1016/j.ijer.2025.102583
Science Achievement in Gulf Cooperation Council (GCC) Countries
Alshaya, F. S., Jakubowski, M., Mansour, N., & Pagliarani, S. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-5743-9.ch002
Self-citationThe Impact of COVID-19 in Brazil Through an Educational Neuroscience Lens: A Preliminary Study
Fonseca, C. G., Dias, C., Barbosa, M. L. L., Hermida, M. J., Carreiro, L. R. R., & Seabra, A. G. (2025) · Brain Sciences · doi:10.3390/brainsci15060548
“Broader evidence from PISA data [5] quantified mathematics losses at 14% of a standard deviation (approximately 7 months), disproportionately affecting marginalised groups and correlating with closure duration.”
The Impact of Campus Teaching Programs on Elementary Students' Literacy Skills
Azizah, R. I. N., & Mahendra, Y. (2025) · MIMBAR PGSD Undiksha · doi:10.23887/jjpgsd.v13i1.91212
Fitriyana, R., Setyawati, A., Lestari, S., & Djatmiko (2025) · Journal of Transdisiplinary Studies in Education · doi:10.64268/jtse.v1i1.9
Kennedy, A., Mejía‐Rodríguez, A. M., & Strietholt, R. (2025) · Learning and Instruction · doi:10.1016/j.learninstruc.2025.102243 · Cited by 1 · see citing works
Teaching Techniques to Cover Academic Gaps in Modern Conditions
Kosharna, N. V., Petryk, L., Dzhurylo, A., Rudnik, Y., & Sytnyk, O. (2025) · International Journal on Culture History and Religion · doi:10.63931/ijchr.v7isi1.2.422 · Cited by 1 · see citing works
“Prolonged periods of distance learning and the shortage of qualified academic and teaching staff negatively impact on the learning process (Della Cruz et al., 2025; Jakubowski et al., 2025; Pasquini, 2025).”
The trend of science education research post-COVID-19 pandemic
Fatkhurrohman, M. A., Hamidah, I., Suhandi, A., & Samsudin, A. (2025) · Multidisciplinary Reviews · doi:10.31893/multirev.2026305
“This indicates that global science education research post-COVID is focused on humans (students) affected by the COVID-19 pandemic (Jakubowski et al., 2025).”
Anders, J., & Holt-White, E. (2026) · Social Indicators Research · doi:10.1007/s11205-025-03776-7
Socioeconomic achievement gap in Taiwan: The impact of COVID-19 on math performance
Tsou, T. (2026) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2026.103536
Helal, A. M., & Wai, J. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14030036 · Cited by 1 · see citing works
Pedro, J. R. S., & Fernandez, A. (2026) · Research Square · doi:10.21203/rs.3.rs-8992386/v1
Hsu, A., Zech, J. M., Zerbe, A., Samaniego, J. A., Maramag, C. C., Castro, M. C., Roquero, L. B., & Abrams, E. J. (2026) · BMC Public Health · doi:10.1186/s12889-026-27305-4
“Other studies found learning loss and decreases in academic performance due to school closures, reflecting the results of perceived learning loss in participants of our study [1, 4, 7, 29].”
Hudson, A. K., Fuchs, D., Artzi, L., et al. (2026) · Reading & Writing Quarterly · doi:10.1080/10573569.2026.2643161
“Although COVID-19 learning loss is well documented (e.g., Jakubowski et al., 2025), it affected U.S. students unevenly, depending on factors such as gender, socio-economic status, and prior achievement.”
Interrogating educational innovation: A scoping review
Egido, M. P., Quilabert, E., Matarranz, M., & Sánchez‐Tarazaga, L. (2026) · European Educational Research Journal · doi:10.1177/14749041261447100
Delprato, M. (2026) · Cogent Education · doi:10.1080/2331186x.2026.2684391
“This new component allows to gather information on school closures’ association with performance (Jakubowski et al., 2025), the impact of the pandemic on students’ wellbeing, belonging and engagement (Li et al., 2025), the roles of digitalisation in educational systems for remote learning (Xu et…”
Becker, D., & Coyle, T. R. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1734988 · Cited by 1 · see citing works
Education lost, labor gained: A systematic review of COVID-19's impact on children in South Asia
Muromba, P., & Keeni, M. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.102949
Aydemir, N. E., & Şahin, A. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14070137
“Furthermore, while countries with the shortest school closure periods during the COVID-19 pandemic experienced relatively less learning loss, those with the longest closure periods experienced greater learning loss (Jakubowski et al., 2025).”
Who lost the most? COVID-19 school closures, PISA achievement, and socioeconomic gaps
Romanello, M., Lamprianou, I., & Michaelides, M. P. (2026) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2026.101652
Henrich, M., Blatt, M., Zielonka, T., et al. (2026) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000005140
When the adjustment fails: population-denominator sensitivity in coverage-adjusted PISA trendsPreprint
Mandalunes, J. A., & Mandalunes, D. J. S. (2026) · Research Square · doi:10.21203/rs.3.rs-10706435/v1
Pitsia, V., & Karakolidis, A. (2026) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2026.101664 · Cited by 1 · see citing works
Asgary, A., & Nayebi, M. (2026) · Springer eBooks · doi:10.1007/978-3-032-30647-0_6
Soares, P. S. M., & Piva, M. E. S. D. (2026) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-026-03213-1
The NUS–Lancet PRIME Commission: transforming pandemic readiness for equity
Calderón‐Larrañaga, S., Bakhti, R., Etemadi, M., Fakih, L., Vandeskog, H. O., Shrestha, P., He, S., Chua, A. Q., Heymann, D., Tangcharoensathien, V., et al. (2026) · The Lancet · doi:10.1016/s0140-6736(26)01146-3 · Cited by 1 · see citing works
- Authors
- Jakubowski, M., Gajderowicz, T., & Patrinos, H. A.
- Affiliations
- Educational Research Institute (PL)
- University of Arkansas at Fayetteville (US)
- Corresponding author
- Jakubowski, Maciej
- Venue
- npj Science of Learning
- Published
- 2025-01-22
- Volume
- 10
- Issue
- 1
- Pages
- 5-5
- ISSN
- 2056-7936 (ISSN-L)
- PMC
- PMC11754741
- OpenAlex
- W4406731361
- Semantic Scholar
- c891dfe0b33af17179898f1c8e5daf939edc83f7
- Cited by
- 39
- References
- 38
- Citations by year
- 2026: 25 · 2025: 14
- FWCI
- 131.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- npj Science of Learning (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Keywords
- COVID-19 pandemic, school closures, learning loss, academic performance, PISA results, mathematics scores, educational disruption, immigrant students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
