The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; none of the 34 sources its claims cite is checked for support yet.
Did Students Learn Less During the COVID-19 Pandemic? Reading and Mathematics Competencies Before and After the First Pandemic Wave
Preprint — published as 10.1080/09243453.2022.2061014 · Paper page
Abstract
The first wave of the COVID-19 pandemic disrupted regular classes in spring 2020. Temporary school closures supposedly led to a considerable learning loss, particularly for low-achieving students. Schools in Baden-Württemberg, Germany, were closed for two months. Although distance learning was implemented, students spent less time learning. Additionally, teachers were faced with organizational and technological challenges of remote learning environments. The present study investigates the competencies of fifth-graders, using large-scale assessment results in reading and mathematics from annual mandatory tests in September (each n > 80,000). In line with studies from other countries, competence scores were slightly lower in 2020 compared with the three previous years (–0.07 standard deviations for reading comprehension, –0.09 for operations, and –0.03 for numbers). Low-achieving readers managed to attain pre-pandemic competence levels. Regarding mathematics competencies, low-achieving students seem to have a learning backlog that deserves attention in future education. School characteristic such as the average socio-cultural capital and the proportion of students with migration background played a minor role in mediating the schools' learning loss. Still, lower socio-cultural capital was positively associated with larger learning loss in mathematics.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Using statewide mandatory “Lernstand 5” data from Baden-Württemberg, this study compared incoming fifth-graders’ reading and mathematics competencies in September 2020 with those from 2017–2019. It found small average declines after the first COVID-19 school closures, with heterogeneous patterns across achievement levels and domains.
Key findings
- Population means were slightly lower in 2020 than in 2017–2019: reading comprehension *d* = −0.07, operations *d* = −0.09, and numbers *d* = −0.03 (all *p* < .001).
- Percentile comparisons showed mixed patterns: in reading, the bottom 5% was similar (2017–2019: 299 vs. 2020: 300), while the 25th–95th percentiles were 9–17 points lower in 2020; in operations, the 5th–75th percentiles were 1–28 points lower, but the 95th percentile was higher in 2020 (656 vs. 667); in numbers, the 50th, 75th, and 95th percentiles were similar, while the 5th and 25th percentiles were 5–7 points lower.
- School-level change scores correlated strongly between the two mathematics domains (*r* = .50, *p* < .001) and moderately between reading and mathematics (*r* = .26 and .29, both *p* < .001). Higher socio-cultural capital was associated with more positive operations change scores (*r* = .36, *p* < .001); other school-characteristic correlations were weak and heterogeneous.
Methods
- Cross-sectional cohort comparison using the mandatory statewide “Lernstand 5” assessment at the beginning of grade 5 in Baden-Württemberg; the 2020 cohort was compared with the pooled 2017–2019 cohorts. Each year’s sample was *n* > 80,000; 2020 reading *n* = 81,810 and mathematics *n* = 81,550, with pooled 2017–2019 *n* = 249,862 for reading and 250,109 for mathematics.
- Three linked competence tests were analyzed: reading comprehension, operations, and numbers. Scores were scaled with a Rasch model, ten plausible values were drawn, and Cohen’s *d* was used for standardized differences.
- Analyses compared population means, selected percentiles (5%, 25%, 50%, 75%, 95%), and school-level change scores. For school-characteristic analyses, *k* = 1,065 schools remained after excluding 152 schools; change scores were correlated with the proportion of students with migration background and schools’ socio-cultural capital (average books at home).
Limitations
- Stated: the design does not allow causal interpretation; results are between-student comparisons, not intra-individual learning gains; background characteristics were available only at the school level, risking ecological fallacy; average books at home was estimated from previous cohorts; year-to-year fluctuation and test-specific variance may affect 2020 scores; the study covers only reading and mathematics competencies, not well-being or social relatedness; suspended grade repetitions may have added low-achieving students, theoretically accounting for up to 0.01 standard deviations.
- Evident from the text: participation was voluntary for students with special needs and absent students did not participate, and participation rates were lower in 2020 (94.9% reading, 94.6% mathematics) than in prior years (about 96%).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
26 Not checked yet · 26 Own result · 16 Own method · 12 Own interpretation · 5 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Like the findings from Hamburg, the present analysis shows an unexpected positive difference for reading competencies between 2019 and 2020.”
- Also stated in: § Comparisons for High-, Average-, and Low-Achieving Groups (RQ2) · plain
“The year-to-year comparison of 2020 with 2019 shows standardized differences of d = +0.07 for reading comprehension, d = -0.07 for operations, and d = -0.02 for numbers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Comparisons for High-, Average-, and Low-Achieving Groups (RQ2) · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 1, there was a considerable year-to-year fluctuation in the prepandemic years.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The correlation between schools' change scores in the two mathematics competence domains were large ( r = .50, p < .001).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Differences in mathematics assessment scores before and after the first wave of the pandemic were on average closer to zero in the numbers domain than in the operations domain.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Students in the groups with lower levels of competence at the 5 %, 25 %, 50 %, and 75 % percentile had scored 1 to 28 points less in 2020 than in the three previous pre-pandemic years.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, the school background variables were highly correlated with the average school competence scores.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Contrary to previous findings (e.g., Maldonado & De Witte, 2021), students' migration background at the school level showed a small positive relation to the reading change score whereas socio-cultural capital had a small negative relation to the reading change score.”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On the other hand, average- and low-achieving students showed less understanding of operations than previous cohorts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The lockdown findings from Baden-Württemberg show slightly diverging patterns for reading and mathematics, depending on the level of student achievement.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The correlations between the reading change score and the mathematics change scores were medium ( r =.26 and r = .29, both p < .001).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 23
- It may state two separate claims as one. 14
- The quote only partly states this claim. 1
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Assuming that the additional students are on the lower end of the empirical ability distribution, this effect may theoretically account for up to 0.01 standard deviations.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- The authors state this as an assumption: The additional students in the 2020 cohort are on the lower end of the empirical ability distribution.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The remaining correlations between the school characteristics and the change scores were weak and heterogeneous (| r | < .3; p < .001, except for numbers both p > .01; RQ3).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In numbers, cut levels were similar for the 50 %, 75 %, and 95 % percentiles and 5 to 7 points lower for the 25 % and 5 % percentiles.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the correlations between the two background variables with each other ( r = -.71, p < .001) as well as with the average school competence scores (| r | > .65, p < .001) were very large.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The annual learning gains in the present study were larger than these projections, ranging from 83-89 % in reading to 84-87 % in numbers and 95-96 % in numbers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Moreover, learning losses were larger, at least in reading tests, for schools with a large proportion of students with a migration background (Maldonado & De Witte, 2021) and for students from less-educated homes (Engzell et al., 2021; Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
- Not checked yetcites Engzell et al., 2021
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The effect was larger in schools with a high proportion of students with a migration background and in schools with a high proportion of students from non-academic families.”
What the sources say
- Not checked yetcites Engzell et al., 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Schools in Belgium were closed for nine weeks in spring 2020. The comparison of test scores from June 2020 with test scores from the previous year showed a learning loss of 0.19 standard deviations in mathematics and of 0.23 standard deviations in Dutch (Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Surprisingly, students' reading scores were actually higher in the pandemic year 2020 compared to the previous year (0.05 standard deviations). Mathematics scores were slightly lower in 2020 compared to 2019 (0.02 standard deviations).”
What the sources say
- Not checked yetcites Depping et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The tests are mandatory for all public secondary schools in the federal state.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Choosing the three previous years as comparison group is in line with the approach taken by Engzell et al. (2021) and by Maldonado and De Witte (2021). Using more than one year for comparisons reduces possible outlier effects.”
What the sources say
- Not checked yetcites Engzell et al., 2021
- Not checked yetcites Maldonado & De Witte, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, there may have been less time than usual to focus on the application of operations in tasks that require multistep problem solving and a translation of mathematical relations to real world settings.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Teachers mark the tests using standardized coding instructions.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A possible explanation could be that the lockdown forced teachers and students to spend more time with basic reading tasks, including distance learning via email, chat, and work sheets.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis uses entire population cohorts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The test results are not affecting grades in the classroom, making 'Lernstand 5' a low-stakes assessment.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The tests in the pilot studies contained new items along with a set of linking items that remained constant throughout the years.”
plainn = 8,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the average number of books at home was an estimate from previous student cohorts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis is based on between-student comparisons.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This heterogeneity within schools may well pose a greater challenge to teachers than a slight pandemic-related attenuation of the schools' mean competence level.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“This result is in line with findings from Hamburg, where social school characteristics played only a minor role in the explanation of school differences (Depping et al., 2021).”
What the sources say
- Not checked yetcites Depping et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“With its focus on specific competencies, the present study only provides insights regarding changes of reading and mathematics competencies.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This may indicate that school closures are not followed by a uniform loss of learning across all groups of students but rather by a heterogeneous learning backlog (RQ2).”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Taken together, the differences in reading comprehension were larger for high-achieving students, whereas in mathematics they were larger for low-achieving students.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This may reflect that teachers put a stronger emphasis on arithmetic tasks that students could practice at home on their own during the pandemic year 2020.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“We suggest that similar factors are also critical when it comes to distance learning (Muijs, 2021).”
What the sources say
- Not checked yetcites Muijs, 2021
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Using a higher number of pre-pandemic years for comparisons, however, could have introduced bias associated with more general long-term trends in student achievement (Stanat et al., 2017).”
What the sources say
- Not checked yetcites Stanat et al., 2017
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Overall, the present findings suggest that teachers' (as well as parents' and students') efforts to adapt to the disruption were often beneficial, because the cohort differences turned out to be smaller than the estimated short-term loss of learning during the actual school lockdown (Engzell et al., 2021; Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Engzell et al., 2021
- Not checked yetcites Maldonado & De Witte, 2021
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Still, the exact causal links remain unclear, because the samples were self-selected and the interventions were at the same time the competence assessment used as outcome (which facilitates method effects).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Advanced digitalization of schools seems to be a necessary precondition for effective distance teaching and learning.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This might also have led to a larger than usual number of low-achieving students in the 2020 cohort.”
hedgedconditionaln = 262- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: In Germany, students with low achievements often have to repeat a grade.
- Rests on an assumption: In the wake of the first COVID-19 wave, mandatory grade repetitions were suspended.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis of school characteristics (RQ3) cannot be translated directly to individual students without risking an ecological fallacy.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The difficulty parameters of the linking items were estimated in 2014 after the first pilot study and retained for the scaling of all subsequent pilot studies (fixed parameter calibration; Kolen & Brennan, 2014, p. 182-183).”
What the sources say
- Not checked yetcites Kolen & Brennan, 2014
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The 2020 scores may reflect not just effects of school closures but also specific variance introduced by the particular tests that were used in 2020 despite the standardized development and scaling.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The background characteristics were only available at the school level.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The 2017-2019 cohorts are a valid pre-pandemic comparison for the 2020 cohort, such that observed differences reflect pandemic effects rather than secular trends or other cohort differences.
“Using more than one year for comparisons reduces possible outlier effects. Using a higher number of pre-pandemic years for comparisons, however, could have introduced bias associated with more general long-term trends in student achievement (Stanat et al., 2017).”
The paper relies on this without support.
In Germany, students with low achievements often have to repeat a grade.
“In Germany, students with low achievements often have to repeat a grade ( n = 262 after the school year 2018/19 in Baden-Württemberg).”
The paper relies on this without support.
In the wake of the first COVID-19 wave, mandatory grade repetitions were suspended.
“In the wake of the first COVID-19 wave, such mandatory grade repetitions were suspended.”
The authors state this as an assumption.
The additional students in the 2020 cohort are on the lower end of the empirical ability distribution.
“Assuming that the additional students are on the lower end of the empirical ability distribution, this effect may theoretically account for up to 0.01 standard deviations.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the other hand, reasons for finding larger learning losses in mathematics than in reading might be that students spent more time on reading during school closures and that supporting children in their reading skills might have been easier to realize for parents than supporting children in improving their competencies in mathematics (e.g., Depping et al., 2021; Schult et al., 2021).”
From Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review (2021) · cites this paper as Schult et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Third, low-performing students were more affected by COVID-19-related school closures in mathematics, while high-performing students were more affected by COVID-19-related school closures in reading (Schult et al., 2021).”
From Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review (2021) · cites this paper as Schult et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Schult et al. (2021) document that fifth-grade students are slightly behind across the board, with the largest impact among low-achieving students in math.”
From Educational Inequality* (2022) · cites this paper as Schult et al. (2021)
In the citing paper
“Schult et al. [9] stated that the dispersion of the competence levels of pupils within the individual schools could pose greater challenges for the teachers than a slight pandemic-related weakening of the average competence level at the school [9] (p.18).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“According to the authors of the study, this difference could be due to the fact that during school closures, more time is spent on basic reading tasks, such as reading e-mails and text in the chat [9] (p.17).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The effect of school characteristics, such as socio-economic capital and the proportion of pupils with an immigrant background, carried hardly any weight in relation to the changes of the competence values at the schools [9] (p.17).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The results of the investigations also suggest that average and weaker pupils in the 2020 test showed greater deficits in terms of mathematical operations than the cohorts from previous years.This difference could not be demonstrated in the high-performance pupils [9] (p.17).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The differences that were found with regard to reading competence were more pronounced in the group of high-performance pupils than in the group of low-performance pupils [9] (p.14).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“When comparing the standardized test values from the years 2017 to 2019 (before school closure, n = 249,862) with those that were collected in 2020 (after school closure, n = 81,810), there is a negative effect of Cohen's d = -0.07[9] (p.29).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The authors came to the conclusion that, based on an augmented competence of 0.65 standard deviations in a normal school year, the pupils lagged behind in their competence development by one month due to the school closures [9] (p.16).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“If, in addition to this, one also takes into account the results that Schult et al. [9] and Kogan and Lavertu [10,11] come to, there is not expected to be any change in this estimated value.”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“Schult et al. [9], referring to the studies by Depping et al. [8], Moldonado and De Witte [5], Meeter [7], Engzell et al. [6] and Kuhfeld et al. [4] stated that the data across all these studies and domains indicate an average learning loss of around 0.1 standard deviations during the first wave of the COVID 19 pandemic in 2020 [9] (p. 7).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Discussion
“84-87 % in numbers”
§ Discussion
“95-96 % in numbers”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Wickham, H. (2016) · Use R! · doi:10.1007/978-3-319-24277-4 · Cited by 34,873
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
The Split-Apply-Combine Strategy for Data Analysis
Wickham, H. (2011) · Journal of Statistical Software · doi:10.18637/jss.v040.i01 · Cited by 2,760
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2022376118 · Cited by 1,848
The theory and practice of item response theory.
Ayala, D. (2009) · Guilford Press eBooks · Cited by 1,812
Ranking the effectiveness of worldwide COVID-19 government interventions
Haug, N., Geyrhofer, L., Londei, A., Dervic, E., Desvars-Larrive, A., Loreto, V., Pinior, B., Thurner, S., & Klimek, P. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-01009-0 · Cited by 1,412
Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement
Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020) · Educational Researcher · doi:10.3102/0013189x20965918 · Cited by 997
Test Equating, Scaling, and Linking
Kolen, M. J., & Brennan, R. L. (2014) · Springer eBooks · doi:10.1007/978-1-4939-0317-7 · Cited by 646
Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.746289 · Cited by 588
Research works printed but not linked: 4
- Ditton, H., & Krüsken, J. (2009). Denn wer hat, dem wird gegeben werden? Eine Längsschnittstudie zur Entwicklung schulischer Leistungen und den Effekten der sozialen Herkunft in der Grundschulzeit [To those who have, will more be given? A longitudinal study concerning the development of school achievement and the effects of social background during primary school]. Journal for Educational Research Online, 1 (1), 33-61.
- Krüsken, J. (2007). Entwicklung von Schülerleistungen und Zensuren in der Grundschule. In: H. Ditton (Ed.), Kompetenzaufbau und Laufbahnen im Schulsystem (pp. 41-62). Waxmann.
- Schulz, A., Leuders, T., & Rangel, U. (2017). Arithmetische Basiskompetenzen am Übergang zu Klasse 5 - eine empirie- und modellgestützte Diagnostik als Grundlage für spezifische Förderentscheidungen. In A. Fritz, S. Schmidt, & G. Ricken (Eds.), Handbuch Rechenschwäche (3rd ed., pp. 396-417). Beltz.
- Stanat, P., Schipolowski, S., Rjosk, C., Weirich, S., & Haag, N. (Eds.) (2017 ). IQBBildungstrend 2016. Kompetenzen in den Fächern Deutsch und Mathematik am Ende der 4. Jahrgangsstufe im zweiten Ländervergleich . Waxmann.
Cited by
Sorted by citations
Items 101–111 of 111
Schulische Aufholprogramme nach Corona – finanzielle Rahmenbedingungen im internationalen Vergleich
Himmelrath, A. (2022) · Waxmann Verlag GmbH eBooks · doi:10.31244/9783830996033.01
Krisenhilfe-Kommunikation: Wissenschaft als Akuthelferin in gesellschaftlichen Erschütterungen
Pasternack, P. (2022) · Springer eBooks · doi:10.1007/978-3-658-39177-5_10
COVID-19 Pandemic and its Effects on Mental Health: Challenges of Senior Citizens in Nepal
Acharya, M., & Adhikari, M. R. (2023) · Interdisciplinary Research in Education · doi:10.3126/ire.v8i1.56727
Min, M., Nelson, R. A., & Bellows, E. (2023) · International Journal on Studies in Education · doi:10.46328/ijonse.182
Hidayah, L., Sueb, S., Sholeh, M., & Widodo, G. S. (2023) · International Research-Based Education Journal · doi:10.17977/um043v5i2p182-193
Amtalao, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5161258
Bangthamai, E. (2025) · International Journal of Information and Education Technology · doi:10.18178/ijiet.2025.15.5.2311
Three Essays on Education and Health Economics
Mudrow, K. (2025) · The Research Repository @ WVU (West Virginia University) · doi:10.33915/etd.12838
Aldea, K., & Dayawon, M. A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5364396
Educational Software to Strengthen Mathematical Skills in First-Year Higher Education Students
Guerrero, S. C., Avila, C. A. M., & Cárdenas, M. A. G. (2025) · Open Education Studies · doi:10.1515/edu-2025-0122
Yılmaz, T. Y., Oğuzcan, H. M., Yücedağ, S., Yalçın, S., & Akarsu, Z. (2026) · Batı Anadolu eğitim bilimleri dergisi · doi:10.51460/baebd.1756914
Metadata
- Authors
- Schult, J., Mahler, N., Fauth, B., & Lindner, M. A.
- Affiliations
- University of Tübingen (DE)
- Leibniz Institute for Science and Mathematics Education (DE)
- Corresponding author
- Schult, Johannes
- Venue
- School Effectiveness and School Improvement
- Publisher
- Center for Open Science
- Published
- 2021-03-11
- Volume
- 33
- Issue
- 4
- Pages
- 544-563
- ISSN
- 0924-3453 (ISSN-L)
- OpenAlex
- W4225792954
- Cited by
- 262
- References
- 30
- Citations by year
- 2026: 15 · 2025: 37 · 2024: 44 · 2023: 71 · 2022: 48 · 2021: 46
- FWCI
- 107.80
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- School Effectiveness and School Improvement (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Keywords
- COVID-19 pandemic, learning loss, school closures, distance education, reading comprehension, mathematical ability, fifth grade students, low-achieving students, socio-cultural capital, migration background, school characteristics, large-scale assessment, Baden-Württemberg
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
