The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; none of the 34 sources its claims cite is checked for support yet.
Did Students Learn Less During the COVID-19 Pandemic? Reading and Mathematics Competencies Before and After the First Pandemic Wave
Preprint — published as 10.1080/09243453.2022.2061014 · Paper page
Abstract
The first wave of the COVID-19 pandemic disrupted regular classes in spring 2020. Temporary school closures supposedly led to a considerable learning loss, particularly for low-achieving students. Schools in Baden-Württemberg, Germany, were closed for two months. Although distance learning was implemented, students spent less time learning. Additionally, teachers were faced with organizational and technological challenges of remote learning environments. The present study investigates the competencies of fifth-graders, using large-scale assessment results in reading and mathematics from annual mandatory tests in September (each n > 80,000). In line with studies from other countries, competence scores were slightly lower in 2020 compared with the three previous years (–0.07 standard deviations for reading comprehension, –0.09 for operations, and –0.03 for numbers). Low-achieving readers managed to attain pre-pandemic competence levels. Regarding mathematics competencies, low-achieving students seem to have a learning backlog that deserves attention in future education. School characteristic such as the average socio-cultural capital and the proportion of students with migration background played a minor role in mediating the schools' learning loss. Still, lower socio-cultural capital was positively associated with larger learning loss in mathematics.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Using statewide mandatory “Lernstand 5” data from Baden-Württemberg, this study compared incoming fifth-graders’ reading and mathematics competencies in September 2020 with those from 2017–2019. It found small average declines after the first COVID-19 school closures, with heterogeneous patterns across achievement levels and domains.
Key findings
- Population means were slightly lower in 2020 than in 2017–2019: reading comprehension *d* = −0.07, operations *d* = −0.09, and numbers *d* = −0.03 (all *p* < .001).
- Percentile comparisons showed mixed patterns: in reading, the bottom 5% was similar (2017–2019: 299 vs. 2020: 300), while the 25th–95th percentiles were 9–17 points lower in 2020; in operations, the 5th–75th percentiles were 1–28 points lower, but the 95th percentile was higher in 2020 (656 vs. 667); in numbers, the 50th, 75th, and 95th percentiles were similar, while the 5th and 25th percentiles were 5–7 points lower.
- School-level change scores correlated strongly between the two mathematics domains (*r* = .50, *p* < .001) and moderately between reading and mathematics (*r* = .26 and .29, both *p* < .001). Higher socio-cultural capital was associated with more positive operations change scores (*r* = .36, *p* < .001); other school-characteristic correlations were weak and heterogeneous.
Methods
- Cross-sectional cohort comparison using the mandatory statewide “Lernstand 5” assessment at the beginning of grade 5 in Baden-Württemberg; the 2020 cohort was compared with the pooled 2017–2019 cohorts. Each year’s sample was *n* > 80,000; 2020 reading *n* = 81,810 and mathematics *n* = 81,550, with pooled 2017–2019 *n* = 249,862 for reading and 250,109 for mathematics.
- Three linked competence tests were analyzed: reading comprehension, operations, and numbers. Scores were scaled with a Rasch model, ten plausible values were drawn, and Cohen’s *d* was used for standardized differences.
- Analyses compared population means, selected percentiles (5%, 25%, 50%, 75%, 95%), and school-level change scores. For school-characteristic analyses, *k* = 1,065 schools remained after excluding 152 schools; change scores were correlated with the proportion of students with migration background and schools’ socio-cultural capital (average books at home).
Limitations
- Stated: the design does not allow causal interpretation; results are between-student comparisons, not intra-individual learning gains; background characteristics were available only at the school level, risking ecological fallacy; average books at home was estimated from previous cohorts; year-to-year fluctuation and test-specific variance may affect 2020 scores; the study covers only reading and mathematics competencies, not well-being or social relatedness; suspended grade repetitions may have added low-achieving students, theoretically accounting for up to 0.01 standard deviations.
- Evident from the text: participation was voluntary for students with special needs and absent students did not participate, and participation rates were lower in 2020 (94.9% reading, 94.6% mathematics) than in prior years (about 96%).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
26 Not checked yet · 26 Own result · 16 Own method · 12 Own interpretation · 5 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Like the findings from Hamburg, the present analysis shows an unexpected positive difference for reading competencies between 2019 and 2020.”
- Also stated in: § Comparisons for High-, Average-, and Low-Achieving Groups (RQ2) · plain
“The year-to-year comparison of 2020 with 2019 shows standardized differences of d = +0.07 for reading comprehension, d = -0.07 for operations, and d = -0.02 for numbers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Comparisons for High-, Average-, and Low-Achieving Groups (RQ2) · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 1, there was a considerable year-to-year fluctuation in the prepandemic years.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The correlation between schools' change scores in the two mathematics competence domains were large ( r = .50, p < .001).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Differences in mathematics assessment scores before and after the first wave of the pandemic were on average closer to zero in the numbers domain than in the operations domain.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Students in the groups with lower levels of competence at the 5 %, 25 %, 50 %, and 75 % percentile had scored 1 to 28 points less in 2020 than in the three previous pre-pandemic years.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, the school background variables were highly correlated with the average school competence scores.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Contrary to previous findings (e.g., Maldonado & De Witte, 2021), students' migration background at the school level showed a small positive relation to the reading change score whereas socio-cultural capital had a small negative relation to the reading change score.”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On the other hand, average- and low-achieving students showed less understanding of operations than previous cohorts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The lockdown findings from Baden-Württemberg show slightly diverging patterns for reading and mathematics, depending on the level of student achievement.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The correlations between the reading change score and the mathematics change scores were medium ( r =.26 and r = .29, both p < .001).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As expected, socio-cultural capital was positively associated with slightly larger school change scores in mathematical operations and, to a lesser degree, numbers.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The year-to-year comparison of 2020 with 2019 shows standardized differences of d = +0.07 for reading comprehension, d = -0.07 for operations, and d = -0.02 for numbers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Contrary to previous findings (e.g., Maldonado & De Witte, 2021), students' migration background at the school level showed a small positive relation to the reading change score whereas socio-cultural capital had a small negative relation to the reading change score.”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A different pattern can be found in the domain of reading comprehension, where low-achieving students attained similar competence levels in 2020 compared to the previous years.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“School characteristic such as the average socio-cultural capital and the proportion of students with migration background did not show substantial relationships with schools' competence change scores (RQ3).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The other cut scores at the 25 %, 50 %, 75 %, and 95 % percentile were 9 to 17 points lower in 2020 than in the three previous years, indicating lower achievement scores for medium- to high-achieving students.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Competencies within one year - and even within many individual schools - are spread across more than three standard deviations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The present cross-sectional study indicates a minor drop in the mean competence scores of incoming fifth-graders after two months of closed schools in early 2020.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In reading comprehension, low-achieving students (i.e., the bottom 5 %) scored beneath a similar cut level (2017-2019: 299 vs. 2020: 300).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The variability in test scores between students has already been large before the COVID-19 pandemic (see 2015 to 2019 in Figure 1).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The year-to-year comparison of 2020 with 2019 shows standardized differences of d = +0.07 for reading comprehension, d = -0.07 for operations, and d = -0.02 for numbers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The group of high-achieving students managed to attain similar competence levels compared to previous cohorts regarding operations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Schools with higher socio-cultural capital tended to have more positive school change scores in operations ( r = .36, p < .001).”
- Also stated in: § Abstract · plain
“Still, lower socio-cultural capital was positively associated with larger learning loss in mathematics.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The standardized difference was d = -0.07 for reading comprehension, d = -0.09 for operations, and d = -0.03 for numbers (all p < .001).”
- Also stated in: § Abstract · plain
“competence scores were slightly lower in 2020 compared with the three previous years (-0.07 standard deviations for reading comprehension, -0.09 for operations, and -0.03 for numbers).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The standardized difference was d = -0.07 for reading comprehension, d = -0.09 for operations, and d = -0.03 for numbers (all p < .001).”
- Also stated in: § Abstract · plain
“competence scores were slightly lower in 2020 compared with the three previous years (-0.07 standard deviations for reading comprehension, -0.09 for operations, and -0.03 for numbers).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In operations, high-achieving students (i.e., those above the 95 % percentile) scored above a higher cut level in 2020 (2017-2019: 656 vs. 2020: 667).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The standardized difference was d = -0.07 for reading comprehension, d = -0.09 for operations, and d = -0.03 for numbers (all p < .001).”
- Also stated in: § Abstract · plain
“competence scores were slightly lower in 2020 compared with the three previous years (-0.07 standard deviations for reading comprehension, -0.09 for operations, and -0.03 for numbers).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Judging from the comparably low 5 % percentile in 2020, the operations domain in mathematics deserves more attention during (and after) potential future school lockdowns.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Eight weeks of closed schools in the Netherlands in spring 2020 were associated with a loss of learning of 3 percentile ranks ( n > 250,000, grades 4 through 7; Engzell et al., 2021).”
What the sources say
- Not checked yetcites Engzell et al., 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“In spring 2020, elementary schools in the federal state of Baden-Württemberg, for example, were closed for two months.”
- Also stated in: § --- insert Table 1 here --- · plain
“Schools in Baden-Württemberg were closed on March 17, 2020 due to the COVID19 pandemic. Classes in grade 4 resumed on-site teaching two months later on May 18, 2020.”
What the sources say
- Not checked yetcites Fickermann & Edelstein, 2020
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § --- insert Table 1 here --- · plain
In the paper
“According to a survey of 1,099 German parents of school-aged children (Grewenig et al., 2021), students spent considerably less time on learning activities during the lockdown (3.6 hours on average) than they did before (7.4 hours).”
- Also stated in: § Abstract · plain
“Although distance learning was implemented, students spent less time learning.”
What the sources say
- Not checked yetcites Grewenig et al., 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
In the paper
“This reduction was more pronounced for low-achieving students than for high-achieving students.”
What the sources say
- Not checked yetcites Grewenig et al., 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Learning loss estimates were slightly larger for mathematics (0.14 standard deviations) than for reading (0.09 standard deviations; Engzell et al., 2021, p. 85).”
What the sources say
- Not checked yetcites Engzell et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Meta-analytic and systematic reviews, which included a preprint of the present study, estimate average learning losses between d = -0.09 (in reading; Hammerstein et al., 2021) and d = -0.17 (in mathematics; Zierer, 2021) during the first wave of the COVID-19 pandemic in the year 2020.”
What the sources say
- Not checked yetcites Hammerstein et al., 2021
- Not checked yetcites Zierer, 2021
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Still, such positive effects, if they exist, do not offset the detrimental consequences of closed schools, as indicated by an overall loss of learning in recent studies (Engzell et al., 2021; Kuhfeld et al., 2020a).”
What the sources say
- Not checked yetcites Engzell et al., 2021
- Not checked yetcites Kuhfeld et al., 2020a
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results provide further evidence that student competencies were attenuated after schools had been closed for about a fifth of the school year (RQ1).”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 2017-2019 cohorts are a valid pre-pandemic comparison for the 2020 cohort, such that observed differences reflect pandemic effects rather than secular trends or other cohort differences.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the results do not reflect intra-individual learning gains.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Plausible values yield unbiased means and variances (Wu, 2005).”
What the sources say
- Not checked yetcites Wu, 2005
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“In a study of 1,835 Chinese ninth-graders, learning loss during a seven-week school lockdown was 0.22 standard deviations smaller for students with online lessons than for students with no formal instruction (Clark et al., 2021).”
What the sources say
- Not checked yetcites Clark et al., 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“It was therefore not possible to statistically isolate specific effects of school characteristics over and above competence scores.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The present population data suggest that the overall efforts of teachers, students, and parents throughout the pandemic-stricken school year might have mitigated the expectable detrimental effects of school closing on student achievement to some degree.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, the cohort differences translate to roughly one month of learning backlog.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The 2017-2019 cohorts are a valid pre-pandemic comparison for the 2020 cohort, such that observed differences reflect pandemic effects rather than secular trends or other cohort differences.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The present study design does not allow a causal interpretation.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Combining the present findings with other large-scale studies yields learning loss estimates that are slightly larger than our effect sizes ( d = -0.10 in native language domains such as reading; d = -0.17 in mathematics; Zierer, 2021).”
What the sources say
- Not checked yetcites Zierer, 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the measure showed little fluctuation across cohorts, and can thus be used to approximate the average socio-cultural capital of the assessed schools.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Data from Hamburg, Germany, from August/September 2019 and September 2020 show similar results (Depping et al., 2021).”
What the sources say
- Not checked yetcites Depping et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests that learning losses (or gains) at a school level could differ considerably between reading and mathematics.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Studies that compared student performance declines directly after the re-opening of schools tend to show larger deficit estimates (Engzell et al., 2021; Kuhfeld et al., 2020a; Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Engzell et al., 2021
- Not checked yetcites Kuhfeld et al., 2020a
- Not checked yetcites Maldonado & De Witte, 2021
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Empirical studies of formative assessments during the first COVID-19 wave showed a performance increase compared to the regular classes in previous years (Meeter, 2021; for older students see Spitzer & Musslick, 2020).”
What the sources say
- Not checked yetcites Meeter, 2021
- Not checked yetcites Spitzer & Musslick, 2020
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Online learning benefits students during a school lockdown.”
What the sources say
- Not checked yetcites Clark et al., 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 23
- It may state two separate claims as one. 14
- The quote only partly states this claim. 1
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Assuming that the additional students are on the lower end of the empirical ability distribution, this effect may theoretically account for up to 0.01 standard deviations.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- The authors state this as an assumption: The additional students in the 2020 cohort are on the lower end of the empirical ability distribution.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The remaining correlations between the school characteristics and the change scores were weak and heterogeneous (| r | < .3; p < .001, except for numbers both p > .01; RQ3).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In numbers, cut levels were similar for the 50 %, 75 %, and 95 % percentiles and 5 to 7 points lower for the 25 % and 5 % percentiles.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the correlations between the two background variables with each other ( r = -.71, p < .001) as well as with the average school competence scores (| r | > .65, p < .001) were very large.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The annual learning gains in the present study were larger than these projections, ranging from 83-89 % in reading to 84-87 % in numbers and 95-96 % in numbers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Moreover, learning losses were larger, at least in reading tests, for schools with a large proportion of students with a migration background (Maldonado & De Witte, 2021) and for students from less-educated homes (Engzell et al., 2021; Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
- Not checked yetcites Engzell et al., 2021
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The effect was larger in schools with a high proportion of students with a migration background and in schools with a high proportion of students from non-academic families.”
What the sources say
- Not checked yetcites Engzell et al., 2021
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Schools in Belgium were closed for nine weeks in spring 2020. The comparison of test scores from June 2020 with test scores from the previous year showed a learning loss of 0.19 standard deviations in mathematics and of 0.23 standard deviations in Dutch (Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Maldonado & De Witte, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Surprisingly, students' reading scores were actually higher in the pandemic year 2020 compared to the previous year (0.05 standard deviations). Mathematics scores were slightly lower in 2020 compared to 2019 (0.02 standard deviations).”
What the sources say
- Not checked yetcites Depping et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The tests are mandatory for all public secondary schools in the federal state.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Choosing the three previous years as comparison group is in line with the approach taken by Engzell et al. (2021) and by Maldonado and De Witte (2021). Using more than one year for comparisons reduces possible outlier effects.”
What the sources say
- Not checked yetcites Engzell et al., 2021
- Not checked yetcites Maldonado & De Witte, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, there may have been less time than usual to focus on the application of operations in tasks that require multistep problem solving and a translation of mathematical relations to real world settings.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Teachers mark the tests using standardized coding instructions.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A possible explanation could be that the lockdown forced teachers and students to spend more time with basic reading tasks, including distance learning via email, chat, and work sheets.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis uses entire population cohorts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The test results are not affecting grades in the classroom, making 'Lernstand 5' a low-stakes assessment.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The tests in the pilot studies contained new items along with a set of linking items that remained constant throughout the years.”
plainn = 8,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the average number of books at home was an estimate from previous student cohorts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis is based on between-student comparisons.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This heterogeneity within schools may well pose a greater challenge to teachers than a slight pandemic-related attenuation of the schools' mean competence level.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“This result is in line with findings from Hamburg, where social school characteristics played only a minor role in the explanation of school differences (Depping et al., 2021).”
What the sources say
- Not checked yetcites Depping et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“With its focus on specific competencies, the present study only provides insights regarding changes of reading and mathematics competencies.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This may indicate that school closures are not followed by a uniform loss of learning across all groups of students but rather by a heterogeneous learning backlog (RQ2).”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Taken together, the differences in reading comprehension were larger for high-achieving students, whereas in mathematics they were larger for low-achieving students.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This may reflect that teachers put a stronger emphasis on arithmetic tasks that students could practice at home on their own during the pandemic year 2020.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“We suggest that similar factors are also critical when it comes to distance learning (Muijs, 2021).”
What the sources say
- Not checked yetcites Muijs, 2021
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Using a higher number of pre-pandemic years for comparisons, however, could have introduced bias associated with more general long-term trends in student achievement (Stanat et al., 2017).”
What the sources say
- Not checked yetcites Stanat et al., 2017
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Overall, the present findings suggest that teachers' (as well as parents' and students') efforts to adapt to the disruption were often beneficial, because the cohort differences turned out to be smaller than the estimated short-term loss of learning during the actual school lockdown (Engzell et al., 2021; Maldonado & De Witte, 2021).”
What the sources say
- Not checked yetcites Engzell et al., 2021
- Not checked yetcites Maldonado & De Witte, 2021
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Still, the exact causal links remain unclear, because the samples were self-selected and the interventions were at the same time the competence assessment used as outcome (which facilitates method effects).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Advanced digitalization of schools seems to be a necessary precondition for effective distance teaching and learning.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This might also have led to a larger than usual number of low-achieving students in the 2020 cohort.”
hedgedconditionaln = 262- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: In Germany, students with low achievements often have to repeat a grade.
- Rests on an assumption: In the wake of the first COVID-19 wave, mandatory grade repetitions were suspended.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis of school characteristics (RQ3) cannot be translated directly to individual students without risking an ecological fallacy.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The difficulty parameters of the linking items were estimated in 2014 after the first pilot study and retained for the scaling of all subsequent pilot studies (fixed parameter calibration; Kolen & Brennan, 2014, p. 182-183).”
What the sources say
- Not checked yetcites Kolen & Brennan, 2014
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The 2020 scores may reflect not just effects of school closures but also specific variance introduced by the particular tests that were used in 2020 despite the standardized development and scaling.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The background characteristics were only available at the school level.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The 2017-2019 cohorts are a valid pre-pandemic comparison for the 2020 cohort, such that observed differences reflect pandemic effects rather than secular trends or other cohort differences.
“Using more than one year for comparisons reduces possible outlier effects. Using a higher number of pre-pandemic years for comparisons, however, could have introduced bias associated with more general long-term trends in student achievement (Stanat et al., 2017).”
The paper relies on this without support.
In Germany, students with low achievements often have to repeat a grade.
“In Germany, students with low achievements often have to repeat a grade ( n = 262 after the school year 2018/19 in Baden-Württemberg).”
The paper relies on this without support.
In the wake of the first COVID-19 wave, mandatory grade repetitions were suspended.
“In the wake of the first COVID-19 wave, such mandatory grade repetitions were suspended.”
The authors state this as an assumption.
The additional students in the 2020 cohort are on the lower end of the empirical ability distribution.
“Assuming that the additional students are on the lower end of the empirical ability distribution, this effect may theoretically account for up to 0.01 standard deviations.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the other hand, reasons for finding larger learning losses in mathematics than in reading might be that students spent more time on reading during school closures and that supporting children in their reading skills might have been easier to realize for parents than supporting children in improving their competencies in mathematics (e.g., Depping et al., 2021; Schult et al., 2021).”
From Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review (2021) · cites this paper as Schult et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Third, low-performing students were more affected by COVID-19-related school closures in mathematics, while high-performing students were more affected by COVID-19-related school closures in reading (Schult et al., 2021).”
From Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review (2021) · cites this paper as Schult et al., 2021
The cited source says what the claim attributes to it.
In the citing paper
“Schult et al. (2021) document that fifth-grade students are slightly behind across the board, with the largest impact among low-achieving students in math.”
From Educational Inequality* (2022) · cites this paper as Schult et al. (2021)
In the citing paper
“Schult et al. [9] stated that the dispersion of the competence levels of pupils within the individual schools could pose greater challenges for the teachers than a slight pandemic-related weakening of the average competence level at the school [9] (p.18).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“According to the authors of the study, this difference could be due to the fact that during school closures, more time is spent on basic reading tasks, such as reading e-mails and text in the chat [9] (p.17).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The effect of school characteristics, such as socio-economic capital and the proportion of pupils with an immigrant background, carried hardly any weight in relation to the changes of the competence values at the schools [9] (p.17).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The results of the investigations also suggest that average and weaker pupils in the 2020 test showed greater deficits in terms of mathematical operations than the cohorts from previous years.This difference could not be demonstrated in the high-performance pupils [9] (p.17).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The differences that were found with regard to reading competence were more pronounced in the group of high-performance pupils than in the group of low-performance pupils [9] (p.14).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“When comparing the standardized test values from the years 2017 to 2019 (before school closure, n = 249,862) with those that were collected in 2020 (after school closure, n = 81,810), there is a negative effect of Cohen's d = -0.07[9] (p.29).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“The authors came to the conclusion that, based on an augmented competence of 0.65 standard deviations in a normal school year, the pupils lagged behind in their competence development by one month due to the school closures [9] (p.16).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“If, in addition to this, one also takes into account the results that Schult et al. [9] and Kogan and Lavertu [10,11] come to, there is not expected to be any change in this estimated value.”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
In the citing paper
“Schult et al. [9], referring to the studies by Depping et al. [8], Moldonado and De Witte [5], Meeter [7], Engzell et al. [6] and Kuhfeld et al. [4] stated that the data across all these studies and domains indicate an average learning loss of around 0.1 standard deviations during the first wave of the COVID 19 pandemic in 2020 [9] (p. 7).”
From Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review (2021) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Discussion
“84-87 % in numbers”
§ Discussion
“95-96 % in numbers”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Wickham, H. (2016) · Use R! · doi:10.1007/978-3-319-24277-4 · Cited by 34,873
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
The Split-Apply-Combine Strategy for Data Analysis
Wickham, H. (2011) · Journal of Statistical Software · doi:10.18637/jss.v040.i01 · Cited by 2,760
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2022376118 · Cited by 1,848
The theory and practice of item response theory.
Ayala, D. (2009) · Guilford Press eBooks · Cited by 1,812
Ranking the effectiveness of worldwide COVID-19 government interventions
Haug, N., Geyrhofer, L., Londei, A., Dervic, E., Desvars-Larrive, A., Loreto, V., Pinior, B., Thurner, S., & Klimek, P. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-01009-0 · Cited by 1,412
Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement
Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020) · Educational Researcher · doi:10.3102/0013189x20965918 · Cited by 997
Test Equating, Scaling, and Linking
Kolen, M. J., & Brennan, R. L. (2014) · Springer eBooks · doi:10.1007/978-1-4939-0317-7 · Cited by 646
Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.746289 · Cited by 588
Research works printed but not linked: 4
- Ditton, H., & Krüsken, J. (2009). Denn wer hat, dem wird gegeben werden? Eine Längsschnittstudie zur Entwicklung schulischer Leistungen und den Effekten der sozialen Herkunft in der Grundschulzeit [To those who have, will more be given? A longitudinal study concerning the development of school achievement and the effects of social background during primary school]. Journal for Educational Research Online, 1 (1), 33-61.
- Krüsken, J. (2007). Entwicklung von Schülerleistungen und Zensuren in der Grundschule. In: H. Ditton (Ed.), Kompetenzaufbau und Laufbahnen im Schulsystem (pp. 41-62). Waxmann.
- Schulz, A., Leuders, T., & Rangel, U. (2017). Arithmetische Basiskompetenzen am Übergang zu Klasse 5 - eine empirie- und modellgestützte Diagnostik als Grundlage für spezifische Förderentscheidungen. In A. Fritz, S. Schmidt, & G. Ricken (Eds.), Handbuch Rechenschwäche (3rd ed., pp. 396-417). Beltz.
- Stanat, P., Schipolowski, S., Rjosk, C., Weirich, S., & Haag, N. (Eds.) (2017 ). IQBBildungstrend 2016. Kompetenzen in den Fächern Deutsch und Mathematik am Ende der 4. Jahrgangsstufe im zweiten Ländervergleich . Waxmann.
Cited by
Sorted by citations
Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.746289 · Cited by 588
Learning loss during Covid-19: An early systematic review
Donnelly, R., & Patrinos, H. A. (2022) · PROSPECTS · doi:10.1007/s11125-021-09582-6 · Cited by 445
The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis
Di Pietro, G. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100530 · Cited by 173
COVID-19 learning losses: Early grade reading in South Africa
Ardington, C., Wills, G., & Kotze, J. (2021) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2021.102480 · Cited by 164
Hevia, F. J., Vergara-Lope, S., Velásquez-Durán, A., & Calderón, D. (2022) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2021.102515 · Cited by 158
Metadata
- Authors
- Schult, J., Mahler, N., Fauth, B., & Lindner, M. A.
- Affiliations
- University of Tübingen (DE)
- Leibniz Institute for Science and Mathematics Education (DE)
- Corresponding author
- Schult, Johannes
- Venue
- School Effectiveness and School Improvement
- Publisher
- Center for Open Science
- Published
- 2021-03-11
- Volume
- 33
- Issue
- 4
- Pages
- 544-563
- ISSN
- 0924-3453 (ISSN-L)
- OpenAlex
- W4225792954
- Cited by
- 262
- References
- 30
- Citations by year
- 2026: 15 · 2025: 37 · 2024: 44 · 2023: 71 · 2022: 48 · 2021: 46
- FWCI
- 107.80
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- School Effectiveness and School Improvement (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › COVID-19 Pandemic Impacts
- Keywords
- COVID-19 pandemic, learning loss, school closures, distance education, reading comprehension, mathematical ability, fifth grade students, low-achieving students, socio-cultural capital, migration background, school characteristics, large-scale assessment, Baden-Württemberg
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
