CoChat
Sign Up
Cited by 302Meta-analysis and systematic reviewsEducational Research and MethodsStatistics Education and Methodologies

Beyond small, medium, or large: points of consideration when interpreting effect sizes

Open access
Bakker, Arthur, Cai, Jinfa, English, Lyn, Kaiser, Gabriele, Mesa, Vilma, Van Dooren, Wim·2019·Educational Studies in Mathematics
DOI 10.1007/s10649-019-09908-4
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10649-019-09908-4

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Wise interventions disproportionately benefit low performing groups.

    “Indeed, wise interventions are often low cost, disproportionately benefit low performing groups, have effects that can be sustained over time and context, and, in contrast to interventions that target specialized knowledge, can have effects on a broad range of academic, behavioral, and psychological outcomes.”

    From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Bakker et al., 2019

    Section 4, point 10 (“What was the sample?”; page unavailable) says interventions with historically marginalized students tend to have larger effect sizes than interventions with privileged students who already score highly. That supports a disparity in benefits, but the passage does not establish the claim specifically for wise interventions or equate historically marginalized students with low-performing students. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • These characteristics make wise interventions particularly suitable for scaling up, even where the average size of their effect on the overall population is relatively small.

    “These characteristics make wise interventions particularly suitable for scaling up, even where the average size of their effect on the overall population is relatively small (Bakker et al., 2019;Kraft, 2020;Slavin & Madden, 2011).”

    From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Bakker et al., 2019

    Section 4, “Twelve points to consider when interpreting effect sizes” (page unavailable), says that scalability and cost matter alongside effect size, and that some interventions are scalable and cheap. Section 5, “A call for better guidelines” (page unavailable), adds that a small effect can be impressive in context. Neither passage establishes that wise interventions in particular are suitable for scaling up, even when their average effect across the overall population is small. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Wise interventions can have effects on a broad range of academic, behavioral, and psychological outcomes.

    “Indeed, wise interventions are often low cost, disproportionately benefit low performing groups, have effects that can be sustained over time and context, and, in contrast to interventions that target specialized knowledge, can have effects on a broad range of academic, behavioral, and psychological outcomes.”

    From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Bakker et al., 2019

    Section 4 discusses effect sizes for interventions in particular student groups and separately compares achievement with attitudes, interest, and motivation. It does not say that wise interventions affect a broad range of academic, behavioral, and psychological outcomes. Searches for wise interventions and for mind-set interventions across those outcomes found no passage establishing that claim; page numbers were not available in our copy. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

  • Wise interventions have effects that can be sustained over time and context.

    “Indeed, wise interventions are often low cost, disproportionately benefit low performing groups, have effects that can be sustained over time and context, and, in contrast to interventions that target specialized knowledge, can have effects on a broad range of academic, behavioral, and psychological outcomes.”

    From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Bakker et al., 2019

    Section 4, “Twelve points to consider when interpreting effect sizes” (page unavailable), discusses intervention length and how context can affect effect sizes, but does not say that wise interventions have effects sustained over time and across contexts. Searches for “wise interventions sustained over time and context” and for long-term persistence across contexts found no passage addressing that claim. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Wise interventions are often low cost.

    “Indeed, wise interventions are often low cost, disproportionately benefit low performing groups, have effects that can be sustained over time and context, and, in contrast to interventions that target specialized knowledge, can have effects on a broad range of academic, behavioral, and psychological outcomes.”

    From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Bakker et al., 2019

    Section 4 says that some interventions are cheap while many are expensive, but it does not identify wise interventions or say how often they are low cost. Searches for wise interventions and inexpensive psychological interventions found no passage addressing that specific claim; page numbers were unavailable in our copy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 41 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Coe, R. (2002). It's the effect size, stupid. What effect size is and why it is important. Paper presented at the British Educational Research Association annual conference, Exeter, UK. Retrieved July 24, 2019 from https://www.cem.org/attachments/ebe/ESguide.pdf
  2. Wilkinson, L., & APA Taskforce. (1999). Statistical methods in psychology journals. American Psychologist, 54(8), 594-604.

Cited by

Sorted by citations

Related

Authors
Bakker, A., Cai, J., English, L., Kaiser, G., Mesa, V., & Van Dooren, W.
Affiliations
  • Utrecht University (NL)
  • University of Delaware (US)
  • Queensland University of Technology (AU)
  • Universität Hamburg (DE)
  • University of Michigan (US)
Corresponding author
Bakker, Arthur
ORCID
Venue
Educational Studies in Mathematics
Publisher
Springer Science and Business Media LLC
Published
2019-07-30
Volume
102
Issue
1
Pages
1-8
ISSN
  • 0013-1954 (ISSN-L)
  • 1573-0816
DOI
10.1007/s10649-019-09908-4
OpenAlex
W2966650813
Cited by
302
References
41
Citations by year
2026: 26 · 2025: 48 · 2024: 43 · 2023: 52 · 2022: 52 · 2021: 55 · 2020: 20 · 2019: 6
FWCI
22.25
Type
journal article
Language
en
Open access
Licence
Locations
  • Educational Studies in Mathematics (publishedVersion)
  • QUT ePrints (Queensland University of Technology) (submittedVersion)
  • Lirias (KU Leuven) (submittedVersion, other-oa)
  • Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
  • Utrecht University Repository (Utrecht University) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Education › Educational Research and Methods
  • Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
Keywords
effect size, statistical significance, mathematics education research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review