CoChat
Sign Up
Cited by 45Cognitive Abilities and TestingCognitive and developmental aspects of mathematical skillsCognitive Functions and Memory

What counts as evidence for working memory training? Problems with correlated gains and dichotomization

Tidwell, Joe W., Dougherty, Michael R., Chrabaszcz, Jeffrey R., Thomas, Rick P., Mendoza, Jorge L.·2014·Psychonomic Bulletin & Review
DOI 10.3758/s13423-013-0560-7
Open this paper in CoChat — free account

Abstract

The question of whether computerized cognitive training leads to generalized improvements of intellectual abilities has been a popular, yet contentious, topic within both the psychological and neurocognitive literatures. Evidence for the effective transfer of cognitive training to nontrained measures of cognitive abilities is mixed, with some studies showing apparent successful transfer, while others have failed to obtain this effect. At the same time, several authors have made claims about both successful and unsuccessful transfer effects on the basis of a form of responder analysis, an analysis technique that shows that those who gain the most on training show the greatest gains on transfer tasks. Through a series of Monte Carlo experiments and mathematical analyses, we demonstrate that the apparent transfer effects observed through responder analysis are illusory and are independent of the effectiveness of cognitive training. We argue that responder analysis can be used neither to support nor to refute hypotheses related to whether cognitive training is a useful intervention to obtain generalized cognitive benefits. We end by discussing several proposed alternative analysis techniques that incorporate training gain scores and argue that none of these methods are appropriate for testing hypotheses regarding the effectiveness of cognitive training.

Integrity signals

Registry

Crossref · 10.3758/s13423-013-0560-7

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Increases in working memory capacity are typically thought to mediate the effects of working memory training on fluid intelligence or academic attainment.

    “This is because effects of working memory training on fluid intelligence or academic attainment are typically seen as being mediated by (i.e. dependent upon) increases in working memory capacity (Harrison et al., 2013; Lange & Süß, 2015; Tidwell et al., 2014; von Bastian & Oberauer, 2014).”

    From Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of “Far Transfer” (2016) · cites this paper as Tidwell et al., 2014

  • These weaknesses include the ongoing use of passive controls and inappropriate analyses that correlate training gains with transfer gains.

    “One of these areas of methodological deficiencies entails the continued use of passive controls and the other the use of inappropriate analysis techniques that entails correlating training gains with transfer gains (see Tidwell et al., 2014).”

    From Reevaluating the effectiveness of n-back training on transfer through the Bayesian lens: Support for the null (2016) · cites this paper as Tidwell et al., 2014

  • Despite repeated calls for improvement, the working-memory training literature still contains methodological weaknesses in design and analysis.

    “methodological deficiencies in both design and analysis persist in the WM training literature, despite many prior suggestions for remediation (Boot et al., 2013;Shipstead, Redick, & Engle, 2012;Tidwell et al., 2014).”

    From Reevaluating the effectiveness of n-back training on transfer through the Bayesian lens: Support for the null (2016) · cites this paper as Tidwell et al., 2014

  • Previous studies typically compared low versus high training gain subgroups, and this responder analysis can be problematic.

    “Previous studies have examined the relationship between training gain and far transfer, typically by comparing subgroups of participants who show low versus high training gain31,38,39; however, this type of responder analysis can be problematic40.”

    From Near transfer to an unrelated N-back task mediates the effect of N-back working memory training on matrix reasoning (2022) · cites this paper as [40]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Tidwell, J. W., Dougherty, M. R., Chrabaszcz, J. R., Thomas, R. P., & Mendoza, J. L.
Affiliations
  • University of Maryland, College Park (US)
  • University of Oklahoma (US)
Corresponding author
  • Tidwell, Joe W.
  • Dougherty, Michael R.
ORCID
Venue
Psychonomic Bulletin & Review
Publisher
Springer Science and Business Media LLC
Published
2013-12-05
Volume
21
Issue
3
Pages
620-628
ISSN
  • 1069-9384 (ISSN-L)
  • 1531-5320
DOI
10.3758/s13423-013-0560-7
OpenAlex
W2091141148
Cited by
45
References
41
Citations by year
2026: 1 · 2025: 1 · 2024: 1 · 2023: 1 · 2022: 3 · 2021: 7 · 2020: 5 · 2019: 3 · 2018: 2 · 2017: 2 · 2016: 7 · 2015: 7 · 2014: 5
FWCI
1.06
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Psychonomic Bulletin & Review (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
  • Physical Sciences › Mathematics › Statistics and Probability › Cognitive and developmental aspects of mathematical skills
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
Keywords
working memory training, cognitive training, transfer of training, responder analysis, dichotomization, evidence evaluation, Monte Carlo simulation
MeSH terms
Data Interpretation, Statistical, Humans, Intelligence, Memory, Short-Term, Transfer, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review