Assessing the validity of computer-game-like tests of processing speed and working memory
Open accessAbstract
Processing speed (Gs) and working memory (WM) tasks have received considerable interest as correlates of more complex cognitive performance measures. Gs and WM tasks are often repetitive and are often rigidly presented, however. The effects of Gs and WM may, therefore, be confounded with those of motivation and anxiety. In an effort to address this problem, we assessed the concurrent and predictive validity of computer-game-like tests of Gs (Space Code) and WM (Space Matrix) across two experiments. In Experiment 1, within a university sample (N = 70), Space Matrix exhibited concurrent validity as a WM measure, whereas Space Code appeared to be a mixed-ability measure. In Experiment 2, Space Matrix exhibited concurrent validity as well as predictive validity (as a predictor of school grades) within a school-aged sample (N = 94), but the results for Space Code were less encouraging. Relationships between computer-game-like tests and gender, handedness, and computer-game experience are also discussed.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Space Code had similar mixed-domain problems.
“Space Code [17,18] had similar problems.”
From Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy (2016) · cites this paper as [18]
A second paper found that Space Code's correlations with measures of working memory, visuospatial ability, and processing speed were not stable.
“However, a second paper detailing 2 experiments which aimed to replicate the previous finding found that Space Code’s correlations with measures of working memory, visuospatial ability, and processing speed were not stable [18].”
From Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy (2016) · cites this paper as [18]
There were some indications that gamification may actually worsen data quality.
“and there were some indications that it may actually worsen it [17,18,34,36].”
From Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy (2016) · cites this paper as [18]
Results of references [9,17,18,38] show that gamified tasks can be used to improve motivation, while still maintaining a scientifically valid task.
“The results of references [9,17,18,38], show that gamified tasks can be used to improve motivation, while still maintaining a scientifically valid task.”
From Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy (2016) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554 · see citing works
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587 · see citing works
Statistical Methods for Psychology.
Gudgeon, A. C., & Howell, D. C. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348956 · Cited by 8,277 · see citing works
A capacity theory of comprehension: individual differences in working memory
Just, M. A., & Carpenter, P. A. (1992) · Psychological Review · doi:10.1037/0033-295x.99.1.122 · Cited by 4,153 · see citing works
Human Cognitive Abilities: A Survey of Factor-Analytic Studies
Carroll, J. B. (1993) · doi:10.1017/cbo9780511571312 · Cited by 4,121 · see citing works
O’connor, B. P. (2000) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200807 · Cited by 4,116 · see citing works
Human cognitive abilities a survey of factor-analytic studies
Carroll, J. B. (2004) · Cited by 3,396 · see citing works
Working memory, short-term memory, and general fluid intelligence: A latent-variable approach.
Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.128.3.309 · Cited by 3,078 · see citing works
The g factor: the science of mental ability
(1998) · Choice Reviews Online · doi:10.5860/choice.36-2443 · Cited by 2,760 · see citing works
Henson, R. K., & Roberts, J. K. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405282485 · Cited by 2,454 · see citing works
Determining the number of components from the matrix of partial correlations
Velicer, W. F. (1976) · Psychometrika · doi:10.1007/bf02293557 · Cited by 2,438 · see citing works
Reasoning ability is (little more than) working-memory capacity?!
Kyllonen, P. C., & Christal, R. E. (1990) · Intelligence · doi:10.1016/s0160-2896(05)80012-1 · Cited by 1,810 · see citing works
Carpenter, P. A., Just, M. A., & Shell, P. (1990) · Psychological Review · doi:10.1037/0033-295x.97.3.404 · Cited by 1,648 · see citing works
Contemporary Intellectual Assessment: Theories, Tests, and Issues
Gallo, J. L. (2013) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/act011 · Cited by 1,259 · see citing works
Contemporary intellectual assessment: theories, tests, and issues
(1997) · Choice Reviews Online · doi:10.5860/choice.34-5370 · Cited by 1,238 · see citing works
Miyake, A., Friedman, N. P., Rettinger, D. A., Shah, P., & Hegarty, M. (2001) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.130.4.621 · Cited by 1,139 · see citing works
Miyake, A., Friedman, N. P., Rettinger, D. A., Shah, P., & Hegarty, M. (2001) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.130.4.621 · Cited by 1,137 · see citing works
Conway, A. R. A., Cowan, N., Bunting, M. F., Therriault, D. J., & Minkoff, S. R. B. (2002) · Intelligence · doi:10.1016/s0160-2896(01)00096-4 · Cited by 1,108 · see citing works
The Development of Hierarchical Factor Solutions
Schmid, J., & Leiman, J. M. (1957) · Psychometrika · doi:10.1007/bf02289209 · Cited by 1,098 · see citing works
Working Memory and Intelligence: The Same or Different Constructs?
Ackerman, P. L., Beier, M. E., & Boyle, M. O. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.1.30 · Cited by 969 · see citing works
Processing Speed, Working Memory, and Fluid Intelligence: Evidence for a Developmental Cascade
Fry, A. F., & Hale, S. (1996) · Psychological Science · doi:10.1111/j.1467-9280.1996.tb00366.x · Cited by 776 · see citing works
Gathercole, S. E., & Pickering, S. J. (2000) · British Journal of Educational Psychology · doi:10.1348/000709900158047 · Cited by 685 · see citing works
Relationships among processing speed, working memory, and fluid intelligence in children
Fry, A. F., & Hale, S. (2000) · Biological Psychology · doi:10.1016/s0301-0511(00)00051-x · Cited by 606 · see citing works
Working-memory capacity explains reasoning ability—and a little bit more
Süß, H., Oberauer, K., Wittmann, W. W., Wilhelm, O., & Schulze, R. (2002) · Intelligence · doi:10.1016/s0160-2896(01)00100-3 · Cited by 591 · see citing works
Kane, M. J., Hambrick, D. Z., & Conway, A. R. A. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.1.66 · Cited by 545 · see citing works
Reaction Time and Psychometric g
Jensen, A. R. (1982) · Springer eBooks · doi:10.1007/978-3-642-68664-1_4 · Cited by 493 · see citing works
Castel, A. D., Pratt, J., & Drummond, E. (2005) · Acta Psychologica · doi:10.1016/j.actpsy.2005.02.004 · Cited by 481 · see citing works
Oberauer, K., Schulze, R., Wilhelm, O., & Süß, H.-M. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.1.61 · Cited by 410 · see citing works
A normative study of family media habits
Gentile, D. A., & Walsh, D. A. (2002) · Journal of Applied Developmental Psychology · doi:10.1016/s0193-3973(02)00102-8 · Cited by 376 · see citing works
The Structural Characteristics of Video Games: A Psycho-Structural Analysis
Wood, R. T. A., Griffiths, M. D., Chappell, D., & Davies, M. N. O. (2004) · CyberPsychology & Behavior · doi:10.1089/109493104322820057 · Cited by 340 · see citing works
Ackerman, P. L., Beier, M. E., & Boyle, M. D. (2002) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.131.4.567 · Cited by 327 · see citing works
Human cognitive capabilities: Gf-Gc theory.
Horn, J. L., & Noll, J. G. (1997) · Cited by 323 · see citing works
The intelligence test desk reference (ITDR) : Gf-Gc cross-battery assessment
McGrew, K. S., & Flanagan, D. P. (1998) · Medical Entomology and Zoology · Cited by 322 · see citing works
Bors, D. A., & Stokes, T. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058003002 · Cited by 303 · see citing works
Montanelli, R. G., & Humphreys, L. G. (1976) · Psychometrika · doi:10.1007/bf02293559 · Cited by 273 · see citing works
Hitch, G. J., Towse, J. N., & Hutton, U. M. Z. (2001) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.130.2.184 · Cited by 271 · see citing works
Nash Unsworth, A., & Engle, R. W. (2004) · Intelligence · doi:10.1016/j.intell.2004.08.003 · Cited by 249 · see citing works
The relationship between computer-game preference, gender, and mental-rotation ability
Quaiser-Pohl, C. M., Geiser, C., & Lehmann, W. (2005) · Personality and Individual Differences · doi:10.1016/j.paid.2005.07.015 · Cited by 237 · see citing works
The scientific study of general intelligence : tribute to Arthur R. Jensen
Jensen, A. R., & Nyborg, H. (2003) · Cited by 236 · see citing works
The Cattell-Horn-Carroll Theory of Cognitive Abilities
Flanagan, D. P., & Dixon, S. G. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0431 · Cited by 225 · see citing works
Swanson, L., & Kim, K. (2006) · Intelligence · doi:10.1016/j.intell.2006.07.001 · Cited by 218 · see citing works
Complexity, Metacognition, and Fluid Intelligence
Stankov, L. (2000) · Intelligence · doi:10.1016/s0160-2896(99)00033-1 · Cited by 196 · see citing works
Attention and working memory as predictors of intelligence
SCHWEIZER, K. W., & Moosbrugger, H. (2004) · Intelligence · doi:10.1016/j.intell.2004.06.006 · Cited by 188 · see citing works
Memory span and general intelligence: A latent-variable approach
Colom, R., Abad, F. J., Rebollo, I., & Shih, P. C. (2005) · Intelligence · doi:10.1016/j.intell.2005.05.006 · Cited by 172 · see citing works
Foundations for Better Understanding of Cognitive Abilities.
Horn, J. L., & Blankson, A. N. (2005) · Cited by 170 · see citing works
Wolff, H., & Preising, K. (2005) · Behavior Research Methods · doi:10.3758/bf03206397 · Cited by 163 · see citing works
Prokosch, M. D., Yeo, R. A., & Miller, G. F. (2004) · Intelligence · doi:10.1016/j.intell.2004.07.007 · Cited by 163 · see citing works
Visuospatial skills and computer game experience influence the performance of virtual endoscopy
Enochsson, L., Isaksson, B., Tour, R., Kjellin, A., Hedman, L. R., Wredmark, T., & Felländer‐Tsai, L. (2004) · Journal of Gastrointestinal Surgery · doi:10.1016/j.gassur.2004.06.015 · Cited by 138 · see citing works
Processing speed interacts with working memory efficiency in multiple sclerosis
Lengenfelder, J., BRYANT, D., DIAMOND, B., Kalmar, J. H., MOORE, N., & DeLuca, J. (2006) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2005.12.001 · Cited by 136 · see citing works
Human intelligence differences: a recent history
Deary, I. J. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01621-1 · Cited by 126 · see citing works
Sex differences on three factors identified in Raven's Standard Progressive Matrices
Lynn, R., Allïk, J., & Paul Irwing, B. (2004) · Intelligence · doi:10.1016/j.intell.2004.06.007 · Cited by 121 · see citing works
Individual differences in working memory: Introduction to the special section.
Miyake, A. (2001) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.130.2.163 · Cited by 109 · see citing works
Individual differences in working memory: Introduction to the special section.
Miyake, A. (2001) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.130.2.163 · Cited by 107 · see citing works
Rushton, J. P., Skuy, M., & Fridjhon, P. (2003) · Intelligence · doi:10.1016/s0160-2896(02)00140-x · Cited by 75 · see citing works
The games psychologists play (and the data they provide)
Washburn, D. A. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03202541 · Cited by 71 · see citing works
Mental speed is not the ‘basic’ process of intelligence
Stankov, L., & Roberts, R. D. (1997) · Personality and Individual Differences · doi:10.1016/s0191-8869(96)00163-8 · Cited by 70 · see citing works
Inspection time in the structure of cognitive abilities
Burns, N. R., & Nettelbeck, T. (2003) · Intelligence · doi:10.1016/s0160-2896(02)00120-4 · Cited by 67 · see citing works
Self-citationThe criterion validity of tasks of basic cognitive processes
Luo, D., Thompson, L. A., & Detterman, D. K. (2005) · Intelligence · doi:10.1016/j.intell.2004.12.003 · Cited by 65 · see citing works
Middle school students' technology practices and preferences: re-examining gender differences
Miller, L., Schweingruber, H. A., & Brandenburg, C. L. (2001) · Cited by 58 · see citing works
An Overview of Research into the Cognitive Basis of Intelligence
SCHWEIZER, K. W. (2005) · Journal of Individual Differences · doi:10.1027/1614-0001.26.1.43 · Cited by 55 · see citing works
Law, D. J., Morrin, K. A., & Pellegrino, J. W. (1995) · Learning and Individual Differences · doi:10.1016/1041-6080(95)90011-x · Cited by 44 · see citing works
Gs Invaders: Assessing a computer game-like test of processing speed
McPherson, J., & Burns, N. R. (2007) · Behavior Research Methods · doi:10.3758/bf03192982 · Cited by 43 · see citing works
Self-citationVirtual Kids of the 21st Century: Understanding the Children in Schools Today
Yelland, N., & Lloyd, M. (2001) · Cited by 36 · see citing works
Mental Speed: On Frameworks, Paradigms, and a Platform for the Future
Danthiir, V., Roberts, R. D., Schulze, R., & Wilhelm, O. (2005) · Sage eBooks · doi:10.4135/9781452233529.n3 · Cited by 36 · see citing works
Colom, R., & Shih, P. C. (2004) · Intelligence · doi:10.1016/j.intell.2004.06.011 · Cited by 29 · see citing works
Computer games: Paradigms of opportunity
Porter, D. B. (1995) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204737 · Cited by 27 · see citing works
A speeded coding task using a computer-based mouse response
McPherson, J., & Burns, N. R. (2005) · Behavior Research Methods · doi:10.3758/bf03192725 · Cited by 23 · see citing works
Self-citationInterpretation of the Woodcock-Johnson III Tests of Cognitive Abilities
Floyd, R. G., Shaver, R. B., & McGrew, K. S. (2003) · Elsevier eBooks · doi:10.1016/b978-012628982-4/50002-7 · Cited by 21 · see citing works
Floyd, R. G. (2005) · Cited by 13 · see citing works
O’Connor, B. P. (2001) · Applied Psychological Measurement · doi:10.1177/01466216010251011 · Cited by 11 · see citing works
Measuring Auditory Inspection Time in Primary School Children
Zajac, I. T., & Burns, N. R. (2007) · Journal of Individual Differences · doi:10.1027/1614-0001.28.1.45 · Cited by 10 · see citing works
Self-citationCognitive Neuroscience: Working Memory Linked to Intelligence
Wickelgren, I. (1997) · Science · doi:10.1126/science.275.5306.1581 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- ment: Theories, tests, and issues (2nd ed., pp. 203-233). New York: Guilford.
- Aliya, K. S. (2002). The role of computer games in the development of theoretical analysis, flexibility and reflective thinking in children: A longitudinal study. International Journal of Psychophysiology, 45, 149.
- Mitchell, A., & Savill-Smith, C. (2004). The use of computer and video games for learning: A review of the literature. London: Learning and Skills Development Agency.
- Raven, J. C., Raven, J. E., & Court, J. H. (1998). Progressive matrices. Oxford: Oxford Psychologists Press.
Cited by
Sorted by citations
Lumsden, J., Edwards, E. A., Lawrence, N. S., Coyle, D., & Munafò, M. R. (2016) · JMIR Serious Games · doi:10.2196/games.5888 · Cited by 555 · see citing works
“However, a second paper detailing 2 experiments which aimed to replicate the previous finding found that Space Code’s correlations with measures of working memory, visuospatial ability, and processing speed were not stable [18].”
BackgroundMethodInfluentialGender differences in processing speed: A review of recent research
Roivainen, E. (2011) · Learning and Individual Differences · doi:10.1016/j.lindif.2010.11.021 · Cited by 161 · see citing works
Intrinsic motivation and attentional capture from gamelike features in a visual search task
Miranda, A. T., & Palmer, E. M. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0357-7 · Cited by 86 · see citing works
Gamelike features might not improve data
Hawkins, G. E., Rae, B., Nesbitt, K. V., & Brown, S. D. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0264-3 · Cited by 61 · see citing works
TENI: A comprehensive battery for cognitive assessment based on games and technology
Tenorio Delgado, M., Arango Uribe, P., Aparicio Alonso, A., & Rosas Díaz, R. (2016) · Child Neuropsychology · doi:10.1080/09297049.2014.977241 · Cited by 34 · see citing works
- Authors
- McPherson, J., & Burns, N. R.
- Affiliations
- The University of Adelaide (AU)
- Corresponding author
- McPherson, Jason
- ORCID
- Burns, Nicholas R.
- Venue
- Behavior Research Methods
- Published
- 2008-11
- Volume
- 40
- Issue
- 4
- Pages
- 969-981
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2048716432
- Cited by
- 69
- References
- 72
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 2 · 2023: 12 · 2022: 3 · 2021: 6 · 2020: 7 · 2019: 4 · 2018: 4 · 2017: 3 · 2016: 7 · 2015: 5 · 2014: 4 · 2013: 3 · 2012: 1
- FWCI
- 0.91
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Behavior Research Methods (publishedVersion)
- PubMed (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- processing speed, working memory, SPACE code, SPACE matrix, concurrent validity, predictive validity, motivation, anxiety, school grades, gender differences, handedness
- MeSH terms
- Adolescent, Adult, Child, Computers, Factor Analysis, Statistical, Female, Humans, Male, Memory, Short-Term, Psychology, Reaction Time, Visual Perception, Reproducibility of Results, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
