CoChat

The case for the development and use of ecologically valid measures of executive function in experimental and clinical neuropsychology

Open access

The DOI matches its Crossref record; 173 of the 173 references it cites were found in a registry or an index; 10 of the 103 sources its claims cite were checked for support.

Abstract

This article considers the scientific process whereby new and better clinical tests of executive function might be developed, and what form they might take. We argue that many of the traditional tests of executive function most commonly in use (e.g., the Wisconsin Card Sorting Test; Stroop) are adaptations of procedures that emerged almost coincidentally from conceptual and experimental frameworks far removed from those currently in favour, and that the prolongation of their use has been encouraged by a sustained period of concentration on "construct-driven" experimentation in neuropsychology. This resulted from the special theoretical demands made by the field of executive function, but was not a necessary consequence, and may not even have been a useful one. Whilst useful, these tests may not therefore be optimal for their purpose. We consider as an alternative approach a function-led development programme which in principle could yield tasks better suited to the concerns of the clinician because of the transparency afforded by increased "representativeness" and "generalisability." We further argue that the requirement of such a programme to represent the interaction between the individual and situational context might also provide useful constraints for purely experimental investigations. We provide an example of such a programme with reference to the Multiple Errands and Six Element tests.

Overview

CoChat Summary

## TL;DR

The paper argues that many widely used executive-function tests were not designed for their current clinical purposes and may be poor matches to real-world behavior. It advocates a “function-led” approach to building ecologically valid tasks, using examples such as the Multiple Errands Test and Six Element Test.

## Key findings

  • Traditional executive-function tests such as the Wisconsin Card Sorting Test (WCST), Stroop, and Tower of London were developed from earlier experimental or conceptual frameworks, not for clinical assessment of everyday functioning.
  • The authors argue that “ecological validity” should include both representativeness (similarity to real-world situations) and generalisability (predicting behavior outside the lab).
  • The paper cites evidence that more naturalistic tasks can relate more closely to everyday problems than traditional tests: for example, Dawson et al. (2005a) found Multiple Errands performance correlated with everyday-function measures in 11 CVA patients and 11 healthy controls, with MET-AMPS-M correlations of 2.63 and 2.51 as reported in the text; Dawson et al. (2005b) reported correlations of between .5 and .8 in 30 people with acquired brain injury and 30 matched controls.
  • Knight et al. (2002) reported inter-rater reliabilities for Multiple Errands measures of .81 to 1.00 and internal consistency (Cronbach’s alpha) of .77; Dawson et al. (2005b) reported hospital-version inter-class correlations of ≥ 0.90 in 30 controls and 30 people with acquired brain injury.
  • In Alderman et al. (2003), the paper reports a group-level dissociation on the MET-SV in 50 patients total, with 19 rule breakers and 26 task failers; the contingency table shows 224 vs 65 rule breaks/task failures for rule breakers and 78 vs 171 for task failers.

## Methods

  • Conceptual/review article: the paper synthesizes historical, theoretical, and psychometric arguments about executive-function testing rather than reporting a new empirical study.
  • It uses examples from prior work, especially the Multiple Errands Test, Six Element Test, Wisconsin Card Sorting Test, and related lesion, imaging, and psychometric studies.
  • The main analysis is a level-of-explanation argument distinguishing constructs, operations, and functions, and proposing a function-led route for developing and validating clinical tests.

## Limitations

  • The paper explicitly notes that it is too early to tell whether the further theoretical investigations into constructs underpinning these functions will yield lab-type tasks of clinical utility.
  • It also states that some of the supporting studies have small group sizes; for example, the Dawson et al. (2005a) correlation results are described as coming from 11 CVA patients and 11 healthy controls.
  • Evident from the text: much of the evidence is indirect, drawn from prior studies and examples rather than from a single unified empirical dataset in this paper.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1017/s1355617706060310

Cites

173 of 173 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 14 Not valid · 57 Not checked · 56 Own results and interpretations

67 more claims

Create a free account to see all 135 claims and to verify the 2 references still open.

See all claims — free account
  • It may state two separate claims as one. 28
  • Probably not a substantial claim. 40
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 173 references

Research works printed but not linked: 5

  1. Grady, C.L. (1999).Neuroimaging and activation of the frontal lobes. In B.L. Miller & J.L. Cummings (Eds.),The human frontal lobes: Function and disorders (pp.196–230).New York:Guilford Press.
  2. Petrides, M. (1994).Frontal lobes and working memory: Evidence from investigations ofthe effects of cortical excisions in nonhuman primates. In F. Boller & J. Grafman (Eds.),Handbook of Neuropsychology,Vol. 9 (pp.59–82).Amsterdam:Elsevier.
  3. Simons, J.S. , Schölvinck, M. , Gilbert, S.J. , Frith, C.D. , & Burgess, P.W. (in press).Differential components of prospective memory? Evidence fromfMRI. Neuropsychologia.
  4. Weigl, E. (1927).Zur Psychologie sogenannter Abstraktionsprozesse.Zeitschrift fur Psychologie,103, 2–45.[Translated by M. Rioch and reprinted as: On the psychologyof so-called processes of abstraction. Journal of Abnormal and SocialPsychology, 36, 3–33, 1948].
  5. Burgess, P.W. , Alderman, N. , Emslie, H. , Evans, J.J. , & Wilson, B.A. (1996a).The dysexecutive questionnaire. In B.A. Wilson , N. Alderman , P.W. Burgess , H. Emslie , & J.J. Evans (Eds.),Behavioural assessment of the dysexecutive syndrome.Bury St. Edmunds, U.K.:Thames Valley Test Company.

Cited by

See all 671

Metadata

Authors
BURGESS, P. W., ALDERMAN, N., FORBES, C., COSTELLO, A., M-A.COATES, L., DAWSON, D. R., ANDERSON, N. D., GILBERT, S. J., DUMONTHEIL, I., & CHANNON, S.
Affiliations
  • University College London (GB)
  • St Andrew's Healthcare (GB)
  • King's College Hospital (GB)
  • University of London (GB)
  • University of Toronto (CA)
Venue
Journal of the International Neuropsychological Society
Published
2006-03
Volume
12
Issue
2
Pages
194-209
ISSN
  • 1355-6177 (ISSN-L)
  • 1469-7661
OpenAlex
W2161907647
Cited by
663
References
173
Citations by year
2026: 21 · 2025: 24 · 2024: 37 · 2023: 39 · 2022: 38 · 2021: 41 · 2020: 31 · 2019: 47 · 2018: 41 · 2017: 48 · 2016: 35 · 2015: 35 · 2014: 32 · 2013: 44 · 2012: 29
FWCI
28.96
Type
journal article
Language
en
Open access
Locations
  • Journal of the International Neuropsychological Society (publishedVersion)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion)
Funders
Wellcome Trust (061171)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
  • Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
executive function, test development, clinical neuropsychological assessment, Wisconsin Card Sorting Test, Stroop task, Multiple Errands Test, generalizability
MeSH terms
Cognition Disorders, Frontal Lobe, Humans, Neuropsychological Tests, Neuropsychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The case for the development and use of ecologically valid measures of executive function in experimental and clinical neuropsychology. https://app.cochat.ai/papers/10.1017/s1355617706060310
The paper
BURGESS, P. W., ALDERMAN, N., FORBES, C., COSTELLO, A., M-A.COATES, L., DAWSON, D. R., ANDERSON, N. D., GILBERT, S. J., DUMONTHEIL, I., & CHANNON, S. (2006). The case for the development and use of ecologically valid measures of executive function in experimental and clinical neuropsychology. Journal of the International Neuropsychological Society. https://doi.org/10.1017/s1355617706060310

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.