The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index; 3 of the 27 sources its claims cite were checked for support.
The ecological validity of tests of executive function
Open accessAbstract
Ninety-two mixed etiology neurological patients and 216 control participants were assessed on a range of neuropsychological tests, including 10 neuropsychological measures of executive function derived from 6 different tests. People who knew the patients well (relatives or carers) completed a questionnaire about the patient's dysexecutive problems in everyday life, and this paper reports the extent to which the tests predicted the patients' everyday life problems. All of the tests were significantly predictive of at least some of the behavioral and cognitive deficits reported by patients' carers. However, factor analysis of the patients' dysexecutive symptoms suggested a fractionation of the dysexecutive syndrome, with neuropsychological tests loading differentially on 3 underlying cognitive factors (Inhibition, Intentionality, and Executive Memory), supporting the conclusions that different tests measure different cognitive processes, and that there may be limits to the fractionation of the executive system.
Overview
CoChat Summary
TL;DR
In 92 mixed-etiology neurological patients and 216 controls, executive-function tests were generally correlated with carer-reported everyday dysexecutive problems, but different tests predicted different symptom factors. The dysexecutive syndrome fractionated at the behavioral level into five factors, with the first three (Inhibition, Intentionality, Executive Memory) showing differential relations to executive tests.
Key findings
- All executive tests were significantly predictive of at least some carer-reported behavioral/cognitive deficits; patients performed worse than controls on all executive measures except Cognitive Estimates (t(127) = −.98, p = .33). Under Bonferroni correction, significant patient-control differences remained for MWCST Categories and Perseverative Errors, all three Trail Making measures, and SET.
- On the Dysexecutive Questionnaire (DEX), patients rated themselves as showing fewer/less severe dysexecutive signs than observers (t(160) = −2.72, p < .01), whereas controls rated themselves higher than informants (t(331) = 3.65, p < .001), suggesting lack of insight in patients.
- Factor analysis of carer ratings yielded five factors explaining 67.2% of variance (21.3, 15.5, 11.6, 10.6, 8.2%); the first three were Inhibition, Intentionality, and Executive Memory. Executive tests loaded differentially: Factor 1 related to all executive tasks except MWCST; Factor 2 related most strongly to SET (r = .46); Factor 3 related to MWCST and verbal fluency.
- Sensitivity did not explain the factor-test pattern: SET was the second most sensitive measure (33.7% of patients at/below the 5th percentile of controls), Trails B 38.9%, Trails A 32.1%, B-A 25.9%, MWCST Perseverative Errors 28.6%, FAS 10.8%, Animals 8.2%, and Cognitive Estimates 3.6%.
Methods
- 92 neurological patients of varying etiologies (59% head injury, 13% dementia, 8.5% cerebrovascular accidents, 6.5% encephalitis, 13% other; mean age 38.5 years, SD = 15.1; mean NART IQ 103.2; mean WAIS-R FSIQ 92.1) and 216 controls (mean age 46.1 years, SD = 19.8; mean NART IQ 102.7).
- The DEX questionnaire covered 20 dysexecutive symptoms rated 0–4 by a relative/carer and by the participant; the neuropsychological battery included six executive tests (MWCST, Cognitive Estimates, Verbal Fluency FAS, Verbal Fluency Animals, Trail Making, SET) plus background tests of intellect, memory, language, and visual perception.
- Analyses included correlations between DEX total/self/other-self scores and tests, varimax rotated factor analysis of independent raters’ scores, factor score–test correlations, and sensitivity analysis using the proportion of patients at or below the 5th percentile of controls.
Limitations
- Stated: interpretation of Factor 1 was “perhaps premature”; the emotional/personality factors (4 and 5) “would benefit from further research”; and the possible one-way relation between confabulation and perseveration remained unresolved.
- Evident: the control group was significantly older than the patient group (t = 3.46, df = 306, p = .001), though the paper calls this conservative; sample sizes varied widely across tests (e.g., N = 42 to 84), and the design is correlational, so causality cannot be inferred.
- Evident: the DEX relies on subjective self- and carer ratings, and the mixed-etiology sample may limit generalizability to specific neurological groups.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 12 Not checked · 74 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“By far the strongest relation, however, was with the SET, which supports previous interpretations of the test as tapping processes involved in the creation, maintenance and activation of intentions (Burgess, 1997; Burgess & Shallice, 1997; Goldstein et al., 1993; Shallice & Burgess, 1991a, 1991b).”
What the sources say
- Not checked — no full text of the sourcecites Burgess, 1997
- Not checked — no full text of the sourcecites Burgess & Shallice, 1997
- Partly supportedcites Goldstein et al., 1993 sourceExperimental — accuracy not yet measured
In Additional Tests, the paper uses the Six Element Test (SET) to examine planning ability, which supports the broad interpretation of the test. The Discussion links prospective internal markers for behaviour change to the patient's errors on the Multiple Errands task, not specifically to the SET. The paper does not report a relation between SET performance and Intentionality; the citing sentence appears to present that relation as its own finding. (LLM, conf=0.82)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“By far the strongest relation, however, was with the SET, which supports previous interpretations of the test as tapping processes involved in the creation, maintenance and activation of intentions (Burgess, 1997; Burgess & Shallice, 1997; Goldstein et al., 1993; Shallice & Burgess, 1991a, 1991b).”
What the sources say
- Not checked — no full text of the sourcecites Burgess, 1997
- Not checked — no full text of the sourcecites Burgess & Shallice, 1997
- Partly supportedcites Goldstein et al., 1993 sourceExperimental — accuracy not yet measured
The Additional Tests section describes the Six Element Test as examining planning ability through six tasks completed within a fixed time. The Discussion links prospective triggers for changing behaviour to the patient's difficulties on the Multiple Errands task, not specifically to the Six Element Test. The paper therefore supports a planning-related interpretation of the SET but does not establish the full claim about creating, maintaining and activating intentions. (LLM, conf=0.82)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“factor analysis of the patients' dysexecutive symptoms suggested a fractionation of the dysexecutive syndrome”
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with neuropsychological tests loading differentially on 3 underlying cognitive factors (Inhibition, Intentionality, and Executive Memory)”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All of the tests were significantly predictive of at least some of the behavioral and cognitive deficits reported by patients' carers.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
56 more claims
Create a free account to see all 90 claims and every source verdict.
- Probably not a substantial claim. 13
- It may state two separate claims as one. 19
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Most troubling from a clinical perspective is that EF tests have low or no ecological validity when judged against ratings of EF in daily life or direct observations of EF performance in natural settings.”
From Impairment in Occupational Functioning and Adult ADHD: The Predictive Utility of Executive Function (EF) Ratings Versus EF Tests (2010) · cites this paper as Burgess et al., 1998
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“These studies typically find that the variance shared between any single EF test and EF ratings ranges from just 0% to 10%.”
From Impairment in Occupational Functioning and Adult ADHD: The Predictive Utility of Executive Function (EF) Ratings Versus EF Tests (2010) · cites this paper as Burgess et al., 1998
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Thus these results add to the evidence that performance on more 'ecologically valid' tasks can show closer concordance with observed symptoms in everyday life than performance on traditional experimentally derived tasks (Burgess et al., 1998), and can be adapted to a clinical setting.”
From The case for the development and use of ecologically valid measures of executive function in experimental and clinical neuropsychology (2006) · cites this paper as Burgess et al., 1998
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Past studies found EF tests to have low-order correlations that are often nonsignificant with patient self-ratings and those of significant others of the dysexecutive symptoms they observe in patients with frontal lobe injuries”
From Impairment in Occupational Functioning and Adult ADHD: The Predictive Utility of Executive Function (EF) Ratings Versus EF Tests (2010) · cites this paper as Burgess et al., 1998
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Combinations of EF tests do not fair much better, sharing just 12% -20% of the variance with EF ratings.”
From Impairment in Occupational Functioning and Adult ADHD: The Predictive Utility of Executive Function (EF) Ratings Versus EF Tests (2010) · cites this paper as Burgess et al., 1998
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Abstract
“including 10 neuropsychological measures of executive function derived from 6 different tests.”
§ INTRODUCTION
“compares the ecological validity of eleven measures of executive function taken from six different tests”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Correlations Between Overall Severity of Dysexecutive Disorder and the Neuropsychological Test Measures
“all of the tests measuring executive and general intellectual functions, visual perception, and language abilities showed correlations significantly different from zero.”
§ Correlations Between Overall Severity of Dysexecutive Disorder and the Neuropsychological Test Measures
“| Cognitive Estimates ( N 5 52) | .29 n.s. | .18 n.s. | .10 n.s. |”
Citations
Cites
From Neuropsychology to Mental Structure
Shallice, T. (1988) · doi:10.1017/cbo9780511526817 · Cited by 4,072
Amick, D. J., Cooley, W. W., & Lohnes, P. R. (1972) · Contemporary Sociology · doi:10.2307/2062821 · Cited by 2,915
The somatic marker hypothesis and the possible functions of the prefrontal cortex
Damasio, A. R. (1996) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1996.0125 · Cited by 2,847 · cited 2×
“cal groups: Recent evidence suggests that the dysexecutive syndrome may be fractionable, at least as regards the features of cognitive dysfunction (e.g., Burgess, 1997; Burgess & Shallice, 1994, 1996c; Damasio, 1996; Duncan et al., 1995; Owen et al., 1995; Robbins, 1996; Robbins et al., 1995; Shallice & Burgess, 1991a…”
Background(1995) · Cited by 2,809
Stuss, D. T., & Benson, D. F. (1986) · Raven Press eBooks · Cited by 2,636
A Modified Card Sorting Test Sensitive to Frontal Lobe Defects
Nelson, H. E. (1976) · Cortex · doi:10.1016/s0010-9452(76)80035-4 · Cited by 2,454
Negative Symptoms in Schizophrenia: Definition and Reliability
Andreasen, N. C. (1982) · Archives of General Psychiatry · doi:10.1001/archpsyc.1982.04290070020005 · Cited by 2,247
Effects of Different Brain Lesions on Card Sorting: The Role of the Frontal Lobes
MILNER, B. (1963) · Archives of Neurology · doi:10.1001/archneur.1963.00460070100010 · Cited by 2,217 · cited 2×
“…of some brain process(es) that are presumed to be utilized in situations outside the strict test situation; after all, an isolated impairment in performing the Wisconsin Card Sorting Test (WCST; Milner, 1963), with no other problems in any other situation, would be of little clinical significance.”
BackgroundDEFICITS IN STRATEGY APPLICATION FOLLOWING FRONTAL LOBE DAMAGE IN MAN
SHALLICE, T., & BURGESS, P. W. (1991) · Brain · doi:10.1093/brain/114.2.727 · Cited by 2,017 · cited 5×
“cal groups: Recent evidence suggests that the dysexecutive syndrome may be fractionable, at least as regards the features of cognitive dysfunction (e.g., Burgess, 1997; Burgess & Shallice, 1994, 1996c; Damasio, 1996; Duncan et al., 1995; Owen et al., 1995; Robbins, 1996; Robbins et al., 1995; Shallice & Burgess, 1991a…”
BackgroundMethodResultInfluentialSelf-citationThe neuropsychology of schizophrenia
Gray, J. A., Feldon, J., Rawlins, J. N. P., Hemsley, D. R., & Smith, A. D. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065055 · Cited by 1,350
Research works printed but not linked: 4
- Burgess, P.W., Alderman, N., Wilson, B.A., Evans, J.J., & Emslie, H. (1996b). The Dysexecutive Questionnaire. In B.A. Wilson, N. Alderman, P.W. Burgess, H. Emslie, & J.J. Evans (Eds.), Behavioural assessment of the dysexecutive syndrome . Bury St. Edmunds, U.K.: Thames Valley Test Company.
- Burgess, P.W. & Wood, R.Ll. (1990). Neuropsychology of behaviour disorders following brain injury. In R.Ll. Wood (Ed.), Neurobehavioural sequelae of traumatic brain injury (pp. 110133). London: Taylor & Francis.
- Cicerone, K.D. & DeLuca, J. (1990). Neuropsychological predictors of head injury rehabilitation outcome. Journal of Clinical and Experimental Neuropsychology , 12 , 92 [Abstract].
- Stuss, D.T., Shallice, T., Alexander, M.P., & Picton, T.W. (1995). Amultidisciplinary approach to executive functions. Annals of the New York Academy of Sciences , 769 , 191-211.
Cited by
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0734 · Cited by 15,798
Executive Function and the Frontal Lobes: A Meta-Analytic Review
Alvarez, J. A., & Emory, E. (2006) · Neuropsychology Review · doi:10.1007/s11065-006-9002-x · Cited by 1,981
The elusive nature of executive functions: a review of our current understanding.
Jurado, M. B., & Rosselli, M. (2007) · Neuropsychology Review · doi:10.1007/s11065-007-9040-z · Cited by 1,684
Assessment of executive functions: Review of instruments and identification of critical issues.
CHAN, R., SHUM, D., TOULOPOULOU, T., & CHEN, E. (2008) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2007.08.010 · Cited by 1,401
“…to exhibit very impressive validity and sensitivity to patients with frontal lesions and other patients with neurological and psychiatric disorders (Burgess et al., 1998; Chan & Manly, 2002; Chan et al., 2003; Chan, Chen, et al., 2004; Chan, Chen, Cheung, et al., 2006; Chan, Chen, & Law, 2006;…”
BackgroundToplak, M. E., West, R. F., & Stanovich, K. E. (2013) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12001 · Cited by 1,381
Metadata
- Authors
- BURGESS, P. W., ALDERMAN, N., EVANS, J., EMSLIE, H., & WILSON, B. A.
- Affiliations
- University College London (GB)
- St Andrew's Healthcare (GB)
- Medical Research Council (GB)
- Venue
- Journal of the International Neuropsychological Society
- Publisher
- Cambridge University Press (CUP)
- Published
- 1998-11
- Volume
- 4
- Issue
- 6
- Pages
- 547-558
- ISSN
- 1355-6177 (ISSN-L)
- 1469-7661
- OpenAlex
- W2168551754
- Cited by
- 932
- References
- 47
- Citations by year
- 2026: 16 · 2025: 21 · 2024: 23 · 2023: 21 · 2022: 30 · 2021: 45 · 2020: 37 · 2019: 35 · 2018: 31 · 2017: 50 · 2016: 55 · 2015: 40 · 2014: 55 · 2013: 46 · 2012: 55
- FWCI
- 9.00
- Type
- journal article
- Language
- en
- Open access
- green
- discovery.ucl.ac.uk
- Locations
- Journal of the International Neuropsychological Society (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Wellcome Trust
- Topics
- Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- executive function, dysexecutive syndrome, ecological validity, neuropsychological assessment, everyday problems, inhibition, intentionality, neurological patients
- MeSH terms
- Adult, Mental Disorders, Brain Diseases, Cognition Disorders, Frontal Lobe, Humans, Neuropsychological Tests, Surveys and Questionnaires, Severity of Illness Index, Syndrome, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The ecological validity of tests of executive function. https://app.cochat.ai/papers/10.1017/s1355617798466037
- The paper
- BURGESS, P. W., ALDERMAN, N., EVANS, J., EMSLIE, H., & WILSON, B. A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society. https://doi.org/10.1017/s1355617798466037
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
