The DOI matches its Crossref record; 97 of the 98 references it cites were found in a registry or an index; none of the 39 sources its claims cite is checked for support yet.
Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy
Open accessAbstract
Cognitive flexibility is a widely studied construct and is considered an important treatment target for several psychological disorders. The convergence of several independent fields of research has led to assumptions about the assessment of cognitive flexibility - assumptions that are not empirically supported and often conflate different notions of flexibility. This critical review discusses how the conflation of self-report and neurocognitive assessments has seemingly arisen from literature on eating disorders. We describe how seminal early observations of "inflexible" personality characteristics, communication competence research, and investigations of frontal lobe function after injury led to two methods of assessing "cognitive flexibility". We discuss the impact that conflation of self-report and neurocognitive assessments has had on the field, and we provide recommendations for assessing cognitive flexibility in both research and clinical settings.
Overview
CoChat Summary
TL;DR
This critical review argues that self-report and neurocognitive assessments of “cognitive flexibility” have been conflated through the jingle fallacy, especially in eating disorders research, despite little empirical evidence that they measure the same construct. It recommends ceasing interchangeable use, using precise construct language, and selecting assessments aligned with the intended research or clinical question.
Key findings
- The review states that self-report questionnaires and neurocognitive tasks of cognitive flexibility are often used interchangeably, but empirical evidence shows they do not relate well in clinical or non-clinical samples.
- Cited primary studies report little-to-no relationship: Gelonch et al. (2016) found no relationship between a shift subscale and TMT/WCST outcomes; Lounes et al. (2011) found no relationship between the Cognitive Flexibility Scale and the Brixton Spatial Anticipation Task; Gonzalez et al. (2013) reported no association between the Cognitive Flexibility Scale and TMT/novel flexibility task; Johnco et al. (2014) found no correlation in older adults with comorbid depressive and anxiety symptoms, though small correlations (r ≈ 0.28) appeared in the healthy comparison group.
- Two recent meta-analytic reviews (Howlett et al., 2021, 2022) failed to detect a relationship between self-report and neurocognitive assessments in non-clinical or clinical cohorts; Miles et al. (2022) and Sternheim et al. (2022) also reported null correlations.
- The paper identifies three apparent conceptual manifestations in the field—inflexible thoughts/behaviours causing neurocognitive inflexibility, neurocognitive inflexibility causing inflexible thoughts/behaviours, and a single construct assessed by either method—and argues the evidence conflicts with all three.
Methods
- Design: critical review with a historical overview of how self-report and neurocognitive assessments of cognitive flexibility developed and became conflated.
- Data/sample: no primary data; the paper synthesizes prior literature, focusing largely on eating disorders research and extending to other fields.
- Analysis: narrative conceptual critique, including examination of rationales in prior papers, discussion of the jingle fallacy, and recommendations for research and clinical practice.
Limitations
- The paper states that there is no clear empirical evidence to support causal claims between neurocognitive inflexibility and inflexible thoughts/behaviours, and that it is not yet possible to determine whether one or neither assessment accurately captures cognitive flexibility.
- The paper notes limitations of existing measures: neurocognitive tasks may lack ecological validity, have reliability problems, and suffer from task impurity; self-report questionnaires may be confounded by negative affect, insight, meta-cognitive awareness, and response biases.
- Evident from the text: as a critical review rather than a systematic review or meta-analysis, it does not provide new empirical data, and its heavy focus on eating disorders may limit generalizability to other fields.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 2 Not valid · 18 Not checked · 31 Own results and interpretations
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There remains scant evidence in support of the claim that poor performance on neurocognitive tasks will be mirrored by poor performance on self-report questionnaires within individuals, even though there is building evidence that certain clinical cohorts perform poorly as a group on both types of assessments.”
§ Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“One criticism of self-report questionnaires is that they can be confounded by negative affect, with studies reporting an association between self-reported cognitive flexibility, and depression and anxiety (Dennis & Vander Wal, 2010; Johnco et al., 2014; Lounes et al., 2011; Miles et al., 2022; Wang et al., 2019; Yu et al., 2020).”
§ Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy > Limitations of existing research and future recommendations
What the sources say
- Not checked yetcites Dennis & Vander Wal, 2010
- Not checked yetcites Johnco et al., 2014
- Not checked yetcites Lounes et al., 2011
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, breakdowns in cognitive flexibility, as indexed by neurocognitive tasks, may be attributable to deficits in other executive processes such as working memory or inhibition.”
§ Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy > Limitations of existing research and future recommendations
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, given that there is no evidence of a 1:1 relationship between flexible thinking and flexible behaviour, we should also be precise in our language when describing these distinct, but potentially related facets of flexibility.”
§ Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy > Limitations of existing research and future recommendations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Despite empirical evidence that self-report and neurocognitive assessments do not relate well in neither clinical nor non-clinical, samples (e.g., Howlett et al., 2021, 2022; Johnco et al., 2014; Lounes et al., 2011; Miles et al., 2022; Sternheim et al., 2022), these assessments are often used interchangeably, and considered to be equivalent in assessing the construct of cognitive flexibility.”
§ Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy > Abstract
What the sources say
- Not checked yetcites Howlett et al., 2021, 2022
- Not checked yetcites Johnco et al., 2014
- Not checked yetcites Lounes et al., 2011
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
27 more claims
Create a free account to see all 51 claims and to verify the 39 references still open.
- Probably not a substantial claim. 15
- It may state two separate claims as one. 9
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the same time, the nature and structure of executive function remains actively debated in the cognitive literature.184”
From Executive function deficits in attention-deficit/hyperactivity disorder and autism spectrum disorder (2024) · cites this paper as [184]
Experimental — accuracy not yet measured
Citations
Cites
Diamond, A. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143750 · Cited by 13,268
Validity of the Trail Making Test as an Indicator of Organic Brain Damage
Reitan, R. M. (1958) · Perceptual and Motor Skills · doi:10.2466/pms.1958.8.3.271 · Cited by 7,928
Wisconsin Card Sorting Test Manual – Revised and Expanded
Heaton, R. K., Chelune, C., Talley, J., Kay, G., & Curtiss, G. (1981) · Medical Entomology and Zoology · Cited by 2,987
A simple objective technique for measuring flexibility in thinking
Berg, E. A. (1948) · The Journal of General Psychology · doi:10.1080/00221309.1948.9918159 · Cited by 2,255
Grant, D. A., & Berg, E. (1948) · Journal of Experimental Psychology · doi:10.1037/h0059831 · Cited by 2,244
The relation of the Trail Making Test to organic brain damage.
Reitan, R. M. (1955) · Journal of Consulting Psychology · doi:10.1037/h0044509 · Cited by 1,798
Guilford, J. P. (1956) · Psychological Bulletin · doi:10.1037/h0040755 · Cited by 1,508
Toplak, M. E., West, R. F., & Stanovich, K. E. (2013) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12001 · Cited by 1,381 · cited 1×
“The executive functioning literature more broadly has encountered similar problems where self-report questionnaires are presumed to capture identical or similar underlying processes that are assessed by neurocognitive tasks (Toplak et al., 2013).”
BackgroundA New Measure of Cognitive Flexibility
Martin, M. M., & Rubin, R. B. (1995) · Psychological Reports · doi:10.2466/pr0.1995.76.2.623 · Cited by 1,156
Dennis, J. P., & Vander Wal, J. S. (2010) · Cognitive Therapy and Research · doi:10.1007/s10608-009-9276-4 · Cited by 1,118 · cited 8×
“Johnco et al. (2014) reported that the Cognitive Flexibility Scale and Cognitive Flexibility Inventory did not correlate with a range of neurocognitive tasks in older adults with comorbid depressive and anxiety symptoms, but small correlations ( r ≈ 0.28) were detected in the healthy comparison group.”
BackgroundMethodInfluential
Research works printed but not linked: 2
- 10.1007/978-3-642-39360-0-33 · doi:10.1007/978-3-642-39360-0-33
- Uznadze, D. (1966). The psychology of set. Consultants’ Bureau.
Cited by
Hohl, K., & Dolcos, S. M. (2024) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2024.1331960 · Cited by 145
Moseley, G. L., Leake, H. B., Beetsma, A. J., Watson, J. A., Butler, D. S., van der Mee, A., Stinson, J., Harvie, D. S., Palermo, T. M., Meeus, M., et al. (2023) · Journal of Pain · doi:10.1016/j.jpain.2023.11.008 · Cited by 99
Self-citationExecutive function deficits in attention-deficit/hyperactivity disorder and autism spectrum disorder
Kofler, M. J., Soto, E. F., Singh, L. J., Harmon, S. L., Jaisle, E. M., Smith, J. N., Feeney, K. E., & Musser, E. D. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00350-9 · Cited by 51
“However, these distinctions are problematic because using the same term (‘executive functions’) for minimally related constructs evokes the ‘jingle fallacy’ — the logical error in which two tests are assumed to assess the same construct because they share similar names or labels 184 .”
Exploring Biomarkers of Mental Flexibility in Healthy Aging: A Computational Psychometric Study
Borghesi, F., Chirico, A., Pedroli, E., Cipriani, G. E., Canessa, N., Amanzio, M., & Cipresso, P. (2023) · Sensors · doi:10.3390/s23156983 · Cited by 16
Almutawa, S. S., Alshehri, N., AlNoshan, A. A., AbuDujain, N. M., Almutawa, K. S., & Almutawa, A. S. (2025) · BMC Medical Education · doi:10.1186/s12909-024-06445-4 · Cited by 14
Metadata
- Authors
- Howlett, C. A., Miles, S., Berryman, C., Phillipou, A., & Moseley, G. L.
- Affiliations
- University of South Australia (AU)
- Swinburne University of Technology (AU)
- The University of Melbourne (AU)
- Corresponding author
- Moseley, G. Lorimer
- Venue
- Australian Journal of Psychology
- Publisher
- Informa UK Limited
- Published
- 2023-02-19
- Volume
- 75
- Issue
- 1
- Pages
- 2174684-2174684
- ISSN
- 0004-9530 (ISSN-L)
- 1742-9536
- PMC
- PMC12175648
- OpenAlex
- W4321351338
- Cited by
- 22
- References
- 98
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 9 · 2023: 3
- FWCI
- 0.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Australian Journal of Psychology (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- UniSA Research Outputs Repository (University of South Australia) (submittedVersion, cc-by-nc)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by-nc)
- Funders
- funding bodies h
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Keywords
- cognitive flexibility, self-report measures, neuropsychological assessment, eating disorders, communication competence, frontal lobe function, frontal lobe lesions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy. https://app.cochat.ai/papers/10.1080/00049530.2023.2174684
- The paper
- Howlett, C. A., Miles, S., Berryman, C., Phillipou, A., & Moseley, G. L. (2023). Conflation between self-report and neurocognitive assessments of cognitive flexibility: a critical review of the Jingle Fallacy. Australian Journal of Psychology. https://doi.org/10.1080/00049530.2023.2174684
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.