Executive Functions: Formative Versus Reflective Measurement
The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
The primary objective of this article was to critically evaluate the routine use of confirmatory factor analysis (CFA) for representing an individual’s performance across a battery of executive function tasks. A conceptual review and statistical reanalysis of N = 10 studies that used CFA methods of EF tasks was undertaken. Despite evidence of excellent global model fit for CFA models, studies were uniformly characterized by weak correlations among EF tasks and weak to moderate levels of maximal reliability of latent EF constructs. Vanishing tetrad tests raised the possibility that individual tasks may be better conceptualized as causal rather than as effect indicators of the latent construct of EF. The use of CFA methods with EF tasks that are weakly correlated results in latent variables that have poor to modest maximal reliability. This undermines efforts to use latent variables of EF as predictors or outcomes in empirical studies and has implications for theory development. Greater attention to and resolution of the discrepancies between conceptual definitions and statistical approaches that are used to model EF has implications for the long-term value of using EF in research and clinical settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0734 · Cited by 15,798
Missing data analysis: making it work in the real world
Graham, J. W. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085530 · Cited by 6,163
Authentic leadership development: Getting to the root of positive forms of leadership
Avolio, B. J., & Gardner, W. L. (2005) · The Leadership Quarterly · doi:10.1016/j.leaqua.2005.03.001 · Cited by 5,478
A gradient of childhood self-control predicts health, wealth, and public safety
Moffitt, T. E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R. J., Harrington, H., Houts, R., Poulton, R., Roberts, B. W., Ross, S., et al. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1010076108 · Cited by 4,885 · cited 2×
“…including school readiness and academic competence, financial well-being, psychopathology, and even performance among professional athletes (Blair, 2002; Mischel, Shoda, & Rodriguez, 1989; Moffitt et al., 2011; Pennington & Ozonoff, 1996; Vestberg, Gustafson, Maurex, Ingvar, & Petrovic, 2012).”
BackgroundMiyake, A., & Friedman, N. P. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721411429458 · Cited by 3,944 · cited 2×
“For example, IC tasks may be best represented as causal indicators, while WM and AS tasks may be best represented as effect indicators (see e.g., Huizinga et al., 2006; Miyake & Friedman, 2012).”
BackgroundConventional wisdom on measurement: A structural equation perspective.
Bollen, K., & Lennox, R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.305 · Cited by 3,618
Delay of gratification in children.
Mischel, W., Shoda, Y., & Rodriguez, M. L. (1989) · Science · doi:10.1126/science.2658056 · Cited by 3,397
Stress signalling pathways that impair prefrontal cortex structure and function
Arnsten, A. F. T. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2648 · Cited by 3,299
Executive functions and developmental psychopathology.
Pennington, B. F., & Ozonoff, S. (1996) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1996.tb01380.x · Cited by 3,150 · cited 1×
“…including school readiness and academic competence, financial well-being, psychopathology, and even performance among professional athletes (Blair, 2002; Mischel, Shoda, & Rodriguez, 1989; Moffitt et al., 2011; Pennington & Ozonoff, 1996; Vestberg, Gustafson, Maurex, Ingvar, & Petrovic, 2012).”
Executive function in preschoolers: A review using an integrative framework.
Garon, N., Bryson, S. E., & Smith, I. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.31 · Cited by 2,695
A Developmental Perspective on Executive Function
Best, J. R., & Miller, P. H. (2010) · Child Development · doi:10.1111/j.1467-8624.2010.01499.x · Cited by 2,692
Ego depletion and the strength model of self-control: a meta-analysis
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010) · Psychological Bulletin · doi:10.1037/a0019486 · Cited by 2,265
Davidson, M. C., Amso, D., Anderson, L. C., & Diamond, A. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.02.006 · Cited by 2,226
Developmentally Sensitive Measures of Executive Function in Preschool Children
Carlson, S. M. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_3 · Cited by 1,638 · cited 1×
“A voluminous number of performance-based tasks exist to measure EF across the life span (Carlson, 2005; Chan, Shum, Toulopoulou, & Chen, 2008).”
BackgroundAge-related change in executive function: Developmental trends and a latent variable analysis
Huizinga, M., Dolan, C. V., & van der Molen, M. W. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.01.010 · Cited by 1,636 · cited 1×
“For example, IC tasks may be best represented as causal indicators, while WM and AS tasks may be best represented as effect indicators (see e.g., Huizinga et al., 2006; Miyake & Friedman, 2012).”
BackgroundMeta-Analysis of Neurobehavioral Outcomes in Very Preterm and/or Very Low Birth Weight Children
Aarnoudse-Moens, C. S. H., Weisglas-Kuperus, N., van Goudoever, J. B., & Oosterlaan, J. (2009) · Pediatrics · doi:10.1542/peds.2008-2816 · Cited by 1,587
Individual differences in executive functions are almost entirely genetic in origin.
Friedman, N. P., Miyake, A., Young, S. E., DeFries, J. C., Corley, R. P., & Hewitt, J. K. (2008) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.137.2.201 · Cited by 1,547
Socioeconomic status and the brain: mechanistic insights from human and animal research
Hackman, D. A., Farah, M. J., & Meaney, M. J. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2897 · Cited by 1,409
Assessment of executive functions: Review of instruments and identification of critical issues.
CHAN, R., SHUM, D., TOULOPOULOU, T., & CHEN, E. (2008) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2007.08.010 · Cited by 1,401
Bernier, A., Carlson, S. M., & Whipple, N. (2010) · Child Development · doi:10.1111/j.1467-8624.2009.01397.x · Cited by 1,388
Best, J. R. (2010) · Developmental Review · doi:10.1016/j.dr.2010.08.001 · Cited by 1,078 · cited 1×
“…birth, traumatic brain injury) influences (Aarnoudse-Moens, Weisglas-Kuperus, van Goudoever, & Oosterlaan, 2009; Bernier, Carlson, & Whipple, 2010; Best, 2010; Blair et al., 2011; Canfield, Gendle, & Cory-Slechta, 2004; Chaddock, Pontifex, Hillman, & Kramer, 2011; Haase, McCaffrey,…”
Wiebe, S. A., Espy, K. A., & Charak, D. (2008) · Developmental Psychology · doi:10.1037/0012-1649.44.2.575 · Cited by 829 · cited 1×
“Only one study had a latent variable that had a maximal reliability greater than .80 (Wiebe et al., 2008).”
BackgroundCallicott, J. H. (1999) · Cerebral Cortex · doi:10.1093/cercor/9.1.20 · Cited by 771
Activities and Programs That Improve Children’s Executive Functions
Diamond, A. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721412453722 · Cited by 750 · cited 2×
“Despite the fact that nearly all of these results come from observational designs, which provide a weak basis of inference regarding the potentially causal role of EF in predicting outcomes, there is a growing interest in developing programs designed to enhance or facilitate EF (Bryck & Fisher, 2012; Diamond, 2012).”
BackgroundA shift from diffuse to focal cortical activity with development
Durston, S., Davidson, M. C., Tottenham, N., Galvan, A., Spicer, J., Fossella, J. A., & Casey, B. J. (2006) · Developmental Science · doi:10.1111/j.1467-7687.2005.00454.x · Cited by 692
Blair, C., Granger, D. A., Willoughby, M., Mills-Koonce, R., Cox, M., Greenberg, M. T., Kivlighan, K. T., Fortunato, C. K., & the FLP Investigators (2011) · Child Development · doi:10.1111/j.1467-8624.2011.01643.x · Cited by 579 · cited 1×
“…brain injury) influences (Aarnoudse-Moens, Weisglas-Kuperus, van Goudoever, & Oosterlaan, 2009; Bernier, Carlson, & Whipple, 2010; Best, 2010; Blair et al., 2011; Canfield, Gendle, & Cory-Slechta, 2004; Chaddock, Pontifex, Hillman, & Kramer, 2011; Haase, McCaffrey, Santiago-Rivera, Morse, &…”
Self-citationA Model of the Development of Frontal Lobe Functioning: Findings From a Meta-Analysis
Romine, C. B., & Reynolds, C. R. (2005) · Applied Neuropsychology · doi:10.1207/s15324826an1204_2 · Cited by 526 · cited 1×
“EF exhibits a protracted course of development, beginning with nascent abilities that are observable at the end of the first year of life and that continue to change across the life span (Davidson, Amso, Anderson, & Diamond, 2006; De Luca & Leventer, 2008; Gur et al., 2012; Romine & Reynolds, 2005).”
BackgroundGur, R. C., Richard, J., Calkins, M. E., Chiavacci, R., Hansen, J. A., Bilker, W. B., Loughead, J., Connolly, J. J., Qiu, H., Mentch, F. D., et al. (2012) · Neuropsychology · doi:10.1037/a0026712 · Cited by 507 · cited 1×
“EF exhibits a protracted course of development, beginning with nascent abilities that are observable at the end of the first year of life and that continue to change across the life span (Davidson, Amso, Anderson, & Diamond, 2006; De Luca & Leventer, 2008; Gur et al., 2012; Romine & Reynolds, 2005).”
BackgroundThe Fallacy of Formative Measurement
Edwards, J. R. (2011) · Organizational Research Methods · doi:10.1177/1094428110378369 · Cited by 495
Chaddock, L., Pontifex, M. B., Hillman, C. H., & Kramer, A. F. (2011) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617711000567 · Cited by 369
Ecological validity of a simplified version of the multiple errands shopping test
ALDERMAN, N., BURGESS, P. W., KNIGHT, C., & HENMAN, C. (2003) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617703910046 · Cited by 337
Neural correlates of cognitive efficiency
Rypma, B., Berger, J. S., Prabhakaran, V., Martin Bly, B., Kimberg, D. Y., Biswal, B. B., & D'Esposito, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.05.065 · Cited by 334
Memory Maintenance and Inhibitory Control Differentiate from Early Childhood to Adolescence
Shing, Y. L., Lindenberger, U., Diamond, A., Li, S.-C., & Davidson, M. C. (2010) · Developmental Neuropsychology · doi:10.1080/87565641.2010.508546 · Cited by 239
Through the looking glass: Differential noradenergic modulation of prefrontal cortical function
Arnsten, A. F. T. (2000) · Neural Plasticity · doi:10.1155/np.2000.133 · Cited by 229 · cited 1×
“However, as catecholamine levels continue to rise beyond an optimal level, these receptors become saturated and neurons in the PFC become inactive (Arnsten, 2000, 2009; Arnsten & Simon, 2000).”
BackgroundLillard, A. S. (2012) · Journal of School Psychology · doi:10.1016/j.jsp.2012.01.001 · Cited by 224
Executive Functions and the Frontal Lobes
(2010) · doi:10.4324/9780203837863 · Cited by 158
Developmental shifts in fMRI activations during visuospatial relational reasoning
Eslinger, P. J., Blair, C., Wang, J., Lipovsky, B., Realmuto, J., Baker, D., Thorne, S., Gamson, D., Zimmerman, E., Rohrer, L., et al. (2009) · Brain and Cognition · doi:10.1016/j.bandc.2008.04.010 · Cited by 75
Self-citationWilloughby, M. T., Family Life Project Investigators, Pek, J., & Blair, C. B. (2013) · Psychological Assessment · doi:10.1037/a0031747 · Cited by 70
Self-citationOn the Relationship Between Maximal Reliability and Maximal Validity of Linear Composites
Penev, S., & Raykov, T. (2006) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr4102_1 · Cited by 26 · cited 1×
“…reliability for the latent construct of EF (or its subdimensions IC, WM, AS) are problematic in that they contribute to a lack of replicability of results across studies and reduce statistical power for group comparisons involving latent variables of EF (Hancock, 2001; Penev & Raykov, 2006).”
Background
Cited by
Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018) · Psychological Bulletin · doi:10.1037/bul0000160 · Cited by 637
“…the low reliability (Baggetta & Alexander, 2016; Schmidt, 2003) and low intertest correlations often observed for executive function tests (Willoughby et al., 2014), the manifest variables included in the reanalysis could have had limited construct variance related to the factor(s) on…”
InfluentialAnomalous results in G-factor models: Explanations and alternatives.
Eid, M., Geiser, C., Koch, T., & Heene, M. (2017) · Psychological Methods · doi:10.1037/met0000083 · Cited by 439
Down syndrome: Cognitive and behavioral functioning across the lifespan
Grieco, J., Pulsifer, M., Seligsohn, K., Skotko, B., & Schwartz, A. (2015) · American Journal of Medical Genetics Part C: Seminars in Medical Genetics · doi:10.1002/ajmg.c.31439 · Cited by 377
Executive function in the first three years of life: Precursors, predictors and patterns
Hendry, A., Jones, E. J. H., & Charman, T. (2016) · Developmental Review · doi:10.1016/j.dr.2016.06.005 · Cited by 297
Developmental science and executive function
Blair, C. (2016) · Current Directions in Psychological Science · doi:10.1177/0963721415622634 · Cited by 271
Self-citation
Metadata
- Authors
- Willoughby, M., Holochwost, S. J., Blanton, Z. E., & Blair, C. B.
- Venue
- Measurement: Interdisciplinary Research and Perspectives
- Publisher
- Informa UK Limited
- Published
- 2014-07-03
- Volume
- 12
- Issue
- 3
- Pages
- 69-95
- ISSN
- 1536-6359 (ISSN-L)
- 1536-6367
- OpenAlex
- W2052053712
- Cited by
- 73
- References
- 39
- FWCI
- 10.27
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.