Measuring executive function in early childhood: A focus on maximal reliability and the derivation of short forms.
Open accessAbstract
This study assesses the maximal reliability of a newly developed battery of executive function (EF) tasks for use in early childhood. It also demonstrates how changes in maximal reliability can inform the selection of different short forms of the battery, depending on child age. Participants included children from the Family Life Project (Vernon-Feagans, Cox, & Family Life Project Investigators, in press)--a prospective longitudinal study (N = 1,292) of families that were recruited at the time of the birth of a new child--who were assessed at ages 3, 4, and 5 years. Results indicate that the EF battery had reasonably good maximal reliability (ρ = .73, 95% confidence interval [CI] = .69, .76) in a mixed-age sample of children randomly selected from assessments at ages 3, 4, and 5. In contrast, maximal reliability of the battery ranged from poor to modest for within-age samples (ρs = .47 [95% CI = .37, .52], .62 [.57, .66], and .61 [.55, .66] at ages 3, 4, and 5, respectively). Although the derivation of a 3-task short form of the battery always resulted in statistically significant decrements in maximal reliability, in some cases the relative decrement was quite modest and may be tolerable given the time savings and potential reduction in participant burden. The benefits of using maximal reliability to both evaluate task batteries and derive short forms are discussed, as well as how a focus on maximal reliability informs ongoing questions about the measurement and conceptualization of EF in early childhood.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The EFs composite included scores for children who passed comprehension criteria for three or more tasks, in line with research showing a three-task battery is acceptable.
“the EFs composite included scores for children who passed comprehension criteria for three or more tasks (93% of children in the current sample, M = 5.36 tasks, SD = 1.05).”
From Early Family Experiences and Neural Activity in Rural Pakistani Children: The Differential Role of Gender (2024) · cites this paper as Willoughby, Pek, and Blair 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0734 · Cited by 15,798 · see citing works
Miyake, A., & Friedman, N. P. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721411429458 · Cited by 3,944 · see citing works
Executive function in preschoolers: A review using an integrative framework.
Garon, N., Bryson, S. E., & Smith, I. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.31 · Cited by 2,695 · see citing works
Kane, M. J., & Engle, R. W. (2003) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.132.1.47 · Cited by 1,725 · cited 1× · see citing works
“This spanlike task is based upon principles described by Engle, Kane and collaborators (e.g., Kane & Engle, 2003).”
The structure of executive function in 3-year-olds.
Wiebe, S. A., Sheffield, T., Nelson, J. M., Clark, C. A. C., Chevalier, N., & Espy, K. A. (2011) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2010.08.008 · Cited by 776 · see citing works
A neural basis for the development of inhibitory control
Durston, S., Thomas, K. M., Yang, Y., Uluğ, A. M., Zimmerman, R. D., & Casey, B. J. (2002) · Developmental Science · doi:10.1111/1467-7687.00235 · Cited by 684 · see citing works
A Model of the Development of Frontal Lobe Functioning: Findings From a Meta-Analysis
Romine, C. B., & Reynolds, C. R. (2005) · Applied Neuropsychology · doi:10.1207/s15324826an1204_2 · Cited by 526 · see citing works
Willoughby, M. T., The Family Life Project Investigators, Blair, C. B., Wirth, R. J., & Greenberg, M. (2010) · Psychological Assessment · doi:10.1037/a0018708 · Cited by 280 · see citing works
Self-citationMemory Maintenance and Inhibitory Control Differentiate from Early Childhood to Adolescence
Shing, Y. L., Lindenberger, U., Diamond, A., Li, S.-C., & Davidson, M. C. (2010) · Developmental Neuropsychology · doi:10.1080/87565641.2010.508546 · Cited by 239 · see citing works
Willoughby, M. T., Wirth, R. J., & Blair, C. B. (2011) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2010.04.007 · Cited by 102 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 23
- Anderson PJ Anderson V Jacobs R Anderson PJ Towards a developmental model of executive function. Executive functions and the frontal lobes: a lifespan perspective 2008 3 22 Psychology Press New York
- Arnsten AFT Catecholamine modulation of prefrontal cortical cogniive function. Trends in Cognitive Sciences 1998 2 436 447 21227275
- Arnsten AFT Stress impairs prefrontal cortical function in rats and monkeys: role of dopamine D1 and norepinephrine alpha-1 receptor mechanisms. Cognition, Emotion and Autonomic Responses: The Integrative Role of the Prefrontal Cortex and Limbic Structures 2000 126 183 192 11105647
- Bryck RL Fisher PA Training the Brain Practical Applications of Neural Plasticity From the Intersection of Cognitive Neuroscience, Developmental Psychology, and Prevention Science. American Psychologist 2012 67 2 87 100 doi: Doi 10.1037/A0024657 21787037 PMC3335430 · DOI
- Cragg L Nation K Self-ordered pointing as a test of working memory in typically developing children. Memory 2007 15 5 526 535 17613795
- Ettenhofer ML Hambrick DZ Abeles N Reliability and stability of executive functioning in older adults. Neuropsychology 2006 20 5 607 613 16938023
- Gerstadt C Hong Y Diamond A The relationship between cognition and action: performance of children 3 1/2 -7 years old on a Stroop-like day-night test. Cognition 1994 53 129 153 7805351
- Graham JW Taylor BJ Olchowski AE Cumsille PE Planned missing data designs in psychological research. Psychological Methods 2006 11 4 323 343 17154750
- Hancock GR Mueller RO The Reliability Paradox in Assessing Structural Relations Within Covariance Structure Models. Educational and Psychological Measurement 2011 71 2 306 324 doi: Doi 10.1177/0013164410384856 · DOI
- Hughes C Ensor R Wilson A Graham A Tracking Executive Function Across the Transition to School: A Latent Variable Approach. Developmental Neuropsychology 2010 35 1 20 36 20390590
- Jacques S Zelazo PD The Flexible Item Selection Task (FIST): A measure of executive function in preschoolers. Developmental Neuropsychology 2001 20 3 573 591 12002094
- Nunnally JC Bernstein IH Psychometric Theory 1994 McGraw-Hill New York 3rd ed.
- Penev S Raykov T On the relationship between maximal reliability and maximal validity of linear composites. Multivariate Behavioral Research 2006 41 2 105 126 26782906
- Petrides M Milner B Deficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man. Neuropsychologia 1982 20 3 249 262 7121793
- Rabbitt P Rabbitt P Introduction: Methodologies and models in the study of executive function. Methodology of Frontal and Executive Function 1997 1 38 Psychology Press
- Raykov T Estimation of maximum reliability: A note on a covariance structure modeling approach. British Journal of Mathematical & Statistical Psychology 2004 57 21 27 15171799
- Raykov T Hancock GR Examining change in maximal reliability for multiple-component measuring instruments. British Journal of Mathematical & Statistical Psychology 2005 58 65 82 doi: Doi 10.1348/000711005x38753 15969840 · DOI
- Schoemaker K Bunte T Wiebe SA Espy KA Dekovic M Matthys W Executive function deficits in preschool children with ADHD and DBD. Journal of Child Psychology and Psychiatry 2012 53 2 111 119 doi: DOI 10.1111/j.1469-7610.2011.02468.x 22022931 · DOI
- Vernon-Feagans L Cox M Investigators FLPK The Family Life Project: An Epidemiological and Developmental Study of Young Children Living in Poor Rural Communities Monographs of the Society for Research in Child Development (in press) 24147448
- Wiebe SA Espy KA Charak D Using confirmatory factor analysis to understand executive control in preschool children: I. Latent structure. Developmental Psychology 2008 44 2 575 587 18331145
- Willoughby MT Blair CB Test-retest reliability of a new executive function battery for use in early childhood. Child Neuropsychology 2011 17 6 564 579 21714751
- Willoughby MT Blair CB Wirth RJ Greenberg M Investigators FLP The measurement of executive function at age 5: Psychometric properties and relationship to academic achievement. Psychological Assessment 2012 24 1 226 239 21966934 PMC3315058
- Willoughby MT Wirth RJ Blair CB Investigators FLP Executive function in early childhoood: Longitudinal measurement invariance and developmental change. Psychological Assessment 2012 24 2 418 431 22023561 PMC3495312
Cited by
Sorted by citations
Developmental science and executive function
Blair, C. (2016) · Current Directions in Psychological Science · doi:10.1177/0963721415622634 · Cited by 271 · see citing works
Self-citationMeasuring executive function in early childhood: A case for formative measurement.
Willoughby, M. T., The Family Life Project Investigators, & Blair, C. B. (2016) · Psychological Assessment · doi:10.1037/pas0000152 · Cited by 120 · see citing works
Self-citationDick, A. S. (2014) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2014.01.021 · Cited by 77 · see citing works
A framework for executive function in the late elementary years.
Cirino, P. T., Ahmed, Y., Miciak, J., Taylor, W. P., Gerst, E. H., & Barnes, M. A. (2018) · Neuropsychology · doi:10.1037/neu0000427 · Cited by 71 · see citing works
Lopera‐Perez, D. C., Obradović, J., Yousafzai, A. K., Keehn, B., Siyal, S., Nelson, C. A., & Tarullo, A. R. (2024) · Developmental Psychobiology · doi:10.1002/dev.22534 · Cited by 1 · see citing works
- Authors
- Willoughby, M. T., Family Life Project Investigators, Pek, J., & Blair, C. B.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- York University (CA)
- New York University (US)
- Venue
- Psychological Assessment
- Published
- 2013
- Volume
- 25
- Issue
- 2
- Pages
- 664-670
- ISSN
- 1040-3590 (ISSN-L)
- 1939-134X
- PMC
- PMC4203363
- OpenAlex
- W2045150180
- Cited by
- 70
- References
- 10
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 8 · 2023: 2 · 2022: 8 · 2021: 1 · 2020: 1 · 2019: 3 · 2018: 10 · 2017: 4 · 2016: 10 · 2015: 2 · 2014: 7 · 2013: 3
- FWCI
- 10.00
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Psychological Assessment (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Child Health and Human Development (R01 HD51502 and P01 HD39667)
- National Institute on Drug Abuse
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
- Social Sciences › Education › Early Childhood Education and Development
- Keywords
- executive function, early childhood, test battery development, child age differences, Family Life Project, cognitive assessment
- MeSH terms
- Child Development, Child, Preschool, Cross-Sectional Studies, Female, Humans, Male, Neuropsychological Tests, Prospective Studies, Psychometrics, United States, Reproducibility of Results, Executive Function
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
