Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration
Open accessAbstract
This study demonstrates the merits of evaluating a newly developed battery of executive function tasks, designed for use in early childhood, from the perspective of item response theory (IRT). The battery was included in the 48-month assessment of the Family Life Project, a prospective longitudinal study of 1292 children oversampled from low-income and African American families. IRT models were applied to a select set of tasks to demonstrate empirically (a) a principled method for item evaluation, including the utility of item characteristic curves; (b) how to explicitly test whether the measurement properties of executive function tasks are invariant across mutually exclusive subgroups of youths; (c) how the precision of measurement of a given task can vary according to underlying child ability; and (d) the utility of using IRT-based versus percentage correct scores. Results are discussed with respect to the importance of developing psychometrically sound and scalable instruments that facilitate the measurement of interindividual differences in intraindividual change of executive function across the early childhood period.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Self-ordered searching tasks are used with preschoolers and require information, usually visual-spatial, to be maintained and updated in short-term memory.
“Another working memory paradigm used with preschoolers is self-ordered searching tasks, which requires information, usually visual-spatial, to be maintained and updated in short-term memory.”
From Assessing Executive Function in Preschoolers (2012) · cites this paper as M. T. Willoughby, Wirth, & Blair, 2011
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · cited 1× · see citing works
“A series of exact and approximate fit indices were used to evaluate the fit of each model as per the guidelines outlined by Hu and Bentler (1999).”
MethodCoefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · cited 1× · see citing works
“Second, reliability measures (e.g., coefficient alpha) for CTT-based scores assume that all scores are equally reliable (see, for example, Cronbach, 1951).”
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0734 · Cited by 15,798 · cited 1× · see citing works
“N IH -PA Author M anuscript N IH -PA Author M anuscript N IH -PA Author M anuscript early childhood (Miyake et al, 2000; Wiebe et al, 2008).”
BackgroundAn Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · cited 1× · see citing works
“…NC 27599 2 Department of Applied Psychology, 239 Greene St, East Bldg 500, New York University, New York NY 10003 Executive functions refer to cognitive abilities involved in the control and coordination of information in the service of goal-directed actions (Fuster, 1997; Miller & Cohen, 2001).”
BackgroundRozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536 · cited 2× · see citing works
“Testing for such differences is broadly known in the psychometric literature as testing for measurement invariance (Meredith, 1993).”
Background(2000) · Cited by 4,106 · see citing works
Executive function in preschoolers: A review using an integrative framework.
Garon, N., Bryson, S. E., & Smith, I. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.31 · Cited by 2,695 · see citing works
ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES1
Samejima, F. (1968) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1968.tb00153.x · Cited by 2,605 · cited 1× · see citing works
“Ordinal scored (Likert type) items cannot be accommodated by the 2-parameter logistic model; hence the Pick the Picture task was fit using an alternative model— Samejima’s (1969) graded response model.”
MethodThe Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe
Fuster, J. M. (1980) · Cited by 2,367 · see citing works
Davidson, M. C., Amso, D., Anderson, L. C., & Diamond, A. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.02.006 · Cited by 2,226 · see citing works
Hambleton, R. K., & Swaminathan, H. (1985) · doi:10.1007/978-94-017-1988-9 · Cited by 1,777 · see citing works
Principles of Frontal Lobe Function
Stuss, D. T. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195134971.001.0001 · Cited by 1,708 · see citing works
Developmentally Sensitive Measures of Executive Function in Preschool Children
Carlson, S. M. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_3 · Cited by 1,638 · cited 2× · see citing works
“It is probably not too far of an exaggeration to suggest that there exists a “cottage industry” around the development of Executive Function tasks for use with young children (see e.g., Carlson, 2005).”
BackgroundGerstadt, C. L., Hong, Y. J., & Diamond, A. (1994) · Cognition · doi:10.1016/0010-0277(94)90068-x · Cited by 1,598 · cited 1× · see citing works
“This task was derived from the Day-Night task developed by Gerstadt, Hong, and Diamond (1994).”
MethodLinks between behavioral regulation and preschoolers’ literacy, vocabulary, and math skills.
McClelland, M. M., Cameron, C. E., Connor, C. M., Farris, C. L., Jewkes, A. M., & Morrison, F. J. (2007) · Developmental Psychology · doi:10.1037/0012-1649.43.4.947 · Cited by 1,561 · cited 1× · see citing works
“For example, on measures such as peg tapping (Diamond & Taylor, 1996) or the Head-ToesKnees-Shoulders (McClelland et al., 2007), many children are able to successfully complete the first several items, and it is later rather than earlier items that discriminate ability on these measures.”
BackgroundDeficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man
Petrides, M., & Milner, B. A. (1982) · Neuropsychologia · doi:10.1016/0028-3932(82)90100-2 · Cited by 1,272 · cited 1× · see citing works
“This is a Self-Ordered Pointing task (Petrides & Milner, 1982; Cragg & Nation, 2007).”
MethodThe prefrontal cortex: Anatomy, physiology, and neuropsychology of the frontal lobe
Holmes, G. L. (1989) · Journal of Epilepsy · doi:10.1016/0896-6974(89)90035-2 · Cited by 1,210 · see citing works
Socioeconomic gradients predict individual differences in neurocognitive abilities.
Noble, K. G., McCandliss, B. D., & Farah, M. J. (2007) · Developmental Science · doi:10.1111/j.1467-7687.2007.00600.x · Cited by 1,041 · cited 1× · see citing works
“Second, an emerging body of work has demonstrated that poverty is predictive of executive functions in early childhood (Blair et al, submitted; Noble et al, 2007).”
BackgroundItem factor analysis: Current approaches and future directions.
Wirth, R. J., & Edwards, M. C. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.58 · Cited by 917 · cited 3× · see citing works
“The discrimination parameter ranges from 0 to infinity, but values greater than approximately 4 for a uni-dimensional model are generally considered problematic (Wirth & Edwards, 2007).”
BackgroundInfluentialSelf-citationWiebe, S. A., Espy, K. A., & Charak, D. (2008) · Developmental Psychology · doi:10.1037/0012-1649.44.2.575 · Cited by 829 · cited 1× · see citing works
“N IH -PA Author M anuscript N IH -PA Author M anuscript N IH -PA Author M anuscript early childhood (Miyake et al, 2000; Wiebe et al, 2008).”
BackgroundExecutive Function in Typical and Atypical Development
Zelazo, P. D., & Müller, U. (2010) · The Wiley‐Blackwell Handbook of Childhood Cognitive Development · doi:10.1002/9781444325485.ch22 · Cited by 784 · cited 1× · see citing works
“…of prepotent or automatized responding when engaged in task completion; and mental flexibility, defined as the ability to shift attentional or cognitive set among distinct but related dimensions or aspects of a given task (Davidson et al., 2006; Garon, Bryson, & Smith, 2008; Zelazo & Müller, 2002).”
BackgroundConflict adaptation effects in the absence of executive control
Mayr, U., Awh, E., & Laurey, P. (2003) · Nature Neuroscience · doi:10.1038/nn1051 · Cited by 765 · cited 2× · see citing works
“As noted above, however, it is very likely given the nature of EF and the inherent difficulty of switching between congruent and incongruent items on a given task, that items involving switches rather than repetitions are more informative than items that maintain a pattern of responding, whether congruent or incongrue…”
BackgroundThe domain of supervisory processes and temporal organization of behaviour.
Shallice, T., & Burgess, P. (1996) · Philosophical transactions of the Royal Society of London. Series B, Biological sciences · doi:10.1098/rstb.1996.0124 · Cited by 728 · cited 1× · see citing works
“As such, executive function can be defined as a supervisory system that is important for planning, reasoning ability, and the integration of thought and action (Shallice & Burgess, 1996).”
BackgroundOn the Relationship between iTem Response Theory and Factor Analysis of Discretized Variables
Takane, Y., & de Leeuw, J. (1987) · Psychometrika · doi:10.1007/bf02294363 · Cited by 699 · see citing works
Full-Information Item Bi-Factor Analysis
Gibbons, R. D., & Hedeker, D. (1992) · Psychometrika · doi:10.1007/bf02295430 · Cited by 599 · see citing works
Blair, C., Granger, D. A., Willoughby, M., Mills-Koonce, R., Cox, M., Greenberg, M. T., Kivlighan, K. T., Fortunato, C. K., & the FLP Investigators (2011) · Child Development · doi:10.1111/j.1467-8624.2011.01643.x · Cited by 579 · see citing works
Self-citationThe domain of supervisory processes and the temporal organization of behaviour
Shallice, T., & Burgess, P. W. (1998) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198524410.003.0003 · Cited by 500 · see citing works
The Flexible Item Selection Task (FIST): A Measure of Executive Function in Preschoolers
Jacques, S., & Zelazo, P. D. (2001) · Developmental Neuropsychology · doi:10.1207/s15326942dn2003_2 · Cited by 332 · see citing works
The Measurement of Executive Function in Early Childhood
Blair, C., Zelazo, P. D., & Greenberg, M. T. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_1 · Cited by 328 · see citing works
Self-citationThe Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe, 3rd ed.
Spinks, R. A. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.6.1038 · Cited by 286 · see citing works
Full-Information Item Bifactor Analysis of Graded Response Data
Gibbons, R. D., Bock, R. D., Hedeker, D., Weiss, D. J., Segawa, E., Bhaumik, D. K., Kupfer, D. J., Frank, E., Grochocinski, V. J., & Stover, A. M. (2006) · Applied Psychological Measurement · doi:10.1177/0146621606289485 · Cited by 271 · see citing works
Recent Developments in the Factor Analysis of Categorical Variables
Mislevy, R. J. (1986) · Journal of Educational Statistics · doi:10.3102/10769986011001003 · Cited by 257 · cited 1× · see citing works
“…confirmatory factor analysis (CFA) methods were used to examine the dimensionality of tasks while IRT methods were used to better understand item and score characteristics (also see Mislevy, 1986, and Takane & de Leeuw, 1987, for information regarding the relationship between item-factor models).”
MethodLloyd, K. E. (1994) · The Behavior Analyst · doi:10.1007/bf03392658 · Cited by 221 · see citing works
The Blackwell companion to sociology
(2001) · Choice Reviews Online · doi:10.5860/choice.39-1248 · Cited by 188 · see citing works
Woods, C. M., & Thissen, D. (2006) · Psychometrika · doi:10.1007/s11336-004-1175-8 · Cited by 121 · cited 1× · see citing works
“The bottom panel presents ivThe assumption of an underlying normally distributed population is not required (Woods & Thissen, 2006).”
BackgroundRecent Developments in the Factor Analysis of Categorical Variables
Mislevy, R. J. (1986) · Journal of Educational Statistics · doi:10.2307/1164846 · Cited by 110 · see citing works
Hill, C., Edwards, M. C., Thissen, D., Langer, M., Wirth, R., Burwinkle, T. M., & Varni, J. W. (2007) · Medical Care · doi:10.1097/01.mlr.0000259879.05499.eb · Cited by 95 · cited 1× · see citing works
“If model fit was poor (i.e., CFA < .90, TLI < .90, & RMSEA > .08), modification indices, standardized residuals (see, for example, Hill et al., 2007), and discussions among the authors about possible item dependencies were used to inform the fitting of bi-factor models, which take into account…”
MethodSelf-citationModeling Ecological and Contextual Effects in Longitudinal Studies of Human Development
Card, N. A., Little, T. D., & Bovaird, J. A. (2007) · doi:10.4324/9780203936825-5 · Cited by 93 · see citing works
Self-ordered pointing as a test of working memory in typically developing children
Cragg, L., & Nation, K. (2007) · Memory · doi:10.1080/09658210701390750 · Cited by 85 · cited 1× · see citing works
“This is a Self-Ordered Pointing task (Petrides & Milner, 1982; Cragg & Nation, 2007).”
MethodMeasurement and the Study of Change
Edwards, M. C., & Wirth, R. J. (2009) · Research in Human Development · doi:10.1080/15427600902911163 · Cited by 78 · cited 3× · see citing works
“Incorrectly weighting items can lead to scale scores that are biased and thereby less accurate when comparing individuals (or groups) within or over time (Edwards & Wirth, 2009; Wirth 2008).”
BackgroundSelf-citationThe Blackwell Companion to Sociology
(2004) · Blackwell Publishing Ltd eBooks · doi:10.1111/b.9781405122672.2004.x · Cited by 67 · see citing works
Special Topics in Item Response Theory
Edwards, M. C., & Edelen, M. O. (2009) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020994.n8 · Cited by 26 · cited 1× · see citing works
“This study used the non-anchor (also commonly referred to constrain-all/test-all) approach for testing for (see Edwards & Edelen, 2009, for a recent review of DIF methods).”
MethodDill, B. T., & Myers, S. L. (2004) · doi:10.1002/9780470693452.ch14 · Cited by 22 · see citing works
The effects of measurement non-invariance on parameter estimation in latent growth models
Wirth, R. J. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/tgzh-a739 · Cited by 13 · see citing works
Self-citationDiamond, A., & Taylor, C. (1996) · Developmental Psychobiology · doi:10.1002/(sici)1098-2302(199605)29:4
Flagged first, then by citations
3 of the 50 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Cai, L., du Toit, H. S. C., & Thissen, D. (in press). IRTPro: Flexible, multidimensional, multiple categorical IRT modeling [computer software]. Chicago: SSI International.
Cited by
Sorted by citations
McClelland, M. M., & Cameron, C. E. (2011) · Child Development Perspectives · doi:10.1111/j.1750-8606.2011.00191.x · Cited by 660 · see citing works
Executive function in early childhood: Longitudinal measurement invariance and developmental change.
Willoughby, M. T., Family Life Project Investigators, Wirth, R. J., & Blair, C. (2011) · Psychological Assessment · doi:10.1037/a0025779 · Cited by 368 · see citing works
Self-citationChanges and challenges in 20 years of research into the development of executive functions
Hughes, C. (2011) · Infant and Child Development · doi:10.1002/icd.736 · Cited by 347 · see citing works
“Applying this approach, Willoughby et al. (2010) were able to show that three distinct EF tasks (a self-ordered search test of working memory, a ‘Silly Sounds’ non-verbal Stroop test of inhibitory control, and an attention set shifting task) showed similar ICCs over repeated intervals and for…”
BackgroundMethodReview of measures of social and emotional development
Halle, T., & Darling, K. E. (2016) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2016.02.003 · Cited by 354 · see citing works
Assessing Executive Function in Preschoolers
Anderson, P. J., & Reidy, N. (2012) · Neuropsychology Review · doi:10.1007/s11065-012-9220-3 · Cited by 293 · see citing works
“Examples of this approach are the Self-ordered Pointing and Pick the Picture tests (Hongwanishkul et al. 2005; Willoughby et al. 2011).”
BackgroundMethodPace, A. E., Alper, R. M., Burchinal, M., Golinkoff, R. M., & Hirsh‐Pasek, K. (2018) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2018.04.001 · Cited by 279 · see citing works
Blair, C., Family Life Project Investigators, Raver, C. C., & Berry, D. J. (2013) · Developmental Psychology · doi:10.1037/a0033647 · Cited by 235 · see citing works
“Details on the tasks and administration procedures as well as psychometric characteristics are available in Willoughby et al. (2011) and Willoughby et al. (2010).”
Self-citationFitzpatrick, C., McKinnon, R. D., Blair, C., & Willoughby, M. T. (2013) · Learning and Instruction · doi:10.1016/j.learninstruc.2013.11.003 · Cited by 216 · see citing works
Self-citationDevelopment and Self‐Regulation
McClelland, M., Geldhof, G., Cameron, C., & Wanless, S. B. (2015) · Wiley eBooks · doi:10.1002/9781118963418.childpsy114 · Cited by 193 · see citing works
Further examination of the immediate impact of television on children's executive function
Lillard, A. S., Drell, M. B., Richey, E. M., Boguszewski, K., & Smith, E. D. (2015) · Developmental Psychology · doi:10.1037/a0039097 · Cited by 210 · see citing works
Sulik, M. J., Blair, C., Mills-Koonce, R., Berry, D. J., Greenberg, M. T., The Family Life Project Investigators, Vernon-Feagans, L., Burchinal, M. R., Cox, M., Garrett-Peters, P. T., et al. (2015) · Child Development · doi:10.1111/cdev.12386 · Cited by 188 · see citing works
Self-citationIndividual development and evolution: Experiential canalization of self-regulation.
Blair, C., & Raver, C. C. (2012) · Developmental Psychology · doi:10.1037/a0026472 · Cited by 174 · see citing works
Self-citationBlair, C., Family Life Project Investigators, Ursache, A., Greenberg, M., & Vernon-Feagans, L. (2015) · Developmental Psychology · doi:10.1037/a0038813 · Cited by 234 · see citing works
“A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript approach for the battery are available in Willoughby, Wirth, and Blair (2011).”
BackgroundMethodInfluentialSelf-citationMontessori Preschool Elevates and Equalizes Child Outcomes: A Longitudinal Study
Lillard, A. S., Heise, M. J., Richey, E. M., Tong, X., Hart, A., & Bray, P. M. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01783 · Cited by 150 · see citing works
“Here executive function was measured with two tasks; a full battery of tests would have been desirable (Willoughby et al., 2011; Lipsey et al., 2017), but time constraints only allowed two.”
Methodvan der Ven, S. H. G., Kroesbergen, E. H., Boom, J., & Leseman, P. P. M. (2013) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.2012.02079.x · Cited by 149 · see citing works
Clark, C. A. C., Sheffield, T. D., Wiebe, S. A., & Espy, K. A. (2012) · Child Development · doi:10.1111/j.1467-8624.2012.01854.x · Cited by 152 · see citing works
Executive functioning predicts reading, mathematics, and theory of mind during the elementary years
Cantin, R. H., Gnaedinger, E. K., Gallaway, K. C., Hesson-McInnis, M. S., & Hund, A. M. (2016) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2016.01.014 · Cited by 152 · see citing works
Nature and measurement of attention control.
Burgoyne, A. P., Tsukahara, J. S., Mashburn, C. A., Pak, R., & Engle, R. W. (2023) · Journal of Experimental Psychology General · doi:10.1037/xge0001408 · Cited by 126 · see citing works
“As such, considerable time and effort have been invested in research on the nature of individual differences in attention control and their measurement (Lezak, 1982; McCabe et al., 2010; Willoughby et al., 2011; Zelazo et al., 2013).”
Test-retest reliability of a new executive function battery for use in early childhood
Willoughby, M. T., & Blair, C. (2011) · Child Neuropsychology · doi:10.1080/09297049.2011.554390 · Cited by 107 · see citing works
“We have reported the precision of measurement of our newly developed battery of EF tasks elsewhere, with the general conclusion being that the battery does a particularly good job of measuring low levels (less proficient) of EF ability ( Willoughby, Blair, Wirth, Greenberg, and the Family Life Project Investigators 20…”
MethodSelf-citationWilloughby, M. T., Magnus, B. E., Vernon‐Feagans, L., Blair, C., & Family Life Project Investigators (2016) · Journal of Learning Disabilities · doi:10.1177/0022219415619754 · Cited by 111 · see citing works
Self-citationFathers’ sensitive parenting and the development of early executive functioning.
Towe‐Goodman, N. R., The Family Life Project Key Investigators, Willoughby, M. T., Blair, C., Gustafsson, H., Mills-Koonce, R., & Cox, M. J. (2014) · Journal of Family Psychology · doi:10.1037/a0038128 · Cited by 108 · see citing works
Self-citationA Hierarchical Integrated Model of Self-Regulation
Blair, C., & Ku, S. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.725828 · Cited by 116 · see citing works
Self-citationAn Open Trial of a Metacognitive Executive Function Training for Young Children With ADHD
Tamm, L., Nakonezny, P. A., & Hughes, C. W. (2014) · Journal of Attention Disorders · doi:10.1177/1087054712445782 · Cited by 98 · see citing works
Television and Children's Executive Function
Lillard, A. S., Li, H., & Boguszewski, K. E. (2015) · Advances in child development and behavior · doi:10.1016/bs.acdb.2014.11.006 · Cited by 94 · see citing works
Lipsey, M. W., Nesbitt, K. T., Farran, D. C., Dong, N., Fuhs, M. W., & Wilson, S. J. (2017) · Journal of Educational Psychology · doi:10.1037/edu0000203 · Cited by 87 · see citing works
Executive Function Skills, Early Mathematics, and Vocabulary in Head Start Preschool Children
Harvey, H. A., & Miller, G. E. (2016) · Early Education and Development · doi:10.1080/10409289.2016.1218728 · Cited by 86 · see citing works
Brandes-Aitken, A. N., Braren, S. H., Swingler, M. M., Voegtline, K., & Blair, C. (2019) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2019.04.006 · Cited by 89 · see citing works
Self-citationWilloughby, M. T., Family Life Project Investigators, Pek, J., & Blair, C. B. (2013) · Psychological Assessment · doi:10.1037/a0031747 · Cited by 70 · see citing works
Self-citationMeasurement models for studying child executive functioning: Questioning the status quo.
Camerota, M., Willoughby, M. T., & Blair, C. (2020) · Developmental Psychology · doi:10.1037/dev0001127 · Cited by 69 · see citing works
“These EAP scores have been shown to have increased precision to differentiate children’s EF ability compared to raw (i.e., percent correct) scores (Willoughby et al., 2011).”
BackgroundSelf-citationDaneri, M. P., Blair, C. B., & Kuhn, L. J. (2018) · Child Development · doi:10.1111/cdev.13065 · Cited by 74 · see citing works
Self-citationTesting the association between physical activity and executive function skills in early childhood
Willoughby, M. T., Wylie, A. C., & Catellier, D. (2018) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2018.03.004 · Cited by 61 · see citing works
Self-citationPerry, R. E., Braren, S. H., Blair, C., & The Family Life Project Key Investigators (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01544 · Cited by 63 · see citing works
“, 2006), a BackwardWord Span task assessing working memory (Davis and Pratt, 1996), and a Dimensional Change Card Sort task (Willoughby et al., 2011).”
MethodSelf-citationCheng, N., Lu, S., Archer, M., & Wang, Z. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.02293 · Cited by 58 · see citing works
“Although we used only one task with proven validity (e.g., Hughes and Ensor, 2005; Willoughby et al., 2011) to assess working memory at 25 and 38 months, the similarity of results at both time points suggest that the effects of maternal parenting on working memory may not be as strong as previously…”
MethodBlair, C., & McKinnon, R. D. (2015) · Learning and Instruction · doi:10.1016/j.learninstruc.2015.10.001 · Cited by 62 · see citing works
“A full description of each task as well as psychometric information can be found in a number of publications (Willoughby & Blair, 2011; Willoughby et al., 2011).”
MethodSelf-citationVogel, S. C., Perry, R. E., Brandes-Aitken, A. N., Braren, S. H., & Blair, C. (2020) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2020.100907 · Cited by 58 · see citing works
“more precise way to estimate children’s executive function abilities than percentage correct scores (Willoughby et al., 2011).”
Self-citationCaughy, M. O., Mills, B. A., Owen, M. T., & DeLuna, J. H. (2013) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2013.07.017 · Cited by 55 · see citing works
Child Conduct Problems Across Home and School Contexts: a Person-Centered Approach
The Family Life Project Investigators, Sulik, M. J., Blair, C., & Greenberg, M. T. (2016) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-016-9564-8 · Cited by 50 · see citing works
“Full details regarding the administration rules, psychometric properties, and scoring for each of these tasks have been presented elsewhere (Willoughby et al. 2011).”
Self-citationXiao-hui, Y., Chen, Z., Wang, Z., & Zhu, L. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01833 · Cited by 49 · see citing works
“, 2008) and aggregated measures of EF are stable and robust (Willoughby et al., 2011).”
BackgroundAndrews, K., Dunn, J. R., Prime, H., Duku, E., Atkinson, L. R., Tiwari, A., & González, A. (2021) · BMC Psychology · doi:10.1186/s40359-021-00651-1 · Cited by 48 · see citing works
Does early executive function predict teacher–child relationships from kindergarten to second grade?
McKinnon, R. D., The Family Life Project Investigators, & Blair, C. (2018) · Developmental Psychology · doi:10.1037/dev0000584 · Cited by 41 · see citing works
“A full description of each task can be found in a number of publications (Willoughby & Blair, 2011; Willoughby et al., 2011).”
Self-citationMeasuring executive function skills in young children in Kenya
Willoughby, M. T., Piper, B., Kwayumba, D., & McCune, M. (2018) · Child Neuropsychology · doi:10.1080/09297049.2018.1486395 · Cited by 44 · see citing works
“Third, elsewhere we have emphasized the value of using modern psychometric methods to investigate EF task performance and to test for equivalence in item parameters across subgroups (Willoughby et al., 2011).”
BackgroundSelf-citationMoreno, A. J., Shwayder, I., & Friedman, I. D. (2016) · Early Childhood Education Journal · doi:10.1007/s10643-016-0777-y · Cited by 40 · see citing works
Concordance of teacher-rated and performance-based measures of executive functioning in preschoolers
Tamm, L., & Peugh, J. (2019) · Child Neuropsychology · doi:10.1080/09297049.2018.1484085 · Cited by 35 · see citing works
White, R. E., & Carlson, S. M. (2021) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2021.105090 · Cited by 41 · see citing works
EEG power and coherence during preschoolers' performance of an executive function battery
Swingler, M. M., Willoughby, M. T., & Calkins, S. D. (2011) · Developmental Psychobiology · doi:10.1002/dev.20588 · Cited by 35 · see citing works
Self-citationIncome, family context and self-regulation in 5-year-old children
Li, M., Riis, J. L., Ghazarian, S., & Johnson, S. B. (2017) · Journal of Developmental and Behavioral Pediatrics · doi:10.1097/dbp.0000000000000380 · Cited by 28 · see citing works
“These tasks are valid and reliable measures of executive function in this age group.(22) The first task, The Silly Sounds Game, is an auditory Stroop task, which assesses a child’s ability to inhibit an automatic response.”
Assessment of preschool‐age executive functions: A systematic review
Silva, C., Sousa‐Gomes, V., Fávero, M., Oliveira, S., Merendeiro, C. S., Oliveira, J., & Moreira, D. S. (2022) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2718 · Cited by 35 · see citing works
Simpson, A., Ruwaili, R. A., Jolley, R. P., Leonard, H. C., Geeraert, N., & Riggs, K. J. (2017) · Child Development · doi:10.1111/cdev.12949 · Cited by 31 · see citing works
“…response inhibition and fine motor control in 3- and 4-year-olds: the time when inhibition is developing most rapidly (e.g., Weibe et al., 2012; Willoughby et al., 2011). association: first that effective inhibition leads to effective fine motor control, and second, that embodied cognition…”
BackgroundQu, L., Finestone, D. L., Qin, L. J., & Leong, R. Z. X. (2012) · Emotion · doi:10.1037/a0027263 · Cited by 30 · see citing works
Burchinal, M., Carr, R. C., Vernon‐Feagans, L., Blair, C., & Cox, M. J. (2018) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2018.07.002 · Cited by 23 · see citing works
Self-citationBishara, S., & Kaplan, S. (2021) · International Journal of Disability Development and Education · doi:10.1080/1034912x.2021.1925878 · Cited by 21 · see citing works
Investigating the other‐race effect in working memory
Stelter, M., & Degner, J. (2018) · British Journal of Psychology · doi:10.1111/bjop.12304 · Cited by 20 · see citing works
Dekkers, L. M. S., Bexkens, A., Hofman, A. D., de Boeck, P. A. L., d’Escury-Koenigs, A. L. C., & Huizenga, H. M. (2017) · Assessment · doi:10.1177/1073191117698754 · Cited by 19 · see citing works
“…over participants, is a specific case of the more general family of generalized linear mixed models (Cho et al., 2013; De Boeck et al., 2011; De Boeck & Partchev, 2012; De Boeck & Wilson, 2004, 2015; Molenaar, Tuerlincks, & van der Maas, 2015; Partchev & De Boeck, 2012; Willoughby et al., 2011).”
BackgroundBerger, P., Friederici, A. D., & Wiesmann, C. G. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.2235-21.2022 · Cited by 18 · see citing works
COMMENTARY ON THE CHANGING NATURE OF EXECUTIVE CONTROL IN PRESCHOOL
Willoughby, M. T. (2016) · Monographs of the Society for Research in Child Development · doi:10.1111/mono.12276 · Cited by 18 · see citing works
Self-citationBrandes-Aitken, A. N., Braren, S. H., Gandhi, J., Perry, R. E., Rowe‐Harriott, S., & Blair, C. (2020) · Developmental Psychology · doi:10.1037/dev0001089 · Cited by 17 · see citing works
Self-citationUsai, M. C., Viterbori, P., Gandolfi, E., & Zanobini, M. (2020) · Infant Behavior and Development · doi:10.1016/j.infbeh.2020.101495 · Cited by 15 · see citing works
Werchan, D. M., Ku, S., Berry, D. J., & Blair, C. (2022) · Developmental Science · doi:10.1111/desc.13293 · Cited by 15 · see citing works
Self-citationMaxwell, R. W., & Katyal, S. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.863091 · Cited by 15 · see citing works
Liu, J., Lin, H., Hu, B., Zhou, Z., Agyeiwaah, E., & Xu, Y. L. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2021.12.013 · Cited by 15 · see citing works
Blanco-Villaseñor, Á., & Escolano-Pérez, E. (2017) · Anales de Psicología · doi:10.6018/analesps.33.3.271021 · Cited by 13 · see citing works
“These aspects cannot be studied by direct observation and researchers must therefore analyze the outputs of these processes and draw inferences (Willoughby et al., 2011).”
BackgroundInhibition: Mental Control Process or Mental Resource?
Im‐Bolter, N., Johnson, J., Ling, D. S., & Pascual‐Leone, J. (2014) · Journal of Cognition and Development · doi:10.1080/15248372.2014.930743 · Cited by 11 · see citing works
Executive Function and BMI Trajectories Among Rural, Poor Youth at High Risk for Obesity
Rollins, B. Y., Riggs, N. R., Francis, L. A., & Blair, C. (2020) · Obesity · doi:10.1002/oby.23064 · Cited by 10 · see citing works
“The battery included three inhibitory tasks (a Simon-like spatial conflict task, a Stroop-like silly sounds task, and the farm animal go-no-go task), two working memory tasks (a span-like task and a self-ordered pointing task), and one cognitive flexibility task modeled on the Dimensional Change Card Sort task (20).”
MethodSelf-citationDevelopment of Cognitive Flexibility
Blaye, A. (2022) · Wiley eBooks · doi:10.1002/9781119902737.ch2 · Cited by 11 · see citing works
Cumming, M. M., Poling, D. V., Qiu, Y., Pham, A. V., Daunic, A. P., Corbett, N., & Smith, S. W. (2021) · Assessment · doi:10.1177/10731911211032289 · Cited by 10 · see citing works
“Thus, when identifying students at risk for behavior and mental health problems in schools and developing targeted services (Wass et al., 2012; Willoughby et al., 2011; Willoughby et al., 2016), school-based professionals (e.”
BackgroundA unified framework of parental socialization of executive function development
Hanlin, P., & Zhou, Q. (2024) · Developmental Review · doi:10.1016/j.dr.2024.101149 · Cited by 9 · see citing works
Schirmbeck, K., Runge, R. A., Rao, N., Wang, R., Richards, B., Chan, S. W. Y., & Mähler, C. (2022) · Journal of Cultural Cognitive Science · doi:10.1007/s41809-022-00112-0 · Cited by 7 · see citing works
“However, very few tasks used to assess EF in cross-cultural research have been tested for measurement equivalence across the different cultural contexts; in fact, many measures aiming to assess EF at preschool age have not undergone formal psychometric evaluation (Willoughby et al., 2011).”
BackgroundMeldrum, S. J., Heaton, A. E., Foster, J. K., Prescott, S. L., & Simmer, K. N. (2020) · British Journal Of Nutrition · doi:10.1017/s000711452000135x · Cited by 8 · see citing works
“The utility and merit of these tests have been evaluated, and experts agree that there is a pressing need for psychometrically sound instruments capable of detecting subtle differences in executive functioning in children(43).”
BackgroundStress and the Development of Executive Functions: Experiential Canalization of Brain and Behavior
Blair, C. (2013) · Wiley eBooks · doi:10.1002/9781118732373.ch5 · Cited by 9 · see citing works
Self-citationHouts, C. R., Savord, A., & Wirth, R. J. (2021) · Journal of Pediatric Neuropsychology · doi:10.1007/s40817-021-00117-7 · Cited by 8 · see citing works
“An additional publication is available from this research group that provides an overview of factor analysis and IRT using a specific EF scale development project as a basis for discussion (Willoughby et al., 2011).”
MethodSelf-citationSchulz, D., Richter, T., Schindler, J., Lenhard, W., & Mangold, M. (2022) · Journal of Cognition and Development · doi:10.1080/15248372.2022.2119977 · Cited by 7 · see citing works
“Similar to the manual Silly Sound Stroop task introduced by Willoughby et al. (2011), children are presented with an image of a dog and an image of a cat on a touchscreen in every trial.”
MethodSeeking Racial and Ethnic Parity in Preschool Outcomes
Lillard, A. S., Tong, X., & Bray, P. M. (2023) · Journal of Montessori Research · doi:10.17161/jomr.v9i1.19540 · Cited by 7 · see citing works
“A second rationale for combining the two scores is that single measures of executive function are less reliable than composite measures created from more than one test (Willoughby et al., 2011).”
BackgroundCarr, R. C., Peisner-Feinberg, E. S., Kaplan, R. L., et al. (2021) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2021.101317 · Cited by 7 · see citing works
Development and validation of the NIH Baby Toolbox® Executive Function and Memory measures
Flynn, R. M., Han, Y.-H., Kaat, A. J., et al. (2025) · Infant Behavior and Development · doi:10.1016/j.infbeh.2025.102118 · Cited by 6 · see citing works
Willoughby, M. T. (2014) · Measurement Interdisciplinary Research and Perspectives · doi:10.1080/15366367.2014.981074 · Cited by 6 · see citing works
Self-citationWu, Q., Jalapa, K., Han, S. J., Tawfiq, D., & Cui, M. (2022) · Development and Psychopathology · doi:10.1017/s0954579422000037 · Cited by 6 · see citing works
“The EAP scores are suitable to process dichotomous or polytomous to indicate a latent trait score for an individual by accounting for the expected value of the posterior probability distribution (Willoughby et al., 2011).”
Executive Functions in the Classroom
Blair, C., & Steinhard, N. (2013) · Cited by 6 · see citing works
Self-citationPatwardhan, I., Gordon, C. T., & Mason, W. A. (2023) · Developmental Psychology · doi:10.1037/dev0001597 · Cited by 5 · see citing works
Braren, S. H., Brandes-Aitken, A. N., Perry, R. E., Williams, K. T., Lyons, K., Rowe‐Harriott, S., & Blair, C. (2020) · Mind Brain and Education · doi:10.1111/mbe.12267 · Cited by 5 · see citing works
Self-citationToor, H. K., & Hanif, R. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274431 · Cited by 5 · see citing works
“Along the dimension of theta, the peak of the test information curve represents by the difficulty parameters of the items that make up the test [23].”
BackgroundMethodResultExecutive Functions in Autism Spectrum Disorder
McCrimmon, A. W., Matchullis, R. L., Altomare, A. A., & Smith-Demers, A. D. (2016) · Autism and child psychopathology series · doi:10.1007/978-3-319-27171-2_21 · Cited by 4 · see citing works
Nasledov, A., Александрович, М. С., Tkacheva, L. O., Zashchirinskaia, O. V., & Goncharov, V. A. (2020) · Psychology in Russia State of Art · doi:10.11621/pir.2020.0302 · Cited by 5 · see citing works
“…e key elements of the executive function are: the ability to extrapolate; arbitrary attention; behavioral control and self-regulation; working memory; the ability to plan and organize cognitive activities; and the use of e ff ective strategies to solve problems (Willoughby, Wirth, & Blair, 2011).”
BackgroundSimpson, A., & Carroll, D. J. (2017) · Cognition · doi:10.1016/j.cognition.2017.10.008 · Cited by 3 · see citing works
Age-Adapted Stress Task in Preschoolers Does not Lead to Uniform Stress Responses
Stülb, K., Messerli‐Bürgy, N., Kakebeeke, T. H., Arhab, A., Zysset, A. E., Leeger‐Aschmann, C. S., Schmutz, E. A., Meyer, A. H., García‐Burgos, D., Ehlert, U., et al. (2018) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-018-0475-x · Cited by 3 · see citing works
Measuring executive functions in context: Self-regulating behavior in the kindergarten classroom
Maloney, J., Oberle, E., Weber, B., & Schonert-Reichl, K. A. (2025) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2025.07.006 · Cited by 2 · see citing works
Enelamah, N. V., Rao, S., Lombe, M., Yu, M., Newransky, C., Villodas, M. L., Foell, A., & Nebbitt, V. E. (2024) · Children · doi:10.3390/children11030361 · Cited by 2 · see citing works
“Investigating the ECDI solely based on the face value of the CTT assumes that caregivers responding on children’s behalf attribute their development levels to reflect their actual ability in that domain [31].”
BackgroundAndrews, K., Dunn, J. R., Prime, H., Duku, E., Atkinson, L. R., Tiwari, A., & González, A. (2023) · doi:10.32920/24135090 · Cited by 2 · see citing works
de Zarate, A. E. R., Körner, J. K. A., Haas, P., Gunzenhauser, C., Rauch, W. A., & Gawrilow, C. (2025) · Scientific Reports · doi:10.1038/s41598-025-96791-0 · Cited by 1 · see citing works
“In contrast to the accuracy score computed by the EF Touch 45 , which is based on the number of items a child sees, in line with Ulitzka et al. 44 we computed the accuracy based on the total number of items of each task.”
Karonen, A., Juntunen, P., Yada, A., Takio, F., Nordenswan, E., Eskola, E., Deater-Deckard, K. D., Bridgett, D. J., Fernandes, M., Mainela‐Arnold, E., et al. (2026) · Child Development · doi:10.1093/chidev/aacaf053 · Cited by 1 · see citing works
ChiSungae, & 백지혜 (2016) · Korean Journal of Early Childhood Education · doi:10.18023/kjece.2016.36.6.003 · Cited by 1 · see citing works
Tng, G. Y. Q., & Yang, H. (2024) · Cognitive development · doi:10.1016/j.cogdev.2024.101516 · Cited by 1 · see citing works
Rodrigues, C. Y. D. S. (2016) · International Journal of Developmental and Educational Psychology Revista INFAD de psicología · doi:10.17060/ijodaep.2014.n1.v5.653
Hubert, B., Guimard, P., Florin, A., & Bouderbela, S. (2020) · HAL
Neurocognitive Aspects of Mathematical Achievement in Children
Raj, P. M., & Padakannaya, P. (2021) · Springer eBooks · doi:10.1007/978-3-030-72400-9_10
Идентификация прогнозных маркеров задержки психического развития 6-летних детей
Наследов, А., Александрович, М. С., & Tkacheva, L. O. (2019) · Психологические исследования · doi:10.54359/ps.v12i64.229
Andrews, K., Dunn, J. R., Prime, H., Duku, E., Atkinson, L. R., Tiwari, A., & González, A. (2023) · doi:10.32920/24135090.v1
- Authors
- Willoughby, M. T., Wirth, R. J., & Blair, C. B.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- New York University (US)
- Corresponding author
- Willoughby, Michael T.
- Venue
- Journal of Experimental Child Psychology
- Publisher
- Elsevier BV
- Published
- 2011-03
- Volume
- 108
- Issue
- 3
- Pages
- 414-435
- ISSN
- 0022-0965 (ISSN-L)
- PMC
- PMC2953563
- OpenAlex
- W2083472871
- Semantic Scholar
- 262b8e7d22ba4a2661ea0c9d5abb3565bd78a409
- Cited by
- 102
- References
- 50
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 3 · 2023: 5 · 2022: 12 · 2021: 8 · 2020: 9 · 2019: 3 · 2018: 12 · 2017: 8 · 2016: 11 · 2015: 6 · 2014: 3 · 2013: 6 · 2012: 6
- FWCI
- 7.39
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Experimental Child Psychology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- UNC Libraries
- Funders
- National Institutes of Health
- National Institute of Child Health and Human Development (P01 HD 39667, HD39667)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (P01 HD39667, P01HD039667, R01HD051502)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Keywords
- executive function, early childhood, item response theory, measurement invariance, item characteristic curves, measurement precision, psychometric properties, Family Life Project, low-income households, African American children, individual differences
- MeSH terms
- Attention, Black or African American, Child, Preschool, Female, Humans, Longitudinal Studies, Male, Memory, Short-Term, Neuropsychological Tests, North Carolina, Pennsylvania, Poverty, Prospective Studies, Psychometrics, Task Performance and Analysis, Executive Function, Stroop Test
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
