The DOI matches its Crossref record; 47 of the 50 references it cites were found in a registry or an index.
Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration
Open accessAbstract
This study demonstrates the merits of evaluating a newly developed battery of executive function tasks, designed for use in early childhood, from the perspective of item response theory (IRT). The battery was included in the 48-month assessment of the Family Life Project, a prospective longitudinal study of 1292 children oversampled from low-income and African American families. IRT models were applied to a select set of tasks to demonstrate empirically (a) a principled method for item evaluation, including the utility of item characteristic curves; (b) how to explicitly test whether the measurement properties of executive function tasks are invariant across mutually exclusive subgroups of youths; (c) how the precision of measurement of a given task can vary according to underlying child ability; and (d) the utility of using IRT-based versus percentage correct scores. Results are discussed with respect to the importance of developing psychometrically sound and scalable instruments that facilitate the measurement of interindividual differences in intraindividual change of executive function across the early childhood period.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another working memory paradigm used with preschoolers is self-ordered searching tasks, which requires information, usually visual-spatial, to be maintained and updated in short-term memory.”
From Assessing Executive Function in Preschoolers (2012) · cites this paper as M. T. Willoughby, Wirth, & Blair, 2011
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · cited 1×
“A series of exact and approximate fit indices were used to evaluate the fit of each model as per the guidelines outlined by Hu and Bentler (1999).”
MethodCoefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · cited 1×
“Second, reliability measures (e.g., coefficient alpha) for CTT-based scores assume that all scores are equally reliable (see, for example, Cronbach, 1951).”
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0734 · Cited by 15,798 · cited 1×
“N IH -PA Author M anuscript N IH -PA Author M anuscript N IH -PA Author M anuscript early childhood (Miyake et al, 2000; Wiebe et al, 2008).”
BackgroundAn Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · cited 1×
“…NC 27599 2 Department of Applied Psychology, 239 Greene St, East Bldg 500, New York University, New York NY 10003 Executive functions refer to cognitive abilities involved in the control and coordination of information in the service of goal-directed actions (Fuster, 1997; Miller & Cohen, 2001).”
BackgroundRozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536 · cited 2×
“Testing for such differences is broadly known in the psychometric literature as testing for measurement invariance (Meredith, 1993).”
Background(2000) · Cited by 4,106
Executive function in preschoolers: A review using an integrative framework.
Garon, N., Bryson, S. E., & Smith, I. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.31 · Cited by 2,695
ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES1
Samejima, F. (1968) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1968.tb00153.x · Cited by 2,670 · cited 1×
“Ordinal scored (Likert type) items cannot be accommodated by the 2-parameter logistic model; hence the Pick the Picture task was fit using an alternative model— Samejima’s (1969) graded response model.”
MethodThe Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe
Fuster, J. M. (1980) · Cited by 2,367
Davidson, M. C., Amso, D., Anderson, L. C., & Diamond, A. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.02.006 · Cited by 2,226
Hambleton, R. K., & Swaminathan, H. (1985) · doi:10.1007/978-94-017-1988-9 · Cited by 1,777
Principles of Frontal Lobe Function
Stuss, D. T. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195134971.001.0001 · Cited by 1,731
Developmentally Sensitive Measures of Executive Function in Preschool Children
Carlson, S. M. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_3 · Cited by 1,638 · cited 2×
“It is probably not too far of an exaggeration to suggest that there exists a “cottage industry” around the development of Executive Function tasks for use with young children (see e.g., Carlson, 2005).”
BackgroundGerstadt, C. L., Hong, Y. J., & Diamond, A. (1994) · Cognition · doi:10.1016/0010-0277(94)90068-x · Cited by 1,598 · cited 1×
“This task was derived from the Day-Night task developed by Gerstadt, Hong, and Diamond (1994).”
MethodLinks between behavioral regulation and preschoolers’ literacy, vocabulary, and math skills.
McClelland, M. M., Cameron, C. E., Connor, C. M., Farris, C. L., Jewkes, A. M., & Morrison, F. J. (2007) · Developmental Psychology · doi:10.1037/0012-1649.43.4.947 · Cited by 1,561 · cited 1×
“For example, on measures such as peg tapping (Diamond & Taylor, 1996) or the Head-ToesKnees-Shoulders (McClelland et al., 2007), many children are able to successfully complete the first several items, and it is later rather than earlier items that discriminate ability on these measures.”
BackgroundDeficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man
Petrides, M., & Milner, B. A. (1982) · Neuropsychologia · doi:10.1016/0028-3932(82)90100-2 · Cited by 1,317 · cited 1×
“This is a Self-Ordered Pointing task (Petrides & Milner, 1982; Cragg & Nation, 2007).”
MethodThe prefrontal cortex: Anatomy, physiology, and neuropsychology of the frontal lobe
Holmes, G. L. (1989) · Journal of Epilepsy · doi:10.1016/0896-6974(89)90035-2 · Cited by 1,211
Socioeconomic gradients predict individual differences in neurocognitive abilities.
Noble, K. G., McCandliss, B. D., & Farah, M. J. (2007) · Developmental Science · doi:10.1111/j.1467-7687.2007.00600.x · Cited by 1,041 · cited 1×
“Second, an emerging body of work has demonstrated that poverty is predictive of executive functions in early childhood (Blair et al, submitted; Noble et al, 2007).”
BackgroundItem factor analysis: Current approaches and future directions.
Wirth, R. J., & Edwards, M. C. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.58 · Cited by 958 · cited 3×
“The discrimination parameter ranges from 0 to infinity, but values greater than approximately 4 for a uni-dimensional model are generally considered problematic (Wirth & Edwards, 2007).”
BackgroundInfluentialSelf-citationWiebe, S. A., Espy, K. A., & Charak, D. (2008) · Developmental Psychology · doi:10.1037/0012-1649.44.2.575 · Cited by 829 · cited 1×
“N IH -PA Author M anuscript N IH -PA Author M anuscript N IH -PA Author M anuscript early childhood (Miyake et al, 2000; Wiebe et al, 2008).”
BackgroundConflict adaptation effects in the absence of executive control
Mayr, U., Awh, E., & Laurey, P. (2003) · Nature Neuroscience · doi:10.1038/nn1051 · Cited by 793 · cited 2×
“As noted above, however, it is very likely given the nature of EF and the inherent difficulty of switching between congruent and incongruent items on a given task, that items involving switches rather than repetitions are more informative than items that maintain a pattern of responding, whether congruent or incongrue…”
BackgroundExecutive Function in Typical and Atypical Development
Zelazo, P. D., & Müller, U. (2010) · The Wiley‐Blackwell Handbook of Childhood Cognitive Development · doi:10.1002/9781444325485.ch22 · Cited by 784 · cited 1×
“…of prepotent or automatized responding when engaged in task completion; and mental flexibility, defined as the ability to shift attentional or cognitive set among distinct but related dimensions or aspects of a given task (Davidson et al., 2006; Garon, Bryson, & Smith, 2008; Zelazo & Müller, 2002).”
BackgroundOn the Relationship between iTem Response Theory and Factor Analysis of Discretized Variables
Takane, Y., & de Leeuw, J. (1987) · Psychometrika · doi:10.1007/bf02294363 · Cited by 699
The domain of supervisory processes and temporal organization of behaviour
Shallice, T., & Burgess, P. W. (1996) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1996.0124 · Cited by 633 · cited 1×
“As such, executive function can be defined as a supervisory system that is important for planning, reasoning ability, and the integration of thought and action (Shallice & Burgess, 1996).”
BackgroundFull-Information Item Bi-Factor Analysis
Gibbons, R. D., & Hedeker, D. (1992) · Psychometrika · doi:10.1007/bf02295430 · Cited by 599
Blair, C., Granger, D. A., Willoughby, M., Mills-Koonce, R., Cox, M., Greenberg, M. T., Kivlighan, K. T., Fortunato, C. K., & the FLP Investigators (2011) · Child Development · doi:10.1111/j.1467-8624.2011.01643.x · Cited by 579
Self-citationThe domain of supervisory processes and the temporal organization of behaviour
Shallice, T., & Burgess, P. W. (1998) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198524410.003.0003 · Cited by 551
The Measurement of Executive Function in Early Childhood
Blair, C., Zelazo, P. D., & Greenberg, M. T. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_1 · Cited by 354
Self-citationThe Flexible Item Selection Task (FIST): A Measure of Executive Function in Preschoolers
Jacques, S., & Zelazo, P. D. (2001) · Developmental Neuropsychology · doi:10.1207/s15326942dn2003_2 · Cited by 353
The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe, 3rd ed.
Spinks, R. A. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.6.1038 · Cited by 285
Full-Information Item Bifactor Analysis of Graded Response Data
Gibbons, R. D., Bock, R. D., Hedeker, D., Weiss, D. J., Segawa, E., Bhaumik, D. K., Kupfer, D. J., Frank, E., Grochocinski, V. J., & Stover, A. M. (2006) · Applied Psychological Measurement · doi:10.1177/0146621606289485 · Cited by 271
Recent Developments in the Factor Analysis of Categorical Variables
Mislevy, R. J. (1986) · Journal of Educational Statistics · doi:10.3102/10769986011001003 · Cited by 257 · cited 1×
“…confirmatory factor analysis (CFA) methods were used to examine the dimensionality of tasks while IRT methods were used to better understand item and score characteristics (also see Mislevy, 1986, and Takane & de Leeuw, 1987, for information regarding the relationship between item-factor models).”
MethodLloyd, K. E. (1994) · The Behavior Analyst · doi:10.1007/bf03392658 · Cited by 221
The Blackwell companion to sociology
(2001) · Choice Reviews Online · doi:10.5860/choice.39-1248 · Cited by 188
Woods, C. M., & Thissen, D. (2006) · Psychometrika · doi:10.1007/s11336-004-1175-8 · Cited by 121 · cited 1×
“The bottom panel presents ivThe assumption of an underlying normally distributed population is not required (Woods & Thissen, 2006).”
BackgroundRecent Developments in the Factor Analysis of Categorical Variables
Mislevy, R. J. (1986) · Journal of Educational Statistics · doi:10.2307/1164846 · Cited by 110
Hill, C., Edwards, M. C., Thissen, D., Langer, M., Wirth, R., Burwinkle, T. M., & Varni, J. W. (2007) · Medical Care · doi:10.1097/01.mlr.0000259879.05499.eb · Cited by 95 · cited 1×
“If model fit was poor (i.e., CFA < .90, TLI < .90, & RMSEA > .08), modification indices, standardized residuals (see, for example, Hill et al., 2007), and discussions among the authors about possible item dependencies were used to inform the fitting of bi-factor models, which take into account…”
MethodSelf-citationModeling Ecological and Contextual Effects in Longitudinal Studies of Human Development
Card, N. A., Little, T. D., & Bovaird, J. A. (2007) · doi:10.4324/9780203936825-5 · Cited by 93
Self-ordered pointing as a test of working memory in typically developing children
Cragg, L., & Nation, K. (2007) · Memory · doi:10.1080/09658210701390750 · Cited by 85 · cited 1×
“This is a Self-Ordered Pointing task (Petrides & Milner, 1982; Cragg & Nation, 2007).”
MethodMeasurement and the Study of Change
Edwards, M. C., & Wirth, R. J. (2009) · Research in Human Development · doi:10.1080/15427600902911163 · Cited by 78 · cited 3×
“Incorrectly weighting items can lead to scale scores that are biased and thereby less accurate when comparing individuals (or groups) within or over time (Edwards & Wirth, 2009; Wirth 2008).”
BackgroundSelf-citationThe Blackwell Companion to Sociology
(2004) · Blackwell Publishing Ltd eBooks · doi:10.1111/b.9781405122672.2004.x · Cited by 67
Special Topics in Item Response Theory
Edwards, M. C., & Edelen, M. O. (2009) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020994.n8 · Cited by 26 · cited 1×
“This study used the non-anchor (also commonly referred to constrain-all/test-all) approach for testing for (see Edwards & Edelen, 2009, for a recent review of DIF methods).”
MethodDill, B. T., & Myers, S. L. (2004) · Wiley eBooks · doi:10.1002/9780470693452.ch14 · Cited by 22
The effects of measurement non-invariance on parameter estimation in latent growth models
Wirth, R. J. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/tgzh-a739 · Cited by 13
Self-citationDiamond, A., & Taylor, C. (1996) · Developmental Psychobiology · doi:10.1002/(sici)1098-2302(199605)29:4
Research works printed but not linked: 1
- Cai, L., du Toit, H. S. C., & Thissen, D. (in press). IRTPro: Flexible, multidimensional, multiple categorical IRT modeling [computer software]. Chicago: SSI International.
Cited by
Sorted by citations
McClelland, M. M., & Cameron, C. E. (2011) · Child Development Perspectives · doi:10.1111/j.1750-8606.2011.00191.x · Cited by 740
Executive function in early childhood: Longitudinal measurement invariance and developmental change.
Willoughby, M. T., Family Life Project Investigators, Wirth, R. J., & Blair, C. (2011) · Psychological Assessment · doi:10.1037/a0025779 · Cited by 413
Self-citationChanges and challenges in 20 years of research into the development of executive functions
Hughes, C. (2011) · Infant and Child Development · doi:10.1002/icd.736 · Cited by 347
“Applying this approach, Willoughby et al. (2010) were able to show that three distinct EF tasks (a self-ordered search test of working memory, a ‘Silly Sounds’ non-verbal Stroop test of inhibitory control, and an attention set shifting task) showed similar ICCs over repeated intervals and for…”
BackgroundMethodReview of measures of social and emotional development
Halle, T., & Darling, K. E. (2016) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2016.02.003 · Cited by 401
Assessing Executive Function in Preschoolers
Anderson, P. J., & Reidy, N. (2012) · Neuropsychology Review · doi:10.1007/s11065-012-9220-3 · Cited by 293
“Examples of this approach are the Self-ordered Pointing and Pick the Picture tests (Hongwanishkul et al. 2005; Willoughby et al. 2011).”
BackgroundMethod
Metadata
- Authors
- Willoughby, M. T., Wirth, R. J., & Blair, C. B.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- New York University (US)
- Corresponding author
- Willoughby, Michael T.
- Venue
- Journal of Experimental Child Psychology
- Publisher
- Elsevier BV
- Published
- 2011-03
- Volume
- 108
- Issue
- 3
- Pages
- 414-435
- ISSN
- 0022-0965 (ISSN-L)
- PMC
- PMC2953563
- OpenAlex
- W2083472871
- Semantic Scholar
- 262b8e7d22ba4a2661ea0c9d5abb3565bd78a409
- Cited by
- 102
- References
- 50
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 3 · 2023: 5 · 2022: 12 · 2021: 8 · 2020: 9 · 2019: 3 · 2018: 12 · 2017: 8 · 2016: 11 · 2015: 6 · 2014: 3 · 2013: 6 · 2012: 6
- FWCI
- 7.39
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Experimental Child Psychology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- UNC Libraries
- Funders
- National Institutes of Health
- National Institute of Child Health and Human Development (P01 HD 39667, HD39667)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (P01 HD39667, P01HD039667, R01HD051502)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Keywords
- executive function, early childhood, item response theory, measurement invariance, item characteristic curves, measurement precision, psychometric properties, Family Life Project, low-income households, African American children, individual differences
- MeSH terms
- Attention, Black or African American, Child, Preschool, Female, Humans, Longitudinal Studies, Male, Memory, Short-Term, Neuropsychological Tests, North Carolina, Pennsylvania, Poverty, Prospective Studies, Psychometrics, Task Performance and Analysis, Executive Function, Stroop Test
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
