Executive Function in Preschool Children: Test–Retest Reliability
Open accessAbstract
Research suggests that executive function (EF) may distinguish between children who are well- or ill-prepared for kindergarten, however, little is known about the test-retest reliability of measures of EF for children. We aimed to establish a battery of EF measures that are sensitive to both development and individual differences across the preschool period using Conflict and Delay subtests that had a cool (abstract) or hot (extrinsic reward) focus. Results from 151 children in three age groups (2.5, 3.5, and 4.5) suggested acceptable same-day test-retest reliability on all but Delay-Cool subtasks. These findings will inform appropriate measurement selection and development for future studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Both the standard version of the Dimensional Change Card Sort and a more challenging version have shown excellent test-retest reliability in childhood (ICCs = .90-.94).
“Both the standard version of this task and a more challenging version have shown excellent test–retest reliability in childhood (ICCs = .90–.94; Beck, Schaefer, Pang, & Carlson, 2011).”
From NIH Toolbox Cognition Battery (CB): Validation of Executive Function Measures in Adults (2014) · cites this paper as Beck, Schaefer, Pang, & Carlson, 2011
The Discussion reports an ICC of .94 for the standard (cool) Integrated DCCS in preschool children. The Results discuss reliability for the more challenging Advanced DCCS, but the passages available from our copy do not establish an ICC of .90 for that version; they describe cool subtasks collectively as having moderate to excellent reliability. (LLM, conf=0.84)
Experimental — accuracy not yet measuredSame-day test-retest reliability for certain preschool executive function measures was good (ICCs ≥ 0.75).
“a recent study reported that same-day test-retest reliability for certain preschool executive function measures was good (ICCs ≥ 0.75)”
From Assessing Executive Function in Preschoolers (2012) · cites this paper as Beck, Schaefer, Pang, & Carlson, 2011
Child-friendly tasks will reduce the high refusal rate experienced when assessing preschoolers.
“Child friendly tasks will reduce the high refusal rate experienced when assessing preschoolers”
From Assessing Executive Function in Preschoolers (2012) · cites this paper as Beck, Schaefer, Pang, & Carlson, 2011
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Foundations of Clinical Research: Applications to Practice,
(2002) · Survey of Ophthalmology · doi:10.1016/s0039-6257(02)00362-4 · Cited by 9,485 · see citing works
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · see citing works
Ego depletion: Is the active self a limited resource?
Baumeister, R. F., Bratslavsky, E., Muraven, M., & Tice, D. M. (1998) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.74.5.1252 · Cited by 5,312 · see citing works
Handbook of Emotion Regulation
Lambert, K. G. (2007) · JAMA · doi:10.1001/jama.298.15.1808 · Cited by 4,900 · see citing works
Testing the efficiency and independence of attentional networks
Fan, J., McCandliss, B. D., Sommer, T., Raz, A., & Posner, M. I. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317361886 · Cited by 3,717 · see citing works
Blair, C., & Razza, R. P. (2007) · Child Development · doi:10.1111/j.1467-8624.2007.01019.x · Cited by 3,140 · see citing works
A hot/cool-system analysis of delay of gratification: dynamics of willpower
Metcalfe, J., & Mischel, W. (1999) · Psychological Review · doi:10.1037/0033-295x.106.1.3 · Cited by 2,931 · see citing works
Executive function in preschoolers: A review using an integrative framework.
Garon, N., Bryson, S. E., & Smith, I. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.31 · Cited by 2,695 · see citing works
Grant, D. A., & Berg, E. (1948) · Journal of Experimental Psychology · doi:10.1037/h0059831 · Cited by 2,244 · see citing works
Individual Differences in Inhibitory Control and Children's Theory of Mind
Carlson, S. M., & Moses, L. J. (2001) · Child Development · doi:10.1111/1467-8624.00333 · Cited by 2,072 · see citing works
Self-citationKochanska, G., Murray, K. T., & Harlan, E. T. (2000) · Developmental Psychology · doi:10.1037/0012-1649.36.2.220 · Cited by 1,993 · see citing works
Monographs of the Society for Research in Child Development
Ash, M. J., & Torres, J. L. (2008) · Encyclopedia of Special Education · doi:10.1002/9780470373699.speced1386 · Cited by 1,842 · see citing works
Developmentally Sensitive Measures of Executive Function in Preschool Children
Carlson, S. M. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_3 · Cited by 1,638 · see citing works
Self-citationThe Dimensional Change Card Sort (DCCS): A method of assessing executive function in children
Zelazo, P. D. (2006) · Nature Protocols · doi:10.1038/nprot.2006.46 · Cited by 1,579 · see citing works
Bernier, A., Carlson, S. M., & Whipple, N. (2010) · Child Development · doi:10.1111/j.1467-8624.2009.01397.x · Cited by 1,388 · see citing works
Self-citationThe nature of adolescent competencies predicted by preschool delay of gratification.
Mischel, W., Shoda, Y., & Peake, P. K. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.4.687 · Cited by 1,285 · see citing works
Shoda, Y., Mischel, W., & Peake, P. K. (1990) · Developmental Psychology · doi:10.1037/0012-1649.26.6.978 · Cited by 1,249 · see citing works
Handbook of Developmental Cognitive Neuroscience
(2008) · The MIT Press eBooks · doi:10.7551/mitpress/7437.001.0001 · Cited by 1,188 · see citing works
Research Methods and Statistics in Psychology
Coolican, H., & Coolican, H. (2014) · Psychology Press eBooks · doi:10.4324/9780203769836 · Cited by 1,174 · see citing works
Foundations of clinical research application to practice
Portney, L. (1993) · Medical Entomology and Zoology · Cited by 1,133 · see citing works
Bilingual experience and executive functioning in young children
Carlson, S. M., & Meltzoff, A. N. (2008) · Developmental Science · doi:10.1111/j.1467-7687.2008.00675.x · Cited by 1,052 · see citing works
Self-citationAttention and inhibition in bilingual children: evidence from the dimensional change card sort task
Bialystok, E., & Martin, M. M. (2004) · Developmental Science · doi:10.1111/j.1467-7687.2004.00351.x · Cited by 970 · see citing works
Theory of mind and rule-based reasoning
Frye, D., Zelazo, P. D., & Palfai, T. (1995) · Cognitive Development · doi:10.1016/0885-2014(95)90024-1 · Cited by 923 · see citing works
Carlson, S. M., Moses, L. J., & Breton, C. (2002) · Infant and Child Development · doi:10.1002/icd.298 · Cited by 838 · see citing works
Self-citationResearch Methods and Statistics in Psychology
Coolican, H., & Coolican, H. (2013) · Routledge eBooks · doi:10.4324/9780203769669 · Cited by 838 · see citing works
Executive function in preschoolers: Links with theory of mind and verbal ability
Hughes, C. H. (1998) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.1998.tb00921.x · Cited by 715 · see citing works
Inhibitory control and emotion regulation in preschool children
Carlson, S. M., & Wang, T. S. (2007) · Cognitive Development · doi:10.1016/j.cogdev.2007.08.002 · Cited by 701 · see citing works
Self-citationFoundations of clinical research. Applications to practice
Fortney, L. (1993) · Medical Entomology and Zoology · Cited by 700 · see citing works
Executive function and theory of mind: Stability and prediction from ages 2 to 3.
Carlson, S. M., Mandell, D. J., & Williams, L. (2004) · Developmental Psychology · doi:10.1037/0012-1649.40.6.1105 · Cited by 684 · see citing works
Self-citationHongwanishkul, D., Happaney, K. R., Lee, W. S. C., & Zelazo, P. D. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_4 · Cited by 654 · see citing works
Berlin, H. A. (2004) · Brain · doi:10.1093/brain/awh135 · Cited by 541 · see citing works
Brock, L. L., Rimm‐Kaufman, S. E., Nathanson, L., & Grimm, K. J. (2009) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2009.06.001 · Cited by 538 · see citing works
Executive Function: Mechanisms Underlying Emotion Regulation.
Zelazo, P. D., & Cunningham, W. A. (2007) · Cited by 424 · see citing works
Executive Function and Theory of Mind in 2 Year Olds: A Family Affair?
Hughes, C. H., & Ensor, R. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_5 · Cited by 394 · see citing works
Predicting Cognitive Control From Preschool to Late Adolescence and Young Adulthood
Eigsti, I.-M., Zayas, V., Mischel, W., Shoda, Y., Ayduk, O., Dadlani, M. B., Davidson, M. C., Aber, J. L., & Casey, B. J. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01732.x · Cited by 365 · see citing works
Identification and Description of New Tests of Executive Functioning in Children
Archibald, S. J., & Kerns, K. A. (1999) · Child Neuropsychology · doi:10.1076/chin.5.2.115.3167 · Cited by 360 · see citing works
Lowe, C., & Rabbitt, P. (1998) · Neuropsychologia · doi:10.1016/s0028-3932(98)00036-0 · Cited by 344 · see citing works
Carlson, S. M., Davis, A., & Leach, J. G. (2005) · Psychological Science · doi:10.1111/j.1467-9280.2005.01583.x · Cited by 311 · see citing works
Self-citationClinical, imaging, lesion, and genetic approaches toward a model of cognitive control
Casey, B. J., Tottenham, N., & Fossella, J. (2002) · Developmental Psychobiology · doi:10.1002/dev.10030 · Cited by 269 · see citing works
Diamond, A. D., Carlson, S. M., & Beck, D. M. (2005) · Developmental Neuropsychology · doi:10.1207/s15326942dn2802_7 · Cited by 268 · see citing works
Self-citationThe development of future-oriented prudence and altruism in preschoolers
Thompson, C., Barresi, J., & Moore, C. L. (1997) · Cognitive Development · doi:10.1016/s0885-2014(97)90013-7 · Cited by 260 · see citing works
Bernier, A., Carlson, S. M., Bordeleau, S., & Carrier, J. (2010) · Child Development · doi:10.1111/j.1467-8624.2010.01507.x · Cited by 226 · cited 1× · see citing works
“For example, children’s performance on conflict EF, but not delay EF, is significantly related to parenting quality (Bernier et al., 2010), to children’s theory of mind performance (Carlson & Moses; Carlson, Moses, & Breton, 2002), and to effective teaching (Davis-Unger & Carlson).”
BackgroundSelf-citationChild development at the intersection of emotion and cognition.
(2010) · American Psychological Association eBooks · doi:10.1037/12059-000 · Cited by 223 · see citing works
Cognitive person variables in the delay of gratification of older children at risk.
Rodríguez, M., Mischel, W., & Shoda, Y. (1989) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.57.2.358 · Cited by 219 · see citing works
Aman, C. J., Ralph J., J. R., & Pennington, B. F. (1998) · Developmental Psychology · doi:10.1037/0012-1649.34.5.956 · Cited by 215 · see citing works
Overman, W. (2004) · Brain and Cognition · doi:10.1016/s0278-2626(03)00279-3 · Cited by 215 · see citing works
Development of Affective Decision Making for Self and Other
Prencipe, A., & Zelazo, P. D. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01564.x · Cited by 191 · see citing works
Neural origin of cognitive shifting in young children
Moriguchi, Y., & Hiraki, K. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0809747106 · Cited by 187 · see citing works
Cognitive person variables in the delay of gratification of older children at risk.
Rodriguez, M. L., Mischel, W., & Shoda, Y. (1989) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.57.2.358 · Cited by 163 · see citing works
Educating people to be emotionally intelligent
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1145 · Cited by 154 · see citing works
Hot executive function: Emotion and the development of cognitive control.
Zelazo, P. D., Qu, L., & Kesek, A. C. (2010) · American Psychological Association eBooks · doi:10.1037/12059-006 · Cited by 139 · see citing works
Bishop, D., Aamodt‐Leeper, G., Creswell, C., McGurk, R., & Skuse, D. (2001) · Journal of Child Psychology and Psychiatry · doi:10.1111/1469-7610.00749 · Cited by 126 · see citing works
Development of Executive Function During Childhood
Boschloo, A., Krabbendam, L., Aben, A., et al. (2016) · Frontiers research topics · doi:10.3389/978-2-88919-800-9 · Cited by 84 · see citing works
The facilitative effect of positive stimuli on 3-year-olds’ flexible rule use
Qu, L., & Zelazo, P. D. (2007) · Cognitive Development · doi:10.1016/j.cogdev.2007.08.010 · Cited by 72 · see citing works
Validation of executive function tasks with young children
Gnys, J. A., & Willis, W. G. (1991) · Developmental Neuropsychology · doi:10.1080/87565649109540507 · Cited by 66 · see citing works
Children’s Teaching Skills: The Role of Theory of Mind and Executive Function
Davis‐Unger, A. C., & Carlson, S. M. (2008) · Mind Brain and Education · doi:10.1111/j.1751-228x.2008.00043.x · Cited by 38 · see citing works
Self-citationTemporal stability of the Wisconsin Card Sorting Test in an untreated patient sample
Ingram, F., Greve, K. W., Fishel-Ingram, P., & Soukup, V. M. (1999) · British Journal of Clinical Psychology · doi:10.1348/014466599162764 · Cited by 27 · see citing works
Tate, R. L., Perdices, M., & Maggiotto, S. (1998) · The Clinical Neuropsychologist · doi:10.1076/clin.12.3.348.1988 · Cited by 26 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Hughes C Ensor R 2005 Theory of mind and executive function: A family affair? Developmental Neuropsychology 28 645 668 16144431
- Roid GH 2005 Stanford-Binet Intelligence Scales for Early Childhood, Fifth Edition, Manual Itasca, IL Riverside Publishing
Cited by
Sorted by citations
II. NIH TOOLBOX COGNITION BATTERY (CB): MEASURING EXECUTIVE FUNCTION AND ATTENTION
Zelazo, P. D., Anderson, J. E., Richler, J., Wallner-Allen, K., Beaumont, J. L., & Weintraub, S. (2013) · Monographs of the Society for Research in Child Development · doi:10.1111/mono.12032 · Cited by 823 · see citing works
Rudolph, M. D., Graham, A. M., Feczko, E., Miranda-Dominguez, O., Rasmussen, J. M., Nardos, R., Entringer, S., Wadhwa, P. D., Buss, C., & Fair, D. A. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0128-y · Cited by 374 · see citing works
NIH Toolbox Cognition Battery (CB): Validation of Executive Function Measures in Adults
Zelazo, P. D., Anderson, J. E., Richler, J., Wallner-Allen, K., Beaumont, J. L., Conway, K. P., Gershon, R., & Weintraub, S. (2014) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617714000472 · Cited by 350 · see citing works
Assessing Executive Function in Preschoolers
Anderson, P. J., & Reidy, N. (2012) · Neuropsychology Review · doi:10.1007/s11065-012-9220-3 · Cited by 293 · see citing works
Graham, A. M., Rasmussen, J. M., Rudolph, M. D., Heim, C. M., Gilmore, J. H., Styner, M., Potkin, S. G., Entringer, S., Wadhwa, P. D., Fair, D. A., et al. (2018) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.05.027 · Cited by 282 · see citing works
Hostinar, C. E., Stellern, S. A., Schaefer, C., Carlson, S. M., & Gunnar, M. R. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1121246109 · Cited by 234 · see citing works
Self-citationCircle of Security–Parenting: A randomized controlled trial in Head Start
Cassidy, J., Brett, B. E., Gross, J. T., Stern, J. A., Martin, D. R., Mohr, J. J., & Woodhouse, S. S. (2017) · Development and Psychopathology · doi:10.1017/s0954579417000244 · Cited by 167 · see citing works
What would Batman do? Self‐distancing improves executive function in young children
White, R. E., & Carlson, S. M. (2016) · Developmental Science · doi:10.1111/desc.12314 · Cited by 130 · see citing works
Self-citationThe potential of infant fMRI research and the study of early life stress as a promising exemplar
Graham, A. M., Pfeifer, J. H., Fisher, P. A., Lin, W., Gao, W., & Fair, D. A. (2015) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2014.09.005 · Cited by 114 · see citing works
Associations between white matter microstructure and infants' working memory
Short, S. J., Elison, J. T., Goldman, B. D., Styner, M., Gu, H., Connelly, M., Maltbie, E., Woolson, S., Lin, W., Gerig, G., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.021 · Cited by 105 · see citing works
Lipsey, M. W., Nesbitt, K. T., Farran, D. C., Dong, N., Fuhs, M. W., & Wilson, S. J. (2017) · Journal of Educational Psychology · doi:10.1037/edu0000203 · Cited by 87 · see citing works
Schmidt, M., Mavilidi, M. F., Singh, A., & Englert, C. (2020) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2020.101908 · Cited by 70 · see citing works
Complementary assessments of executive function in preterm and full-term preschoolers
Loe, I. M., Chatav, M., & Alduncin, N. (2015) · Child Neuropsychology · doi:10.1080/09297049.2014.906568 · Cited by 58 · see citing works
Doom, J. R., Gunnar, M. R., Georgieff, M. K., Kroupina, M. G., Frenn, K., Fuglestad, A. J., & Carlson, S. M. (2014) · Child Development · doi:10.1111/cdev.12231 · Cited by 40 · see citing works
Self-citationKälin, S., & Roebers, C. M. (2022) · Metacognition and Learning · doi:10.1007/s11409-022-09306-x · Cited by 27 · see citing works
- Authors
- Beck, D. M., Schaefer, C., Pang, K., & Carlson, S. M.
- Affiliations
- Simpson University (US)
- University of Minnesota (US)
- University of Washington (US)
- Corresponding author
- Carlson, Stephanie M.
- Venue
- Journal of Cognition and Development
- Publisher
- Informa UK Limited
- Published
- 2011-04
- Volume
- 12
- Issue
- 2
- Pages
- 169-193
- ISSN
- 1524-8372 (ISSN-L)
- 1532-7647
- PMC
- PMC3105625
- OpenAlex
- W2029517916
- Cited by
- 262
- References
- 59
- Citations by year
- 2026: 12 · 2025: 18 · 2024: 22 · 2023: 19 · 2022: 26 · 2021: 22 · 2020: 20 · 2019: 13 · 2018: 22 · 2017: 20 · 2016: 20 · 2015: 14 · 2014: 13 · 2013: 12 · 2012: 7
- FWCI
- 52.06
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Cognition and Development (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD051495)
- Topics
- Social Sciences › Education › Early Childhood Education and Development
- Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
- Social Sciences › Psychology › Developmental and Educational Psychology › Child and Animal Learning Development
- Keywords
- executive function, preschool children, test-retest reliability, extrinsic rewards, individual differences, kindergarten readiness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
