Reliability and Validity of Composite Scores from the NIH Toolbox Cognition Battery in Adults
Open accessAbstract
This study describes psychometric properties of the NIH Toolbox Cognition Battery (NIHTB-CB) Composite Scores in an adult sample. The NIHTB-CB was designed for use in epidemiologic studies and clinical trials for ages 3 to 85. A total of 268 self-described healthy adults were recruited at four university-based sites, using stratified sampling guidelines to target demographic variability for age (20–85 years), gender, education, and ethnicity. The NIHTB-CB contains seven computer-based instruments assessing five cognitive sub-domains: Language, Executive Function, Episodic Memory, Processing Speed, and Working Memory. Participants completed the NIHTB-CB, corresponding gold standard validation measures selected to tap the same cognitive abilities, and sociodemographic questionnaires. Three Composite Scores were derived for both the NIHTB-CB and gold standard batteries: “Crystallized Cognition Composite,” “Fluid Cognition Composite,” and “Total Cognition Composite” scores. NIHTB Composite Scores showed acceptable internal consistency (Cronbach’s alphas=0.84 Crystallized, 0.83 Fluid, 0.77 Total), excellent test–retest reliability (r: 0.86–0.92), strong convergent (r: 0.78–0.90) and discriminant (r: 0.19–0.39) validities versus gold standard composites, and expected age effects ( r =0.18 crystallized, r =−0.68 fluid, r =−0.26 total). Significant relationships with self-reported prior school difficulties and current health status, employment, and presence of a disability provided evidence of external validity. The NIH Toolbox Cognition Battery Composite Scores have excellent reliability and validity, suggesting they can be used effectively in epidemiologic and clinical studies. ( JINS , 2014, 20 , 1–11)
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This improvement is greater than the small-to-medium practice effect reported in the normative sample for this battery.
“this improvement is greater than the small-to-medium practice effect reported in the normative sample for this battery [44]”
From A phase I trial of accelerated intermittent theta burst rTMS for amnestic MCI (2024) · cites this paper as [44]
The Results section (Test–Retest Reliability and Practice Effects; page not provided) reports a small-to-medium practice effect for the Fluid Cognition composite (d = 0.42). But that estimate came from 89 retested participants described as a convenience sample, not a normative sample; the Discussion says national norming was still under way. The cited paper also cannot establish whether the citing study’s improvement was greater without that improvement’s magnitude. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Delis-Kaplan Executive Function System
Delis, D. C., Kaplan, E., & Kramer, J. H. (2012) · PsycTESTS Dataset · doi:10.1037/t15082-000 · Cited by 2,683 · see citing works
Paced Auditory Serial-Addition Task: A Measure of Recovery from Concussion
Gronwall, D. M. A. (1977) · Perceptual and Motor Skills · doi:10.2466/pms.1977.44.2.367 · Cited by 2,395 · see citing works
Cognition assessment using the NIH Toolbox
Weintraub, S., Dikmen, S. S., Heaton, R. K., Tulsky, D. S., Zelazo, P. D., Bauer, P. J., Carlozzi, N. E., Slotkin, J., Blitz, D., Wallner-Allen, K., et al. (2013) · Neurology · doi:10.1212/wnl.0b013e3182872ded · Cited by 1,287 · see citing works
Self-citationAkshoomoff, N., Beaumont, J. L., Bauer, P. J., Dikmen, S. S., Gershon, R. C., Mungas, D., Slotkin, J., Tulsky, D., Weintraub, S., Zelazo, P. D., et al. (2013) · Monographs of the Society for Research in Child Development · doi:10.1111/mono.12038 · Cited by 299 · see citing works
Self-citationOrganization of Data on Life-Span Development of Human Abilities
HORN, J. L. (1970) · Life-Span Developmental Psychology · doi:10.1016/b978-0-12-293850-4.50022-4 · Cited by 226 · see citing works
Flagged first, then by citations
Research works printed but not linked: 14
- Byrd DA Miller SW Reilly J Weber S Wall TL Heaton RK Early environmental factors, ethnicity, and adult cognitive test performance The Clinical Neuropsychologist 2006 20 243 260 16690545
- Cattell RB Abilities: Their structure, growth, and action 1971 Cambridge Cambridge University Press
- Cohen J A power primer Psychological Bulletin 1992 112 155 159 19565683
- Cysique L Franklin D Abramson I Ellis R Letendre S Collier A Heaton RK Normative data and validation of a regression based summary score for assessing meaningful neuropsychological change Journal of Clinical and Experimental Neuropsychology 2011 33 5 505 522 21391011 PMC3151558
- Dunn LM Dunn LM Peabody Picture Vocabulary Test-Fourth Edition (PPVT-4) 2007 Circle Pines, MN American Guidance Services
- Heaton RK Ryan L Grant I Grant I Adams KM Demographic influences and use of demographically corrected norms in neuropsychological assessment Neuropsychological assessment of neuropsychiatric and neuromedical disorders 2009 New York Oxford University Press 127 155
- Heaton RK Taylor MJ Manly J Tulsky DS Saklofske DH Chelune GJ Heaton RK Ivnik RJ Bornstein R Prifitera A Ledbetter MF Demographic effects and use of demographically corrected norms with the WAIS-III and WMS III Clinical interpretation of the WAIS-III and WMS-III 2003 San Diego, CA Academic Press 181 210
- Horn JL Organization of abilities and the development of intelligence Psychological Review 1968 75 242 259 4875815
- Horn JL Cattell RB Age differences in fluid and crystallized intelligence Acta Psychologica 1967 26 107 129 6037305
- Li S-C Lindenberger U Hommel B Aschersleben G Prinz W Baltes P Transformations in the couplings among intellectual abilities and constituent cognitive processes across the life span Psychological Science 2004 15 155 163 15016286
- Mungas D Heaton RK Tulsky D Zelazo PD Slotkin J Blitz D Gershon R Factor structure, convergent validity, and discriminant validity of the NIH Toolbox Cognitive Health Battery (NIHTB-CHB) in Adults Journal of the International Neuropsychological Society (submitted). PMC4103956 24960474
- Mungas D Widaman K Zelazo PD Tulsky D Heaton R Slotkin J Gershon R Zelazo PD Bauer PJ NIH Toolbox Cognitive Function Battery (CFB): Factor structure for 3- to 15- year-olds National Institutes of Health Toolbox-Cognitive Function Battery (NIH Toolbox CFB):validation for children between 3 and 15 years. Ann Arbor, MI: Society for Research in Child Development Monographs 2013 78 4 103 118 PMC3950958 23952205
- Norman MA Moore DJ Taylor M Franklin D Cysique L Ake C Heaton RK Demographically corrected norms for African Americans and Caucasians on the Hopkins Verbal Learning Test-Revised, Brief Visuospatial Memory Test- Revised, Stroop Color and Word Test, and Wisconsin Card Sorting Test 64-Card Version Journal of Clinical and Experimental Neuropsychology 2011 33 7 793 804 21547817 PMC3154384
- Wechsler D Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) 2008 San Antonio, TX The Psychological Corporation
Cited by
Sorted by citations
Persistent neurologic symptoms and cognitive dysfunction in non-hospitalized COVID-19 “long haulers”
Graham, E. L., Clark, J. R., Orban, Z. S., Lim, P. H., Szymanski, A. L., Taylor, C., DiBiase, R. M., Jia, D. T., Balabanov, R., Ho, S. U., et al. (2021) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.51350 · Cited by 693 · see citing works
Hartman, S. J., Nelson, S. H., Myers, E., Natarajan, L., Sears, D. D., Palmer, B. W., Weiner, L. S., Parker, B. A., & Patterson, R. E. (2018) · Cancer · doi:10.1002/cncr.30987 · Cited by 188 · see citing works
Jonklaas, J., Bianco, A. C., Cappola, A. R., Celi, F. S., Fliers, E., Heuer, H., McAninch, E. A., Moeller, L. C., Nygaard, B., Sawka, A. M., et al. (2021) · Thyroid® · doi:10.1089/thy.2020.0720 · Cited by 172 · see citing works
Hackman, D. A., Cserbik, D., Chen, J.-C., Berhane, K., Minaravesh, B., McConnell, R., & Herting, M. M. (2021) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2021.0426 · Cited by 111 · see citing works
Dhamala, E., Yeo, B. T. T., & Holmes, A. J. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.09.024 · Cited by 91 · see citing works
Jonklaas, J., Bianco, A. C., Cappola, A. R., Celi, F. S., Fliers, E., Heuer, H., McAninch, E. A., Moeller, L. C., Nygaard, B., Sawka, A. M., et al. (2021) · European Thyroid Journal · doi:10.1159/000512970 · Cited by 78 · see citing works
Scott, E. P., Sorrell, A., & Benitez, A. (2019) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617719000614 · Cited by 69 · see citing works
Using the NIH Toolbox Cognition Battery (NIHTB-CB) in individuals with traumatic brain injury.
Tulsky, D. S., Carlozzi, N. E., Holdnack, J., Heaton, R. K., Wong, A., Goldsmith, A., & Heinemann, A. W. (2017) · Rehabilitation Psychology · doi:10.1037/rep0000174 · Cited by 67 · see citing works
Self-citationChadwick, L., Roth, E., Minich, N. M., Taylor, H. G., Bigler, E. D., Cohen, D. M., Bacevice, A., Mihalov, L. K., Bangert, B. A., Zumberge, N. A., et al. (2021) · Journal of Neurotrauma · doi:10.1089/neu.2020.7513 · Cited by 50 · see citing works
A phase I trial of accelerated intermittent theta burst rTMS for amnestic MCI
Aghamoosa, S., Lopez, J., Rbeiz, K., Fleischmann, H. H., Horn, O., Madden, K., Caulfield, K. A., Antonucci, M. U., Revuelta, G., McTeague, L. M., et al. (2024) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2023-332680 · Cited by 12 · see citing works
Aghamoosa, S., Nolin, S. A., Chen, A. A., Caulfield, K. A., Lopez, J., Rbeiz, K., Fleischmann, H. H., Horn, O., Madden, K., Antonucci, M., et al. (2025) · Brain Stimulation · doi:10.1016/j.brs.2025.04.012 · Cited by 6 · see citing works
- Authors
- Heaton, R. K., Akshoomoff, N., Tulsky, D., Mungas, D., Weintraub, S., Dikmen, S., Beaumont, J., Casaletto, K. B., Conway, K., Slotkin, J., & Gershon, R.
- Affiliations
- University of California San Diego (US)
- Kessler Foundation (US)
- University of California, Davis (US)
- University of Washington (US)
- Northwestern University (US)
- San Diego State University (US)
- National Institutes of Health (US)
- Corresponding author
- Heaton, Robert K.
- ORCID
- Venue
- Journal of the International Neuropsychological Society
- Publisher
- Cambridge University Press (CUP)
- Published
- 2014-06-24
- Volume
- 20
- Issue
- 6
- Pages
- 588-598
- ISSN
- 1355-6177 (ISSN-L)
- 1469-7661
- PMC
- PMC4103963
- OpenAlex
- W2128607849
- Cited by
- 437
- References
- 5
- Citations by year
- 2026: 37 · 2025: 55 · 2024: 50 · 2023: 53 · 2022: 56 · 2021: 58 · 2020: 34 · 2019: 39 · 2018: 17 · 2017: 24 · 2016: 10 · 2015: 3
- FWCI
- 17.62
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the International Neuropsychological Society (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- Kessler Foundation
- California Department of Public Health
- Brown University
- Ohio State University
- Northwestern University
- University of Washington
- Temple University
- Davee Foundation
- NorthShore University HealthSystem
- National Institutes of Health (HHS-N-260–2006-00007-C)
- University of California, San Diego (P30MH062512)
- University of California, San Francisco
- National Institute on Disability and Rehabilitation Research (H133B090024, H133G070138, H133N060022)
- National Institute on Aging (P30AG010129, N01-AG-6-0007, R01AG010220, RC2AG036498, P30AG013854, AG-260-06-01)
- National Heart, Lung, and Blood Institute (NHLBI K23, K23HL085766 NIA, K23HL085766)
- National Institute of Mental Health (U01MH083506, R01MH058076, R01MH081861, R01MH083552, R01MH078737, R01MH078748, R01MH073433, R01MH060720, P30MH062512)
- National Institute on Drug Abuse (P50DA026306, F31DA035708, P01DA012065, T32DA031098)
- National Institute on Deafness and Other Communication Disorders (R01DC008552)
- National Institute of Diabetes and Digestive and Kidney Diseases (U01DK082342)
- National Institute of Neurological Disorders and Stroke (R01NS058302, U01 NS 056 975 02, H133B090024)
- National Institute of Arthritis and Musculoskeletal and Skin Diseases (U54AR057943, U01AR057929)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD061400)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- NIH Toolbox Cognition Battery, composite score, psychometric properties, reliability statistics, validity, fluid cognition, cognitive aging, executive function, episodic memory, processing speed, working memory, language, test-retest reliability, convergent validity, generalizability, healthy adults
- MeSH terms
- Adult, Age Factors, Aged, Aged, 80 and over, Cognition, Female, Humans, Language, Male, Memory, Short-Term, Middle Aged, National Institutes of Health (U.S.), Neuropsychological Tests, Psychometrics, Surveys and Questionnaires, Reaction Time, United States, Reproducibility of Results, Young Adult, Executive Function, Memory, Episodic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
