Reliability and validity of the UK Biobank cognitive tests
Open accessPublished version of a preprint
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
UK Biobank is a health resource with data from over 500,000 adults. The cognitive assessment in UK Biobank is brief and bespoke, and is administered without supervision on a touchscreen computer. Psychometric information on the UK Biobank cognitive tests are limited. Despite the non-standard nature of these tests and the limited psychometric information, the UK Biobank cognitive data have been used in numerous scientific publications. The present study examined the validity and short-term test-retest reliability of the UK Biobank cognitive tests. A sample of 160 participants (mean age = 62.59, SD = 10.24) was recruited who completed the UK Biobank cognitive assessment and a range of well-validated cognitive tests ('reference tests'). Fifty-two participants returned 4 weeks later to repeat the UK Biobank tests. Correlations were calculated between UK Biobank tests and reference tests. Two measures of general cognitive ability were created by entering scores on the UK Biobank cognitive tests, and scores on the reference tests, respectively, into separate principal component analyses and saving scores on the first principal component. Four-week test-retest correlations were calculated for UK Biobank tests. UK Biobank cognitive tests showed a range of correlations with their respective reference tests, i.e. those tests that are thought to assess the same underlying cognitive ability (mean Pearson r = 0.53, range = 0.22 to 0.83, p≤.005). The measure of general cognitive ability based on the UK Biobank cognitive tests correlated at r = 0.83 (p < .001) with a measure of general cognitive ability created using the reference tests. Four-week test-retest reliability of the UK Biobank tests were moderate-to-high (mean Pearson r = 0.55, range = 0.40 to 0.89, p≤.003). Despite the brief, non-standard nature of the UK Biobank cognitive tests, some tests showed substantial concurrent validity and test-retest reliability. These psychometric results provide currently-lacking information on the validity of the UK Biobank cognitive tests.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Although the UK Biobank cognitive battery contains adapted versions of commonly used tests and tests designed for the UK Biobank, the g-factor from well-validated tests correlates highly with the g-factor from the UK Biobank cognitive tests (r = 0.83; [34]).”
From High intelligence is not associated with a greater propensity for mental health disorders (2023) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728 · cited 7×
“The number of studies using the UK Biobank cognitive data will likely increase now that UK Biobank have released genotyping data on all 500,000 participants, and brain imaging data is available on a subsample of UK Biobank participants [1].”
BackgroundMethodInfluential"General Intelligence," Objectively Determined and Measured
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412107 · Cited by 6,207
Human Cognitive Abilities: A Survey of Factor-Analytic Studies
Carroll, J. B. (1993) · doi:10.1017/cbo9780511571312 · Cited by 4,121
Peabody Picture Vocabulary Test
Campbell, J. M., & Dommestrup, A. K. (2010) · The Corsini Encyclopedia of Psychology · doi:10.1002/9780470479216.corpsy0649 · Cited by 3,787
Human cognitive abilities a survey of factor-analytic studies
Carroll, J. B. (2004) · Cited by 3,400
Delis-Kaplan Executive Function System
Delis, D. C., Kaplan, E., & Kramer, J. H. (2012) · PsycTESTS Dataset · doi:10.1037/t15082-000 · Cited by 2,683
Peabody Picture Vocabulary Test
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n336 · Cited by 1,743
Shunk, A. W., Davis, A. S., & Dean, R. S. (2006) · Applied Neuropsychology · doi:10.1207/s15324826an1304_9 · Cited by 1,523
Cognition assessment using the NIH Toolbox
Weintraub, S., Dikmen, S. S., Heaton, R. K., Tulsky, D. S., Zelazo, P. D., Bauer, P. J., Carlozzi, N. E., Slotkin, J., Blitz, D., Wallner-Allen, K., et al. (2013) · Neurology · doi:10.1212/wnl.0b013e3182872ded · Cited by 1,287 · cited 15×
“Test-retest interval range 7 to 21 days, n = 89 [13].”
MethodInfluentialSmith, A. (2016) · PsycTESTS Dataset · doi:10.1037/t27513-000 · Cited by 1,277
Vision assessment using the NIH Toolbox
Varma, R., McKean-Cowdin, R., Vitale, S., Slotkin, J., & Hays, R. D. (2013) · Neurology · doi:10.1212/wnl.0b013e3182876e0a · Cited by 1,178
Selective review of cognitive aging
SALTHOUSE, T. A. (2010) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617710000706 · Cited by 1,166 · cited 4×
“A previous study using UK Biobank baseline data [18] found that scores on the UKB Fluid IQ test showed mean values that remained relatively stable between the ages of 40 and 60 years and therefore did not show the age-related decline across the adult lifespan that is the hallmark of fluid ability [2].”
BackgroundMethodResultInfluentialHsieh, S., Schubert, S., Hoon, C., Mioshi, E., & Hodges, J. R. (2013) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000351671 · Cited by 973
Peabody Picture Vocabulary Test--Fourth Edition
Dunn, L. M., & Dunn, D. M. (2012) · PsycTESTS Dataset · doi:10.1037/t15144-000 · Cited by 914
DIKMEN, S. S., HEATON, R. K., GRANT, I., & TEMKIN, N. R. (1999) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617799544056 · Cited by 473 · cited 1×
“Test-retest interval mean = 11 months, n = 384 [22].”
CAMDEX : the Cambridge examination for mental disorders of the elderly
Roth, M. (1988) · Medical Entomology and Zoology · Cited by 425
Just one g: consistent results from three test batteries
Johnson, W., Bouchard, T. J., Krueger, R. F., McGue, M., & Gottesman, I. I. (2004) · Intelligence · doi:10.1016/s0160-2896(03)00062-x · Cited by 353 · cited 1×
“These correlations reported here were lower than those reported in one study [5] that found that three measures of general cognitive ability created using three entirely different cognitive test batteries correlated nearly perfectly (r� 0.”
ResultMETASTROKE Consortium, I. C. F. B. P. G., Hagenaars, S. P., SpiroMeta Consortium, CHARGE Consortium Pulmonary Group, C. C. A. A. L. G., Harris, S. E., Davies, G., Hill, W. D., Liewald, D. C. M., Ritchie, S. J., Marioni, R. E., et al. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.225 · Cited by 350
Self-citationDeary, I. J., Liewald, D., & Nissan, J. (2011) · Behavior Research Methods · doi:10.3758/s13428-010-0024-1 · Cited by 328 · cited 3×
“Processing speed Reaction Time Test UKB RT Deary-Liewald Reaction Time Test Simple Reaction Time and Choice Reaction Time [17] DLRT Simple RT DLRT Choice RT”
MethodInfluentialSelf-citationStill just 1 g: Consistent results from five test batteries
Johnson, W., Nijenhuis, J. T., & Bouchard, T. J. (2008) · Intelligence · doi:10.1016/j.intell.2007.06.001 · Cited by 304
The Mini-Addenbrooke's Cognitive Examination: A New Assessment Tool for Dementia
Hsieh, S., McGrory, S., Leslie, F., Dawson, K., Ahmed, S., Butler, C. R., Rowe, J. B., Mioshi, E., & Hodges, J. R. (2015) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000366040 · Cited by 277 · cited 2×
“Global cognitive function Mini Addenbrooke’s Cognitive Examination [19] M-ACE”
BackgroundMethodLyall, D. M., Cullen, B., Allerhand, M., Smith, D. J., Mackay, D., Evans, J., Anderson, J., Fawns-Ritchie, C., McIntosh, A. M., Deary, I. J., et al. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154222 · Cited by 256 · cited 10×
“49 on this component [3], confirming the scores were positively correlated and that a g component was present in the original UK Biobank tests.”
BackgroundMethodInfluentialSelf-citationA proposed panel of biomarkers of healthy ageing
Lara, J., Cooper, R., Nissan, J., Ginty, A. T., Khaw, K.-T., Deary, I. J., Lord, J. M., Kuh, D., & Mathers, J. C. (2015) · BMC Medicine · doi:10.1186/s12916-015-0470-9 · Cited by 223 · cited 1×
“75) with the NART, which is often used as an estimate of crystallised cognitive ability [14, 15].”
MethodSelf-citationDeary, I. J. (2013) · Current Biology · doi:10.1016/j.cub.2013.07.021 · Cited by 139
Self-citationHalstead-Reitan Neuropsychological Test Battery
Hussey, J., & Allen, D. N. (2018) · Springer eBooks · doi:10.1007/978-3-319-57111-9_189 · Cited by 131
The Rivermead Behavioural Memory Test - Extended Version
Wilson, B., Clare, L., Cockburn, J., Baddeley, A., Tate, R. L., & Watson, P. (1999) · UCL Discovery (University College London) · Cited by 94
Hypoconnectivity and Hyperfrontality in Retired American Football Players
Hampshire, A., MacDonald, A., & Owen, A. M. (2013) · Scientific Reports · doi:10.1038/srep02972 · Cited by 87
COGNITO: Computerized Assessment of Information Processing
Ritchie, K. A. (2014) · Journal of Psychology & Psychotherapy · doi:10.4172/2161-0487.1000136 · Cited by 65 · cited 3×
“Test-retest interval range 2 to 3 weeks, n = 78 [23].”
BackgroundMethodDeary, I. J., Harris, S. E., & Hill, W. D. (2019) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2018.07.005 · Cited by 56 · cited 1×
“A composite score, which is created by saving scores based on the first unrotated principal component, typically accounts for about 40% of the variance in a wide range of different cognitive tests [8, 9].”
BackgroundSelf-citationHalstead-Reitan Neuropsychological Test Battery
Halstead, W. C., & Reitan, R. M. (2021) · PsycTESTS Dataset · doi:10.1037/t00275-000 · Cited by 49
CAMDEX-R : the Cambridge examination for mental disorders of the elderly
Roth, M. (1998) · Medical Entomology and Zoology · Cited by 47
Schmidt, F. L. (2017) · Archives of Scientific Psychology · doi:10.1037/arc0000033 · Cited by 45 · cited 1×
“It is an error not to acknowledge this, as Schmidt [20] discusses in detail.”
BackgroundCooper, C. (2003) · Research Portal (Queen's University Belfast) · Cited by 2
Research works printed but not linked: 6
- Wechsler D. WMS-IV Technical and Interpretive Manual Bloomington, MN: Pearson; 2009.
- Delis DC, Kaplan E, Kramer JH. D-KEFS Technical Manual. Bloomington, MN: Pearson; 2001.
- Mathers JC, Deary IJ, Kuh D, Lord JM, Khaw KT, Lara J, et al Guidelines for biomarkers of healthy ageing. Medical Research Council, 2015.
- Wechsler D. WMS-IV Administration and Scoring Manual. London: Pearson; 2010.
- Wechsler D. WAIS-IV Administration and Scoring Manual. London: Pearson; 2010.
- Wechsler D. WAIS-IV Technical and Interpretive Manual. San Antonio, Texas: Pearson; 2008.
Cited by
SARS-CoV-2 is associated with changes in brain structure in UK Biobank
Douaud, G., Lee, S., Alfaro-Almagro, F., Arthofer, C., Wang, C., McCarthy, P., Lange, F., Andersson, J. L. R., Griffanti, L., Duff, E., et al. (2022) · Nature · doi:10.1038/s41586-022-04569-5 · Cited by 1,646
Confound modelling in UK Biobank brain imaging
Alfaro-Almagro, F., McCarthy, P., Afyouni, S., Andersson, J. L. R., Bastiani, M., Miller, K. L., Nichols, T. E., & Smith, S. M. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117002 · Cited by 249
“…images and the independent variables were a set of features that are related to structural motion and QC (e.g. smoothness estimates in X, Y, and Z (Flitney and Jenkinson, 2000), average Euler number of the FreeSurfer surfaces (Rosen et al., 2018), Qoala-T quality metric of FreeSurfer output…”
BackgroundData Integration Challenges for Machine Learning in Precision Medicine
Martínez-García, M., & Hernández-Lemus, E. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2021.784455 · Cited by 185
“…15 years has been generating massive data corpora used for their own means but also encouraging and feeding other data-intensive analytical efforts from genetic disease association (29) to brain imaging (30) to psychology (31) and social determinants of health (32), to name just a few instances.”
BackgroundMorys, F., Dadar, M., & Dagher, A. (2021) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgab135 · Cited by 166
Impact of sleep duration on executive function and brain structure
Tai, X. Y., Chen, C., Manohar, S., & Husain, M. (2022) · Communications Biology · doi:10.1038/s42003-022-03123-3 · Cited by 148
Metadata
- Authors
- Fawns-Ritchie, C., & Deary, I. J.
- Affiliations
- University of Edinburgh (GB)
- Corresponding author
- Deary, Ian J.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-04-20
- Volume
- 15
- Issue
- 4
- Pages
- e0231627
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7170235
- OpenAlex
- W3016980863
- Cited by
- 396
- References
- 33
- Citations by year
- 2026: 65 · 2025: 82 · 2024: 94 · 2023: 61 · 2022: 55 · 2021: 28 · 2020: 9
- FWCI
- 36.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- medRxiv (acceptedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- Discovery Research Portal (University of Dundee) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Biotechnology and Biological Sciences Research Council (grant number MR/K026992/1)
- Medical Research Council (grant number MR/K026992/1, MR/L023784/2)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Keywords
- UK Biobank, cognitive tests, test-retest reliability, concurrent validity, general intelligence, psychometric properties, reference tests
- MeSH terms
- Aged, Cognition, Female, United Kingdom, Humans, Male, Middle Aged, Neuropsychological Tests, Reproducibility of Results, Databases as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.