Empirical examination of the replicability of associations between brain structure and psychological variables
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Linking interindividual differences in psychological phenotype to variations in brain structure is an old dream for psychology and a crucial question for cognitive neurosciences. Yet, replicability of the previously-reported ‘structural brain behavior’ (SBB)-associations has been questioned, recently. Here, we conducted an empirical investigation, assessing replicability of SBB among heathy adults. For a wide range of psychological measures, the replicability of associations with gray matter volume was assessed. Our results revealed that among healthy individuals 1) finding an association between performance at standard psychological tests and brain morphology is relatively unlikely 2) significant associations, found using an exploratory approach, have overestimated effect sizes and 3) can hardly be replicated in an independent sample. After considering factors such as sample size and comparing our findings with more replicable SBB-associations in a clinical cohort and replicable associations between brain structure and non-psychological phenotype, we discuss the potential causes and consequences of these findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
A fast diffeomorphic image registration algorithm
Ashburner, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.007 · Cited by 8,122
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 5×
“Finally, in line with the replication literature in psychological sciences and neurosciences (Button et al., 2013; Poldrack et al., 2017; Turner et al., 2018), we devoted our last section to sample size and power issues in SBB studies in healthy adults and proposed some recommendations.”
InfluentialAshburner, J., & Friston, K. J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.02.018 · Cited by 7,507
LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS
Scoville, W. B., & Milner, B. (1957) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.20.1.11 · Cited by 7,322
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
SMITH, S., & NICHOLS, T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.061 · Cited by 5,250
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Testing the efficiency and independence of attentional networks
Fan, J., McCandliss, B. D., Sommer, T., Raz, A., & Posner, M. I. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317361886 · Cited by 3,717 · cited 1×
“The testing included: the attention network task (ANT) probing attention sub-functions (Fan et al., 2002), the Delis-Kaplan testing battery assessing different aspects of executive functions (Delis et al., 2001) (including trail-making test, color-word interference task, verbal fluency, 20…”
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487 · cited 1×
“Directly addressing this limitation, several strategies for multiple comparison correction have been proposed to control the rate of false positives (Eklund et al., 2016).”
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Delis-Kaplan Executive Function System
Delis, D. C., Kaplan, E., & Kramer, J. H. (2012) · PsycTESTS Dataset · doi:10.1037/t15082-000 · Cited by 2,683
Changes in grey matter induced by training
Draganski, B., Gaser, C., Busch, V., Schuierer, G., Bogdahn, U., & May, A. (2004) · Nature · doi:10.1038/427311a · Cited by 2,373
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Multimodal population brain imaging in the UK Biobank prospective epidemiological study
Miller, K. L., Alfaro-Almagro, F., Bangerter, N. K., Thomas, D. L., Yacoub, E., Xu, J., Bartsch, A. J., Jbabdi, S., Sotiropoulos, S. N., Andersson, J. L. R., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4393 · Cited by 2,020
Thinning of the Cerebral Cortex in Aging
Salat, D. H. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh032 · Cited by 1,851
Stress Effects on Neuronal Structure: Hippocampus, Amygdala, and Prefrontal Cortex
McEwen, B. S., Nasca, C., & Gray, J. D. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2015.171 · Cited by 1,574
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410
Pipkins, H. R., Bradshaw, J. L., Keller, L. E., Swiatlo, E., & McDaniel, L. S. (2017) · PLOS ONE · doi:10.1371/journal.pone.0179159 · Cited by 1,338
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127
The structural basis of inter-individual differences in human behaviour and cognition
Kanai, R., & Rees, G. (2011) · Nature reviews. Neuroscience · doi:10.1038/nrn3000 · Cited by 1,055 · cited 2×
“…significant SBB associations in healthy individuals: Importance of reporting null findings According to the scientific literature, associations between psychological phenotype (cognitive performance and psychological trait) and local brain structure are not uncommon (Kanai and Rees, 2011).”
Measurement error and the replication crisis
Loken, E., & Gelman, A. (2017) · Science · doi:10.1126/science.aal3618 · Cited by 628 · cited 1×
“Correlations in small samples in the presence of noise for both type of variables is 434 likely to produce spurious significant results (Loken and Gelman, 2017) by fitting a 435 correlation or regression between random noise in both variables.”
Stress effects on the hippocampus: a critical review
Kim, E. J., Pellman, B., & Kim, J. J. (2015) · Learning & Memory · doi:10.1101/lm.037291.114 · Cited by 620 · cited 1×
“research (Ismaylova et al., 2018; Kim et al., 2015; Luders et al., 2013, 2012; McEwen et al., 78 2016), but also possibly hold an important key for our understanding of brain functions 79 (Genon et al.”
Yuan, P., & Raz, N. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.02.005 · Cited by 608
Understanding brain networks and brain organization
Pessoa, L. (2014) · Physics of Life Reviews · doi:10.1016/j.plrev.2014.03.005 · Cited by 464 · cited 2×
“Following this philosophical segregation between psychological sciences and neurosciences, it is now widely acknowledged that there is no one-to-one mapping between behavioral functions and brain regions (Anderson, 2016; Genon et al., 2018; Pessoa, 2014).”
Multivariate data analysis for neuroimaging data: overview and application to Alzheimer’s disease
Initiative, T. A. D. N., Habeck, C., & Stern, Y. (2010) · Cell Biochemistry and Biophysics · doi:10.1007/s12013-010-9093-0 · Cited by 127
Cited by
Reproducible brain-wide association studies require thousands of individuals
Marek, S., Tervo-Clemmens, B., Calabro, F. J., Montez, D. F., Kay, B. P., Hatoum, A. S., Donohue, M. R., Foran, W., Miller, R. L., Hendrickson, T. J., et al. (2022) · Nature · doi:10.1038/s41586-022-04492-9 · Cited by 1,928
Practical recommendations to conduct a neuroimaging meta-analysis for neuropsychiatric disorders
Tahmasian, M., Sepehry, A. A., Samea, F., Khodadadifar, T., Soltaninejad, Z., Javaheripour, N., Khazaie, H., Zarei, M., Eickhoff, S. B., & Eickhoff, C. R. (2019) · Human Brain Mapping · doi:10.1002/hbm.24746 · Cited by 126
Self-citationPushing the Boundaries of Psychiatric Neuroimaging to Ground Diagnosis in Biology
Saggar, M., & Uddin, L. Q. (2019) · eneuro · doi:10.1523/eneuro.0384-19.2019 · Cited by 38
Metadata
- Authors
- Kharabian Masouleh, S., Eickhoff, S. B., Hoffstaedter, F., Genon, S., & Initiative, A. D. N.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2019-03-13
- Volume
- 8
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2949262129
- Cited by
- 230
- References
- 28
- FWCI
- 27.98
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.