False positives in neuroimaging genetics using voxel-based morphometry data
Open accessAbstract
Voxel-wise statistical inference is commonly used to identify significant experimental effects or group differences in both functional and structural studies of the living brain. Tests based on the size of spatially extended clusters of contiguous suprathreshold voxels are also widely used due to their typically increased statistical power. In "imaging genetics", such tests are used to identify regions of the brain that are associated with genetic variation. However, concerns have been raised about the adequate control of rejection rates in studies of this type. A previous study tested the effect of a set of 'null' SNPs on brain structure and function, and found that false positive rates were well-controlled. However, no similar analysis of false positive rates in an imaging genetic study using cluster size inference has yet been undertaken. We measured false positive rates in an investigation of the effect of 700 pre-selected null SNPs on grey matter volume using voxel-based morphometry (VBM). As VBM data exhibit spatially-varying smoothness, we used both non-stationary and stationary cluster size tests in our analysis. Image and genotype data on 181 subjects with mild cognitive impairment were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI). At a nominal significance level of 5%, false positive rates were found to be well-controlled (3.9-5.6%), using a relatively high cluster-forming threshold, α(c)=0.001, on images smoothed with a 12 mm Gaussian kernel. Tests were however anticonservative at lower cluster-forming thresholds (α(c)=0.01, 0.05), and for images smoothed using a 6mm Gaussian kernel. Here false positive rates ranged from 9.8 to 67.6%. In a further analysis, false positive rates using simulated data were observed to be well-controlled across a wide range of conditions. While motivated by imaging genetics, our findings apply to any VBM study, and suggest that parametric cluster size inference should only be used with high cluster-forming thresholds and smoothness. We would advocate the use of nonparametric methods in other cases.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In whole-brain analysis, reporting was limited to clusters that passed both the nonparametric and parametric procedures.
“For whole-brain analysis, only clusters that survived following both the nonparametric and parametric statistical procedures are reported (61).”
From Epigenetic modification of <i>OXT</i> and human sociability (2016) · cites this paper as [61]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688 · see citing works
Voxel-Based Morphometry—The Methods
Ashburner, J., & Friston, K. J. (2000) · NeuroImage · doi:10.1006/nimg.2000.0582 · Cited by 8,676 · see citing works
A fast diffeomorphic image registration algorithm
Ashburner, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.007 · Cited by 8,122 · see citing works
The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods
Jack, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P. J., L. Whitwell, J., Ward, C., et al. (2008) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21049 · Cited by 4,347 · see citing works
Frackowiak, R. S. J. (1997) · Cited by 1,713 · see citing works
Intermediate phenotypes and genetic mechanisms of psychiatric disorders
Meyer-Lindenberg, A., & Weinberger, D. R. (2006) · Nature reviews. Neuroscience · doi:10.1038/nrn1993 · Cited by 1,216 · see citing works
Controlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T. E., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,210 · see citing works
Self-citationDetecting activations in PET and fMRI: levels of inference and power
Friston, K. J., Holmes, A., Poline, J.-B., Price, C. J., & Frith, C. D. (1996) · NeuroImage · doi:10.1006/nimg.1996.0074 · Cited by 1,148 · see citing works
A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data
Calhoun, V. D., Liu, J., & Adalı, T. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.057 · Cited by 1,148 · see citing works
Combining spatial extent and peak intensity to test for activations in functional imaging
Poline, J.-B., Worsley, K. J., Evans, A. C., & Friston, K. J. (1997) · NeuroImage · doi:10.1006/nimg.1996.0248 · Cited by 670 · see citing works
Nonstationary cluster-size inference with random field and permutation methods
Hayasaka, S., Phan, K. L., Liberzon, I., Worsley, K. J., & Nichols, T. E. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.01.041 · Cited by 661 · see citing works
Self-citationValidating cluster size inference: random field and permutation methods
Hayasaka, S. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.08.003 · Cited by 491 · see citing works
False discovery rate revisited: FDR and topological inference using Gaussian random fields
Chumbley, J. R., & Friston, K. J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.05.021 · Cited by 458 · see citing works
Shen, L., Kim, S., Risacher, S. L., Nho, K. T., Swaminathan, S., West, J. D., Foroud, T., Pankratz, N., Moore, J. H., Sloan, C., et al. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.042 · Cited by 379 · see citing works
Hariri, A. R., Drabant, E. M., & Weinberger, D. R. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.11.005 · Cited by 336 · see citing works
Bergouignan, L., Marie Chupin, A., Czechowska, Y., Kinkingnéhun, S., Lemogne, C., Bastard, G. L., Lepage, M., Garnero, L., Olivier Colliot, A., & Fossati, P. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.11.006 · Cited by 265 · see citing works
Neuroimaging endophenotypes: Strategies for finding genes influencing brain structure and function
Glahn, D. C., Thompson, P. M., & Blangero, J. (2007) · Human Brain Mapping · doi:10.1002/hbm.20401 · Cited by 259 · see citing works
Distributional assumptions in voxel-based morphometry
Salmond, C. H., Ashburner, J., Vargha-Khadem, F., Connelly, A., Gadian, D. G., & Friston, K. J. (2002) · NeuroImage · doi:10.1006/nimg.2002.1153 · Cited by 256 · see citing works
A Genome-Wide Association Study of Schizophrenia Using Brain Activation as a Quantitative Phenotype
Potkin, S. G., Turner, J. A., Guffanti, G., Lakatos, A., Fallon, J. H., Nguyen, D. D., Mathalon, D. H., Ford, J. M., Lauriello, J., & Macciardi, F. M. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn155 · Cited by 219 · see citing works
Schwartz, D. L., Mitchell, A. D., Lahna, D. L., Luber, H. S., Huckans, M. S., Mitchell, S. H., & Hoffman, W. F. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.056 · Cited by 171 · see citing works
Filippini, N., Rao, A., Wetten, S., Gibson, R. A., Borrie, M., Guzman, D., Kertesz, A., Loy-English, I., Williams, J., Nichols, T., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.003 · Cited by 143 · see citing works
Self-citationFalse positives in imaging genetics
Meyer‐Lindenberg, A., Nicodemus, K. K., Egan, M. F., Callicott, J. H., Mattay, V. S., & Weinberger, D. R. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.11.058 · Cited by 117 · see citing works
Neuroanatomical correlates of cognitive self-appraisal in neurodegenerative disease
Rosen, H. J., Alcantar, O., ROTHLIND, J. C., Sturm, V. E., Kramer, J. H., Weiner, M. W., & Miller, B. L. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.041 · Cited by 111 · see citing works
Joyner, A. H., Cooper Roddey, J., Bloss, C. S., Bakken, T. E., Rimol, L. M., Melle, I. S., Agartz, I., Djurovic, S., Topol, E. J., Schork, N. J., et al. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0901866106 · Cited by 111 · see citing works
Potkin, S. G., Turner, J. A., Fallon, J. A., Lakatos, A., Keator, D. B., Guffanti, G., Macciardi, F., & FBIRN (2009) · Molecular Psychiatry · doi:10.1038/mp.2008.127 · Cited by 97 · see citing works
Random fields of multivariate test statistics, with applications to shape analysis
Taylor, J., & Worsley, K. J. (2008) · The Annals of Statistics · doi:10.1214/009053607000000406 · Cited by 83 · see citing works
Anomalous neural circuit function in schizophrenia during a virtual Morris water task
Folley, B. S., Astur, R. S., Jagannathan, K., Calhoun, V. D., & Pearlson, G. D. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.034 · Cited by 72 · see citing works
Mapping the effect of APOE ε4 on gray matter loss in Alzheimer's disease in vivo
Pievani, M., Rasser, P. E., Galluzzi, S., Benussi, L., Ghidoni, R., Sabattoli, F., Bonetti, M., Binetti, G. G., Thompson, P. M., & Frisoni, G. B. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.01.009 · Cited by 71 · see citing works
Roffman, J. L., Weiss, A. P., GOFF, D. C., Rauch, S. L., & Weinberger, D. R. (2006) · Harvard Review of Psychiatry · doi:10.1080/10673220600642945 · Cited by 53 · see citing works
Moorhead, T. W. J., Job, D. E., Spencer, M. D., Whalley, H. C., Johnstone, E. C., & Lawrie, S. M. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.04.045 · Cited by 47 · see citing works
Non-normality and transformations of random fields, with an application to voxel-based morphometry
Viviani, R., Beschoner, P., Ehrhard, K., Schmitz, B., & Thöne, J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.11.037 · Cited by 30 · see citing works
Correlation-based multivariate analysis of genetic influence on brain volume
Hardoon, D. R., Ettinger, U., Mourão-Miranda, J., et al. (2008) · Neuroscience Letters · doi:10.1016/j.neulet.2008.11.035 · Cited by 26 · see citing works
Investigating association of brain volumes with intracranial capacity in schizophrenia
Ueda, K., Fujiwara, H., Miyata, J., Hirao, K., Saze, T., Kawada, R., Fujimoto, S., Tanaka, Y. Z., Sawamoto, N., Fukuyama, H., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.09.006 · Cited by 18 · see citing works
Worsley (1996) · doi:10.1002/(sici)1097-0193(1996)4:1
Xiong (1999) · doi:10.1002/(sici)1097-0193(1999)8:2/3
Flagged first, then by citations
Research works printed but not linked: 1
- Worsley K.J. Marrett S. Neelin P. Evans A.C. Searching scale space for activation in PET images Hum. Brain Mapp. 4 1 1996 74 90 20408187
Cited by
Sorted by citations
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487 · see citing works
Self-citationCluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations
Woo, C.-W., Krishnan, A., & Wager, T. D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.12.058 · Cited by 1,253 · see citing works
Weiner, M. W., Veitch, D. P., Aisen, P. S., Beckett, L. A., Cairns, N. J., Cedarbaum, J., Green, R. C., Harvey, D., Jack, C. R., Jagust, W., et al. (2015) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2014.11.001 · Cited by 348 · see citing works
The impact of television viewing on brain structures: Cross-sectional and longitudinal analyses
Takeuchi, H., Taki, Y., Hashizume, H., Asano, K., Asano, M., Sassa, Y., Yokota, S., Kotozaki, Y., Nouchi, R., & Kawashima, R. (2015) · Cerebral Cortex · doi:10.1093/cercor/bht315 · Cited by 140 · see citing works
Epigenetic modification of <i>OXT</i> and human sociability
Haas, B. W., Filkowski, M. M., Cochran, R. N., Denison, L., Ishak, A., Nishitani, S., & Smith, A. K. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602809113 · Cited by 91 · see citing works
Jha, S. C., Meltzer-Brody, S., Steiner, R. J., Cornea, E., Woolson, S., Ahn, M., Verde, A. R., Hamer, R. M., Zhu, H., Styner, M., et al. (2016) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2016.05.004 · Cited by 66 · see citing works
Patterns of Co-Occurring Gray Matter Concentration Loss across the Huntington Disease Prodrome
Ciarochi, J. A., Calhoun, V. D., Lourens, S., Long, J. D., Johnson, H. J., Bockholt, H. J., Liu, J., Plis, S. M., Paulsen, J. S., & Turner, J. A. (2016) · Frontiers in Neurology · doi:10.3389/fneur.2016.00147 · Cited by 31 · see citing works
- Authors
- Silver, M., Montana, G., & Nichols, T. E.
- Affiliations
- Imperial College London (GB)
- University of Warwick (GB)
- Corresponding author
- Nichols, Thomas E.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2011-01
- Volume
- 54
- Issue
- 2
- Pages
- 992-1000
- ISSN
- 1053-8119 (ISSN-L)
- PMC
- PMC3063336
- OpenAlex
- W2104524472
- Cited by
- 146
- References
- 35
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 2 · 2023: 4 · 2022: 5 · 2021: 6 · 2020: 9 · 2019: 17 · 2018: 8 · 2017: 15 · 2016: 15 · 2015: 19 · 2014: 15 · 2013: 10 · 2012: 7
- FWCI
- 4.26
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- NeuroImage (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- Funders
- Wellcome Trust
- National Institutes of Health (U01 AG024904, P30 AG010129, K01 AG030514)
- National Institute on Aging
- National Institute of Biomedical Imaging and Bioengineering
- Dana Foundation
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Keywords
- false positive rate, imaging genetics, voxel-based morphometry, gray matter volume, mild cognitive impairment, Alzheimer's disease, spatial smoothing, nonparametric statistics
- MeSH terms
- Brain, Cognition Disorders, False Positive Reactions, Genotype, Humans, Magnetic Resonance Imaging, Polymorphism, Single Nucleotide
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
