The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations
Open accessAbstract
Cluster-extent based thresholding is currently the most popular method for multiple comparisons correction of statistical maps in neuroimaging studies, due to its high sensitivity to weak and diffuse signals. However, cluster-extent based thresholding provides low spatial specificity; researchers can only infer that there is signal somewhere within a significant cluster and cannot make inferences about the statistical significance of specific locations within the cluster. This poses a particular problem when one uses a liberal cluster-defining primary threshold (i.e., higher p-values), which often produces large clusters spanning multiple anatomical regions. In such cases, it is impossible to reliably infer which anatomical regions show true effects. From a survey of 814 functional magnetic resonance imaging (fMRI) studies published in 2010 and 2011, we show that the use of liberal primary thresholds (e.g., p<.01) is endemic, and that the largest determinant of the primary threshold level is the default option in the software used. We illustrate the problems with liberal primary thresholds using an fMRI dataset from our laboratory (N=33), and present simulations demonstrating the detrimental effects of liberal primary thresholds on false positives, localization, and interpretation of fMRI findings. To avoid these pitfalls, we recommend several analysis and reporting procedures, including 1) setting primary p<.001 as a default lower limit; 2) using more stringent primary thresholds or voxel-wise correction methods for highly powered studies; and 3) adopting reporting practices that make the level of spatial precision transparent to readers. We also suggest alternative and supplementary analysis methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“As the latter criterion has been recommended to avoid false positives (Woo et al., 2014), the results reported later are all significant at P < 0.05 (corrected) based on a voxel-wise error of P < 0.001, unless otherwise stated.”
From The brain's conversation with itself: neural substrates of dialogic inner speech (2016) · cites this paper as Woo et al., 2014
In the citing paper
“Next, we performed a regression analysis with OXT methylation values entered as the predictor variable and contrast estimates (emotion perspectivetaking > shape match) as the criterion variable using a threshold of P < 0.005, voxel extent of k = 60, which is sufficient to preserve the balance between sensitivity and false-positive rates (55,56).”
From Epigenetic modification of OXT and human sociability (2016) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, R. P., Hyman, B. T., et al. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.021 · Cited by 13,324
Applied Multivariate Statistical Analysis.
Johnson, A., & Wichern, D. W. (1988) · Biometrics · doi:10.2307/2531616 · Cited by 11,374
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Nichols, T. E., & Holmes, A. P. (2002) · Human Brain Mapping · doi:10.1002/hbm.1058 · Cited by 6,256
SMITH, S., & NICHOLS, T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.061 · Cited by 5,250
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202
Applied Multivariate Statistical Analysis
Härdle, W. K., & Simar, L. (2007) · Springer eBooks · doi:10.1007/978-3-540-72244-1 · Cited by 2,103
A three-dimensional statistical analysis for CBF activation studies in human brain
Worsley, K. J., Evans, A. C., Marrett, S., & Neelin, P. (1992) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1992.127 · Cited by 2,036
Assessing the significance of focal activations using their spatial extent
Friston, K. J., Worsley, K. J., Frackowiak, R. S. J., Mazziotta, J. C., & Evans, A. C. (1994) · Human Brain Mapping · doi:10.1002/hbm.460010306 · Cited by 1,907
A whole brain fMRI atlas generated via spatially constrained spectral clustering
Craddock, R. C., James, G. A., Holtzheimer, P. E., Hu, X. P., & Mayberg, H. S. (2012) · Human Brain Mapping · doi:10.1002/hbm.21333 · Cited by 1,636
An fMRI-Based Neurologic Signature of Physical Pain
Wager, T. D., Atlas, L. Y., Lindquist, M. A., Roy, M., Woo, C.-W., & Kross, E. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1204471 · Cited by 1,597
Self-citationControlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T. E., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,249
Wager, T. D., Phan, K. L., Liberzon, I., & Taylor, S. F. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00078-8 · Cited by 1,150
Self-citationNonstationary cluster-size inference with random field and permutation methods
Hayasaka, S., Phan, K. L., Liberzon, I., Worsley, K. J., & Nichols, T. E. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.01.041 · Cited by 661
Neuroimaging studies of shifting attention: a meta-analysis
Wager, T. D., Jonides, J., & Reading, S. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.03.052 · Cited by 654
Self-citationValidating cluster size inference: random field and permutation methods
Hayasaka, S. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.08.003 · Cited by 491
The principled control of false positives in neuroimaging
Bennett, C. M., Wolford, G. L., & Miller, M. B. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp053 · Cited by 486
The secret lives of experiments: Methods reporting in the fMRI literature
Carp, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.004 · Cited by 463
Meta-analysis of functional neuroimaging data: current and future directions
Wager, T. D., Lindquist, M., & Kaplan, L. (2007) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsm015 · Cited by 434
Self-citationWager, T. D., van Ast, V. A., Hughes, B. L., Davidson, M. L., Lindquist, M. A., & Ochsner, K. N. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.05.044 · Cited by 264
Self-citationFalse Discovery Rates for Spatial Signals
Benjamini, Y., & Heller, R. (2007) · Journal of the American Statistical Association · doi:10.1198/016214507000000941 · Cited by 224
Multiple testing corrections, nonparametric methods, and random field theory
Nichols, T. E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.014 · Cited by 205
Cluster-based analysis of FMRI data
Heller, R., Stanley, D. A., Yekutieli, D., Rubin, N., & Benjamini, Y. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.04.233 · Cited by 202
(2000) · Elsevier eBooks · doi:10.1016/b978-0-12-481460-8.x5000-8 · Cited by 158
False positives in neuroimaging genetics using voxel-based morphometry data
Silver, M., Montana, G., & Nichols, T. E. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.08.049 · Cited by 146
Independent component analysis of fMRI data in the complex domain
Calhoun, V. D., Adalı, T., Pearlson, G. D., van Zijl, P. C. M., & Pekar, J. J. (2002) · Magnetic Resonance in Medicine · doi:10.1002/mrm.10202 · Cited by 146
Meta Analysis of Functional Neuroimaging Data via Bayesian Spatial Point Processes
Kang, J., Johnson, T., Nichols, T. E., & Wager, T. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.ap09735 · Cited by 68
Self-citationExperimental Design and Statistical Issues
Friston, K. J. (2000) · Elsevier eBooks · doi:10.1016/b978-012481460-8/50005-6 · Cited by 27
Cited by
Sorted by citations
DPABI: Data Processing & Analysis for (Resting-State) Brain Imaging
Yan, C., Wang, X., Zuo, X., & Zang, Y. (2016) · Neuroinformatics · doi:10.1007/s12021-016-9299-4 · Cited by 3,935
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
Ten simple rules for neuroimaging meta-analysis
Müller, V. I., Cieslik, E. C., Laird, A. R., Fox, P. T., Radua, J., Mataix-Cols, D., Tench, C. R., Yarkoni, T., Nichols, T. E., Turkeltaub, P. E., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.11.012 · Cited by 805
Self-citationEickhoff, S. B., Nichols, T. E., Laird, A. R., Hoffstaedter, F., Amunts, K., Fox, P. T., Bzdok, D., & Eickhoff, C. R. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.04.072 · Cited by 683
“Searches for studies investigating major brain disorders such as schizophrenia, depression, dementia, stroke or Parkinson’s disease likewise revealed many hundred clinical neuroimaging papers.”
Measuring shared responses across subjects using intersubject correlation
Nastase, S. A., Gazzola, V., Hasson, U., & Keysers, C. (2019) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsz037 · Cited by 419
Metadata
- Authors
- Woo, C.-W., Krishnan, A., & Wager, T. D.
- Affiliations
- University of Colorado Boulder (US)
- Corresponding author
- Wager, Tor D.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2014-05
- Volume
- 91
- Pages
- 412-419
- ISSN
- 1053-8119 (ISSN-L)
- PMC
- PMC4214144
- OpenAlex
- W2169463832
- Cited by
- 1,253
- References
- 31
- Citations by year
- 2026: 77 · 2025: 79 · 2024: 81 · 2023: 81 · 2022: 96 · 2021: 110 · 2020: 138 · 2019: 112 · 2018: 158 · 2017: 135 · 2016: 93 · 2015: 77 · 2014: 16
- FWCI
- 59.70
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH076136)
- National Institute on Drug Abuse (R01DA035484)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Keywords
- multiple testing correction, spatial specificity, false positives, statistical maps
- MeSH terms
- Computer Simulation, Data Interpretation, Statistical, False Positive Reactions, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Software, Cluster Analysis, Normal Distribution, Signal-To-Noise Ratio, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
