The DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Open accessAbstract
Requiring only minimal assumptions for validity, nonparametric permutation testing provides a flexible and intuitive methodology for the statistical analysis of data from functional neuroimaging experiments, at some computational expense. Introduced into the functional neuroimaging literature by Holmes et al. ([ 1996 ]: J Cereb Blood Flow Metab 16:7–22), the permutation approach readily accounts for the multiple comparisons problem implicit in the standard voxel‐by‐voxel hypothesis testing framework. When the appropriate assumptions hold, the nonparametric permutation approach gives results similar to those obtained from a comparable Statistical Parametric Mapping approach using a general linear model with multiple comparisons corrections derived from random field theory. For analyses with low degrees of freedom, such as single subject PET/SPECT experiments or multi‐subject PET/SPECT or f MRI designs assessed for population effects, the nonparametric approach employing a locally pooled (smoothed) variance estimate can outperform the comparable Statistical Parametric Mapping approach. Thus, these nonparametric techniques can be used to verify the validity of less computationally expensive parametric approaches. Although the theory and relative advantages of permutation approaches have been discussed by various authors, there has been no accessible explication of the method, and no freely distributed software implementing it. Consequently, there have been few practical applications of the technique. This article, and the accompanying MATLAB software, attempts to address these issues. The standard nonparametric randomization and permutation testing ideas are developed at an accessible level, using practical examples from functional neuroimaging, and the extensions for multiple comparisons described. Three worked examples from PET and f MRI are presented, with discussion, and comparisons with standard parametric approaches made where appropriate. Practical considerations are given throughout, and relevant statistical concepts are expounded in appendices. Hum. Brain Mapping 15:1–25, 2001. © 2001 Wiley‐Liss, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Statistical significance of decoding performance was assessed via permutation test, randomly permuting the sample labels and repeating the decoding analysis 1,000 times, corrected for multiple comparisons using a cluster-based correction (cluster mass inference with cluster forming threshold at P < 0.05)105,106.”
From Adversarial testing of global neuronal workspace and integrated information theories of consciousness (2025) · cites this paper as [105]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The permutation p-value in Equation ( 10) is super-uniform under H 0 (i.e., P H0 (p ≤ α) ≤ α for all α ∈ [0, 1]), a standard consequence of randomization inference (Nichols & Holmes, 2002;Phipson & Smyth, 2010;Good, 2005)”
From Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2026) · cites this paper as Nichols & Holmes, 2002
The cited source says what the claim attributes to it.
In the citing paper
“While resampling-based “maxT” procedures (Westfall & Young, 1993;Nichols & Holmes, 2002) can calibrate such aggregates, this has not yet been applied in representational similarity studies.”
From Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2026) · cites this paper as Nichols & Holmes, 2002
In the citing paper
“To find the true type I error rate, we performed Monte Carlo simulations that permute the sign of the whole-brain maps from individual subjects58.”
From Anchoring the neural compass: coding of local spatial reference frames in human medial parietal lobe (2014) · cites this paper as [58]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688
Self-citationKendall, M. G. (1957) · Biometrika · doi:10.2307/2333282 · Cited by 6,442
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202
Analysis of fMRI Time-Series Revisited
Friston, K. J., Holmes, A. P., Poline, J., Grasby, P. M., Williams, S., Frackowiak, R. S. J., & Turner, R. (1995) · NeuroImage · doi:10.1006/nimg.1995.1007 · Cited by 2,504
Self-citationBevan, J. M., & Kendall, M. G. (1971) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2986801 · Cited by 2,458
Multiple Comparison Procedures
Hochberg, Y., & Tamhane, A. C. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316672 · Cited by 2,389
Analysis of fMRI Time-Series Revisited—Again
Worsley, K. J., & Friston, K. J. (1995) · NeuroImage · doi:10.1006/nimg.1995.1023 · Cited by 2,198
Multiple Comparison Procedures
Hahs-Vaughn, D., & Lomax, R. (2013) · Routledge eBooks · doi:10.4324/9780203137819-17 · Cited by 2,132
Edgington, E. S., Edgington, E., & Onghena, P. (1964) · Springer eBooks · doi:10.1002/0471667196.ess2169 · Cited by 2,052
Flagged first, then by citations
1 of the 56 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- 10.1080/00223980.1964.9916711 · DOI
- Fisher RA, 1990, Bennett JH
- HolmesAP(1994): Statistical issues in functional brain mapping PhD thesis. University of Glasgow.http://www.fil.ion.ucl.ac.uk/spm/papers/APH_thesis
- Pitman EJG, 1937, Significance tests which may be applied to samples from any population. II. The correlation coefficient test, J R Stat Soc, 4, 224
- Worsley KJ, 1995, A unified statistical approach for determining significant signals in images of cerebral activation, Hum Brain Mapp, 4, 58, 10.1002/(SICI)1097-0193(1996)4:1<58::AID-HBM4>3.0.CO;2-O · DOI
Cited by
Sorted by citations
Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W., & Smith, S. M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.015 · Cited by 11,314
Nonparametric statistical testing of EEG- and MEG-data
Maris, E., & Oostenveld, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.03.024 · Cited by 8,496
Tract-based spatial statistics: Voxelwise analysis of multi-subject diffusion data
Smith, S. M., Jenkinson, M., Johansen‐Berg, H., Rueckert, D., Nichols, T. E., Mackay, C. E., Watkins, K. E., Ciccarelli, O., Cader, M. Z., Matthews, P. M., et al. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.02.024 · Cited by 6,598
Self-citationMEG and EEG data analysis with MNE-Python
Gramfort, A. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00267 · Cited by 3,882
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
Self-citation
Metadata
- Authors
- Nichols, T. E., & Holmes, A. P.
- Affiliations
- University of Michigan (US)
- UCL Queen Square Institute of Neurology (GB)
- ORCID
- Nichols, Thomas E.
- Venue
- Human Brain Mapping
- Publisher
- Wiley
- Published
- 2001-10-15
- Volume
- 15
- Issue
- 1
- Pages
- 1-25
- ISSN
- 1065-9471 (ISSN-L)
- 1097-0193
- PMC
- PMC6871862
- OpenAlex
- W2138790588
- Cited by
- 6,256
- References
- 56
- Citations by year
- 2026: 223 · 2025: 264 · 2024: 255 · 2023: 275 · 2022: 321 · 2021: 350 · 2020: 392 · 2019: 368 · 2018: 381 · 2017: 368 · 2016: 343 · 2015: 385 · 2014: 365 · 2013: 356 · 2012: 314
- FWCI
- 16.77
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Human Brain Mapping (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Funders
- Wellcome Trust
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
- Keywords
- permutation tests, functional neuroimaging, multiple testing correction, statistical parametric mapping, random field theory, PET, functional MRI, population effects, MATLAB software
- MeSH terms
- Brain, Brain Mapping, Data Interpretation, Statistical, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Signal Processing, Computer-Assisted, Tomography, Emission-Computed, Statistical Distributions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
