CoChat
Sign Up
Cited by 6,256Functional Brain Connectivity StudiesAdvanced MRI Techniques and ApplicationsMedical Imaging Techniques and Applications

Nonparametric permutation tests for functional neuroimaging: A primer with examples

Open access
Nichols, Thomas E., Holmes, Andrew P.·2002·Human Brain Mapping
DOI 10.1002/hbm.1058PMC PMC6871862
Open this paper in CoChat — free accountOpen PDF

Abstract

Requiring only minimal assumptions for validity, nonparametric permutation testing provides a flexible and intuitive methodology for the statistical analysis of data from functional neuroimaging experiments, at some computational expense. Introduced into the functional neuroimaging literature by Holmes et al. ([ 1996 ]: J Cereb Blood Flow Metab 16:7–22), the permutation approach readily accounts for the multiple comparisons problem implicit in the standard voxel‐by‐voxel hypothesis testing framework. When the appropriate assumptions hold, the nonparametric permutation approach gives results similar to those obtained from a comparable Statistical Parametric Mapping approach using a general linear model with multiple comparisons corrections derived from random field theory. For analyses with low degrees of freedom, such as single subject PET/SPECT experiments or multi‐subject PET/SPECT or f MRI designs assessed for population effects, the nonparametric approach employing a locally pooled (smoothed) variance estimate can outperform the comparable Statistical Parametric Mapping approach. Thus, these nonparametric techniques can be used to verify the validity of less computationally expensive parametric approaches. Although the theory and relative advantages of permutation approaches have been discussed by various authors, there has been no accessible explication of the method, and no freely distributed software implementing it. Consequently, there have been few practical applications of the technique. This article, and the accompanying MATLAB software, attempts to address these issues. The standard nonparametric randomization and permutation testing ideas are developed at an accessible level, using practical examples from functional neuroimaging, and the extensions for multiple comparisons described. Three worked examples from PET and f MRI are presented, with discussion, and comparisons with standard parametric approaches made where appropriate. Practical considerations are given throughout, and relevant statistical concepts are expounded in appendices. Hum. Brain Mapping 15:1–25, 2001. © 2001 Wiley‐Liss, Inc.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/hbm.1058

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Statistical significance of decoding performance was assessed via permutation tests with 1,000 permutations, corrected for multiple comparisons using cluster-based correction.

    “Statistical significance of decoding performance was assessed via permutation test, randomly permuting the sample labels and repeating the decoding analysis 1,000 times, corrected for multiple comparisons using a cluster-based correction (cluster mass inference with cluster forming threshold at P < 0.05)105,106.”

    From Adversarial testing of global neuronal workspace and integrated information theories of consciousness (2025) · cites this paper as [105]

    The Relabeling enumeration section describes a test using 1,000 labelings, and Suprathreshold cluster tests explains permutation-based correction for multiple comparisons. It mentions cluster mass as a possible variation, but its worked cluster analysis uses cluster size; our copy does not establish a P < 0.05 cluster-forming threshold or describe the citing study’s decoding analysis. No page numbers were provided for these passages. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The permutation p-value is super-uniform under H0, giving finite-sample Type-I control for calibrated scores.

    “The permutation p-value in Equation ( 10) is super-uniform under H 0 (i.e., P H0 (p ≤ α) ≤ α for all α ∈ [0, 1]), a standard consequence of randomization inference (Nichols & Holmes, 2002;Phipson & Smyth, 2010;Good, 2005)”

    From Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2026) · cites this paper as Nichols & Holmes, 2002

  • Resampling-based maxT procedures can calibrate such aggregates, but this has not yet been applied in representational similarity studies.

    “While resampling-based “maxT” procedures (Westfall & Young, 1993;Nichols & Holmes, 2002) can calibrate such aggregates, this has not yet been applied in representational similarity studies.”

    From Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2026) · cites this paper as Nichols & Holmes, 2002

  • Monte Carlo simulations permuting the sign of whole-brain maps were performed to find the true type I error rate.

    “To find the true type I error rate, we performed Monte Carlo simulations that permute the sign of the whole-brain maps from individual subjects58.”

    From Anchoring the neural compass: coding of local spatial reference frames in human medial parietal lobe (2014) · cites this paper as [58]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 56 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Holmes AP (1994): Statistical issues in functional brain mapping, PhD thesis. University of Glasgow. http://www.fil.ion.ucl.ac. uk/spm/papers/APH_thesis

Cited by

Sorted by citations

Related

Authors
Nichols, T. E., & Holmes, A. P.
Affiliations
  • University of Michigan (US)
  • UCL Queen Square Institute of Neurology (GB)
ORCID
Nichols, Thomas E.
Venue
Human Brain Mapping
Publisher
Wiley
Published
2001-10-15
Volume
15
Issue
1
Pages
1-25
ISSN
  • 1065-9471 (ISSN-L)
  • 1097-0193
DOI
10.1002/hbm.1058
PMC
PMC6871862
OpenAlex
W2138790588
Cited by
6,256
References
56
Citations by year
2026: 223 · 2025: 264 · 2024: 255 · 2023: 275 · 2022: 321 · 2021: 350 · 2020: 392 · 2019: 368 · 2018: 381 · 2017: 368 · 2016: 343 · 2015: 385 · 2014: 365 · 2013: 356 · 2012: 314
FWCI
16.77
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • Human Brain Mapping (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Deep Blue (University of Michigan) (submittedVersion)
Funders
Wellcome Trust
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
Keywords
permutation tests, functional neuroimaging, multiple testing correction, statistical parametric mapping, random field theory, PET, functional MRI, population effects, MATLAB software
MeSH terms
Brain, Brain Mapping, Data Interpretation, Statistical, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Signal Processing, Computer-Assisted, Tomography, Emission-Computed, Statistical Distributions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review