Cited by 157Functional Brain Connectivity StudiesAdvanced Neuroimaging Techniques and ApplicationsNeural and Behavioral Psychology Studies

Valid population inference for information-based imaging: From the second-level t -test to prevalence inference

Open access
Allefeld, Carsten, Görgen, Kai, Haynes, John-Dylan·2016·NeuroImage

Abstract

In multivariate pattern analysis of neuroimaging data, 'second-level' inference is often performed by entering classification accuracies into a t-test vs chance level across subjects. We argue that while the random-effects analysis implemented by the t-test does provide population inference if applied to activation differences, it fails to do so in the case of classification accuracy or other 'information-like' measures, because the true value of such measures can never be below chance level. This constraint changes the meaning of the population-level null hypothesis being tested, which becomes equivalent to the global null hypothesis that there is no effect in any subject in the population. Consequently, rejecting it only allows to infer that there are some subjects in which there is an information effect, but not that it generalizes, rendering it effectively equivalent to fixed-effects analysis. This statement is supported by theoretical arguments as well as simulations. We review possible alternative approaches to population inference for information-based imaging, converging on the idea that it should not target the mean, but the prevalence of the effect in the population. One method to do so, 'permutation-based information prevalence inference using the minimum statistic', is described in detail and applied to empirical data.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2016.07.040

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The nonparametric permutation approach used for group-level hypothesis testing does not depend on assumptions about the distribution of classification accuracies.

    “This nonparametric approach does not depend on assumptions about the distribution of classification accuracies94,95.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Bateson, G., 1972. Steps to an ecology of mind. University of Chicago Press, Chicago.
  2. Huber, P., Ronchetti, E., 2009. Robust statistics, 2nd ed. John Wiley & Sons, Hoboken.
  3. Mudholkar, G., George, E., 1979. The logit method for combining probabilities, in: Symposium on Optimizing Methods in Statistics. Academic Press, New York, pp. 345-366.
  4. Searle, S., Casella, G., McCulloch, C., 1992. Variance components. John Wiley & Sons, Hoboken.

Cited by

Sorted by citations

Authors
Allefeld, C., Görgen, K., & Haynes, J.-D.
Affiliations
  • Bernstein Center for Computational Neuroscience Berlin (DE)
  • Humboldt-Universität zu Berlin (DE)
Corresponding author
Allefeld, Carsten
Venue
NeuroImage
Publisher
Elsevier BV
Published
2016-11
Volume
141
Pages
378-392
ISSN
1053-8119 (ISSN-L)
OpenAlex
W2469334922
Cited by
157
References
86
Citations by year
2026: 11 · 2025: 7 · 2024: 12 · 2023: 7 · 2022: 16 · 2021: 24 · 2020: 22 · 2019: 24 · 2018: 17 · 2017: 12 · 2016: 4
FWCI
12.09
Type
journal article
Language
en
Open access
Locations
  • NeuroImage (publishedVersion)
  • PubMed (publishedVersion)
  • City Research Online (City University London) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
German Research Foundation (GRK1589/1, FK:JA945/3-1)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
population inference, multivariate pattern analysis, classification accuracy, minimum statistics
MeSH terms
Brain, Brain Mapping, Computer Simulation, Data Interpretation, Statistical, Humans, Image Interpretation, Computer-Assisted, Sensitivity and Specificity, Reproducibility of Results, Models, Statistical, Multivariate Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review