The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Cluster‐based permutation tests of MEG/EEG data do not establish significance of effect latency or location
Abstract
Cluster‐based permutation tests are gaining an almost universal acceptance as inferential procedures in cognitive neuroscience. They elegantly handle the multiple comparisons problem in high‐dimensional magnetoencephalographic and EEG data. Unfortunately, the power of this procedure comes hand in hand with the allure for unwarranted interpretations of the inferential output, the most prominent of which is the overestimation of the temporal, spatial, and frequency precision of statistical claims. This leads researchers to statements about the onset or offset of a certain effect that is not supported by the permutation test. In this article, we outline problems and common pitfalls of using and interpreting cluster‐based permutation tests. We illustrate these with simulated data in order to promote a more intuitive understanding of the method. We hope that raising awareness about these issues will be beneficial to common scientific practices, while at the same time increasing the popularity of cluster‐based permutation procedures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371
Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2011) · Computational Intelligence and Neuroscience · doi:10.1155/2011/156869 · Cited by 10,323
Nonparametric statistical testing of EEG- and MEG-data
Maris, E., & Oostenveld, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.03.024 · Cited by 8,496
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
(2005) · Cited by 5,977
MEG and EEG data analysis with MNE-Python
Gramfort, A. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00267 · Cited by 3,784
Removing electroencephalographic artifacts by blind source separation
Jung, T., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V. J., & Sejnowski, T. J. (2000) · Psychophysiology · doi:10.1017/s0048577200980259 · Cited by 3,334
Mass univariate analysis of event‐related brain potentials/fields I: A critical tutorial review
Groppe, D. M., Urbach, T. P., & Kutas, M. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01273.x · Cited by 1,169
Characterizing the dynamics of mental representations: the temporal generalization method
King, J.-R., & Dehaene, S. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.01.002 · Cited by 919
Statistical methods of estimation and inference for functional MR image analysis
Bullmore, E., Brammer, M., Williams, S. C. R., Rabe‐Hesketh, S., Janot, N., David, A., Mellers, J., Howard, R., & Sham, P. (1996) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910350219 · Cited by 683
Registered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Kiesel, A., Miller, J., Jolicœur, P., & Brisson, B. (2008) · Psychophysiology · doi:10.1111/j.1469-8986.2007.00618.x · Cited by 424
Tanner, D., Morgan‐Short, K., & Luck, S. J. (2015) · Psychophysiology · doi:10.1111/psyp.12437 · Cited by 354
Pernet, C. R., Latinus, M., Nichols, T. E., & Rousselet, G. A. (2015) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2014.08.003 · Cited by 300
Statistical testing in electrophysiological studies
Maris, E. (2012) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01320.x · Cited by 232
The intractability of scaling scalp distributions to infer neuroelectric sources
Urbach, T. P., & Kutas, M. (2002) · Psychophysiology · doi:10.1111/1469-8986.3960791 · Cited by 212
Advanced EEG analysis using threshold-free cluster-enhancement and non-parametric statistics
Mensen, A., & Khatami, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.10.027 · Cited by 176
Does Filtering Preclude Us from Studying ERP Time-Courses?
Rousselet, G. A. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00131 · Cited by 117
Bias in a common EEG and MEG statistical analysis and how to avoid it
Kilner, J. M. (2013) · Clinical Neurophysiology · doi:10.1016/j.clinph.2013.03.024 · Cited by 96
Vul, E., Harris, C. R., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01132.x · Cited by 80
Why jackknifing yields good latency estimates
Miller, J. O., Ulrich, R., & Schwarz, W. (2009) · Psychophysiology · doi:10.1111/j.1469-8986.2008.00761.x · Cited by 63
Measures of Spatial Similarity and Response Magnitude in MEG and Scalp EEG
Tian, X., & Huber, D. E. (2008) · Brain Topography · doi:10.1007/s10548-007-0040-3 · Cited by 30
Piai, V., Dahlslätt, K., & Maris, E. (2014) · Psychophysiology · doi:10.1111/psyp.12335 · Cited by 27
Cited by
ERP CORE: An open resource for human event-related potential research
Kappenman, E. S., Farrens, J. L., Zhang, W., Stewart, A. X., & Luck, S. J. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117465 · Cited by 223
Vishne, G., Gerber, E. M., Knight, R. T., & Deouell, L. Y. (2023) · Cell Reports · doi:10.1016/j.celrep.2023.112752 · Cited by 46
Thalamic oscillations distinguish natural states of consciousness in humans
Chowdhury, A., Wu, X., Beilner, T., Schreiner, T., Koeglsperger, T., Mehrkens, J.-H., Remi, J., Vollmar, C., Kaufmann, E., & Staudigl, T. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02446-z · Cited by 1
Metadata
- Authors
- Sassenhagen, J., & Draschkow, D.
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 2019-01-18
- Volume
- 56
- Issue
- 6
- Pages
- e13335-e13335
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2911019338
- Cited by
- 545
- References
- 23
- FWCI
- 28.66
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.