The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
How to get statistically significant effects in any ERP experiment (and why you shouldn't)
Open accessAbstract
ERP experiments generate massive datasets, often containing thousands of values for each participant, even after averaging. The richness of these datasets can be very useful in testing sophisticated hypotheses, but this richness also creates many opportunities to obtain effects that are statistically significant but do not reflect true differences among groups or conditions (bogus effects). The purpose of this paper is to demonstrate how common and seemingly innocuous methods for quantifying and analyzing ERP effects can lead to very high rates of significant but bogus effects, with the likelihood of obtaining at least one such bogus effect exceeding 50% in many experiments. We focus on two specific problems: using the grand‐averaged data to select the time windows and electrode sites for quantifying component amplitudes and latencies, and using one or more multifactor statistical analyses. Reanalyses of prior data and simulations of typical experimental designs are used to show how these problems can greatly increase the likelihood of significant but bogus results. Several strategies are described for avoiding these problems and for increasing the likelihood that significant effects actually reflect true differences among groups or conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2011) · Computational Intelligence and Neuroscience · doi:10.1155/2011/156869 · Cited by 10,323
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Nonparametric statistical testing of EEG- and MEG-data
Maris, E., & Oostenveld, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.03.024 · Cited by 8,496
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Estimating how often this happens is difficult, but there is growing evidence that many published results in psychology (Open Science Collaboration, 2015), neuroscience (Button et al., 2013), and oncology (Prinz, Schlange, & Asadullah, 2011) are not replicable.”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
(2005) · Cited by 5,977
The Oxford Handbook of Event-Related Potential Components
(2011) · doi:10.1093/oxfordhb/9780195374148.001.0001 · Cited by 2,263
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Cited by
Rubin, M. (2021) · Synthese · doi:10.1007/s11229-021-03276-4 · Cited by 371
Large-scale replication study reveals a limit on probabilistic prediction in language comprehension
Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., Von Grebmer Zu Wolfsthurn, S., Bartolozzi, F., Kogan, V., Ito, A., et al. (2018) · eLife · doi:10.7554/elife.33468 · Cited by 299
Coll, M.-P., Hobson, H., Bird, G., & Murphy, J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.012 · Cited by 262
ERP CORE: An open resource for human event-related potential research
Kappenman, E. S., Farrens, J. L., Zhang, W., Stewart, A. X., & Luck, S. J. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117465 · Cited by 223
Self-citationUnderstanding neural signals of post-decisional performance monitoring: An integrative review
Desender, K., Ridderinkhof, K. R., & Murphy, P. R. (2021) · eLife · doi:10.7554/elife.67556 · Cited by 77
Metadata
- Authors
- Luck, S. J., & Gaspelin, N.
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 2016-12-20
- Volume
- 54
- Issue
- 1
- Pages
- 146-157
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2562953409
- Cited by
- 1,195
- References
- 35
- FWCI
- 41.92
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to get statistically significant effects in any ERP experiment (and why you shouldn't). https://app.cochat.ai/papers/10.1111/psyp.12639
- The paper
- Luck, S. J., & Gaspelin, N. (2017). How to get statistically significant effects in any ERP experiment (and why you shouldn't). Psychophysiology. https://doi.org/10.1111/psyp.12639
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.