The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
How to get statistically significant effects in any ERP experiment (and why you shouldn't)
Open accessAbstract
ERP experiments generate massive datasets, often containing thousands of values for each participant, even after averaging. The richness of these datasets can be very useful in testing sophisticated hypotheses, but this richness also creates many opportunities to obtain effects that are statistically significant but do not reflect true differences among groups or conditions (bogus effects). The purpose of this paper is to demonstrate how common and seemingly innocuous methods for quantifying and analyzing ERP effects can lead to very high rates of significant but bogus effects, with the likelihood of obtaining at least one such bogus effect exceeding 50% in many experiments. We focus on two specific problems: using the grand‐averaged data to select the time windows and electrode sites for quantifying component amplitudes and latencies, and using one or more multifactor statistical analyses. Reanalyses of prior data and simulations of typical experimental designs are used to show how these problems can greatly increase the likelihood of significant but bogus results. Several strategies are described for avoiding these problems and for increasing the likelihood that significant effects actually reflect true differences among groups or conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2011) · Computational Intelligence and Neuroscience · doi:10.1155/2011/156869 · Cited by 10,323
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Nonparametric statistical testing of EEG- and MEG-data
Maris, E., & Oostenveld, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.03.024 · Cited by 8,496
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Estimating how often this happens is difficult, but there is growing evidence that many published results in psychology (Open Science Collaboration, 2015), neuroscience (Button et al., 2013), and oncology (Prinz, Schlange, & Asadullah, 2011) are not replicable.”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
(2005) · Cited by 5,977
The Oxford Handbook of Event-Related Potential Components
(2011) · doi:10.1093/oxfordhb/9780195374148.001.0001 · Cited by 2,263
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Cited by
Rubin, M. (2021) · Synthese · doi:10.1007/s11229-021-03276-4 · Cited by 371
Large-scale replication study reveals a limit on probabilistic prediction in language comprehension
Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., Von Grebmer Zu Wolfsthurn, S., Bartolozzi, F., Kogan, V., Ito, A., et al. (2018) · eLife · doi:10.7554/elife.33468 · Cited by 299
Coll, M.-P., Hobson, H., Bird, G., & Murphy, J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.012 · Cited by 262
ERP CORE: An open resource for human event-related potential research
Kappenman, E. S., Farrens, J. L., Zhang, W., Stewart, A. X., & Luck, S. J. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117465 · Cited by 223
Self-citationUnderstanding neural signals of post-decisional performance monitoring: An integrative review
Desender, K., Ridderinkhof, K. R., & Murphy, P. R. (2021) · eLife · doi:10.7554/elife.67556 · Cited by 77
Clayson, P. E., Baldwin, S. A., & Larson, M. J. (2021) · Psychophysiology · doi:10.1111/psyp.13762 · Cited by 84
N400 and P600 modulation in presupposition accommodation: The effect of different trigger types
Domaneschi, F., Canal, P., Masia, V., Lombardi Vallauri, E., & Bambini, V. (2018) · Journal of Neurolinguistics · doi:10.1016/j.jneuroling.2017.08.002 · Cited by 69
Brush, C. J., Hajcak, G., Bocchine, A. J., Ude, A. A., Muniz, K. M., Foti, D., & Alderman, B. L. (2022) · Psychological Medicine · doi:10.1017/s0033291720002573 · Cited by 68
Eklund, R., & Wiens, S. (2018) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-018-0562-z · Cited by 54
Inner speech is accompanied by a temporally-precise and content-specific corollary discharge
Jack, B. N., Le Pelley, M. E., Han, N., Harris, A. W. F., Spencer, K. M., & Whitford, T. J. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.04.038 · Cited by 50
Boen, R., Quintana, D. S., Ladouceur, C. D., & Tamnes, C. K. (2022) · Psychophysiology · doi:10.1111/psyp.14003 · Cited by 40
“Thus, in line with previous recommendations (Luck & Gaspelin, 2017), we urge developmental researchers who are using EEG to consider their methodological choices a priori.”
ResultNarcissism and risky decisions: a neurophysiological approach
Yang, Z., Sedikides, C., Gu, R., Luo, Y. L. L., Wang, Y., & Cai, H. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy053 · Cited by 35
Brush, C. J., Foti, D., Bocchine, A. J., Muniz, K. M., Gooden, M. J., Spaeth, A. M., Miller, M. W., & Alderman, B. L. (2020) · Mental health and physical activity · doi:10.1016/j.mhpa.2020.100339 · Cited by 29
Clayson, P. E., Mcdonald, J. B., Park, B., Holbrook, A., Baldwin, S. A., Riesel, A., & Larson, M. J. (2025) · Psychophysiology · doi:10.1111/psyp.14496 · Cited by 21
Perri, R. L., Bianco, V., Facco, E., & Di Russo, F. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.600083 · Cited by 18
Yammine, L., Verrico, C. D., Versace, F., Webber, H. E., Suchting, R., Weaver, M. F., Kosten, T. R., Alibhai, H., Cinciripini, P. M., Lane, S. D., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-072707 · Cited by 13
Developmental language disorder as syntactic prediction impairment
Hestvik, A., Epstein, B., Schwartz, R. G., & Shafer, V. L. (2022) · Frontiers in Communication · doi:10.3389/fcomm.2021.637585 · Cited by 11
“This approach reduces experimenter bias related to selecting electrode channels and time windows (Luck and Gaspelin, 2017) and reduces multiple comparison problems, as it delivers data-driven constructs of time-windows and electrode regions.”
BackgroundRauchbauer, B., Lorenz, C., Lamm, C., & Pfabigan, D. M. (2021) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-021-00878-w · Cited by 8
Watanabe, H., Nakajima, K., Takagi, S., Mizuyama, R., Saito, M., Furusawa, K., Nakatani, K., Yokota, Y., Kataoka, H., Nakajima, H., et al. (2021) · Frontiers in Neuroergonomics · doi:10.3389/fnrgo.2021.644449 · Cited by 7
LoTemplio, S., Silcox, J., Strayer, D. L., & Payne, B. R. (2026) · Psychophysiology · doi:10.1111/psyp.70216 · Cited by 3
Metadata
- Authors
- Luck, S. J., & Gaspelin, N.
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 2016-12-20
- Volume
- 54
- Issue
- 1
- Pages
- 146-157
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2562953409
- Cited by
- 1,195
- References
- 35
- FWCI
- 41.92
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.