The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Methodological reporting behavior, sample sizes, and statistical power in studies of event‐related potentials: Barriers to reproducibility and replicability
Open accessAbstract
Methodological reporting guidelines for studies of ERPs were updated in Psychophysiology in 2014. These guidelines facilitate the communication of key methodological parameters (e.g., preprocessing steps). Failing to report key parameters represents a barrier to replication efforts, and difficulty with replicability increases in the presence of small sample sizes and low statistical power. We assessed whether guidelines are followed and estimated the average sample size and power in recent research. Reporting behavior, sample sizes, and statistical designs were coded for 150 randomly sampled articles from five high‐impact journals that frequently published ERP research from 2011 to 2017. An average of 63% of guidelines were reported, and reporting behavior was similar across journals, suggesting that gaps in reporting is a shortcoming of the field rather than any specific journal. Publication of the guidelines article had no impact on reporting behavior, suggesting that editors and peer reviewers are not enforcing these recommendations. The average sample size per group was 21. Statistical power was conservatively estimated as .72‒.98 for a large effect size, .35‒.73 for a medium effect, and .10‒.18 for a small effect. These findings indicate that failing to report key guidelines is ubiquitous and that ERP studies are primarily powered to detect large effects. Such low power and insufficient following of reporting guidelines represent substantial barriers to replication efforts. The methodological transparency and replicability of studies can be improved by the open sharing of processing code and experimental tasks and by a priori sample size calculations to ensure adequately powered studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many previous studies that have examined associations between subjective and objective measures suffer from low statistical power; a problem that is by no means confined to this particular field of inquiry (Clayson et al., 2019).”
From Effortful listening under the microscope: Examining relations between pupillometric and subjective markers of effort and tiredness from listening (2021) · cites this paper as Clayson et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 3,001
Schuirmann, D. J. (1987) · Journal of Pharmacokinetics and Biopharmaceutics · doi:10.1007/bf01068419 · Cited by 2,696
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Cited by
McGarrigle, R., Rakusen, L., & Mattys, S. (2021) · Psychophysiology · doi:10.1111/psyp.13703 · Cited by 43
Clayson, P. E., Kappenman, E. S., Gehring, W. J., Miller, G. A., & Larson, M. J. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.117932 · Cited by 28
Self-citationDudda, L., Kormann, E., Kozula, M., DeVito, N. J., Klebel, T., Dewi, A. P. M., Spijker, R., Stegeman, I., Van den Eynden, V., Ross-Hellauer, T., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.242057 · Cited by 28
Clayson, P. E., Mcdonald, J. B., Park, B., Holbrook, A., Baldwin, S. A., Riesel, A., & Larson, M. J. (2025) · Psychophysiology · doi:10.1111/psyp.14496 · Cited by 21
Self-citationFiles, B. T., Pollard, K. A., Oiknine, A. H., Khooshabeh, P., & Passaro, A. D. (2021) · Psychophysiology · doi:10.1111/psyp.13856 · Cited by 9
“Our smallest group size was 24, which is somewhat larger than the average group size reported in a recent survey of methodology in eventrelated potentials studies (Clayson et al., 2019), but it is not so large as to eliminate concerns about sampling error.”
Background
Metadata
- Authors
- Clayson, P. E., Carbine, K. A., Baldwin, S. A., & Larson, M. J.
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 2019-07-19
- Volume
- 56
- Issue
- 11
- Pages
- e13437-e13437
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2962993737
- Cited by
- 184
- References
- 29
- FWCI
- 11.52
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methodological reporting behavior, sample sizes, and statistical power in studies of event‐related potentials: Barriers to reproducibility and replicability. https://app.cochat.ai/papers/10.1111/psyp.13437
- The paper
- Clayson, P. E., Carbine, K. A., Baldwin, S. A., & Larson, M. J. (2019). Methodological reporting behavior, sample sizes, and statistical power in studies of event‐related potentials: Barriers to reproducibility and replicability. Psychophysiology. https://doi.org/10.1111/psyp.13437
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.