May the power be with you: are there highly powered studies in neuroscience, and how can we get more of them?
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Statistical power is essential for robust science and replicability, but a meta-analysis by Button et al. in 2013 diagnosed a “power failure” for neuroscience. In contrast, Nord et al. ( J Neurosci 37: 8051–8061, 2017) reanalyzed these data and suggested that some studies feature high power. We illustrate how publication and researcher bias might have inflated power estimates, and review recently introduced techniques that can improve analysis pipelines and increase power in neuroscience studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 10×
“Button et al. reported an alarmingly low median power of only 0.21, which means that only once in five times could studies detect the effect they were investigating. Button et al. performed a “metameta-analysis” on all meta-analyses published in neuroscience in 2011 (N 49), assuming that all studies stemmed from the s…”
BackgroundInfluentialA manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“Publication bias is one main reason: significant results are more likely to be accepted for publication than nonsignificant results (Dwan et al. 2008).”
ResultSzucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761 · cited 2×
“Recent estimates suggest that, for this reason, more than 50% of published findings in neuroscience are likely to be false positives (Szucs and Ioannidis 2017): treatments that are reported to work may not work reliably, genes that are reported to contribute to a phenotype may contribute little,…”
BackgroundMeasurement error and the replication crisis
Loken, E., & Gelman, A. (2017) · Science · doi:10.1126/science.aal3618 · Cited by 628 · cited 5×
“The probability that a reported power estimate reflects truly high power (case 1) can be inferred from three assumptions (Szucs and Ioannidis, 2017): 1) Only few effects are truly large, but many are small; 2) in typical small samples, small effects can only become significant if they are inflated (Loken and Gelman 20…”
BackgroundInfluential
Cited by
Open science challenges, benefits and tips in early career and beyond
Allen, C., & Mehler, D. M. A. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000246 · Cited by 608
Self-citationRos, T., Enriquez-Geppert, S., Zotev, V., Young, K. D., Wood, G., Whitfield-Gabrieli, S., Wan, F., Vuilleumier, P., Vialatte, F., Van De Ville, D., et al. (2020) · Brain · doi:10.1093/brain/awaa009 · Cited by 399
Self-citationPsychological measurement and the replication crisis: Four sacred cows
Lilienfeld, S. O., & Strother, A. N. (2020) · Canadian Psychology / Psychologie canadienne · doi:10.1037/cap0000236 · Cited by 144
“Instead, positive findings emanating from underpowered studies are more likely than findings emanating from adequately powered studies to be type I errors (false positives), a statistical phenomenon termed the winner’s curse (Algermissen & Mehler, 2018; Button et al., 2013).”
BackgroundProactive vs. reactive aggression within two modified versions of the Taylor aggression paradigm
Boccadoro, S., Wagels, L., Henn, A. T., Hüpen, P., Graben, L., Raine, A., & Neuner, I. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.749041 · Cited by 23
Metadata
- Authors
- Algermissen, J., & Mehler, D. M. A.
- Venue
- Journal of Neurophysiology
- Publisher
- American Physiological Society
- Published
- 2018-06-01
- Volume
- 119
- Issue
- 6
- Pages
- 2114-2117
- ISSN
- 0022-3077 (ISSN-L)
- 1522-1598
- OpenAlex
- W2788033485
- Cited by
- 70
- References
- 5
- FWCI
- 4.62
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.