Power-up: A Reanalysis of 'Power Failure' in Neuroscience Using Mixture Modeling
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Recently, evidence for endemically low statistical power has cast neuroscience findings into doubt. If low statistical power plagues neuroscience, then this reduces confidence in the reported effects. However, if statistical power is not uniformly low, then such blanket mistrust might not be warranted. Here, we provide a different perspective on this issue, analyzing data from an influential study reporting a median power of 21% across 49 meta-analyses (Button et al., 2013). We demonstrate, using Gaussian mixture modeling, that the sample of 730 studies included in that analysis comprises several subcomponents so the use of a single summary statistic is insufficient to characterize the nature of the distribution. We find that statistical power is extremely low for studies included in meta-analyses that reported a null result and that it varies substantially across subfields of neuroscience, with particularly low power in candidate gene association studies. Therefore, whereas power in neuroscience remains a critical issue, the notion that studies are systematically underpowered is not the full story: low power is far from a universal problem. SIGNIFICANCE STATEMENT Recently, researchers across the biomedical and psychological sciences have become concerned with the reliability of results. One marker for reliability is statistical power: the probability of finding a statistically significant result given that the effect exists. Previous evidence suggests that statistical power is low across the field of neuroscience. Our results present a more comprehensive picture of statistical power in neuroscience: on average, studies are indeed underpowered—some very seriously so—but many studies show acceptable or even exemplary statistical power. We show that this heterogeneity in statistical power is common across most subfields in neuroscience. This new, more nuanced picture of statistical power in neuroscience could affect not only scientific understanding, but potentially policy and funding decisions for neuroscience research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
(2007) · doi:10.5555/1283383.1283494 · Cited by 7,364
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Anthropometric Reference Data for Children and Adults: United States, 2003-2006: (623932009-001)
McDowell, M. A., Fryar, C. D., Ogden, C. L., & Flegal, K. M. (2008) · PsycEXTRA Dataset · doi:10.1037/e623932009-001 · Cited by 1,378
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Duncan, L. E., & Keller, M. C. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11020191 · Cited by 1,032
Delayed reward discounting and addictive behavior: a meta-analysis
MacKillop, J., Amlung, M. T., Few, L. R., Ray, L. A., Sweet, L. H., & Munafò, M. R. (2011) · Psychopharmacology · doi:10.1007/s00213-011-2229-0 · Cited by 1,021
Frontocingulate Dysfunction in Depression: Toward Biomarkers of Treatment Response
Pizzagalli, D. A. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.166 · Cited by 945
Veehof, M. M., Oskam, M.-J., Schreurs, K. M. G., & Bohlmeijer, E. T. (2011) · PAIN · doi:10.1016/j.pain.2010.11.002 · Cited by 792
Cited by
Sex Differences in the Adult Human Brain: Evidence from 5216 UK Biobank Participants
Ritchie, S. J., Cox, S. R., Shen, X., Lombardo, M. V., Reus, L. M., Alloza, C., Harris, M. A., Alderson, H. L., Hunter, S., Neilson, E., et al. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy109 · Cited by 855
“Given the surfeit of small-n studies in neuroscience (Button et al. 2013; Nord et al. 2017), it is of great importance to test hypotheses in large, well-powered samples, especially given that many neural sex differences are modest in size (Joel et al.”
BackgroundHippocampal volume across age: Nomograms derived from over 19,700 people in UK Biobank
Nobis, L., Manohar, S. G., Smith, S. M., Alfaro-Almagro, F., Jenkinson, M., Mackay, C. E., & Husain, M. (2019) · NeuroImage: Clinical · doi:10.1016/j.nicl.2019.101904 · Cited by 236
Disrupted Dorsal Mid-Insula Activation During Interoception Across Psychiatric Disorders
Nord, C. L., Lawson, R. P., & Dalgleish, T. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20091340 · Cited by 171
Self-citation10 years of Bayesian theories of autism: A comprehensive review
Angeletos Chrysaitis, N., & Seriès, P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.105022 · Cited by 92
Sex/gender differences in cognitive abilities
Kheloui, S., Jacmin-Park, S., Larocque, O., Kerr, P., Rossi, M., Cartier, L., & Juster, R.-P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105333 · Cited by 78
Metadata
- Authors
- Nord, C. L., Valton, V., Wood, J., & Roiser, J. P.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2017-07-13
- Volume
- 37
- Issue
- 34
- Pages
- 8051-8061
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2735379969
- Cited by
- 86
- References
- 21
- FWCI
- 10.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Power-up: A Reanalysis of 'Power Failure' in Neuroscience Using Mixture Modeling. https://app.cochat.ai/papers/10.1523/jneurosci.3592-16.2017
- The paper
- Nord, C. L., Valton, V., Wood, J., & Roiser, J. P. (2017). Power-up: A Reanalysis of 'Power Failure' in Neuroscience Using Mixture Modeling. The Journal of Neuroscience. https://doi.org/10.1523/jneurosci.3592-16.2017
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.