Do studies of statistical power have an effect on the power of studies?
Open accessThe DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Abstract
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal that Cohen reviewed (now the Journal of Abnormal Psychology) has not increased over the past 24 years. In 1960 the median power (i.e., the probability that a significant result will be obtained if there is a true effect) was.46 for a medium size effect, whereas in 1984 it was only.37. The decline of power is a result of alpha-adjusted procedures. Low power seems to go unnoticed: only 2 out of 64 experi-ments mentioned power, and it was never estimated. Nonsignificance was generally interpreted as confirmation of the null hypothesis (if this was the research hypothesis), although the median power was as low as.25 in these cases. We discuss reasons for the ongoing neglect of power. Since J. Cohen's (1962) classical study on the statistical power of the studies published in the 1960 volume of the Journal of Abnormal and Social Psychology, a number of power analyses have been performed. These studies exhorted researchers to pay attention to the power of their tests rather than to focus exclu-sively on the level of significance. Historically, the concept of
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A little over two decades after Cohen’s original paper, two meta-analyses by Sedlmeier & Gigerenzer [66] and Rossi [67] examined a total of 25 reviews of statistical power in the psychological and social science literature between 1960 and 1984.”
From The natural selection of bad science (2016) · cites this paper as [66]
In the citing paper
“In fact, the average power of studies published in the 1984 volume of the Journal of Abnormal Psychology (a successor to the JASP) had dropped slightly compared with Cohen's (1962) results.”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
In the citing paper
“Sedlmeier and Gigerenzer (1989) investigated the impact of power analysis studies and textbooks on the power of recent psychological studies. Surprisingly, these authors found no significant increase in power values since 1962 when Cohen published his power study of the 1960 volume of the Journal of Abnormal & Social Psychology (JASP).”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Fundamental Statistics in Psychology and Education.
Smith, B. B., & Guilford, J. P. (1951) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2980908 · Cited by 3,455
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
Statistical Methods and Scientific Inference.
Bartlett, M. S., & Fisher, R. A. (1957) · Biometrika · doi:10.2307/2333276 · Cited by 1,278
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022
Handbook of clinical psychology
Wolman, B. B. (1965) · McGraw-Hill eBooks · Cited by 960
Cited by
Items 701–753 of 753
Outcome Expectancy, Working Alliance, and Symptom Reduction in Social Anxiety Disorder
Benbow, A. (2017) · Digital Archive @ GSU · doi:10.57709/10932754
Cashen, L. H. (2005) · Louisiana State University · doi:10.31390/gradschool_dissertations.2915
Siddaway, A. P. (2017) · Stirling Online Research Repository (University of Stirling)
Modulation of food consumption in healthy adults by exposure to coloured lighting
Edmondson-Jones, K. (2017) · Sheffield Hallam University · doi:10.7190/shu-thesis-00022
Schmidt, F. L. (2012) · Wiley eBooks · doi:10.1002/9781118133880.hop202020
Mellenbergh, G. J. (2019) · Springer eBooks · doi:10.1007/978-3-030-12272-0_12
Nagowski, N., Kirsch, P., Kübler, A., & Quednow, B. B. (2018) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-160317
To Block or Covary a Concomitant Variable: Which Is More Powerful?.
Wu, Y., & McLean, J. E. (1994)
Estimating Rationality in Economics: A History of Statistical Methods in Experimental EconomicsPreprint
Jullien, D., & Vallois, N. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3116220 · Cited by 2
Transparency and replication in alcohol research
Drax, K., & Munafo, M. R. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-816720-5.00005-0
Recent Work in Probability and Inference
Acree, M. C. (2021) · Springer eBooks · doi:10.1007/978-3-030-73257-8_10
Conclusions and the Future of Psychological Research
Acree, M. C. (2021) · Springer eBooks · doi:10.1007/978-3-030-73257-8_11
Managing Speech Perception Data Sets
Cutler, A., Ernestus, M., Warner, N. L., & Weber, A. (2022) · The MIT Press eBooks · doi:10.7551/mitpress/12200.003.0055
Nesselroade, K. P., Jr., & Grimm, L. G. (2018) · Wiley eBooks · doi:10.1002/9781119531708.refs
(2021) · Wiley eBooks · doi:10.1002/9781119558378.ref
Towards a simplified justification of the chosen sample sizePreprint
Broers, N. J., & Otgaar, H. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/kmx75 · Cited by 2
Treat, T. A., & Weersing, V. R. (2014) · Wiley eBooks · doi:10.1002/9781118445112.stat06732
Howell, D. C. (2005) · Wiley eBooks · doi:10.1002/0470013192.bsa490
Murphy, K. R. (2010) · The Corsini Encyclopedia of Psychology · doi:10.1002/9780470479216.corpsy0944
Gigerenzer, G. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.003.0014
Self-citationKirk, R. E. (2014) · Wiley eBooks · doi:10.1002/9781118445112.stat06242
Voelkle, M. C., & Erdfelder, E. (2021) · Springer eBooks · doi:10.1007/978-3-319-69909-7_2230-2
Nagowski, N., Kirsch, P., Kübler, A., & Quednow, B. B. (2018) · Psychologische Rundschau · doi:10.1026/0033-3042/a000418
(2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.002.0004
(2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.002.0006
(2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.002.0005
Cohen’s Convention, the Seriousness of Errors, and the Body of Knowledge in Behavioral SciencePreprint
Arslan, A., & Zenker, F. (2022) · doi:10.31234/osf.io/mjb2x
Compatibility effects with destination and origin of motion
Scerrati, E., Nicoletti, R., Rubichi, S., Scorolli, C., & Lugli, L. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281829
Emotional Intelligence of Dental Students and Its Correlation with Leadership Behaviour
SR, S., Y, F., SI, F., A, M., & AF, A. (2023) · Medicine & health · doi:10.17576/mh.2023.1801.17
Delfin, C. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1253452 · Cited by 12
Impact du langage probabiliste sur l'usage des tests de décision
M’Hamedi, Z. E. (2023) · Éducation & didactique · doi:10.4000/educationdidactique.11833 · Cited by 1
Voelkle, M. C., & Erdfelder, E. (2023) · Springer eBooks · doi:10.1007/978-3-031-17299-1_2230
An Open Letter to Our Students: Doing Interdisciplinary Moral Psychology (2017)
Machery, É., & Doris, J. M. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780198719601.005.0001 · Cited by 2
Doris, J. M. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780198719601.002.0007
Doris, J. M. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780198719601.002.0005
Doris, J. M. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780198719601.002.0003
Doris, J. M. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780198719601.002.0004
Pretscher, A., Paul, K. I., & Zechmann, A. (2025) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026/0932-4089/a000442
Anikin, A. (2025) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02740-3
“…studies have been repeatedly raised since the 1960s (Button et al., 2013; Cohen, 1962; Ioannidis, 2005; Kelley et al., 2003; Schmidt, 1996; Sedlmeier & Gigerenzer, 1992; Smaldino & McElreath, 2016), with each new review repeating the warnings and concluding that there is little sign of…”
Pehlivan, N. N., & Coşkun, H. (2025) · BMC Psychology · doi:10.1186/s40359-025-03523-0 · Cited by 1
“According to Sedlmeier and Gigerenzer (1989) [32], small samples that cannot adequately represent individual differences and random fluctuations may hinder the clear establishment of cause and effect relationships.”
The Perils of p-Values: Why Tests of Statistical Significance Impede the Progress of Research
Kelley, J. M. (2008) · Humana Press eBooks · doi:10.1007/978-1-59745-444-5_16
“Empirical evidence suggests that this may be a generous estimate of statistical power in the social sciences [7, 8 ]. This means that of the 100 studies in which the research hypothesis is correct, only 50 will be statistically significant.”
BackgroundResearcher Degrees of Freedom in the Psychology of ReligionPreprint
Bartlett, J. E., Uzdavines, A., Coleman, T. J., III, Messick, K. J., & Charles, S. J. (2018) · OSF Preprints (OSF Preprints)
Mosen, J., Voracek, M., Kirchler, E., & Olsen, J. (2019) · OSF Preprints (OSF Preprints)
Psychology’s RenaissancePreprint
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2022) · OSF Preprints (OSF Preprints)
Terry, D., Büttner, F., & Iverson, G. L. (2020) · OSF Preprints (OSF Preprints)
[Andreas M. Brandmaier and Markus Werkle-Bergner contributed equally to this work.]
Buchberger, E., Ngo, C., Peikert, A., Brandmaier, A., & Werkle‐Bergner, M. (2024) · MPG.PuRe (Max Planck Society)
975 Statistical power and power analysis in quantitative communication science
Sun, Y., & Qian, S. (2026) · De Gruyter eBooks · doi:10.1515/9783111087139-005
Polito, N. D., Gambarota, F., Hampson, E., & Herlitz, A. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106877
Liang, X. (2026) · Journal of Intercultural Communication Research · doi:10.1080/17475759.2026.2748553
Mendis, A. S. J. R. (2014) · Figshare
Using Interactive Diagrams to Teach Graduate Students About Statistical Power
Hunter, E. (2008) · ScholarsArchive (Brigham Young University)
(2009) · PubMed · Cited by 13
Metadata
- Authors
- Sedlmeier, P., & Gigerenzer, G.
- Venue
- Psychological Bulletin
- Published
- 1989-03
- Volume
- 105
- Issue
- 2
- Pages
- 309-316
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2155449866
- Cited by
- 748
- References
- 55
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 7 · 2023: 14 · 2022: 22 · 2021: 30 · 2020: 23 · 2019: 30 · 2018: 34 · 2017: 35 · 2016: 26 · 2015: 26 · 2014: 30 · 2013: 24 · 2012: 25
- FWCI
- 27.89
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- Locations
- Psychological Bulletin (publishedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- statistical power, power analysis, effect size, multiple comparisons, non-significant results, null hypothesis significance testing, history of statistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.