Do studies of statistical power have an effect on the power of studies?
Open accessThe DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Abstract
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal that Cohen reviewed (now the Journal of Abnormal Psychology) has not increased over the past 24 years. In 1960 the median power (i.e., the probability that a significant result will be obtained if there is a true effect) was.46 for a medium size effect, whereas in 1984 it was only.37. The decline of power is a result of alpha-adjusted procedures. Low power seems to go unnoticed: only 2 out of 64 experi-ments mentioned power, and it was never estimated. Nonsignificance was generally interpreted as confirmation of the null hypothesis (if this was the research hypothesis), although the median power was as low as.25 in these cases. We discuss reasons for the ongoing neglect of power. Since J. Cohen's (1962) classical study on the statistical power of the studies published in the 1960 volume of the Journal of Abnormal and Social Psychology, a number of power analyses have been performed. These studies exhorted researchers to pay attention to the power of their tests rather than to focus exclu-sively on the level of significance. Historically, the concept of
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A little over two decades after Cohen’s original paper, two meta-analyses by Sedlmeier & Gigerenzer [66] and Rossi [67] examined a total of 25 reviews of statistical power in the psychological and social science literature between 1960 and 1984.”
From The natural selection of bad science (2016) · cites this paper as [66]
In the citing paper
“In fact, the average power of studies published in the 1984 volume of the Journal of Abnormal Psychology (a successor to the JASP) had dropped slightly compared with Cohen's (1962) results.”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
In the citing paper
“Sedlmeier and Gigerenzer (1989) investigated the impact of power analysis studies and textbooks on the power of recent psychological studies. Surprisingly, these authors found no significant increase in power values since 1962 when Cohen published his power study of the 1960 volume of the Journal of Abnormal & Social Psychology (JASP).”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Fundamental Statistics in Psychology and Education.
Smith, B. B., & Guilford, J. P. (1951) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2980908 · Cited by 3,455
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
Statistical Methods and Scientific Inference.
Bartlett, M. S., & Fisher, R. A. (1957) · Biometrika · doi:10.2307/2333276 · Cited by 1,278
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022
Handbook of clinical psychology
Wolman, B. B. (1965) · McGraw-Hill eBooks · Cited by 960
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 911
Statistical Methods and Scientific Inference.
Bailey, N. T. J., & Fisher, R. (1957) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2342557 · Cited by 760
The Empire of chance: how probability changed science and everyday life
(1989) · Choice Reviews Online · doi:10.5860/choice.27-2084 · Cited by 597
Statistical Methods and Scientific Induction
Fisher, R. (1955) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1955.tb00180.x · Cited by 596
Applied Statistics: Analysis of Variance and Regression.
Imrey, P. B., Dunn, O. J., & Clark, V. A. (1976) · Journal of the American Statistical Association · doi:10.2307/2285782 · Cited by 591
Cognition as Intuitive Statistics
Gigerenzer, G., & Murray, D. J. (2015) · Psychology Press eBooks · doi:10.4324/9781315668796 · Cited by 453
Self-citationStatistical Inference: A Commentary for the Social and Behavioural Sciences
Oakes, M. W. (1986) · Medical Entomology and Zoology · Cited by 388
Magnitude of experimental effect and a table for its rapid estimation.
Friedman, H. (1968) · Psychological Bulletin · doi:10.1037/h0026258 · Cited by 312
Statistical Power and Effect Size in Marketing Research
Sawyer, A. G., & Ball, A. D. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800302 · Cited by 269
Effect of violation of normality on the t test of the correlation coefficient.
Edgell, S. E., & Noon, S. M. (1984) · Psychological Bulletin · doi:10.1037//0033-2909.95.3.576 · Cited by 192
The Present State of Multiple Comparison Methods
O'Neill, R. G., & Wetherill, G. B. (1971) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1971.tb00874.x · Cited by 169
Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias.
Coursol, A., & Wagner, E. E. (1986) · Professional Psychology: Research and Practice · doi:10.1037//0735-7028.17.2.136 · Cited by 165
Petrinovich, L. F., & Hardyck, C. D. (1969) · Psychological Bulletin · doi:10.1037/h0026861 · Cited by 139
Pearson, E. S. (1939) · Biometrika · doi:10.2307/2332648 · Cited by 134
Analysis of variance and the "second discipline" of scientific psychology: A historical account.
Rucci, A. J., & Tweney, R. D. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.87.1.166 · Cited by 120
A statistical power analysis of applied psychological research.
Chase, L. J., & Chase, R. B. (1976) · Journal of Applied Psychology · doi:10.1037/0021-9010.61.2.234 · Cited by 115
Cohen, J. (1970) · Educational and Psychological Measurement · doi:10.1177/001316447003000404 · Cited by 111
Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982) · Journal of Counseling Psychology · doi:10.1037/0022-0167.29.2.189 · Cited by 109
Birnbaum, A. (1977) · Synthese · doi:10.1007/bf00485690 · Cited by 104
On the Power of Statistical Tests in the American Educational Research Journal
Brewer, J. K. (1972) · American Educational Research Journal · doi:10.3102/00028312009003391 · Cited by 100
Introduction to statistical methods
Harnett, D. L. (1970) · Cited by 97
Lehrbuch der Statistik : für Sozialwissenschaftler
Bortz, J. (1977) · Springer eBooks · Cited by 84
Haase, R. F., Waechter, D. M., & Solomon, G. S. (1982) · Journal of Counseling Psychology · doi:10.1037/0022-0167.29.1.58 · Cited by 83
Probabilistic thinking and the fight against subjectivity
Gigerenzer, G. (1987) · Max Planck Digital Library · Cited by 74
Self-citationIntroduction to Statistics for Psychology and Education
Clayton, K. N. (1984) · Medical Entomology and Zoology · Cited by 71
The probabilistic revolution: Vol. 2. Ideas in the sciences
Krüger, L., Gigerenzer, G., & Morgan, M. S. (1987) · Cited by 55
Self-citationExpected Effect Sizes: Estimates for Statistical Power Analysis in Social Psychology
Cooper, H., & Findley, M. (1982) · Personality and Social Psychology Bulletin · doi:10.1177/014616728281026 · Cited by 53
An Analysis of the Use of Statistical Testing in Communication Research
Katzer, J., & Sodt, J. (1973) · Journal of Communication · doi:10.1111/j.1460-2466.1973.tb00947.x · Cited by 50
The analysis of variance in experimental psychology: 1934–1945
Lovie, A. D. (1979) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1979.tb00591.x · Cited by 50
A power‐analytic examination of contemporary communication research
Chase, L. J., & Tucker, R. K. (1975) · Speech Monographs · doi:10.1080/03637757509375874 · Cited by 49
STATISTICAL POWER OF TRAINING EVALUATION DESIGNS
Arvey, R. D., Cole, D. A., Hazucha, J. F., & Hartanto, F. M. (1985) · Personnel Psychology · doi:10.1111/j.1744-6570.1985.tb00556.x · Cited by 44
Statistical Power: Derivation, Development, and Data-Analytic Implications
Chase, L. J., & Tucker, R. K. (1976) · The Psychological Record · doi:10.1007/bf03394413 · Cited by 43
Error Probabilities in Educational and Psychological Research
Westermann, R., & Hager, W. (1986) · Journal of Educational Statistics · doi:10.2307/1164973 · Cited by 36
Communication disorders: A power analytic assessment of recent research
Kroll, R. M., & Chase, L. J. (1975) · Journal of Communication Disorders · doi:10.1016/0021-9924(75)90016-7 · Cited by 34
How significant is a significant difference? Problems with the measurement of magnitude of effect.
Murray, L. W., & Dosser, D. A. (1987) · Journal of Counseling Psychology · doi:10.1037/0022-0167.34.1.68 · Cited by 33
An Assessment of Quantitative Research in Mass Communication
Chase, L. J., & Baran, S. J. (1976) · Journalism Quarterly · doi:10.1177/107769907605300217 · Cited by 25
Cohen, L. H. (1979) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.47.2.421 · Cited by 23
Power Analysis of Research in Counselor Education
Haase, R. F. (1974) · Counselor Education and Supervision · doi:10.1002/j.1556-6978.1974.tb00854.x · Cited by 21
Statistical Power and Research in Occupational Therapy
Ottenbacher, K. J. (1982) · The Occupational Therapy Journal of Research · doi:10.1177/153944928200200102 · Cited by 17
Krüger, L., Gigerenzer, G., & Morgan, M. S. (1987) · Medical Entomology and Zoology · Cited by 16
Self-citationStatistical power of the controlled research on wheat gluten and schizophrenia
King, D. S. (1985) · Biological Psychiatry · doi:10.1016/0006-3223(85)90157-x · Cited by 12
On the Assumptions Concerning the Assumptions of atTest
Trachtman, J. N., Giambalvo, V., & Dippner, R. (1978) · The Journal of General Psychology · doi:10.1080/00221309.1978.9920901 · Cited by 11
Hager, W., & Möller, H. (1986) · Biometrical Journal · doi:10.1002/bimj.4710280602 · Cited by 9
F., W. T., & Harter, H. L. (1970) · Biometrics · doi:10.2307/2528742 · Cited by 9
Cited by
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Cohen, J. (1992) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10768783 · Cited by 5,870
GPOWER: A general power analysis program
Erdfelder, E., Faul, F., & Buchner, A. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03203630 · Cited by 4,864
“According to Sedlmeier and Gigerenzer (1989), a major reason for this confusion is the "hybridization" of the Fisherian and the Neyman-Pearson theories ofstatistical inference in the psychological literature (see also Giger enzer, 1993).”
BackgroundIssues and opinion on structural equation modeling
Chin, W. W. (1998) · Journal of the Association for Information Systems · doi:10.5555/290231.290235 · Cited by 4,577
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
“He reminded researchers that, given the fact that the nil hypothesis is always false, the rate of Type I errors is 0%, not 5%, and that only Type II errors can be made, which run typically at about 50% (Cohen, 1962; Sedlmeier & Gigerenzer, 1989).”
Background
Metadata
- Authors
- Sedlmeier, P., & Gigerenzer, G.
- Venue
- Psychological Bulletin
- Published
- 1989-03
- Volume
- 105
- Issue
- 2
- Pages
- 309-316
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2155449866
- Cited by
- 748
- References
- 55
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 7 · 2023: 14 · 2022: 22 · 2021: 30 · 2020: 23 · 2019: 30 · 2018: 34 · 2017: 35 · 2016: 26 · 2015: 26 · 2014: 30 · 2013: 24 · 2012: 25
- FWCI
- 27.89
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- Locations
- Psychological Bulletin (publishedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- statistical power, power analysis, effect size, multiple comparisons, non-significant results, null hypothesis significance testing, history of statistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.