The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies.
Abstract
Underpowered studies persist in the psychological literature. This article examines reasons for their persistence and the effects on efforts to create a cumulative science. The "curse of multiplicities" plays a central role in the presentation. Most psychologists realize that testing multiple hypotheses in a single study affects the Type I error rate, but corresponding implications for power have largely been ignored. The presence of multiple hypothesis tests leads to 3 different conceptualizations of power. Implications of these 3 conceptualizations are discussed from the perspective of the individual researcher and from the perspective of developing a coherent literature. Supplementing significance tests with effect size measures and confidence intervals is shown to address some but not necessarily all problems associated with multiple testing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
(2002) · Cited by 13,211
Power analysis and determination of sample size for covariance structure modeling.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.130 · Cited by 10,418
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
(2013) · Handbooks of Sociology and Social Research · doi:10.1007/978-94-007-6772-0 · Cited by 7,486
(2004) · Choice Reviews Online · doi:10.5860/choice.41-3182 · Cited by 4,690
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
How many subjects does it take to do a regression analysis
Green, S. B. (1991) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2603_7 · Cited by 3,916 · cited 2×
“…articles and book chapters been written on the topic of statistical power (e.g., Allison, Al-lison, Faith, Paultre, & Pi-Sunyer, 1997; Cohen, 1992; Green, 1991; Kraemer, 1991; McClelland, 1997; O ’ Brien & Muller, 1993) but in addition several entire books devoted to the topic have been published…”
BackgroundStatistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,401
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Meehl, P. E. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.4.806 · Cited by 1,947
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
On judging the significance of differences by examining the overlap between confidence intervals.
Schenker, N., & Gentleman, J. F. (2001) · The American Statistician · doi:10.1198/000313001317097960 · Cited by 1,218
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,171
Handbook of Applied Social Research Methods
Bickman, L., & Rog, D. J. (1998) · Cited by 1,169
Design Sensitivity: Statistical Power for Experimental Research.
Eliason, S. R., & Lipsey, M. W. (1991) · Contemporary Sociology · doi:10.2307/2072167 · Cited by 1,040
Designing Experiments and Analyzing Data: A Model Comparison Perspective, Third Edition
Maxwell, S. E., & Delaney, H. D. (1999) · Medical Entomology and Zoology · Cited by 975
Self-citationHow Many Subjects?: Statistical Power Analysis in Research
Kraemer, H. C., & Blasey, C. (2016) · SAGE Publications Ltd eBooks · doi:10.4135/9781483398761 · Cited by 910
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 830
Designing Experiments and Analyzing Data: A Model Comparison Perspective.
Serlin, R. C., Maxwell, S. E., & Delaney, H. D. (1991) · Journal of the American Statistical Association · doi:10.2307/2289744 · Cited by 812
Self-citationStatistical power analysis: a simple and general model for traditional and modern hypothesis tests
(1998) · Choice Reviews Online · doi:10.5860/choice.36-1020 · Cited by 792
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037//1082-989x.1.2.115 · Cited by 783
Design Sensitivity: Statistical Power for Experimental Research.
Lachenbruch, P. A., & Lipsey, M. W. (1990) · Journal of the American Statistical Association · doi:10.2307/2289636 · Cited by 782
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748
What If There Were No Significance Tests?
(2013) · Psychology Press eBooks · doi:10.4324/9781315827353 · Cited by 667
What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology.
Schmidt, F. L. (1992) · American Psychologist · doi:10.1037/0003-066x.47.10.1173 · Cited by 564
The Continuing Unethical Conduct of Underpowered Clinical Trials
Halpern, S. D. (2002) · JAMA · doi:10.1001/jama.288.3.358 · Cited by 547
Experimentation in social psychology.
Aronson, E., Wilson, T. D., & Brewer, M. B. (1998) · Cited by 519
How meta-analysis increases statistical power.
Cohn, L. D., & Becker, B. J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.3.243 · Cited by 498
Vote-counting methods in research synthesis.
Hedges, L. V., & Olkin, I. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.2.359 · Cited by 495
Power of the Likelihood Ratio Test in Covariance Structure Analysis
Satorra, A., & Saris, W. E. (1985) · Psychometrika · doi:10.1007/bf02294150 · Cited by 485
The science of prevention: Methodological advances from alcohol and substance abuse research.
(1997) · American Psychological Association eBooks · doi:10.1037/10222-000 · Cited by 472
Investigating and Dealing with Publication and Other Biases
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch11 · Cited by 449
How hard is hard science, how soft is soft science? The empirical cumulativeness of research.
Hedges, L. V. (1987) · American Psychologist · doi:10.1037//0003-066x.42.5.443 · Cited by 435
A General Linear Model for Estimating Effect Size in the Presence of Publication Bias
Vevea, J. L., & Hedges, L. V. (1995) · Psychometrika · doi:10.1007/bf02294384 · Cited by 413
How many subjects? Statistical power analysis in research
(1988) · Evaluation and Program Planning · doi:10.1016/0149-7189(88)90053-5 · Cited by 373
Statistical power of psychological research: What have we gained in 20 years?
Rossi, J. S. (1990) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.58.5.646 · Cited by 323
The role of method in treatment effectiveness research: Evidence from meta-analysis.
Wilson, D. B., & Lipsey, M. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.413 · Cited by 261
Chalmers, T. C., Levin, H., Sacks, H. S., Reitman, D., Berrier, J., & Nagalingam, R. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060320 · Cited by 246
A Review of Statistical Power Analysis Software
(1997) · The Bulletin of the Ecological Society of America · doi:10.2307/20168137 · Cited by 239
Predictive ability of meta-analyses of randomised controlled trials
Villar, J., Carroli, G., & Belizán, J. M. (1995) · The Lancet · doi:10.1016/s0140-6736(95)90646-0 · Cited by 225
Hedges, L. V., & Vevea, J. L. (1996) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165338 · Cited by 221
Optimal design in psychological research.
McClelland, G. H. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.1.3 · Cited by 219
Meta-Analysis in Medicine and Health Policy
(2000) · CRC Press eBooks · doi:10.1201/9780203909935 · Cited by 211
Writing the Empirical Journal Article
Bem, D. J. (2021) · Psychology Press eBooks · doi:10.4324/9781315808314-10 · Cited by 162
Statistical Power and Effect Sizes of Clinical Neuropsychology Research
Bezeau, S., & Graves, R. E. (2001) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.23.3.399.1181 · Cited by 161
Foundations of the Theory of Prediction
Glass, G. V., & Rozeboom, W. W. (1967) · American Educational Research Journal · doi:10.2307/1161775 · Cited by 157
Power and money: Designing statistically powerful studies while minimizing financial costs.
Allison, D. B., Allison, R. L., Faith, M. S., Paultre, F., & Pi‐Sunyer, F. X. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.1.20 · Cited by 143
Kraemer, H. C., Gardner, C., Brooks, J. O., & Yesavage, J. A. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.23 · Cited by 141
Howard, G., Maxwell, S., & Fleming, K. J. (2000) · Psychological methods · doi:10.1037/1082-989x.5.3.315 · Cited by 141
Self-citationDesign Sensitivity: Statistical Power for Applied Experimental Research
Lipsey, M. W., & Hurley, S. (2009) · Sage eBooks · doi:10.4135/9781483348858.n2 · Cited by 135
Causal inference and generalization in field settings: Experimental and quasi-experimental designs.
West, S. G., Biesanz, J. C., & Pitts, S. C. (2000) · Cited by 126
Unified power analysis for t-tests through multivariate hypotheses.
OʼBrien, R. G., & Muller, K. E. (1993) · Cited by 87
THE PERCEPTIONS AND USAGE OF STATISTICAL POWER IN APPLIED PSYCHOLOGY AND MANAGEMENT RESEARCH
Mone, M. A., Mueller, G. C., & MAULAND, W. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01793.x · Cited by 84
Automatic Emotion: Attention and Facial Efference
Hansen, C., & Hansen, R. D. (1994) · Elsevier eBooks · doi:10.1016/b978-0-12-410560-7.50016-8 · Cited by 72
To increase power in randomized clinical trials without increasing sample size.
KRAEMER, H. C. (1991) · PubMed · Cited by 71
The Compleat academic : a practical guide for the beginning social scientist
mark-p-zanna-and-john-m-darley (1986) · Cited by 70
The account taken of statistical power in research published in the British Journal of Psychology
Clark-Carter, D. D. (1997) · British Journal of Psychology · doi:10.1111/j.2044-8295.1997.tb02621.x · Cited by 53
Psychotherapy and behavior change: social, cultural, and methodological perspectives
(1989) · Choice Reviews Online · doi:10.5860/choice.26-4148 · Cited by 52
A Tour of UnifyPow: A SAS Module/Macro for Sample-Size Analysis
OʼBrien, R. G. (1998) · Cited by 48
Statistical power of articles published in three health-psychology related journals.
Maddock, J. E., & Rossi, J. S. (2001) · Health Psychology · doi:10.1037/0278-6133.20.1.76 · Cited by 48
Seven Ways To Increase Power Without Increasing N
Hansen, W. B., & Collins, L. M. (1994) · PsycEXTRA Dataset · doi:10.1037/e495862006-008 · Cited by 48
Power and precision : a computer program for statistical power analysis and confidence intervals
Borenstein, M., Rothstein, H. R., & Cohen, J. (1997) · Medical Entomology and Zoology · Cited by 32
Practical power analysis for substance abuse health services research.
Dennis, M. L., Lennox, R., & Foss, M. A. (1997) · American Psychological Association eBooks · doi:10.1037/10222-011 · Cited by 32
Lilford, R., & Stevens, A. J. (2002) · British journal of surgery · doi:10.1046/j.1365-2168.2002.01989.x · Cited by 31 · cited 1×
“In fact, this perspective has led some medical researchers to maintain that conducting underpowered clinical trials is unethical to research participants (Halpern, Karla-wish, & Berlin, 2002; but for contrary views see Janosky, 2002; Lilford & Stevens, 2002).”
BackgroundThe Ethics of Underpowered Clinical Trials
Lilford, R. J. (2002) · JAMA · doi:10.1001/jama.288.17.2118-b · Cited by 23
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (2016) · doi:10.4324/9781315629049-20 · Cited by 22
Zanna, M. P., & Darley, J. M. (2021) · Psychology Press eBooks · doi:10.4324/9781315808314 · Cited by 19
Cited by
Edwards, J. R., & Lambert, L. S. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.1 · Cited by 4,711
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013) · Educational and Psychological Measurement · doi:10.1177/0013164413495237 · Cited by 3,931
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Metadata
- Authors
- Maxwell, S. E.
- Venue
- Psychological Methods
- Published
- 2004
- Volume
- 9
- Issue
- 2
- Pages
- 147-163
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2074813973
- Cited by
- 654
- References
- 71
- FWCI
- 14.66
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.