The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Abstract
It is well known that statistical power calculations can be valuable in planning an experiment. There is also a large literature advocating that power calculations be made whenever one performs a statistical test of a hypothesis and one obtains a statistically nonsignificant result. Advocates of such post-experiment power calculations claim the calculations should be used to aid in the interpretation of the experimental results. This approach, which appears in various forms, is fundamentally flawed. We document that the problem is extensive and present arguments to demonstrate the flaw in the logic.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Post-hoc power analysis is criticized because the type II error calculated using the results of negative clinical trials is always high, which sometimes leads to incorrect conclusions regarding power.
“Post-hoc power analysis is criticized because the type II error calculated using the results of negative clinical trials is always high, which sometimes leads to incorrect conclusions regarding power [13,14].”
From Sample size determination and power analysis using the G*Power software (2021) · cites this paper as [14]
Cites
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,385 · see citing works
Cohen, J. (1992) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10768783 · Cited by 5,870 · see citing works
Fundamentals of Biostatistics.
(1969) · Annals of Internal Medicine · doi:10.7326/0003-4819-70-4-873 · Cited by 4,573 · see citing works
Schuirmann, D. J. (1987) · Journal of Pharmacokinetics and Biopharmaceutics · doi:10.1007/bf01068419 · Cited by 2,696 · see citing works
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,188 · see citing works
Toward Evidence-Based Medical Statistics. 2: The Bayes Factor
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00019 · Cited by 913 · see citing works
Statistical Power Analysis can Improve Fisheries Research and Management
Peterman, R. M. (1990) · Canadian Journal of Fisheries and Aquatic Sciences · doi:10.1139/f90-001 · Cited by 788 · see citing works
The Journal of Wildlife Management
Voller, J., & Voller, J. (2011) · Journal of Wildlife Management · doi:10.1002/(issn)1937-2817 · Cited by 709 · see citing works
Goodman, S. N., & Berlin, J. A. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-3-199408010-00008 · Cited by 658 · see citing works
Bioequivalence trials, intersection-union tests and equivalence confidence sets
Berger, R. L., & Hsu, J. C. (1996) · Statistical Science · doi:10.1214/ss/1032280304 · Cited by 621 · see citing works
Statistical Power Analysis in Wildlife Research
Steidl, R. J., Hayes, J. P., & Schauber, E. M. (1997) · Journal of Wildlife Management · doi:10.2307/3802582 · Cited by 420 · see citing works
Thomas, L. (1997) · Conservation Biology · doi:10.1046/j.1523-1739.1997.96102.x · Cited by 410 · see citing works
Statistical power and design requirements for environmental monitoring
Fairweather, P. G. (1991) · Australian Journal of Marine and Freshwater Research · doi:10.1071/mf9910555 · Cited by 354 · see citing works
The Behavior of the P-Value When the Alternative Hypothesis is True
Hung, H., O’Neill, R. T., Bauer, P., & Kohne, K. (1997) · Biometrics · doi:10.2307/2533093 · Cited by 281 · see citing works
Detecting Community-Wide Patterns: Estimating Power Strengthens Statistical Inference
Toft, C. A., & Shea, P. J. (1983) · The American Naturalist · doi:10.1086/284162 · Cited by 269 · see citing works
PValues: What They are and What They are Not
Schervish, M. J. (1996) · The American Statistician · doi:10.1080/00031305.1996.10474380 · Cited by 248 · see citing works
A Review of Statistical Power Analysis Software
Thomas, L., & Krebs, C. J. (1997) · Bulletin of the Ecological Society of America · doi:10.2307/20168137 · Cited by 238 · see citing works
The importance of statistical power analysis: an example fromAnimal Behaviour
Thomas, L., & Juanes, F. (1996) · Animal Behaviour · doi:10.1006/anbe.1996.0232 · Cited by 217 · see citing works
Reversal of the Burden of Proof in Fisheries Management
Dayton, P. K. (1998) · Science · doi:10.1126/science.279.5352.821 · Cited by 208 · see citing works
P Values: What They Are and What They Are Not
Schervish, M. J. (1996) · The American Statistician · doi:10.2307/2684655 · Cited by 200 · see citing works
The Importance of Reporting Statistical Power: The Forest Decline and Acidic Deposition Example
Peterman, R. M. (1990) · Ecology · doi:10.2307/1937612 · Cited by 187 · see citing works
A case for bayesianism in clinical trials
Berry, D. A. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121504 · Cited by 165 · see citing works
Limits of Retrospective Power Analysis
Gerard, P. D., Smith, D. R. R., & Weerakkody, G. J. (1998) · Journal of Wildlife Management · doi:10.2307/3802357 · Cited by 158 · see citing works
Statistical Power Analysis and Community-Wide Patterns
Rotenberry, J. T., & Wiens, J. A. (1985) · The American Naturalist · doi:10.1086/284335 · Cited by 128 · see citing works
Statistical power analysis and the precautionary principle
Peterman, R. M., & M’Gonigle, M. (1992) · Marine Pollution Bulletin · doi:10.1016/0025-326x(92)90559-o · Cited by 113 · see citing works
Increasing scientific power with statistical power
Muller, K. E., & Benignus, V. A. (1992) · Neurotoxicology and Teratology · doi:10.1016/0892-0362(92)90019-7 · Cited by 113 · see citing works
Statistical power: Concepts, procedures, and applications
Hallahan, M., & Rosenthal, R. (1996) · Behaviour Research and Therapy · doi:10.1016/0005-7967(95)00082-8 · Cited by 89 · see citing works
Statistical power and aquacultural research
Searcy‐Bernal, R. (1994) · Aquaculture · doi:10.1016/0044-8486(94)90239-9 · Cited by 87 · see citing works
Statistical Power Analysis and Amphibian Population Trends
Hayes, J. P., & Steidl, R. J. (1997) · Conservation Biology · doi:10.1046/j.1523-1739.1997.96034.x · Cited by 80 · see citing works
The role ofP‐values in analysing trial results
Freeman, P. R. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121510 · Cited by 78 · see citing works
Fagley, N. (1985) · Journal of Counseling Psychology · doi:10.1037/0022-0167.32.3.391 · Cited by 57 · see citing works
Assessment of “Nondeclining” Amphibian Populations Using Power Analysis
Reed, J. M., & Blaustein, A. R. (1995) · Conservation Biology · doi:10.1046/j.1523-1739.1995.9051295.x-i1 · Cited by 57 · see citing works
The Power of a Statistical Test What Does Insignificance Mean?
Markel, M. D. (1991) · Veterinary Surgery · doi:10.1111/j.1532-950x.1991.tb00336.x · Cited by 54 · see citing works
Application of Statistical Power Analysis to the Oregon Coho Salmon (Oncorhynchus kisutch) Problem
Peterman, R. M. (1989) · Canadian Journal of Fisheries and Aquatic Sciences · doi:10.1139/f89-154 · Cited by 42 · see citing works
Biologically Significant Population Declines and Statistical Power
Reed, J. M., & Blaustein, A. R. (1997) · Conservation Biology · doi:10.1046/j.1523-1739.1997.96339.x · Cited by 38 · see citing works
Statistical Power in Research in English Education
Daly, J. A., & Hexamer, A. (1983) · Research in the Teaching of English · doi:10.58680/rte198315715 · Cited by 15 · see citing works
Assessment of "Nondeclining" Amphibian Populations Using Power Analysis
Reed, J. M., & Blaustein, A. R. (1995) · Conservation Biology · doi:10.1046/j.1523-1739.1995.9051299.x · Cited by 14 · see citing works
Power analysis in biological anthropology
Hodges, D. C., & Schell, L. M. (1988) · American Journal of Physical Anthropology · doi:10.1002/ajpa.1330770205 · Cited by 10 · see citing works
Improving outcome research in rehabilitation psychology: Some methodological recommendations.
McAweeney, M. J., Forchheimer, M. B., & Tate, D. G. (1997) · Rehabilitation Psychology · doi:10.1037/0090-5550.42.2.125 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
The ASA Statement on <i>p</i> -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · see citing works
Effect size estimates: Current use, calculations, and interpretation.
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012) · Journal of Experimental Psychology: General · doi:10.1037/a0024338 · Cited by 5,199 · see citing works
Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359 · see citing works
Distance‐Based Tests for Homogeneity of Multivariate Dispersions
Anderson, M. J. (2006) · Biometrics · doi:10.1111/j.1541-0420.2005.00440.x · Cited by 3,304 · see citing works
Inbreeding effects in wild populations
Keller, L. F. (2002) · Trends in Ecology & Evolution · doi:10.1016/s0169-5347(02)02489-8 · Cited by 3,297 · see citing works
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · see citing works
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200 · see citing works
Sample size determination and power analysis using the G*Power software
Kang, H. (2021) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2021.18.17 · Cited by 2,572 · see citing works
<scp>SIMR</scp> : an R package for power analysis of generalized linear mixed models by simulation
Green, P., & MacLeod, C. J. (2016) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12504 · Cited by 2,467 · see citing works
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461 · see citing works
Un sistema de clasificación de los diseños de investigación en psicología
Ato, M., López-García, J. J., & Benavente, A. (2013) · Anales de Psicología · doi:10.6018/analesps.29.3.178511 · Cited by 2,254 · see citing works
A farewell to Bonferroni: the problems of low statistical power and publication bias
Nakagawa, S. (2004) · Behavioral Ecology · doi:10.1093/beheco/arh107 · Cited by 2,102 · see citing works
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962 · see citing works
Bayesian estimation supersedes the t-test
Kruschke, J. K. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0029146 · Cited by 1,530 · see citing works
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343 · see citing works
How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294 · see citing works
Aguinis, H., Beaty, J. C., Boik, R. J., & Pierce, C. A. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.94 · Cited by 1,216 · see citing works
Some Practical Guidelines for Effective Sample Size Determination
Lenth, R. (2001) · The American Statistician · doi:10.1198/000313001317098149 · Cited by 1,200 · see citing works
Brysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996 · see citing works
Sample Size Estimation: How Many Individuals Should Be Studied?
Eng, J. (2003) · Radiology · doi:10.1148/radiol.2272012051 · Cited by 811 · see citing works
Mccarthy, M. A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511802454 · Cited by 758 · see citing works
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · see citing works
MorePower 6.0 for ANOVA with relational confidence intervals and Bayesian analysis
Campbell, J. I. D., & Thompson, V. A. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0186-0 · Cited by 661 · see citing works
Darnall, N., & Edwards, D. D. (2006) · Strategic Management Journal · doi:10.1002/smj.518 · Cited by 623 · see citing works
Power analysis for random-effects meta-analysis
Jackson, D., & Turner, R. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1240 · Cited by 619 · see citing works
Kon, E., Mandelbaum, B. R., Buda, R., Filardo, G., Delcogliano, M., Timoncini, A., Fornasari, P. M., Giannini, S., & Marcacci, M. (2011) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2011.05.011 · Cited by 582 · see citing works
Estimating power in (generalized) linear mixed models: An open introduction and tutorial in R
Kumle, L., Võ, M. L.-H., & Draschkow, D. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01546-0 · Cited by 552 · see citing works
Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093735 · Cited by 542 · see citing works
Hemming, K., Taljaard, M., McKenzie, J. E., Hooper, R., Copas, A., Thompson, J. A., Dixon-Woods, M., Aldcroft, A., Doussau, A., Grayling, M., et al. (2018) · BMJ · doi:10.1136/bmj.k1614 · Cited by 475 · see citing works
Why most conservation monitoring is, but need not be, a waste of time.
Legg, C., & Nagy, L. (2006) · Journal of Environmental Management · doi:10.1016/j.jenvman.2005.04.016 · Cited by 480 · see citing works
Contingent Political Capital and International Alliances: Evidence from South Korea
Siegel, J. I. (2007) · Administrative Science Quarterly · doi:10.2189/asqu.52.4.621 · Cited by 479 · see citing works
Suggestions for Presenting the Results of Data Analyses
Anderson, D. R., Link, W. A., Johnson, D. H., & Burnham, K. P. (2001) · Journal of Wildlife Management · doi:10.2307/3803088 · Cited by 468 · see citing works
The ironic effect of significant results on the credibility of multiple-study articles.
Schimmack, U. (2012) · Psychological Methods · doi:10.1037/a0029487 · Cited by 455 · see citing works
Fusar-Poli, P., Cappucciati, M., Borgwardt, S., Woods, S. W., Addington, J., Nelson, B., Nieman, D. H., Stahl, D. R., Rutigliano, G., Riecher-Rössler, A., et al. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.2324 · Cited by 440 · see citing works
Dziak, J. J., Lanza, S. T., & Tan, X. (2014) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2014.919819 · Cited by 441 · see citing works
Robust misinterpretation of confidence intervals
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0572-3 · Cited by 425 · see citing works
Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide
Hu, X. J. (2004) · Journal of the American Statistical Association · doi:10.1198/jasa.2004.s366 · Cited by 440 · see citing works
Fraley, R. C., & Vazire, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109019 · Cited by 409 · see citing works
Biau, D. J., Kernéis, S., & Porcher, R. (2008) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-008-0346-9 · Cited by 428 · see citing works
Quintana, D. S., & Williams, D. R. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1761-4 · Cited by 415 · see citing works
Testing differences between nested covariance structure models: Power analysis and null hypotheses.
Maccallum, R., Browne, M., & Cai, L. (2006) · Psychological methods · doi:10.1037/1082-989x.11.1.19 · Cited by 374 · see citing works
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376 · see citing works
Post hoc power analysis: is it an informative and meaningful analysis?
Zhang, Y., Hedo, R., Rivera, A. C., Rull, R. P., Richardson, S. M., & Tu, X. M. (2019) · General Psychiatry · doi:10.1136/gpsych-2019-100069 · Cited by 374 · see citing works
Lan, L., & Lian, Z. (2009) · Building and Environment · doi:10.1016/j.buildenv.2009.11.002 · Cited by 359 · see citing works
Martin, J. G. A., Nussey, D. H., Wilson, A. J., & Réale, D. J. F. (2010) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2010.00084.x · Cited by 358 · see citing works
Sun, S.-Y., Pan, W., & Wang, L. (2010) · Journal of Educational Psychology · doi:10.1037/a0019507 · Cited by 329 · see citing works
Global mycorrhizal plant distribution linked to terrestrial carbon stocks
Soudzilovskaia, N. A., van Bodegom, P. M., Terrer, C., Zelfde, M. V., McCallum, I., McCormack, M., Fisher, J. B., Brundrett, M. C., de Sá, N. C., & Tedersoo, L. (2019) · Nature Communications · doi:10.1038/s41467-019-13019-2 · Cited by 340 · see citing works
Muiesan, M. L., Salvetti, M., Monteduro, C., Bonzi, B., Paini, A., Viola, S., Poisa, P., Rizzoni, D., Castellano, M., & Agabiti‐Rosei, E. (2004) · Hypertension · doi:10.1161/01.hyp.0000121223.44837.de · Cited by 330 · see citing works
The effectiveness of Multisystemic Therapy (MST): A meta-analysis
van der Stouwe, T., Asscher, J. J., Stams, G. J. J. M., Deković, M., & van der Laan, P. H. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.06.006 · Cited by 322 · see citing works
Maman, E., Harris, C., White, L. M., TOMLINSON, G. A., Shashank, M., & Boynton, E. L. (2009) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.g.01335 · Cited by 300 · see citing works
A survey of the statistical power of research in behavioral ecology and animal behavior
Jennions, M. D. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.438 · Cited by 278 · see citing works
Voxelwise genome-wide association study (vGWAS)
Stein, J. L., Hua, X., Lee, S., Ho, A. J., Leow, A., Toga, A. W., Saykin, A. J., Shen, L., Foroud, T., Pankratz, N., et al. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.02.032 · Cited by 272 · see citing works
Repeatable evaluation of search services in dynamic environments
Jensen, E. C., Beitzel, S. M., Chowdhury, A., & Frieder, O. (2007) · ACM Transactions on Information Systems · doi:10.1145/1292591.1292592 · Cited by 269 · see citing works
Finch, H. (2005) · Methodology · doi:10.1027/1614-1881.1.1.27 · Cited by 267 · see citing works
Mansournia, M. A., Collins, P. G. S., Nielsen, R. O., Nazemipour, M., Jewell, N. P., Altman, D. G., & Campbell, M. J. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-103652 · Cited by 268 · see citing works
Lehrer, E. J., Peterson, J., Zaorsky, N. G., Brown, P. D., Sahgal, A., Chiang, V., Chao, S. T., Sheehan, J. P., & Trifiletti, D. M. (2018) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.10.038 · Cited by 263 · see citing works
Intravenous Contrast Material Exposure Is Not an Independent Risk Factor for Dialysis or Mortality
McDonald, R. J., McDonald, J. S., Carter, R. E., Hartman, R. P., Katzberg, R. W., Kallmes, D. F., & Williamson, E. E. (2014) · Radiology · doi:10.1148/radiol.14132418 · Cited by 262 · see citing works
David, S. P., Murthy, N. V., Rabiner, E. A., Munafo, M. R., Johnstone, E. C., Jacob, R., Walton, R. T., & Grasby, P. M. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.3769-04.2005 · Cited by 257 · see citing works
Colegrave, N. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.446 · Cited by 251 · see citing works
Sample size determination for image classification accuracy assessment and comparison
Foody, G. M. (2009) · International Journal of Remote Sensing · doi:10.1080/01431160903130937 · Cited by 248 · see citing works
McCluskey, A., & Lovarini, M. (2005) · BMC Medical Education · doi:10.1186/1472-6920-5-40 · Cited by 246 · see citing works
O’Keefe, D. J. (2007) · Communication Methods and Measures · doi:10.1080/19312450701641375 · Cited by 249 · see citing works
Improving the Dependability of Research in Personality and Social Psychology
Funder, D. C., Levine, J. M., Mackie, D. M., Morf, C. C., Sansone, C., Vazire, S., & West, S. G. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313507536 · Cited by 254 · see citing works
Franco, E. A. D., O’Brien, J., Adair, C. D., Lewis, D. F., Hall, D. R., Fusey, S., Soma‐Pillay, P., Porter, K. B., How, H., Schackis, R. C., et al. (2007) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.5159 · Cited by 239 · see citing works
Thomas, E. J. (2009) · JAMA · doi:10.1001/jama.2009.1902 · Cited by 237 · see citing works
Sommet, N., Weissman, D. L., Cheutin, N., & Elliot, A. J. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231178728 · Cited by 274 · see citing works
Current sample size conventions: Flaws, harms, and alternatives
Bacchetti, P. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-17 · Cited by 233 · see citing works
What meta-analyses reveal about the replicability of psychological research.
Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018) · Psychological Bulletin · doi:10.1037/bul0000169 · Cited by 307 · see citing works
Foody, G. M. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2009.03.014 · Cited by 232 · see citing works
Cuypers, I. R. P., & Martín, X. (2009) · Journal of International Business Studies · doi:10.1057/jibs.2009.17 · Cited by 232 · see citing works
Uttley, J. (2019) · LEUKOS The Journal of the Illuminating Engineering Society of North America · doi:10.1080/15502724.2018.1533851 · Cited by 247 · see citing works
Power to Detect What? Considerations for Planning and Evaluating Sample Size
Giner‐Sorolla, R., Montoya, A. K., Reifman, A. S., Carpenter, T., Lewis, N. A., Aberson, C. L., Bostyn, D. H., Conrique, B. G., Ng, B., Schoemann, A. M., et al. (2024) · Personality and Social Psychology Review · doi:10.1177/10888683241228328 · Cited by 292 · see citing works
I can see clearly now: Reinterpreting statistical significance
Dushoff, J. G., Kain, M. P., & Bolker, B. M. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13159 · Cited by 207 · see citing works
The statistical significance filter leads to overoptimistic expectations of replicability
Vasishth, S., Mertzen, D., Jäger, L. A., & Gelman, A. (2018) · Journal of Memory and Language · doi:10.1016/j.jml.2018.07.004 · Cited by 194 · see citing works
The interpretation of statistical power after the data have been gathered
Dziak, J. J., Dierker, L., & Abar, B. W. (2018) · Current Psychology · doi:10.1007/s12144-018-0018-1 · Cited by 208 · see citing works
Bergmann’s Rule in Ectotherms: A Test Using Freshwater Fishes
Belk, M. C., & Houston, D. D. (2002) · The American Naturalist · doi:10.1086/343880 · Cited by 198 · see citing works
Direct Access Compared With Referred Physical Therapy Episodes of Care: A Systematic Review
Ojha, H. A., Snyder, R. S., & Davenport, T. E. (2013) · Physical Therapy · doi:10.2522/ptj.20130096 · Cited by 189 · see citing works
Osteoporosis-related fracture case definitions for population-based administrative data
Lix, L. M., Azimaee, M., Osman, B. A., Caetano, P. A., Morin, S. N., Metge, C. J., Goltzman, D., Kreiger, N., Prior, J. C., & Leslie, W. D. (2012) · BMC Public Health · doi:10.1186/1471-2458-12-301 · Cited by 185 · see citing works
The “Costs” of Urinary Incontinence for Women
Subak, L. L., Brown, J. S., Kraus, S. R., Brubaker, L. T., Lin, F., Richter, H. E., Bradley, C. S., & Grady, D. G. (2006) · Obstetrics and Gynecology · doi:10.1097/01.aog.0000206213.48334.09 · Cited by 183 · see citing works
Barlow, K. E., Briggs, P. A., Haysom, K. A., HUTSON, A. M., Lechiara, N. L., Racey, P. A., Walsh, A. L., & Langton, S. D. (2014) · Biological Conservation · doi:10.1016/j.biocon.2014.11.022 · Cited by 182 · see citing works
The Disclosure and Consequences of U.S. Critical Audit Matters
Burke, J. J., Hoitash, R., Hoitash, U., & Xiao, S. (2022) · The Accounting Review · doi:10.2308/tar-2021-0013 · Cited by 182 · see citing works
Cunningham, L. M., Li, C., Stein, S. E., & Wright, N. S. (2018) · The Accounting Review · doi:10.2308/accr-52305 · Cited by 178 · see citing works
A power calculation guide for fMRI studies
Mumford, J. A. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nss059 · Cited by 171 · see citing works
Shemilt, I., Simon, A., Hollands, G. J., Marteau, T. M., Ogilvie, D., O'Mara‐Eves, A., Kelly, M. P., & Thomas, J. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1093 · Cited by 170 · see citing works
The impact of transgenic plants on natural enemies: a critical review of laboratory studies
Löveï, G. L., & Arpaia, S. (2005) · Entomologia Experimentalis et Applicata · doi:10.1111/j.0013-8703.2005.00235.x · Cited by 170 · see citing works
Statistical power calculations1
Lenth, R. V. (2007) · Journal of Animal Science · doi:10.2527/jas.2006-449 · Cited by 169 · see citing works
Enhancing Rigor in Quantitative Entrepreneurship Research
MAULA, M. V. J., & Stam, W. (2019) · Entrepreneurship Theory and Practice · doi:10.1177/1042258719891388 · Cited by 172 · see citing works
IDENTIFYING ECOLOGICAL CHANGE AND ITS CAUSES: A CASE STUDY ON CORAL REEFS
Fabricius, K., & De’ath, G. (2004) · Ecological Applications · doi:10.1890/03-5320 · Cited by 168 · see citing works
Statistical considerations for reporting and planning heart rate variability case‐control studies
Quintana, D. S. (2017) · Psychophysiology · doi:10.1111/psyp.12798 · Cited by 192 · see citing works
Practical Sample Size Calculations for Surveillance and Diagnostic Investigations
Fosgate, G. T. (2009) · Journal of Veterinary Diagnostic Investigation · doi:10.1177/104063870902100102 · Cited by 162 · see citing works
On the Post Hoc Power in Testing Mean Differences
Yuan, K.-H., & Maxwell, S. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030002141 · Cited by 168 · see citing works
Quality of life in patients with allergic contact dermatitis
Kadyk, D. L., McCarter, K. S., Achen, F., & Belsito, D. V. (2003) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(03)02112-1 · Cited by 160 · see citing works
Cafri, G., Kromrey, J. D., & Brannick, M. T. (2010) · Multivariate Behavioral Research · doi:10.1080/00273171003680187 · Cited by 156 · see citing works
Differing effects of denosumab and alendronate on cortical and trabecular bone
Zebaze, R., Libanati, C. R., Austin, M., Ghasemzadeh, A., Hanley, D. A., Zanchetta, J., Thomas, T., Boutroy, S., Bogado, C. E., Bilezikian, J. P., et al. (2013) · Bone · doi:10.1016/j.bone.2013.11.016 · Cited by 157 · see citing works
Alday, P. M. (2019) · Psychophysiology · doi:10.1111/psyp.13451 · Cited by 155 · see citing works
Nacasch, N., Huppert, J. D., Su, Y., Kivity, Y., Dinstein, Y., Yeh, R., & Foa, E. B. (2014) · Behavior Therapy · doi:10.1016/j.beth.2014.12.002 · Cited by 152 · see citing works
The response of Arctic vegetation and soils following an unusually severe tundra fire
Bret-Harte, M. S., Mack, M. C., Shaver, G. R., Huebner, D. C., Johnston, M. R., Mojica, C. A., Pizano, C., & Reiskind, J. A. (2013) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2012.0490 · Cited by 151 · see citing works
Monte Carlo based statistical power analysis for mediation models: methods and software
Zhang, Z. J. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0424-0 · Cited by 150 · see citing works
- Authors
- Hoenig, J. M., & Heisey, D. M.
- Affiliations
- University of Wisconsin–Madison (US)
- ORCID
- Hoenig, John M
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2001-02
- Volume
- 55
- Issue
- 1
- Pages
- 19-24
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W2069523442
- Cited by
- 2,046
- References
- 38
- Citations by year
- 2026: 213 · 2025: 104 · 2024: 80 · 2023: 97 · 2022: 111 · 2021: 111 · 2020: 124 · 2019: 131 · 2018: 103 · 2017: 87 · 2016: 90 · 2015: 79 · 2014: 84 · 2013: 89 · 2012: 79
- FWCI
- 42.63
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The American Statistician (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- statistical power, non-significant results, experimental design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
