Post Hoc Power Analysis: An Idea Whose Time Has Passed?
Abstract
Using a hypothetical scenario typifying the experience that authors have when submitting manuscripts that report results of negative clinical trials, the pitfalls of a post hoc analysis are illustrated. We used the same scenario to explain how confidence intervals are used in interpreting results of clinical trials. We showed that confidence intervals better inform readers about the possibility of an inadequate sample size than do post hoc power calculations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Post-hoc power analysis is criticized because the type II error calculated using the results of negative clinical trials is always high, which sometimes leads to incorrect conclusions regarding power.
“Post-hoc power analysis is criticized because the type II error calculated using the results of negative clinical trials is always high, which sometimes leads to incorrect conclusions regarding power [13,14].”
From Sample size determination and power analysis using the G*Power software (2021) · cites this paper as [13]
Cites
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Sample size determination and power analysis using the G*Power software
Kang, H. (2021) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2021.18.17 · Cited by 2,572 · see citing works
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461 · see citing works
Un sistema de clasificación de los diseños de investigación en psicología
Ato, M., López-García, J. J., & Benavente, A. (2013) · Anales de Psicología · doi:10.6018/analesps.29.3.178511 · Cited by 2,254 · see citing works
Ismail-Beigi, F., Craven, T., Banerji, M. A., Basile, J., Calles, J., Cohen, R. M., Cuddihy, R., Cushman, W. C., Genuth, S., Grimm, R. H., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60576-4 · Cited by 1,428 · see citing works
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · see citing works
Fusar-Poli, P., Cappucciati, M., Borgwardt, S., Woods, S. W., Addington, J., Nelson, B., Nieman, D. H., Stahl, D. R., Rutigliano, G., Riecher-Rössler, A., et al. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.2324 · Cited by 440 · see citing works
A power calculation guide for fMRI studies
Mumford, J. A. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nss059 · Cited by 171 · see citing works
Improving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2019) · Journal of Neuroscience Research · doi:10.1002/jnr.24340 · Cited by 83 · see citing works
Tzikos, G., Alexiou, I., Tsagkaropoulos, S., Menni, A.-E., Chatziantoniou, G., Doutsini, S., Papavramidis, T., Grosomanidis, V., Stavrou, G., & Kotzampassi, K. (2023) · Journal of Personalized Medicine · doi:10.3390/jpm13030473 · Cited by 28 · see citing works
Plasma oxytocin is modulated by mental training, but does not mediate its stress-buffering effect
Hoehne, K., Vrtička, P., Engert, V., & Singer, T. (2022) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2022.105734 · Cited by 17 · see citing works
Early life stress, FKBP5 polymorphisms, and quantitative glycemic traits
Suarez, A., Lahti, J., Kajantie, E., Eriksson, J. G., & Räikkönen, K. (2017) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000439 · Cited by 8 · see citing works
Mabbe, Soenens, Vansteenkiste, & Pauw, D. (2020) · Merrill-Palmer Quarterly · doi:10.13110/merrpalmquar1982.66.3.0308 · Cited by 5 · see citing works
- Authors
- Levine, M., & Ensom, M. H. H.
- Affiliations
- University of British Columbia (CA)
- Corresponding author
- Levine, Marc
- ORCID
- Ensom, Mary H. H.
- Venue
- Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy
- Publisher
- Wiley
- Published
- 2001-04
- Volume
- 21
- Issue
- 4
- Pages
- 405-409
- ISSN
- 0277-0008 (ISSN-L)
- 1875-9114
- OpenAlex
- W2087491399
- Cited by
- 473
- References
- 1
- Citations by year
- 2026: 23 · 2025: 27 · 2024: 34 · 2023: 39 · 2022: 50 · 2021: 35 · 2020: 33 · 2019: 40 · 2018: 28 · 2017: 23 · 2016: 24 · 2015: 22 · 2014: 18 · 2013: 17 · 2012: 11
- FWCI
- 3.10
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- post hoc power analysis, confidence intervals, sample size adequacy, statistical interpretation, clinical trial reporting, manuscript submission
- MeSH terms
- Humans, Periodicals as Topic, Reproducibility of Results, Confidence Intervals, Randomized Controlled Trials as Topic, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
