The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
A farewell to Bonferroni: the problems of low statistical power and publication bias
Abstract
Recently, Jennions and Møller (2003) carried out a meta-analysis on statistical power in the field of behavioral ecology and animal behavior, reviewing 10 leading journals including Behavioral Ecology. Their results showed dismayingly low average statistical power (note that a meta-analytic review of statistical power is different from post hoc power analysis as criticized in Hoenig and Heisey, 2001). The statistical power of a null hypothesis (Ho) significance test is the probability that the test will reject Ho when a research hypothesis (Ha) is true. Knowledge of effect size is particularly important for statistical power analysis (for statistical power analysis, see Cohen, 1988; Nakagawa and Foster, in press). There are many kinds of effect size measures available (e.g., Pearson's r, Cohen's d, Hedges's g), but most of these fall into one of two major types, namely the r family and the d family (Rosenthal, 1994). The r family shows the strength of relationship between two variables while the d family shows the size of difference between two variables. As a benchmark for research planning and evaluation, Cohen (1988) proposed ‘conventional’ values for small, medium, and large effects: r =.10,.30, and.50 and d =.20,.50, and.80, respectively (in the way that p values of.05,.01, and.001 are conventional points, although these conventional values of effect size have been criticized; e.g., Rosenthal et al., 2000).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046 · cited 2×
“Their results showed dismayingly low average statistical power (note that a meta-analytic review of statistical power is different from post hoc power analysis as criticized in Hoenig and Heisey, 2001).”
BackgroundMethodThings I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778 · cited 1×
“…observed (standardized) effect size, such as Pearson’s r and Cohen’s d, in journal articles (r and d are readily convertible; Rosenthal, 1994), as also recommended by Stoehr (1999) and Jennions and Møller (2003) (see these papers for various benefits of reporting effect size; see also Kirk, 1996).”
BackgroundArguments for rejecting the sequential Bonferroni in ecological studies
Moran, M. D. (2003) · Oikos · doi:10.1034/j.1600-0706.2003.12010.x · Cited by 1,484 · cited 3×
“The sequential Bonferroni procedure also incurs reduction in power, but to a lesser extent (which is the reason that the sequential procedure is used in preference by some researchers; Moran, 2003).”
BackgroundMethodInfluentialA survey of the statistical power of research in behavioral ecology and animal behavior
Jennions, M. D. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.438 · Cited by 278 · cited 5×
“Department of Animal and Plant Sciences, University of Sheffield, Sheffield S10 2TN, United Kingdom Recently, Jennions and Møller (2003) carried out a metaanalysis on statistical power in the field of behavioral ecology and animal behavior, reviewing 10 leading journals including Behavioral Ecology.”
BackgroundMethodInfluential
Cited by
Mind-sets matter: A meta-analytic review of implicit theories and self-regulation.
Burnette, J. L., O'Boyle, E. H., VanEpps, E. M., Pollack, J. M., & Finkel, E. J. (2013) · Psychological Bulletin · doi:10.1037/a0029531 · Cited by 1,367
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436
Cognitive reserve and cognitive function in healthy older people: a meta-analysis
Opdebeeck, C., Martyr, A., & Clare, L. (2016) · Aging, Neuropsychology, and Cognition · doi:10.1080/13825585.2015.1041450 · Cited by 465
Wolever, T. M., Tosh, S. M., Gibbs, A. L., Brand-Miller, J., Duncan, A. M., Hart, V., Lamarche, B., Thomson, B. A., Duss, R., & Wood, P. J. (2010) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.2010.29174 · Cited by 398
Questionable research practices in ecology and evolution
Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F. (2018) · PLOS ONE · doi:10.1371/journal.pone.0200303 · Cited by 346
Self-citationMindfulness training affects attention—Or is it attentional effort?
Jensen, C. G., Vangkilde, S., Frokjaer, V., & Hasselbalch, S. G. (2012) · Journal of Experimental Psychology: General · doi:10.1037/a0024931 · Cited by 284
Lawrence, N. S., Hinton, E. C., Parkinson, J. A., & Lawrence, A. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.070 · Cited by 242
Assessing the Relative Impact of Diverse Stressors among Public Safety Personnel
Carleton, R. N., Afifi, T. O., Taillieu, T., Turner, S., Mason, J. E., Ricciardelli, R., McCreary, D. R., Vaughan, A. D., Anderson, G. S., Krakauer, R. L., et al. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17041234 · Cited by 225
“We did not use additional Bonferroni corrections, despite the multiple comparisons, because contemporary theory suggests against such a correction in this type of analytic scenario [62,63].”
MethodStrohmaier, S. (2020) · Mindfulness · doi:10.1007/s12671-020-01319-4 · Cited by 163
Brooks, E., Sloman, K., Sims, D., & Danylchuk, A. (2011) · Endangered Species Research · doi:10.3354/esr00331 · Cited by 156
Vanhove, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069172 · Cited by 158
Strohmaier, S., Jones, F. W., & Cane, J. E. (2021) · Mindfulness · doi:10.1007/s12671-020-01512-5 · Cited by 128
Kushalnagar, P., Topolski, T. D., Schick, B., Edwards, T. C., Skalicky, A. M., & Patrick, D. L. (2011) · Journal of Deaf Studies and Deaf Education · doi:10.1093/deafed/enr015 · Cited by 111
Simon, N., Roberts, N. P., Lewis, C. E., van Gelderen, M. J., & Bisson, J. I. (2019) · European Journal of Psychotraumatology · doi:10.1080/20008198.2019.1573129 · Cited by 107
Fitak, R. R., & Johnsen, S. (2017) · Journal of Experimental Biology · doi:10.1242/jeb.167056 · Cited by 92
Douglass, J. G., Duffy, J. E., & Bruno, J. F. (2008) · Ecology Letters · doi:10.1111/j.1461-0248.2008.01175.x · Cited by 90
Cazzell, S., Vayser, D., Pham, H., Walters, J., Reyzelman, A., Samsell, B., Dorsch, K., & Moore, M. (2017) · Wound Repair and Regeneration · doi:10.1111/wrr.12551 · Cited by 78
Xia, J., Qi, X., Wu, Y., Zhu, B., Xu, L., Zhang, L., Gao, X., Chen, Y., Li, J., & Gao, H. (2016) · Mammalian Genome · doi:10.1007/s00335-016-9635-x · Cited by 77
Recovering from stillbirth: the effects of making and sharing memories on maternal mental health
Crawley, R., Lomax, S., & Ayers, S. (2013) · Journal of Reproductive and Infant Psychology · doi:10.1080/02646838.2013.795216 · Cited by 73
Schnell, T., Fuchs, D., & Hefti, R. (2020) · Journal of Religion and Health · doi:10.1007/s10943-020-01008-5 · Cited by 69
Becker, S. P., Duraccio, K. M., Sidol, C. A., Fershtman, C. E. M., Byars, K. C., & Harvey, A. G. (2022) · Journal of Attention Disorders · doi:10.1177/10870547211056965 · Cited by 58
Briones, E. M., & Benham, G. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0710-8 · Cited by 48
Pettersson, M., Höglund, A. T., & Hedström, M. (2018) · PLOS ONE · doi:10.1371/journal.pone.0206550 · Cited by 43
Reconstruction of the colonization of southern Madagascar by introduced Rattus rattus
Hingston, M., Goodman, S. M., Ganzhorn, J. U., & Sommer, S. (2005) · Journal of Biogeography · doi:10.1111/j.1365-2699.2005.01311.x · Cited by 38
Sato, M. (2014) · System · doi:10.1016/j.system.2014.05.004 · Cited by 37
Little, K., Olsson, C. A., Youssef, G. J., Whittle, S., Simmons, J. G., Yücel, M., Sheeber, L. B., Foley, D. L., & Allen, N. B. (2015) · Journal of Abnormal Psychology · doi:10.1037/abn0000101 · Cited by 30
Little, K., Olsson, C. A., Whittle, S., Youssef, G. J., Byrne, M. L., Simmons, J. G., Yücel, M., Foley, D. L., & Allen, N. B. (2014) · Translational Psychiatry · doi:10.1038/tp.2014.85 · Cited by 29
Day, M. A., Thorn, B. E., Ehde, D. M., Burns, J. W., Barnier, A., Mattingley, J. B., Matthews, N., & Jensen, M. P. (2020) · The Journal of Pain · doi:10.1016/j.jpain.2019.06.006 · Cited by 28
Day, M. A., & Thorn, B. E. (2017) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2017.06.009 · Cited by 27
Treatment validity for intervention studies.
Chambless, D. L., & Hollon, S. D. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-028 · Cited by 24
Infrasound in mating displays: a peacock's tale
Freeman, A. R., & Hare, J. F. (2015) · Animal Behaviour · doi:10.1016/j.anbehav.2015.01.029 · Cited by 23
Anxiety in high-functioning autism: A pilot study of experience sampling using a mobile platform
Hare, D. J., Gracey, C., & Wood, C. (2016) · Autism · doi:10.1177/1362361315604817 · Cited by 22
Becker, S. P. (2017) · Journal of Attention Disorders · doi:10.1177/1087054717692882 · Cited by 17
Arbeider, M., Sharpe, C., Carr-Harris, C., & Moore, J. (2019) · Marine Ecology Progress Series · doi:10.3354/meps12896 · Cited by 8
A meta-analysis of the effects in alpha acoustic indices
Wang, Y., Zhang, Y., Xia, C., Pape Møller, A., & 1. Ministry of Education Key Laboratory for Biodiversity Science and Ecological Engineering, C. O. L. S. (2023) · Biodiversity Science · doi:10.17520/biods.2022369 · Cited by 7
Balestra, C., Maj, C., Müller, E., & Mayr, A. (2023) · PLOS ONE · doi:10.1371/journal.pone.0282699 · Cited by 4
Metadata
- Authors
- Nakagawa, S.
- Venue
- Behavioral Ecology
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-11
- Volume
- 15
- Issue
- 6
- Pages
- 1044-1045
- ISSN
- 1045-2249 (ISSN-L)
- 1465-7279
- OpenAlex
- W1986694300
- Cited by
- 2,097
- References
- 8
- FWCI
- 61.50
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
