When to use the B onferroni correction
Abstract
Purpose The Bonferroni correction adjusts probability (p) values because of the increased risk of a type I error when making multiple statistical tests. The routine use of this test has been criticised as deleterious to sound statistical judgment, testing the wrong hypothesis, and reducing the chance of a type I error but at the expense of a type II error; yet it remains popular in ophthalmic research. The purpose of this article was to survey the use of the Bonferroni correction in research articles published in three optometric journals, viz. Ophthalmic & Physiological Optics, Optometry & Vision Science, and Clinical & Experimental Optometry, and to provide advice to authors contemplating multiple testing. Recent findings Some authors ignored the problem of multiple testing while others used the method uncritically with no rationale or discussion. A variety of methods of correcting p values were employed, the Bonferroni method being the single most popular. Bonferroni was used in a variety of circumstances, most commonly to correct the experiment-wise error rate when using multiple 't' tests or as a post-hoc procedure to correct the family-wise error rate following analysis of variance (anova). Some studies quoted adjusted p values incorrectly or gave an erroneous rationale. Summary Whether or not to use the Bonferroni correction depends on the circumstances of the study. It should not be used routinely and should be considered if: (1) a single test of the 'universal null h…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
To enhance regression reliability, a Bonferroni correction was included as this established new alpha numbers to determine significance.
“To enhance regression reliability, a Bonferroni correction (Armstrong, 2014) was included as this established new alpha numbers to determine significance.”
From An Investigation into the Perception and Prevalence of Mixed-Species Exhibits in Zoos and Aquaria (2022) · cites this paper as Armstrong, 2014
Cites
No adjustments are needed for multiple comparisons.
Rothman, K. J. (1990) · Epidemiology · doi:10.1097/00001648-199001000-00010 · Cited by 5,965 · see citing works
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900 · see citing works
Multiplicity in randomised trials I: endpoints and treatments
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66461-6 · Cited by 366 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A gut-derived metabolite alters brain activity and anxiety behaviour in mice
Needham, B. D., Funabashi, M., Adame, M. D., Wang, Z., Boktor, J. C., Haney, J., Wu, W.-L., Rabut, C., Ladinsky, M. S., Hwang, S.-J., et al. (2022) · Nature · doi:10.1038/s41586-022-04396-8 · Cited by 489 · see citing works
Rubin, M. (2021) · Synthese · doi:10.1007/s11229-021-03276-4 · Cited by 371 · see citing works
Kozhimannil, K. B., Hung, P., Henning-Smith, C., Casey, M. M., & Prasad, S. (2018) · JAMA · doi:10.1001/jama.2018.1830 · Cited by 325 · see citing works
Trends in Beverage Consumption Among Children and Adults, 2003‐2014
Bleich, S. N., Vercammen, K. A., Koma, J. W., & Li, Z. (2018) · Obesity · doi:10.1002/oby.22056 · Cited by 318 · see citing works
Improving inferences about null effects with Bayes factors and equivalence tests.
Lakens, D., McLatchie, N., Isager, P. M., Scheel, A. M., & Dienes, Z. (2020) · The Journals of Gerontology: Series B · doi:10.1093/geronb/gby065 · Cited by 279 · see citing works
Gambin, M., Sękowski, M., Woźniak-Prus, M., Wnuk, A., Oleksy, T., Cudo, A., Hansen, K., Huflejt-Łukasik, M., Kubicka, K., Łyś, A. E., et al. (2021) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2020.152222 · Cited by 202 · see citing works
Microbiota influence the development of the brain and behaviors in C57BL/6J mice
Lu, J., Synowiec, S., Lu, L., Yu, Y., Bretherick, T., Takada, S., Yarnykh, V., Caplan, J., Caplan, M., Claud, E. C., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0201829 · Cited by 159 · see citing works
Hirosawa, T., Kawamura, R., Harada, Y., Mizuta, K., Tokumasu, K., Kaji, Y., Suzuki, T., & Shimizu, T. (2023) · JMIR Medical Informatics · doi:10.2196/48808 · Cited by 130 · see citing works
Zhao, L., Dong, Q., Luo, C., Wu, Y., Bu, D., Qi, X., Luo, Y., & Zhao, Y. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.04.067 · Cited by 127 · see citing works
Childhood adversity and psychiatric disorder in young adulthood: An analysis of 107,704 Swedes
Björkenstam, E., Burström, B., Vinnerljung, B., & Kosidou, K. (2016) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2016.02.018 · Cited by 77 · see citing works
Gashi, S., Di Lascio, E., & Santini, S. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3314400 · Cited by 74 · see citing works
High-throughput target trial emulation for Alzheimer’s disease drug repurposing with real-world data
Zang, C., Zhang, H., Xu, J., Zhang, H., Fouladvand, S., Havaldar, S., Cheng, F., Chen, K., Chen, Y., Glicksberg, B. S., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-43929-1 · Cited by 69 · see citing works
Morid, M. A., Sheng, O. R. L., Kawamoto, K., & Abdelrahman, S. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103565 · Cited by 49 · see citing works
Graham, L. M., Ranapurwala, S. I., Zimmer, C., Macy, R. J., Rizo, C. F., Lanier, P., & Martin, S. L. (2021) · PLoS ONE · doi:10.1371/journal.pone.0246477 · Cited by 33 · see citing works
O'Connor, E. E., Holly, L. E., Chevalier, L. L., Pincus, D. B., & Langer, D. A. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.22941 · Cited by 35 · see citing works
Trinh, M., Kalloniatis, M., Alonso-Caneiro, D., & Nivison-Smith, L. (2022) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.63.5.36 · Cited by 29 · see citing works
Åkerblom, S., Perrin, S., Rivano Fischer, M., & McCracken, L. M. (2018) · The Clinical Journal of Pain · doi:10.1097/ajp.0000000000000561 · Cited by 33 · see citing works
Day, M. A., Thorn, B. E., Ehde, D. M., Burns, J. W., Barnier, A., Mattingley, J. B., Matthews, N., & Jensen, M. P. (2020) · The Journal of Pain · doi:10.1016/j.jpain.2019.06.006 · Cited by 28 · see citing works
Gort, C., Marcusson-Clavertz, D., & Kuehner, C. (2021) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/s10942-020-00353-4 · Cited by 29 · see citing works
Day, M. A., & Thorn, B. E. (2017) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2017.06.009 · Cited by 27 · see citing works
Kablo, E., & Arias-Cabarcos, P. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3623164 · Cited by 14 · see citing works
Trinh, M., Khou, V., Kalloniatis, M., & Nivison-Smith, L. (2021) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.62.13.13 · Cited by 14 · see citing works
Blaney, C., Hitchon, C. A., Marrie, R. A., Mackenzie, C., Holens, P., & El-Gabalawy, R. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100385 · Cited by 14 · see citing works
“We did not perform a correction for multiple tests because of the exploratory nature of the study (Armstrong, 2014).”
MethodA Novel Diagnostic Decision Support System for Medical Professionals: Prospective Feasibility Study
Timiliotis, J., Blümke, B., Serfözö, P. D., Gilbert, S., Ondrésik, M., Türk, E., Hirsch, M. C., & Eckstein, J. (2022) · JMIR Formative Research · doi:10.2196/29943 · Cited by 13 · see citing works
Wright, P. S. R., Briggs, K. A., Thomas, R., Smith, G. F., Maglennon, G., Mikulskis, P., Chapman, M., Greene, N., Phillips, B. U., & Bender, A. (2023) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2022.105308 · Cited by 8 · see citing works
An Investigation into the Perception and Prevalence of Mixed-Species Exhibits in Zoos and Aquaria
Green, H., Palmer, G. A., & Brereton, J. E. (2022) · Journal of Research in Social Science and Humanities · doi:10.56397/jrssh.2022.12.02 · Cited by 4 · see citing works
“To enhance regression reliability, a Bonferroni correction (Armstrong, 2014) was included as this established new alpha numbers to determine significance.”
MethodHoetink, A. E., Kaldenbach, S., Lieftink, A., Versnel, H., & Stokroos, R. J. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1451741 · Cited by 3 · see citing works
When design deceives: consumer reactions to dark patterns in quick commerce
Agrawal, S., & Narayanan, S. (2026) · Journal of Indian Business Research · doi:10.1108/jibr-05-2025-0188 · Cited by 4 · see citing works
- Authors
- Armstrong, R. A.
- Affiliations
- Aston University (GB)
- Corresponding author
- Armstrong, Richard A.
- Venue
- Ophthalmic and Physiological Optics
- Published
- 2014-04-02
- Volume
- 34
- Issue
- 5
- Pages
- 502-508
- ISSN
- 0275-5408 (ISSN-L)
- 1475-1313
- OpenAlex
- W2076646346
- Cited by
- 3,031
- References
- 3
- Citations by year
- 2026: 314 · 2025: 436 · 2024: 435 · 2023: 346 · 2022: 350 · 2021: 340 · 2020: 243 · 2019: 196 · 2018: 161 · 2017: 102 · 2016: 63 · 2015: 34 · 2014: 9 · 2012: 2
- FWCI
- 8.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Ophthalmic and Physiological Optics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- Bonferroni correction, multiple comparisons, type I error, type II error, family-wise error rate, post-hoc correction, p-value adjustment, ophthalmology research, optometry journals, statistical reporting
- MeSH terms
- Data Interpretation, Statistical, Humans, Ophthalmology, Optometry, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
