The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Do multiple outcome measures require p-value adjustment?
Open accessAbstract
BACKGROUND: Readers may question the interpretation of findings in clinical trials when multiple outcome measures are used without adjustment of the p-value. This question arises because of the increased risk of Type I errors (findings of false "significance") when multiple simultaneous hypotheses are tested at set p-values. The primary aim of this study was to estimate the need to make appropriate p-value adjustments in clinical trials to compensate for a possible increased risk in committing Type I errors when multiple outcome measures are used. DISCUSSION: The classicists believe that the chance of finding at least one test statistically significant due to chance and incorrectly declaring a difference increases as the number of comparisons increases. The rationalists have the following objections to that theory: 1) P-value adjustments are calculated based on how many tests are to be considered, and that number has been defined arbitrarily and variably; 2) P-value adjustments reduce the chance of making type I errors, but they increase the chance of making type II errors or needing to increase the sample size. SUMMARY: Readers should balance a study's statistical significance with the magnitude of effect, the quality of the study and with findings from other studies. Researchers facing multiple outcome measures might want to either select a primary outcome measure or use a global assessment measure, rather than adjusting the p-value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Due to this statistical approach and to decrease the chances of making a Type 1 error, the criterion for significance was set at p ≤ 0.01 [32].”
From The Effects of an Eight over Cricket Bowling Spell upon Pace Bowling Biomechanics and Performance within Different Delivery Lengths (2019) · cites this paper as [32]
In the citing paper
“However, given that the approach here is of planned comparisons, where all analyses are pre-specified, it is not necessarily appropriate to apply these corrections (Feise, 2002; Schulz and Grimes, 2005).”
From Sleep duration and psychotic experiences in patients at risk of psychosis: a secondary analysis of the EDIE-2 trial (2019) · cites this paper as Feise, 2002
In the citing paper
“The overall α level was set at 0.05, using Bonferroni's criterion for the primary outcome but not for the secondary outcomes, to balance between type I and type II error (Feise, 2002).”
From Efficacy of 8- and 4-Session Mindfulness-Based Interventions in a Non-clinical Population: A Controlled Study (2017) · cites this paper as Feise, 2002
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526
No adjustments are needed for multiple comparisons.
Rothman, K. J. (1990) · Epidemiology · doi:10.1097/00001648-199001000-00010 · Cited by 5,965
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900
Multiple significance tests: the Bonferroni method
Bland, J. M., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.310.6973.170 · Cited by 3,341
Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
Šidák, Z. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482935 · Cited by 2,904
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396
Multiple Comparison Procedures
Hahs-Vaughn, D., & Lomax, R. (2013) · Routledge eBooks · doi:10.4324/9780203137819-17 · Cited by 2,132
A Stagewise Rejective Multiple Test Procedure Based on a Modified Bonferroni Test
Hommel, G. (1988) · Biometrika · doi:10.1093/biomet/75.2.383 · Cited by 1,367
Adjusting for multiple testing when reporting research results: the Bonferroni vs Holm methods.
Aickin, M., & Gensler, H. (1996) · American Journal of Public Health · doi:10.2105/ajph.86.5.726 · Cited by 1,346
ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE
Neyman, J., & Pearson, E. S. (1928) · Biometrika · doi:10.1093/biomet/20a.3-4.263 · Cited by 1,164
Cited by
Sorted by citations
Adjust for multiple comparisons? It’s not that simple
Althouse, A. D. (2016) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2015.11.024 · Cited by 825
Lonsdorf, T. B., Menz, M. M., Andreatta, M., Fullana, M. A., Golkar, A., Haaker, J., Heitland, I., Hermann, A., Kuhn, M., Kruse, O., et al. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.026 · Cited by 837
COVID-19 and vaccine hesitancy: A longitudinal study
Fridman, A., Gershon, R., & Gneezy, A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0250123 · Cited by 697
Differences in Rural and Urban Health Information Access and Use
Chen, X., Orom, H., Hay, J. L., Waters, E. A., Schofield, E., Li, Y., & Kiviniemi, M. T. (2018) · The Journal of Rural Health · doi:10.1111/jrh.12335 · Cited by 635
Escaping the Bonferroni iron claw in ecological studies
García, L. V. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.13046.x · Cited by 556
Metadata
- Authors
- Feise, R. J.
- Corresponding author
- Feise, Ronald J
- ORCID
- Feise, Ronald J
- Venue
- BMC Medical Research Methodology
- Published
- 2002-06-17
- Volume
- 2
- Issue
- 1
- Pages
- 8-8
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC117123
- OpenAlex
- W2139498473
- Cited by
- 1,299
- References
- 42
- Citations by year
- 2026: 55 · 2025: 67 · 2024: 75 · 2023: 101 · 2022: 79 · 2021: 133 · 2020: 95 · 2019: 115 · 2018: 83 · 2017: 98 · 2016: 72 · 2015: 45 · 2014: 47 · 2013: 33 · 2012: 32
- FWCI
- 5.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, other-oa)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- p-value adjustment, type I error, multiple comparisons, clinical trials, primary outcomes, statistical significance, sample size
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Confidence Intervals, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
