Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 194

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Use and misuse of corrections for multiple testing

Open access
García-Pérez, Miguel A. · 2023 · Methods in Psychology

Abstract

Current psychological research addresses multifaceted questions demanding multiple analyses of data. Statistical analyses regarded as instances of multiple testing are often subjected to alpha adjustments to guard against inflation of Type-I errors. A review of papers published in the last two years in two major psychology journals shows inconsistent and discretionary use of alpha adjustments in a broad diversity of statistical analyses that are formally identical across papers. Authoritative sources also do not clarify the circumstances in which alpha adjustments should or should not be used. This paper describes the workings of Bonferroni and false-discovery-rate adjustments, showing that they only control the Type-I error rate for an (omnibus) hypothesis stating that all its individual (surrogate) nulls are true. For individual nulls, alpha adjustment only has the trivial consequences of the use of a lower alpha level, without reducing the occurrence of Type-I errors or Type-II errors below their expected rates. In practice, then, corrections for multiple testing only come down to testing individual hypotheses at a lower alpha level without preventing the rejection of true nulls and without favoring the rejection of false nulls. Thus, use of alpha adjustments is only justifiable for inferences about an omnibus null for which a one-shot statistical test does not exist and which must instead be tested piecewise via several surrogates that collectively speak about the omnibus null. Recommendations for the use and reporting of alpha adjustments are given for a variety of statistical analyses with which they are often implemented.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.metip.2023.100120

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 19 references

Cited by

See all 194

Metadata

Authors
García-Pérez, M. A.
Venue
Methods in Psychology
Publisher
Elsevier BV
Published
2023-11
Volume
8
Pages
100120
ISSN
2590-2601 (ISSN-L)
OpenAlex
W4327923045
Cited by
194
References
19
FWCI
38.29
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Use and misuse of corrections for multiple testing. https://app.cochat.ai/papers/10.1016/j.metip.2023.100120
The paper
García-Pérez, M. A. (2023). Use and misuse of corrections for multiple testing. Methods in Psychology. https://doi.org/10.1016/j.metip.2023.100120

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.