Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 187

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

p Value Adjustments for Multiple Tests in Multivariate Binomial Models

Westfall, Peter H., Young, S. Stanley · 1989 · Journal of the American Statistical Association

Abstract

Data from rodent carcinogenicity (preclinical) and clinical studies involving new drugs may be modeled as having come from multivariate binomial distributions. In two-year rodent carcinogenicity studies, there are typically 20–50 tissues examined for occurrence of any of several possible lesions. For a particular treatment group, the number of occurrences of a particular lesion at a particular tissue may be modeled as binomial, and the vector of such frequencies may be considered multivariate binomial with unspecified dependence structure. The same model may also apply to clinical side-effects data; in this case the marginal frequencies may represent occurrences of events ranging from headaches to ingrown toenails. Frequently, the goal of such studies is to isolate site-specific significant differences between treatment and control groups. For example, in rodent carcinogenicity analyses it is generally not sufficient to claim that a new compound causes an increase in tumors at some unspecified site; rather, the report should identify the particular sites where unusual increases are noted. Such an analysis requires separate tests for each site. False significances may easily occur when multiple tests are performed. When a marginal significance criterion p ≤ .05 is used, experimentwise false significance rates as large as 44% have been reported (Haseman, Winbush, and O'Donnell 1986). Others have reported the experimentwise false significance rate much lower; for example, Gart, Chu, and Tarone (1979) reported 8%–10% for each sex and species combination of a two-sex, two-species experiment. In this article it is proposed that the experimentwise false significance rate be controlled by adjusting all p values for the multiplicity of testing using vector-based resampling methods. This analysis is an extension of the bootstrap method described by Westfall (1985) to the multisample case, with particular application to models useful in clinical and preclinical biopharmaceutical analyses; it is also similar to the methodology proposed by Brown and Fears (1981). Assuming no differences between treatment and control groups (the null case), one may estimate the multivariate binomial distribution or permutation distribution conveniently via vector resampling. Using this estimated distribution, one may easily estimate (via Monte Carlo) the probability that the smallest p value in the study is smaller than any given threshold. An adjusted p value is then defined as the probability that the smallest p value in the study is less than or equal to the observed p value for the given test. This methodology is compared to the usual Bonferronistyle adjustments, and it is demonstrated that these adjustments are grossly conservative in certain instances because of their failure to account for dependence between tests and the discreteness of the data. Results of bootstrap and permutation resampling adjustments tend to be similar, particularly for large sample sizes. The approaches are philosophically different: Bootstrap resampling is preferable if an unconditional analysis is desired [Upton (1982) demonstrated that nominal and actual Type I errors are closer and that statistical power is greater in the univariate two-sample case] whereas permutation resampling gives essentially exact results and is preferable if a conditional analysis is desired [Yates (1984) gave philosophical arguments for favoring the conditional approach].

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/01621459.1989.10478837

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Cited by

See all 187

Metadata

Authors
Westfall, P. H., & Young, S. S.
Venue
Journal of the American Statistical Association
Published
1989-09
Volume
84
Issue
407
Pages
780-786
ISSN
  • 0162-1459 (ISSN-L)
  • 1537-274X
OpenAlex
W2013215086
Cited by
187
References
2
FWCI
6.89
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: p Value Adjustments for Multiple Tests in Multivariate Binomial Models. https://app.cochat.ai/papers/10.1080/01621459.1989.10478837
The paper
Westfall, P. H., & Young, S. S. (1989). p Value Adjustments for Multiple Tests in Multivariate Binomial Models. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1989.10478837

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.