The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
p Value Adjustments for Multiple Tests in Multivariate Binomial Models
Abstract
Data from rodent carcinogenicity (preclinical) and clinical studies involving new drugs may be modeled as having come from multivariate binomial distributions. In two-year rodent carcinogenicity studies, there are typically 20–50 tissues examined for occurrence of any of several possible lesions. For a particular treatment group, the number of occurrences of a particular lesion at a particular tissue may be modeled as binomial, and the vector of such frequencies may be considered multivariate binomial with unspecified dependence structure. The same model may also apply to clinical side-effects data; in this case the marginal frequencies may represent occurrences of events ranging from headaches to ingrown toenails. Frequently, the goal of such studies is to isolate site-specific significant differences between treatment and control groups. For example, in rodent carcinogenicity analyses it is generally not sufficient to claim that a new compound causes an increase in tumors at some unspecified site; rather, the report should identify the particular sites where unusual increases are noted. Such an analysis requires separate tests for each site. False significances may easily occur when multiple tests are performed. When a marginal significance criterion p ≤ .05 is used, experimentwise false significance rates as large as 44% have been reported (Haseman, Winbush, and O'Donnell 1986). Others have reported the experimentwise false significance rate much lower; for example, Gart, Chu, and Tarone (1979) reported 8%–10% for each sex and species combination of a two-sex, two-species experiment. In this article it is proposed that the experimentwise false significance rate be controlled by adjusting all p values for the multiplicity of testing using vector-based resampling methods. This analysis is an extension of the bootstrap method described by Westfall (1985) to the multisample case, with particular application to models useful in clinical and preclinical biopharmaceutical analyses; it is also similar to the methodology proposed by Brown and Fears (1981). Assuming no differences between treatment and control groups (the null case), one may estimate the multivariate binomial distribution or permutation distribution conveniently via vector resampling. Using this estimated distribution, one may easily estimate (via Monte Carlo) the probability that the smallest p value in the study is smaller than any given threshold. An adjusted p value is then defined as the probability that the smallest p value in the study is less than or equal to the observed p value for the given test. This methodology is compared to the usual Bonferronistyle adjustments, and it is demonstrated that these adjustments are grossly conservative in certain instances because of their failure to account for dependence between tests and the discreteness of the data. Results of bootstrap and permutation resampling adjustments tend to be similar, particularly for large sample sizes. The approaches are philosophically different: Bootstrap resampling is preferable if an unconditional analysis is desired [Upton (1982) demonstrated that nominal and actual Type I errors are closer and that statistical power is greater in the univariate two-sample case] whereas permutation resampling gives essentially exact results and is preferable if a conditional analysis is desired [Yates (1984) gave philosophical arguments for favoring the conditional approach].
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Some Methods for Strengthening the Common χ 2 Tests
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001616 · Cited by 3,718
The Analysis of Multiple Endpoints in Clinical Trials
Pocock, S. J., Geller, N. L., & Tsiatis, A. A. (1987) · Biometrics · doi:10.2307/2531989 · Cited by 524
Cited by
Sisvar: a Guide for its Bootstrap procedures in multiple comparisons
Ferreira, D. F. (2014) · Ciência e Agrotecnologia · doi:10.1590/s1413-70542014000200001 · Cited by 1,864
Identifying differentially expressed genes using false discovery rate controlling procedures
Reiner, A., Yekutieli, D., & Benjamini, Y. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btf877 · Cited by 1,765
Do multiple outcome measures require p-value adjustment?
Feise, R. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-8 · Cited by 1,299
Adjusted P-Values for Simultaneous Inference
Wright, S. P. (1992) · Biometrics · doi:10.2307/2532694 · Cited by 1,026
Phase III Randomized Trial of Amifostine as a Radioprotector in Head and Neck Cancer
Brizel, D. M., Wasserman, T. H., Henke, M., Strnad, V., Rudat, V., Monnier, A., Eschwege, F., Zhang, J., Russell, L., Oster, W., et al. (2000) · Journal of Clinical Oncology · doi:10.1200/jco.2000.18.19.3339 · Cited by 828
Metadata
- Authors
- Westfall, P. H., & Young, S. S.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1989-09
- Volume
- 84
- Issue
- 407
- Pages
- 780-786
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2013215086
- Cited by
- 187
- References
- 2
- FWCI
- 6.89
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: p Value Adjustments for Multiple Tests in Multivariate Binomial Models. https://app.cochat.ai/papers/10.1080/01621459.1989.10478837
- The paper
- Westfall, P. H., & Young, S. S. (1989). p Value Adjustments for Multiple Tests in Multivariate Binomial Models. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1989.10478837
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.