The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Higher criticism for detecting sparse heterogeneous mixtures
Open accessAbstract
Higher criticism, or second-level significance testing, is a multiple-comparisons concept mentioned in passing by Tukey. It concerns a situation where there are many independent tests of significance and one is interested in rejecting the joint null hypothesis. Tukey suggested comparing the fraction of observed significances at a given α-level to the expected fraction under the joint null. In fact, he suggested standardizing the difference of the two quantities and forming a z-score; the resulting z-score tests the significance of the body of significance tests. We consider a generalization, where we maximize this z-score over a range of significance levels 0<α≤α0. We are able to show that the resulting higher criticism statistic is effective at resolving a very subtle testing problem: testing whether n normal means are all zero versus the alternative that a small fraction is nonzero. The subtlety of this “sparse normal means” testing problem can be seen from work of Ingster and Jin, who studied such problems in great detail. In their studies, they identified an interesting range of cases where the small fraction of nonzero means is so small that the alternative hypothesis exhibits little noticeable effect on the distribution of the p-values either for the bulk of the tests or for the few most highly significant tests. In this range, when the amplitude of nonzero means is calibrated with the fraction of nonzero means, the likelihood ratio test for a precisely specified alternative would still succeed in separating the two hypotheses. We show that the higher criticism is successful throughout the same region of amplitude sparsity where the likelihood ratio test would succeed. Since it does not require a specification of the alternative, this shows that higher criticism is in a sense optimally adaptive to unknown sparsity and size of the nonnull effects. While our theoretical work is largely asymptotic, we provide simulations in finite samples and suggest some possible applications. We also show that higher critcism works well over a range of non-Gaussian cases.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Continuous Univariate Distributions.
Omori, Y., Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995) · Journal of the American Statistical Association · doi:10.2307/2291547 · Cited by 9,084
Simultaneous Statistical Inference
Miller, R. G. (1981) · Springer Series in Statistics · doi:10.1007/978-1-4613-8122-8 · Cited by 5,604
Bayesian Inference in Statistical Analysis.
Kadane, J. B., Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.2307/2285412 · Cited by 3,824
Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes
Anderson, T. W., & Darling, D. A. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729437 · Cited by 3,542
Multiple Comparison Procedures
Hochberg, Y., & Tamhane, A. C. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316672 · Cited by 2,389
Empirical Processes with Applications to Statistics
Shorack, G. R., & Wellner, J. A. (2009) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898719017 · Cited by 2,310
Multiple Comparison Procedures
(2013) · An Introduction to Statistical Concepts · doi:10.4324/9780203137819-17 · Cited by 2,109
Statistical Methods for Research Workers.
Grove, C. C., & Fisher, R. A. (1930) · The American Mathematical Monthly · doi:10.2307/2299007 · Cited by 1,349
(1953) · Medical Entomology and Zoology · Cited by 1,321
Adapting to unknown sparsity by controlling the false discovery rate
Abramovich, F., Benjamini, Y., Donoho, D. L., & Johnstone, I. M. (2006) · The Annals of Statistics · doi:10.1214/009053606000000074 · Cited by 455
Self-citationA failure of likelihood asymptotics for normal mixtures
Hartigan, J. A. (1985) · Medical Entomology and Zoology · Cited by 205
Modeling the joint statistics of images in the wavelet domain:
Simoncelli, E. P. (1999) · SPIE Proceedings · doi:10.1117/12.366779 · Cited by 193
A limit theorem for the maximum of normalized sums of independent random variables
Darling, D. A., & Erdös, P. (1956) · Duke Mathematical Journal · doi:10.1215/s0012-7094-56-02313-4 · Cited by 190
Wellner, J. A. (1978) · Probability Theory and Related Fields · doi:10.1007/bf00635964 · Cited by 136
Combining significance levels.
Becker, B. J. (1994) · Cited by 115
Reliable and questionable significance in a series of statistical tests.
Brožek, J., & Tiede, K. (1952) · Psychological Bulletin · doi:10.1037/h0058274 · Cited by 54
A Note on the Asymptotic Distribution of Berk—Jones Type Statistics under the Null Hypothesis
Wellner, J. A., & Koltchinskii, V. I. (2003) · Birkhäuser Basel eBooks · doi:10.1007/978-3-0348-8059-6_19 · Cited by 24
Cited by
Analyzing gene expression data in terms of gene sets: methodological issues
Goeman, J. J., & Bühlmann, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm051 · Cited by 802
Genome-wide association analysis of metabolic traits in a birth cohort from a founder population
Sabatti, C., Service, S. K., Hartikainen, A.-L., Pouta, A., Ripatti, S., Brodsky, J., Jones, C. G., Zaitlen, N. A., Varilo, T., Kaakinen, M., et al. (2009) · Nature Genetics · doi:10.1038/ng.271 · Cited by 736
Discovering the False Discovery Rate
Benjamini, Y. (2010) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00746.x · Cited by 629
Liu, Y., & Xie, J. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1554485 · Cited by 477
ACAT: A Fast and Powerful p Value Combination Method for Rare-Variant Analysis in Sequencing Studies
Liu, Y., Chen, S., Li, Z., Morrison, A. C., Boerwinkle, E., & Lin, X. (2019) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2019.01.002 · Cited by 450
Metadata
- Authors
- Donoho, D., & Jin, J.
- Venue
- The Annals of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2004-06-01
- Volume
- 32
- Issue
- 3
- Pages
- 962-994
- ISSN
- 0090-5364 (ISSN-L)
- OpenAlex
- W1981106668
- Cited by
- 721
- References
- 19
- FWCI
- 9.51
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.