The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
The harmonic mean p -value for combining dependent tests
Open accessPublished version of a preprint
Abstract
Significance: The widespread use of Bonferroni correction encumbers the scientific process and wastes opportunities for discovery presented by big data, because it discourages exploratory analyses by overpenalizing the total number of statistical tests performed. In this paper, I introduce the harmonic mean p -value (HMP), a simple to use and widely applicable alternative to Bonferroni correction motivated by Bayesian model averaging that greatly improves statistical power while maintaining control of the gold standard false positive rate. The HMP has a range of desirable properties and offers a different way to think about large-scale exploratory data analysis in classical statistics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We then aggregated the values of porig using the harmonic mean p-value to test the global null hypothesis that none of the pairs were statistically inconsistent (Wilson, 2019), yielding an aggregated porig of 0.0005, which is strong evidence of some inconsistency in the tested pairs.”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Wilson, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 3×
“Controlling the FDR guarantees that, among the significant tests, the proportion in which the null hypothesis is incorrectly rejected in favor of the alternative is limited (9).”
BackgroundMethodEstimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Guindon, S., Dufayard, J.-F., Lefort, V., Anisimova, M., Hordijk, W., & Gascuel, O. (2010) · Systematic Biology · doi:10.1093/sysbio/syq010 · Cited by 18,376
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 1×
“The widely used Benjamini–Hochberg (BH) procedure (9) for controlling the FDR shares with the Bonferroni method a robustness to positive correlation between tests (10) but is less conservative.”
MethodStatistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692
Handbook of Personality: Theory and Research
Krueger, R. F., & Johnson, W. (2008) · Cited by 7,884
A haplotype map of the human genome
The International HapMap Consortium (2005) · Nature · doi:10.1038/nature04226 · Cited by 5,858
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
METAL: fast and efficient meta-analysis of genomewide association scans
Willer, C. J., Li, Y., & Abecasis, G. R. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq340 · Cited by 5,205
Research works printed but not linked: 1
- ; The Royal Society Machine Learning: The Power and Promise of Computers That Learn by Example (The Royal Society, London, 2017).
Cited by
Major Impacts of Widespread Structural Variation on Gene Expression and Crop Improvement in Tomato
Alonge, M., Wang, X., Benoit, M., Soyk, S., Pereira, L., Zhang, L., Suresh, H., Ramakrishnan, S., Maumus, F., Ciren, D., et al. (2020) · Cell · doi:10.1016/j.cell.2020.05.021 · Cited by 819
“The adjusted p values for each annotation and tissue type were aggregated using two methods: Fisher’s method and a harmonic mean estimate (Sitgreaves, 1960; Wilson, 2019), and are listed in Table S4A.”
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
Machine learning identifies candidates for drug repurposing in Alzheimer’s disease
Rodriguez, S., Hug, C., Todorov, P., Moret, N., Boswell, S. A., Evans, K., Zhou, G., Johnson, N. T., Hyman, B. T., Sorger, P. K., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21330-0 · Cited by 299
Lee, H., Fenster, R. J., Pineda, S. S., Gibbs, W. S., Mohammadi, S., Davila-Velderrain, J., Garcia, F. J., Therrien, M., Novis, H. S., Gao, F., et al. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.021 · Cited by 279
“We then combined individual p-value s (Wilson, 2019) for each transcription factor to estimate a meta p-value for its activity.”
MethodThe Global Flourishing Study: Study Profile and Initial Results on Flourishing
VanderWeele, T. J., Johnson, B. R., Białowolski, P., Bonhag, R., Bradshaw, M., Breedlove, T., Case, B. W., Chen, Y., Chen, Z. J., Counted, V., et al. (2025) · Nature Mental Health · doi:10.1038/s44220-025-00423-5 · Cited by 253
Metadata
- Authors
- Wilson, D. J.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Wilson, Daniel J.
- ORCID
- Wilson, Daniel J.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2019-01-04
- Volume
- 116
- Issue
- 4
- Pages
- 1195-1200
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6347718
- OpenAlex
- W2907099021
- Cited by
- 449
- References
- 48
- Citations by year
- 2026: 58 · 2025: 115 · 2024: 44 · 2023: 54 · 2022: 61 · 2021: 57 · 2020: 45 · 2019: 12
- FWCI
- 34.33
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-10-07)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Wellcome (101237/Z/13/Z)
- Royal Society (101237/Z/13/Z)
- RCUK | Medical Research Council (MR/K01532X/1)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- family-wise error rate, statistical power, generalized central limit theorem, group testing, genome-wide association study, neuroticism, Benjamini-Hochberg procedure, false discovery rate
- MeSH terms
- Hepatitis C, Humans, Models, Statistical, Hepacivirus, Viral Load, Genome-Wide Association Study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The harmonic mean p -value for combining dependent tests. https://app.cochat.ai/papers/10.1073/pnas.1814092116
- The paper
- Wilson, D. J. (2019). The harmonic mean p -value for combining dependent tests. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1814092116
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.