The harmonic mean p -value for combining dependent tests
Open accessPublished version of a preprint
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Significance: The widespread use of Bonferroni correction encumbers the scientific process and wastes opportunities for discovery presented by big data, because it discourages exploratory analyses by overpenalizing the total number of statistical tests performed. In this paper, I introduce the harmonic mean p -value (HMP), a simple to use and widely applicable alternative to Bonferroni correction motivated by Bayesian model averaging that greatly improves statistical power while maintaining control of the gold standard false positive rate. The HMP has a range of desirable properties and offers a different way to think about large-scale exploratory data analysis in classical statistics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We then aggregated the values of porig using the harmonic mean p-value to test the global null hypothesis that none of the pairs were statistically inconsistent (Wilson, 2019), yielding an aggregated porig of 0.0005, which is strong evidence of some inconsistency in the tested pairs.”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Wilson, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 3×
“Controlling the FDR guarantees that, among the significant tests, the proportion in which the null hypothesis is incorrectly rejected in favor of the alternative is limited (9).”
BackgroundMethodEstimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Guindon, S., Dufayard, J.-F., Lefort, V., Anisimova, M., Hordijk, W., & Gascuel, O. (2010) · Systematic Biology · doi:10.1093/sysbio/syq010 · Cited by 18,376
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 1×
“The widely used Benjamini–Hochberg (BH) procedure (9) for controlling the FDR shares with the Bonferroni method a robustness to positive correlation between tests (10) but is less conservative.”
MethodStatistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692
Handbook of Personality: Theory and Research
Krueger, R. F., & Johnson, W. (2008) · Cited by 7,884
A haplotype map of the human genome
The International HapMap Consortium (2005) · Nature · doi:10.1038/nature04226 · Cited by 5,858
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
METAL: fast and efficient meta-analysis of genomewide association scans
Willer, C. J., Li, Y., & Abecasis, G. R. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq340 · Cited by 5,205
Research works printed but not linked: 1
- The Royal Society 2017 Machine Learning: The Power and Promise of Computers That Learn by Example The Royal Society London
Cited by
Items 301–400 of 460
A Test of the Dependence Assumptions for the Simes-Test-based Multiple Test Procedures
Gou, J. (2023) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2023.2190930 · Cited by 3
“We can combine p -values using other methods (Fisher 1925; Stouffer et al. 1949; Simes 1986; Gou et al. 2014; Gou and Tamhane 2014; Wilson 2019; Gou 2022a, 2022b).”
MethodApplication of Portfolio Optimization to Achieve Persistent Time Series
Zlatniczki, Á., & Telcs, A. (2024) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-024-02426-1 · Cited by 3
Simple tests on multiple correlation coefficient in high-dimensional normal data
Abusaleh, S., & Najarzadeh, D. (2024) · Statistical Methods & Applications · doi:10.1007/s10260-024-00759-9 · Cited by 4
“This test possesses several beneficial properties as outlined by Wilson (2019a), including its robustness to positive dependency between p -values, insensitivity to the specific number of tests conducted, and sensitivity to the smallest p -values.”
InfluentialTwo P or Not Two P: Mendel Random Variables in Combining Fake and Genuine p-Values
Brilhante, M. F., Gomes, M. I., Mendonça, S., Pestana, D., & Santos, R. (2024) · AppliedMath · doi:10.3390/appliedmath4030060 · Cited by 3
“Wilson [66] recommends the use of the harmonic mean p -value, i.e., T H n ( P 1 , . . . , P n ) = n ∑ nk = 1 1/ P k , for combining dependent p -values, since it controls the overall type I error, i.e., the probability of falsely rejecting the overall null hypothesis H ∗ 0 in favor of at least one…”
Sleep-Wake States Are Encoded across Emotion Regulation Regions of the Mouse Brain
Walder-Christensen, K. K., Goffinet, J., Bey, A. L., Syed, R., Benton, J., Mague, S. D., Adamson, E., Vera, S., Soliman, H. A., Kansagra, S. M., et al. (2025) · eNeuro · doi:10.1523/eneuro.0291-25.2025 · Cited by 3
“In comparison to procedures such as Benjamini – Hochberg that control the false discovery rate, the HMP is more sensitive in detecting signi fi cant groups of hypotheses at the expense of being less sensitive in detecting signi fi cant individual hypotheses (Wilson, 2019).”
InfluentialArning, N., Fryer, H. R., & Wilson, D. J. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2514138122 · Cited by 3
Self-citationChildhood experiences and adult self-rated physical health in 22 countries
Bradshaw, M., Kent, B. V., Wortham, J. S., Pertel, N. L., VanderWeele, T. J., & Johnson, B. R. (2026) · BMC Global and Public Health · doi:10.1186/s44263-026-00253-2 · Cited by 3
A cross-national analysis of demographic variation in perceived social trust across 22 countries
Kim, Y.-I., Jang, S. J., VanderWeele, T. J., & Johnson, B. R. (2026) · Scientific Reports · doi:10.1038/s41598-026-56453-1 · Cited by 3
A Python programozási nyelvről statisztikusoknak
Sóti, A. (2020) · Statisztikai Szemle · doi:10.20311/stat2020.4.hu0324 · Cited by 3
Exploring links between pathogen avoidance motivation, COVID ‐19 case counts, and immune function
Muehlenbein, M. P., Gassen, J., Nowak, T. J., Henderson, A. D., Thum, E., Weaver, S. P., & Baker, E. J. (2022) · American Journal of Human Biology · doi:10.1002/ajhb.23833 · Cited by 3
“…harmonic mean p-value (HMP) presents a method for controlling false positive rate in exploratory research with dependent p-values while overcoming the shortcomings of methods that operate poorly under test dependency and/or are excessively conservative (e.g., Bonferroni corrections) (Wilson, 2019).”
MethodInfluentialInferring differential subcellular localisation in comparative spatial proteomics using BANDLEPreprint
Crook, O. M., Davies, C. T. R., Gatto, L., Kirk, P., & Lilley, K. S. (2021) · Research Square · doi:10.21203/rs.3.rs-145772/v1 · Cited by 3
The Lévy combination testPreprint
Wilson, D. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01501 · Cited by 3
“Recently [1] I employed Bayesian model-averaging arguments in the development of the harmonic mean”
BackgroundMethodSelf-citationSystematic identification of disease-associated 3D neighborhoods in protein structuresPreprint
Nason, E., Gerges, S., Satterstrom, F. K., Gorissen, B. L., Liao, R., Panagiotaropoulou, G., Guez, J., Karczewski, K. J., Daly, M. J., & Finucane, H. K. (2026) · medRxiv · doi:10.64898/2026.05.29.26354366 · Cited by 3
Sadeqi, M. B., Ballvora, A., Dadshani, S., Siddiqui, M. N., Kamruzzaman, M., Koua, A. P., & Léon, J. (2025) · Plant Direct · doi:10.1002/pld3.70036 · Cited by 2
“The first attempts to avoid this risk were to consider the threshold as nonlinear concept (Wilson 2019; McCaw et al. 2022; Asif et al. 2020), and q-value (s) was introduced as an alternative approach to the adjusted p-value (s) method for thresholding.”
metacp: a versatile software package for combining dependent or independent p-values
Nikolitsa, E. K., Kontou, P. I., & Bagos, P. G. (2025) · BMC Bioinformatics · doi:10.1186/s12859-025-06126-z · Cited by 5
Yamaguchi, T., Uehara, T., Okadome, T., Mukaino, T., Shimogawa, T., Mukae, N., Shigeto, H., & Isobe, N. (2025) · International Journal of Neural Systems · doi:10.1142/s0129065726500024 · Cited by 2
Automated detection of short-term slow slip events using GNSS data via change-point analysis
Ma, Y.-M., Anastasiou, A., & Montiel, F. (2025) · Geophysical Journal International · doi:10.1093/gji/ggaf517 · Cited by 2
DiagnoDating: diagnostics for dated phylogenies in microbial population genetics
Didelot, X., Carson, J., Ribeca, P., & Volz, E. (2026) · Molecular Biology and Evolution · doi:10.1093/molbev/msag093 · Cited by 2
“We used a false discovery rate (FDR) correction (Benjamini and Hochberg 1995) although other options would also be possible for example using a harmonic mean P -value (Wilson 2019).”
Ma, Y. X., Su, M., Liu, X. H., Wang, W., Wu, Y., Wang, Z., Peng, X., Xiong, X., Lei, J., & Li, Y. (2026) · The Oncologist · doi:10.1093/oncolo/oyag147 · Cited by 2
Microbiome mediation analysis: methods, assumptions, and practical considerations
Jiang, Z., & Li, G. (2026) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2026.1832981 · Cited by 2
“For community-level inference, LDM-med aggregates taxon-speci fi c p values using the harmonic mean method (Wilson, 2019) and assesses signi fi cance through permutation to test the overall mediation effects.”
Jumanne, A. S., Njoroge, J., & Moi, E. J. (2023) · American Journal of Public Policy and Administration · doi:10.47672/ajppa.1327 · Cited by 3
Bounded strategic reasoning explains crisis emergence in multi-agent market games
Evans, B. P., & Prokopenko, M. (2023) · Royal Society Open Science · doi:10.1098/rsos.221164 · Cited by 2
A novel performance scoring quantification framework for stress test set-ups
Kozlovski, T., Hausdorff, J. M., Davidov, O., Giladi, N., Mirelman, A., & Benjamini, Y. (2023) · PLoS ONE · doi:10.1371/journal.pone.0284083 · Cited by 2
“Each measure yielded 10 such pvalues, where the respective minimum and a combined Harmonic Mean P-value (HMP) [32] are provided for.”
Sharma, N. K., Gaznepoglu, Ü. E., Robotham, T., & Habets, E. A. P. (2023) · JASA Express Letters · doi:10.1121/10.0017936 · Cited by 2
Bauermeister, J. A., Lin, W., Tingler, R. C., Liu, A., Chariyalertsak, S., Hoesley, C. J., Gonzáles, P., Ho, K. S., Kayange, N., Palanee‐Phillips, T., et al. (2024) · Journal of the International AIDS Society · doi:10.1002/jia2.26219 · Cited by 2
Furk, D., Silva, L., Dias, M., Fujão, C., Probst, P., Liu, H., & Gamboa, H. F. S. (2024) · Applied Sciences · doi:10.3390/app14093921 · Cited by 2
“For further reliability, the statistical test was repeated 500 times with a different random seed for the sample generation such that the final result equaled the harmonic mean of the p -values [47].”
MethodA general consonance principle for closure tests based on p-values
Zehetmayer, S., König, F., & Posch, M. (2024) · Statistical Methods in Medical Research · doi:10.1177/09622802241269624 · Cited by 2
Bernstein, J. A., Seth, A. K., Rao, M., Katz, J. C., Singh, U. P., Greisner, W. A., Lierl, M. B., Prince, B. T., Mavunda, K., Bernstein, C. K., et al. (2025) · Journal of Allergy and Clinical Immunology Global · doi:10.1016/j.jacig.2025.100427 · Cited by 3
“35 For the subgroup with CASI >_ 4, in a post hoc analysis, multiple imputation chain equation (aka MICE) was used for imputation.”
Yu, X., Zhang, L., Srinivasan, A., Xie, M., & Xue, L. (2025) · Biometrics · doi:10.1093/biomtc/ujaf001 · Cited by 5
“Moreover, the framework subsumes the recently developed dependent combination methods, including the vanilla Cauchy combination and also the closely related works of Wilson (2019) and Fang et al. (2023).”
BackgroundMethodInfluentialElliott, M., Hsu, M., Uddin, L. Q., Modavi, K., & Johnson, S. L. (2025) · Clinical Psychological Science · doi:10.1177/21677026241305377 · Cited by 2
“To guard against violations of the independence assumption required for binomial probabilities, we also calculated the asymptotically exact harmonic-mean-p (HMP) value of the five predicted significant and seven predicted nonsignificant tests (Wilson, 2019).”
InfluentialDistribution of the Harmonic Mean of P-values with Application to Multiple Testing
Gou, J., & Tamhane, A. C. (2025) · Sankhya B · doi:10.1007/s13571-025-00358-y · Cited by 2
“4.1 Wilson’s HMP Procedure Wilson (2019b) proposed the following corrected version of his original HMP procedure Wilson (2019a) In particular, any elementary hypothesis H i is rejected if The following are the issues with this HMP. (a) It tests different subset hypotheses independently without…”
InfluentialVariations in belonging across 22 countries in the Global Flourishing Study
Counted, V., Allen, K., Johnson, B. R., & VanderWeele, T. J. (2025) · The Journal of Positive Psychology · doi:10.1080/17439760.2025.2569083 · Cited by 2
“Within each country, a global test of variation of outcome across levels of each particular demographic variable was conducted and a pooled p-value (Wilson, 2019) across countries reported concerning evidence for variation within any country.”
Wang, Z., Kojima, R., Kiji, R., Fujita, K., Tachibana, R., Tsuchiya, R., Uchiyama, T., Minagawa, Y., Mizuno, T., Igarashi, K., et al. (2026) · Journal of the American Chemical Society · doi:10.1021/jacs.5c14173 · Cited by 2
Cowden, R. G., Padgett, R. N., Felton, C., Wilkinson, R., Macchia, L., Chen, Z. J., Węziak‐Białowolska, D., Koning, S. M., Gundersen, C., Johnson, B. R., et al. (2026) · Communications Medicine · doi:10.1038/s43856-026-01693-5 · Cited by 2
“For each outcome, a global p - value corresponding to an omnibus Wald-type test for evidence of association in any sample is reported 79 .”
Combining dependent p-values by gamma distributions
Chien, L. (2020) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2019-0057 · Cited by 2
Identification of factors impacting on the transmission and mortality of COVID-19
Zhang, P., Dong, T., Li, N., & Liang, F. (2021) · Journal of Applied Statistics · doi:10.1080/02664763.2021.1953449 · Cited by 2
Alt, E. M., Psioda, M. A., & Ibrahim, J. G. (2023) · Statistics in Medicine · doi:10.1002/sim.9696 · Cited by 3
Fang, Y., Chang, C., & Tseng, G. C. (2023) · Statistica Sinica · doi:10.5705/ss.202022.0261 · Cited by 2
“1), one can show that the above procedures are equivalent to directly using L ∑L i=1 1/pi as p-value for statistic 1 L ∑L i=1 1/pi, which is suggested by Wilson (2019).”
MethodMeta-analysis of Genuine and Fake p-Values
Brilhante, M. F., Gomes, M. I., Mendonça, S., Pestana, D., & Santos, R. (2025) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-025-00445-3 · Cited by 2
“Wilson [24] recommends the use of the harmonic mean T H n , in (6), for combining mutually exclusive tests, not necessarily independent, assuming that the p -values being combined are valid.”
Lee, C. G., Kwon, E., Paltzer, J., Okafor, C. N., VanderWeele, T. J., Johnson, B. R., & Kwon, J. (2025) · BMC Public Health · doi:10.1186/s12889-025-23430-8 · Cited by 3
Trade-off between validity and efficiency of merging p-values under arbitrary dependencePreprint
Chen, Y., Liu, P., Tan, K. S., & Wang, R. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3569329 · Cited by 3
“For example, the Fisher method (Fisher (1948)) corresponds to the geometric mean with φ(p) = log(p); the averaging methods of Vovk and Wang (2019) and Wilson (2019) correspond to the functions φ(p) = pr, and r ∈ [−∞,∞] (including limit cases), and the Cauchy combination method of Liu and Xie (2020)…”
Multi-hallmark long noncoding RNA maps reveal non-small cell lung cancer vulnerabilitiesPreprint
Esposito, R., Polidori, T., Meise, D. F., Pulido-Quetglas, C., Chouvardas, P., Forster, S., Schaerer, P., Kobel, A., Schlatter, J., Roemmele, M., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.19.464956 · Cited by 2
Global Patterns of Belonging: A Cross-National Study of 22 CountriesPreprint
Counted, V., Allen, K., Johnson, B. R., & VanderWeele, T. J. (2024) · Research Square · doi:10.21203/rs.3.rs-5292945/v1 · Cited by 2
Hamilton, N. H., Huang, Y.-C. E., McMichael, B. D., Love, M. I., & Furey, T. S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.05.636702 · Cited by 2
“Future work may consider alternative approaches that are more robust to unknown dependencies between p -values [55] and do not treat each as independent supporting evidence.”
Repurposing without Reinventing the Wheel - Ensemble Models for Differential AnalysisPreprint
Chatterjee, S., Paul, E., Liu, Z., Gao, J., Basak, P., Bhattacharyya, A., Banerjee, C., & Mallick, H. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.07.647549 · Cited by 2
Pestana, D., Brilhante, M. F., & Mendonça, S. (2011) · Springer eBooks · doi:10.1007/978-3-662-69359-9_459 · Cited by 2
Wang, Y., Jayaweera, I. M. L. N., & Zou, J. (2025) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.70015 · Cited by 1
“Recent powerful combined p -value tests include the Harmonic Mean p -value (HMP) approach [5] and Cauchy Combination Test (CCT) approach [6], which can handle arbitrary correlation structure among the test statistics although HMP approach does not have the exact additive property shown in (12).”
Comprehensive analysis of mutational processes across 20 000 adult and pediatric tumors
Villa, M., Malighetti, F., Sano, L. D., Villa, A. M., Cordani, N., Aroldi, A., Antoniotti, M., Caravagna, G., Graudenzi, A., Piazza, R., et al. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf648 · Cited by 2
“Moreover, a confidence estimate of the global activities of each signature in the considered dataset can be computed as the harmonic mean P -value [ 31 ] of the results by the Wilcoxon signed-rank test.”
Sheikh, I., Ahmed, S., Kumar, J., Tyagi, V., Vasistha, N. K., Sharma, S., Roy, J. K., & Dhaliwal, H. S. (2025) · Physiological and Molecular Plant Pathology · doi:10.1016/j.pmpp.2025.102978 · Cited by 1
Karanasios, P. I., Schmeller, D. S., Hsu, Z.-T., & Lin, Y.-P. (2026) · Nature Conservation · doi:10.3897/natureconservation.61.171624 · Cited by 1
“To avoid redundancy, Spearman’s correlation coefficient (|r| > 0.7) was applied to remove highly correlated predictors, which can bias variable importance estimation; predictors with stronger HMP support were retained.”
InfluentialDNA methylation signatures associated with early-onset schizophrenia in Chinese patients
Zhan, N., Leung, P. B. M., Zhong, Y., Wong, K. C. Y., Hui, T. C. K., So, H.-C., Sham, P. C., Wong, C. C. Y., & Lui, S. S. Y. (2026) · Translational Psychiatry · doi:10.1038/s41398-026-03869-y · Cited by 1
Zhang, S., Qi, J., Jiang, Y., Lin, H., Wang, X., Wang, T., Cao, H., & Zeng, P. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag103 · Cited by 1
“With Y 0 and G 0 as well as Y 1 and G 1 , the summary statistics of the auxiliary and target populations could be easily obtained using the single-marker linear analysis [42].”
InfluentialGregorio, J. D., Sánchez, D., & Toral, R. (2026) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0307092 · Cited by 1
Park, K., Sun, Z., Liao, R., Bresnick, E. H., & Keleş, S. (2026) · Nature Communications · doi:10.1038/s41467-026-70652-4 · Cited by 1
MACHINE LEARNING BASED BIAS CORRECTION OF CMAQ USING EMISSION SOURCE DECOUPLED FEATURES IN THAILAND
Sukprasert, S. (2026) · International Journal of Geomate · doi:10.21660/2026.139.g15298 · Cited by 4
Berger, A., Latypov, T. H., Suresh, H., Mithani, K., Danthine, V., Wong, S. M., Coleman, S. C., Rutka, J. T., Weiss, S. K., Sham, L., et al. (2026) · Neuromodulation Technology at the Neural Interface · doi:10.1016/j.neurom.2026.03.009 · Cited by 1
A two-timepoint framework for sensitive and specific single-cell activity screening
Ramirez, A., Kyzar, E. J., Rogerson, L., Berland, C., Rodriguez, E., Guerrero, J., Setara, R., Eisengart, M., Virkar, S., Hammond, L. A., et al. (2026) · Neuron · doi:10.1016/j.neuron.2026.05.026 · Cited by 1
薛潇, Feng, Y., Lin, Y., Chan, P.-C., & So, H. (2026) · Influenza and Other Respiratory Viruses · doi:10.1111/irv.70334 · Cited by 1
Utilizing anatomical information for signal detection in functional magnetic resonance imaging
Peitek, N., Brechmann, A., Tabelow, K., & Dickhaus, T. (2025) · F1000Research · doi:10.12688/f1000research.166549.1 · Cited by 1
On Samuel’s p -value model and the Simes test under dependence
Xiong, P., & Hu, T. (2022) · Statistics & Probability Letters · doi:10.1016/j.spl.2022.109509 · Cited by 1
Kim, J., & Park, J. (2023) · Statistica Sinica · doi:10.5705/ss.202023.0117 · Cited by 1
Jumanne, A. S., Njoroge, J., & Moi, E. J. (2023) · American Journal of Public Policy and Administration · doi:10.47672/ajppa.1702 · Cited by 1
“In iaddition, composite index for ithe ivariables iof ithe istudy iwere icomputed by iharmonic imean formula (Wilson, 2019).”
MethodDid Karl Marx party in 1891? The effect of SPD 's Erfurt program on Karl Marx's citations
Magness, P., & Makovi, M. (2024) · Southern Economic Journal · doi:10.1002/soej.12747 · Cited by 1
Too Many, Too Improbable: Testing joint hypotheses and closed testing shortcuts
Mogensen, P. B., & Markussen, B. (2025) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2025.106311 · Cited by 1
“In recent years the Cauchy Combination Test (Liu and Xie, 2020) and the Harmonic Mean p-value (Wilson, 2019) have been proposed and Vovk and Wang (2020) derive a large family of combination tests by using the Kolmogorov generalized f -mean.”
BackgroundMethodInfluentialA one-shot, lossless algorithm for cross-cohort learning in mixed-outcomes analysis
Li, R., Benz, L., Duan, R., Denny, J. C., Hakonarson, H. H., Mosley, J. D., Smoller, J. W., Wei, W., Lumley, T., Ritchie, M. D., et al. (2025) · Patterns · doi:10.1016/j.patter.2025.101321 · Cited by 1
“PheWAS methods combining p values via ACAT or HMP obtained higher power than any possible score-based test in the fully dense setting with positive phenotype correlation, but the power of all PheWAS methods was dramatically influenced by the direction of correlation.”
Religious Reading across Borders and Demographics: A Study of 22 Countries
Weziak-Bialowolska, D., Białowolski, P., Cowden, R. G., Fogleman, A., Bradshaw, M., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2026) · Studia Religiologica · doi:10.4467/20844077sr.26.002.23351 · Cited by 1
Mapping Religious and Spiritual Comfort Worldwide: Demographic Patterns Across 22 Countries
Weziak-Bialowolska, D., Cowden, R. G., Paltzer, J., Białowolski, P., Bradshaw, M., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2026) · Critical Research on Religion · doi:10.1177/20503032261451246 · Cited by 1
Ferté, T., Villain, L., Thiébaut, R., & Hejblum, B. P. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag262 · Cited by 1
“The Cauchy and Harmonic Mean tests rely on P -value combination approaches [ 30– 32].”
Nakamura, J. S., Wilkinson, R., Woodberry, R. D., Weziak-Bialowolska, D., Shiba, K., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2026) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-026-00312-x · Cited by 2
“For each outcome, a global p -value for an omnibus Wald-type test of evidence of association in any sample is reported (Wilson, 2019).”
Higher Criticism Tuned Regression For Weak And Sparse SignalsPreprint
Jiang, T., London, S. J., Lee, M. K., Mychaleckyj, J. C., & Motsinger‐Reif, A. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00130 · Cited by 1
“These B p -values are aggregated by harmonic mean p -value Wilson (2019).”
BackgroundMethodStabilizing selection of seasonal influenza receptor binding in populations with partial immunityPreprint
Hay, J. A., Junus, A., Riley, S., & Yuan, H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.27.175190 · Cited by 1
A New Approach to Testing Mediation of the Microbiome using the LDMPreprint
Yue, Y., & Hu, Y. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.12.468449 · Cited by 1
“Here we chose the p-value to be Zj = max(p1,j, p2,j), and we adopt the Harmonic mean method [30] to aggregate Zjs, which is more robust to the dependence structure among taxa than Fisher’s method.”
MethodIF1 Protein Controls Aging RatePreprint
Forrest, M. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.28.466310 · Cited by 1
The superior colliculus response space has globally high– and locally low-dimensionalityPreprint
Schwartz, O. S., & Yonehara, K. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.06.565916 · Cited by 1
harmonicmeanp: Harmonic Mean p-Values and Model Averaging by Mean Maximum Likelihood
Wilson, D. J. (2017) · doi:10.32614/cran.package.harmonicmeanp · Cited by 1
Self-citationSex-Specific Differential DNA Methylation in Mild Cognitive Impairment and Alzheimer’s DiseasePreprint
Bergmann, P. E., Madrid, A., Papale, L. A., Clark, L. R., Asthana, S., Johnson, S. C., Keleş, S., Hogan, K. J., & Alisch, R. S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.05.15.654279 · Cited by 1
“33 The harmonic mean P -value controls the family-wise error rate of multiple statistical tests and does not assume independence among tests.”
D’Archivio, S., Brusini, L., Trindade, S., Figueiredo, L. M., Gadelha, C., & Wickstead, B. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.27.734978 · Cited by 1
Geirnaert, S., Bertrand, A., Francart, T., & Vanthornhout, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.10887 · Cited by 1
“Multiple comparisons can be naturally controlled by combining p -values using the harmonic mean (Wilson, 2019).”
Exploring allele-specific expression mechanisms in health and disease
Goovaerts, T. (2020) · Ghent University Academic Bibliography (Ghent University)
Umeton, R., Bellucci, G., Bigi, R., Romano, S., Buscarinu, M. C., Reniè, R., Rinaldi, V., Umeton, R. P., Morena, E., Romano, C., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.12.434773
Cinar, O. (2021) · Research Publications (Maastricht University) · doi:10.26481/dis.20210519oc
Did the Reform Work? Evidence from VIX Settlement AnomaliesPreprint
Baumgartner, T., & Guettler, A. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3874249
Richmanz, S. C., Lyman, C. A., Morris, M. C., Caoy, H., Nesterovay, A., Yuryevy, A., Cheadley, C., & Broderick, G. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669536
Anderson relaxation test for intrinsic dimension selection in model-based clustering
Kim, N., & Browne, R. P. (2022) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2022.2069769
Temporal Characteristics of High-Frequency Oscillations as a Biomarker of Human Epilepsy
Scott, J. (2021) · Deep Blue (University of Michigan) · doi:10.7302/1506
Housing Returns and Intertemporal Substitution in Consumption: Estimates for Industrial EconomiesPreprint
Pozzi, L. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4179082 · Cited by 1
“From Wilson (2019), with N = 15 and at the 5% level of significance, the critical value is about 0.04, i.e., the null hypothesis of the test is rejected at the 5% level of significance if the HMP is below 0.04.”
BackgroundMethodResultInfluentialAnnotation-agnostic discovery of associations between novel gene isoforms and phenotypesPreprint
Hjörleifsson, K. E., Pachter, L. S., & Melsted, P. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.02.518787
“…same gene are not generally independent, the resulting p -values for the individual unitigs are aggregated using the Harmonic Mean P-value (HMP) (Wilson, 2019) with weights equal to the log-transformed mean counts normalized to 1, i.e. given a dBG with unitigs with mean counts 𝑁 , the weight…”
MethodInfluentialJumanne, A. S., Njoroge, J., & Moi, E. J. (2023) · American Journal of Public Policy and Administration · doi:10.47672/ajppa.1324
MULTIPLE TESTING CORRECTION IN TIME SERIES ROLLING WINDOW ANALYSIS WITH APPLICATION OF GWAS METHODS
Wang, S. (2022) · Digital Commons - Michigan Tech (Michigan Technological University) · doi:10.37099/mtu.dc.etdr/1468
Qin, T., Lee, C., Li, S., Cavalcante, R. G., Orchard, P., Yao, H., Zhang, H., Wang, S., Patil, S. B., Boyle, A. P., et al. (2022) · Figshare · doi:10.6084/m9.figshare.19663872
mvMAPIT: Multivariate Genome Wide Marginal Epistasis Test
Stamp, J., & Crawford, L. A. (2022) · doi:10.32614/cran.package.mvmapit
Wang, Z., Gu, H., Qin, P., & Wang, J. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e33582
The Ashley and Patterson (1986) test for serial independence in daily stock returns, revisited
Ashley, R., & Najafi, F. (2024) · Annals of Operations Research · doi:10.1007/s10479-024-06355-0
“One of these methods – notably the “HMP P-value Combination Method” put forth by Wilson (2019) is very promising in this context, and may well represent a future path for the topic at hand.”
Metadata
- Authors
- Wilson, D. J.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Wilson, Daniel J.
- ORCID
- Wilson, Daniel J.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2019-01-04
- Volume
- 116
- Issue
- 4
- Pages
- 1195-1200
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6347718
- OpenAlex
- W2907099021
- Cited by
- 449
- References
- 48
- Citations by year
- 2026: 58 · 2025: 115 · 2024: 44 · 2023: 54 · 2022: 61 · 2021: 57 · 2020: 45 · 2019: 12
- FWCI
- 34.33
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-10-07)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Wellcome (101237/Z/13/Z)
- Royal Society (101237/Z/13/Z)
- RCUK | Medical Research Council (MR/K01532X/1)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- family-wise error rate, statistical power, generalized central limit theorem, group testing, genome-wide association study, neuroticism, Benjamini-Hochberg procedure, false discovery rate
- MeSH terms
- Hepatitis C, Humans, Models, Statistical, Hepacivirus, Viral Load, Genome-Wide Association Study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.