The harmonic mean p -value for combining dependent tests
Open accessPublished version of a preprint
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Significance: The widespread use of Bonferroni correction encumbers the scientific process and wastes opportunities for discovery presented by big data, because it discourages exploratory analyses by overpenalizing the total number of statistical tests performed. In this paper, I introduce the harmonic mean p -value (HMP), a simple to use and widely applicable alternative to Bonferroni correction motivated by Bayesian model averaging that greatly improves statistical power while maintaining control of the gold standard false positive rate. The HMP has a range of desirable properties and offers a different way to think about large-scale exploratory data analysis in classical statistics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“We then aggregated the values of porig using the harmonic mean p-value to test the global null hypothesis that none of the pairs were statistically inconsistent (Wilson, 2019), yielding an aggregated porig of 0.0005, which is strong evidence of some inconsistency in the tested pairs.”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Wilson, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 3×
“Controlling the FDR guarantees that, among the significant tests, the proportion in which the null hypothesis is incorrectly rejected in favor of the alternative is limited (9).”
BackgroundMethodEstimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Guindon, S., Dufayard, J.-F., Lefort, V., Anisimova, M., Hordijk, W., & Gascuel, O. (2010) · Systematic Biology · doi:10.1093/sysbio/syq010 · Cited by 18,376
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 1×
“The widely used Benjamini–Hochberg (BH) procedure (9) for controlling the FDR shares with the Bonferroni method a robustness to positive correlation between tests (10) but is less conservative.”
MethodStatistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692
Handbook of Personality: Theory and Research
Krueger, R. F., & Johnson, W. (2008) · Cited by 7,884
A haplotype map of the human genome
The International HapMap Consortium (2005) · Nature · doi:10.1038/nature04226 · Cited by 5,858
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
METAL: fast and efficient meta-analysis of genomewide association scans
Willer, C. J., Li, Y., & Abecasis, G. R. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq340 · Cited by 5,205
Research works printed but not linked: 1
- The Royal Society 2017 Machine Learning: The Power and Promise of Computers That Learn by Example The Royal Society London
Cited by
Items 101–200 of 460
Fast, flexible analysis of differences in cellular composition with crumblr
Hoffman, G. E., & Roussos, P. (2026) · Nature Communications · doi:10.1038/s41467-026-75681-7 · Cited by 18
“In order to study immune response to SARS-CoV-2 infection, the COMBAT Consortium 32 collected blood from patients hospitalized with COVID-19, along with blood from non-hospitalized COVID-19 patients, healthy controls, and patients with sepsis or fl u.”
Lomas, T., Padgett, R. N., Ritchie‐Dunham, J. L., Pawelski, J. O., Shiba, K., Johnson, B. R., & VanderWeele, T. J. (2025) · Applied Research in Quality of Life · doi:10.1007/s11482-024-10407-9 · Cited by 17
“Within each country, a global test of variation of outcome across levels of each particular demographic variable was conducted, and a pooled p-value (Wilson, 2019) across countries reported concerning evidence for variation within any country.”
Polygenic Selection within a Single Generation Leads to Subtle Divergence among Ecological Niches
Ehrlich, M. A., Wagner, D. N., Oleksiak, M. F., & Crawford, D. L. (2020) · Genome Biology and Evolution · doi:10.1093/gbe/evaa257 · Cited by 19
The effect of unpredictability on the perception of pain: a systematic review and meta-analysis
Pavy, F., Zaman, J., Noortgate, W. V. D., Scarpa, A., von Leupoldt, A., & Torta, D. M. E. (2024) · Pain · doi:10.1097/j.pain.0000000000003199 · Cited by 18
A powerful and versatile colocalization test
Deng, Y., & Pan, W. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007778 · Cited by 19
“In addition, we tried the harmonic mean p-value (HMP) by [42] for the conditional method to replace the Bonferroni correction, but the improvement was very limited (so we did not include the results).”
MethodDuddek, P., Papritz, A., Ahmed, M. A., Lovrić, G., & Carminati, A. (2024) · Plant and Soil · doi:10.1007/s11104-024-06582-1 · Cited by 16
“The resulting sample-wise p-values of the DCLF-tests were pooled bycalculatingtheirharmonicmeanusingthe‘hmp.stat’ function of the ‘harmonicmeanp’ package (Wilson 2019).”
Cowden, R. G., Worthington, E. L., Chung, C. A., Kock, J. H. D., Węziak‐Białowolska, D., Yancey, G., Shiba, K., Padgett, R. N., Bradshaw, M., Johnson, B. R., et al. (2025) · Scientific Reports · doi:10.1038/s41598-024-82502-8 · Cited by 17
Childhood predictors of volunteering across 22 countries in the Global Flourishing Study
Nakamura, J. S., Woodberry, R. D., Gibson, C. B., Lee, M. T., Kim, Y.-I., Shiba, K., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-024-81639-w · Cited by 16
Chen, H., Zhang, J., Wang, T., Zhang, S., Lai, Q., Huang, S., & Zeng, P. (2021) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgab465 · Cited by 17
Friction is preferred over grasp configuration in precision grip grasping
Klein, L. K., Maiello, G., Fleming, R. W., & Voudouris, D. (2021) · Journal of Neurophysiology · doi:10.1152/jn.00021.2021 · Cited by 17
The harmonic mean p -value: Strong versus weak control, and the assumption of independence
Goeman, J. J., Rosenblatt, J. D., & Nichols, T. E. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1909339116 · Cited by 16
Visual Inference and Graphical Representation in Regression Discontinuity DesignsPreprint
Korting, C., Lieberman, C., Matsudaira, J. D., Pei, Z., & Shen, Y. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4114427 · Cited by 17
Long-read RNA sequencing reveals allele-specific N 6 -methyladenosine modifications
Park, D., & Cenik, C. (2024) · Genome Research · doi:10.1101/gr.279270.124 · Cited by 17
PhyloMed: a phylogeny-based test of mediation effect in microbiome
Hong, Q., Chen, G., & Tang, Z.-Z. (2023) · Genome biology · doi:10.1186/s13059-023-02902-3 · Cited by 16
“(HMP) combination method [30] in the global mediation test.”
Lomas, T., Padgett, R. N., Ritchie‐Dunham, J. L., Lee, M. T., Pawelski, J. O., Shiba, K., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-024-83353-z · Cited by 16
“Within each country, a global test of association of each childhood predictor variable group with outcome was conducted, and a pooled p-value 58 across countries reported concerning evidence for association within any country.”
Yin, L., Feng, Y., Shi, Y., Lau, A., Qiu, J., Sham, P. C., & So, H. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00906-7 · Cited by 15
Infinite-mean models in risk management: Discussions and recent advances
Chen, Y., & Wang, R. (2024) · Risk sciences. · doi:10.1016/j.risk.2024.100003 · Cited by 14
“…are (i) Fisher’s combination method (Fisher (1948)): ϕ ( x ) = − log( x ) for x ∈ [0 , 1], and w = (1 /n, . . . , 1 /n ); (ii) the harmonic mean method (Wilson (2019)): ϕ ( x ) = 1 /x for x ∈ [0 , 1]; (iii) the generalized mean method (Vovk and Wang (2020)): ϕ ( x ) = x r for x ∈ [0 , 1], where…”
MethodThe distribution of optimism across sociodemographic groups in 22 countries
Chen, Y., Kubzansky, L. D., Kim, E. S., Koga, H. K., Shiba, K., Padgett, R. N., Wilkinson, R., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-024-77257-1 · Cited by 16
“Additional methodological details are reported elsewhere 30,32 .”
Demographic variation in pain across 22 countries
Macchia, L., Okafor, C. N., Breedlove, T., Shiba, K., Piper, A., Johnson, B. R., & VanderWeele, T. J. (2025) · Communications Medicine · doi:10.1038/s43856-025-00858-y · Cited by 15
Seeds of Hope: Cross-National Analysis of Childhood Predictors of Hope in 22 Countries
Counted, V., Long, K. N. G., Cowden, R. G., Witvliet, C. V., Gibson, C. B., Cortright, A., Walsh, J. A., Purcell, E., Garzon, F. L., Hathaway, W., et al. (2025) · Applied Research in Quality of Life · doi:10.1007/s11482-025-10450-0 · Cited by 14
“Within each country, a global test of association of each childhood predictor variable group with the outcome of hope was conducted, and a pooled p-value (Wilson, 2019) across countries reported concerning evidence for association within any country.”
Barthelson, K., Newman, M., & Lardelli, M. T. (2022) · Disease Models & Mechanisms · doi:10.1242/dmm.049187 · Cited by 16
“We used the harmonic mean p-value to determine the overall significance of changes to gene expression within gene sets because this method does not assume that component p-values are independent (Wilson, 2019).”
MethodInfluentialShao, Z., Wang, T., Qiao, J., Zhang, Y., Huang, S., & Zeng, P. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04897-3 · Cited by 14
Large‐scale simultaneous inference under dependence
Tian, J., Chen, X., Katsevich, E., Goeman, J. J., & Ramdas, A. (2022) · Scandinavian Journal of Statistics · doi:10.1111/sjos.12614 · Cited by 14
“Particularly, for r ∈ {−∞, 0,∞}, we define αr,m as limr→∞(r + 1)1/r = 1, limr→0(r + 1)1/r = e, and limr→−∞ rr+1m 1+1/r = m. Follow-up work [13, 14] explored the conservativeness of such calibration under some special dependence structure: Wilson [9] derived asymptotic valid (in the sense of m → ∞) calibration under in…”
BackgroundMethodInfluentialWyenberg-Henzler, T., Patterson, R. T., & Mallon, J. C. (2021) · Historical Biology · doi:10.1080/08912963.2021.2010191 · Cited by 14
“This method is capable of handling large datasets and controls for familywise error rate (Wilson 2019).”
MethodA generalised seasonality test and applications for cryptocurrency and stock market seasonality
Shanaev, S., & Ghimire, B. (2022) · The Quarterly Review of Economics and Finance · doi:10.1016/j.qref.2022.07.002 · Cited by 15
Kriukov, D., Khrameeva, E. E., Gladyshev, V. N., Dmitriev, S. E., & Tyshkovskiy, A. (2022) · doi:10.1101/2022.12.12.520058 · Cited by 13
“Aggregation of p-values across the signatures 562 The aggregation of p-values within each group of signatures was conducted using the harmonic 563 mean (72).”
MethodKim, E. S., Bradshaw, M., Padgett, R. N., Chen, Y., Shiba, K., Ritchie‐Dunham, J. L., Case, B. W., Johnson, B. R., & VanderWeele, T. J. (2025) · npj Mental Health Research · doi:10.1038/s44184-025-00127-9 · Cited by 13
“Withineachcountry,aglobaltestofassociationofeachchildhood correlatevariablegroupwithoutcomewasconducted,andapooled p -value 92 across countries reported concerning evidence for association within any country.”
A cross-national analysis of demographic variation in self-rated mental health across 22 countries
Okafor, C. N., Jean-Berluche, D., Paltzer, J., Kwon, E., Grigoropoulou, N., VanderWeele, T. J., & Johnson, B. R. (2025) · Communications Medicine · doi:10.1038/s43856-025-01038-8 · Cited by 14
Combining independent p-values in replicability analysis: a comparative study
Hoang, A., & Dickhaus, T. (2022) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2021.2022678 · Cited by 20
Yano, K., & Kano, M. (2022) · Journal of Geophysical Research Solid Earth · doi:10.1029/2021jb023258 · Cited by 13
“Finally, we combine station-wise nonconfidence values 𝐴𝐴 { 𝑝𝑝𝑠𝑠 ′ 𝑖𝑖 ∶ 𝑠𝑠′ ∈ secondary stations } by using the combined p-value technique (Good, 1958; Wilson, 2019): 𝐴𝐴 𝐴𝐴−1 𝑖𝑖 = ∑ 𝑠𝑠′ 1∕𝐴𝐴𝑠𝑠 ′ 𝑖𝑖 .”
Respiratory brain impulse propagation in focal epilepsy
Elabasy, A., Suhonen, M., Rajna, Z., Hosni, Y., Kananen, J., Annunen, J., Ansakorpi, H., Korhonen, V. O., Seppänen, T., & Kiviniemi, V. J. (2023) · Scientific Reports · doi:10.1038/s41598-023-32271-7 · Cited by 13
Combining exchangeable P -values
Gasparin, M., Wang, R., & Ramdas, A. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2410849122 · Cited by 27
“The harmonic mean p-value was studied by Wilson (2019).”
Valgardson, J. D., Cosbey, R., Houser, P. R., Rupp, M., Bronkhorst, R. V., Lee, M., Jagodzinski, F., & Amacher, J. F. (2019) · Protein Science · doi:10.1002/pro.3741 · Cited by 13
“105”
Zhu, Z., Satten, G. A., & Hu, Y. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac181 · Cited by 14
Bellucci, G., Umeton, R., Bigi, R., Romano, S., Buscarinu, M. C., Reniè, R., Rinaldi, V., Umeton, R. P., Morena, E., Romano, C., et al. (2021) · Scientific Reports · doi:10.1038/s41598-022-11444-w · Cited by 14
Barrett, A., Varol, E., Weinreb, A., Taylor, S. R., McWhirter, R. M., Cros, C., Vidal, B., Basavaraju, M., Poff, A., Tipps, J. A., et al. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.05.487209 · Cited by 13
Averaging p -values under exchangeability
Choi, W., & Kim, I. (2022) · Statistics & Probability Letters · doi:10.1016/j.spl.2022.109748 · Cited by 19
Religious centrality across 22 countries
Woodberry, R. D., Johnson, K. A., Case, B. W., Bradshaw, M., VanderWeele, T. J., & Johnson, B. R. (2025) · Scientific Reports · doi:10.1038/s41598-025-99183-6 · Cited by 14
Nakamura, J. S., Woodberry, R. D., Węziak‐Białowolska, D., Kubzansky, L. D., Shiba, K., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-024-77950-1 · Cited by 14
Cowden, R. G., Worthington, E. L., Węziak‐Białowolska, D., Yancey, G., Witvliet, C. V., Shiba, K., Padgett, R. N., Bradshaw, M., Johnson, B. R., & VanderWeele, T. J. (2025) · Applied Research in Quality of Life · doi:10.1007/s11482-025-10451-z · Cited by 12
“A pooled p -value (Wilson, 2019) was used across countries to report whether there is evidence of an asso - ciation in at least one country.”
The roots of belonging: childhood predictors of belonging in 22 countries
Counted, V., Allen, K., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-14410-4 · Cited by 13
Chen, Y., Case, B. W., Long, K. N. G., Woodberry, R. D., Kim, E. S., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-22806-5 · Cited by 13
“We reported pooled global p-values 63 across countries to evaluate whether the proportion estimates varied statistically significantly by each sociodemographic variable within at least one country.”
Lee, M. T., Wilkinson, R., Long, K. N. G., Case, B. W., Ritchie‐Dunham, J. L., Bradshaw, M., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-31380-9 · Cited by 12
“…of love/care expression across levels of each sociodemographic characteristic (or childhood predictor) was conducted, and a pooled p value 74 was reported across countries to evaluate if a given association for the sociodemographic characteristic (or childhood predictor) with love/care…”
Stamp, J., DenAdel, A., Weinreich, D. M., & Crawford, L. A. (2023) · G3 Genes Genomes Genetics · doi:10.1093/g3journal/jkad118 · Cited by 16
“The second ap - proach assumes an unknown dependency structure be-tween each of the T tests and computes a harmonic mean (Wilson 2019a) P -value, where ˚ p = t w t / t w t / p t .”
Peters, B., Wiedrick, J. T., & Baylor, C. R. (2023) · American Journal of Speech-Language Pathology · doi:10.1044/2023_ajslp-22-00346 · Cited by 12
Multi-Dimensional Anomalous Entity Detection via Poisson Tensor Factorization
Eren, M. E., Moore, J. S., & Alexandrov, B. S. (2020) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi49825.2020.9280524 · Cited by 12
“We achieve this summarization using p-value fusion of dependent p-values [20].”
MethodBarthelson, K., Pederson, S., Newman, M., & Lardelli, M. T. (2021) · Journal of Alzheimer s Disease · doi:10.3233/jad-201383 · Cited by 14
“Since each method gave different levels of significance, we combined the raw p -values from each method by calculating the harmonic mean p -value [27].”
Barthelson, K., Dong, Y., Newman, M., & Lardelli, M. T. (2021) · Journal of Alzheimer s Disease · doi:10.3233/jad-210128 · Cited by 14
“Our method, inspired by the EGSEA framework [74], involves 341 calculation of the harmonic mean p-value [75] from the raw p-values of three different rank342 based gene set testing methods: fry [58], camera [59] and GSEA [60, 61].”
MethodOn the Bayesian interpretation of the harmonic mean p -value
Held, L. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900671116 · Cited by 12
Rustamov, R. M., & Majumdar, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15285 · Cited by 12
“Hypothesis testing is performed by conducting t-tests on each dimension of the embedding and combining the resulting p-values using recently introduced p-value combination techniques that are robust to dependencies [15, 35, 24].”
BackgroundMethodProteomic Assessment of the Risk of Secondary Cardiovascular Events among Individuals with CKD
Deo, R., Dubin, R. F., Ren, Y., Wang, J., Feldman, H. I., Shou, H., Coresh, J., Grams, M. E., Surapaneni, A., Cohen, J. B., et al. (2024) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000502 · Cited by 11
“The overall performance of the proteomics model was compared with the clinical or hybrid (comprising proteomic and clinical variables) model using the harmonic mean P value approach 26 to generate a combined P value that accommodates dependent tests.”
Exploring the Morality of Tax Morale
Puklavec, Ž., Stavrova, O., Kogler, C., & Zeelenberg, M. (2025) · Journal of Community & Applied Social Psychology · doi:10.1002/casp.70042 · Cited by 11
Childhood predictors of suffering in adulthood across 22 countries
Cowden, R. G., Chen, Z. J., Wilkinson, R., Węziak‐Białowolska, D., Breedlove, T., Gundersen, C., Shiba, K., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2025) · Communications Medicine · doi:10.1038/s43856-025-00913-8 · Cited by 11
Where Hope Thrives: Demographic Variation in Hope across 22 Countries
Counted, V., Long, K. N. G., Cowden, R. G., Witvliet, C. V., Gibson, C. B., Cortright, A., Purcell, E., Walsh, J. A., Hathaway, W., Garzon, F. L., et al. (2025) · Journal of Happiness Studies · doi:10.1007/s10902-025-00981-6 · Cited by 11
“Within each country, a global test of variation of outcome across levels of each particular demographic variable was conducted, and a pooled p-value (Wilson, 2019) across countries reported concerning evidence for variation within any country.”
Bradshaw, M., Shiba, K., Jang, S. J., Kent, B. V., Bonhag, R., Johnson, B. R., & VanderWeele, T. J. (2026) · Communications Medicine · doi:10.1038/s43856-025-01366-9 · Cited by 14
Distribution-free changepoint detection tests based on the breaking of records
Castillo‐Mateo, J. (2022) · Environmental and Ecological Statistics · doi:10.1007/s10651-022-00539-2 · Cited by 11
“, the harmonic mean p-value by Wilson (2019) could be used to have a single p-value of all tests.”
BackgroundMethodMusculoskeletal Model to Predict Muscle Activity During Upper Limb Movement
Tamilselvam, Y. K., Ganguly, J., Patel, R. V., & Jog, M. (2021) · IEEE Access · doi:10.1109/access.2021.3103438 · Cited by 11
“combined using the harmonic mean method [38] to obtain a muscle-wise p-value.”
MethodChildhood predictors of balance in life: a cross-national analysis of the global flourishing study
Lomas, T., Padgett, R. N., Ritchie‐Dunham, J. L., Pawelski, J. O., Shiba, K., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-89853-w · Cited by 11
“Within each country, a global test of association of each childhood predictor variable group with outcome was conducted, and a pooled p-value 69 across countries reported concerning evidence for association within any country.”
Decision-making with multiple correlated binary outcomes in clinical trials
Kavelaars, X., Mulder, J., & Kaptein, M. C. (2020) · Statistical Methods in Medical Research · doi:10.1177/0962280220922256 · Cited by 11
“Since a (combined) p-value may be difficult to compute for the multivariate Bernoulli model, Bayesian computational procedures can exploit this equivalence and facilitate computations involved in Type I error control [12, 13].”
MethodBarthelson, K., Pederson, S., Newman, M., Jiang, H., & Lardelli, M. T. (2021) · Journal of Alzheimer s Disease Reports · doi:10.3233/adr-200279 · Cited by 13
“Calculation of the harmonic mean p -value is a method robust to dependent tests [34] and has been validated previously on simulated datasets [20].”
Burger, T. (2023) · PROTEOMICS · doi:10.1002/pmic.202200406 · Cited by 10
“Many p-value combination methods endowed with proof of correct calibration exist in the literature (Harmonic mean [39], Fisher combination [40], etc.”
MethodGliko, O., Mallory, M., Dalley, R. A., Gala, R., Gornet, J., Zeng, H., Sorensen, S. A., & Sümbül, U. (2024) · Nature Communications · doi:10.1038/s41467-024-50728-9 · Cited by 10
Childhood experiences and adult prayer or meditation in 22 countries around the world
Bradshaw, M., Counted, V., Lomas, T., Woodberry, R. D., VanderWeele, T. J., & Johnson, B. R. (2025) · Scientific Reports · doi:10.1038/s41598-025-99796-x · Cited by 10
“Within each country, a global test of association of each childhood predictor variable group with P/M was conducted, and a pooled p-value across countries was reported concerning evidence for association within any country 65 .”
A cross-national analysis of demographic variation in daily smoking across 22 countries
Jang, S. J., de la Rosa, P. A., Padgett, R. N., Bradshaw, M., VanderWeele, T. J., & Johnson, B. R. (2025) · Scientific Reports · doi:10.1038/s41598-024-76318-9 · Cited by 10
“…various aspects of well-being such as happiness and life satisfaction, physical and mental health, meaning and purpose, character and virtue, close social relationships, and financial and material stability 40 , along with other demographic, social, economic, political, religious, personality,…”
Ritchie‐Dunham, J. L., Yancey, G., Managi, S., Bartel, C. A., Bonhag, R., Padgett, R. N., Shiba, K., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-09953-5 · Cited by 10
“Within each country, a global test of association of each childhood predictor variable group with outcome was conducted, and a pooled p-value 55 across countries reported concerning evidence for association within any country.”
Sociodemographic Variation in Gratitude Using a Cross-National Analysis with 22 Countries
Okuzono, S. S., Cowden, R. G., Padgett, R. N., Yancey, G., Johnson, B. R., & VanderWeele, T. J. (2025) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-025-00254-w · Cited by 10
“Within each country, a global test of variation of outcome across levels of each particular demographic variable 1 3 was conducted, and a pooled p -value across countries reported evidence of variation within at least one country (i.e., global p -value) (Wilson, 2019 ).”
Association of the IceCube neutrinos with CAZ blazar light curves
Kouch, P. M., Hovatta, T., Lindfors, E. J., Liodakis, I., Koljonen, K. I. I., & Paggi, A. (2026) · Astronomy and Astrophysics · doi:10.1051/0004-6361/202557584 · Cited by 10
Heavy-Tailed Distribution for Combining Dependent P-Values With Asymptotic Robustness
Fang, Y., Chang, C., Park, Y., & Tseng, G. C. (2022) · Statistica Sinica · doi:10.5705/ss.202022.0046 · Cited by 10
“Wilson (2019b), Wilson (2020), and Vovk, Wang and Wang (2022) also did related work involving combining dependent p -values.”
BackgroundMethodInfluentialA Comparison of Methods for Gene-Based Testing That Account for Linkage Disequilibrium
Cinar, O., & Viechtbauer, W. (2022) · Frontiers in Genetics · doi:10.3389/fgene.2022.867724 · Cited by 10
The impact of violating the independence assumption in meta-analysis on biomarker discovery
Aghababazadeh, F. A., Xu, W. M., & Haibe‐Kains, B. (2023) · Frontiers in Genetics · doi:10.3389/fgene.2022.1027345 · Cited by 11
“Moreover, to combine p -values under arbitrary dependency structures, the Cauchy combination test (Liu et al., 2019; Liu and Xie, 2020) and the harmonic mean p -value (Wilson, 2019) were proposed.”
InfluentialCombined flexion and compression negatively impact the mechanical integrity of the annulus fibrosus
Briar, K. J., & Gregory, D. E. (2023) · European Spine Journal · doi:10.1007/s00586-022-07518-y · Cited by 10
“05 and post hoc multiple pairwise comparisons using a Harmonic Mean p value correction were conducted to further analyse significant effects [18].”
MethodRobot-assisted investigation of sensorimotor control in Parkinson's disease
Tamilselvam, Y. K., Jog, M., & Patel, R. V. (2023) · Scientific Reports · doi:10.1038/s41598-023-31299-z · Cited by 10
Aggregating dependent signals with heavy-tailed combination tests
Gui, L., Jiang, Y., & Wang, J. (2025) · Biometrika · doi:10.1093/biomet/asaf038 · Cited by 15
Overs, B. J., Lenroot, R., Roberts, G. M. P., Green, M. J., Toma, C., Hadzi-Pavlovic, D., Pierce, K. D., Schofield, P. R., Mitchell, P. B., & Fullerton, J. M. (2021) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2021.111258 · Cited by 10
Kawaguchi, E. S., Li, G., Lewinger, J. P., & Gauderman, W. J. (2022) · Statistics in Medicine · doi:10.1002/sim.9319 · Cited by 11
MiRKAT-MC: A Distance-Based Microbiome Kernel Association Test With Multi-Categorical Outcomes
Jiang, Z., He, M., Chen, J., Zhao, N., & Zhan, X. (2022) · Frontiers in Genetics · doi:10.3389/fgene.2022.841764 · Cited by 12
“We propose an omnibus test that combines the individual p -values from tests with different kernels through the harmonic mean procedure (HMP) (Wilson, 2019).”
MethodInfluentialWilson, D. J. (2020) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15761.1 · Cited by 10
Self-citationThe Harmonic Meanχ2-Test to Substantiate Scientific Findings
Held, L. (2020) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/rssc.12410 · Cited by 13
“It is worth noting that the proposed method is different from the harmonic mean p-value (Good, 1958; Wilson, 2019), where the null distribution is more difficult to compute.”
MethodA cross-national analysis of sociodemographic variation in educational attainment
Padgett, R. N., Pertel, N. L., Hinton, C., Kettler, T., Ho, A. D., Chen, Y., Bradshaw, M., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-87406-9 · Cited by 11
“Within each country, a global test of variation of outcome across levels of each particular demographic variable was conducted, and a pooled p-value 38 across countries was reported concerning evidence for variation within any country.”
When the glass is half full: early life experiences and adult optimism in 22 countries
Chen, Y., Kubzansky, L. D., Kim, E. S., Koga, H. K., Shiba, K., Padgett, R. N., Wilkinson, R., Johnson, B. R., & VanderWeele, T. J. (2025) · npj Mental Health Research · doi:10.1038/s44184-024-00109-3 · Cited by 10
“Apooledglobal p -value 45 wasreported across countries to evaluate if the association between each childhood factor and adult optimism holds within at least one country.”
EBV strain interacts with host HLA to drive nasopharyngeal carcinoma risk
Chen, Y., Liang, J., Zhang, W., Zhang, X., Zhang, X., Wang, C., Yao, F., Zhang, S., Zhou, X., Ye, W., et al. (2026) · Nature · doi:10.1038/s41586-026-10416-8 · Cited by 10
“1, Supplementary Tables 16 and 17 and Supplementary Fig.”
Gait Analysis by Causal Decomposition
Peng, X., Feng, Y., Ji, S., Amos, T. J., Wang, W., Li, M., Ai, S., Qiu, X., Dong, Y., Ma, D., et al. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2021.3082936 · Cited by 9
“(2) Previous p-value combination techniques like Fisher’s method makes the strong assumption that tests are independent, but HMP is robust to dependency between pvalues and thereby avoids the restrictive assumption [21].”
BackgroundMethodInfluentialAn integrated cluster‐wise significance measure for fMRI analysis
Ge, Y., Chen, G., Waltz, J. A., Hong, L. E., Kochunov, P., & Chen, S. (2022) · Human Brain Mapping · doi:10.1002/hbm.25795 · Cited by 10
“Various combining methods, such as Fisher’s combined probability test (Fisher, 1992), Stouffer’s statistic (Stouffer, 1949), Tippett’s method (Tippett, 1931), and recent approaches with a Cauchy distribution (Liu & Xie, 2020) or a harmonic mean p-value (Wilson, 2019), have been developed and commonly used due to their…”
MethodSynergism Paradigm and Effective Teamwork Mechanism
(2019) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.l1180.10812s19 · Cited by 9
Lardelli, M. T., Baer, L., Hin, N., Allen, A. G., Pederson, S., & Barthelson, K. (2024) · Journal of Alzheimer s Disease · doi:10.3233/jad-230522 · Cited by 10
Zhang, S., Cao, H., Chen, K., Gao, T., Zhao, H., Zheng, C., Wang, T., Zeng, P., & Wang, K. (2024) · International Journal of Public Health · doi:10.3389/ijph.2024.1606868 · Cited by 9
“The advantage of HMP is that it effectively aggregates a group of correlated p -values based on the model-averaged mean maximum likelihood ratio and is robust against unknown positive dependence [32, 33].”
MethodMadrid, A., Papale, L. A., Bergmann, P. E., Breen, C. E., Clark, L. R., Asthana, S., Johnson, S. C., Keleş, S., Hogan, K. J., & Alisch, R. S. (2025) · Alzheimer s & Dementia · doi:10.1002/alz.14474 · Cited by 9
Aglozo, E. Y., Johnson, K. A., Case, B. W., Padgett, R. N., Johnson, B. R., & VanderWeele, T. J. (2025) · Scientific Reports · doi:10.1038/s41598-024-79103-w · Cited by 9
Ritchie‐Dunham, J. L., Yancey, G., Managi, S., Bartel, C. A., Bonhag, R., Padgett, R. N., Shiba, K., Johnson, B. R., & VanderWeele, T. J. (2025) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-025-00226-0 · Cited by 9
“Within each country, a global test of variation of outcome across levels of each particular demographic variable was conducted, and a pooled p-value (Wilson, 2019) across countries was reported concerning evidence for variation within any country.”
Differential detection workflows for multi-sample single-cell RNA-seq data
Gilis, J., Perin, L., Malfait, M., Crowell, H. L., den Berge, K. V., Assefa, A. T., Verbist, B. M. P., Risso, D., & Clement, L. (2025) · BMC Genomics · doi:10.1186/s12864-025-12102-x · Cited by 9
Trade-off Between Validity and Efficiency of Merging P-Values Under Arbitrary Dependence
Chen, Y., Liu, P., Tan, K. S., & Wang, R. (2021) · Statistica Sinica · doi:10.5705/ss.202021.0071 · Cited by 18
Rapidly spreading seizures arise from large-scale functional brain networks in focal epilepsy
Uehara, T., Shigeto, H., Mukaino, T., Yokoyama, J., Okadome, T., Yamasaki, R., Ogata, K., Mukae, N., Sakata, A., Tobimatsu, S., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118104 · Cited by 9
“For each patient, the individual seizure P values were combined nto the harmonic mean P value with equal weights ( Wilson, 2019 ) to Table 1 Patient characteristics.”
MethodWang, T., Qiao, J., Zhang, S., Wei, Y., & Zeng, P. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac038 · Cited by 10
Development of an early alert model for pandemic situations in Germany
Wang, D., Lentzen, M., Botz, J., Valderrama, D., Deplante, L., Perrio, J., Génin, M., Thommes, E. W., Coudeville, L., & Fröhlich, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-48096-3 · Cited by 10
“We then combined these synonyms using the harmonic mean P -value (HMP) method 33 .”
MethodD3C: Reducing the Price of Anarchy in Multi-Agent Learning
Gemp, I., McKee, K. R., Everett, R., Duéñez‐Guzmán, E. A., Bachrach, Y., Balduzzi, D., & Tacchetti, A. (2022) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/ztul8959 · Cited by 25
Freff, J., Beins, E. C., Bröker, L., Schwarte, K., Dantas, R. L., Maj, C., Arolt, V., Dannlowski, U., Nöthen, M. M., Baune, B. T., et al. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.05.003 · Cited by 10
Zaffanello, M., Pietrobelli, A., Nosetti, L. M., Antoniazzi, F., Frassoldati, R., & PIACENTINI, G. L. (2025) · Pharmaceutics · doi:10.3390/pharmaceutics17050588 · Cited by 9
“Moreover, this method determines the overall significance of the association between the intervention and outcome [22,23].”
Chung, H., Choi, D., Gregor, A. N., Sim, E., Lau, A., Black, D., Scanga, H. L., Linshinski, A., Pennesi, M. E., Sahel, J., et al. (2025) · Journal of Inherited Metabolic Disease · doi:10.1002/jimd.70060 · Cited by 8
Barthelson, K., Protzman, R. A., Snel, M. F., Hemsley, K. M., & Lardelli, M. T. (2025) · Biochimica et Biophysica Acta (BBA) - Molecular Basis of Disease · doi:10.1016/j.bbadis.2024.167651 · Cited by 8
Uncovering the genetic architecture and evolutionary roots of androgenetic alopecia in African men
Janivara, R., Hazra, U., Pfennig, A., Harlemon, M. S., Kim, M. S., Eaaswarkhanth, M., Chen, W. C., Ogunbiyi, A. O., Kachambwa, P., Petersen, L. N., et al. (2025) · Human Genetics and Genomics Advances · doi:10.1016/j.xhgg.2025.100428 · Cited by 8
Metadata
- Authors
- Wilson, D. J.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Wilson, Daniel J.
- ORCID
- Wilson, Daniel J.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2019-01-04
- Volume
- 116
- Issue
- 4
- Pages
- 1195-1200
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6347718
- OpenAlex
- W2907099021
- Cited by
- 449
- References
- 48
- Citations by year
- 2026: 58 · 2025: 115 · 2024: 44 · 2023: 54 · 2022: 61 · 2021: 57 · 2020: 45 · 2019: 12
- FWCI
- 34.33
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-10-07)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Wellcome (101237/Z/13/Z)
- Royal Society (101237/Z/13/Z)
- RCUK | Medical Research Council (MR/K01532X/1)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- family-wise error rate, statistical power, generalized central limit theorem, group testing, genome-wide association study, neuroticism, Benjamini-Hochberg procedure, false discovery rate
- MeSH terms
- Hepatitis C, Humans, Models, Statistical, Hepacivirus, Viral Load, Genome-Wide Association Study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.