H‐MAGMA, inheriting a shaky statistical foundation, yields excess false positives
Open accessPublished version of a preprint
Abstract
All data used in the manuscript are available at https://github.com/ryurko/HMAGMA-comment. Supplementary Fig. 1. Comparison of covariance approximations for two-sided tests (green-solid) versus Brown's one-sided approximation described in MAGMA paper (red-dashed) and the code implemented in MAGMA with r2 (cyan-dashed). Supplementary Fig. 2. Comparison of the type 1 error rate control, plus/minus two standard errors, at target a = 0:05 (denoted by dashed red line) by gene-level method for seven example genes of different sizes. Supplementary Fig. 3. Comparison of multiple testing family-wise error rate (FWER), plus/minus two standard errors, at target a = 0:05 (denoted with dashed red line) by gene-level method. Supplementary Fig. 4. a, Comparison of the proportion of simulations with zero to six false positives for each method, excluding MAGMA: paper, using the Bonferroni correction at target a = 0:05 (plus/minus two standard errors). Supplementary Fig. 5. Comparison of gene-set analysis type 1 error, plus/minus two-standard errors, at target a = 0:05 (denoted with dashed red line) by gene-level method for each considered gene-set analysis method (from left to right) MAGMA: gene-set, Fisher's combination test, and Stouffer's z-test. Supplementary Table 1. Comparison of number of BH discoveries (union across adult and fetal Hi-C data annotations) with H-MAGMA versus corrected Monte Carlo based approach. Supplementary Fig. 6. Comparison of null histograms by gene-set analysis method (columns) and methods for computing gene-level p-values (rows) for gene-sets of size G = 45 genes. Red dashed lines indicate expected 2.5% and 97.5% quantiles for uniform p-values across 200,000 simulations. Supplementary Fig. 7. Comparison of distributions for MAGMA gene-set analysis standardized test statistic by methods for computing gene-level p-values. Red density curve displays the expected t(G 1) distribution, with the red vertical dashed line denoting the center at zero. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Gene set enrichment analyses were performed in MAGMA with correction for the number of variants in each gene, LD between variants and LD between genes.
“Gene set enrichment analyses were performed in MAGMA v1.08 (refs. 65,66), with correction for the number of variants in each gene, LD between variants and LD between genes.”
From New insights into the genetic etiology of Alzheimer’s disease and related dementias (2022) · cites this paper as [66]
The main text (page not supplied) describes MAGMA as a gene-level method using reference LD data, discusses gene-set enrichment, and identifies v1.08 as a revised version. It does not say that v1.08 corrects for the number of variants per gene or for LD between genes, so the full set of corrections attributed to this source is not established. (LLM, conf=0.85)
Experimental — accuracy not yet measuredPathway analyses using MAGMA corrected for the number of variants in each gene, LD between variants and LD between genes.
“Pathway analyses were performed using MAGMA (v1.08)41,42, with correction for the number of variants in each gene, LD between variants and LD between genes.”
From Consensus meta-analysis of genome-wide association studies for Alzheimer’s disease and related dementias (2026) · cites this paper as [42]
The main text says MAGMA uses reference LD data and describes adjustment for LD-induced covariance among SNP-level p-values. It does not say that pathway analyses correct for the number of variants in each gene or for LD between genes. Although it mentions a corrected testing strategy in v1.08, it does not specify those corrections. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
An integrated map of genetic variation from 1,092 human genomes
The 1000 Genomes Project Consortium (2012) · Nature · doi:10.1038/nature11632 · Cited by 7,711 · see citing works
METAL: fast and efficient meta-analysis of genomewide association scans
Willer, C. J., Li, Y., & Abecasis, G. R. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq340 · Cited by 5,205 · see citing works
MAGMA: Generalized Gene-Set Analysis of GWAS Data
de Leeuw, C. A., Mooij, J. M., Heskes, T., & Posthuma, D. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004219 · Cited by 3,653 · see citing works
Identification of common genetic risk variants for autism spectrum disorder
Autism Spectrum Disorder Working Group of the Psychiatric Genomics Consortium, Grove, J., BUPGEN, Major Depressive Disorder Working Group of the Psychiatric Genomics Consortium, 23andMe Research Team, Ripke, S., Als, T. D., Mattheisen, M., Walters, R. K., Won, H., et al. (2019) · Nature Genetics · doi:10.1038/s41588-019-0344-8 · Cited by 2,421 · see citing works
Self-citationADHD Working Group of the Psychiatric Genomics Consortium (PGC), Demontis, D., Early Lifecourse & Genetic Epidemiology (EAGLE) Consortium, 23andMe Research Team, Walters, R. K., Martin, J., Mattheisen, M., Als, T. D., Agerbo, E., Baldursson, G., et al. (2019) · Nature Genetics · doi:10.1038/s41588-018-0269-7 · Cited by 2,203 · see citing works
GERAD1 Consortium, Pardiñas, A. F., CRESTAR Consortium, Holmans, P., Pocklington, A. J., Escott-Price, V., Ripke, S., Carrera, N., Legge, S. E., Bishop, S., et al. (2018) · Nature Genetics · doi:10.1038/s41588-018-0059-2 · Cited by 1,552 · see citing works
Genome-wide association study identifies 30 loci associated with bipolar disorder
eQTLGen Consortium, Stahl, E. A., BIOS Consortium, the Bipolar Disorder Working Group of the Psychiatric Genomics Consortium, Breen, G., Forstner, A. J., McQuillin, A., Ripke, S., Trubetskoy, V., Mattheisen, M., et al. (2019) · Nature Genetics · doi:10.1038/s41588-019-0397-8 · Cited by 1,519 · see citing works
A global overview of pleiotropy and genetic architecture in complex traits
Watanabe, K., Stringer, S., Frei, O., Umićević Mirkov, M., de Leeuw, C., Polderman, T. J. C., van der Sluis, S., Andreassen, O. A., Neale, B. M., & Posthuma, D. (2019) · Nature Genetics · doi:10.1038/s41588-019-0481-0 · Cited by 1,332 · see citing works
A Versatile Gene-Based Test for Genome-wide Association Studies
Liu, J. Z., McRae, A. F., Nyholt, D. R., Medland, S. E., Wray, N. R., Brown, K. M., Hayward, N. K., Montgomery, G. W., Visscher, P. M., Martin, N. G., et al. (2010) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2010.06.009 · Cited by 791 · see citing works
Howard, D. M., Adams, M. J., Shirali, M., Clarke, T.-K., Marioni, R. E., Davies, G., Coleman, J. R. I., Alloza, C., Shen, X., Barbu, M. C., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-03819-3 · Cited by 612 · see citing works
400: A Method for Combining Non-Independent, One-Sided Tests of Significance
Brown, M. B. (1975) · Biometrics · doi:10.2307/2529826 · Cited by 473 · see citing works
Computing the Local Field Potential (LFP) from Integrate-and-Fire Network Models
Mazzoni, A., Lindén, H., Cuntz, H., Lansner, A., Panzeri, S., & Einevoll, G. T. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004584 · Cited by 461 · see citing works
Sey, N. Y. A., Hu, B., Mah, W., Fauni, H., McAfee, J. C., Rajarajan, P., Brennand, K. J., Akbarian, S., & Won, H. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0603-0 · Cited by 297 · see citing works
The statistical properties of gene-set analysis
de Leeuw, C., Neale, B. M., Heskes, T., & Posthuma, D. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.29 · Cited by 286 · see citing works
VEGAS2: Software for More Flexible Gene-Based Testing
Mishra, A., & Macgregor, S. (2015) · Twin Research and Human Genetics · doi:10.1017/thg.2014.79 · Cited by 257 · see citing works
Kost, J. T., & McDermott, M. (2002) · Statistics & Probability Letters · doi:10.1016/s0167-7152(02)00310-3 · Cited by 163 · see citing works
A global overview of pleiotropy and genetic architecture in complex traitsPreprint
Watanabe, K., Stringer, S., Frei, O., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/500090 · Cited by 162 · see citing works
Yang, J. J., Li, J., Williams, L. K., & Buu, A. (2016) · BMC Bioinformatics · doi:10.1186/s12859-015-0868-6 · Cited by 51 · see citing works
The Generalized Fisher's Combination and AccurateP-Value Calculation under Dependence
Zhang, H., & Wu, Z. (2022) · Biometrics · doi:10.1111/biom.13634 · Cited by 27 · see citing works
Distribution of Fisher's combination statistic when the tests are dependent
Yang, J. J. (2009) · Journal of Statistical Computation and Simulation · doi:10.1080/00949650802412607 · Cited by 25 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
New insights into the genetic etiology of Alzheimer’s disease and related dementias
Bellenguez, C., Küçükali, F., Jansen, I. E., Kleineidam, L., Moreno-Grau, S., Amin, N., Naj, A. C., Campos-Martin, R., Grenier-Boley, B., Andrade, V., et al. (2022) · Nature Genetics · doi:10.1038/s41588-022-01024-z · Cited by 2,498 · see citing works
Adams, M. J., Streit, F., Meng, X., Awasthi, S., Adey, B. N., Choi, K. W., Chundru, V. K., Coleman, J. R. I., Ferwerda, B., Foo, J. C., et al. (2025) · Cell · doi:10.1016/j.cell.2024.12.002 · Cited by 313 · see citing works
Integrating multi-omics summary data using a Mendelian randomization framework
Jin, C., Lee, B., Shen, L., Long, Q., & Initiative, A. D. N. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac376 · Cited by 36 · see citing works
Annotating genetic variants to target genes using H-MAGMA
Sey, N. Y. A., Pratt, B. M., & Won, H. (2022) · Nature Protocols · doi:10.1038/s41596-022-00745-z · Cited by 26 · see citing works
Pergola, G., Parihar, M., Sportelli, L., Bharadwaj, R. A., Borcuk, C., Radulescu, E., Bellantuono, L., Blasi, G., Chen, Q., Kleinman, J. E., et al. (2023) · Science Advances · doi:10.1126/sciadv.ade2812 · Cited by 26 · see citing works
Olislagers, M., Rademaker, K., Adan, R. A. H., Lin, B. D., & Luykx, J. J. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01324-6 · Cited by 23 · see citing works
Silberstein, M. D., Nesbit, N., Cai, J., & Lee, P. H. (2021) · Journal of genetics and genomics/Journal of Genetics and Genomics · doi:10.1016/j.jgg.2021.01.007 · Cited by 19 · see citing works
Wang, R., Lin, D.-Y., & Jiang, Y. (2022) · PLoS Genetics · doi:10.1371/journal.pgen.1010251 · Cited by 17 · see citing works
Naj, A. C., Leonenko, G., Jian, X., Grenier‐Boley, B., Dalmasso, M. C., Bellenguez, C., Jin, S., Zhao, Y., van der Lee, S. J., Sims, R., et al. (2021) · medRxiv · doi:10.1101/2021.03.14.21253553 · Cited by 17 · see citing works
Borcuk, C., Parihar, M., Sportelli, L., Kleinman, J. E., Shin, J. H., Hyde, T. M., Bertolino, A., Weinberger, D. R., & Pergola, G. (2024) · Neuron · doi:10.1016/j.neuron.2024.08.005 · Cited by 16 · see citing works
de Leeuw, C., Sey, N. Y. A., Posthuma, D., & Won, H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.25.310722 · Cited by 13 · see citing works
Pratt, B. M., & Won, H. (2021) · Seminars in Cell and Developmental Biology · doi:10.1016/j.semcdb.2021.08.013 · Cited by 12 · see citing works
Verweij, K. J. H., Vink, J. M., Abdellaoui, A., Gillespie, N. A., Derks, E. M., & Treur, J. L. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-02215-2 · Cited by 10 · see citing works
Kany, S., Rämö, J. T., Hou, C. R., Jurgens, S. J., Khurshid, S. S., Nauffal, V., Cunningham, J. W., Lau, E. S., Koyama, S., FinnGen, et al. (2025) · Nature Genetics · doi:10.1038/s41588-025-02397-7 · Cited by 9 · see citing works
Pleiotropic effects of telomere length loci with brain morphology and brain tissue expression
Pathak, G. A., Wendt, F. R., Levey, D. F., Mecca, A. P., van Dyck, C. H., Gelernter, J. E., & Polimanti, R. A. (2021) · Human Molecular Genetics · doi:10.1093/hmg/ddab102 · Cited by 8 · see citing works
EADB, Bellenguez, C., Castillo Morales, A., Amin, N., van der Lee, S. J., Muntaner, M., Parveen, K., Küçükali, F., Grenier-Boley, B., Heikkinen, S., et al. (2026) · Nature Genetics · doi:10.1038/s41588-026-02583-1 · Cited by 7 · see citing works
Zhang, S. (2022) · STAR Protocols · doi:10.1016/j.xpro.2021.101083 · Cited by 7 · see citing works
Lorincz‐Comi, N. J., Song, W., Chen, X., Paz, I. R., Hou, Y., Zhou, Y., Xu, J., Martin, W., Barnard, J., Pieper, A. A., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-69236-z · Cited by 4 · see citing works
An approach to gene-based testing accounting for dependence of tests among nearby genes
Yurko, R., Roeder, K., Devlin, B., & G’Sell, M. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab329 · Cited by 2 · see citing works
Self-citationWang, R., Lin, D. Y., & Jiang, Y. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.09.447805 · Cited by 2 · see citing works
The Genetic Determinants of Aortic DistensionPreprint
Pirruccello, J. P., Choi, S. H., Chaffin, M. D., Nekoui, M., Chou, E. L., Jurgens, S. J., Friedman, S. F., Juric, D., Stone, J. R., Batra, P., et al. (2021) · medRxiv · doi:10.1101/2021.10.16.21265089 · Cited by 2 · see citing works
- Authors
- Yurko, R., Roeder, K., Devlin, B., & G'Sell, M.
- Affiliations
- Carnegie Mellon University (US)
- University of Pittsburgh (US)
- Department of Statistics & Data Science Carnegie Mellon University Pittsburgh Pennsylvania
- Venue
- Annals of Human Genetics
- Publisher
- Wiley
- Published
- 2020-12-29
- Volume
- 85
- Issue
- 3-4
- Pages
- 97-100
- ISSN
- 0003-4800 (ISSN-L)
- 1469-1809
- OpenAlex
- W3115584880
- Cited by
- 23
- References
- 21
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 1 · 2023: 1 · 2022: 6 · 2021: 8 · 2020: 1
- FWCI
- 2.34
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Annals of Human Genetics (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Funders
- National Science Foundation
- National Institute of Mental Health
- Simons Foundation
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- magma, false positives, type I error rate, gene set analysis, multiple testing correction, Bonferroni correction, family-wise error rate, covariance approximation, Hi-C, chromatin interactions, Bayesian hypothesis testing
- MeSH terms
- Mental Disorders, Humans, Models, Genetic, Statistics as Topic, Linkage Disequilibrium, Genome-Wide Association Study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
