Imbalance-Aware Machine Learning for Predicting Rare and Common Disease-Associated Non-Coding Variants
Open accessAbstract
Disease and trait-associated variants represent a tiny minority of all known genetic variation, and therefore there is necessarily an imbalance between the small set of available disease-associated and the much larger set of non-deleterious genomic variation, especially in non-coding regulatory regions of human genome. Machine Learning (ML) methods for predicting disease-associated non-coding variants are faced with a chicken and egg problem - such variants cannot be easily found without ML, but ML cannot begin to be effective until a sufficient number of instances have been found. Most of state-of-the-art ML-based methods do not adopt specific imbalance-aware learning techniques to deal with imbalanced data that naturally arise in several genome-wide variant scoring problems, thus resulting in a significant reduction of sensitivity and precision. We present a novel method that adopts imbalance-aware learning strategies based on resampling techniques and a hyper-ensemble approach that outperforms state-of-the-art methods in two different contexts: the prediction of non-coding variants associated with Mendelian and with complex diseases. We show that imbalance-aware ML is a key issue for the design of robust and accurate prediction algorithms and we provide a method and an easy-to-use software tool that can be effectively applied to this challenging prediction task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Medical datasets frequently suffer from imbalances in phenotypes, like rare diseases or serious clinical outcomes, and cohort identification.
“Medical datasets also frequently suffer from imbalances in phenotypes, like rare diseases or serious clinical outcomes37, and cohort identification7.”
From Probabilistic medical predictions of large language models (2024) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565 · see citing works
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537 · see citing works
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · see citing works
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408 · see citing works
An integrated map of genetic variation from 1,092 human genomes
The 1000 Genomes Project Consortium (2012) · Nature · doi:10.1038/nature11632 · Cited by 7,711 · see citing works
LIBLINEAR: A Library for Large Linear Classification
Fan, R.-E., Chang, K., Hsieh, C., Xiang-rui, W., & Lin, C. (2008) · doi:10.5555/1390681.1442794 · Cited by 6,680 · see citing works
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559 · see citing works
A general framework for estimating the relative pathogenicity of human genetic variants
Kircher, M., Witten, D. M., Jain, P., O'Roak, B. J., Cooper, G. M., & Shendure, J. (2014) · Nature Genetics · doi:10.1038/ng.2892 · Cited by 6,183 · see citing works
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309 · see citing works
Galar, M., Fernández, A., Barrenechea, E., Bustince, H., & Herrera, F. (2011) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2161285 · Cited by 2,820 · see citing works
The NHGRI GWAS Catalog, a curated resource of SNP-trait associations
Welter, D., MacArthur, J., Morales, J., Burdett, T., Hall, P., Junkins, H., Klemm, A., Flicek, P., Manolio, T., Hindorff, L., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1229 · Cited by 2,707 · see citing works
Exploratory Undersampling for Class-Imbalance Learning
Liu, X., Wu, J., & Zhou, Z. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007853 · Cited by 2,477 · see citing works
Visscher, P. M., Brown, M. A., McCarthy, M. I., & Yang, J. (2012) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2011.11.029 · Cited by 2,425 · see citing works
Detection of nonneutral substitution rates on mammalian phylogenies
Pollard, K. S., Hubisz, M. J., Rosenbloom, K. R., & Siepel, A. (2010) · Genome Research · doi:10.1101/gr.097857.109 · Cited by 2,353 · see citing works
Predicting effects of noncoding variants with deep learning–based sequence model
Zhou, J., & Troyanskaya, O. G. (2015) · Nature Methods · doi:10.1038/nmeth.3547 · Cited by 2,338 · see citing works
COSMIC: exploring the world's knowledge of somatic mutations in human cancer
Forbes, S. A., Beare, D. M., Gunasekaran, P., Leung, K., Bindal, N., Boutselakis, H., Ding, M., Bamford, S., Cole, C. G., Ward, S. A., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku1075 · Cited by 2,261 · see citing works
Bishop, C. (1996) · Taylor & Francis eBooks · doi:10.1201/9781420050646.ptb6 · Cited by 1,607 · see citing works
Beyond GWASs: Illuminating the Dark Road from Association to Function
Edwards, S. L., Beesley, J., French, J. D., & Dunning, A. M. (2013) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2013.10.012 · Cited by 784 · see citing works
Shihab, H. A., Rogers, M. F., Gough, J., Mort, M. E., Cooper, D. N., Day, I. N. M., Gaunt, T. R., & Campbell, C. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv009 · Cited by 703 · see citing works
Kuncheva, L. I. (2014) · Wiley eBooks · doi:10.1002/9781118914564 · Cited by 670 · see citing works
A spectral approach integrating functional genomic annotations for coding and noncoding variants
Ionita‐Laza, I., McCallum, K., Xu, B., & Buxbaum, J. D. (2016) · Nature Genetics · doi:10.1038/ng.3477 · Cited by 588 · see citing works
Functional annotation of noncoding sequence variants
Ritchie, G. R. S., Dunham, I., Zeggini, E., & Flicek, P. (2014) · Nature Methods · doi:10.1038/nmeth.2832 · Cited by 505 · see citing works
A method to predict the impact of regulatory variants from DNA sequence
Lee, D., Gorkin, D. U., Baker, M., Strober, B. J., Asoni, A. L., McCallion, A. S., & Beer, M. (2015) · Nature Genetics · doi:10.1038/ng.3331 · Cited by 417 · see citing works
Enredo and Pecan: genome-wide mammalian consistency-based multiple alignment with paralogs.Preprint
Paten, B., Herrero, J., Beal, K., Fitzgerald, S., & Birney, E. (2008) · Genome Research · doi:10.1101/gr.076554.108 · Cited by 348 · see citing works
Smedley, D., Schubach, M., Jacobsen, J. O. B., Köhler, S., Zemojtel, T., Spielmann, M., Jäger, M., Hochheiser, H., Washington, N. L., McMurry, J. A., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.07.005 · Cited by 283 · see citing works
Self-citationRandom Forests for Genetic Association Studies
Goldstein, B. A., Polley, E. C., & Briggs, F. B. S. (2011) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1691 · Cited by 270 · see citing works
Genome-wide nucleotide-level mammalian ancestor reconstructionPreprint
Paten, B., Herrero, J., Fitzgerald, S., Beal, K., Flicek, P., Holmes, I., & Birney, E. (2008) · Genome Research · doi:10.1101/gr.076521.108 · Cited by 195 · see citing works
Ré, M., & Valentini, G. (2012) · Chapman & Hall/CRC data mining and knowledge discovery series · doi:10.1201/b11822-34 · Cited by 105 · see citing works
Self-citationMa, M., Ru, Y., Chuang, L.-S. F., Hsu, N.-Y., Shi, L., Hakenberg, J., Cheng, W., Uzilov, A. V., Ding, W., Glicksberg, B. S., et al. (2015) · BMC Genomics · doi:10.1186/1471-2164-16-s8-s3 · Cited by 59 · see citing works
Enhancer mutations and phenotype modularity
Gordon, C. T., & Lyonnet, S. (2013) · Nature Genetics · doi:10.1038/ng.2861 · Cited by 21 · see citing works
From genes to genomes in the clinic
Veltman, J. A., & Lupski, J. R. (2015) · Genome Medicine · doi:10.1186/s13073-015-0200-0 · Cited by 15 · see citing works
Interpreting noncoding genetic variation in complex traits and human disease
Ward, L. D., & Kellis, M. (2012) · Nature Biotechnology · doi:10.1038/nbt.2422 · Cited by 6 · see citing works
Functional Annotation of Rare Genetic Variants
Ritchie, G. R. S., & Flicek, P. (2015) · Springer eBooks · doi:10.1007/978-1-4939-2824-8_5 · Cited by 5 · see citing works
Diversity in Classifier Ensembles
Kuncheva, L. I. (2014) · doi:10.1002/9781118914564.ch8 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Nicholls, H. L., John, C. R., Watson, D. S., Munroe, P. B., Barnes, M. R., & Cabrera, C. P. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.00350 · Cited by 155 · see citing works
Disease Prediction via Graph Neural Networks
Sun, Z., Yin, H., Chen, H., Chen, T., Cui, L., & Yang, F. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3004143 · Cited by 98 · see citing works
Probabilistic medical predictions of large language models
Gu, B., Desai, R. J., Lin, K. J., & Yang, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01366-4 · Cited by 50 · see citing works
“Medical datasets also frequently suffer from imbalances in phenotypes, like rare diseases or serious clinical outcomes 37 , and cohort identi fi cation 7 .”
Uncertainty-Aware COVID-19 Detection from Imbalanced Sound Data
Xia, T., Han, J., Qendro, L., Dang, T., & Mascolo, C. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1320 · Cited by 20 · see citing works
- Authors
- Schubach, M., Re, M., Robinson, P. N., & Valentini, G.
- Affiliations
- Charité – Universitätsmedizin Berlin (DE)
- University of Connecticut (US)
- University of Milan (IT)
- Corresponding author
- Valentini, Giorgio
- Venue
- Scientific Reports
- Published
- 2017-06-07
- Volume
- 7
- Issue
- 1
- Pages
- 2959-2959
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC5462751
- OpenAlex
- W2620563544
- Cited by
- 97
- References
- 38
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 8 · 2023: 7 · 2022: 18 · 2021: 19 · 2020: 16 · 2019: 10 · 2018: 5 · 2017: 1 · 2016: 1
- FWCI
- 5.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- The Mouseion at the JAXlibrary (Jackson Laboratory) (submittedVersion)
- Funders
- Deutscher Akademischer Austauschdienst (57210259)
- Bundesministerium für Bildung und Forschung (01EC1402B)
- Berlin Institute of Health
- National Institutes of Health (R24OD011883)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- non-coding variants, class imbalance, resampling, monogenic disorders, complex diseases, regulatory elements, rare variants
- MeSH terms
- Machine Learning, Algorithms, Humans, Models, Genetic, Mutation, Software, Genetic Variation, Reproducibility of Results, Genetic Predisposition to Disease, RNA, Untranslated, Genome-Wide Association Study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
