The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Discordance between a deep learning model and clinical-grade variant pathogenicity classification in a rare disease cohort
Open accessPublished version of a preprint
Abstract
Abstract Genetic testing has become an essential component in the diagnosis and management of a wide range of clinical conditions, from cancer to developmental disorders, especially in rare Mendelian diseases. Efforts to identify rare phenotype-associated variants have predominantly focused on protein-truncating variants, while the interpretation of missense variants presents a considerable challenge. Deep learning algorithms excel in various applications across biomedical tasks 1,2 , yet accurately distinguishing between pathogenic and benign genetic variants remains an elusive goal 3-5 . Specifically, even the most sophisticated models encounter difficulties in accurately assessing the pathogenicity of missense variants of uncertain significance (VUS). Our investigation of AlphaMissense (AM) 5 , the latest iteration of deep learning methods for predicting the potential functional impact of missense variants and assessing gene essentiality, reveals important limitations in its ability to identify pathogenic missense variants within a rare disease cohort. Indeed, AM struggles to accurately assess the pathogenicity of variants in intrinsically disordered regions (IDRs), leading to unreliable gene-level essentiality scores for certain genes containing IDRs. This limitation highlights the challenges in applying AM faces in the context of clinical genetics 6 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Richards, S., Aziz, N., Bale, S., Bick, D., Das, S., Gastier-Foster, J., Grody, W. W., Hegde, M., Lyon, E., Spector, E., et al. (2015) · Genetics in Medicine · doi:10.1038/gim.2015.30 · Cited by 32,594
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365
Artificial Intelligence and Machine Learning in Clinical Medicine, 2023
Haug, C. J., & Drazen, J. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2302038 · Cited by 1,215
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995
Disease variant prediction with deep generative models of evolutionary data
Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J. K., Brock, K., Gal, Y., & Marks, D. S. (2021) · Nature · doi:10.1038/s41586-021-04043-8 · Cited by 791
Pejaver, V., Byrne, A. B., Feng, B.-J., Pagel, K. A., Mooney, S. D., Karchin, R., O’Donnell-Luria, A., Harrison, S. M., Tavtigian, S. V., Greenblatt, M. S., et al. (2022) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2022.10.013 · Cited by 589
Flagged first, then by citations
Research works printed but not linked: 31
- Brandes, N., Goldman, G., Wang, C. H., Ye, C. J. & Ntranos, V. Genome-wide prediction of disease variant effects with a deep protein language model. Nat. Genet. 55, 1512–1522 (2023). · DOI
- Cheng, J. et al. Accurate proteome-wide missense variant effect prediction with AlphaMissense. Science 381, eadg7492 (2023). · DOI · Paper page
- Investigators, G. P. P. et al. 100,000 Genomes pilot on rare-disease diagnosis in health care—preliminary report. N. Engl. J. Med.385, 1868–1880 (2021). · DOI · Paper page
- Pagnamenta, A. T. et al. Structural and non-coding variants increase the diagnostic yield of clinical whole genome sequencing for rare diseases. Genome Med. 15, 94 (2023). · DOI
- Landrum, M. J. et al. ClinVar: improvements to accessing data. Nucleic Acids Res. 48, D835–D844 (2020). · DOI · Paper page
- Chandra, A., Tünnermann, L., Löfstedt, T. & Gratz, R. Transformer-based deep learning for predicting protein properties in the life sciences. Elife 12, e82819 (2023). · DOI
- Tang, Z., Toneyan, S. & Koo, P. K. Current approaches to genomic deep learning struggle to fully capture human genetic variation. Nat. Genet. 55, 2021–2022 (2023). · DOI
- Mandl, K. D. et al. The Genomics Research and Innovation Network: creating an interoperable, federated, genomics learning system. Genet. Med. 22, 371–380 (2020). · DOI
- Rockowitz, S. et al. Children’s rare disease cohorts: an integrative research and clinical genomics initiative. NPJ Genom. Med. 5, 29 (2020). · DOI
- Ioannidis, N. M. et al. REVEL: an ensemble method for predicting the pathogenicity of rare missense variants. Am. J. Hum. Genet. 99, 877–885 (2016). · DOI · Paper page
- Feng, B. J. PERCH: a unified framework for disease gene prioritization. Hum. Mutat. 38, 243–251 (2017). · DOI · Paper page
- Burdon, K. P. et al. Specifications of the ACMG/AMP variant curation guidelines for myocilin: Recommendations from the clingen glaucoma expert panel. Hum. Mutat. 43, 2170–2186 (2022). · DOI
- Johnston, J. J. et al. Variant curation expert panel recommendations for RYR1 pathogenicity classifications in malignant hyperthermia susceptibility. Genet. Med. 23, 1288–1295 (2021). · DOI
- Ross, J. E. et al. Specifications of the variant curation guidelines for ITGA2B/ITGB3: ClinGen platelet disorder variant curation panel. Blood Adv. 5, 414–431 (2021). · DOI
- Parsons, M. T. et al. Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel. Am. J. Hum. Genet. 111, 2044–2058 (2024). · DOI
- Tian, Y. et al. REVEL and BayesDel outperform other in silico meta-predictors for clinical variant classification. Sci. Rep. 9, 12752 (2019). · DOI · Paper page
- Jacobsen, J. O. B. et al. Phenotype-driven approaches to enhance variant prioritization and diagnosis of rare disease. Hum. Mutat. 43, 1071–1081 (2022). · DOI
- Stenson, P. D. et al. The Human Gene Mutation Database (HGMD((R))): optimizing its use in a clinical diagnostic or research setting. Hum. Genet. 139, 1197–1207 (2020). · DOI
- Sharo, A. G., Zou, Y., Adhikari, A. N. & Brenner, S. E. ClinVar and HGMD genomic variant classification accuracy has improved over time, as measured by implied disease burden. Genome Med. 15, 51 (2023). · DOI
- Uhlig, H. H. et al. Clinical Genomics for the diagnosis of monogenic forms of inflammatory bowel disease: a position paper from the paediatric IBD Porto Group of European Society of Paediatric Gastroenterology, Hepatology and Nutrition. J. Pediatr. Gastroenterol. Nutr. 72, 456–473 (2021). · DOI
- Collen, L. V. et al. Cytotoxic T lymphocyte antigen 4 haploinsufficiency presenting as refractory celiac-like disease: case report. Front. Immunol. 13, 894648 (2022). · DOI
- Collen, L. V., Newburger, P. E. & Snapper, S. B. Clinical remission of severe Crohn’s disease with empagliflozin monotherapy in a pediatric patient with glycogen storage disease type 1b. JPGN Rep. 4, e356 (2023). · DOI
- Shouval, D. S. et al. Interleukin 1beta mediates intestinal inflammation in mice and patients with interleukin 10 receptor deficiency. Gastroenterology 151, 1100–1104 (2016). · DOI
- Bolton, C. et al. An integrated taxonomy for monogenic inflammatory bowel disease. Gastroenterology 162, 859–876 (2022). · DOI
- Zhang, Y. J. et al. Novel variants in the stem cell niche factor WNT2B define the disease phenotype as a congenital enteropathy with ocular dysgenesis. Eur. J. Hum. Genet. 29, 998–1007 (2021). · DOI
- Holehouse, A. S. & Kragelund, B. B. The molecular basis for cellular function of intrinsically disordered protein regions. Nat. Rev. Mol. Cell Biol. 25, 187–211 (2024). · DOI
- Ahmed, S. S. et al. Characterization of intrinsically disordered regions in proteins informed by human genetic diversity. PLoS Comput. Biol. 18, e1009911 (2022). · DOI
- Goldberg, C. B. et al. To do no harm — and the most good — with AI in health care. Nat. Med. 30, 623–627 (2024). · DOI
- Karczewski, K. J. et al. The mutational constraint spectrum quantified from variation in 141,456 humans. Nature 581, 434–443 (2020). · DOI · Paper page
- Shamsani, J. et al. A plugin for the Ensembl Variant Effect Predictor that uses MaxEntScan to predict variant spliceogenicity. Bioinformatics 35, 2315–2317 (2019). · DOI
- Jaganathan, K. et al. Predicting splicing from primary sequence with deep learning. Cell 176, 535–548.e24 (2019). · DOI · Paper page
Metadata
- Authors
- Kong, S. W., Lee, I.-H., Collen, L. V., Field, M., Manrai, A. K., Snapper, S. B., & Mandl, K. D.
- Affiliations
- Boston Children's Hospital (US)
- Harvard University (US)
- Corresponding author
- Kong, Sek Won
- ORCID
- Venue
- npj Genomic Medicine
- Published
- 2025-02-28
- Volume
- 10
- Issue
- 1
- Pages
- 17-17
- ISSN
- 2056-7944 (ISSN-L)
- PMC
- PMC11871343
- OpenAlex
- W4408046295
- Cited by
- 18
- References
- 8
- Citations by year
- 2026: 13 · 2025: 5
- FWCI
- 10.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- npj Genomic Medicine (publishedVersion, cc-by-nc-nd)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- U.S. Department of Health & Human Services | NIH | National Center for Advancing Translational Sciences (U01TR002623)
- U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (R01NS129188)
- U.S. Department of Health & Human Services | NIH | National Human Genome Research Institute (U54HG012513)
- PrecisionLink Health Discovery and Children’s Rare Disease Cohorts initiative of Boston Children’s Hospital
- Helmsley Charitable Trust, Wolpow Family Chair in IBD Research and Treatment, Egan Family Foundation Chair in Transitional Medicine
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genetic Associations and Epidemiology
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- AlphaMissense, intrinsically disordered regions, gene essentiality, variant interpretation, very early onset inflammatory bowel disease, truncating mutations, deep learning limitations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
