The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations
Open accessAbstract
To deal with the huge number of novel protein-coding variants identified by genome and exome sequencing studies, many computational variant effect predictors (VEPs) have been developed. Such predictors are often trained and evaluated using different variant data sets, making a direct comparison between VEPs difficult. In this study, we use 31 previously published deep mutational scanning (DMS) experiments, which provide quantitative, independent phenotypic measurements for large numbers of single amino acid substitutions, in order to benchmark and compare 46 different VEPs. We also evaluate the ability of DMS measurements and VEPs to discriminate between pathogenic and benign missense variants. We find that DMS experiments tend to be superior to the top-ranking predictors, demonstrating the tremendous potential of DMS for identifying novel human disease mutations. Among the VEPs, DeepSequence clearly stood out, showing both the strongest correlations with DMS data and having the best ability to predict pathogenic mutations, which is especially remarkable given that it is an unsupervised method. We further recommend SNAP2, DEOGEN2, SNPs&GO, SuSPect and REVEL based upon their performance in these analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A method and server for predicting damaging missense mutations.
Adzhubei, I. A., Schmidt, S., Peshkin, L., Ramensky, V. E., Gerasimova, A., Bork, P., Kondrashov, A. S., & Sunyaev, S. R. (2010) · Nature Methods · doi:10.1038/nmeth0410-248 · Cited by 13,194
The mutational constraint spectrum quantified from variation in 141,456 humans
Karczewski, K. J., Francioli, L. C., Tiao, G., Cummings, B. B., Alföldi, J., Wang, Q., Collins, R. L., Laricchia, K. M., Ganna, A., Birnbaum, D. P., et al. (2020) · Nature · doi:10.1038/s41586-020-2308-7 · Cited by 9,434
Amino acid substitution matrices from protein blocks.
Henikoff, S., & Henikoff, J. G. (1992) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.89.22.10915 · Cited by 6,218
CADD: predicting the deleteriousness of variants throughout the human genome
Rentzsch, P., Witten, D., Cooper, G. M., Shendure, J., & Kircher, M. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky1016 · Cited by 3,534
ClinVar: public archive of relationships among sequence variation and human phenotype
Landrum, M. J., Lee, J. M., Riley, G. R., Jang, W., Rubinstein, W. S., Church, D. M., & Maglott, D. R. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1113 · Cited by 3,516
Identifying a High Fraction of the Human Genome to be under Selective Constraint Using GERP++
Davydov, E. V., Goode, D. L., Sirota, M., Cooper, G. M., Sidow, A., & Batzoglou, S. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001025 · Cited by 1,757
Patterns and rates of exonic de novo mutations in autism spectrum disorders
Neale, B. M., Kou, Y., Liu, L., Ma’ayan, A., Samocha, K. E., Sabo, A., Lin, C.-F., Stevens, C., Wang, L.-S., Makarov, V., et al. (2012) · Nature · doi:10.1038/nature11011 · Cited by 1,734 · cited 1×
“…nonsynonymous single nucleotide variant (SNV) per protein-coding gene (Rauch et al , 2012), while de novo mutations lead to roughly one new nonsynonymous SNV per child not observed in either parent (de Ligt et al , 2012; Neale et al , 2012; Epi4K Consortium et al , 2013; Fitzgerald et al , 2015).”
Diagnostic exome sequencing in persons with severe intellectual disability
de Ligt, J., Willemsen, M. H., van Bon, B. W. M., Kleefstra, T., Yntema, H. G., Kroes, T., Vulto-van Silfhout, A. T., Koolen, D. A., de Vries, P., Gilissen, C., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1206524 · Cited by 1,546
De novo mutations in epileptic encephalopathies
Epi4K Consortium, & Epilepsy Phenome/Genome Project (2013) · Nature · doi:10.1038/nature12439 · Cited by 1,514
Deep mutational scanning: a new style of protein science
Fowler, D. M., & Fields, S. (2014) · Nature Methods · doi:10.1038/nmeth.3027 · Cited by 1,287 · cited 1×
“Deep mutational scanning (DMS) experiments combine saturation mutagenesis of a protein with a high-throughput functional test and deep sequencing (Fowler & Fields, 2014).”
Cited by
Accurate proteome-wide missense variant effect prediction with AlphaMissense
Cheng, J., Novati, G., Pan, J., Bycroft, C., Žemgulytė, A., Applebaum, T., Pritzel, A., Wong, L. H., Zielinski, M., Sargeant, T., et al. (2023) · Science · doi:10.1126/science.adg7492 · Cited by 2,163
A guide to machine learning for biologists
Greener, J. G., Kandathil, S. M., Moffat, L., & Jones, D. T. (2022) · Nature Reviews Molecular Cell Biology · doi:10.1038/s41580-021-00407-0 · Cited by 2,127
Disease variant prediction with deep generative models of evolutionary data
Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J. K., Brock, K., Gal, Y., & Marks, D. S. (2021) · Nature · doi:10.1038/s41586-021-04043-8 · Cited by 791
Efficient evolution of human antibodies from general protein language models
Hie, B. L., Shanker, V. R., Xu, D., Bruun, T. U. J., Weidenbacher, P. A., Tang, S., Wu, W., Pak, J. E., & Kim, P. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01763-2 · Cited by 449
Protein design and variant prediction using autoregressive generative models
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., & Marks, D. S. (2021) · Nature Communications · doi:10.1038/s41467-021-22732-w · Cited by 320
Embeddings from protein language models predict conservation and variant effects
(2021) · doi:10.1007/s00439-021-02411-y · Cited by 174
Longitudinal dynamics of clonal hematopoiesis identifies gene-specific fitness effects
Robertson, N. A., Latorre-Crespo, E., Terradas-Terradas, M., Lemos-Portela, J., Purcell, A. C., Livesey, B. J., Hillary, R. F., Murphy, L., Fawkes, A., MacGillivray, L., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01883-3 · Cited by 132
Self-citationLindorff-Larsen, K., & Kragelund, B. B. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167196 · Cited by 85
Predicting the pathogenicity of missense variants using features derived from AlphaFold2
Schmidt, A., Röner, S., Mai, K., Klinkhammer, H., Kircher, M., & Ludwig, K. U. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad280 · Cited by 79
Cross-protein transfer learning substantially improves disease variant prediction
(2023) · doi:10.1186/s13059-023-03024-6 · Cited by 68
Persistent spectral theory-guided protein engineering
(2023) · doi:10.1038/s43588-022-00394-y · Cited by 62
Livesey, B. J., & Marsh, J. A. (2025) · Genome Biology · doi:10.1186/s13059-025-03575-w · Cited by 50
Self-citationGerasimavicius, L., Livesey, B. J., & Marsh, J. A. (2023) · Protein Science · doi:10.1002/pro.4688 · Cited by 41
Self-citationHigh-throughput deep learning variant effect prediction with Sequence UNET
Dunham, A. S., Beltrao, P., & AlQuraishi, M. (2023) · Genome Biology · doi:10.1186/s13059-023-02948-3 · Cited by 31
“It was consistently found among the top VEPs across a diverse set of DMS results [12, 13].”
BackgroundSystematic analysis of NDUFAF6 in complex I assembly and mitochondrial disease
(2024) · doi:10.1038/s42255-024-01039-2 · Cited by 23
Fawzy, M., & Marsh, J. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-76202-6 · Cited by 22
Self-citationDeep mutational scanning quantifies DNA binding and predicts clinical outcomes of PAX6 variants
McDonnell, A. F., Plech, M., Livesey, B. J., Gerasimavicius, L., Owen, L. J., Hall, H. N., FitzPatrick, D. R., Marsh, J. A., & Kudla, G. (2024) · Molecular Systems Biology · doi:10.1038/s44320-024-00043-8 · Cited by 16
Self-citationVariant scoring tools for deep mutational scanning
Çubuk, H., Jin, X., Phipson, B., Marsh, J. A., & Rubin, A. F. (2025) · Molecular Systems Biology · doi:10.1038/s44320-025-00137-x · Cited by 13
Self-citationLuppino, F., Adzhubei, I. A., Cassa, C. A., & Toth-Petroczy, A. (2023) · Nature Communications · doi:10.1038/s41467-023-37661-z · Cited by 12
Why variant effect predictors and multiplexed assays agree and disagree
Livesey, B. J., & Marsh, J. A. (2025) · doi:10.1101/2025.07.31.667868 · Cited by 7
Self-citationCankara, F., & Doğan, T. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.09.017 · Cited by 1
Metadata
- Authors
- Livesey, B. J., & Marsh, J. A.
- Venue
- Molecular Systems Biology
- Published
- 2020-07
- Volume
- 16
- Issue
- 7
- Pages
- e9380-e9380
- ISSN
- 1744-4292 (ISSN-L)
- OpenAlex
- W3038248848
- Cited by
- 170
- References
- 17
- FWCI
- 13.74
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.