Variant effect predictions capture some aspects of deep mutational scanning experiments
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Background: Deep mutational scanning (DMS) studies exploit the mutational landscape of sequence variation by systematically and comprehensively assaying the effect of single amino acid variants (SAVs; also referred to as missense mutations, or non-synonymous Single Nucleotide Variants – missense SNVs or nsSNVs) for particular proteins. We assembled SAV annotations from 22 different DMS experiments and normalized the effect scores to evaluate variant effect prediction methods. Three trained on traditional variant effect data (PolyPhen-2, SIFT, SNAP2), a regression method optimized on DMS data (Envision), and a naïve prediction using conservation information from homologs. Results: On a set of 32,981 SAVs, all methods captured some aspects of the experimental effect scores, albeit not the same. Traditional methods such as SNAP2 correlated slightly more with measurements and better classified binary states (effect or neutral). Envision appeared to better estimate the precise degree of effect. Most surprising was that the simple naïve conservation approach using PSI-BLAST in many cases outperformed other methods. All methods captured beneficial effects (gain-of-function) significantly worse than deleterious (loss-of-function). For the few proteins with multiple independent experimental measurements, experiments differed substantially, but agreed more with each other than with predictions. Conclusions: DMS provides a new powerful experimental means of understanding the dynamics of the protein sequence space. As always, promising new beginnings have to overcome challenges. While our results demonstrated that DMS will be crucial to improve variant effect prediction methods, data diversity hindered simplification and generalization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Richards, S., Aziz, N., Bale, S., Bick, D., Das, S., Gastier-Foster, J., Grody, W. W., Hegde, M., Lyon, E., Spector, E., et al. (2015) · Genetics in Medicine · doi:10.1038/gim.2015.30 · Cited by 32,594
CADD: predicting the deleteriousness of variants throughout the human genome
Rentzsch, P., Witten, D., Cooper, G. M., Shendure, J., & Kircher, M. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky1016 · Cited by 3,534 · cited 2×
“11 Long before experimental DMS, prediction methods had addressed the same task in 12 silico [33, 34] [35] [36-41].”
BackgroundRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280
REVEL: An Ensemble Method for Predicting the Pathogenicity of Rare Missense Variants
Ioannidis, N. M., Rothstein, J. H., Pejaver, V., Middha, S., McDonnell, S. K., Baheti, S., Musolf, A., Li, Q., Holzinger, E., Karyadi, D., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.08.016 · Cited by 2,908
ClinVar: public archive of interpretations of clinically relevant variants
Landrum, M. J., Lee, J. M., Benson, M., Brown, G., Chao, C., Chitipiralla, S., Gu, B., Hart, J., Hoffman, D., Hoover, J., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1222 · Cited by 2,758
Evolution and functional impact of rare coding variation from deep sequencing of human exomes
Tennessen, J. A., Bigham, A. W., O’Connor, T. D., Fu, W., Kenny, E. E., Gravel, S., McGee, S., Do, R., Liu, X., Jun, G., et al. (2012) · Science · doi:10.1126/science.1219240 · Cited by 1,613
Deep mutational scanning: a new style of protein science
Fowler, D. M., & Fields, S. (2014) · Nature Methods · doi:10.1038/nmeth.3027 · Cited by 1,287
Accurate classification of BRCA1 variants with saturation genome editing
Findlay, G. M., Daza, R. M., Martin, B., Zhang, M. D., Leith, A. P., Gasperini, M., Janizek, J. D., Huang, X., Starita, L. M., & Shendure, J. (2018) · Nature · doi:10.1038/s41586-018-0461-z · Cited by 843
Modeling the language of life – Deep Learning Protein Sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · doi:10.1101/614313 · Cited by 606
Self-citation
Cited by
Embeddings from protein language models predict conservation and variant effects
(2021) · doi:10.1007/s00439-021-02411-y · Cited by 174
Predicting the pathogenicity of missense variants using features derived from AlphaFold2
Schmidt, A., Röner, S., Mai, K., Klinkhammer, H., Kircher, M., & Ludwig, K. U. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad280 · Cited by 79
“In addition to the annotation of missense variants, DMS also represent a unique resource for the benchmarking of prediction scores for missense variants (Gray et al., 2018; Livesey and Marsh, 2020; Reeb et al., 2020).”
BackgroundLivesey, B. J., & Marsh, J. A. (2025) · Genome Biology · doi:10.1186/s13059-025-03575-w · Cited by 50
Expert-guided protein language models enable accurate and blazingly fast fitness prediction
Marquet, C., Schlensok, J., Abakarova, M., Rost, B., & Laine, E. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae621 · Cited by 33
Self-citationLuppino, F., Adzhubei, I. A., Cassa, C. A., & Toth-Petroczy, A. (2023) · Nature Communications · doi:10.1038/s41467-023-37661-z · Cited by 12
Metadata
- Authors
- Reeb, J., Wirth, T., & Rost, B.
- Venue
- BMC Bioinformatics
- Published
- 2020-03-17
- Volume
- 21
- Issue
- 1
- Pages
- 107-107
- ISSN
- 1471-2105 (ISSN-L)
- OpenAlex
- W3010879523
- Cited by
- 32
- References
- 19
- FWCI
- 3.01
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Variant effect predictions capture some aspects of deep mutational scanning experiments. https://app.cochat.ai/papers/10.1186/s12859-020-3439-4
- The paper
- Reeb, J., Wirth, T., & Rost, B. (2020). Variant effect predictions capture some aspects of deep mutational scanning experiments. BMC Bioinformatics. https://doi.org/10.1186/s12859-020-3439-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.