Deep generative models of genetic variation capture the effects of mutations
Open accessPublished version of a preprint
Abstract
The functions of proteins and RNAs are defined by the collective interactions of many residues, and yet most statistical models of biological sequences consider sites nearly independently. Recent approaches have demonstrated benefits of including interactions to capture pairwise covariation, but leave higher-order dependencies out of reach. Here we show how it is possible to capture higher-order, context-dependent constraints in biological sequences via latent variable models with nonlinear dependencies. We found that DeepSequence ( https://github.com/debbiemarkslab/DeepSequence ), a probabilistic model for sequence families, predicted the effects of mutations across a variety of deep mutational scanning experiments substantially better than existing methods based on the same evolutionary data. The model, learned in an unsupervised manner solely on the basis of sequence information, is grounded with biologically motivated priors, reveals the latent organization of sequence families, and can be used to explore new parts of sequence space.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Language models learn the probability of a sequence occurring and this can be directly applied to predict the fitness of sequence mutations.
“Language models learn the probability of a sequence occurring and this can be directly applied to predict the fitness of sequence mutations 29-31 .”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [29]
When trained on tens of thousands of evolutionarily related proteins, the learned probability mass function describing the empirical distribution over naturally occurring sequences has shown promise for predicting the fitness of sequence variants.
“When trained on tens of thousands of evolutionarily related proteins, the learned probability mass function describing the empirical distribution over naturally occurring sequences has shown promise for predicting the fitness of sequence variants 29-31.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A method and server for predicting damaging missense mutations.
Adzhubei, I. A., Schmidt, S., Peshkin, L., Ramensky, V. E., Gerasimova, A., Bork, P., Kondrashov, A. S., & Sunyaev, S. R. (2010) · Nature Methods · doi:10.1038/nmeth0410-248 · Cited by 13,194 · see citing works
Accelerated Profile HMM Searches
Eddy, S. R. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002195 · Cited by 6,975 · see citing works
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434 · see citing works
Self-citationDeep mutational scanning: a new style of protein science
Fowler, D. M., & Fields, S. (2014) · Nature Methods · doi:10.1038/nmeth.3027 · Cited by 1,287 · see citing works
Protein 3D Structure Computed from Evolutionary Sequence Variation
Marks, D. S., Colwell, L. J., Sheridan, R., Hopf, T. A., Pagnani, A., Zecchina, R., & Sander, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028766 · Cited by 1,144 · see citing works
Self-citationMutation effects predicted from sequence co-variation
Hopf, T. A., Ingraham, J. B., Poelwijk, F. J., Schärfe, C. P. I., Springer, M., Sander, C., & Marks, D. S. (2017) · Nature Biotechnology · doi:10.1038/nbt.3769 · Cited by 775 · see citing works
Large-scale determination of previously unsolved protein structures using evolutionary information
Ovchinnikov, S., Kinch, L., Park, H., Liao, Y., Pei, J., Kim, D. E., Kamisetty, H., Grishin, N. V., & Baker, D. (2015) · eLife · doi:10.7554/elife.09248 · Cited by 222 · see citing works
The power of multiplexed functional analysis of genetic variants
Gasperini, M., Starita, L., & Shendure, J. (2016) · Nature Protocols · doi:10.1038/nprot.2016.135 · Cited by 160 · see citing works
3D RNA and Functional Interactions from Evolutionary Couplings
Weinreb, C., Riesselman, A. J., Ingraham, J. B., Gross, T., Sander, C., & Marks, D. S. (2016) · Cell · doi:10.1016/j.cell.2016.03.030 · Cited by 159 · see citing works
Self-citationDeconstruction of the Ras switching cycle through saturation mutagenesis
Bandaru, P., Shah, N. H., Bhattacharyya, M., Barton, J. P., Kondo, Y., Cofsky, J. C., Gee, C. L., Chakraborty, A. K., Kortemme, T., Ranganathan, R., et al. (2017) · eLife · doi:10.7554/elife.27810 · Cited by 117 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131 · see citing works
Disease variant prediction with deep generative models of evolutionary data
Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J. K., Brock, K., Gal, Y., & Marks, D. S. (2021) · Nature · doi:10.1038/s41586-021-04043-8 · Cited by 791 · see citing works
Self-citationEvaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719 · see citing works
Language models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.07.09.450648 · Cited by 635 · see citing works
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · see citing works
ProGen2: Exploring the boundaries of protein language models
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.002 · Cited by 480 · see citing works
Illuminating protein space with a programmable generative model
Ingraham, J. B., Baranov, M., Costello, Z., Barber, K. W., Wang, W., Ismail, A., Frappier, V., Lord, D. M., Ng-Thow-Hing, C., Van Vlack, E. R., et al. (2023) · Nature · doi:10.1038/s41586-023-06728-8 · Cited by 444 · see citing works
Self-citationLow-N protein engineering with data-efficient deep learning
Biswas, S., Khimulya, G., Alley, E. C., Esvelt, K. M., & Church, G. M. (2021) · Nature Methods · doi:10.1038/s41592-021-01100-y · Cited by 422 · see citing works
Expanding functional protein sequence spaces using generative adversarial networks
Repecka, D., Jauniskis, V., Karpus, L., Rembeza, E., Rokaitis, I., Zrimec, J., Poviloniene, S., Laurynenas, A., Viknander, S., Abuajwa, W., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00310-5 · Cited by 388 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · see citing works
“The generative picture can be extended to include latent variables parameterized by neural networks that capture higher order interactions than pairwise (Riesselman et al., 2018).”
BackgroundRao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350 · see citing works
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341 · see citing works
Protein design and variant prediction using autoregressive generative models
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., & Marks, D. S. (2021) · Nature Communications · doi:10.1038/s41467-021-22732-w · Cited by 320 · see citing works
Self-citationMason, D. M., Friedensohn, S., Weber, C. R., Jordi, C., Wagner, B., Meng, S. M., Ehling, R. A., Bonati, L., Dahinden, J., Gainza, P., et al. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00699-9 · Cited by 308 · see citing works
Global Genetic Networks and the Genotype-to-Phenotype Relationship
Costanzo, M., Kuzmin, E., van Leeuwen, J., Mair, B., Moffat, J., Boone, C., & Andrews, B. (2019) · Cell · doi:10.1016/j.cell.2019.01.033 · Cited by 241 · see citing works
Deep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227 · see citing works
Rapid protein stability prediction using deep learning representations
Blaabjerg, L. M., Kassem, M. M., Good, L. L., Jonsson, N., Cagiada, M., Johansson, K. E., Boomsma, W., Stein, A., & Lindorff-Larsen, K. (2023) · eLife · doi:10.7554/elife.82593 · Cited by 177 · see citing works
Saka, K., Kakuzaki, T., Metsugi, S., Kashiwagi, D., Yoshida, K., Wada, M., Tsunoda, H., & Teramoto, R. (2021) · Scientific Reports · doi:10.1038/s41598-021-85274-7 · Cited by 144 · see citing works
Protein language model embeddings for fast, accurate, alignment-free protein structure prediction
Weissenow, K., Heinzinger, M., & Rost, B. (2021) · doi:10.1101/2021.07.31.454572 · Cited by 139 · see citing works
Ig-VAE: Generative modeling of protein structure by direct 3D coordinate generation
Eguchi, R. R., Choe, C. A., & Huang, P.-S. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010271 · Cited by 141 · see citing works
Convolutions are competitive with transformers for protein sequence pretraining
Yang, K. K., Fusi, N., & Lu, A. X. (2024) · Cell Systems · doi:10.1016/j.cels.2024.01.008 · Cited by 137 · see citing works
Deciphering protein evolution and fitness landscapes with latent space models
Ding, X., Zou, Z., & Brooks III, C. L. (2019) · Nature Communications · doi:10.1038/s41467-019-13633-0 · Cited by 106 · see citing works
Artificial intelligence in food science and nutrition: A narrative review
Miyazawa, T., Hiratsuka, Y., Toda, M., Hatakeyama, N., Ozawa, H., Abe, C., Cheng, T.-Y., Matsushima, Y., Miyawaki, Y., Ashida, K., et al. (2022) · Nutrition Reviews · doi:10.1093/nutrit/nuac033 · Cited by 91 · see citing works
Manrubia, S., Cuesta, J. A., Aguirre, J., Ahnert, S. E., Altenberg, L., Cano, A. V., Catalán, P., Diaz-Uriarte, R., Elena, S. F., García-Martín, J. A., et al. (2021) · Physics of Life Reviews · doi:10.1016/j.plrev.2021.03.004 · Cited by 88 · see citing works
Lindorff-Larsen, K., & Kragelund, B. B. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167196 · Cited by 85 · see citing works
Harnessing generative AI to decode enzyme catalysis and evolution for enhanced engineering
Xie, W. J., & Warshel, A. (2023) · National Science Review · doi:10.1093/nsr/nwad331 · Cited by 69 · see citing works
An end-to-end framework for the prediction of protein structure and fitness from single sequence
Chen, Y., Xu, Y., Liu, D., Xing, Y., & Gong, H. (2024) · Nature Communications · doi:10.1038/s41467-024-51776-x · Cited by 45 · see citing works
Protein sequence‐to‐structure learning: Is this the end(‐to‐end revolution)?
Laine, E., Eismann, S., Elofsson, A., & Grudinin, S. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26235 · Cited by 44 · see citing works
“diction and protein design can be unified in a common theoretical framework extracting information from protein sequences.(171,172,175)”
BackgroundXu, Y., Liu, D., & Gong, H. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00716-2 · Cited by 41 · see citing works
Protein representations: Encoding biological information for machine learning in biocatalysis
Harding-Larsen, D., Funk, J., Madsen, N. G., Gharabli, H., Acevedo-Rocha, C. G., Mazurenko, S., & Welner, D. H. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108459 · Cited by 38 · see citing works
High-throughput deep learning variant effect prediction with Sequence UNET
Dunham, A. S., Beltrao, P., & AlQuraishi, M. (2023) · Genome Biology · doi:10.1186/s13059-023-02948-3 · Cited by 31 · see citing works
<scp>ESM</scp> ‐scan—A tool to guide amino acid substitutions
Totaro, M. G., Vide, U., Zausinger, R., Winkler, A., & Oberdorfer, G. (2024) · Protein Science · doi:10.1002/pro.5221 · Cited by 19 · see citing works
- Authors
- Riesselman, A. J., Ingraham, J. B., & Marks, D. S.
- Affiliations
- Harvard University (US)
- Corresponding author
- Marks, Debora S.
- Venue
- Nature Methods
- Published
- 2018-09-24
- Volume
- 15
- Issue
- 10
- Pages
- 816-822
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC6693876
- OpenAlex
- W2890223884
- Cited by
- 601
- References
- 10
- Citations by year
- 2026: 70 · 2025: 86 · 2024: 107 · 2023: 91 · 2022: 94 · 2021: 75 · 2020: 43 · 2019: 31 · 2018: 3 · 2017: 1
- FWCI
- 13.52
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Methods (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- U.S. Department of Energy (DEFG0297ER25308)
- National Institute of General Medical Sciences (R01GM106303)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Evolution and Genetic Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Keywords
- deep generative models, genetic variation, mutation effects, deep mutational scanning, latent variable models, nonlinear dependencies, protein sequence families, sequence space exploration, evolutionary constraints, unsupervised learning
- MeSH terms
- Algorithms, Humans, Models, Theoretical, Mutation, Evolution, Molecular, Computational Biology, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
