CoChat

Protein Design and Variant Prediction Using Autoregressive Generative Models

Open access
Shin, Jung-Eun, Riesselman, Adam J., Kollasch, Aaron W., McMahon, Conor, Simon, Elana, Sander, Chris, Manglik, Aashish, Kruse, Andrew C., Marks, Debora S. · 2019 · bioRxiv (Cold Spring Harbor Laboratory)

Preprint — published as 10.1038/s41467-021-22732-w · Paper page

PreprintCited by 32

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Abstract

The ability to design functional sequences and predict effects of variation is central to protein engineering and biotherapeutics. State-of-art computational methods rely on models that leverage evolutionary information but are inadequate for important applications where multiple sequence alignments are not robust. Such applications include the prediction of variant effects of indels, disordered proteins, and the design of proteins such as antibodies due to the highly variable complementarity determining regions. We introduce a deep generative model adapted from natural language processing for prediction and design of diverse functional sequences without the need for alignments. The model performs state-of-art prediction of missense and indel effects and we successfully design and test a diverse 10 5 -nanobody library that shows better expression than a 1000-fold larger synthetic library. Our results demonstrate the power of the ‘alignment-free’ autoregressive model in generalizing to regions of sequence space traditionally considered beyond the reach of prediction and design.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/757252

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 76

  1. 10.1038/nrm2805 · DOI · Paper page
  2. 10.1016/j.copbio.2009.08.005 · DOI · Paper page
  3. 10.1002/pro.481 · DOI · Paper page
  4. 10.1093/protein/gzw042 · DOI
  5. 10.1038/s41594-018-0028-6 · DOI · Paper page
  6. 10.1038/nbt.1791 · DOI · Paper page
  7. Schoof, M. et al. An ultra-potent synthetic nanobody neutralizes SARS-CoV-2 by locking Spike into an inactive conformation. bioRxiv, 2020.2008.2008.238469 (2020).
  8. 10.1074/jbc.REV120.010181 · DOI · Paper page
  9. 2016, Maximum-Entropy Models of Sequenced Immune Repertoires Predict Antigen-Antibody Affinity, PLoS Comput Biol, 12, e1004870, 10.1371/journal.pcbi.1004870 · DOI
  10. 10.1073/pnas.1001705107 · DOI · Paper page
  11. 10.1038/s41467-018-02832-w · DOI · Paper page
  12. 10.1073/pnas.1525510113 · DOI · Paper page
  13. Muyldermans, S. A guide to: generation and design of nanobodies. FEBS J (2020). · DOI · Paper page
  14. Zimmermann, I. et al. Synthetic single domain antibodies for the conformational trapping of membrane proteins. Elife 7 (2018). · DOI · Paper page
  15. 10.1016/j.cell.2012.04.012 · DOI · Paper page
  16. 10.1073/pnas.1314045110 · DOI · Paper page
  17. Green, A.G. et al. Proteome-scale discovery of protein interactions with residue-level resolution using sequence coevolution. bioRxiv, 791293 (2019). · DOI
  18. 2018, Quantitative Missense Variant Effect Prediction Using Large-Scale Mutagenesis Data, Cell Syst, 6, 116, 10.1016/j.cels.2017.11.003 · DOI · Paper page
  19. 10.1371/journal.pcbi.1003776 · DOI · Paper page
  20. 2015, Coevolutionary Landscape Inference and the Context-Dependence of Mutations in Beta-Lactamase TEM-(I), Molecular Biology and Evolution, 33, 268
  21. 2020, Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations, Mol Syst Biol, 16, e9380, 10.15252/msb.20199380 · DOI · Paper page
  22. 10.1093/nar/gks539 · DOI · Paper page
  23. 10.1038/ng.2892 · DOI · Paper page
  24. 10.1093/nar/gkr407 · DOI · Paper page
  25. 10.1016/S0145-305X(02)00039-3 · DOI · Paper page
  26. 10.1021/cr400525m · DOI · Paper page
  27. 10.1093/hmg/ddq400 · DOI · Paper page
  28. 2017, Effects of short indels on protein structure and function in human genomes, Sci Rep, 7, 9313, 10.1038/s41598-017-09287-x · DOI · Paper page
  29. 10.1101/gr.115907.110 · DOI · Paper page
  30. 10.1002/prot.22504 · DOI · Paper page
  31. 10.1016/S1470-2045(17)30516-8 · DOI · Paper page
  32. 2019, Intrinsically disordered proteins and structured proteins with intrinsically disordered regions have different functional roles in the cell, PLoS One, 14, e0217889, 10.1371/journal.pone.0217889 · DOI · Paper page
  33. 10.1186/1471-2164-10-S3-S7 · DOI
  34. Graves, A. , Mohamed, A. & Hinton, G. Speech recognition with deep recurrent neural networks. 2013 IEEE International Conference on Acoustics, Speech and Singal Processing, 6645–6649 (2013). · DOI · Paper page
  35. Bahdanau, D. , Cho, K. & Begio, Y. Neural machine translation by jointly learning to align and translate. arXiv (2014).
  36. Sutskever, I. , Vinyals, O. & Le, Q.V. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 3104–3112 (2014).
  37. 10.1093/bioinformatics/btaa003 · DOI · Paper page
  38. 2019, Conditioning by adaptive sampling for robust design, Proceedings of the 36th International Conference on Machine Learning, 97, 773
  39. 10.1371/journal.ppat.1004552 · DOI
  40. 2017, Naive Human Antibody Libraries for Infectious Diseases, Adv Exp Med Biol, 1053, 35, 10.1007/978-3-319-72077-7_3 · DOI
  41. 2018, Scalable, Continuous Evolution of Genes at Mutation Rates above Genomic Error Thresholds, Cell, 175, 1946, 10.1016/j.cell.2018.10.021 · DOI · Paper page
  42. Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems, 5998–6008 (2017).
  43. Sutskever, I. , Martens, J. & Hinton, G. Generating text with recurrent neural networks. Proceedings of the 28th International Conference on Machine Learning (ICML-11), 1017–1024 (2011).
  44. Pascanu, R. , Mikolov, T. & Begio, Y. On the difficulty of training recurrent neural networks. International Conference on Machine Learning, 1310-1318 (2013).
  45. Cho, K. et al. Learning phrase representations using RNN Encoder-Decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1724–1734 (2014). · DOI · Paper page
  46. van den Oord, A. et al. Wavenet: A generative model for raw audio. arXiv (2016).
  47. Gupta, A. & Rush, A. Dilated convolutions for modeling long-distance genomic dependencies. arXiv (2017). · DOI · Paper page
  48. 2017, Exploiting sequence and stability information for directing nanobody stability engineering, Biochim Biophys Acta Gen Subj, 1861, 2196, 10.1016/j.bbagen.2017.06.014 · DOI
  49. 2017, Evaluation of anti-botulinum neurotoxin single domain antibodies with additional optimization for improved production and stability, Toxicon, 135, 51, 10.1016/j.toxicon.2017.06.002 · DOI
  50. 2015, Improving the biophysical properties of anti-ricin single-domain antibodies, Biotechnol Rep (Amst), 6, 27, 10.1016/j.btre.2015.01.001 · DOI
  51. 10.1371/journal.pone.0077678 · DOI
  52. Durbin, R. , Eddy, S.R. , Krogh, A. & Mitchison, G. Biological sequence analysis: probabilistic models of proteins and nucleic acids. (Cambridge university press, 1998). · DOI · Paper page
  53. 2011, Accelerated profile HMM searches, PLoS computational biology, 7, e1002195, 10.1371/journal.pcbi.1002195 · DOI · Paper page
  54. 10.1016/j.ajhg.2018.03.018 · DOI · Paper page
  55. 2019, An experimental assay of the interactions of amino acids from orthologous sequences shaping a complex fitness landscape, PLoS Genet, 15, e1008079, 10.1371/journal.pgen.1008079 · DOI · Paper page
  56. 10.1016/S0161-5890(00)00081-X · DOI · Paper page
  57. 10.4161/mabs.12187 · DOI
  58. 2012, Antibody polyspecificity: what does it matter?, Adv Exp Med Biol, 750, 213, 10.1007/978-1-4614-3461-0_16 · DOI
  59. 10.4049/jimmunol.1300880 · DOI
  60. 10.1080/19420862.2017.1356528 · DOI
  61. Lim, C.C. , Choong, Y.S. & Lim, T.S. Cognizance of Molecular Methods for the Generation of Mutagenic Phage Display Antibody Libraries for Affinity Maturation. Int J Mol Sci 20 (2019). · DOI
  62. 2017, Antibody repertoire profiling with mimotope arrays, Hum Vaccin Immunother, 13, 314, 10.1080/21645515.2017.1264786 · DOI
  63. 10.1126/science.1086907 · DOI · Paper page
  64. Wellner, A. et al. Rapid generation of potent antibodies by autonomous hypermutation in yeast. bioRxiv (2020). · doi:10.1101/2020.11.11.378778
  65. Ingraham, J.B. , Vikas, G.K. , Barzilay, R. & Jaakkola, T. Generative models for graph-based protein design. 33rd Conference on Neural Information Processing Systems 15794–15805 (2019).
  66. Kim, Y. , Wiseman, S. , Miller, A.C. , Sontag, D. & Rush, A. Semi-amortized variational autoencoders. arXiv (2018).
  67. Yang, Z. , Hu, Z. , Salakhutdinov, R. & Berg-Kirkpatrick, T. Improved variational autoencoders for text modeling using dilated convolutions. arXiv (2017).
  68. 2018, Design of metalloproteins and novel protein folds using variational autoencoders, Sci Rep, 8, 16189, 10.1038/s41598-018-34533-1 · DOI · Paper page
  69. Vijayakumar, A.K. et al. Diverse beam search: Decoding diverse solutions from neural sequence models. arXiv (2016).
  70. Salimans, T. & Kingma, D.P. Weight normalization: a simple reparametrization to accelerate training of deep neural networks. Advances in neural information processing systems, 901–909 (2016).
  71. Ba, J.L. , Kiros, J.R. & Hinton, G. Layer normalization. arXiv (2016).
  72. 2014, Dropout: a simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15, 1929
  73. Kingma, D.P. & Ba, J.L. Adam: a method for stochastic optimization. arXiv (2014).
  74. 10.1016/0022-2836(82)90515-0 · DOI · Paper page
  75. 10.1002/elps.11501401163 · DOI · Paper page
  76. 10.1093/bioinformatics/btp163 · DOI · Paper page

Cited by

See all 32

Metadata

Authors
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., & Marks, D. S.
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2019-09-05
OpenAlex
W2971803505
Cited by
32
References
33
Type
posted content
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.