CoChat

Deep neural language modeling enables functional protein generation across families

Open access
Madani, Ali, Krause, Ben, Greene, Eric R., Subramanian, Subu, Mohr, Benjamin P., Holton, James M., Olmos, Jose Luis, Xiong, Caiming, Sun, Zachary Z., Socher, Richard, Fraser, James S., Naik, Nikhil · 2021 · bioRxiv (Cold Spring Harbor Laboratory)
PreprintCited by 41

The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.

Abstract

Bypassing nature’s evolutionary trajectory, de novo protein generation—defined as creating artificial protein sequences from scratch—could enable breakthrough solutions for biomedical and environmental challenges. Viewing amino acid sequences as a language, we demonstrate that a deep learning-based language model can generate functional artificial protein sequences across families, akin to generating grammatically and semantically correct natural language sentences on diverse topics. Our protein language model is trained by simply learning to predict the next amino acid for over 280 million protein sequences from thousands of protein families, without biophysical or coevolutionary modeling. We experimentally evaluate model-generated artificial proteins on five distinct antibacterial lysozyme families. Artificial proteins show similar activities and catalytic efficiencies as representative natural lysozymes, including hen egg white lysozyme, while reaching as low as 44% identity to any known naturally-evolved protein. The X-ray crystal structure of an enzymatically active artificial protein recapitulates the conserved fold and positioning of active site residues found in natural proteins. We demonstrate our language model’s ability to be adapted to different protein families by accurately predicting the functionality of artificial chorismate mutase and malate dehydrogenase proteins. These results indicate that neural language models successfully perform de novo protein generation across protein families and may prove to be a tool to shortcut evolution.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/2021.07.18.452833

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 24 references

Research works printed but not linked: 45

  1. 10.1073/pnas.1509508112 · DOI · Paper page
  2. 10.1038/nchembio.1966 · DOI · Paper page
  3. 10.1126/science.aad8865 · DOI · Paper page
  4. 10.1093/bioinformatics/bty862 · DOI · Paper page
  5. 2020, Signal Peptides Generated by Attention-Based Neural Networks, ACS Synth. Biol, 9, 2154, 10.1021/acssynbio.0c00219 · DOI · Paper page
  6. 2021, Deep diversification of an AAV capsid protein by machine learning, Nat. Biotechnol, 39, 691, 10.1038/s41587-020-00793-4 · DOI · Paper page
  7. 2021, Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations, Nat Biomed Eng, 5, 613, 10.1038/s41551-021-00689-x · DOI · Paper page
  8. 10.1093/bioinformatics/bth191 · DOI · Paper page
  9. 10.1093/nar/gni148 · DOI · Paper page
  10. 10.1093/nar/gkt1223 · DOI · Paper page
  11. Vaswani, A. et al. Attention Is All You Need. arXiv [cs.CL] (2017).
  12. Zellers, R. et al. Defending Against Neural Fake News. arXiv [cs.CL] (2019).
  13. 10.2307/412586 · DOI · Paper page
  14. Poplack, S. Sometimes I’ll start a sentence in Spanish Y TERMINO EN ESPAÑOL: toward a typology of code-switching1. Linguistics vol. 18 (1980). · DOI · Paper page
  15. Madani, A. et al. ProGen: Language Modeling for Protein Generation. arXiv [q-bio.BM] (2020). · DOI · Paper page
  16. 10.1162/153244303322533223 · DOI · Paper page
  17. 10.21203/rs.2.13774/v1 · DOI
  18. 10.1093/nar/gkr1178 · DOI · Paper page
  19. 10.1016/0005-2795(75)90109-9 · DOI · Paper page
  20. 2018, Catalytic diversity and cell wall binding repeats in the phage-encoded endolysins, Mol. Microbiol, 110, 879, 10.1111/mmi.14134 · DOI
  21. 2020, On the catalytic mechanism of bacteriophage endolysins: Opportunities for engineering, Biochim. Biophys. Acta: Proteins Proteomics, 1868, 140302, 10.1016/j.bbapap.2019.140302 · DOI
  22. Martin, P. P. Potts Models And Related Problems In Statistical Mechanics. (World Scientific, 1991). · DOI · Paper page
  23. 2015, Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models, PLoS Comput. Biol, 11, e1004182, 10.1371/journal.pcbi.1004182 · DOI · Paper page
  24. Goodfellow, I. J. et al. Generative Adversarial Networks. arXiv [stat.ML] (2014).
  25. Koehn, P. Pharaoh: A Beam Search Decoder for Phrase-Based Statistical Machine Translation Models. in Machine Translation: From Real Users to Research 115–124 (Springer Berlin Heidelberg, 2004). · DOI · Paper page
  26. 10.1093/bioinformatics/btab184 · DOI · Paper page
  27. 2009, Evaluation at atomic resolution of the role of strain in destabilizing the temperature-sensitive T4 lysozyme mutant Arg 96 → His, Protein Science, 18, 863, 10.1002/pro.93 · DOI
  28. 10.1126/science.8266098 · DOI · Paper page
  29. 10.1093/nar/gkv360 · DOI · Paper page
  30. 10.1101/846279 · DOI
  31. Ruder, S. , Peters, M. E. , Swayamdipta, S. & Wolf, T. Transfer Learning in Natural Language Processing. in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials 15–18 (Association for Computational Linguistics, 2019). · DOI · Paper page
  32. Huh, M. , Agrawal, P. & Efros, A. A. What makes ImageNet good for transfer learning? arXiv [cs.CV] (2016).
  33. The IUPAC stability constants database. Chem. Int. 28, (2006). · doi:10.1515/ci.2006.28.5.14
  34. 10.1038/75556 · DOI · Paper page
  35. Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. arXiv [cs.LG] (2014).
  36. 2013, in Proceedings of the 30th International Conference on Machine Learning (eds, 28, 1310
  37. 10.1214/12-AOS1000. 1102.4807 · DOI · Paper page
  38. Holtzman, A. , Buys, J. , Du, L. , Forbes, M. & Choi, Y. The Curious Case of Neural Text Degeneration. in Eighth International Conference on Learning Representations (2020).
  39. Sun, Z. Z. et al. Protocols for implementing an Escherichia coli based TX-TL cell-free expression system for synthetic biology. J. Vis. Exp. e50762 (2013). · DOI · Paper page
  40. 10.1107/S0907444909047337 · DOI · Paper page
  41. 10.1107/S0021889807021206 · DOI · Paper page
  42. 10.1107/S2059798318000979 · DOI · Paper page
  43. 2020, Current approaches for automated model building into cryo-EM maps using Buccaneer with CCP-EM, Acta Crystallogr D Struct Biol, 76, 531, 10.1107/S2059798320005513 · DOI · Paper page
  44. 10.1107/S0907444910007493 · DOI · Paper page
  45. 10.1107/S0907444912001308 · DOI · Paper page

Cited by

See all 41

Metadata

Authors
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., Fraser, J. S., & Naik, N.
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2021-07-18
OpenAlex
W3186612807
Cited by
41
References
24
Type
posted content
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Deep neural language modeling enables functional protein generation across families. https://app.cochat.ai/papers/10.1101/2021.07.18.452833
The paper
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., Fraser, J. S., & Naik, N. (2021). Deep neural language modeling enables functional protein generation across families. bioRxiv (Cold Spring Harbor Laboratory). https://doi.org/10.1101/2021.07.18.452833

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.