CoChat
Sign Up
PreprintCited by 83Protein Structure and DynamicsMachine Learning in BioinformaticsComputational Drug Discovery Methods

Illuminating protein space with a programmable generative model

Open access
Ingraham, John, Baranov, Max, Costello, Zak, Frappier, Vincent, Ismail, Ahmed, Tie, Shan, Wang, Wujie, Xue, Vincent +3 more·2022·bioRxiv (Cold Spring Harbor Laboratory)
DOI 10.1101/2022.12.01.518682

Preprint — published as 10.1038/s41586-023-06728-8 · Paper page

Open this paper in CoChat — free accountOpen PDF

Abstract

Three billion years of evolution have produced a tremendous diversity of protein molecules, and yet the full potential of this molecular class is likely far greater. Accessing this potential has been challenging for computation and experiments because the space of possible protein molecules is much larger than the space of those likely to host function. Here we introduce Chroma, a generative model for proteins and protein complexes that can directly sample novel protein structures and sequences and that can be conditioned to steer the generative process towards desired properties and functions. To enable this, we introduce a diffusion process that respects the conformational statistics of polymer ensembles, an efficient neural architecture for molecular systems based on random graph neural networks that enables long-range reasoning with sub-quadratic scaling, equivariant layers for efficiently synthesizing 3D structures of proteins from predicted inter-residue geometries, and a general low-temperature sampling algorithm for diffusion models. We suggest that Chroma can effectively realize protein design as Bayesian inference under external constraints, which can involve symmetries, substructure, shape, semantics, and even natural language prompts. With this unified approach, we hope to accelerate the prospect of programming protein matter for human health, materials science, and synthetic biology.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/2022.12.01.518682

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The in silico success measure used (AF2 single-sequence prediction high confidence, global r.m.s.d. <2 Å, scaffolded site r.m.s.d. <1 Å) has been found to correlate with experimental success and is significantly more stringent than TM-score-based metrics used elsewhere.

    “This measure of in silico success has been found to correlate with experimental success4,7,26 and is significantly more stringent than template modelling (TM)-score-based metrics used elsewhere5,16,27–29”

    From De novo design of protein structure and function with RFdiffusion (2023) · cites this paper as [28]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 68

  1. Po-Ssu Huang, Scott E Boyken, and David Baker. The coming of age of de novo protein design. Nature, 537(7620):320-327, September 2016.
  2. Nobuyasu Koga, Rie Tatsumi-Koga, Gaohua Liu, Rong Xiao, Thomas B Acton, Gaetano T Mon- telione, and David Baker. Principles for designing ideal protein structures. Nature, 491(7423): 222-227, November 2012.
  3. Longxing Cao, Brian Coventry, Inna Goreshnik, Buwei Huang, William Sheffler, Joon Sung Park, Kevin M Jude, Iva Marković, Rameshwar U Kadam, Koen H G Verschueren, Kenneth Ver- straete, Scott Thomas Russell Walsh, Nathaniel Bennett, Ashish Phal, Aerin Yang, Lisa Ko- zodoy, Michelle DeWitt, Lora Picton, Lauren Miller, Eva-Maria Strauch, Nicholas D DeBouver, Allison Pires, Asim K Bera, Samer Halabiya, Bradley Hammerson, Wei Yang, Steffen Bernard, Lance Stewart, Ian A Wilson, Hannele Ruohola-Baker, Joseph Schlessinger, Sangwon Lee, Sav- vas N Savvides, K Christopher Garcia, and David Baker. Design of protein-binding proteins from the target structure alone. Nature, 605(7910):551-560, May 2022.
  4. Hajo Kries, Rebecca Blomberg, and Donald Hilvert. De novo enzymes by computational design. Curr. Opin. Chem. Biol., 17(2):221-228, April 2013.
  5. Nathan H Joh, Tuo Wang, Manasi P Bhate, Rudresh Acharya, Yibing Wu, Michael Grabe, Mei Hong, Gevorg Grigoryan, and William F DeGrado. De novo design of a transmembrane zn2+- transporting four-helix bundle. Science, 346(6216):1520-1524, December 2014.
  6. John Maynard Smith. Natural selection and the concept of a protein space. Nature, 225(5232): 563-564, 1970.
  7. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821-8831. PMLR, 2021.
  8. Adam J Riesselman, John B Ingraham, and Debora S Marks. Deep generative models of genetic variation capture the effects of mutations. Nature methods, 15(10):816-822, 2018.
  9. Joe G Greener, Lewis Moffat, and David T Jones. Design of metalloproteins and novel protein folds using variational autoencoders. Scientific reports, 8(1):1-12, 2018.
  10. John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. Advances in neural information processing systems, 32, 2019.
  11. Namrata Anand, Raphael Eguchi, Irimpan I Mathews, Carla P Perez, Alexander Derry, Russ B Altman, and Po-Ssu Huang. Protein sequence design with a learned potential. Nature commu- nications, 13(1):1-11, 2022.
  12. Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena Hurtado, Aidan N Gomez, Debora Marks, and Yarin Gal. Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. In International Conference on Machine Learning, pages 16990- 17017.
  13. PMLR, 2022.
  14. Namrata Anand and Possu Huang. Generative modeling for protein structures. Advances in neural information processing systems, 31, 2018.
  15. Zeming Lin, Tom Sercu, Yann LeCun, and Alexander Rives. Deep generative models create new and diverse protein structures. In Machine Learning for Structural Biology Workshop, NeurIPS, 2021.
  16. Josh Barnes and Piet Hut. A hierarchical O(n log n) force-calculation algorithm. Nature, 324 (6096):446-449, 1986.
  17. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256-2265. PMLR, 2015.
  18. Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pages 1263-1272. PMLR, 2017.
  19. Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. bioRxiv, 2022.
  20. Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Ad- vances in Neural Information Processing Systems, 34:8780-8794, 2021.
  21. Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. URL https://openreview. net/forum?id=qw8AKxfYbI.
  22. John J Tanner. Empirical power laws for the radii of gyration of protein oligomers. Acta Crystal- lographica Section D: Structural Biology, 72(10):1119-1129, 2016.
  23. Ivan Anishchenko, Samuel J Pellock, Tamuka M Chidyausiku, Theresa A Ramelot, Sergey Ovchinnikov, Jingzhou Hao, Khushboo Bafna, Christoffer Norn, Alex Kang, Asim K Bera, Frank DiMaio, Lauren Carter, Cameron M Chow, Gaetano T Montelione, and David Baker. De novo protein design by deep network hallucination. Nature, 600(7889):547-552, December 2021.
  24. Yang Zhang and Jeffrey Skolnick. Tm-align: a protein structure alignment algorithm based on the tm-score. Nucleic acids research, 33(7):2302-2309, 2005.
  25. Jianfu Zhou, Alexandra E. Panaitiu, and Gevorg Grigoryan. A general-purpose protein de- sign framework based on mining sequence-structure relationships in known protein struc- tures. Proceedings of the National Academy of Sciences, 117(2):1059-1068, 2019. doi: 10.1073/pnas.1908723117. The UniProt Consortium. UniProt: the universal protein knowledgebase in 2021. Nucleic Acids Research, 49(D1):D480-D489, November 2020. doi: 10.1093/nar/gkaa1100. URL https: //doi.org/10.1093/nar/gkaa1100. · DOI
  26. Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5-6):355-607, 2019.
  27. Peter Røgen and Boris Fain. Automatic classification of protein structure by using gauss integrals. Proceedings of the National Academy of Sciences, 100(1):119-124, 2003.
  28. Tim Harder, Mikael Borg, Wouter Boomsma, Peter Røgen, and Thomas Hamelryck. Fast large- scale clustering of protein structures using gauss integrals. Bioinformatics, 28(4):510-515, 2012.
  29. Kevin P Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.
  30. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
  31. Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems, 34:21696-21707, 2021.
  32. Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326, 1982.
  33. Dimitra Maoutsa, Sebastian Reich, and Manfred Opper. Interacting particle solutions of fokker- planck equations through gradient-log-density estimation. Entropy, 22(8):802, 2020.
  34. Bowen Jing, Gabriele Corso, Renato Berlinghieri, and Tommi Jaakkola. Subspace diffusion gen- erative models. arXiv preprint arXiv:2205.01490, 2022. · arXiv
  35. Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
  36. Wouter Kool, Herke Van Hoof, and Max Welling. Stochastic beams and where to find them: The gumbel-top-k trick for sampling sequences without replacement. In International Conference on Machine Learning, pages 3499-3508. PMLR, 2019.
  37. David JC MacKay. Information theory, inference and learning algorithms. Cambridge university press, 2003.
  38. Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribu- tion. Advances in Neural Information Processing Systems, 32, 2019.
  39. Enzo Marinari and Giorgio Parisi. Simulated tempering: a new monte carlo scheme. EPL (Euro- physics Letters), 19(6):451, 1992.
  40. Ulrich HE Hansmann. Parallel tempering algorithm for conformational studies of biological molecules. Chemical Physics Letters, 281(1-3):140-150, 1997.
  41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  42. Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. Big bird: Transformers for longer sequences. Advances in Neural Information Processing Systems, 33:17283-17297, 2020.
  43. Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jian- feng Gao. Focal attention for long-range interactions in vision transformers. In Ad- vances in Neural Information Processing Systems, volume 34, pages 30008-30022.
  44. Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. In 9th ISCA Speech Synthesis Workshop, pages 125-125, 2016.
  45. Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Žídek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706-710, 2020.
  46. Debora S Marks, Thomas A Hopf, and Chris Sander. Protein structure prediction from sequence variation. Nature biotechnology, 30(11):1072-1080, 2012.
  47. David Belanger and Andrew McCallum. Structured prediction energy networks. In International Conference on Machine Learning, pages 983-992. PMLR, 2016.
  48. John Ingraham, Adam Riesselman, Chris Sander, and Debora Marks. Learning protein structure with a differentiable simulator. In International Conference on Learning Representations, 2018.
  49. Samuel Schoenholz and Ekin Dogus Cubuk. Jax md: a framework for differentiable physics. Advances in Neural Information Processing Systems, 33:11428-11441, 2020.
  50. Wujie Wang, Simon Axelrod, and Rafael Gómez-Bombarelli. Differentiable molecular simulations for control and learning. In ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
  51. Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallo- graphica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 32 (5):922-923, 1976.
  52. Sidhartha Chaudhury, Sergey Lyskov, and Jeffrey J Gray. Pyrosetta: a script-based interface for implementing molecular modeling algorithms using rosetta. Bioinformatics, 26(5):689-691, 2010.
  53. Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(11):1026-1028, 2017.
  54. D Frishman and P Argos. Knowledge-based protein secondary structure assignment. Proteins, 23 (4):566-579, 1995.
  55. DN Ivankov, SO Garbuzynskiy, E Alm, KW Plaxco, D Baker, and AV Finkelstein. Contact order revisited: influence of protein size on the folding rate. Protein Sci, 12(9):2057-2062, 2003.
  56. M Borg, K Mardia, W Boomsma, J Frellsen, T Harder, K Stovgaard, J Ferkinghoff-Borg, P Røgen, and T Hamelryck. A probabilistic approach to protein structure prediction: Phaistos in casp9. LASR2009-Statistical tools for challenges in bioinformatics, pages 65-70, 2009.
  57. K T Simons, C Kooperberg, E Huang, and D Baker. Assembly of protein tertiary structures from fragments with similar local sequences using simulated annealing and bayesian scoring func- tions. J. Mol. Biol., 268(1):209-225, April 1997.
  58. Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, and Chris Maddison. Oops i took a gradient: Scalable sampling for discrete distributions. In International Conference on Machine Learning, pages 3831-3841. PMLR, 2021.
  59. Jianfu Zhou and Gevorg Grigoryan. Rapid search for tertiary fragments reveals protein sequence- structure relationships. Protein Sci., 24(4):508-524, April 2015.
  60. Jianfu Zhou and Gevorg Grigoryan. A c++ library for protein sub-structure search. bioRxiv preprint 2020.04.26.062612, 2020b.
  61. David S Goodsell and Arthur J Olson. Structural symmetry and protein function. Annual review of biophysics and biomolecular structure, 29:105, 2000.
  62. Yang Hsia, Jacob B Bale, Shane Gonen, Dan Shi, William Sheffler, Kimberly K Fong, Una Nat- termann, Chunfu Xu, Po-Ssu Huang, Rashmi Ravichandran, et al. Design of a hyperstable 60-subunit protein icosahedron. Nature, 535(7610):136-139, 2016.
  63. Taco Cohen and Max Welling. Group equivariant convolutional networks. In International con- ference on machine learning, pages 2990-2999. PMLR, 2016.
  64. Hagit Zabrodsky, Shmuel Peleg, and David Avnir. Continuous symmetry measures. Journal of the American Chemical Society, 114(20):7843-7851, 1992.
  65. Stephen C Harvey, Robert K-Z Tan, and Thomas E Cheatham III. The flying ice cube: velocity rescaling in molecular dynamics leads to violation of energy equipartition. Journal of computa- tional chemistry, 19(7):726-740, 1998.
  66. Justin Solomon, Gabriel Peyré, Vladimir G Kim, and Suvrit Sra. Entropic metric alignment for correspondence problems. ACM Transactions on Graphics (ToG), 35(4):1-13, 2016.
  67. David Alvarez-Melis and Tommi S Jaakkola. Gromov-wasserstein alignment of word embedding spaces. In EMNLP, 2018.
  68. Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let net- works learn high frequency functions in low dimensional domains. Advances in Neural Infor- mation Processing Systems, 33:7537-7547, 2020.

Cited by

Sorted by citations

Authors
Ingraham, J., Baranov, M., Costello, Z., Frappier, V., Ismail, A., Tie, S., Wang, W., Xue, V., Obermeyer, F., Beam, A., & Grigoryan, G.
Corresponding author
Grigoryan, Gevorg
ORCID
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2022-12-02
DOI
10.1101/2022.12.01.518682
OpenAlex
W4311179098
Cited by
83
References
47
Citations by year
2026: 2 · 2025: 18 · 2024: 26 · 2023: 34 · 2022: 2
Type
posted content
Language
en
Open access
Locations
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Desy publication database (The Deutsches Elektronen-Synchrotron) (submittedVersion, cc-by)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
Keywords
protein design, generative models, protein complexes, diffusion models, graph neural networks, Bayesian inference, protein sequence generation, protein structure generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review