The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Single-sequence protein structure prediction using a language model and deep learning
Open accessPublished version of a preprint
Abstract
AlphaFold2 and related computational systems predict protein structure using deep learning and co-evolutionary relationships encoded in multiple sequence alignments (MSAs). Despite high prediction accuracy achieved by these systems, challenges remain in (1) prediction of orphan and rapidly evolving proteins for which an MSA cannot be generated; (2) rapid exploration of designed structures; and (3) understanding the rules governing spontaneous polypeptide folding in solution. Here we report development of an end-to-end differentiable recurrent geometric network (RGN) that uses a protein language model (AminoBERT) to learn latent structural information from unaligned proteins. A linked geometric module compactly represents C α backbone geometry in a translationally and rotationally invariant way. On average, RGN2 outperforms AlphaFold2 and RoseTTAFold on orphan proteins and classes of designed proteins while achieving up to a 10 6-fold reduction in compute time. These findings demonstrate the practical and theoretical strengths of protein language models relative to MSAs in structure prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“predict protein structure [8]”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
GROMACS 4: Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation
Hess, B., Kutzner, C., van der Spoel, D., & Lindahl, E. R. (2008) · Journal of Chemical Theory and Computation · doi:10.1021/ct700301q · Cited by 15,830
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“…potentially billions of protein sequences are increasingly available 26,35,36 , and the scaling previously observed in natural language applications 47 makes it likely that the performance of RGN2 and similar methods will continue to improve and become broadly performative over intrinsically…”
BackgroundAccurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
Automatic atom type and bond type perception in molecular mechanical calculations
Wang, J., Wang, W., Kollman, P. A., & Case, D. A. (2006) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2005.12.005 · Cited by 5,517
Improved protein structure prediction using potentials from deep learning
Senior, A. W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., Žídek, A., Nelson, A. W. R., Bridgland, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1923-7 · Cited by 3,378
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 3×
“Language models were first developed as a means to extract semantic information from a sequence of words (a key requirement for natural language processing) 26 .”
BackgroundMethodI-TASSER server: new development for protein structure and function predictions
Yang, J., & Zhang, Y. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv342 · Cited by 2,486
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329
Burley, S. K., Bhikadiya, C., Bi, C., Bittrich, S., Chen, L., Crichlow, G. V., Christie, C. H., Dalenberg, K., Di Costanzo, L., Duarte, J. M., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1038 · Cited by 1,581
The Rosetta All-Atom Energy Function for Macromolecular Modeling and Design
Alford, R. F., Leaver-Fay, A., Jeliazkov, J. R., O’Meara, M. J., DiMaio, F. P., Park, H., Shapovalov, M. V., Renfrew, P. D., Mulligan, V. K., Kappel, K., et al. (2017) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.7b00125 · Cited by 1,532
Improved protein structure prediction using predicted interresidue orientations
Yang, J., Anishchenko, I., Park, H., Peng, Z., Ovchinnikov, S., & Baker, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1914677117 · Cited by 1,421
A series of PDB related databases for everyday needs
Joosten, R. P., Beek, T. A. H. T., Krieger, E., Hekkelman, M. L., Hooft, R. W. W., Schneider, R., Sander, C., & Vriend, G. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1105 · Cited by 1,313
THE KINETICS OF FORMATION OF NATIVE RIBONUCLEASE DURING OXIDATION OF THE REDUCED POLYPEPTIDE CHAIN
Anfinsen, C. B., Haber, E., Sela, M., & White, F. H. (1961) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.47.9.1309 · Cited by 1,182
De Novo Computational Design of Retro-Aldol Enzymes
Jiang, L., Althoff, E. A., Clemente, F. R., Doyle, L., Röthlisberger, D., Zanghellini, A., Gallaher, J. L., Betker, J. L., Tanaka, F., Barbas, C. F., et al. (2008) · Science · doi:10.1126/science.1152692 · Cited by 1,174
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131
Self-citationXu, D. H., & Zhang, Y. (2011) · Biophysical Journal · doi:10.1016/j.bpj.2011.10.024 · Cited by 1,039
An Introduction to Sequence Similarity (“Homology”) Searching
Pearson, W. R. (2013) · Current Protocols in Bioinformatics · doi:10.1002/0471250953.bi0301s42 · Cited by 946
A series of PDB-related databanks for everyday needs
Touw, W. G., Baakman, C. A. B., Black, J., Beek, T. A. H. T., Krieger, E., Joosten, R. P., & Vriend, G. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku1028 · Cited by 941
The evolutionary origin of orphan genes
Tautz, D., & Domazet-Lošo, T. (2011) · Nature Reviews Genetics · doi:10.1038/nrg3053 · Cited by 764
Fox, N. K., Brenner, S. E., & Chandonia, J.-M. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1240 · Cited by 762
Uniclust databases of clustered and deeply annotated protein sequences and alignments
Mirdita, M., von den Driesch, L., Galiez, C., Martin, M. J., Söding, J., & Steinegger, M. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw1081 · Cited by 726
RosettaScripts: A Scripting Language Interface to the Rosetta Macromolecular Modeling Suite
Fleishman, S. J., Leaver‐Fay, A., Corn, J. E., Strauch, E., Khare, S. D., Koga, N., Ashworth, J. R., Murphy, P. M., Richter, F. M., Lemmon, G., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020161 · Cited by 659
Language models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.07.09.450648 · Cited by 635
Modeling the language of life – Deep Learning Protein Sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · doi:10.1101/614313 · Cited by 606
Modeling aspects of the language of life through transfer-learning protein sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3220-8 · Cited by 606
Renata, H., Wang, Z. J., & Arnold, F. H. (2015) · Angewandte Chemie International Edition · doi:10.1002/anie.201409470 · Cited by 529
Strategies for training large scale neural network language models
Mikolov, T., Deoras, A., Povey, D., Burget, L., & Černocký, J. (2011) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2011.6163930 · Cited by 516
A wellness study of 108 individuals using personal, dense, dynamic data clouds
Price, N. D., Magis, A. T., Earls, J. C., Glusman, G., Levy, R., Lausted, C., McDonald, D. T., Kusebauch, U., Moss, C. L., Zhou, Y., et al. (2017) · Nature Biotechnology · doi:10.1038/nbt.3870 · Cited by 427
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 424
Relaxation of backbone bond geometry improves protein energy landscape modeling
Conway, P., Tyka, M. D., DiMaio, F., Konerding, D. E., & Baker, D. A. (2013) · Protein Science · doi:10.1002/pro.2389 · Cited by 408
ProGen: Language Modeling for Protein GenerationPreprint
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi, R. R., Huang, P., & Socher, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.07.982272 · Cited by 383
End-to-End Differentiable Learning of Protein Structure
AlQuraishi, M. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.006 · Cited by 371
Self-citationSynthesis of proteins by automated flow chemistry
Hartrampf, N., Saebi, A., Poskus, M., Gates, Z. P., Callahan, A. J., Cowfer, A. E., Hanna, S., Antilla, S., Schissel, C. K., Quartararo, A. J., et al. (2020) · Science · doi:10.1126/science.abb2491 · Cited by 367
De Novo Enzyme Design Using Rosetta3
Richter, F. M., Leaver‐Fay, A., Khare, S. D., Bjelic, S., & Baker, D. A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019230 · Cited by 359
Rao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341
State-of-the-Art Estimation of Protein Model Accuracy Using AlphaFold
Roney, J. P., & Ovchinnikov, S. (2022) · Physical Review Letters · doi:10.1103/physrevlett.129.238101 · Cited by 281
Machine learning in protein structure prediction
AlQuraishi, M. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.04.005 · Cited by 263
Self-citationLeinonen, R., Diez, F. G., Binns, D., Fleischmann, W., López, R., & Apweiler, R. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth191 · Cited by 217
Unexpected features of the dark proteome
Perdigão, N., Heinrich, J., Stolte, C., Sabir, K. S., Buckley, M. J., Tabor, B., Signal, B., Gloss, B., Hammang, C. J., Rost, B., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1508380112 · Cited by 210
Jiang, C., Hei, R., Yang, Y., Zhang, S., Wang, Q., Wang, W., Zhang, Q., Yan, M., Zhu, G., Huang, P., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18240-y · Cited by 193
RECENT ADVANCES IN RATIONAL APPROACHES FOR ENZYME ENGINEERING
Steiner, K., & Schwab, H. (2012) · Computational and Structural Biotechnology Journal · doi:10.5936/csbj.201209010 · Cited by 192
ProteinNet: a standardized data set for machine learning of protein structure
AlQuraishi, M. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2932-0 · Cited by 147
Self-citationEvoEF2: accurate and fast energy function for computational protein design
Huang, X., Pearce, R., & Zhang, Y. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz740 · Cited by 122
Hu, S., Lundgren, M., & Niemi, A. J. (2011) · Physical Review E · doi:10.1103/physreve.83.061908 · Cited by 105
Current structure predictors are not learning the physics of protein folding
Outeiral, C., Nissley, D. A., & Deane, C. M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btab881 · Cited by 82
Penner, R., Knudsen, M., Wiuf, C., & Andersen, J. E. (2010) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20340 · Cited by 57
Sáez-Jiménez, V., Fernández‐Fueyo, E., Medrano, F. J., Romero, A., Martı́nez, Á. T., & Ruiz-Dueñas, F. J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140984 · Cited by 51
Unified rational protein engineering with sequence-only deep representation learningPreprint
Alley, E. C., Khimulya, G., Biswas, S., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/589333 · Cited by 50
Self-citationKandathil, S. M., Greener, J. G., Lau, A. M., & Jones, D. T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2113348119 · Cited by 48
Park, H. J., Joo, J. C., Park, K., Kim, Y. H., & Yoo, Y. J. (2012) · Journal of Biotechnology · doi:10.1016/j.jbiotec.2012.11.006 · Cited by 44
Improved protein structure prediction by deep learning irrespective of co-evolution information
Xu, J., Mcpartlon, M., & Li, J. (2020) · doi:10.1101/2020.10.12.336859 · Cited by 18
Stittrich, A., Ashworth, J. R., Shi, M., Robinson, M., Mauldin, D. E., Brunkow, M. E., Biswas, S., Kim, J., Kwon, K., Jung, J. U., et al. (2016) · Human Genome Variation · doi:10.1038/hgv.2015.60 · Cited by 17
Self-citationKandathil, S. M., Greener, J. G., Lau, A. M., & Jones, D. T. (2020) · doi:10.1101/2020.11.27.401232 · Cited by 8
Machine Learning for Protein Structure Prediction
Selbig, J. (1994) · Studies in classification, data analysis, and knowledge organization · doi:10.1007/978-3-642-46808-7_36 · Cited by 2
Research works printed but not linked: 2
- Ingraham, J., Riesselman, A., Sander, C. & Marks, D. Learning protein structure with a differentiable simulator. in 7th International Conference on Learning Representations. https://openreview.net/forum?id=Byg3y3C9Km (2019).
- Li, J. Universal transforming geometric network. Preprint at https://arxiv.org/abs/1908.00723 (2019). · arXiv
Cited by
Sorted by citations
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Improved prediction of protein-protein interactions using AlphaFold2
Bryant, P., Pozzati, G., & Elofsson, A. (2021) · doi:10.1101/2021.09.15.460468 · Cited by 1,022
A structural biology community assessment of AlphaFold2 applications
Akdel, M., Pires, D. E. V., Pardo, E. P., Jänes, J., Zalevsky, A. O., Mészáros, B., Bryant, P., Good, L. L., Laskowski, R. A., Pozzati, G., et al. (2022) · Nature Structural & Molecular Biology · doi:10.1038/s41594-022-00849-w · Cited by 675
AlphaFold2 and its applications in the fields of biology and medicine
Yang, Z., Zeng, X., Zhao, Y. C., & Chen, R. (2023) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-023-01381-z · Cited by 655
Scaffolding protein functional sites using deep learning
Wang, J., Lisanza, S., Juergens, D., Tischer, D., Watson, J. L., Castro, K. M., Ragotte, R., Saragovi, A., Milles, L. F., Baek, M., et al. (2022) · Science · doi:10.1126/science.abn2100 · Cited by 487
Metadata
- Authors
- Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., Sorger, P. K., & AlQuraishi, M.
- Affiliations
- Harvard University (US)
- Columbia University (US)
- Corresponding author
- Bouatta, Nazim
- Sorger, Peter K.
- AlQuraishi, Mohammed
- ORCID
- Venue
- Nature Biotechnology
- Published
- 2022-10-03
- Volume
- 40
- Issue
- 11
- Pages
- 1617-1623
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC10440047
- OpenAlex
- W4300861364
- Cited by
- 455
- References
- 56
- Citations by year
- 2026: 49 · 2025: 103 · 2024: 135 · 2023: 124 · 2022: 37
- FWCI
- 32.66
- Type
- journal article
- Language
- en
- Update notices
- correction (2022-10-17)
- Open access
- green
- doi.org
- Licence
- Locations
- Nature Biotechnology (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Nvidia
- Defense Advanced Research Projects Agency (HR0011, HR0011-19-2-0022)
- National Cancer Institute (U54CA225088)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Structural Biology › Enzyme Structure and Function
- Keywords
- protein structure prediction, protein language models, orphan proteins, protein design, computational efficiency
- MeSH terms
- Deep Learning, Language, Protein Conformation, Proteins, Sequence Alignment, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
