Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 301–400 of 1,100
A review of deep learning methods for ligand based drug virtual screening
Wu, H., Liu, J., Zhang, R., Lu, Y., Cui, G., Cui, Z., & Ding, Y. (2024) · Fundamental Research · doi:10.1016/j.fmre.2024.02.011 · Cited by 75
Self-citationMasked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers
Choromański, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlós, T., Hawkins, P., Davis, J. Q., Belanger, D., Colwell, L. J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03555 · Cited by 28
Hou, Z., Yang, Y., Ma, Z., Wong, K., & Li, X. (2023) · Communications Biology · doi:10.1038/s42003-023-04462-5 · Cited by 67
Deep Learning Model for Efficient Protein–Ligand Docking with Implicit Side-Chain Flexibility
Masters, M. R., Mahmoud, A. H., Yao, W., & Lill, M. A. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01436 · Cited by 61
Diaz, D. J., Gong, C., Ouyang-Zhang, J., Loy, J. M., Wells, J., Yang, D., Ellington, A. D., Dimakis, A. G., & Klivans, A. R. (2024) · Nature Communications · doi:10.1038/s41467-024-49780-2 · Cited by 60
Progress and opportunities of foundation models in bioinformatics
Li, Q., Hu, Z.-H., Wang, Y., Li, L., Fan, Y.-M., King, I., Jia, G.-J., Wang, S., Song, L., & Li, Y. (2024) · Briefings Bioinform. · doi:10.1093/bib/bbae548 · Cited by 80
Evaluating the representational power of pre-trained DNA language models for regulatory genomics
Tang, Z., Somia, N., Yu, Y., & Koo, P. K. (2025) · Genome biology · doi:10.1186/s13059-025-03674-8 · Cited by 84
Persistent spectral theory-guided protein engineering
(2023) · doi:10.1038/s43588-022-00394-y · Cited by 62
Ziegler, S. J., Mallinson, S. J. B., John, P. C. S., & Bomble, Y. J. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.11.052 · Cited by 68
Lin, B., Luo, X., Liu, Y., & Jin, X. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae289 · Cited by 72
Simple and effective embedding model for single-cell biology built from ChatGPT
Chen, Y. T., & Zou, J. (2024) · Nature Biomedical Engineering · doi:10.1038/s41551-024-01284-6 · Cited by 258
Wang, W., Shuai, Y., Zeng, M., Fan, W., & Li, M. (2025) · Nature Communications · doi:10.1038/s41467-024-54816-8 · Cited by 79
Deep embedding and alignment of protein sequences
Llinares-López, F., Berthet, Q., Blondel, M., Teboul, O., & Vert, J.-P. (2023) · Nature Methods · doi:10.1038/s41592-022-01700-2 · Cited by 58
Li, T., Li, Y., Zhu, X., He, Y., Wu, Y., Ying, T., & Xie, Z. (2023) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2023.02.007 · Cited by 65
Accelerating drug target inhibitor discovery with a deep generative foundation model
Chenthamarakshan, V., Hoffman, S. C., Owen, C. D., Lukacik, P., Strain-Damerell, C. M., Fearon, D., Malla, T. R., Tumber, A., Schofield, C. J., Duyvesteyn, H. M. E., et al. (2023) · Science Advances · doi:10.1126/sciadv.adg7865 · Cited by 78
Dual-view Molecular Pre-training
Zhu, J., Xia, Y., Wu, L., Xie, S., Zhou, W., Qin, T., Li, H., & Liu, T. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599317 · Cited by 94
Integration of pre-trained protein language models into geometric deep learning networks
Wu, F., Wu, L., Radev, D., Xu, J., & Li, S. Z. (2023) · Communications Biology · doi:10.1038/s42003-023-05133-1 · Cited by 57
Deep Learning for Genomics: From Early Neural Nets to Modern Large Language Models
Yue, T., Wang, Y., Zhang, L., Gu, C., Xue, H., Wang, W., Lyu, Q., & Dun, Y. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms242115858 · Cited by 197
Large language models in plant biology
Lam, H. Y. I., Ong, X. E., & Mutwil, M. (2024) · Trends in Plant Science · doi:10.1016/j.tplants.2024.04.013 · Cited by 77
Evaluating generalizability of artificial intelligence models for molecular datasets
Ektefaie, Y., Shen, A., Bykova, D., Marin, M. G., Zitnik, M., & Farhat, M. (2024) · bioRxiv · doi:10.1038/s42256-024-00931-6 · Cited by 80
A high-level programming language for generative protein design
Hie, B., Candido, S., Lin, Z., Kabeli, O., Rao, R., Smetanin, N., Sercu, T., & Rives, A. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.21.521526 · Cited by 57
Self-citationPerturbNet predicts single-cell responses to unseen chemical and genetic perturbations
Yu, H., Qian, W., Song, Y., & Welch, J. D. (2025) · Molecular Systems Biology · doi:10.1038/s44320-025-00131-3 · Cited by 57
Deciphering microbial gene function using natural language processing
Miller, D., Stern, A., & Burstein, D. (2022) · Nature Communications · doi:10.1038/s41467-022-33397-4 · Cited by 55
In vitro continuous protein evolution empowered by machine learning and automation
Yu, T., Boob, A. G., Singh, N., Su, Y., & Zhao, H. (2023) · Cell Systems · doi:10.1016/j.cels.2023.04.006 · Cited by 57
Generative models for protein sequence modeling: recent advances and future directions
Mardikoraem, M., Wang, Z., Pascual, N., & Woldring, D. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad358 · Cited by 73
Next generation of multispecific antibody engineering
Kéri, D., Walker, M., Singh, I., Nishikawa, K. K., & Garcés, F. (2023) · Antibody Therapeutics · doi:10.1093/abt/tbad027 · Cited by 77
Notin, P., Van Niekerk, L., Kollasch, A. W., Ritter, D., Gal, Y., & Marks, D. S. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.07.519495 · Cited by 55
Protein language models are biased by unequal sequence sampling across the tree of life
Ding, F., & Steinhardt, J. (2024) · doi:10.1101/2024.03.07.584001 · Cited by 57
Pantolini, L., Studer, G., Pereira, J., Durairaj, J., Tauriello, G., & Schwede, T. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btad786 · Cited by 55
The Virtual Lab: AI Agents Design New SARS-CoV-2 Nanobodies with Experimental Validation
Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E., & Zou, J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.11.623004 · Cited by 56
Deep contrastive learning enables genome-wide virtual screening
Jia, Y., Gao, B., Tan, J., Zheng, J., Hong, X., Zhu, W., Tan, H., Xiao, Y., Tan, L., Cai, H., et al. (2026) · Science · doi:10.1126/science.ads9530 · Cited by 66
Chen, X., Li, C., Bernards, M. T., Shi, Y., Shao, Q., & He, Y. (2021) · Molecular Systems Design & Engineering · doi:10.1039/d0me00161a · Cited by 61
Chen, B., Cheng, X., Li, P., Geng, Y., Gong, J., Li, S., Bei, Z., Tan, X., Wang, B., Zeng, X., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02636-z · Cited by 240
Methods to Discover and Validate Biofluid-Based Biomarkers in Neurodegenerative Dementias
Teunissen, C. E., Kimble, L. E., Bayoumy, S., Bolsewig, K., Burtscher, F., Coppens, S., Das, S., Gogishvili, D., Gomes, B. F., José, N. G. D. S., et al. (2023) · Molecular & Cellular Proteomics · doi:10.1016/j.mcpro.2023.100629 · Cited by 63
Pei, Q., Zhang, W., Zhu, J., Wu, K., Gao, K., Wu, L., Xia, Y., & Yan, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.70 · Cited by 187
Data science opportunities of large language models for neuroscience and biomedicine
Bzdok, D., Thieme, A., Levkovskyy, O., Wren, P. B., Ray, T. S., & Reddy, S. (2024) · Neuron · doi:10.1016/j.neuron.2024.01.016 · Cited by 72
Large language models for science and medicine
Telenti, A., Auli, M., Hie, B. L., Maher, C., Saria, S., & Ioannidis, J. P. A. (2024) · European Journal of Clinical Investigation · doi:10.1111/eci.14183 · Cited by 54
Zeng, X., Li, S., Lv, S., Wen, M., & Li, Y. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2024.1375522 · Cited by 67
Wang, J., Yang, Z., Chen, C., Ge, Y., Wan, X., Bao, S., Ding, J., Wang, L., & Jiang, H. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae387 · Cited by 76
Dewaker, V., Morya, V. K., Kim, Y. H., Park, S. T., Kim, H. S., & Koh, Y. H. (2025) · Biomarker Research · doi:10.1186/s40364-025-00764-4 · Cited by 63
Fast and effective protein model refinement using deep graph neural networks
Jing, X., & Xu, J. (2021) · Nature Computational Science · doi:10.1038/s43588-021-00098-9 · Cited by 54
Mapping the glycosyltransferase fold landscape using interpretable deep learning
Taujale, R., Zhou, Z., Yeung, W., Moremen, K. W., Li, S., & Kannan, N. (2021) · Nature Communications · doi:10.1038/s41467-021-25975-9 · Cited by 58
De novo mass spectrometry peptide sequencing with a transformer modelPreprint
Yilmaz, M., Fondrie, W. E., Bittremieux, W., Oh, S., & Noble, W. S. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.02.07.479481 · Cited by 111
Generative power of a protein language model trained on multiple sequence alignments
(2023) · doi:10.7554/elife.79854 · Cited by 43
Pantolini, L., Studer, G., Pereira, J., Durairaj, J., & Schwede, T. (2022) · doi:10.1101/2022.12.13.520313 · Cited by 55
The generative capacity of probabilistic protein sequence models
McGee, F., Hauri, S., Novinger, Q., Vucetic, S., Levy, R. M., Carnevale, V., & Haldane, A. (2021) · Nature Communications · doi:10.1038/s41467-021-26529-9 · Cited by 54
Linguistically inspired roadmap for building biologically reliable protein language models
Vu, M. H., Akbar, R., Robert, P. A., Swiatczak, B., Sandve, G. K., Greiff, V., & Haug, D. T. T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00637-1 · Cited by 54
Machine Learning-Enabled Genome Mining and Bioactivity Prediction of Natural Products
Yuan, Y., Shi, C., & Zhao, H. (2023) · ACS Synthetic Biology · doi:10.1021/acssynbio.3c00234 · Cited by 59
A Perspective on Protein Structure Prediction Using Quantum Computers
Doğa, H., Raubenolt, B. A., Cumbo, F., Joshi, J., DiFilippo, F. P., Qin, J., Blankenberg, D. J., & Shehab, O. (2024) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c00067 · Cited by 79
Large language models for drug discovery and development
Zheng, Y., Koh, H. Y., Ju, J., Yang, M., May, L. T., Webb, G. I., Li, L., Pan, S., & Church, G. M. (2025) · Patterns · doi:10.1016/j.patter.2025.101346 · Cited by 133
ProT-VAE: Protein Transformer Variational AutoEncoder for functional protein design
Sevgen, E., Moller, J., Lange, A., Parker, J., Quigley, S., Mayer, J., Srivastava, P., Gayatri, S., Hosfield, D., Dilks, C., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2408737122 · Cited by 52
Pre-Training of Deep Bidirectional Protein Sequence Representations With Structural Information
Min, S., Park, S., Kim, S., Choi, H.-S., Lee, B., & Yoon, S. (2021) · IEEE Access · doi:10.1109/access.2021.3110269 · Cited by 53
Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks
Nambiar, A., Liu, S., Hopkins, M., Heflin, M., Maslov, S., & Ritz, A. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.15.153643 · Cited by 43
SETH predicts nuances of residue disorder from protein embeddings
Ilzhöfer, D., Heinzinger, M., & Rost, B. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.1019597 · Cited by 53
Deep learning methods for protein function prediction
Boadu, F., Lee, A., & Cheng, J. (2025) · PROTEOMICS · doi:10.1002/pmic.202300471 · Cited by 53
Interpretable and explainable predictive machine learning models for data-driven protein engineering
Medina-Ortiz, D., Khalifeh, A., Anvari-Kazemabad, H., & Davari, M. D. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108495 · Cited by 65
GraphBAN: An inductive graph-based approach for enhanced prediction of compound-protein interactions
Hadipour, H., Li, Y. Y., Sun, Y., Deng, C., Lac, L., Davis, R. L., Cardona, S. T., & Hu, P. (2025) · Nature Communications · doi:10.1038/s41467-025-57536-9 · Cited by 61
Protein tertiary structure prediction and refinement using deep learning and Rosetta in CASP14
Anishchenko, I., Baek, M., Park, H., Hiranuma, N., Kim, D. E., Dauparas, J., Mansoor, S., Humphreys, I. R., & Baker, D. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26194 · Cited by 50
Liu, Y., Zhang, R., Li, T., Jiang, J., Ma, J., & Wang, P. (2022) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2022.108344 · Cited by 62
Self-citationUsing machine learning to predict the effects and consequences of mutations in proteins
Diaz, D. J., Kulikova, A. V., Ellington, A. D., & Wilke, C. O. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2022.102518 · Cited by 52
Deep learning for protein complex structure prediction
Bryant, P. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2023.102529 · Cited by 52
Antibody design using deep learning: from sequence and structure design to affinity maturation
Joubbi, S., Micheli, A., Milazzo, P., Maccari, G., Ciano, G., Cardamone, D., & Medini, D. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae307 · Cited by 67
Chen, J.-Y., Wang, J.-F., Hu, Y., Li, X.-H., Qian, Y.-R., & Song, C.-L. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1506508 · Cited by 49
PANDA2: protein function prediction using graph neural networks
Zhao, C.-G., Liu, T., & Wang, Z. (2022) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqac004 · Cited by 45
Improved global protein homolog detection with major gains in function identification
Kilinc, M., Jia, K., & Jernigan, R. L. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2211823120 · Cited by 51
Structure-aware protein self-supervised learning
Chen, C. (., Zhou, J., Wang, F., Liu, X., & Dou, D. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad189 · Cited by 49
Learning protein fitness landscapes with deep mutational scanning data from multiple sources
Chen, L., Zhang, Z., Li, Z., Li, R., Huo, R., Chen, L., Wang, D., Luo, X., Chen, K., Liao, C., et al. (2023) · Cell Systems · doi:10.1016/j.cels.2023.07.003 · Cited by 61
Deep mutational scanning for therapeutic antibody engineering
Hanning, K. R., Minot, M., Warrender, A. K., Kelton, W., & Reddy, S. T. (2021) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2021.11.010 · Cited by 58
DLPacker : Deep learning for prediction of amino acid side chain conformations in proteins
Misiura, M., Shroff, R., Thyer, R., & Kolomeisky, A. B. (2022) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26311 · Cited by 51
Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models
Li, F.-Z., Amini, A. P., Yue, Y., Yang, K. K., & Lu, A. X. (2024) · doi:10.1101/2024.02.05.578959 · Cited by 51
An analysis of protein language model embeddings for fold prediction
Villegas-Morcillo, A., Gomez, A. M., & Sanchez, V. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac142 · Cited by 46
Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence
Duran-Frigola, M., Cigler, M., & Winter, G. E. (2023) · Journal of the American Chemical Society · doi:10.1021/jacs.2c11098 · Cited by 50
DeepNOG: fast and accurate protein orthologous group assignment
Feldbauer, R., Gosch, L., Lüftinger, L., Hyden, P., Flexer, A., & Rattei, T. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa1051 · Cited by 57
LectinOracle: A Generalizable Deep Learning Model for Lectin–Glycan Binding Prediction
(2021) · doi:10.1002/advs.202103807 · Cited by 50
Yang, M., Huang, L., Huang, H., Tang, H., Zhang, N., Yang, H., Wu, J., & Mu, F. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac326 · Cited by 67
Deep learning for protein secondary structure prediction: Pre and post-AlphaFold
Ismi, D. P., Pulungan, R., & Afiahayati (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.11.012 · Cited by 65
The RESP AI model accelerates the identification of tight-binding antibodies
Parkinson, J., Hard, R. L., & Wang, W. (2023) · Nature Communications · doi:10.1038/s41467-023-36028-8 · Cited by 58
Li, P., & Liu, Z.-P. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad288 · Cited by 50
Addressing epistasis in the design of protein function
Lipsh‐Sokolik, R., & Fleishman, S. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314999121 · Cited by 64
Comparative evaluation of methods for the prediction of protein–ligand binding sites
Utgés, J. S., & Barton, G. J. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00923-z · Cited by 61
Akbar, S., Raza, A., Awan, H. H., Zou, Q., Alghamdi, W., & Saeed, A. (2025) · ACS Omega · doi:10.1021/acsomega.4c11449 · Cited by 55
Lu, A. X., Lu, A. X., Pritišanac, I., Zarin, T., Forman-Kay, J. D., & Moses, A. M. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010238 · Cited by 48
Harnessing the power of artificial intelligence to advance cell therapy
Capponi, S., & Daniels, K. G. (2023) · Immunological Reviews · doi:10.1111/imr.13236 · Cited by 54
Neural network extrapolation to distant regions of the protein fitness landscape
Freschlin, C. R., Fahlberg, S. A., Heinzelman, P., & Romero, P. A. (2024) · Nature Communications · doi:10.1038/s41467-024-50712-3 · Cited by 48
Zeng, W., Dou, Y., Pan, L., Xu, L., & Peng, S. (2024) · Nature Communications · doi:10.1038/s41467-024-52293-7 · Cited by 49
Yang, D., Liu, Y., Huang, C., Li, M., Zhao, X., Wang, Y., Yang, K., Wang, Y., Zhai, P., & Zhang, L. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110370 · Cited by 62
pLM4Alg: Protein Language Model-Based Predictors for Allergenic Proteins and Peptides
Du, Z., Xu, Y., Liu, C., & Li, Y. (2023) · Journal of Agricultural and Food Chemistry · doi:10.1021/acs.jafc.3c07143 · Cited by 56
Ullah, M., Akbar, S., Raza, A., Khan, K. A., & Zou, Q. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf026 · Cited by 53
Are protein language models the new universal key?
Weissenow, K., & Rost, B. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.102997 · Cited by 52
Singh, J., Litfin, T., Paliwal, K., Singh, J., Hanumanthappa, A. K., & Zhou, Y. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab316 · Cited by 48
Deep self-supervised learning for biosynthetic gene cluster detection and product classification
Rios-Martinez, C., Bhattacharya, N., Amini, A. P., Crawford, L. A., & Yang, K. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011162 · Cited by 56
Kurgan, L., Hu, G., Wang, K., Ghadermarzi, S., Zhao, B., Malhis, N., Erdős, G., Gsponer, J., Uversky, V. N., & Dosztányi, Z. (2023) · Nature Protocols · doi:10.1038/s41596-023-00876-x · Cited by 50
Artificial intelligence tools for the identification of antibiotic resistance genes
Olatunji, I., Bardají, D. K. R., Miranda, R. R., Savka, M. A., & Hudson, A. O. (2024) · Frontiers in Microbiology · doi:10.3389/fmicb.2024.1437602 · Cited by 56
Liu, Y., Wang, S., Dong, J., Chen, L.-H., Wang, X., Wang, L., Li, F., Wang, C., Zhang, J., Wang, Y., et al. (2024) · Nature Methods · doi:10.1038/s41592-024-02437-w · Cited by 61
ProSST: Protein Language Modeling with Quantized Structure and Disentangled Attention
Hong, L., Li, M., Ma, X., Ouyang, W., Tan, P., Tan, Y., Yu, H., Zhong, B., Zhou, B., & Zhou, Z. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1126 · Cited by 44
Conditional language models enable the efficient design of proficient enzymesPreprint
Munsamy, G., Illanes-Vicioso, R., Funcillo, S., Nakou, I. T., Lindner, S., Ayres, G., Sheehan, L. S., Moss, S., Eckhard, U., Lorenz, P., et al. (2024) · bioRxiv · doi:10.1101/2024.05.03.592223 · Cited by 55
A combinatorially complete epistatic fitness landscape in an enzyme active site
Johnston, K. E., Almhjell, P. J., Watkins-Dulaney, E. J., Liu, G., Porter, N. J., Yang, J., & Arnold, F. H. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2400439121 · Cited by 44
Machine learning applications for therapeutic tasks with genomics data
Huang, K., Xiao, C., Glass, L. M., Critchlow, C. W., Gibson, G. C., & Sun, J. (2021) · Patterns · doi:10.1016/j.patter.2021.100328 · Cited by 52
Systematic prediction of degrons and E3 ubiquitin ligase binding via deep learning
Hou, C., Li, Y., Wang, M., Wu, H., & Li, T. (2022) · BMC Biology · doi:10.1186/s12915-022-01364-6 · Cited by 45
Harmalkar, A., Rao, R., Richard Xie, Y., Honer, J., Deisting, W., Anlahr, J., Hoenig, A., Czwikla, J., Sienz-Widmann, E., Rau, D., et al. (2023) · mAbs · doi:10.1080/19420862.2022.2163584 · Cited by 46
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.