Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 401–500 of 1,102
AI-Driven Enzyme Engineering: Emerging Models and Next-Generation Biotechnological Applications
Khan, M. F., & Khan, M. T. (2025) · Molecules · doi:10.3390/molecules31010045 · Cited by 61
Early computational detection of potential high-risk SARS-CoV-2 variants
(2023) · doi:10.1016/j.compbiomed.2023.106618 · Cited by 46
Early Computational Detection of Potential High Risk SARS-CoV-2 Variants
Beguir, K., Skwark, M. J., Fu, Y., Pierrot, T., Carranza, N. L., Laterre, A., Kadri, I., Korched, A., Lowegard, A. U., Lui, B. G., et al. (2021) · doi:10.1101/2021.12.24.474095 · Cited by 46
Do deep learning models make a difference in the identification of antimicrobial peptides?
García‐Jacas, C. R., Pinacho-Castellanos, S. A., García‐González, L. A., & Brizuela, C. A. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac094 · Cited by 50
Fenoy, E., Edera, A. A., & Stegmayer, G. S. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac232 · Cited by 50
Manfredi, M., Savojardo, C., Martelli, P. L., & Casadio, R. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac678 · Cited by 55
Zhu, Y.-H., Liu, Z., Liu, Y., Ji, Z.-W., & Yu, D.-J. (2024) · Briefings Bioinform. · doi:10.1093/bib/bbae040 · Cited by 43
SEMA 2.0: web-platform for B-cell conformational epitopes prediction using artificial intelligence
Ivanisenko, N. V., Shashkova, T. I., Shevtsov, A., Sindeeva, M., Umerenkov, D., & Kardymon, O. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae386 · Cited by 43
Multi-modal deep learning enables efficient and accurate annotation of enzymatic active sites
Wang, X., Yin, X., Jiang, D., Zhao, H., Wu, Z., Zhang, O., Wang, J., Li, Y., Deng, Y., Liu, H., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-51511-6 · Cited by 63
An end-to-end framework for the prediction of protein structure and fitness from single sequence
Chen, Y., Xu, Y., Liu, D., Xing, Y., & Gong, H. (2024) · Nature Communications · doi:10.1038/s41467-024-51776-x · Cited by 45
ProLLaMA: A Protein Large Language Model for Multitask Protein Language Processing
Lv, L., Lin, Z., Li, H., Liu, Y., Cui, J., Chen, C. Y., Yuan, L., & Tian, Y. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3564914 · Cited by 104
Self-citationDefensePredictor: A machine learning model to discover prokaryotic immune systems
DeWeirdt, P. C., Mahoney, E. M., & Laub, M. T. (2026) · Science · doi:10.1126/science.adv7924 · Cited by 62
Ten quick tips for deep learning in biology
Lee, B. D., Gitter, A., Greene, C. S., Raschka, S., Maguire, F., Titus, A., Kessler, M. D., Lee, A. J., Chevrette, M. G., Stewart, P. A., et al. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009803 · Cited by 50
GraphiT: Encoding Graph Structure in TransformersPreprint
Mialon, G., Chen, D., Selosse, M., & Mairal, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05667 · Cited by 276
Artificial intelligence and machine learning in cell-free-DNA-based diagnostics
Tsui, W., Ding, S. C., Jiang, P., & Lo, Y. M. D. (2025) · Genome Research · doi:10.1101/gr.278413.123 · Cited by 57
Zero-shot antibody design in a 24-well platePreprint
Chai Discovery Team, Boitreaud, J., Dent, J., Geisz, D., McPartlon, M., Meier, J., Qiao, Z., Rogozhnikov, A., Rollins, N., Wollenhaupt, P., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.05.663018 · Cited by 99
Self-citationRapid and automated design of two-component protein nanomaterials using ProteinMPNN
de Haas, R. J., Brunette, N., Goodson, A., Dauparas, J., Yi, S. Y., Yang, E. C., Dowling, Q., Nguyen, H., Kang, A., Bera, A. K., et al. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314646121 · Cited by 42
Protein sequence‐to‐structure learning: Is this the end(‐to‐end revolution)?
Laine, E., Eismann, S., Elofsson, A., & Grudinin, S. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26235 · Cited by 44
Kim, Q., Ko, J.-H., Kim, S., Park, N., & Jhe, W. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab346 · Cited by 47
Protein Design with Deep Learning
Defresne, M., Barbe, S., & Schiex, T. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms222111741 · Cited by 44
Fast and accurate Ab Initio Protein structure prediction using deep learning potentials
Pearce, R., Li, Y., Omenn, G. S., & Zhang, Y. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010539 · Cited by 51
Prevalent bee venom genes evolved before the aculeate stinger and eusociality
Koludarov, I., Velasque, M., Senoner, T., Timm, T., Greve, C., Hamadou, A. B., Gupta, D. K., Lochnit, G., Heinzinger, M., Vilcinskas, A., et al. (2023) · BMC Biology · doi:10.1186/s12915-023-01656-5 · Cited by 64
PLM-ARG: antibiotic resistance gene identification using a pretrained protein language model
Wu, J., Ouyang, J., Qin, H., Zhou, J., Roberts, R. A., Siam, R., Wang, L., Tong, W., Liu, Z., & Shi, T. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad690 · Cited by 53
Pang, Y., & Liu, B. (2024) · BMC Biology · doi:10.1186/s12915-023-01803-y · Cited by 51
Accurate prediction of protein function using statistics-informed graph networks
Jang, Y. J., Qin, Q.-Q., Huang, S., Peter, A. T. J., Ding, X., & Kornmann, B. (2024) · Nature Communications · doi:10.1038/s41467-024-50955-0 · Cited by 60
Optimizing enzyme thermostability by combining multiple mutations using protein language model
Bian, J., Tan, P., Nie, T., Hong, L. Y., & Yang, G. (2024) · mLife · doi:10.1002/mlf2.12151 · Cited by 45
Scaling Unlocks Broader Generation and Deeper Functional Understanding of Proteins
Bhatnagar, A., Jain, S., Beazer, J., Curran, S. C., Hoffnagle, A. M., Ching, K., Martyn, M., Nayfach, S., Ruffolo, J. A., & Madani, A. (2025) · neural information processing systems · doi:10.52202/085713-1536 · Cited by 75
Drug-target interaction/affinity prediction: Deep learning models and advances review
Vefghi, A., Rahmati, Z., & Akbari, M. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110438 · Cited by 58
The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modelingPreprint
Cornman, A., West-Roberts, J. A., Camargo, A. P., Roux, S., Beracochea, M., Mirdita, M., Ovchinnikov, S., & Hwang, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.14.607850 · Cited by 73
Geffen, Y., Ofran, Y., & Unger, R. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac474 · Cited by 40
Lin, S., Shi, C., & Chen, J. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04905-6 · Cited by 49
The application of artificial intelligence to accelerate G protein‐coupled receptor drug discovery
Nguyen, A. T. N., Nguyen, D. T. N., Koh, H. Y., Toskov, J., MacLean, W., Xu, A., Zhang, D., Webb, G. I., May, L. T., & Halls, M. L. (2023) · British Journal of Pharmacology · doi:10.1111/bph.16140 · Cited by 52
Fast, sensitive detection of protein homologs using deep dense retrieval
Hong, L., Hu, Z., Sun, S., Tang, X., Wang, J., Tan, Q., Zheng, L., Wang, S., Xu, S., King, I., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-024-02353-6 · Cited by 43
AI-driven de novo enzyme design: Strategies, applications, and future prospects
Cui, X.-C., Zheng, Y., Liu, Y., Yuchi, Z., & Yuan, Y. (2025) · Biotechnology Advances · doi:10.1016/j.biotechadv.2025.108603 · Cited by 47
Modeling protein–ligand interactions for drug discovery in the era of deep learning
Wang, Y., Li, Y., Chen, J., & Lai, L. (2025) · Chemical Society Reviews · doi:10.1039/d5cs00415b · Cited by 53
Synthetic molecular evolution of antimicrobial peptides
Chen, C. H., Bepler, T., Pepper, K., Fu, D., & Lu, T. K. (2022) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2022.102718 · Cited by 48
A Deep Dive into Single-Cell RNA Sequencing Foundation Models
Boiarsky, R., Singh, N., Buendia, A., Getz, G., & Sontag, D. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.19.563100 · Cited by 43
Pre‐Training of Equivariant Graph Matching Networks with Conformation Flexibility for Drug Binding
Wu, F., Jin, S., Jiang, Y., Jin, X., Tang, B., Niu, Z., Liu, X., Zhang, Q., Zeng, X., & Li, S. Z. (2022) · Advanced Science · doi:10.1002/advs.202203796 · Cited by 66
Learning deep representations of enzyme thermal adaptation
Li, G., Buric, F., Zrimec, J., Viknander, S., Nielsen, J., Zelezniak, A., & Engqvist, M. K. M. (2022) · Protein Science · doi:10.1002/pro.4480 · Cited by 42
Chen, P., Jiao, R., Liu, J., Liu, Y., & Lu, Y. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c00876 · Cited by 45
Self-citationZhong, X., Gu, Y., Luo, Y., Zeng, X., & Liu, G. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04228-2 · Cited by 44
Wang, L., Huang, C., Wang, M., Xue, Z., & Wang, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad077 · Cited by 45
ADOPT: intrinsic protein disorder prediction through deep bidirectional transformers
Redl, I., Fisicaro, C., Dutton, O., Hoffmann, F., Henderson, L., Owens, B. M. J., Heberling, M., Paci, E., & Tamiola, K. (2023) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad041 · Cited by 42
Wang, S., Song, X., Zhang, Y., Zhang, K., Liu, Y., Ren, C., & Pang, S. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24098326 · Cited by 47
Karlsen, S. T., Rau, M. H., Sánchez, B. J., Jensen, K. L., & Zeidan, A. A. (2023) · FEMS Microbiology Reviews · doi:10.1093/femsre/fuad030 · Cited by 47
Deploying synthetic coevolution and machine learning to engineer protein-protein interactions
Yang, A., Jude, K. M., Lai, B., Minot, M., Kocyła, A., Glassman, C. R., Nishimiya, D., Kim, Y. S., Reddy, S. T., Khan, A. A., et al. (2023) · Science · doi:10.1126/science.adh1720 · Cited by 51
Single-sequence protein structure prediction by integrating protein language models
Jing, X., Wu, F., Luo, X., & Xu, J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308788121 · Cited by 57
Glycolysis-compatible urethanases for polyurethane recycling
Chen, Y., Sun, J., Shi, K., Zhu, T., Li, R., Li, R., Liu, X., Xie, X., Ding, C., Geng, W., et al. (2025) · Science · doi:10.1126/science.adw4487 · Cited by 44
Nearest neighbor search on embeddings rapidly identifies distant protein relations
Schütze, K., Heinzinger, M., Steinegger, M., & Rost, B. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.1033775 · Cited by 42
Rapid and automated design of two-component protein nanomaterials using ProteinMPNN
de Haas, R. J., Brunette, N., Goodson, A., Dauparas, J., Yi, S. Y., Yang, E. C., Dowling, Q., Nguyen, H., Kang, A., Bera, A. K., et al. (2023) · doi:10.1101/2023.08.04.551935 · Cited by 42
Applications of artificial intelligence to lipid nanoparticle delivery
Yuan, Y., Wu, Y., Cheng, J., Yang, K., Xia, Y., Wu, H., & Pan, X. (2023) · Particuology · doi:10.1016/j.partic.2023.11.014 · Cited by 47
Genome-scale annotation of protein binding sites via language model and geometric deep learning
Yuan, Q., Tian, C., & Yang, Y. (2024) · eLife · doi:10.7554/elife.93695.3 · Cited by 9
Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers
Abdine, H., Chatzianastasis, M., Bouyioukos, C., & Vazirgiannis, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.28948 · Cited by 104
Protein Sequence Classification Through Deep Learning and Encoding Strategies
Tasnim, F., Habiba, S. U., Mahmud, T., Nahar, L., Hossain, M. S., & Andersson, K. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.06.106 · Cited by 45
DepoScope: Accurate phage depolymerase annotation and domain delineation using large language models
Concha-Eloko, R., Stock, M., Bernard De Baets, C., Briers, Y., Sanjuán, R., Domingo‐Calap, P., & Boeckaerts, D. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011831 · Cited by 46
Small data, big challenges: Machine- and deep-learning strategies for data-limited drug discovery
Pallikkavaliyaveetil, N., & Chandrasekaran, S. (2025) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2025.115762 · Cited by 71
Deep neural language modeling enables functional protein generation across families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.07.18.452833 · Cited by 41
Xu, Y., Liu, D., & Gong, H. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00716-2 · Cited by 41
Therapeutic enzyme engineering using a generative neural network
Giessel, A., Dousis, A., Ravichandran, K., Smith, K., Sur, S., McFadyen, I., Zheng, W., & Licht, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-05195-x · Cited by 40
Lin, P., Yan, Y., & Huang, S.-Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbac499 · Cited by 40
Machine learning enables prediction of metabolic system evolution in bacteria
Konno, N., & Iwasaki, W. (2023) · Science Advances · doi:10.1126/sciadv.adc9130 · Cited by 40
AFTGAN: prediction of multi-type PPI based on attention free transformer and graph attention network
Kang, Y., Elofsson, A., Jiang, Y., Huang, W., Yu, M., & Li, Z. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad052 · Cited by 46
AI models for protein design are driving antibody engineering
Chungyoun, M. F., & Gray, J. J. (2023) · Current Opinion in Biomedical Engineering · doi:10.1016/j.cobme.2023.100473 · Cited by 39
Improving antibody language models with native pairing
Burbach, S. M., & Briney, B. S. (2024) · Patterns · doi:10.1016/j.patter.2024.100967 · Cited by 50
GPSFun: geometry-aware protein sequence function predictions with language models
Yuan, Q., Tian, C., Song, Y., Ou, P., Zhu, M., Zhao, H., & Yang, Y. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae381 · Cited by 51
Self-supervised learning in drug discovery
Chen, Y., Wang, Z., Wang, J., Chu, Y., Zhang, Q., Li, Z. A., & Zeng, X. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4453-4 · Cited by 42
Nielsen, H. (2025) · Methods in molecular biology · doi:10.1007/978-1-0716-4623-6_10 · Cited by 73
Cui, F., Zhang, Z., Cao, C., Zou, Q., Chen, D., & Su, X. (2022) · PROTEOMICS · doi:10.1002/pmic.202100197 · Cited by 40
Protein design using structure-based residue preferences
(2024) · doi:10.1038/s41467-024-45621-4 · Cited by 38
Nguyen, T. M., Nguyen, T., & Tran, T. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac269 · Cited by 50
Pakhrin, S. C., Pokharel, S., Aoki-Kinoshita, K. F., Beck, M. R., Dam, T. K., Caragea, D., & KC, D. B. (2023) · Glycobiology · doi:10.1093/glycob/cwad033 · Cited by 42
CapsNet-MHC predicts peptide-MHC class I binding based on capsule neural networks
Kalemati, M., Darvishi, S., & Koohi, S. (2023) · Communications Biology · doi:10.1038/s42003-023-04867-2 · Cited by 44
Structural Modeling of Nanobodies: A Benchmark of State-of-the-Art Artificial Intelligence Programs
(2023) · doi:10.3390/molecules28103991 · Cited by 39
Jiang, R., Yue, Z., Shang, L., Wang, D., & Wei, N. (2024) · Metabolic Engineering Communications · doi:10.1016/j.mec.2024.e00248 · Cited by 43
GenerRNA: A generative pre-trained language model for de novo RNA design
Zhao, Y., Oono, K., Takizawa, H., & Kotera, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0310814 · Cited by 64
Applications of Machine and Deep Learning in Adaptive Immunity
Pertseva, M., Gao, B., Neumeier, D., Yermanos, A., & Reddy, S. T. (2021) · Annual Review of Chemical and Biomolecular Engineering · doi:10.1146/annurev-chembioeng-101420-125021 · Cited by 43
Recent Advances in NMR Protein Structure Prediction with ROSETTA
Leman, J. K., & Künze, G. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24097835 · Cited by 42
CoCoNat: a novel method based on deep learning for coiled-coil prediction
Madeo, G., Savojardo, C., Manfredi, M., Martelli, P. L., & Casadio, R. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad495 · Cited by 38
Applications of artificial intelligence and machine learning in dynamic pathway engineering
Merzbacher, C., & Oyarzún, D. A. (2023) · Biochemical Society Transactions · doi:10.1042/bst20221542 · Cited by 41
Aparicio, J. A. B., Olivares‐Gil, A., Rodríguez, J. J., García‐Osorio, C., & Díez-Pastor, J.-F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102035 · Cited by 43
Predicting enzymatic function of protein sequences with attention
Buton, N., Coste, F., & Cunff, Y. L. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad620 · Cited by 49
Protein Engineering with Lightweight Graph Denoising Neural Networks
Zhou, B., Zheng, L., Wu, B., Tan, Y., Lv, O., Yi, K., Fan, G., & Hong, L. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c00036 · Cited by 38
Protein representations: Encoding biological information for machine learning in biocatalysis
Harding-Larsen, D., Funk, J., Madsen, N. G., Gharabli, H., Acevedo-Rocha, C. G., Mazurenko, S., & Welner, D. H. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108459 · Cited by 38
Predicting absolute protein folding stability using generative models
Cagiada, M., Ovchinnikov, S., & Lindorff‐Larsen, K. (2024) · Protein Science · doi:10.1002/pro.5233 · Cited by 51
Wohlwend, J., Nathan, A., Shalon, N., Crain, C. R., Tano-Menka, R., Goldberg, B., Richards, E., Gaiha, G. D., & Barzilay, R. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00971-y · Cited by 53
Engineering highly active nuclease enzymes with machine learning and high-throughput screening.
Thomas, N., Belanger, D., Xu, C., Lee, H., Hirano, K., Iwai, K., Polic, V., Nyberg, K. D., Hoff, K., Frenz, L., et al. (2025) · Cell Systems · doi:10.1016/j.cels.2025.101236 · Cited by 35
Lin, A., Ye, J., Qi, C., Zhu, L., Mou, W., Gan, W., Zeng, D., Tang, B., Xiao, M., Chu, G., et al. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf357 · Cited by 58
Machine Learning-Driven Enzyme Mining: Opportunities, Challenges, and Future Perspectives
Moorhoff, F., Zhang, Y., Qiu, S., Dong, W., Medina-Ortiz, D., Zhao, J. B., & Davari, M. D. (2025) · ACS Catalysis · doi:10.1021/acscatal.5c04814 · Cited by 50
Kandathil, S. M., Greener, J. G., Lau, A. M., & Jones, D. T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2113348119 · Cited by 38
Studies of cell-penetrating peptides by biophysical methods
Zorko, M., & Langel, Ü. (2022) · Quarterly Reviews of Biophysics · doi:10.1017/s0033583522000026 · Cited by 44
Lee, D., Yang, J., & Kim, S. (2022) · Nature Communications · doi:10.1038/s41467-022-34152-5 · Cited by 38
Nitrogenase resurrection and the evolution of a singular enzymatic mechanism
Garcia, A. K., Harris, D. F., Rivier, A. J., Carruthers, B. M., Pinochet-Barros, A., Seefeldt, L. C., & Kaçar, B. (2023) · eLife · doi:10.7554/elife.85003 · Cited by 37
Protein Design and Variant Prediction Using Autoregressive Generative Models
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., & Marks, D. S. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/757252 · Cited by 32
Predicting plant Rubisco kinetics from RbcL sequence data using machine learning
Iqbal, W. A., Lisitsa, A., & Kapralov, M. V. (2022) · Journal of Experimental Botany · doi:10.1093/jxb/erac368 · Cited by 39
AI for organic and polymer synthesis
Hong, X., Yang, Q., Liao, K., Pei, J., Chen, M., Mo, F., Lu, H., Zhang, W.-B., Zhou, H., Chen, J., et al. (2024) · Science China Chemistry · doi:10.1007/s11426-024-2072-4 · Cited by 45
Deep Learning for Antimicrobial Peptides: Computational Models and Databases
Zhou, X., Liu, G., Cao, S., & Lv, J. Z. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c00006 · Cited by 43
Qian, H., Wang, Y., Zhou, X., Gu, T., Wang, H., Lyu, H., Li, Z., Li, Z., Zhou, H., Guo, C., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-58521-y · Cited by 38
AI-guided discovery and optimization of antimicrobial peptides through species-aware language model
Bae, D., Kim, M., Seo, J., & Nam, H. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf343 · Cited by 50
PLM-interact: extending protein language models to predict protein-protein interactions
Liu, D., Young, F., Lamb, K. D., Quiros, A. C., Pancheva, A., Miller, C. J., Macdonald, C., Robertson, D. L., & Yuan, K. (2025) · Nature Communications · doi:10.1038/s41467-025-64512-w · Cited by 72
Lu, Y., Huang, W., Li, Y., Xu, Y., Wei, Q., Sha, C., & Guo, P. (2025) · npj Precision Oncology · doi:10.1038/s41698-025-01159-2 · Cited by 56
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.