Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 701–800 of 1,102
Structure-inducing pre-training
McDermott, M. B. A., Yap, B., Szolovits, P., & Žitnik, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00647-z · Cited by 33
Burnim, A. A., Spence, M. A., Xu, D., Jackson, C. J., & Ando, N. (2022) · eLife · doi:10.7554/elife.79790 · Cited by 23
EPT: A data-driven transformer model for earthquake prediction
Zhang, B., Hu, Z., Wu, P., Huang, H., & Xiang, J. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106176 · Cited by 24
Jaszczyszyn, I., Bielska, W., Gawłowski, T., Dudzic, P., Satława, T., Kończak, J., Wilman, W., Janusz, B., Wróbel, S., Chomicz, D., et al. (2023) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2023.1214424 · Cited by 26
Deep learning in preclinical antibody drug discovery and development
Zhou, Y., Huang, Z., Li, W., Wei, J., Jiang, Q., Yang, W., & Huang, J. H. (2023) · Methods · doi:10.1016/j.ymeth.2023.07.003 · Cited by 26
Israeli, S., & Louzoun, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae084 · Cited by 26
Can Protein Structure Prediction Methods Capture Alternative Conformations of Membrane Transporters?
Xie, T., & Huang, J. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01936 · Cited by 23
Prediction of virus-host associations using protein language models and multiple instance learning
Liu, D., Young, F., Lamb, K. D., Robertson, D. L., & Yuan, K. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012597 · Cited by 32
Jamasb, A. R., Viñas, R., Ma, E. J., Harris, C., Huang, K., Hall, D., Lió, P., & Blundell, T. L. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.15.204701 · Cited by 39
Training Compute-Optimal Protein Language ModelsPreprint
Cheng, X., Chen, B., Li, P., Gong, J., Tang, J., & Song, L. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.06.597716 · Cited by 32
Adams, E., Bai, L., Lee, M., Yu, Y.-Y., & Alquraishi, M. K. S. (2025) · bioRxiv · doi:10.1101/2025.02.06.636901 · Cited by 65
Interpreting Potts and Transformer Protein Models Through the Lens of Simplified Attention
Bhattacharya, N., Thomas, N., Rao, R., Dauparas, J., Koo, P. K., Baker, D., Song, Y. S., & Ovchinnikov, S. (2021) · Biocomputing 2022 · doi:10.1142/9789811250477_0004 · Cited by 20
Protein language model-embedded geometric graphs power inter-protein contact prediction
Si, Y., & Yan, C. (2024) · eLife · doi:10.7554/elife.92184.3
Evaluating large language models for annotating proteins
Vitale, R., Bugnon, L. A., Fenoy, E., Milone, D. H., & Stegmayer, G. S. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae177 · Cited by 26
Li, F., Lang, S., Tian, Y., Hong, B., Rolf, B., Noortwyck, R., Schulz, R., & Reggelin, T. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-024-02470-8 · Cited by 40
pLM4CPPs: Protein Language Model-Based Predictor for Cell Penetrating Peptides
Kumar, N., Du, Z.-J., & Li, Y.-H. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01338 · Cited by 25
Vongsouthi, V., Georgelin, R., Matthews, D. S., Saunders, J. W., Lee, B. M., Ton, J., Damry, A. M., Frkic, R. L., Spence, M. A., & Jackson, C. J. (2025) · Science Advances · doi:10.1126/sciadv.ads8318 · Cited by 27
Protein functional site annotation using local structure embeddings
Derry, A., Tartici, A., & Altman, R. B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2513219122 · Cited by 26
Efficient Exploration of Sequence Space by Sequence-Guided Protein Engineering and Design
Clifton, B. E., Kozome, D., & Laurino, P. (2022) · Biochemistry · doi:10.1021/acs.biochem.1c00757 · Cited by 25
PRECOGx: e x ploring GPCR signaling mechanisms with deep protein representations
Matic, M., Singh, G., Carli, F., Rosa, N. D. O., Miglionico, P., Magni, L., Gutkind, J. S., Russell, R. B., Inoue, A., & Raimondi, F. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac426 · Cited by 24
NeuroPpred-SVM: A New Model for Predicting Neuropeptides Based on Embeddings of BERT
Liu, Y., Wang, S., Li, X., Liu, Y., & Zhu, X. (2023) · Journal of Proteome Research · doi:10.1021/acs.jproteome.2c00363 · Cited by 23
Large-scale predicting protein functions through heterogeneous feature fusion
Zheng, R., Huang, Z., & Deng, L. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad243 · Cited by 25
A self-supervised deep learning method for data-efficient training in genomics
Gündüz, H. A., Binder, M., To, X.-Y., Mreches, R., Bischl, B., McHardy, A. C., Münch, P. C., & Rezaei, M. (2023) · Communications Biology · doi:10.1038/s42003-023-05310-2 · Cited by 32
Malik, M. S., & Ou, Y. (2023) · Methods · doi:10.1016/j.ymeth.2023.10.008 · Cited by 23
In silico protein function prediction: the rise of machine learning-based approaches
Chen, J., Gu, Z., Lai, L., & Pei, J. (2023) · Medical Review · doi:10.1515/mr-2023-0038 · Cited by 28
Mohseni Behbahani, Y., Laine, E., & Carbone, A. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad231 · Cited by 23
Deep embeddings to comprehend and visualize microbiome protein space
Odrzywolek, K., Karwowska, Z., Majta, J., Byrski, A., Milanowska-Zabel, K., & Kosciolek, T. (2022) · Scientific Reports · doi:10.1038/s41598-022-14055-7 · Cited by 22
Zhou, K., Lei, C., Zheng, J., Huang, Y., & Zhang, Z. (2023) · Plant Methods · doi:10.1186/s13007-023-01119-6 · Cited by 28
Raisinghani, N., Alshahrani, M., Gupta, G., Tian, H., Xiao, S., Tao, P., & Verkhivker, G. M. (2024) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c00222 · Cited by 17
Foundation models for bioinformatics
Chen, Z., Wei, L., & Gao, G. (2024) · Quantitative Biology · doi:10.1002/qub2.69 · Cited by 31
TAWFN: a deep learning framework for protein function prediction
Meng, L., & Wang, X. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae571 · Cited by 23
Leveraging ancestral sequence reconstruction for protein representation learning
Matthews, D. S., Spence, M. A., Mater, A. C., Nichols, J., Pulsford, S. B., Sandhu, M., Kaczmarski, J. A., Miton, C. M., Tokuriki, N., & Jackson, C. J. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00935-2 · Cited by 26
Engineering CAR‐T Therapeutics for Enhanced Solid Tumor Targeting
Zhu, D., Kim, W., Lee, H., Bao, X., & Kim, P. (2025) · Advanced Materials · doi:10.1002/adma.202414882 · Cited by 32
Recent advances in deep learning and language models for studying the microbiome
Yan, B., Nam, Y., Li, L., Deek, R. A., Li, H., & Ma, S. (2025) · Frontiers in Genetics · doi:10.3389/fgene.2024.1494474 · Cited by 33
Generative Artificial Intelligence in Medicine
Shanmugam, D., Agrawal, M., Movva, R., Chen, I. Y., Ghassemi, M., Jacobs, M. L., & Pierson, E. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-095332 · Cited by 41
Discovery of CRISPR-Cas12a clades using a large language model
Feng, Y., Shi, J., Li, Z., Li, Y., Yang, J., Huang, S., Zheng, J., Han, W., Qiao, Y., Zhang, J., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-63160-4 · Cited by 21
Large language models for bioinformatics
Ruan, W., Lyu, Y., Zhang, J., Cai, J., Shu, P., Yang, G., Lu, Y., Gao, S., Wang, Y., Wang, P., et al. (2025) · Quantitative Biology · doi:10.1002/qub2.70014 · Cited by 48
Generalizable and scalable protein stability prediction with rewired protein generative models
Li, Z., & Luo, Y. (2025) · Nature Communications · doi:10.1038/s41467-025-67609-4 · Cited by 37
Protein and genomic language models uncover the unexplored diversity of bacterial immunity
Mordret, E., Hervé, A., Tesson, F., Vaysset, H., Clabby, T., Loubat, A., Shomar, H., Planel, R., Lavenir, R., Cury, J., et al. (2026) · Science · doi:10.1126/science.adv8275 · Cited by 41
PDBench: evaluating computational methods for protein-sequence design
Castorina, L. V., Petrenas, R., Subr, K., & Wood, C. W. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad027 · Cited by 21
Exploiting pretrained biochemical language models for targeted drug design
Uludoğan, G., Özkırımlı, E., Ülgen, K. Ö., Karalı, N., & Özgür, A. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac482 · Cited by 30
Yu, H., & Luo, X. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac476 · Cited by 21
Cai, T., Xie, L., Zhang, S., Chen, M., He, D., Badkul, A., Liu, Y., Krishna, N. H., Dorogan, M., Harding, W. W., et al. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010851 · Cited by 23
Predicting mechanical properties of silk from its amino acid sequences via machine learning
Kim, Y., Yoon, T., Park, W. B., & Na, S. (2023) · Journal of the mechanical behavior of biomedical materials/Journal of mechanical behavior of biomedical materials · doi:10.1016/j.jmbbm.2023.105739 · Cited by 21
Pélissier, A., Luo, S., Stratigopoulou, M., Guikema, J. E. J., & Martínez, M. R. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1123968 · Cited by 23
Machine Learning for Protein Engineering
Johnston, K. E., Fannjiang, C., Wittmann, B. J., Hie, B., Yang, K., & Wu, Z. (2023) · Challenges and advances in computational chemistry and physics · doi:10.1007/978-3-031-37196-7_9 · Cited by 30
Ge, F., Arif, M., Yan, Z., Alahmadi, H., Worachartcheewan, A., Yu, D., & Shoombuatong, W. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00950 · Cited by 21
Insights into the inner workings of transformer models for protein function prediction
Wenzel, M. A., Grüner, E., & Strodthoff, N. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae031 · Cited by 26
Giants among Cnidaria: Large Nuclear Genomes and Rearranged Mitochondrial Genomes in Siphonophores
Ahuja, N., Cao, X., Schultz, D. T., Picciani, N., Lord, A., Shao, S., Jia, K., Burdick, D. R., Haddock, S. H. D., Li, Y., et al. (2024) · Genome Biology and Evolution · doi:10.1093/gbe/evae048 · Cited by 21
Protein Design by Directed Evolution Guided by Large Language Models
Tran, T. P., & Hy, T. S. (2024) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2024.3439690 · Cited by 39
Simple, Efficient, and Scalable Structure-Aware Adapter Boosts Protein Language Models
Tan, Y., Li, M., Zhou, B., Zhong, B., Zheng, L., Tan, P., Zhou, Z., Yu, H., Fan, G., & Hong, L. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c00689 · Cited by 34
Resimulation-based self-supervised learning for pretraining physics foundation models
Harris, P., Krupa, J., Kagan, M., Maier, B., & Woodward, N. (2025) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.111.032010 · Cited by 65
PredictProtein – Predicting Protein Structure and Function for 29 YearsPreprint
Bernhofer, M., Dallago, C., Karl, T., Satagopam, V., Heinzinger, M., Littmann, M., Olenyi, T., Qiu, J., Schütze, K., Yachdav, G., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.23.432527 · Cited by 21
Novel CRISPR-Cas12a Clades Discovery Using Large Language Model
Huang, X., Feng, Y., Shi, J., Yang, J., Li, Z., Li, Y., Tang, J., Yang, Y., Zheng, J., Han, W., et al. (2024) · doi:10.21203/rs.3.rs-4817511/v1 · Cited by 21
Recent advances in generative biology for biotherapeutic discovery
Mock, M. L., LANGMEAD, C. J., Grandsard, P. J., Edavettal, S. C., & Russell, A. J. (2024) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2024.01.003 · Cited by 23
Zimmerman, L., Alon, N., Levin, I., Koganitsky, A., Shpigel, N., Brestel, C., & Lapidoth, G. D. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2313809121 · Cited by 25
Assessing the role of evolutionary information for enhancing protein language model embeddings
Erckert, K., & Rost, B. (2024) · Scientific Reports · doi:10.1038/s41598-024-71783-8 · Cited by 21
Cai, L., Yue, G., Chen, Y., Wang, L., Yao, X., Zou, Q., Fu, X., & Cao, D. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae654 · Cited by 24
Savojardo, C., Manfredi, M., Martelli, P. L., & Casadio, R. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btaf019 · Cited by 28
Multimeric protein interaction and complex prediction: Structure, dynamics and function
Lu, D., Yu, S., Huang, Y., & Gong, X. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.05.009 · Cited by 24
Wolf, B., Shehu, P., Brenker, L., von Bachmann, A.-L., Kroell, A.-S. H., Southern, N. T., Holderbach, S., Eigenmann, J., Aschenbrenner, S., Mathony, J., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02741-z · Cited by 28
Phage therapy against Klebsiella pneumoniae: An evolving perspective
Xing, J., Zhang, H., Zheng, L., Zhao, J., Zhang, Y., Xu, Z., Zhai, Y., Hu, G., Li, J., & Wu, H. (2025) · Biotechnology Advances · doi:10.1016/j.biotechadv.2025.108689 · Cited by 23
Abbasi, K., Razzaghi, P., Gharizadeh, A., Ghareyazi, A., Dehnad, A., Rabiee, H. R., & Mofrad, M. R. K. (2025) · Pharmacological Reviews · doi:10.1016/j.pharmr.2025.100095 · Cited by 30
Protein Language Models: Applications and Perspectives
Leclercq, M., & Droit, A. (2025) · Journal of Proteome Research · doi:10.1021/acs.jproteome.5c00506 · Cited by 43
Protein language models trained on biophysical dynamics inform mutation effects
Hou, C., Zhao, H., & Shen, Y. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2530466123 · Cited by 30
Research on Students’ Mental Health Based on Data Mining Algorithms
Luo, M. (2021) · Journal of Healthcare Engineering · doi:10.1155/2021/1382559 · Cited by 20
Cai, T., Abbu, K. A., Liu, Y., & Xie, L. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac154 · Cited by 21
Learning Strategies in Protein Directed Evolution
Cadet, X. F., Gelly, J., van Noord, A., Cadet, F., & Acevedo‐Rocha, C. G. (2022) · Methods in molecular biology · doi:10.1007/978-1-0716-2152-3_15 · Cited by 22
Hallucinating structure-conditioned antibody libraries for target-specific binders
Mahajan, S. P., Ruffolo, J. A., Frick, R., & Gray, J. J. (2022) · Frontiers in Immunology · doi:10.3389/fimmu.2022.999034 · Cited by 20
PITHIA: Protein Interaction Site Prediction Using Multiple Sequence Alignments and Attention
Hosseini, S., & Ilie, L. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms232112814 · Cited by 23
Fast and accurate protein intrinsic disorder prediction by using a pretrained language model
Song, Y., Yuan, Q., Chen, S., Chen, K., Zhou, Y., & Yang, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad173 · Cited by 21
Improved the heterodimer protein complex prediction with protein language models
Chen, B., Xie, Z., Qiu, J., Ye, Z., Xu, J., & Tang, J. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad221 · Cited by 35
Extracellular vesicles for developing targeted hearing loss therapy
Pan, X., Li, Y., Huang, P., Staecker, H., & He, M. (2024) · Journal of Controlled Release · doi:10.1016/j.jconrel.2023.12.050 · Cited by 25
Hu, Y.-M., Wang, Y.-J., Hu, X., Chao, H.-Y., Li, S.-D., Ni, Q., Zhu, Y., Hu, Y.-X., Zhao, Z., & Chen, M. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.01.015 · Cited by 22
Wang, J.-K., Quan, L.-J., Jin, Z., Wu, H., Ma, X., Wang, X., Xie, J., Pan, D., Chen, T., Wu, T.-F., et al. (2024) · IEEE journal of biomedical and health informatics · doi:10.1109/jbhi.2024.3400521 · Cited by 19
Sequential Optimal Experimental Design of Perturbation Screens Guided by Multi-modal Priors
Huang, K., Lopez, R., Hütter, J.-C., Kudo, T., Rios, A., & Regev, A. (2024) · Lecture notes in computer science · doi:10.1007/978-1-0716-3989-4_2 · Cited by 39
Tan, Y., Li, M.-C., Zhou, Z.-Y., Tan, P., Yu, H.-Q., Fan, G.-S., & Hong, L. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00884-3 · Cited by 27
The Role of Protein Engineering in Biomedical Applications of Mammalian Synthetic Biology
Bojar, D., & Fussenegger, M. (2019) · Small · doi:10.1002/smll.201903093 · Cited by 21
Transfer learning for predicting virus-host protein interactions for novel virus sequences
Lanchantin, J., Weingarten, T., Sekhon, A., Miller, C. L., & Qi, Y. (2021) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3459930.3469527 · Cited by 27
Principles and Methods in Computational Membrane Protein Design
Vorobieva, A. A. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167154 · Cited by 20
Deep metric learning for accurate protein secondary structure prediction
Yang, W. P., Liu, Y., & Xiao, C. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108356 · Cited by 28
Improving the topology prediction of α-helical transmembrane proteins with deep transfer learning
Wang, L., Zhong, H., Xue, Z., & Wang, Y. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.04.024 · Cited by 23
Function-guided protein design by deep manifold samplingPreprint
Gligorijević, V., Berenberg, D., Ra, S., Watkins, A. M., Kelow, S. P., Cho, K., & Bonneau, R. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.12.22.473759 · Cited by 26
Proximal Exploration for Model-guided Protein Sequence DesignPreprint
Ren, Z., Li, J., Ding, F., Zhou, Y., Ma, J., & Peng, J. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.12.487986 · Cited by 53
Self-citationPhyloformer: towards fast and accurate phylogeny estimation with self-attention networksPreprint
Nesterenko, L., Boussau, B., & Jacob, L. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.06.24.496975 · Cited by 20
Fast and accurate modeling of TCR-peptide-MHC complexes using tFold-TCRPreprint
Wu, F., Zhao, Y. S., Xiao, Y., Qin, C., Wang, F., Wu, Z., Huang, L.-K., Liu, X., Song, J., He, B., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.01.12.632367 · Cited by 27
A-Prot: protein structure modeling using MSA transformer
Hong, Y., Lee, J., & Ko, J. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04628-8 · Cited by 19
Huang, C., Zhang, L., Tang, T., Wang, H., Jiang, Y., Ren, H., Zhang, Y., Fang, J., Zhang, W., Jia, X., et al. (2024) · JACS Au · doi:10.1021/jacsau.4c00284 · Cited by 19
Choi, S., & Kim, D. (2024) · Scientific Reports · doi:10.1038/s41598-024-78506-z · Cited by 24
ESM ‐scan—A tool to guide amino acid substitutions
Totaro, M. G., Vide, U., Zausinger, R., Winkler, A., & Oberdorfer, G. (2024) · Protein Science · doi:10.1002/pro.5221 · Cited by 19
Artificial Intelligence Transforming Post-Translational Modification Research
Kim, D. N., Yin, T., Zhang, T., Im, A. K., Cort, J., Rozum, J. C., Pollock, D. D., Qian, W., & Feng, S. (2024) · Bioengineering · doi:10.3390/bioengineering12010026 · Cited by 25
Hu, W., & Ohue, M. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.01.022 · Cited by 26
Yu, C., Fang, X., Tian, S., & Liu, H. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00973-w · Cited by 23
Pathogen genomic surveillance and the AI revolution
Lytras, S., Lamb, K. D., Ito, J., Grove, J., Yuan, K., Sato, K., Hughes, J., & Robertson, D. L. (2025) · Journal of Virology · doi:10.1128/jvi.01601-24 · Cited by 28
LLMDTA: Improving Cold-Start Prediction in Drug-Target Affinity With Biological LLM
Tang, W., Zhao, Q., & Wang, J. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2025.3541634 · Cited by 27
FusionESP: Improved Enzyme–Substrate Pair Prediction by Fusing Protein and Chemical Knowledge
Du, Z., Fu, W., Guo, X., Caragea, D., & Li, Y. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c02357 · Cited by 24
Yan, C., Zhang, Z., Xu, J., Yu, M., Yan, S., Wei, L., Zou, Q., Zhang, Q., & Cui, F. (2025) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2025.142309 · Cited by 23
Wan, F., Torres, M. D. T., Guan, C., & de la Fuente‐Núñez, C. (2025) · Nature Protocols · doi:10.1038/s41596-025-01144-w · Cited by 24
Can protein expression be ‘solved’?
Baranowski, C., Martín, H. G., Oyarzún, D. A., Spinner, A., Desai, B. J., Petzold, C. J., Nikolados, E.-M., Kraatz, S. H. W., Gaber, A., Chalkley, R. J., et al. (2025) · Trends in biotechnology · doi:10.1016/j.tibtech.2025.04.021 · Cited by 21
Universal cell embedding provides a foundation model for cell biology
Rosen, Y., Roohani, Y., Agrawal, A., Samotorcan, L., Quake, S. R., & Leskovec, J. (2026) · Nature · doi:10.1038/s41586-026-10689-z · Cited by 285
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.