Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 901–1,000 of 1,102
How artificial intelligence is reengineering protein engineering
Listgarten, J., & Jiang, H. (2026) · Science · doi:10.1126/science.aec8444 · Cited by 31
TLCrys: Transfer Learning Based Method for Protein Crystallization Prediction
Jin, C., Shi, Z., Kang, C., Lin, K. Y., & Zhang, H. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23020972 · Cited by 21
Han, W. J., Wu, Y., Zeng, L., & Zhao, S. (2022) · Science China Life Sciences · doi:10.1007/s11427-021-2081-6 · Cited by 20
Protein sequence profile prediction using ProtAlbert transformer
Behjati, A., Zare‐Mirakabad, F., Arab, S. S., & Nowzari-Dalini, A. (2022) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2022.107717 · Cited by 25
(2022) · doi:10.1002/pro.4541 · Cited by 16
Mardikoraem, M., & Woldring, D. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15051337 · Cited by 20
Wu, J., Qing, H., Ouyang, J., Zhou, J., Gao, Z., Mason, C. E., Liu, Z., & Shi, T. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad311 · Cited by 19
Group Activity Recognition in Basketball Tracking Data – Neural Embeddings in Team Sports (NETS)
Hauri, S., & Vučetić, S. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230373 · Cited by 23
Zeng, W., Lv, D., Liu, X., Chen, G., Liu, W., & Peng, S. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385509 · Cited by 17
Using explainable machine learning to uncover the kinase–substrate interaction landscape
Zhou, Z., Yeung, W., Soleymani, S., Gravel, N., Salcedo, M., Li, S., & Kannan, N. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae033 · Cited by 21
Language model and its interpretability in biomedicine: A scoping review
Lyu, D., Wang, X., Chen, Y., & Wang, F. (2024) · iScience · doi:10.1016/j.isci.2024.109334 · Cited by 26
Accurate prediction of CDR-H3 loop structures of antibodies with deep learning
(2024) · doi:10.7554/elife.91512.4 · Cited by 16
Boshar, S., Trop, E., de Almeida, B. P., Copoiu, L., & Pierrot, T. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae529 · Cited by 16
Model Breadcrumbs: Scaling Multi-task Model Merging with Sparse Masks
Davari, M., & Belilovsky, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73226-3_16 · Cited by 165
Protein engineering in the deep learning era
Zhou, B., Tan, Y., Hu, Y., Zheng, L., Zhong, B., & Hong, L. (2024) · mLife · doi:10.1002/mlf2.12157 · Cited by 25
Annotating protein functions via fusing multiple biological modalities
Ma, W., Bi, X., Jiang, H., Wei, Z., & Zhang, S. (2024) · Communications Biology · doi:10.1038/s42003-024-07411-y · Cited by 18
Accelerating protein engineering with fitness landscape modeling and reinforcement learning
Sun, H., He, L., Deng, P., Liu, G., Zhao, Z.-Y., Jiang, Y.-L., Cao, C., Ju, F.-S., Wu, L.-J., Liu, H.-G., et al. (2025) · bioRxiv · doi:10.1038/s42256-025-01103-w · Cited by 20
Application of Sequence Embedding in Protein Sequence-Based Predictions
Ibtehaz, N., & Kihara, D. (2022) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811258589_0002 · Cited by 25
Not All Memories are Created Equal: Learning to Forget by ExpiringPreprint
Sukhbaatar, S., Ju, D., Poff, S., Roller, S., Szlam, A. D., Weston, J., & Fan, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.06548 · Cited by 21
Chen, T., Dumas, M., Watson, R., Vincoff, S., Peng, C., Zhao, L., Hong, L., Pertsemlidis, S., Shaepers-Cheu, M., Wang, T. Z., et al. (2023) · PubMed Central · doi:10.48550/arxiv.2310.03842 · Cited by 43
Fast and accurate modeling and design of antibody-antigen complex using tFoldPreprint
Wu, F., Zhao, Y. S., Wu, J., Jiang, B., He, B., Huang, L.-K., Qin, C., Yang, F., Huang, N., Xiao, Y., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.02.05.578892 · Cited by 18
ProteinCLIP: enhancing protein language models with natural languagePreprint
Wu, K., Chang, H. Y., & Zou, J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.14.594226 · Cited by 25
Ertelt, M., Moretti, R., Meiler, J., & Schoeder, C. T. (2025) · Science Advances · doi:10.1126/sciadv.adr7338 · Cited by 15
AlphaEvolve: A coding agent for scientific and algorithmic discovery
Novikov, A., Vũ, N., Eisenberger, M., Dupont, E., Huang, P.-S., Wagner, A. Z., Shirobokov, S., Kozlovskii, B., Ruiz, F. J. R., Mehrabian, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.13131 · Cited by 15
Estevam, G. O., Linossi, E., Rao, J., Macdonald, C. B., Ravikumar, A., Chrispens, K. M., Capra, J. A., Coyote-Maestas, W., Pimentel, H., Collisson, E. A., et al. (2025) · eLife · doi:10.7554/elife.101882 · Cited by 17
Xie, J., Dong, R., Zhu, J., Lin, H., Wang, S., & Lai, L. (2025) · Advanced Science · doi:10.1002/advs.202410981 · Cited by 20
Phytopathogen Effector Biology in the Burgeoning AI Era
Jones, D., & Raffaele, S. (2025) · Annual Review of Phytopathology · doi:10.1146/annurev-phyto-121823-081033 · Cited by 19
Computational methods and data resources for predicting tumor neoantigens
Zhao, X., Wei, L., & Zhang, X. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf302 · Cited by 16
Artificial intelligence-based methods for protein structure prediction: a survey
Zhan, Z., Hong, J., Li, J.-Y., Wang, C., He, L., Xu, Z., & Zhang, J. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11325-4 · Cited by 21
Mehdiabadi, M., Conte, A. D., Nugnes, M. V., Aspromonte, M. C., Tosatto, S. C. E., & Piovesan, D. (2025) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.70045 · Cited by 31
T cell receptor specificity landscape revealed through de novo peptide design
Visani, G. M., Pun, M. N., Minervina, A. A., Bradley, P. H., Thomas, P. G., & Nourmohammad, A. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2504783122 · Cited by 23
A comprehensive survey of genome language models in bioinformatics
Shu, L., Tang, J., Guan, X., & Zhang, D. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf724 · Cited by 25
Evotuning protocols for Transformer-based variant effect prediction on multi-domain proteins
Yamaguchi, H., & Saitō, Y. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab234 · Cited by 15
Advances in antibody discovery from human BCR repertoires
Xu, Z., Ismanto, H. S., Zhou, H., Saputri, D. S., Sugihara, F., & Standley, D. M. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.1044975 · Cited by 19
Pham, T. V., Nguyen, V. V., Vu, D., Henneman, A. A., Richardson, R. A., Piersma, S. R., & Jiménez, C. R. (2023) · PROTEOMICS · doi:10.1002/pmic.202200041 · Cited by 15
Protein Function Prediction With Functional and Topological Knowledge of Gene Ontology
Zhao, Y., Yang, Z., Hong, Y., Yang, Y., Wang, L., Zhang, Y., Lin, H., & Wang, J. (2023) · IEEE Transactions on NanoBioscience · doi:10.1109/tnb.2023.3278033 · Cited by 15
Protein engineering of pores for separation, sensing, and sequencing
Samineni, L., Acharya, B., Behera, H., Oh, H., Kumar, M., & Chowdhury, R. (2023) · Cell Systems · doi:10.1016/j.cels.2023.07.004 · Cited by 17
A novel hybrid CNN and BiGRU-Attention based deep learning model for protein function prediction
Sharma, L., Deepak, A., Ranjan, A., & Krishnasamy, G. (2023) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2022-0057 · Cited by 11
Chung, C., Liou, J.-T., Wu, L.-C., Horng, J., & Lee, T.-Y. (2023) · iScience · doi:10.1016/j.isci.2023.108250 · Cited by 15
Barghout, R. A., Xu, Z.-Q. J., Betala, S., & Mahadevan, R. (2023) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2023.103007 · Cited by 17
Computational Enzyme Redesign Enhances Tolerance to Denaturants for Peptide C-Terminal Amidation
Zhu, T., Sun, J., Pang, H., & Wu, B. (2024) · JACS Au · doi:10.1021/jacsau.3c00792 · Cited by 15
Wang, W., Shuai, Y., Yang, Q., Zhang, F., Zeng, M., & Li, M. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae050 · Cited by 22
Enhancing cryo-EM structure prediction with DeepTracer and AlphaFold2 integration
Chen, J., Zia, A., Luo, A., Meng, H., Wang, F., Hou, J., Cao, R., & Si, D. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae118 · Cited by 21
Protein function prediction through multi-view multi-label latent tensor reconstruction
Armah-Sekum, R. E., Szedmák, S., & Rousu, J. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05789-4 · Cited by 18
Kalemati, M., Emani, M. Z., & Koohi, S. (2024) · BMC Genomics · doi:10.1186/s12864-024-10326-x · Cited by 18
Sudnawa, K. K., Li, W., Calamia, S., Kanner, C. H., Bain, J., Abdelhakim, A. H., Geltzeiler, A., Mebane, C. M., Provenzano, F. A., Sands, T. T., et al. (2024) · Genetics in Medicine · doi:10.1016/j.gim.2024.101169 · Cited by 15
Machine learning for predicting protein properties: A comprehensive review
Wang, Y., Zhang, Y., Zhan, X., He, Y., Yang, Y., Cheng, L., & Alghazzawi, D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128103 · Cited by 18
Boadu, F., & Cheng, J. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae120 · Cited by 20
Decoding Missense Variants by Incorporating Phase Separation via Machine Learning
Feng, M., Wei, X., Zheng, X., Liu, L., Lin, L., Xia, M., He, G., Shi, Y., & Lu, Q. (2024) · Nature Communications · doi:10.1038/s41467-024-52580-3 · Cited by 15
Asim, M. N., Ibrahim, M. A., Asif, T., & Dengel, A. R. (2025) · Heliyon · doi:10.1016/j.heliyon.2024.e41488 · Cited by 21
Ünlü, A., Ulusoy, E., Yiğit, M. G., Darcan, M., & Doğan, T. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103017 · Cited by 17
Aggregating residue-level protein language model embeddings with optimal transport
Naderializadeh, N., & Singh, R. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbaf060 · Cited by 21
Frisby, T. S., & LANGMEAD, C. J. (2021) · Algorithms for Molecular Biology · doi:10.1186/s13015-021-00195-4 · Cited by 24
Wang, Q., Wang, B., Xu, Z., Wu, J., Zhao, P., Li, Z., Wang, S., Huang, J., & Cui, S. R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16141 · Cited by 22
Monti, R., Rautenstrauch, P., Ghanbari, M., James, A. R., Kirchler, M., Ohler, U., Konigorski, S., & Lippert, C. (2022) · Nature Communications · doi:10.1038/s41467-022-32864-2 · Cited by 16
A primer on model-guided exploration of fitness landscapes for biological sequence designPreprint
Sinai, S., & Kelsic, E. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10614 · Cited by 35
Generating novel protein sequences using Gibbs sampling of masked language models
Johnson, S. R., Monaco, S., Massie, K., & Syed, Z. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.01.26.428322 · Cited by 15
Improving Generalizability of Protein Sequence Models with Data AugmentationsPreprint
Shen, H., Price, L. C., Bahadori, M. T., & Seeger, F. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.18.431877 · Cited by 19
Learning the Language of Antibody HypervariabilityPreprint
Singh, R., Im, C., Qiu, Y., Mackness, B. C., Gupta, A., Sorenson, T., Sledzieski, S., Erlach, L., Wendt, M., Fomekong‐Nanfack, Y., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.04.26.538476 · Cited by 16
Exploring the Protein Sequence Space with Global Generative Models
Romero‐Romero, S., Lindner, S., & Ferruz, N. (2023) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041471 · Cited by 21
(2023) · doi:10.1101/2023.03.20.533508 · Cited by 15
Malbranke, C., Rostain, W., Depardieu, F., Cocco, S., Monasson, R., & Bikard, D. (2023) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011621 · Cited by 15
Bielska, W., Jaszczyszyn, I., Dudzic, P., Janusz, B., Chomicz, D., Wrobel, S., Greiff, V., Feehan, R., Adolf‐Bryfogle, J., & Krawczyk, K. (2025) · Frontiers in Immunology · doi:10.3389/fimmu.2025.1571371 · Cited by 22
Tan, Y., Wang, R.-L., Wu, B.-H., Hong, L., & Zhou, B.-X. (2025) · Bioinform. · doi:10.1093/bioinformatics/btaf189 · Cited by 30
Multi-dimensional metagenomics
Peng, H., Ruiz-Moreno, A. J., & Fu, J. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00346-x · Cited by 21
AF2BIND: predicting small-molecule binding sites using the pair representation of AlphaFold2
Gazizov, A., Lian, A., Goverde, C. A., Mou, J., Ovchinnikov, S., & Polizzi, N. F. (2026) · Nature Methods · doi:10.1038/s41592-026-03011-2 · Cited by 38
Evolutionary Context-Integrated Deep Sequence Modeling for Protein Engineering
Luo, Y., Vo, L. T., Ding, H., Su, Y., Liu, Y., Qian, W. W., Zhao, H., & Peng, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45257-5_30 · Cited by 18
A deep learning algorithm for joint direct tracking and classification of manoeuvring sources
Yu, W., Yu, H., Du, J., Zhang, M., & Wang, D. (2022) · IET Radar Sonar & Navigation · doi:10.1049/rsn2.12254 · Cited by 15
Huang, D., He, H., Ouyang, J., Zhao, C., Dong, X. W., & Xie, J. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-05101-2 · Cited by 17
Shu, Y., Hai, Y., Cao, L., & Wu, J. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.01.021 · Cited by 16
miProBERT: identification of microRNA promoters based on the pre-trained model BERT
Wang, X., Gao, X., Wang, G., & Li, D. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad093 · Cited by 15
Therapies for Tau-associated neurodegenerative disorders: targeting molecules, synapses, and cells
Robbins, M. (2023) · Neural Regeneration Research · doi:10.4103/1673-5374.373670 · Cited by 16
DeepPeptide predicts cleaved peptides in proteins using conditional random fields
Teufel, F., Refsgaard, J. C., Madsen, C. T., Stahlhut, C., Grønborg, M., Winther, O., & Madsen, D. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad616 · Cited by 20
Yang, Z.-S., Li, T.-S., Huang, Y.-S., Chang, C., & Chien, C. (2024) · Scientific Reports · doi:10.1038/s41598-024-53111-2 · Cited by 14
High-Temperature Tolerance Protein Engineering through Deep Evolution
Chu, H., Tian, Z., Hu, L., Zhang, H., Chang, H., Bai, J., Liu, D., Lu, L., Cheng, J., & Jiang, H. (2024) · BioDesign Research · doi:10.34133/bdr.0031 · Cited by 14
Cui, M., Wang, M., Sun, H., Yu, L., Su, Z., Zhang, X., Zheng, Y., Xia, M., Shen, Y., & Wang, M. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.132272 · Cited by 17
Wang, Z., Meng, J., Dai, Q., Li, H., Xia, S., Yang, R., & Luan, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124168 · Cited by 16
Dias, A. H. S., Cao, Y., Skaf, M. S., & de Visser, S. P. (2024) · Physical Chemistry Chemical Physics · doi:10.1039/d4cp01282h · Cited by 14
Zhang, H., Zhou, Y., Zhang, Z., Sun, H., Pan, Z., Mou, M., Zhang, W., Ye, Q., Hou, T., Li, H., et al. (2024) · Analytical Chemistry · doi:10.1021/acs.analchem.4c01793 · Cited by 18
Kabir, A., Moldwin, A., Bromberg, Y., & Shehu, A. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae119 · Cited by 16
Lin, Y.-J., Menon, A. S., Hu, Z., & Brenner, S. E. (2024) · Human Genomics · doi:10.1186/s40246-024-00663-z · Cited by 18
Yan, H., Wang, S., Liu, H., Mamitsuka, H., & Zhu, S. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae401 · Cited by 23
Liu, Y., Xia, X., Gong, Y., Song, B., & Zeng, X. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102983 · Cited by 15
Chuang, C.-C., Liu, Y., & Ou, Y. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.136252 · Cited by 19
Large language models transform biological research: from architecture to utilization
Wang, T., & Luo, Z. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4466-3 · Cited by 14
SensorFormer: Efficient Many-to-Many Sensor Calibration With Learnable Input Subsampling
Cheng, Y., Saukh, O., & Thiele, L. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3177948 · Cited by 16
Tubiana, J., Schneidman‐Duhovny, D., & Wolfson, H. J. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167758 · Cited by 16
Advancing variant effect prediction using protein language models
Livesey, B., & Marsh, J. A. (2023) · Nature Genetics · doi:10.1038/s41588-023-01470-3 · Cited by 21
Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence ModelsPreprint
Sturmfels, P., Vig, J., Madani, A., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00195 · Cited by 26
Filipavicius, M., Manica, M., Cadow, J., & Martínez, M. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03084 · Cited by 27
Now What Sequence? Pre-trained Ensembles for Bayesian Optimization of Protein SequencesPreprint
Yang, Z., Milas, K. A., & White, A. D. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.08.05.502972 · Cited by 17
Structure-enabled enzyme function prediction unveils elusive terpenoid biosynthesis in archaeaPreprint
Samusevich, R., Akmeşe, S. M., Hebra, T., Bushuiev, R., Engst, M., Kulhánek, J., Bushuiev, A., Smith, J. D., Čalounová, T., Smrčková, H., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.01.29.577750 · Cited by 20
Robust Prediction of Enzyme Variant Kinetics with RealKcatPreprint
Sajeevan, K. A., Osinuga, A., Arunraj, B., Ferdous, S., Shahreen, N., Noor, M., Koneru, S., Santa-Correa, L. M., Salehi, R., Chowdhury, N. B., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.10.637555 · Cited by 14
Fine-Tuning Protein Language Models Unlocks the Potential of Underrepresented Viral Proteomes
Sawhney, R., Ferrell, B., Dejean, T., Schreiber, Z., Harrigan, W., Polson, S. W., Wommack, K. E., & Belcaid, M. (2025) · doi:10.1101/2025.04.17.649224 · Cited by 14
Kulikova, A. V., Diaz, D. J., Chen, T., Cole, T. J., Ellington, A. D., & Wilke, C. O. (2023) · Scientific Reports · doi:10.1038/s41598-023-40247-w · Cited by 15
Xu, S., & Onoda, A. (2025) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c01288 · Cited by 13
Machine-learning-assisted universal protein activation in living mice
Wang, X., Liu, Y., Wang, Z., Zeng, X., Ngai, W. S. C., Wang, J., Zhang, H., Xie, X., Zhu, R., Fan, X., et al. (2025) · Cell · doi:10.1016/j.cell.2025.05.006 · Cited by 16
AIR-Nets: An Attention-Based Framework for Locally Conditioned Implicit Representations
Giebenhain, S., & Goldlüecke, B. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00113 · Cited by 16
Running ahead of evolution—AI-based simulation for predicting future high-risk SARS-CoV-2 variants
Chen, J., Nie, Z., Wang, Y., Wang, K., Xu, F., Hu, Z., Zheng, B., Wang, Z., Song, G., Zhang, J., et al. (2023) · The International Journal of High Performance Computing Applications · doi:10.1177/10943420231188077 · Cited by 12
Predicting Protein Functions Based on Heterogeneous Graph Attention Technique
Zhao, Y., Yang, Z., Wang, L., Zhang, Y., Lin, H., & Wang, J. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3357834 · Cited by 15
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.