Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 120 of the 120 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 1,001–1,097 of 1,097
Cuturello, F., Celoria, M., Ansuini, A., & Cazzaniga, A. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae447 · Cited by 13
TCR clustering by contrastive learning on antigen specificity
Pertseva, M., Follonier, O. M., Scarcella, D., & Reddy, S. T. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae375 · Cited by 18
Discovery of potential antidiabetic peptides using deep learning
Yue, J., Xu, J., Li, T., Li, Y., Chen, Z., Liang, S., Liu, Z., & Wang, Y. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109013 · Cited by 14
Mayer-Homology Learning Prediction of Protein-Ligand Binding Affinities
Feng, H., Shen, L., Liu, J., & Wei, G.-W. (2024) · Journal of Computational Biophysics and Chemistry · doi:10.1142/s2737416524500613 · Cited by 16
Self-citationZhang, L., & Liu, T. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.135762 · Cited by 14
Current computational tools for protein lysine acylation site prediction
Qin, Z., Ren, H., Zhao, P., Wang, K., Liu, H., Miao, C., Du, Y., Li, J., Wu, L., & Chen, Z. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae469 · Cited by 13
Enzyme catalytic efficiency prediction: employing convolutional neural networks and XGBoost
Alazmi, M. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1446063 · Cited by 17
Fujita, S., & Terada, T. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.11.028 · Cited by 13
Xia, H., Dong, C., Chen, X., Wei, Z., Gu, L., & Zhu, X. (2024) · BMC Genomics · doi:10.1186/s12864-024-11022-6 · Cited by 13
ParaAntiProt provides paratope prediction using antibody and protein language models
Kalemati, M., Noroozi, A., Shahbakhsh, A., & Koohi, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-80940-y · Cited by 19
Luo, J., Zhao, K., Chen, J., Yang, C., Qu, F., Liu, Y., Jin, X., Yan, K., Zhang, Y., & Liu, B. (2024) · Genomics Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzae084 · Cited by 24
Learning a generalized graph transformer for protein function prediction in dissimilar sequences
Fu, Y., Gu, Z., Luo, X., Guo, Q., Lai, L., & Deng, M. (2024) · GigaScience · doi:10.1093/gigascience/giae093 · Cited by 16
Ramon, A., Ni, M., Predeina, O., Gaffey, R., Kunz, P., Onuoha, S., & Sormanni, P. (2025) · mAbs · doi:10.1080/19420862.2024.2442750 · Cited by 13
MultiKD-DTA: Enhancing Drug-Target Affinity Prediction Through Multiscale Feature Extraction
Hu, R., Ge, R., Deng, G., Fan, J., Tang, B., & Wang, C. (2025) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-025-00697-4 · Cited by 14
MULAN: multimodal protein language model for sequence and structure encoding
Frolova, D., Pak, M., Litvin, A., Sharov, I., Ivankov, D., & Oseledets, I. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbaf117 · Cited by 12
Amino acid sequence encodes protein abundance shaped by protein stability at reduced synthesis cost
(2024) · doi:10.1002/pro.5239 · Cited by 12
Accurate identification and mechanistic evaluation of pathogenic missense variants with Rhapsody-2
Banerjee, A., Bogetti, A. T., & Bahar, I. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2418100122 · Cited by 12
Painting Peptides With Antimicrobial Potency Through Deep Reinforcement Learning
Dong, R., Cao, Q., & Song, C. (2025) · Advanced Science · doi:10.1002/advs.202506332 · Cited by 11
Ho, W., Huang, H., & Huang, J. (2023) · Protein Science · doi:10.1002/pro.4739 · Cited by 11
Fan, X., Pan, H., Tian, A., Chung, W. K., & Shen, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbac584 · Cited by 11
ProteinGLUE multi-task benchmark suite for self-supervised protein modeling
Capel, H., Weiler, R., Dijkstra, M., Vleugels, R., Bloem, P., & Feenstra, K. A. (2022) · Scientific Reports · doi:10.1038/s41598-022-19608-4 · Cited by 11
RNA contact prediction by data efficient deep learning
Taubert, O., von der Lehr, F., Bazarova, A., Faber, C., Knechtges, P., Weiel, M., Debus, C., Coquelin, D., Basermann, A., Streit, A., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-05244-9 · Cited by 11
Singh, A., Miranda Bedate, A., von Richthofen, H. J., Vijver, S. V., van der Vlist, M., Kuhn, R., Yermanos, A., Kuball, J. J., Kesmir, C., Pascoal Ramos, M. I., et al. (2024) · eLife · doi:10.7554/elife.92870.3 · Cited by 11
(2022) · doi:10.1101/2022.08.30.505840 · Cited by 11
Fine-tuning protein language models unlocks the potential of underrepresented viral proteomes
Sawhney, R., Ferrell, B. D., Dejean, T., Schreiber, Z., Harrigan, W., Polson, S. W., Wommack, K. E., & Belcaid, M. (2025) · PeerJ · doi:10.7717/peerj.19919 · Cited by 14
Semantic search using protein large language models detects class II microcins in bacterial genomes
Kulikova, A. V., Parker, J. K., Davies, B. W., & Wilke, C. O. (2024) · mSystems · doi:10.1128/msystems.01044-24 · Cited by 10
Discovery of highly active kynureninases for cancer immunotherapy through protein language model
Eom, H., Park, S., Cho, K. S., Lee, J., Kim, H., Kim, S., Yang, J., Han, Y.-H., Lee, J., Seok, C., et al. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkae1245 · Cited by 10
Protein Language Model Identifies Disordered, Conserved Motifs Implicated in Phase Separation
Zhang, Y., Zheng, J., & Zhang, B. (2024) · doi:10.1101/2024.12.12.628175 · Cited by 10
Automated Scientific Discovery: From Equation Discovery to Autonomous Discovery Systems
Kramer, S., Cerrato, M., Brugger, J., Džeroski, S., & King, R. (2023) · arXiv · doi:10.48550/arxiv.2305.02251 · Cited by 10
Semantical and geometrical protein encoding toward enhanced bioactivity and thermostability
Tan, Y., Zhou, B., Zheng, L., Fan, G., & Hong, L. (2025) · eLife · doi:10.7554/elife.98033.4 · Cited by 20
Protein language models uncover carbohydrate-active enzyme function in metagenomics
Thurimella, K., Mohamed, A. M. T., Li, C., Vatanen, T., Graham, D. B., Owens, R. M., La Rosa, S. L., Plichta, D. R., Bacallado, S., & Xavier, R. J. (2025) · BMC Bioinformatics · doi:10.1186/s12859-025-06286-y · Cited by 9
Vidal-Henriquez, E., Holder, T., Lee, N. F., Pompe, C., & Teese, M. G. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.10.026 · Cited by 9
Pre-training with a rational approach for antibody sequence representation
Gao, X., Cao, C., He, C., & Lai, L. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1468599 · Cited by 9
Kandathil, S. M., Greener, J. G., Lau, A. M., & Jones, D. T. (2020) · doi:10.1101/2020.11.27.401232 · Cited by 8
OPUS-Rota5: A highly accurate protein side-chain modeling method with 3D-Unet and RotaFormer
Xu, G., Luo, Z., Yan, Y., Wang, Q., & Ma, J. (2024) · Structure · doi:10.1016/j.str.2024.03.015 · Cited by 8
Self-citationInvestigating the determinants of performance in machine learning for protein fitness prediction
Sandhu, M., Mater, A. C., Matthews, D. S., Spence, M. A., Lenskiy, A. A., & Jackson, C. (2020) · doi:10.1101/2020.09.30.319780 · Cited by 8
A paired sequence language model for protein-protein interaction modeling
Liu, J., Chen, H., & Zhang, Y. (2026) · Nature Communications · doi:10.1038/s41467-026-70457-5 · Cited by 8
Self-citationScoring alignments by embedding vector similarity
(2024) · doi:10.1093/bib/bbae178 · Cited by 8
AlphaFold model quality self‐assessment improvement via deep graph learning
Verburgt, J., Zhang, Z., & Kihara, D. (2025) · Protein Science · doi:10.1002/pro.70274 · Cited by 8
Zhang, Z., He, X., Long, D., Luo, G., & Chen, S. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae240 · Cited by 8
Efficient inference, training, and fine-tuning of protein language models
Çelik, M. H., & Xie, X. (2025) · iScience · doi:10.1016/j.isci.2025.113495 · Cited by 7
AI-enabled alkaline-resistant evolution of protein to apply in mass production
Kang, L., Wu, B., Zhou, B., Tan, P., Kang, Y. (., Yan, Y., Zong, Y., Li, S., Liu, Z., & Hong, L. (2025) · eLife · doi:10.7554/elife.102788 · Cited by 7
(2025) · doi:10.1016/j.jhazmat.2025.139625 · Cited by 7
BAGEL: Protein engineering via exploration of an energy landscape
(2025) · doi:10.1371/journal.pcbi.1013774 · Cited by 7
Protein Language Models Uncover Carbohydrate-Active Enzyme Function in Metagenomics
Thurimella, K., Mohamed, A. M. T., Graham, D. B., Owens, R. M., La Rosa, S. L., Plichta, D. R., Bacallado, S., & Xavier, R. J. (2023) · doi:10.1101/2023.10.23.563620 · Cited by 7
(2025) · doi:10.1101/2025.07.07.663595 · Cited by 7
Simonsen, S., Larsen, F. B., Søgaard, C. K., Jonsson, N., Lindorff‐Larsen, K., Bruheim, P., Otterlei, M., Hartmann‐Petersen, R., & Kragelund, B. B. (2025) · The FEBS Journal · doi:10.1111/febs.70133 · Cited by 7
Exploring the Knowledge of an Outstanding Protein to Protein Interaction Transformer
Yang, S., Cheng, P., Liu, Y., Feng, D., & Wang, S. (2024) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2024.3381825 · Cited by 6
Investigating the determinants of performance in machine learning for protein fitness prediction
(2025) · doi:10.1002/pro.70235 · Cited by 6
Protein language model pseudolikelihoods capture features of in vivo B cell selection and evolution
van Ginneken, D., Samant, A., Daga-Krumins, K., Glänzer, W., Agrafiotis, A., Kladis, E., Reddy, S. T., & Yermanos, A. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf418 · Cited by 6
Estevam, G. O., Linossi, E., Rao, J., Macdonald, C. B., Ravikumar, A., Chrispens, K. M., Capra, J. A., Coyote-Maestas, W., Pimentel, H., Collisson, E. A., et al. (2025) · eLife · doi:10.7554/elife.101882.3 · Cited by 6
HaloClass: Salt-Tolerant Protein Classification with Protein Language Models
Narang, K., Nath, A., Hemstrom, W., & Chu, S. K. S. (2024) · The Protein Journal · doi:10.1007/s10930-024-10236-7 · Cited by 6
E2EDA: Protein Domain Assembly Based on End-to-End Deep Learning
(2023) · doi:10.1021/acs.jcim.3c01387 · Cited by 6
Quantifying the Hardness of Bioactivity Prediction Tasks for Transfer Learning
Fooladi, H., Hirte, S., & Kirchmair, J. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c00160 · Cited by 6
ProteinMCP : An agentic AI framework for autonomous protein engineering
Xu, X., Feng, C., Zha, C., He, W., He, M., Xiao, B., & Gao, X. (2026) · Protein Science · doi:10.1002/pro.70547 · Cited by 6
GOBoost: leveraging long-tail gene ontology terms for accurate protein function prediction
Zhang, L., Wang, Y., Chen, X., Hou, J., Si, D., Ding, R., Jiang, B., Ledenko, H., & Cao, R. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf267 · Cited by 5
Spicer, R., Raychawdhary, N., Danwada, S., Udomprasert, P., Seals, C., & Bhattacharya, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-23319-x · Cited by 5
Direct coupling analysis and the attention mechanism
(2025) · doi:10.1186/s12859-025-06062-y · Cited by 5
Transformers meets neoantigen detection: a systematic literature review
Machaca, V., Goyzueta, V., Cruz, M. G., Sejje, E., Pilco, L. M., López, J., & Túpac, Y. (2024) · Journal of Integrative Bioinformatics · doi:10.1515/jib-2023-0043 · Cited by 5
Protein embeddings and local alignments
Malec, J., Golding, G. B., & Ilie, L. (2026) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.12.002 · Cited by 5
Protein Embeddings and Local Alignments
Malec, J., Golding, G. B., & Ilie, L. (2025) · doi:10.1101/2025.07.23.666343 · Cited by 5
Finucane, H. K., Parsa, S., Guez, J., Kanai, M., Satterstrom, F. K., Nkambule, L. L., Daly, M. J., Seed, C., & Karczewski, K. J. (2024) · doi:10.1101/2024.09.17.613327 · Cited by 4
McWhite, C. D., & Singh, M. (2022) · doi:10.1101/2022.10.21.513099 · Cited by 4
Learning chemical sensitivity reveals mechanisms of cellular response
Connell, W., Garcia, K., Goodarzi, H., & Keiser, M. J. (2024) · Communications Biology · doi:10.1038/s42003-024-06865-4 · Cited by 4
TMVisDB: Annotation and 3D-visualization of Transmembrane Proteins
Olenyi, T., Marquet, C., Grekova, A., Houri, L., Heinzinger, M., Dallago, C., & Rost, B. (2025) · Journal of Molecular Biology · doi:10.1016/j.jmb.2025.168997 · Cited by 4
Protein language model identifies disordered, conserved motifs implicated in phase separation
Zhang, Y., Zheng, J., & Zhang, B. (2025) · eLife · doi:10.7554/elife.105309 · Cited by 4
Thomas, D. P. G., Garcia Fernandez, C. M., Haydarlou, R., & Feenstra, K. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-88445-y · Cited by 4
(2025) · doi:10.1016/j.cels.2025.101400 · Cited by 4
Singh, A., Miranda Bedate, A., von Richthofen, H. J., Vijver, S. V., van der Vlist, M., Kuhn, R., Yermanos, A., Kuball, J. J., Kesmir, C., Pascoal Ramos, M. I., et al. (2024) · eLife · doi:10.7554/elife.92870 · Cited by 4
EGGNet, a Generalizable Geometric Deep Learning Framework for Protein Complex Pose Scoring
Wang, Z., Brand, R., Adolf-Bryfogle, J., Grewal, J., Qi, Y., Combs, S. A., Golovach, N., Alford, R., Rangwala, H., & Clark, P. M. (2024) · ACS Omega · doi:10.1021/acsomega.3c04889 · Cited by 3
RAG-ESM: Improving Pretrained Protein Language Models via Sequence Retrieval
Sgarbossa, D., & Bitbol, A.-F. (2025) · PRX Life · doi:10.1103/db1b-hy16 · Cited by 3
How well do contextual protein encodings learn structure, function, and evolutionary context?
(2025) · doi:10.1016/j.cels.2025.101201 · Cited by 3
Kawano, T., Shiraishi, T., Kuzuyama, T., & Umemura, M. (2026) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1013181 · Cited by 3
Flattening the curve—How to get better results with small deep‐mutational‐scanning datasets
Wirnsberger, G., Pritišanac, I., Oberdorfer, G., & Gruber, K. (2024) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26686 · Cited by 3
From Signal to Symphony: Exploring 2D Sequence Representations for Protein Function Prediction
Wang, Y., Cai, M., Dong, Y., Ma, Y., & Wei, K. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01768 · Cited by 3
Ab-initio amino acid sequence design from protein text description with ProtDAT
Guo, X.-Y., Li, Y.-F., Liu, Y., Pan, X., & Shen, H.-B. (2025) · Nature Communications · doi:10.1038/s41467-025-65562-w · Cited by 3
Enhancing kcat prediction through residue-aware attention mechanism and pre-trained representations
Cai, Y., Ge, F., Zhang, C., Chen, H., Qi, X., Liao, X., & Wang, L. (2026) · Communications Biology · doi:10.1038/s42003-026-09551-9 · Cited by 2
Genome-scale annotation of protein binding sites via language model and geometric deep learning
(2023) · doi:10.1101/2023.11.02.565344 · Cited by 2
Semantical and geometrical protein encoding toward enhanced bioactivity and thermostability
Tan, Y., Zhou, B., Zheng, L., Fan, G., & Hong, L. (2025) · eLife · doi:10.7554/elife.98033 · Cited by 2
Connecting the dots: deep learning-based automated model building methods in cryo-EM
Bansia, H., & des Georges, A. (2026) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2025.1613399 · Cited by 2
(2026) · doi:10.1073/pnas.2524201123 · Cited by 2
Branda, F., Ahmed, M. M., Ciccozzi, M., Guzzi, P. H., & Scarpa, F. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag245 · Cited by 2
Liang, J., Luo, Y., Su, B., & Peng, Z. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag126 · Cited by 1
(2026) · doi:10.1371/journal.pcbi.1013996 · Cited by 1
H3-OPT: Accurate prediction of CDR-H3 loop structures of antibodies with deep learning
Chen, H., Fan, X., Zhu, S., Pei, Y., Zhang, X., Zhang, X., Liu, L., Qian, F., & Tian, B. (2023) · doi:10.1101/2023.08.19.553933 · Cited by 1
Lyu, Q., Wei, H., Chen, S., Peng, Z., & Yang, J. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c02385 · Cited by 1
Zhang, Y., Zhong, S., Xu, S., Wang, Z., Xin, C., Ni, F., Yan, F., Lu, X., Sun, S., Wang, H., et al. (2025) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2025.147637 · Cited by 1
Ankh‐Score Produces Better Sequence Alignments Than AlphaFold3
Malec, J., Rusen, K., Golding, G. B., & Ilie, L. (2026) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.70143 · Cited by 1
Mishra, S., Pandey, N., Rawat, A., Srivastava, D., Ray, A., & Kumar, V. (2023) · IEEE Access · doi:10.1109/access.2023.3337260
Dai, Y., Tan, X., Wang, H., Ma, G., Xiong, Y., & Qiu, X. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.109163
Genome-scale annotation of protein binding sites via language model and geometric deep learning
Yuan, Q., Tian, C., & Yang, Y. (2024) · doi:10.7554/elife.93695.2
Uncovering differential tolerance to deletions versus substitutions with a protein language model
Goldman, G., Chati, P., & Ntranos, V. (2025) · Cell Systems · doi:10.1016/j.cels.2025.101373
AI-enabled alkaline-resistant evolution of protein to apply in mass production
Kang, L., Wu, B., Zhou, B., Tan, P., Kang, Y. (., Yan, Y., Zong, Y., Li, S., Liu, Z., & Hong, L. (2025) · eLife · doi:10.7554/elife.102788.3
Decoding Missense Variants by Incorporating Phase Separation via Machine Learning
Feng, M., Wei, X., Zheng, X., Liu, L., Lin, L., Xia, M., He, G., Shi, Y., & Lu, Q. (2024) · doi:10.1101/2024.04.01.587546
(2020) · International Conference on Machine Learning · Cited by 1,012
Rethinking Attention with Performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.