Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 601–700 of 1,102
Approaching Optimal pH Enzyme Prediction with Large Language Models
Zaretckii, M., Buslaev, P., Kozlovskii, I., Morozov, A., & Popov, P. A. (2024) · ACS Synthetic Biology · doi:10.1021/acssynbio.4c00465 · Cited by 34
Learning a deep language model for microbiomes: The power of large scale unlabeled microbiome data
Pope, Q., Varma, R., Tataru, C. A., David, M. M., & Fern, X. Z. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011353 · Cited by 43
Matsunaga, R., & Tsumoto, K. (2025) · Journal of Biomedical Science · doi:10.1186/s12929-025-01141-x · Cited by 35
miRe2e: a full end-to-end deep model based on transformers for prediction of pre-miRNAs
Raad, J., Bugnon, L. A., Milone, D. H., & Stegmayer, G. S. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab823 · Cited by 34
Protein Function Analysis through Machine Learning
Avery, C., Patterson, J., Grear, T. J., Frater, T. D., & Jacobs, D. J. (2022) · Biomolecules · doi:10.3390/biom12091246 · Cited by 31
GOProFormer: A Multi-Modal Transformer Method for Gene Ontology Protein Function Prediction
Kabir, A., & Shehu, A. (2022) · Biomolecules · doi:10.3390/biom12111709 · Cited by 29
Survey of Protein Sequence Embedding Models
Tran, C. T. T., Khadkikar, S., & Porollo, A. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24043775 · Cited by 39
Transformers and Large Language Models for Chemistry and Drug Discovery
Bran, A. M., & Schwaller, P. (2024) · Springer eBooks · doi:10.1007/978-981-97-4828-0_8 · Cited by 55
Single Layers of Attention Suffice to Predict Protein Contacts
Bhattacharya, N., Thomas, N., Rao, R., Dauparas, J., Koo, P. K., Baker, D., Song, Y. S., & Ovchinnikov, S. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.12.21.423882 · Cited by 29
EvoBind: in silico directed evolution of peptide binders with AlphaFold
Bryant, P., & Elofsson, A. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.07.23.501214 · Cited by 29
Decoding molecular mechanisms for loss-of-function variants in the human proteome
Cagiada, M., Jonsson, N., & Lindorff-Larsen, K. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.21.595203 · Cited by 26
Seq-InSite: sequence supersedes structure for protein interaction site prediction
Hosseini, S., Golding, G. B., & Ilie, L. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btad738 · Cited by 26
Finding functional motifs in protein sequences with deep learning and natural language models
Savojardo, C., Martelli, P. L., & Casadio, R. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2023.102641 · Cited by 31
Leary, A. Y., Scott, D., Gupta, N. T., Waite, J. C., Skokos, D., Atwal, G. S., & Hawkins, P. G. (2024) · Nature Communications · doi:10.1038/s41467-024-48198-0 · Cited by 35
Masked particle modeling on sets: towards self-supervised high energy physics foundation models
Golling, T., Heinrich, L., Kagan, M., Klein, S. B., Leigh, M., Osadchy, M., & Raine, J. A. (2024) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ad64a8 · Cited by 111
Leveraging mRNA technology for antigen based immuno-oncology therapies
Floudas, C. S., Sarkizova, S., Ceccarelli, M., & Zheng, W. (2025) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2024-010569 · Cited by 35
Yeung, W., Zhou, Z., Mathew, L., Gravel, N., Taujale, R., O’Boyle, B. M., Salcedo, M., Venkat, A., Lanzilotta, W. N., Li, S., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbac619 · Cited by 37
TEMPROT: protein function annotation using transformers embeddings and homology search
Oliveira, G. B., Pedrini, H., & Dias, Z. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05375-0 · Cited by 37
Ultra-fast protein structure prediction to capture effects of sequence variation in mutation moviesPreprint
Weißenow, K., Heinzinger, M., Steinegger, M., & Rost, B. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.11.14.516473 · Cited by 31
Luo, S., Su, Y., Wu, Z.-F., Su, C., Peng, J., & Ma, J.-Z. (2024) · bioRxiv · doi:10.1101/2023.02.28.530137 · Cited by 55
Self-citationShanehsazzadeh, A., Alverio, J., Kasun, G. W., Levine, S., Calman, I., Khan, J. A., Chung, C., Diaz, N., Luton, B. K., Tarter, Y., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.12.08.570889 · Cited by 40
Self-citationDesigning DNA With Tunable Regulatory Activity Using Discrete DiffusionPreprint
Sarkar, A., Durán, A., Yu, Y., Lin, D.-W., Kang, Y., Somia, N., Mantilla, P., Zhou, J., Nagai, M., Tang, Z., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.23.595630 · Cited by 52
Protein Language Models: Is Scaling Necessary?
Fournier, Q., Vernon, R. M., van der Sloot, A., Schulz, B., Chandar, S., & Langmead, C. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.09.23.614603 · Cited by 26
Lab-in-the-loop therapeutic antibody design with deep learningPreprint
Frey, N. C., Hötzel, I., Stanton, S., Kelly, R. L., Alberstein, R. G., Makowski, E. K., Martinkus, K., Berenberg, D., Bevers, J. J., Bryson, T., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.19.639050 · Cited by 58
Efficient generation of protein pockets with PocketGen
Zhang, Z., Shen, W. X., Liu, Q., & Zitnik, M. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00920-9 · Cited by 32
Protein embeddings improve phage-host interaction prediction
Gonzales, M. E. M., Ureta, J. C., & Shrestha, A. M. S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289030 · Cited by 31
Toward Unified AI Drug Discovery with Multimodal Knowledge
Luo, Y., Liu, X. Y., Yang, K., Huang, K., Hong, M., Zhang, J., Wu, Y., & Nie, Z. (2024) · Health Data Science · doi:10.34133/hds.0113 · Cited by 43
Li, B., & Ming, D. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05820-8 · Cited by 27
T-cell receptor binding prediction: A machine learning revolution
Weber, A., Pélissier, A., & Martínez, M. R. (2024) · ImmunoInformatics · doi:10.1016/j.immuno.2024.100040 · Cited by 36
Development of a deep learning model for cancer diagnosis by inspecting cell-free DNA end-motifs
Shen, H., Yang, M., Liu, J., Chen, K., & Li, X. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00635-5 · Cited by 39
Self-citationAcrNET: predicting anti-CRISPR with deep learning
Li, Y., Wei, Y., Xu, S. Y., Tan, Q., Zong, L., Wang, J., Wang, Y., Chen, J., Hong, L., & Li, Y. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad259 · Cited by 28
ECOD domain classification of 48 whole proteomes from AlphaFold Structure Database using DPAM2
Schaeffer, R. D., Zhang, J., Medvedev, K. E., Kinch, L. N., Cong, Q., & Grishin, N. V. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011586 · Cited by 22
Enhancing missense variant pathogenicity prediction with protein language models using VariPred
Lin, W., Wells, J., Wang, Z., Orengo, C., & Martin, A. C. R. (2024) · Scientific Reports · doi:10.1038/s41598-024-51489-7 · Cited by 34
Xie, L., & Xie, L. (2023) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010974 · Cited by 26
Yan, Z., Ge, F., Liu, Y., Zhang, Y., Li, F., Song, J., & Yu, D. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c02019 · Cited by 25
Benchmarking uncertainty quantification for protein engineering
Greenman, K. P., Amini, A. P., & Yang, K. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012639 · Cited by 28
Contrastive learning of T cell receptor representations
Nagano, Y., Pyo, A. G. T., Milighetti, M., Henderson, J., Shawe-Taylor, J. S., Chain, B., & Tiffeau-Mayer, A. (2025) · Cell Systems · doi:10.1016/j.cels.2024.12.006 · Cited by 42
Token-Mol 1.0: tokenized drug design with large language models
Wang, J., Qin, R., Wang, M., Fang, M., Zhang, Y., Zhu, Y., Su, Q., Gou, Q., Shen, C., Zhang, O., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-59628-y · Cited by 58
From traditional to AI-driven: The evolution of intelligent enzyme engineering for biocatalysis
Lv, F., Zhang, J., You, S., & Qi, W. (2025) · Biotechnology Advances · doi:10.1016/j.biotechadv.2025.108788 · Cited by 36
Ahmad, B., Ouahada, K., & Hamam, H. (2026) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.12.033 · Cited by 38
Learning the language of protein-protein interactions
(2026) · doi:10.1038/s41467-025-67971-3 · Cited by 26
Clustering FunFams using sequence embeddings improves EC purity
Littmann, M., Bordin, N., Heinzinger, M., Schütze, K., Dallago, C., Orengo, C., & Rost, B. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab371 · Cited by 26
A switchable Cas12a enabling CRISPR-based direct histone deacetylase activity detection
Kang, W., Liu, L., Yu, P., Zhang, T., Lei, C., & Nie, Z. (2022) · Biosensors and Bioelectronics · doi:10.1016/j.bios.2022.114468 · Cited by 26
DrugormerDTI: Drug Graphormer for drug–target interaction prediction
Hu, J., Yu, W., Pang, C., Jin, J., Pham, N. T., Manavalan, B., & Wei, L. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106946 · Cited by 30
Deorphanizing Peptides Using Structure Prediction
Teufel, F., Refsgaard, J. C., Kasimova, M. A., Deibler, K., Madsen, C. T., Stahlhut, C., Grønborg, M., Winther, O., & Madsen, D. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00378 · Cited by 26
MultiGML: Multimodal graph machine learning for prediction of adverse drug events
Krix, S., DeLong, L. N., Madan, S., Domingo‐Fernándéz, D., Ahmad, A., Gul, S., Zaliani, A., & Fröhlich, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19441 · Cited by 33
tFold-Ab: Fast and Accurate Antibody Structure Prediction without Sequence Homologs
Wu, J., Wu, F., Jiang, B., Liu, W., & Zhao, P. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.11.10.515918 · Cited by 26
Learning the language of protein-protein interactions
Ullanat, V., Jing, B., Sledzieski, S., & Berger, B. (2025) · doi:10.1101/2025.03.09.642188 · Cited by 26
Deeper evaluation of a single-cell foundation model
Boiarsky, R., Singh, N. M., Buendia, A., Amini, A. P., Getz, G., & Sontag, D. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00949-w · Cited by 25
Xu, S., & Onoda, A. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01202 · Cited by 15
ACPScanner: Prediction of Anticancer Peptides by Integrated Machine Learning Methodologies
Zhong, G., & Deng, L. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01860 · Cited by 23
Using protein language models for protein interaction hot spot prediction with limited data
Sargsyan, K., & Lim, C. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05737-2 · Cited by 24
Heinzinger, M., & Rost, B. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.102986 · Cited by 26
Dong, R., Liu, R., Liu, Z., Liu, Y., Zhao, G., Li, H., Hou, S., Ma, X., Kang, H., Liu, J., et al. (2025) · eLife · doi:10.7554/elife.97330.3 · Cited by 25
Self-citationComputationally designed proteins mimic antibody immune evasion in viral evolution
Youssef, N., Gurev, S. F., Ghantous, F., Brock, K. P., Jaimes, J. A., Thadani, N. N., Dauphin, A., Sherman, A. C., Yurkovetskiy, L., Soto, D., et al. (2025) · Immunity · doi:10.1016/j.immuni.2025.04.015 · Cited by 36
Shrivastava, A. D., & Kell, D. B. (2021) · Molecules · doi:10.3390/molecules26072065 · Cited by 25
Liu, Y., Liu, Y., Wang, G.-A., Cheng, Y., Bi, S., & Zhu, X. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.834153 · Cited by 24
Peng, X., Wang, X., Guo, Y., Ge, Z., Li, F., Gao, X., & Song, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac215 · Cited by 26
Si, Y., & Yan, C. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad039 · Cited by 25
Gim, M., Choe, J., Baek, S., Park, J., Lee, C., Ju, M., Lee, S., & Kang, J. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad207 · Cited by 34
Steering and controlling evolution — from bioengineering to fighting pathogens
Lässig, M., Mustonen, V., & Nourmohammad, A. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00623-8 · Cited by 32
Jiang, Z., Shen, Y.-Y., & Liu, R. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011428 · Cited by 27
Language models can identify enzymatic binding sites in protein sequences
Teukam, Y. G. N., Dassi, L. K., Manica, M., Probst, D., Schwaller, P., & Laino, T. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.04.012 · Cited by 36
DR-BERT: A protein language model to annotate disordered regions
Nambiar, A., Forsyth, J. M., Liu, S., & Maslov, S. (2024) · Structure · doi:10.1016/j.str.2024.04.010 · Cited by 30
Decoding biology with massively parallel reporter assays and machine learning
Fleur, A. L., Shi, Y., & Seelig, G. (2024) · Genes & Development · doi:10.1101/gad.351800.124 · Cited by 31
Protein Language Model Fitness Is a Matter of PreferencePreprint
Gordon, C., Lu, A. X., & Abbeel, P. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.03.616542 · Cited by 37
PerturbNet predicts single-cell responses to unseen chemical and genetic perturbations
Yu, H., & Welch, J. D. (2022) · doi:10.1101/2022.07.20.500854 · Cited by 57
Improved prediction of MHC-peptide binding using protein language models
Hashemi, N., Hao, B., Ignatov, M., Paschalidis, I. C., Vakili, P., Vajda, S., & Kozakov, D. (2023) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2023.1207380 · Cited by 25
Han, B., Ren, C., Wang, W., Li, J., & Gong, X. (2023) · Genes · doi:10.3390/genes14020432 · Cited by 28
Genetic control over biogenic crystal morphogenesis in zebrafish
Deis, R., Lerer-Goldshtein, T., Baiko, O., Eyal, Z., Brenman-Begin, D., Goldsmith, M., Kaufmann, S., Heinig, U., Dong, Y., Lushchekina, S., et al. (2025) · Nature Chemical Biology · doi:10.1038/s41589-024-01722-1 · Cited by 24
Yang, Y., Jerger, A., Feng, S., Wang, Z., Brasfield, C., Cheung, M. S., Zucker, J., & Guan, Q. (2024) · Communications Biology · doi:10.1038/s42003-024-07359-z · Cited by 24
UAPT: an underwater acoustic target recognition method based on pre-trained Transformer
Tang, J., Ma, E., Qu, Y., Gao, W., Zhang, Y., & Gan, L. (2025) · Multimedia Systems · doi:10.1007/s00530-024-01614-3 · Cited by 30
Yue, P., Wu, J., Sun, Y., Zhang, Y., Wang, Q., & Shao, S. (2025) · Nature Communications · doi:10.1038/s41467-025-56526-1 · Cited by 28
Artificial intelligence in bioinformatics: a survey
Jiang, J., Li, Y., Cao, S., Shan, Y., Liu, Y., Fei, T., Yu, Y., Feng, Y., Li, Y., Li, Y., et al. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf576 · Cited by 33
Ligand Binding Prediction Using Protein Structure Graphs and Residual Graph Attention Networks
Pandey, M. K., Radaeva, M., Mslati, H., Garland, O., Fernández, M., Ester, M., & Cherkasov, A. R. (2022) · Molecules · doi:10.3390/molecules27165114 · Cited by 28
Alkuhlani, A., Gad, W. K., Roushdy, M. I., Voskoglou, M. G., & Salem, A.-B. M. (2022) · Axioms · doi:10.3390/axioms11090469 · Cited by 25
CLADE 2.0: Evolution-Driven Cluster Learning-Assisted Directed Evolution
Qiu, Y., & Wei, G. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01046 · Cited by 27
García‐Jacas, C. R., García‐González, L. A., Martínez-Rios, F., Tapia-Contreras, I. P., & Brizuela, C. A. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac428 · Cited by 26
Machine learning methods for predicting protein structure from single sequences
Kandathil, S. M., Lau, A. M., & Jones, D. T. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2023.102627 · Cited by 30
Lobo, F., González, M. S., Boto, A., & de la Lastra, J. M. P. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms241210270 · Cited by 27
Ghosh, S., & Mitra, P. (2023) · Comput. Methods Programs Biomed. · doi:10.1016/j.cmpb.2023.107955 · Cited by 23
KinomeMETA: meta-learning enhanced kinome-wide polypharmacology profiling
Ren, Q., Qu, N., Sun, J., Zhou, J., Liu, J., Ni, L., Tong, X., Zhang, Z., Kong, X., Wen, Y., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad461 · Cited by 26
Self-citationBest practices for machine learning in antibody discovery and development
Wossnig, L., Furtmann, N., Buchanan, A. G., Kumar, S., & Greiff, V. (2024) · Drug Discovery Today · doi:10.1016/j.drudis.2024.104025 · Cited by 31
Rationally seeded computational protein design of ɑ-helical barrels
Albanese, K. I., Petrenas, R., Pirro, F., Naudin, E. A., Borucu, U., Dawson, W., Scott, D. A., Leggett, G. J., Weiner, O. D., Oliver, T. A. A., et al. (2024) · Nature Chemical Biology · doi:10.1038/s41589-024-01642-0 · Cited by 33
Combining evolutionary and assay-labelled data for protein fitness predictionPreprint
Hsu, C., Nisonoff, H. M., Fannjiang, C., & Listgarten, J. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.28.437402 · Cited by 24
DIPS-Plus: The enhanced database of interacting protein structures for interface prediction
Morehead, A., Chen, C., Sedova, A., & Cheng, J. (2023) · Scientific Data · doi:10.1038/s41597-023-02409-3 · Cited by 28
Carter, J. T., Feddema, J., Kothe, D. B., Neely, R., Pruet, J., Stevens, R. L., Balaprakash, P., Beckman, P., Foster, I. T., Iskra, K. A., et al. (2023) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/1986455 · Cited by 41
Genome-wide prediction of disease variants with a deep protein language model
Brandes, N., Goldman, G., Wang, C. H., Ye, C. J., & Ntranos, V. (2022) · doi:10.1101/2022.08.25.505311 · Cited by 24
GTalign: spatial index-driven protein structure alignment, superposition, and search
(2024) · doi:10.1038/s41467-024-51669-z · Cited by 23
SPOT: A machine learning model that predicts specific substrates for transport proteins
Kroll, A., Niebuhr, N., Butler, G., & Lercher, M. J. (2024) · PLoS Biology · doi:10.1371/journal.pbio.3002807 · Cited by 30
SSEmb: A joint embedding of protein sequence and structure enables robust variant effect predictions
Blaabjerg, L. M., Jonsson, N., Boomsma, W., Stein, A., & Lindorff‐Larsen, K. (2024) · Nature Communications · doi:10.1038/s41467-024-53982-z · Cited by 39
Zhang, C., Wang, Q., Li, Y., Teng, A., Hu, G., Wuyun, Q., & Zheng, W. H. (2024) · Biomolecules · doi:10.3390/biom14121531 · Cited by 38
Gonzales, M. E. M., Ureta, J. C., & Shrestha, A. M. S. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btaf016 · Cited by 23
Rannon, E., Shaashua, S., & Burstein, D. E. (2025) · Microbiome · doi:10.1186/s40168-025-02055-4 · Cited by 29
Comprehensive benchmarking of large language models for RNA secondary structure prediction
Zablocki, L. I., Bugnon, L. A., Gerard, M. F., Persia, L. E. D., Stegmayer, G. S., & Milone, D. H. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf137 · Cited by 38
In search of adequate models of pedagogy
Nussbaum, M., Bekerman, Z., Archambault, L., Rainieri, M., Yılmaz, R., Zou, D., van Braak, J., & Tsai, C. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105345 · Cited by 30
Machine learning to predict de novo protein–protein interactions
Gaínza, P., Bunker, R. D., Townson, S. A., & Castle, J. C. (2025) · Trends in biotechnology · doi:10.1016/j.tibtech.2025.04.013 · Cited by 27
Language model-guided anticipation and discovery of mammalian metabolites
Qiang, H., Wang, F., Lu, W., Xing, X., Kim, H., Mérette, S. A. M., Ayres, L. B., Oler, E., AbuSalim, J. E., Roichman, A., et al. (2026) · Nature · doi:10.1038/s41586-025-09969-x · Cited by 53
RNA Backbone Torsion and Pseudotorsion Angle Prediction Using Dilated Convolutional Neural Networks
Singh, J., Paliwal, K. K., Singh, J., & Zhou, Y. (2021) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.1c00153 · Cited by 26
Learning the local landscape of protein structures with convolutional neural networks
(2021) · doi:10.1007/s10867-021-09593-6 · Cited by 23
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.