Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Open accessPublished version of a preprint
The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.
Abstract
Significance: Learning biological properties from sequence data is a logical step toward generative and predictive artificial intelligence for biology. Here, we propose scaling a deep contextual language model with unsupervised learning to sequences spanning evolutionary diversity. We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect. Unsupervised representation learning enables state-of-the-art supervised prediction of mutational effect and secondary structure and improves state-of-the-art features for long-range contact prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The current state-of-the-art in sequence-based modeling is the use of AI/ML embeddings pretrained on multimillion-scale protein collections.118”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [118]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It has been shown that the resulting representations contain rich information about the structure and the function of the proteins 17 , 32 , 33 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the evolutionary scale modeling (ESM) family (Rives et al. 2021), comprising transformer protein language models such as ESM-2 and ESMFold, surpasses earlier single-sequence protein language models, demonstrating enhanced capability in generating accurate structure predictions based on protein sequences.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Rives et al. 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [32]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
Research works printed but not linked: 4
- A. Radford . Language models are unsupervised multitask learners. OpenAI Blog [Preprint] (2019). https://openai.com/blog/better-language-models (Accessed 6 August 2020).
- 10.1016/S0022-2836(05)80360-2 · doi:10.1016/s0022-2836(05)80360-2 · Paper page
- A. Radford K. Narasimhan T. Salimans I. Sutskever Improving language understanding by generative pre-training. OpenAI Blog [Preprint] (2018). https://openai.com/blog/language-unsupervised (Accessed 6 August 2020).
- T. Mikolov . Subword language modeling with neural networks. The website of T. Mikolov [Preprint] (2012). http://www.fit.vutbr.cz/∼imikolov/rnnlm/char.pdf (Accessed 14 March 2021).
Cited by
Items 801–900 of 1,102
Luo, H., Shan, W., Chen, C., Ding, P., & Luo, L. (2022) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-022-00537-9 · Cited by 31
Antibody apparent solubility prediction from sequence by transfer learning
Feng, J.-Y., Jiang, M., Shih, J., & Chai, Q. (2022) · iScience · doi:10.1016/j.isci.2022.105173 · Cited by 21
A large-scale benchmark study of tools for the classification of protein-coding and non-coding RNAs
Singh, D., & Roy, J. K. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac1092 · Cited by 22
LambdaPP : Fast and accessible protein‐specific phenotype predictions
(2022) · doi:10.1002/pro.4524 · Cited by 19
Manfredi, M., Savojardo, C., Martelli, P. L., & Casadio, R. (2023) · Journal of Molecular Biology · doi:10.1016/j.jmb.2023.167963 · Cited by 19
Elucidating the functional roles of prokaryotic proteins using big data and artificial intelligence
Ardern, Z., Chakraborty, S., Lenk, F., & Kaster, A. (2023) · FEMS Microbiology Reviews · doi:10.1093/femsre/fuad003 · Cited by 23
Inferring protein fitness landscapes from laboratory evolution experiments
D’Costa, S., Hinds, E. C., Freschlin, C. R., Song, H., & Romero, P. A. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010956 · Cited by 23
Deep learning‐assisted prediction of protein–protein interactions in Arabidopsis thaliana
Zheng, J., Yang, X., Huang, Y., Yang, S., Wuchty, S., & Zhang, Z. (2023) · The Plant Journal · doi:10.1111/tpj.16188 · Cited by 19
Plug & play directed evolution of proteins with gradient-based discrete MCMC
Emami, P., Perreault, A., Law, J. N., Biagioni, D. J., & John, P. C. S. (2022) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/accacd · Cited by 37
Surface ID: a geometry-aware system for protein molecular surface comparison
Riahi, S., Lee, J. H., Sorenson, T., Wei, S., Jäger, S., Olfati‐Saber, R., Zhou, Y., Park, A., Wendt, M., Minoux, H., et al. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad196 · Cited by 24
Features and algorithms: facilitating investigation of secreted effectors in Gram-negative bacteria
Zhao, Z., Hu, Y., Hu, Y., White, A. P., & Wang, Y. (2023) · Trends in Microbiology · doi:10.1016/j.tim.2023.05.011 · Cited by 18
A robust deep learning workflow to predict CD8 + T-cell epitopes
Lee, C. H.-J., Huh, J., Buckley, P. R., Jang, M., Pinho, M. P., Fernandes, R. A., Antanaviciute, A., Simmons, A., & Koohy, H. (2023) · Genome Medicine · doi:10.1186/s13073-023-01225-z · Cited by 21
Wee, J., Chen, J., Xia, K., & Wei, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107918 · Cited by 26
IDP-LM: Prediction of protein intrinsic disorder and disorder functions based on language models
Pang, Y., & Liu, B. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011657 · Cited by 23
Machine learning-based approaches for ubiquitination site prediction in human proteins
Pourmirzaei, M., Ramazi, S., Esmaili, F., Shojaeilangari, S., & Allahvardi, A. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05581-w · Cited by 21
HyperPCM: Robust Task-Conditioned Modeling of Drug–Target Interactions
Svensson, E., Hoedt, P.-J., Hochreiter, S., & Klambauer, G. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01417 · Cited by 25
O’Connor, E., Micklefield, J., & Cai, Y. (2024) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2024.103125 · Cited by 21
Elisée, E., Ducrot, L., Méheust, R., Bastard, K., Fossey‐Jouenne, A., Grogan, G., Pelletier, É., Petit, J., Stam, M. R., de Berardinis, V., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-49009-2 · Cited by 20
EGPDI: identifying protein–DNA binding sites based on multi-view graph embedding fusion
Zheng, M., Sun, G., Li, X., & Fan, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae330 · Cited by 23
Zhang, L., & Liu, T. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.136147 · Cited by 20
Ensembling machine learning models to boost molecular affinity prediction
Druchok, M., Yarish, D., Garkot, S., Nikolaienko, T. Y., & Gurbych, O. (2021) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2021.107529 · Cited by 21
Constructing benchmark test sets for biological sequence analysis using independent set algorithms
Petti, S., & Eddy, S. R. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009492 · Cited by 22
Modeling Protein Using Large-scale Pretrain Language ModelPreprint
Xiao, Y., Qiu, J., Li, Z., Hsieh, C., & Tang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07435 · Cited by 48
Design of Peptide-Based Protein Degraders via Contrastive Deep Learning
Palepu, K., Ponnapati, M., Bhat, S., Tysinger, E., Stan, T., Brixi, G., Koseki, S. R. T., & Chatterjee, P. (2022) · doi:10.1101/2022.05.23.493169 · Cited by 17
Levy, B., Xu, Z., Zhao, L., Kremling, K. A. G., Altman, R., Wong, P., & Tanner, C. (2022) · Research Square · doi:10.21203/rs.3.rs-1927200/v1 · Cited by 20
Protein Representation Learning via Knowledge Enhanced Primary Structure ModelingPreprint
Zhou, H.-Y., Fu, Y., Zhang, Z., Bian, C., & Yu, Y. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.26.525795 · Cited by 38
On Pre-trained Language Models for AntibodyPreprint
Wang, D., Ye, F., & Hao, Z. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.29.525793 · Cited by 29
Variant Effect Prediction in the Age of Machine Learning
Bromberg, Y., Prabakaran, R., Kabir, A., & Shehu, A. (2024) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041467 · Cited by 23
ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction PredictionPreprint
Jin, M., Xue, H., Wang, Z., Kang, B., Ye, R., Zhou, K., Du, M., & Zhang, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.04.18.590025 · Cited by 54
Integrating Computational Design and Experimental Approaches for Next-Generation Biologics
Son, A., Park, J., Kim, W., Lee, W., Yoon, Y., Ji, J., & Kim, H. (2024) · Biomolecules · doi:10.3390/biom14091073 · Cited by 29
Protein A-like Peptide Design Based on Diffusion and ESM2 Models
Zhao, L., He, Q., Song, H., Zhou, T., Luo, A., Wen, Z., Wang, T., & Lin, X. (2024) · Molecules · doi:10.3390/molecules29204965 · Cited by 20
Li, B., Luo, S., Wang, W., Xu, J., Liu, D., Shameem, M., Mattila, J., Franklin, M. C., Hawkins, P. G., & Atwal, G. S. (2025) · mAbs · doi:10.1080/19420862.2025.2474521 · Cited by 22
Wang, Y., Cheng, L., Zhang, Y., Cao, Y., & Alghazzawi, D. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf187 · Cited by 24
DeepMolecules: a web server for predicting enzyme and transporter–small molecule interactions
Kroll, A., Rousset, Y., Spitzlei, T., & Lercher, M. J. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf343 · Cited by 20
Li, Y.-F., Pan, X., & Shen, H.-B. (2025) · Patterns · doi:10.1016/j.patter.2025.101262 · Cited by 18
Tran, D. T., Pham, N. T., Nguyen, N. D. H., Wei, L., & Manavalan, B. (2025) · Journal of Pharmaceutical Analysis · doi:10.1016/j.jpha.2025.101410 · Cited by 23
Zhang, W., Meng, Q., Wang, J., & Guo, F. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac169 · Cited by 18
Villalobos-Alva, J., Ochoa-Toledo, L., Villalobos-Alva, M. J., Aliseda, A., Escamirosa, F. P., Altamirano-Bustamante, N. F., Ochoa-Fernández, F., Zamora-Solís, R., Villalobos-Alva, S., Revilla, M., et al. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.788300 · Cited by 27
Ten quick tips for sequence-based prediction of protein properties using machine learning
Hou, Q., Waury, K., Gogishvili, D., & Feenstra, K. A. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010669 · Cited by 24
Sun, J., Kulandaisamy, A., Liu, J., Hu, K., Gromiha, M. M., & Zhang, Y. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.01.036 · Cited by 24
Self-citationDeCOIL: Optimization of Degenerate Codon Libraries for Machine Learning-Assisted Protein Engineering
Yang, J., Ducharme, J., Johnston, K. E., Li, F.-Z., Yue, Y., & Arnold, F. H. (2023) · ACS Synthetic Biology · doi:10.1021/acssynbio.3c00301 · Cited by 22
Simplifying complex antibody engineering using machine learning
Makowski, E. K., Chen, H.-T., & Tessier, P. M. (2023) · Cell Systems · doi:10.1016/j.cels.2023.04.009 · Cited by 21
An Effective Plant Small Secretory Peptide Recognition Model Based on Feature Correction Strategy
Wang, R., Zhou, Z., Wu, X., Jiang, X., Zhuo, L., Liu, M., Li, H., Fu, X., & Yao, X. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00868 · Cited by 18
Fuse feeds as one: cross-modal framework for general identification of AMPs
Zhang, W., Xu, Y., Wang, A., Chen, G., & Zhao, J. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad336 · Cited by 21
Praljak, N., Lian, X., Ranganathan, R., & Ferguson, A. L. (2023) · ACS Synthetic Biology · doi:10.1021/acssynbio.3c00261 · Cited by 18
Assessing protein model quality based on deep graph coupled networks using protein language model
(2023) · doi:10.1093/bib/bbad420 · Cited by 18
Self-citationWhen Protein Structure Embedding Meets Large Language Models
Ali, S., Chourasia, P., & Patterson, M. (2023) · Genes · doi:10.3390/genes15010025 · Cited by 23
Wang, H., Hao, X., He, Y., & Fan, L. (2024) · PLoS ONE · doi:10.1371/journal.pone.0296737 · Cited by 28
Martínez‐Mauricio, K. L., García‐Jacas, C. R., & Cordoves‐Delgado, G. (2024) · Protein Science · doi:10.1002/pro.4928 · Cited by 20
Biophysical principles predict fitness of SARS-CoV-2 variants
Wang, D., Huot, M., Mohanty, V., & Shakhnovich, E. I. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314518121 · Cited by 23
Soleymani, F., Paquet, E., Viktor, H. L., & Michalowski, W. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.06.021 · Cited by 19
Draizen, E. J., Veretnik, S., Mura, C., & Bourne, P. E. (2024) · Nature Communications · doi:10.1038/s41467-024-52020-2 · Cited by 21
Deep learning in template-free de novo biosynthetic pathway design of natural products
Xie, X., Gui, L., Qiao, B., Wang, G., Huang, S., Zhao, Y., & Sun, S. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae495 · Cited by 21
Knot data analysis using multiscale Gauss link integral
Shen, L., Feng, H., Li, F., Lei, F., Wu, J., & Wei, G. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2408431121 · Cited by 29
Anti-symmetric framework for balanced learning of protein–protein interactions
Tang, T., Li, T., Li, W., Cao, X., Liu, Y., & Zeng, X. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae603 · Cited by 23
Quake, S. R. (2024) · Cell · doi:10.1016/j.cell.2024.10.029 · Cited by 24
Considerations in the search for epistasis
Balvert, M., Cooper‐Knock, J., Stamp, J., Byrne, R. P., Mourragui, S. M. C., van Gils, J. H. M., Benónísdóttir, S., Schlüter, J., Kenna, K. P., Abeln, S., et al. (2024) · Genome biology · doi:10.1186/s13059-024-03427-z · Cited by 23
Deep protein representations enable recombinant protein expression prediction
Martiny, H.-M., Armenteros, J. J. A., Johansen, A. R., Salomon, J., & Nielsen, H. (2021) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2021.107596 · Cited by 17
Artificial Intelligence in the 21st Century
奥宮, 正. (2023) · Research on Intelligent Manufacturing and Assembly · doi:10.25082/rima.2023.01.002 · Cited by 21
End-to-end multitask learning, from protein language to protein features without alignmentsPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., & Rost, B. (2019) · bioRxiv · doi:10.1101/864405 · Cited by 19
Fast and flexible design of novel proteins using graph neural networks
Strokach, A., Becerra, D., Corbi-Verge, C., Perez-Riba, A., & Kim, P. M. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/868935 · Cited by 14
Microdroplet screening rapidly profiles a biocatalyst to enable its AI-assisted engineeringPreprint
Gantz, M., Mathis, S. V., Nintzel, F. E. H., Penner, M., Zurek, P. J., Knaus, T., Tseliou, V., Patel, E., Boros, D., Weberling, F.-M., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.04.08.588565 · Cited by 27
Mixture of Experts Enable Efficient and Effective Protein Understanding and DesignPreprint
Sun, N., Zou, S., Tao, T., Mahbub, S., Li, D., Zhuang, Y., Wang, H., Cheng, X., Song, L., & Xing, E. P. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.29.625425 · Cited by 30
Enzyme functional classification using artificial intelligence
Kim, H. R., Ji, H., Kim, G. B., & Lee, S. Y. (2025) · Trends in biotechnology · doi:10.1016/j.tibtech.2025.03.003 · Cited by 21
Foundation neural-networks quantum states as a unified Ansatz for multiple hamiltonians
Rende, R., Viteritti, L. L., Becca, F., Scardicchio, A., Laio, A., & Carleo, G. (2025) · Nature Communications · doi:10.1038/s41467-025-62098-x · Cited by 74
From machine learning to multimodal models: The AI revolution in enzyme engineering
Shi, Z., Xu, S., Xue, S., Chen, K., Lu, Y., Wang, F., Long, S., Tian, Y., Zhang, P., Wang, J., et al. (2025) · BioDesign Research · doi:10.1016/j.bidere.2025.100044 · Cited by 22
Evaluation of Machine Learning-Assisted Directed Evolution Across Diverse Combinatorial Landscapes
Li, F.-Z., Yang, J., Johnston, K. E., Gürsoy, E., Yue, Y., & Arnold, F. H. (2024) · Cell Systems · doi:10.1016/j.cels.2025.101387 · Cited by 35
Cerchione, R., Liccardo, G., & Passaro, R. (2025) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2025.100866 · Cited by 44
LDBT instead of DBTL: combining machine learning and rapid cell-free testing
Clark‐ElSayed, A., Harrison, I. M., Olsen, M. L., Lazar, J. T., Jewett, M. C., & Ellington, A. D. (2025) · Nature Communications · doi:10.1038/s41467-025-65281-2 · Cited by 21
Transforming microfluidics for single-cell analysis with robotics and artificial intelligence
Cheng, J., Anne, R., & Chen, Y. (2025) · Lab on a Chip · doi:10.1039/d5lc00216h · Cited by 23
Engineered base editors with reduced bystander editing through directed evolution
Perrotta, R. M., Vinke, S., Ferreira, R., Moret, M., Mahas, A., Chiappino-Pepe, A., Riedmayr, L. M., Mehra, A.-T., Lehmann, L. S., & Church, G. M. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02937-w · Cited by 35
Artificial intelligence for microbiology and microbiome research
Wang, X.-W., Wang, T. D., & Liu, Y. (2026) · Cell Systems · doi:10.1016/j.cels.2026.101531 · Cited by 44
Transformers and the Representation of Biomedical Background Knowledge
Wysocki, O., Zhou, Z., O’Regan, P., Ferreira, D. M., Wysocka, M., Landers, D., & Freitas, A. (2022) · Computational Linguistics · doi:10.1162/coli_a_00462 · Cited by 19
Artificial intelligence for template-free protein structure prediction: a comprehensive review
Mufassirin, M. M. M., Newton, M. A. H., & Sattar, A. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10350-x · Cited by 19
The intramembrane COOH-terminal domain of PRRT2 regulates voltage-dependent Na+ channels
Franchi, F., Marte, A., Corradi, B., Sterlini, B., Alberini, G., Romei, A., Fusco, A. D., Vogel, A., Maragliano, L., Baldelli, P., et al. (2023) · Journal of Biological Chemistry · doi:10.1016/j.jbc.2023.104632 · Cited by 18
Deciphering the protein landscape with ProtFlash, a lightweight language model
Wang, L., Zhang, H., Xu, W. M., Xue, Z., & Wang, Y. (2023) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2023.101600 · Cited by 28
Accurate prediction of functional effect of single amino acid variants with deep learning
Derbel, H., Zhao, Z., & Liu, Q. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.017 · Cited by 17
Wang, L., Zeng, Z., Xue, Z., & Wang, Y. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.12.004 · Cited by 18
Learning the shape of protein microenvironments with a holographic convolutional neural network
Pun, M. N., Ivanov, A., Bellamy, Q., Montague, Z., LaMont, C., Bradley, P. H., Otwinowski, J., & Nourmohammad, A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2300838121 · Cited by 23
Palacios, A. V., Acharya, P., Peidl, A. S., Beck, M. R., Blanco, E., Mishra, A., Bawa‐Khalfe, T., & Pakhrin, S. C. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae011 · Cited by 18
Partial order relation–based gene ontology embedding improves protein function prediction
Li, W., Wang, B., Dai, J., Kou, Y., Chen, X., Pan, Y. F., Hu, S., & Xu, Z. Z. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae077 · Cited by 21
Huang, Z., Chen, S., He, K., Yu, T., Fu, J., Gao, S., & Li, H. (2024) · Horticulture Research · doi:10.1093/hr/uhae082 · Cited by 21
KEGG orthology prediction of bacterial proteins using natural language processing
Chen, J., Wu, H., & Wang, N. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05766-x · Cited by 18
Enabling high-throughput enzyme discovery and engineering with a low-cost, robot-assisted pipeline
Norton‐Baker, B., Denton, M. C. R., Murphy, N. P., Fram, B. J., Lim, S., Erickson, E., Gauthier, N. P., & Beckham, G. T. (2024) · Scientific Reports · doi:10.1038/s41598-024-64938-0 · Cited by 20
Tao, W., Lin, X., Liu, Y., Li, Z., Ma, T., Cheng, N., Jiang, J., Zeng, X., & Yuan, S. (2024) · BMC Biology · doi:10.1186/s12915-024-02049-y · Cited by 17
Deep Learning for Protein–Protein Interaction Site Prediction
Jamasb, A. R., Day, B., Cangea, C., Lió, P., & Blundell, T. L. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1641-3_16 · Cited by 27
PRESTO: Rapid protein mechanical strength prediction with an end-to-end deep learning model
Liu, F. Y. C., Ni, B., & Buehler, M. J. (2022) · Extreme Mechanics Letters · doi:10.1016/j.eml.2022.101803 · Cited by 17
Language modelling for biological sequences – curated datasets and baselinesPreprint
Armenteros, J. J. A., Johansen, A. R., Winther, O., & Nielsen, H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.09.983585 · Cited by 19
Protein Structural Alignments From Sequence
Morton, J. T., Strauss, C. E. M., Blackwell, R., Berenberg, D., Gligorijevic, V., & Bonneau, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.11.03.365932 · Cited by 17
Dual-view Molecule Pre-trainingPreprint
Zhu, J., Xia, Y., Qin, T., Zhou, W., Li, H., & Liu, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.10234 · Cited by 53
TM-Vec: template modeling vectors for fast homology detection and alignment
Hamamsy, T., Morton, J. T., Berenberg, D., Carriero, N., Gligorijevic, V., Blackwell, R., Strauss, C. E. M., Leman, J. K., Cho, K., & Bonneau, R. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.07.25.501437 · Cited by 17
RNAGenesis: A Generalist Foundation Model for Functional RNA TherapeuticsPreprint
Zhang, Z., Jin, R., Chao, L., Xu, G., Zhang, Y., Zhou, G., Yin, D., Guo, Y., Fu, Y., Yang, Y., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.30.630826 · Cited by 30
ProtMamba: a homology-aware but alignment-free protein state space model
Sgarbossa, D., Malbranke, C., & Bitbol, A.-F. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf348 · Cited by 27
Do protein language models learn phylogeny?
Tule, S., Foley, G., & Bodén, M. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf047 · Cited by 19
Thrift, W. J., Lounsbury, N., Broadwell, Q., Heidersbach, A. J., Freund, E. C., Abdolazimi, Y., Phung, Q. T., Chen, J., Capietto, A.-H., Tong, A.-J., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-54887-7 · Cited by 28
Predicting pathogen evolution and immune evasion in the age of artificial intelligence
Hamelin, D. J., Scicluna, M., Saadie, I., Mostefai, F., Grenier, J., Baron, C., Caron, É., & Hussin, J. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.03.044 · Cited by 25
Dublino, R., & Ercolano, M. R. (2025) · The Crop Journal · doi:10.1016/j.cj.2025.05.008 · Cited by 21
Shi, J., Zhang, C., Deng, R., Fernie, A. R., Chen, M., & Zhang, Y. (2025) · Plant Communications · doi:10.1016/j.xplc.2025.101460 · Cited by 18
AlphaBind, a domain-specific model to predict and optimize antibody–antigen binding affinity
Agarwal, A. A., Harrang, J., Noble, D., McGowan, K. L., Lange, A. W., Engelhart, E., Lahman, M. C., Adamo, J., Yu, X., Serang, O., et al. (2025) · mAbs · doi:10.1080/19420862.2025.2534626 · Cited by 21
Quantifying uncertainty in protein representations across models and tasks
Prabakaran, R., & Bromberg, Y. (2026) · Nature Methods · doi:10.1038/s41592-026-03028-7 · Cited by 30
Metadata
- Authors
- Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R.
- Affiliations
- Meta (United States) (US)
- New York University (US)
- Harvard University (US)
- Yale University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-04-05
- Volume
- 118
- Issue
- 15
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8053943
- OpenAlex
- W3146944767
- Cited by
- 3,286
- References
- 100
- Citations by year
- 2026: 787 · 2025: 813 · 2024: 697 · 2023: 448 · 2022: 305 · 2021: 162
- FWCI
- 126.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
- University of Chicago
- Funders
- National Science Foundation (1339362)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, unsupervised learning, protein evolution, protein structure prediction, secondary structure prediction, protein tertiary structure, remote homology
- MeSH terms
- Unsupervised Machine Learning, Amino Acids, Protein Conformation, Sequence Homology, Amino Acid, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.