Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Rives, Alexander, Meier, Joshua, Sercu, Tom, Goyal, Siddharth, Lin, Zeming, Liu, Jason, Guo, Demi, Ott, Myle +3 more·2019·Proceedings of the National Academy of Sciences
DOI 10.1101/622803
Cited by (1,005)
Items 501–600 of 1,005
- Alignment-free estimation of sequence conservation for identifying functional sites using protein sequence embeddingsWayland Yeung, Zhongliang Zhou, Sheng Li, Natarajan Kannan · 2023 · Briefings in Bioinformatics
- scGGAN: single-cell RNA-seq imputation by graph-based generative adversarial networkZimo Huang, Jun Wang, Xudong Lü, Azlan Mohd Zain, Guoxian Yu · 2023 · Briefings in Bioinformatics
- Meta learning addresses noisy and under-labeled data in machine learning-guided antibody engineeringMinot, Mason, Reddy, Sai T. · 2024 · Cell Systems
- PLM_Sol: predicting protein solubility by benchmarking multiple protein language models with the updated Escherichia coli protein solubility datasetX.J. Zhang, Xiaoxuan Hu, Tongtong Zhang, Ling Yang, Chunhong Liu, Ning Xu, Haoyi Wang, Wen Sun · 2024 · Briefings in Bioinformatics
- Deep Learning Methods for De Novo Peptide SequencingWout Bittremieux, Varun Ananth, William E. Fondrie, Carlo Melendez, Marina Pominova, Justin Sanders, Bo Wen, Melih Yilmaz, William Stafford Noble · 2024 · Mass Spectrometry Reviews
- SE(3)-equivariant ternary complex prediction towards target protein degradationFanglei Xue, M L Zhang, Shuqi Li, Xinyu Gao, James Akira Wohlschlegel, Wenbing Huang, Yi Yang, Weixian Deng · 2025 · Nature Communications
- Critical assessment of missense variant effect predictors on disease-relevant variant dataRuchir Rastogi, Ryan Chung, Sindy Li, Chang Li, Kyoungyeul Lee, Junwoo Woo, Dongwook Kim, Changwon Keum, Giulia Babbi, Pier Luigi Martelli et al. · 2025 · Human Genetics
- Democratizing protein language model training, sharing and collaborationSu, Jin, Li, Zhikai, Tao, Tianli, Han, Chenchen, He, Yan, Dai, Fengyuan, Yuan, Qingyan, Gao, Yuan, Si, Tong, Zhang, Xuting et al. · 2026 · Nature Biotechnology
- Pretraining model for biological sequence dataBosheng Song, Zimeng Li, Xuan Lin, Jianmin Wang, Tian Wang, Xiangzheng Fu · 2021 · Briefings in Functional Genomics
- Alzheimer's Disease Classification Through Imaging Genetic Data With IGnetJade Xiaoqing Wang, Yimei Li, Xintong Li, Zhaohua Lu · 2022 · Frontiers in Neuroscience
- 3D-equivariant graph neural networks for protein model quality assessmentChen, Chen, Chen, Xiao, Morehead, Alex, Wu, Tianqi, Cheng, Jianlin · 2023 · Bioinformatics
- S‐PLM: Structure‐Aware Protein Language Model via Contrastive Learning Between Sequence and StructureWang, Duolin, Pourmirzaei, Mahdi, Abbas, Usman L., Zeng, Shuai, Manshour, Negin, Esmaili, Farzaneh, Poudel, Biplab, Jiang, Yuexu, Shao, Qing, Chen, Jin et al. · 2025 · Advanced Science
- Flexible protein–protein docking with a multitrack iterative transformerChu, Lee‐Shin, Ruffolo, Jeffrey A., Harmalkar, Ameya, Gray, Jeffrey J. · 2024 · Protein Science
- GlyNet: a multi-task neural network for predicting protein–glycan interactionsEric J. Carpenter, Shaurya Seth, Noel Yue, Russell Greiner, Ratmir Derda · 2022 · Chemical Science
- Phosformer: an explainable transformer model for protein kinase-specific phosphorylation predictionsZhongliang Zhou, Wayland Yeung, Nathan Gravel, Mariah Victoria Salcedo, Saber Soleymani, Sheng Li, Natarajan Kannan · 2023 · Bioinformatics
- The promises of large language models for protein design and modelingGiorgio Valentini, Dario Malchiodi, Jessica Gliozzo, Marco Mesiti, Mauricio Soto-Gomez, Alberto Cabri, Justin Reese, Elena Casiraghi, Peter N. Robinson · 2023 · Frontiers in Bioinformatics
- Accurate prediction of antibody function and structure using bio-inspired antibody language modelHongtai Jing, Zhengtao Gao, Sheng Yong Xu, Tao Shen, Zhangzhi Peng, Shwai He, Tao You, Shuang Ye, Wei Jen Lin, Siqi Sun · 2024 · Briefings in Bioinformatics
- General-purpose foundation models for increased autonomy in robot-assisted surgerySamuel Schmidgall, Ji Woong Kim, Alan Kuntz, Ahmed Ghazi, Axel Krieger · 2024 · Nature Machine Intelligence
- Conditional generative modeling for de novo protein design with hierarchical functionsTim Kucera, Matteo Togninalli, Laetitia Meng-Papaxanthos · 2022 · Bioinformatics
- Machine Learning Approaches to TCR Repertoire AnalysisYotaro Katayama, Ryo Yokota, Taishin Akiyama, Tetsuya Kobayashi · 2022 · Frontiers in Immunology
- Pre-trained Language Models in Biomedical Domain: A Systematic SurveyBenyou Wang, Qianqian Xie, Jiahuan Pei, Chen, Zhihong, Prayag Tiwari, Zhao Li, Jie Fu · 2021 · arXiv (Cornell University)
- CodonBERT: Large Language Models for mRNA design and optimizationSizhen Li, Saeed Moayedpour, Ruijiang Li, Michael Bailey, Saleh Riahi, Lorenzo Kogler-Anele, Milad Miladi, Jacob C. Miner, Dinghai Zheng, Jun Wang et al. · 2023 · bioRxiv (Cold Spring Harbor Laboratory)
- Language Modeling Materializes a World Model of Protein BiologySalvatore Candido, Thomas Hayes, Alexander Derry, Roshan Rao, Zeming Lin, Robert Verkuil, Bryan Z. Wu, Jin Sub Lee, Elise S. Bruguera, Jehan A. Keval et al. · 2026 · bioRxiv (Cold Spring Harbor Laboratory)
- Leveraging protein language models for accurate multiple sequence alignments2023
- Protein representations: Encoding biological information for machine learning in biocatalysisHarding-Larsen, David, Funk, Jonathan, Madsen, Niklas Gesmar, Gharabli, Hani, Acevedo-Rocha, Carlos G., Mazurenko, Stanislav, Welner, Ditte Hededam · 2024 · Biotechnology Advances
- An Interpretable Double-Scale Attention Model for Enzyme Protein Class Prediction Based on Transformer Encoders and Multi-Scale ConvolutionsKen Lin, Xiong-Wen Quan, Chengcai Jin, Zhuangwei Shi, Jinglong Yang · 2022 · Frontiers in Genetics
- A Multimodal Protein Representation Framework for Quantifying Transferability Across Biochemical Downstream TasksFan Hu, Yishen Hu, Weihong Zhang, Huazhen Huang, Yi Feng Pan, Peng Yin · 2023 · Advanced Science
- Superior protein thermophilicity prediction with protein language model embeddingsFlorian Haselbeck, Maura John, Yuqi Zhang, Jonathan Pirnay, J. Werner, Rubén D. Costa, Dominik G. Grimm · 2023 · NAR Genomics and Bioinformatics
- Accurately identifying nucleic-acid-binding sites through geometric graph learning on language model predicted structuresYidong Song, Qianmu Yuan, Huiying Zhao, Yuedong Yang · 2023 · Briefings in Bioinformatics
- Machine learning in RNA structure prediction: Advances and challengesSicheng Zhang, Jun Li, Shi‐Jie Chen · 2024 · Biophysical Journal
- Leveraging Artificial Intelligence to Expedite Antibody Design and Enhance Antibody–Antigen InteractionsDoo Nam Kim, Andrew D. McNaughton, Neeraj Kumar · 2024 · Bioengineering
- FusPB-ESM2: Fusion model of ProtBERT and ESM-2 for cell-penetrating peptide predictionFan Zhang, Jinfeng Li, Zhenguo Wen, Chun Fang · 2024 · Computational Biology and Chemistry
- Enhanced Protein Secondary Structure Prediction Through Multi-View Multi-Feature Evolutionary Deep Fusion MethodYining Qian, Li-Jie Su, Meiling Xu, Lixin Tang · 2025 · IEEE Transactions on Emerging Topics in Computational Intelligence
- Deep indel mutagenesis reveals the impact of amino acid insertions and deletions on protein stability and functionMagdalena Topolska, Antoni Beltran, Ben Lehner · 2025 · Nature Communications
- Massive experimental quantification allows interpretable deep learning of protein aggregationMichael Thompson, Mariano Martín, Trinidad Sanmartín Olmo, Chandana Rajesh, Peter K. Koo, Benedetta Bolognesi, Ben Lehner · 2025 · Science Advances
- Machine Learning-Guided Identification of PET Hydrolases from Natural DiversityBrenna Norton‐Baker, Evan Komp, Japheth E. Gado, Mackenzie C. R. Denton, Irimpan I. Mathews, Natasha P. Murphy, Erika Erickson, Olateju O. Storment, Ritimukta Sarangi, Nicholas Paul Gauthier et al. · 2025 · ACS Catalysis
- Large-scale design and refinement of stable proteins using sequence-only modelsJedediah M. Singer, Scott Novotney, Devin Strickland, Hugh K. Haddox, Nicholas Leiby, Gabriel J. Rocklin, Cameron M. Chow, Anindya Roy, Asim K. Bera, Francis C. Motta et al. · 2022 · PLoS ONE
- Parameter-efficient fine-tuning on large protein language models improves signal peptide predictionShuai Zeng, Duolin Wang, Lei Jiang, Dong He Xu · 2024 · Genome Research
- High-throughput deep learning variant effect prediction with Sequence UNETDunham, Alistair S., Beltrao, Pedro, AlQuraishi, Mohammed · 2023 · Genome Biology
- SVSBI: sequence-based virtual screening of biomolecular interactionsLi Shen, Hongsong Feng, Yuchi Qiu, Guo‐Wei Wei · 2023 · Communications Biology
- DeepSecE: A Deep-Learning-Based Framework for Multiclass Prediction of Secreted Proteins in Gram-Negative BacteriaYumeng Zhang, Jiahao Guan, Chen Li, Zhikang Wang, Zixin Deng, Robin B. Gasser, Jiangning Song, Hong‐Yu Ou · 2023 · Research
- TCR-ESM: Employing protein language embeddings to predict TCR-peptide-MHC bindingShashank Yadav, Dhvani Sandip Vora, Durai Sundar, Jaspreet Kaur Dhanjal · 2023 · Computational and Structural Biotechnology Journal
- Multimodal pretraining for unsupervised protein representation learningViet Thanh Duy Nguyen, Truong Son Hy · 2024 · Biology Methods and Protocols
- Natural Language Processing Methods for the Study of Protein–Ligand InteractionsJames Michels, Ramya Bandarupalli, Amin Ahangar Akbari, Thai Le, Hong Yu Xiao, Jing Li, Erik F. Y. Hom · 2025 · Journal of Chemical Information and Modeling
- Variant effect predictions capture some aspects of deep mutational scanning experimentsJonas Reeb, Theresa Wirth, Burkhard Rost · 2020 · BMC Bioinformatics
- T4SEfinder: a bioinformatics tool for genome-scale prediction of bacterial type IV secreted effectors using pre-trained protein language modelYumeng Zhang, Yangming Zhang, Yi Xiong, Hui Wang, Zixin Deng, Jiangning Song, Hong‐Yu Ou · 2021 · Briefings in Bioinformatics
- QuoteTarget : A sequence‐based transformer protein language model to identify potentially druggable protein targetsJiaxiao Chen, Zhonghui Gu, Youjun Xu, Minghua Deng, Luhua Lai, Jianfeng Pei · 2022 · Protein Science
- Is Transfer Learning Necessary for Protein Landscape Prediction?Amir Shanehsazzadeh, David Belanger, David Dohan · 2020 · arXiv (Cornell University)
- Self-supervised learning on millions of pre-mRNA sequences improves sequence-based RNA splicing predictionKen Chen, Yue Zhou, Maolin Ding, Yu Wang, Zhixiang Ren, Yuedong Yang · 2023 · bioRxiv (Cold Spring Harbor Laboratory)
- PerturbNet predicts single-cell responses to unseen chemical and genetic perturbationsYu, Hengshi, Qian, Weizhou, Song, Yuxuan, Welch, Joshua D · 2025 · Molecular Systems Biology
- Evolutionary-scale enzymology enables exploration of a rugged catalytic landscape2025
- RamanFormer: A Transformer-Based Quantification Approach for Raman Mixture ComponentsKoyun, Onur Can, Keser, Reyhan Kevser, Şahin, Safa Onur, Bulut, Damla, Yorulmaz, Mustafa, Yücesoy, Veysel, Töreyin, Behçet Uğur · 2024 · ACS Omega
- Machine learning approaches for biomolecular, biophysical, and biomaterials research.C. Rickert, O. Lieleg · 2022 · Biophysical Reviews
- pLMSNOSite: an ensemble-based approach for predicting protein S-nitrosylation sites by integrating supervised word embedding and embedding from pre-trained protein language modelPawel Pratyush, Suresh Pokharel, Hiroto Saigo, Dukka B. KC · 2023 · BMC Bioinformatics
- Domain-PFP allows protein function prediction using function-aware domain embedding representationsNabil Ibtehaz, Yuki Kagaya, Daisuke Kihara · 2023 · Communications Biology
- Unbiased organism-agnostic and highly sensitive signal peptide predictor with deep protein language modelJun-Bo Shen, Qinze Yu, Shen-Xue Chen, Qingxiong Tan, Jing-Chao Li, Yu Li · 2023 · Nature Computational Science
- Advancing bioinformatics with language models: components, applications, and perspectivesJiajia Liu, Mengyuan Yang, Yankai Yu, Haixia Xu, Tiangang Wang, Kang Li, Xiaobo Zhou · 2026 · Briefings in Bioinformatics
- Learnt representations of proteins can be used for accurate prediction of small molecule binding sites on experimentally determined and predicted protein structuresAnna Carbery, Martin Buttenschoen, R. Skyner, F. von Delft, Charlotte Mary Deane · 2024 · Journal of Cheminformatics
- MDF-DTA: A Multi-Dimensional Fusion Approach for Drug-Target Binding Affinity PredictionAmit Ranjan, Adam Bess, Chris Alvin, Supratik Mukhopadhyay · 2024 · Journal of Chemical Information and Modeling
- AttABseq: an attention-based deep learning prediction method for antigen–antibody binding affinity changes based on protein sequencesRuofan Jin, Qing Ye, Jike Wang, Zheng Cao, Dejun Jiang, Tianyue Wang, Yu Kang, Wanting Xu, Chang‐Yu Hsieh, Tingjun Hou · 2024 · Briefings in Bioinformatics
- Machine-Learning-Guided Peptide Drug Discovery: Development of GLP-1 Receptor Agonists with Improved Drug PropertiesJens Christian Froslev Nielsen, Claudia U. Hjørringgaard, Mads Mørup Nygaard, Anita Wester, Lisbeth Elster, Trine Porsgaard, Randi Bonke Mikkelsen, Silas Anselm Rasmussen, Andreas Nygaard Madsen, Morten Schlein et al. · 2024 · Journal of Medicinal Chemistry
- Learning the language of antibody hypervariabilityRohit Singh, Chiho Im, Yu Qiu, Brian C. Mackness, Abhinav Gupta, Taylor Joren, Samuel Sledzieski, Lena Erlach, Maria Wendt, Yves Fomekong‐Nanfack et al. · 2024 · Proceedings of the National Academy of Sciences
- Artificial intelligence-assisted next-generation biomaterials: From design and preparation to medical applicationsBixia Zhou, Xin Li, Yuchen Pan, Bingfang He, Bingbing Gao · 2025 · Colloids and Surfaces B Biointerfaces
- MSA-Regularized Protein Sequence Transformer toward Predicting Genome-Wide Chemical-Protein Interactions: Application to GPCRome DeorphanizationTian Cai, Hansaim Lim, Kyra Alyssa Abbu, Yue Qiu, Ruth Nussinov, Lei Xie · 2021 · Journal of Chemical Information and Modeling
- Machine learning in energy chemistry: introduction, challenges and perspectivesYuzhi Xu, Jiankai Ge, Cheng‐Wei Ju · 2023 · Energy Advances
- PocketAnchor: Learning structure-based pocket representations for protein-ligand interaction predictionShuya Li, Tingzhong Tian, Ziting Zhang, Ziheng Zou, Dan Yang Zhao, Jianyang Zeng · 2023 · Cell Systems
- Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodiesJeffrey A. Ruffolo, Lee‐Shin Chu, Sai Pooja Mahajan, Jeffrey J. Gray · 2022 · bioRxiv (Cold Spring Harbor Laboratory)
- BEND: Benchmarking DNA Language Models on biologically meaningful tasksF. I. Marin, Felix Teufel, Marc Horlacher, Dennis Madsen, Dennis Pultz, Ole Winther, Wouter Boomsma · 2023 · International Conference on Learning Representations
- Beyond sequence: Structure-based machine learningJanani Durairaj, Dick de Ridder, Aalt D. J. van Dijk · 2022 · Computational and Structural Biotechnology Journal
- Computationally guided AAV engineering for enhanced gene deliveryJingxuan Guo, Li F. Lin, Sydney V. Oraskovich, Julio A. Rivera de Jesús, Jennifer Listgarten, David V. Schaffer · 2024 · Trends in Biochemical Sciences
- PLMACPred prediction of anticancer peptides based on protein language model and wavelet denoising transformationMuhammad Arif, Saleh Musleh, Huma Fida, Tanvir Alam · 2024 · Scientific Reports
- DeepMineLys: Deep mining of phage lysins from human microbiome.Yiran Fu, Shuting Yu, Jianfeng Li, Zisha Lao, Xiao-Feng Yang, Zhang-Lin Lin · 2024 · Cell Reports
- Structure-informed protein language models are robust predictors for variant effects2024
- PGAT-ABPp: harnessing protein language models and graph attention networks for antibacterial peptide identification with remarkable accuracyYuelei Hao, Xuyang Liu, Haohao Fu, Xueguang Shao, Wensheng Cai · 2024 · Bioinformatics
- Evaluation of residue-residue contact prediction methods: From retrospective to prospectiveHuiling Zhang, Zhendong Bei, Wenhui Xi, Min Hao, Zhen Ju, Konda Mani Saravanan, Haiping Zhang, Ning Guo, Yanjie Wei · 2021 · PLoS Computational Biology
- Self-Supervised Pre-training for Protein Embeddings Using Tertiary StructuresYuzhi Guo, Jiaxiang Wu, Hehuan Ma, Junzhou Huang · 2022 · Proceedings of the AAAI Conference on Artificial Intelligence
- Medium-sized protein language models perform well at transfer learning on realistic datasetsVieira, Luiz C., Handojo, Morgan L., Wilke, Claus O. · 2025 · Scientific Reports
- HNetGO: protein function prediction via heterogeneous network transformerXiaoshuai Zhang, Huannan Guo, Fan Zhang, Xuan Wang, Kaitao Wu, Shizheng Qiu, Bo Liu, Yadong Wang, Yang Hu, Junyi Li · 2021 · Briefings in Bioinformatics
- Data-driven strategies for the computational design of enzyme thermal stability: trends, perspectives, and prospectsZhixin Dou, Yuqing Sun, Xukai Jiang, Xiuyun Wu, Yingjie Li, Bin Gong, Lushan Wang · 2023 · Acta Biochimica et Biophysica Sinica
- DG-Affinity: predicting antigen–antibody affinity with language models from sequencesYe Yuan, Qushuo Chen, Jun Mao, Guipeng Li, Xiaoyong Pan · 2023 · BMC Bioinformatics
- M-Ionic: prediction of metal-ion-binding sites from sequence using residue embeddingsAditi Shenoy, Yogesh Kalakoti, Durai Sundar, Arne Elofsson · 2024 · Bioinformatics
- ALDELE: All-Purpose Deep Learning Toolkits for Predicting the Biocatalytic Activities of EnzymesXiangwen Wang, Derek J. Quinn, Thomas S. Moody, Meilan Huang · 2024 · Journal of Chemical Information and Modeling
- Approaching Optimal pH Enzyme Prediction with Large Language ModelsMark Zaretckii, Pavel Buslaev, Igor Kozlovskii, Alexander Morozov, Petr A. Popov · 2024 · ACS Synthetic Biology
- Learning a deep language model for microbiomes: The power of large scale unlabeled microbiome dataQuintin Pope, Rohan Varma, Christine A. Tataru, Maude M. David, Xiaoli Z. Fern · 2025 · PLoS Computational Biology
- Accelerating antibody discovery and optimization with high-throughput experimentation and machine learningRyo Matsunaga, Kouhei Tsumoto · 2025 · Journal of Biomedical Science
- miRe2e: a full end-to-end deep model based on transformers for prediction of pre-miRNAsJonathan Raad, Leandro A. Bugnon, Diego Humberto Milone, Georgina S. Stegmayer · 2021 · Bioinformatics
- Protein Function Analysis through Machine LearningChris Avery, John Patterson, Tyler J. Grear, Theodore D. Frater, Donald J. Jacobs · 2022 · Biomolecules
- GOProFormer: A Multi-Modal Transformer Method for Gene Ontology Protein Function PredictionAnowarul Kabir, Amarda Shehu · 2022 · Biomolecules
- Survey of Protein Sequence Embedding ModelsChau Thi Thuy Tran, Siddharth Khadkikar, Aleksey Porollo · 2023 · International Journal of Molecular Sciences
- Transformers and Large Language Models for Chemistry and Drug DiscoveryAndres M Bran, Philippe Schwaller · 2024 · Springer eBooks
- Single Layers of Attention Suffice to Predict Protein ContactsNicholas Bhattacharya, Neil Thomas, Roshan Rao, Justas Dauparas, Peter K. Koo, David A. Baker, Yun S. Song, Sergey Ovchinnikov · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
- EvoBind: in silico directed evolution of peptide binders with AlphaFoldPatrick Bryant, Arne Elofsson · 2022 · bioRxiv (Cold Spring Harbor Laboratory)
- Decoding molecular mechanisms for loss-of-function variants in the human proteomeMatteo Cagiada, Nicolas Jonsson, Kresten Lindorff‐Larsen · 2024 · bioRxiv (Cold Spring Harbor Laboratory)
- Seq-InSite: sequence supersedes structure for protein interaction site predictionHosseini, SeyedMohsen, Golding, G Brian, Ilie, Lucian · 2024 · Bioinformatics
- Finding functional motifs in protein sequences with deep learning and natural language modelsCastrense Savojardo, Pier Luigi Martelli, Rita Casadio · 2023 · Current Opinion in Structural Biology
- Designing meaningful continuous representations of T cell receptor sequences with deep generative modelsAllen Y. Leary, Darius Scott, Namita T. Gupta, Janelle C. Waite, Dimitris Skokos, Gurinder S. Atwal, Peter G. Hawkins · 2024 · Nature Communications
- Masked particle modeling on sets: towards self-supervised high energy physics foundation modelsT. Golling, L. Heinrich, M. Kagan, S. B. Klein, Matthew Leigh, Margarita Osadchy, J. A. Raine · 2024 · Machine Learning Science and Technology
- Leveraging mRNA technology for antigen based immuno-oncology therapiesCharalampos S. Floudas, Siranush Sarkizova, Michele Ceccarelli, Wei Zheng · 2025 · Journal for ImmunoTherapy of Cancer
- Tree visualizations of protein sequence embedding space enable improved functional clustering of diverse protein superfamiliesWayland Yeung, Zhongliang Zhou, Liju Mathew, Nathan Gravel, Rahil Taujale, Brady M. O’Boyle, Mariah Victoria Salcedo, Aarya Venkat, William N. Lanzilotta, Sheng Li et al. · 2023 · Briefings in Bioinformatics
- TEMPROT: protein function annotation using transformers embeddings and homology searchGabriel B. Oliveira, Hélio Pedrini, Zanoni Dias · 2023 · BMC Bioinformatics