PreprintCited by 606Machine Learning in BioinformaticsProtein Structure and DynamicsFractal and DNA sequence analysis

The DOI matches its Crossref record; 103 of the 105 references it cites were found in a registry or an index.

Modeling the language of life – Deep Learning Protein Sequences

Open access

Heinzinger, Michael, Elnaggar, Ahmed, Wang, Yu, Dallago, Christian, Nechaev, Dmitrii, Matthes, Florian, Rost, Burkhard·2019·BMC Bioinformatics

DOI 10.1101/614313PMC PMC6918593

Preprint — published as 10.1186/s12859-019-3220-8 · Paper page

Open this paper in CoChat — free account Open PDF

Abstract

Background: One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results: We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion: We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability: SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1101/614313

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
The Protein Data Bank
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Untitled work
(2014) · Cited by 33,419
Untitled work
(2008) · Journal of Machine Learning Research · Cited by 33,051
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568

Flagged first, then by citations

Show all 105 references

2 of the 105 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

10.1007/s10479-011-0841-3. · DOI

Sorted by citations

Items 201–300 of 599

Deep embeddings to comprehend and visualize microbiome protein space
Odrzywolek, K., Karwowska, Z., Majta, J., Byrski, A., Milanowska-Zabel, K., & Kosciolek, T. (2022) · Scientific Reports · doi:10.1038/s41598-022-14055-7 · Cited by 22
Pathogen genomic surveillance and the AI revolution
Lytras, S., Lamb, K. D., Ito, J., Grove, J., Yuan, K., Sato, K., Hughes, J., & Robertson, D. L. (2025) · Journal of Virology · doi:10.1128/jvi.01601-24 · Cited by 24
pLM4CPPs: Protein Language Model-Based Predictor for Cell Penetrating Peptides
Kumar, N., Du, Z., & Li, Y. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01338 · Cited by 22
Large-scale predicting protein functions through heterogeneous feature fusion
Zheng, R., Huang, Z., & Deng, L. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad243 · Cited by 23
CollaPPI: A Collaborative Learning Framework for Predicting Protein-Protein Interactions
Ma, W., Bi, X., Jiang, H., Zhang, S., & Wei, Z. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3375621 · Cited by 22
Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream TasksPreprint
Filipavicius, M., Manica, M., Cadow, J., & Martínez, M. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03084 · Cited by 25
Fast and accurate protein intrinsic disorder prediction by using a pretrained language model
Song, Y., Yuan, Q., Chen, S., Chen, K., Zhou, Y., & Yang, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad173 · Cited by 21
MultiModRLBP: A Deep Learning Approach for Multi-Modal RNA-Small Molecule Ligand Binding Sites Prediction
Wang, J., Quan, L., Jin, Z. X., Wu, H., Ma, X., Wang, X., Xie, J., Pan, D., Chen, T., Wu, T., et al. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3400521 · Cited by 22
Deep metric learning for accurate protein secondary structure prediction
Yang, W. P., Liu, Y., & Xiao, C. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108356 · Cited by 23
Recent advances in generative biology for biotherapeutic discovery
Mock, M. L., LANGMEAD, C. J., Grandsard, P. J., Edavettal, S. C., & Russell, A. J. (2024) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2024.01.003 · Cited by 20
Assessing the role of evolutionary information for enhancing protein language model embeddings
Erckert, K., & Rost, B. (2024) · Scientific Reports · doi:10.1038/s41598-024-71783-8 · Cited by 21
Self-citation
Phage therapy against Klebsiella pneumoniae: An evolving perspective
Xing, J., Zhang, H., Zheng, L., Zhao, J., Zhang, Y., Xu, Z., Zhai, Y., Hu, G., Li, J., & Wu, H. (2025) · Biotechnology Advances · doi:10.1016/j.biotechadv.2025.108689 · Cited by 22
ISPRED-SEQ: Deep Neural Networks and Embeddings for Predicting Interaction Sites in Protein Sequences
Manfredi, M., Savojardo, C., Martelli, P. L., & Casadio, R. (2023) · Journal of Molecular Biology · doi:10.1016/j.jmb.2023.167963 · Cited by 20
Elucidating the functional roles of prokaryotic proteins using big data and artificial intelligence
Ardern, Z., Chakraborty, S., Lenk, F., & Kaster, A. (2023) · FEMS Microbiology Reviews · doi:10.1093/femsre/fuad003 · Cited by 22
HyperPCM: Robust Task-Conditioned Modeling of Drug–Target Interactions
Svensson, E., Hoedt, P.-J., Hochreiter, S., & Klambauer, G. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01417 · Cited by 21
T4SEpp: A pipeline integrating protein language models to predict bacterial type IV secreted effectors
Hu, Y., Wang, Y., Hu, X., Chao, H., Li, S., Ni, Q., Zhu, Y., Hu, Y., Zhao, Z., & Chen, M. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.01.015 · Cited by 23
Self-citation
The Role of Protein Engineering in Biomedical Applications of Mammalian Synthetic Biology
Bojar, D., & Fussenegger, M. (2019) · Small · doi:10.1002/smll.201903093 · Cited by 20
Mul-SNO: A Novel Prediction Tool for S-Nitrosylation Sites Based on Deep Learning Methods
Zhao, Q., Ma, J., Wang, Y., Xie, F., Lv, Z., Xu, Y., Shi, H., & Han, K. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3123503 · Cited by 20
Self-citation
Critical Assessment of Protein Intrinsic Disorder Round 3 ‐ Predicting Disorder in the Era of Protein Language Models
Mehdiabadi, M., Conte, A. D., Nugnes, M. V., Aspromonte, M. C., Tosatto, S. C. E., & Piovesan, D. (2025) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.70045 · Cited by 20
Protein Science Meets Artificial Intelligence: A Systematic Review and a Biochemical Meta-Analysis of an Inter-Field
Villalobos-Alva, J., Ochoa-Toledo, L., Villalobos-Alva, M. J., Aliseda, A., Escamirosa, F. P., Altamirano-Bustamante, N. F., Ochoa-Fernández, F., Zamora-Solís, R., Villalobos-Alva, S., Revilla, M., et al. (2022) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2022.788300 · Cited by 19
Ten quick tips for sequence-based prediction of protein properties using machine learning
Hou, Q., Waury, K., Gogishvili, D., & Feenstra, K. A. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010669 · Cited by 20
When Protein Structure Embedding Meets Large Language Models
Ali, S., Chourasia, P., & Patterson, M. (2023) · Genes · doi:10.3390/genes15010025 · Cited by 15
Unsupervised Representation Learning for Proteochemometric Modeling
Kim, P. T., Winter, R., & Clevert, D.-A. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms222312882 · Cited by 19
Hierarchical deep learning for predicting GO annotations by integrating protein knowledge
Merino, G. A., Saidi, R., Milone, D. H., Stegmayer, G. S., & Martin, M. J. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac536 · Cited by 19
LambdaPP : Fast and accessible protein‐specific phenotype predictions
(2022) · doi:10.1002/pro.4524 · Cited by 19
Self-citation
Deep Neural Network Framework Based on Word Embedding for Protein Glutarylation Sites Prediction
Liu, C.-M., Ta, V.-D., Le, N. Q. K., Addis, D., & Shi, C. (2022) · Life · doi:10.3390/life12081213 · Cited by 18
TPepPro: a deep learning model for predicting peptide–protein interactions
Xiao-hong, J., Chen, Z., Yu, D., Jiang, Q., Chen, Z., Yan, B., Qin, J., Liu, Y., & Wang, J. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae708 · Cited by 18
Language modelling for biological sequences – curated datasets and baselinesPreprint
Armenteros, J. J. A., Johansen, A. R., Winther, O., & Nielsen, H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.09.983585 · Cited by 19
Predicting pathogen evolution and immune evasion in the age of artificial intelligence
Hamelin, D. J., Scicluna, M., Saadie, I., Mostefai, F., Grenier, J., Baron, C., Caron, É., & Hussin, J. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.03.044 · Cited by 21
Improving prediction for medical institution with limited patient data: Leveraging hospital-specific data based on multicenter collaborative research network
Li, J., Tian, Y., Li, R., Zhou, T., Li, J., Ding, K., & Li, J. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102024 · Cited by 18
Protein sequence profile prediction using ProtAlbert transformer
Behjati, A., Zare‐Mirakabad, F., Arab, S. S., & Nowzari-Dalini, A. (2022) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2022.107717 · Cited by 22
Prediction of Protein-Protein Interactions Using Vision Transformer and Language Model
Jha, K., Saha, S., & Karmakar, S. (2023) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2023.3248797 · Cited by 18
Exploring the Potential of GANs in Biological Sequence Analysis
Murad, T., Ali, S., & Patterson, M. (2023) · Biology · doi:10.3390/biology12060854 · Cited by 17
Deciphering the protein landscape with ProtFlash, a lightweight language model
Wang, L., Zhang, H., Xu, W. M., Xue, Z., & Wang, Y. (2023) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2023.101600 · Cited by 20
Self-citation
SumoPred-PLM: human SUMOylation and SUMO2/3 sites Prediction using Pre-trained Protein Language Model
Palacios, A. V., Acharya, P., Peidl, A. S., Beck, M. R., Blanco, E., Mishra, A., Bawa‐Khalfe, T., & Pakhrin, S. C. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae011 · Cited by 17
Language model and its interpretability in biomedicine: A scoping review
Lyu, D., Wang, X., Chen, Y., & Wang, F. (2024) · iScience · doi:10.1016/j.isci.2024.109334 · Cited by 17
KEGG orthology prediction of bacterial proteins using natural language processing
Chen, J., Wu, H., & Wang, N. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05766-x · Cited by 17
Prediction of protein secondary structure by the improved TCN-BiLSTM-MHA model with knowledge distillation
Zhao, L., Li, J., Zhan, W., Jiang, X., & Zhang, B. (2024) · Scientific Reports · doi:10.1038/s41598-024-67403-0 · Cited by 17
iDeepSubMito: identification of protein submitochondrial localization with deep learning
Hou, Z., Yang, Y., Li, H., Wong, K., & Li, X. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab288 · Cited by 17
End-to-end multitask learning, from protein language to protein features without alignmentsPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., & Rost, B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/864405 · Cited by 17
Self-citation
Protein Structural Alignments From SequencePreprint
Morton, J. T., Strauss, C. E. M., Blackwell, R., Berenberg, D., Gligorijević, V., & Bonneau, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.11.03.365932 · Cited by 17
TM-Vec: template modeling vectors for fast homology detection and alignmentPreprint
Hamamsy, T., Morton, J. T., Berenberg, D., Carriero, N. J., Gligorijević, V., Blackwell, R., Strauss, C. E. M., Leman, J. K., Cho, K., & Bonneau, R. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.07.25.501437 · Cited by 16
AttSec: protein secondary structure prediction by capturing local patterns from attention map
(2023) · doi:10.1186/s12859-023-05310-3 · Cited by 16
Multi-dimensional metagenomics
Peng, H., Ruiz-Moreno, A. J., & Fu, J. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00346-x · Cited by 19
MMSMAPlus: a multi-view multi-scale multi-attention embedding model for protein function prediction
Wang, Z., Deng, Z., Zhang, W., Lou, Q., Choi, K., Wei, Z., Wang, L., & Wu, J. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad201 · Cited by 16
A novel hybrid CNN and BiGRU-Attention based deep learning model for protein function prediction
Sharma, L., Deepak, A., Ranjan, A., & Krishnasamy, G. (2023) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2022-0057 · Cited by 18
Application of Sequence Embedding in Protein Sequence-Based Predictions
Ibtehaz, N., & Kihara, D. (2022) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811258589_0002 · Cited by 18
Prediction of Protein–Protein Interaction Sites Based on Stratified Attentional Mechanisms
Tang, M., Wu, L., Yu, X., Chu, Z., Jin, S., & Liu, J. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.784863 · Cited by 16
Protein Fitness Prediction Is Impacted by the Interplay of Language Models, Ensemble Learning, and Sampling Methods
Mardikoraem, M., & Woldring, D. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15051337 · Cited by 16
AAontology: An Ontology of Amino Acid Scales for Interpretable Machine Learning
Breimann, S., Kamp, F., Steiner, H., & Frishman, D. (2024) · Journal of Molecular Biology · doi:10.1016/j.jmb.2024.168717 · Cited by 15
Recent Advances in the Prediction of Subcellular Localization of Proteins and Related Topics
Nakai, K., & Wei, L. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.910531 · Cited by 15
Protein Function Prediction With Functional and Topological Knowledge of Gene Ontology
Zhao, Y., Yang, Z., Hong, Y., Yang, Y., Wang, L., Zhang, Y., Lin, H., & Wang, J. (2023) · IEEE Transactions on NanoBioscience · doi:10.1109/tnb.2023.3278033 · Cited by 15
Sequence, Structure, and Functional Space of Drosophila De Novo Proteins
Middendorf, L., Iyengar, B. R., & Eicholt, L. A. (2024) · Genome Biology and Evolution · doi:10.1093/gbe/evae176 · Cited by 15
Capturing Protein Domain Structure and Function Using Self-Supervision on Domain Architectures
Melidis, D. P., Malone, B., & Nejdl, W. (2021) · Algorithms · doi:10.3390/a14010028 · Cited by 15
PSSM-Distil: Protein Secondary Structure Prediction (PSSP) on Low-Quality PSSM by Knowledge Distillation with Contrastive Learning
Wang, Q., Wang, B., Xu, Z., Wu, J., Zhao, P., Li, Z., Wang, S., Huang, J., & Cui, S. R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16141 · Cited by 18
Unifying the known and unknown microbial coding sequence spacePreprint
Vanni, C., Schechter, M. S., Acinas, S. G., Barberán, A., Buttigieg, P. L., Casamayor, E. O., Delmont, T. O., Duarte, C. M., Eren, A. M., Finn, R. D., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.30.180448 · Cited by 15
Improving Generalizability of Protein Sequence Models with Data AugmentationsPreprint
Shen, H., Price, L. C., Bahadori, M. T., & Seeger, F. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.18.431877 · Cited by 16
Major advances in protein function assignment by remote homolog detection with protein language models – A review
Kilinc, M., Jia, K., & Jernigan, R. L. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.102984 · Cited by 15
TripletProt: Deep Representation Learning of Proteins Based On Siamese Networks
Nouranı, E., Asgari, E., McHardy, A. C., & Mofrad, M. R. K. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3108718 · Cited by 14
ScanNet: A Web Server for Structure-based Prediction of Protein Binding Sites with Geometric Deep Learning
Tubiana, J., Schneidman‐Duhovny, D., & Wolfson, H. J. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167758 · Cited by 15
RPPSP: A Robust and Precise Protein Solubility Predictor by Utilizing Novel Protein Sequence Encoder
Mehmood, F., Arshad, S., & Shoaib, M. (2023) · IEEE Access · doi:10.1109/access.2023.3284464 · Cited by 14
Family-specific analysis of variant pathogenicity prediction tools
Zaucha, J. M., Heinzinger, M., Tarnovskaya, S. I., Rost, B., & Frishman, D. (2020) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaa014 · Cited by 14
Self-citation
Recent Advancements in Subcellular Proteomics: Growing Impact of Organellar Protein Niches on the Understanding of Cell Biology
Bhushan, V., & Nita‐Lazar, A. (2024) · Journal of Proteome Research · doi:10.1021/acs.jproteome.3c00839 · Cited by 15
HemoDL: Hemolytic peptides prediction by double ensemble engines from Rich sequence-derived and transformer-enhanced information
Yang, S., & Xu, P. (2024) · Analytical Biochemistry · doi:10.1016/j.ab.2024.115523 · Cited by 15
In the twilight zone of protein sequence homology: do protein language models learn protein structure?
Kabir, A., Moldwin, A., Bromberg, Y., & Shehu, A. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae119 · Cited by 13
Protein transfer learning improves identification of heat shock protein families
Min, S., Kim, H., Lee, B., & Yoon, S. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251865 · Cited by 13
Motif2Mol: Prediction of New Active Compounds Based on Sequence Motifs of Ligand Binding Sites in Proteins Using a Biochemical Language Model
Yoshimori, A., & Bajorath, J. (2023) · Biomolecules · doi:10.3390/biom13050833 · Cited by 13
How Deep Learning Tools Can Help Protein Engineers Find Good Sequences
Osadchy, M., & Kolodny, R. (2021) · The Journal of Physical Chemistry B · doi:10.1021/acs.jpcb.1c02449 · Cited by 16
Decoding functional proteome information in model organisms using protein language models
Barrios-Núñez, I., Martínez-Redondo, G. I., Medina-Burgos, P., Cases, I., Fernández, R., & Rojas, A. M. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae078 · Cited by 13
Improving viral annotation with artificial intelligence
Flamholz, Z., Li, C., & Kelly, L. (2024) · mBio · doi:10.1128/mbio.03206-23 · Cited by 12
MULAN: multimodal protein language model for sequence and structure encoding
Frolova, D., Pak, M., Litvin, A., Sharov, I., Ivankov, D., & Oseledets, I. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbaf117 · Cited by 12
Machine-Guided Dual-Objective Protein Engineering for Deimmunization and Therapeutic Functions
Wolfsberg, E., Paul, J.-S., Tycko, J., Chen, B., Bassik, M. C., Bintu, L., Alizadeh, A. A., & Gao, X. J. (2025) · Cell Systems · doi:10.1016/j.cels.2025.101299 · Cited by 12
Predictive and therapeutic applications of protein language models
Furui, K., Sakano, K., & Ohue, M. (2025) · Allergology International · doi:10.1016/j.alit.2025.08.004 · Cited by 12
NLP-BCH-Ens: NLP-based intelligent computational model for discrimination of malaria parasite
Hayat, M., Tahir, M., Alarfaj, F. K., Alturki, R., & Gazzawe, F. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105962 · Cited by 12
SPRoBERTa: protein embedding learning with local fragment modeling
Wu, L., Yin, C., Zhu, J., Wu, Z., He, L., Xia, Y., Xie, S., Qin, T., & Liu, T. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac401 · Cited by 12
DeepBSRPred: deep learning-based binding site residue prediction for proteins
Nikam, R., Yugandhar, K., & Gromiha, M. M. (2022) · Amino Acids · doi:10.1007/s00726-022-03228-3 · Cited by 12
Redesigning plant specialized metabolism with supervised machine learning using publicly available reactome data
Lim, P. K., Julca, I., & Mutwil, M. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.01.013 · Cited by 12
AI4Protein: transforming the future of protein design
Wang, D., Tan, Z., Gao, J., Zhang, S., Shen, J., & Lu, Y. (2025) · Science China Life Sciences · doi:10.1007/s11427-024-2906-3 · Cited by 12
Structure, Interactions, and Assembly of Membrane-Active Antimicrobial Polypeptides
Lee, T.-H., Charchar, P., Sani, M., Le, D.-H., Le, T. C., Yarovsky, I., Separovic, F., & Aguilar, M. (2026) · Chemical Reviews · doi:10.1021/acs.chemrev.5c00994 · Cited by 12
MPI-VGAE: protein–metabolite enzymatic reaction link learning by variational graph autoencoders
Wang, C., Yuan, C., Wang, Y., Chen, R., Shi, Y., Zhang, T., Xue, F., Patti, G. J., Wei, L., & Hou, Q. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad189 · Cited by 13
Self-citation
PANDA-3D: protein function prediction based on AlphaFold models
Zhao, C., Liu, T., & Wang, Z. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae094 · Cited by 11
Computational analysis of pathogen-host interactome for fast and low-risk in-silico drug repurposing in emerging viral threats like Mpox
Paul, D., Saha, S., Basu, S., & Chakraborti, T. (2024) · Scientific Reports · doi:10.1038/s41598-024-69617-8 · Cited by 11
TransConv: convolution-infused transformer for protein secondary structure prediction
Das, S., Ghosh, S., & Jana, N. D. (2025) · Journal of Molecular Modeling · doi:10.1007/s00894-024-06259-7 · Cited by 12
Predicting purification process fit of monoclonal antibodies using machine learning
Maier, A. J., Cha, M., Burgess, S., Wang, A., Cuellar, C., Kim, S. Y., Rajan, N. S., Neyyan, J., SENGUPTA, R., O’Connor, K., et al. (2025) · mAbs · doi:10.1080/19420862.2024.2439988 · Cited by 11
A systematic evaluation of the language-of-viral-escape model using multiple machine learning frameworks
Allman, B. E., Vieira, L., Diaz, D. J., & Wilke, C. O. (2025) · Journal of The Royal Society Interface · doi:10.1098/rsif.2024.0598 · Cited by 11
Protein Sequence Analysis landscape: A Systematic Review of Task Types, Databases, Datasets, Word Embeddings Methods, and Language Models
Asim, M. N., Asif, T., Hassan, F., & Dengel, A. (2025) · Database · doi:10.1093/database/baaf027 · Cited by 11
DegronMD: Leveraging Evolutionary and Structural Features for Deciphering Protein-Targeted Degradation, Mutations, and Drug Response to Degrons
Xu, H., Hu, R., & Zhao, Z. (2023) · Molecular Biology and Evolution · doi:10.1093/molbev/msad253 · Cited by 11
Enhancing predictions of protein stability changes induced by single mutations using MSA-based language models
Cuturello, F., Celoria, M., Ansuini, A., & Cazzaniga, A. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae447 · Cited by 11
PatchProt: hydrophobic patch prediction using protein foundation models
Gogishvili, D., Minois-Genin, E., van Eck, J., & Abeln, S. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae154 · Cited by 11
Domain-Specific Contextualized Embedding: A Systematic Literature Review
Yunianto, I., Permanasari, A. E., & Widyawan, W. (2020) · doi:10.1109/icitee49829.2020.9271752 · Cited by 12
FoldHSphere: deep hyperspherical embeddings for protein fold recognition
Villegas-Morcillo, A., Sanchez, V., & Gomez, A. M. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04419-7 · Cited by 11
ProtPlat: an efficient pre-training platform for protein classification based on FastText
Jin, Y., & Yang, Y. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04604-2 · Cited by 11
ProteinGLUE multi-task benchmark suite for self-supervised protein modeling
Capel, H., Weiler, R., Dijkstra, M., Vleugels, R., Bloem, P., & Feenstra, K. A. (2022) · Scientific Reports · doi:10.1038/s41598-022-19608-4 · Cited by 11
EmbedCaps-DBP: Predicting DNA-Binding Proteins Using Protein Sequence Embedding and Capsule Network
Mursalim, M. K. N., Mengko, T. L., Hertadi, R., Purwarianti, A., & Susanty, M. (2023) · IEEE Access · doi:10.1109/access.2023.3328960 · Cited by 10
MetaDegron: multimodal feature-integrated protein language model for predicting E3 ligase targeted degrons
Zheng, M., Lin, S., Chen, K., Hu, R., Wang, L., Zhao, Z., & Xu, H. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae519 · Cited by 11
Locality-aware pooling enhances protein language model performance across varied applications
Hoang, M., & Singh, M. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf178 · Cited by 10
Transfer learning on protein language models improves antimicrobial peptide classification
Georgoulis, E., Zervou, M. A., & Pantazis, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-21223-y · Cited by 13
Deep learning model predicts water interaction sites on the surface of proteins using limited-resolution data
Zaucha, J. M., Softley, C. A., Sattler, M., Frishman, D., & Popowicz, G. M. (2020) · Chemical Communications · doi:10.1039/d0cc04383d · Cited by 12
PROVAL: A framework for comparison of protein sequence embeddings
Väth, P., Münch, M., Raab, C., & Schleif, F.-M. (2022) · Journal of Computational Mathematics and Data Science · doi:10.1016/j.jcmds.2022.100044 · Cited by 11
Improving protein structure prediction using templates and sequence embedding
Wu, F., Jing, X., Luo, X., & Xu, J. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac723 · Cited by 12

Authors: Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B.
Affiliations: Technical University of Munich (DE)
Leibniz Supercomputing Centre (DE)
Corresponding author: Heinzinger, Michael
ORCID: Heinzinger, Michael
Elnaggar, Ahmed
Wang, Yu
Dallago, Christian
Matthes, Florian
Rost, Burkhard
Venue: BMC Bioinformatics
Publisher: openRxiv
Published: 2019-04-19
Volume: 20
Issue: 1
Pages: 723-723
ISSN: 1471-2105 (ISSN-L)
DOI: 10.1101/614313
PMC: PMC6918593
OpenAlex: W2995514860
Cited by: 606
References: 105
Citations by year: 2026: 53 · 2025: 103 · 2024: 115 · 2023: 117 · 2022: 99 · 2021: 83 · 2020: 29 · 2019: 7
FWCI: 18.03
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
Locations: BMC Bioinformatics (publishedVersion, cc-by)
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
Columbia Academic Commons (Columbia University)
Funders: Alexander von Humboldt-Stiftung
Nvidia
Deutsche Forschungsgemeinschaft (DFG-GZ: RO1320/4-1, RO1320/4‐1)
Bundesministerium für Bildung und Forschung
Leibniz-Gemeinschaft
Topics: Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
Keywords: protein sequence embeddings, ELMo, transfer learning, protein function prediction, protein structure prediction, secondary structure prediction, subcellular localization prediction, membrane proteins, dark proteome, metaproteomics, microbiome analysis
MeSH terms: Machine Learning, Amino Acid Sequence, Natural Language Processing, Proteins, Neural Networks, Computer, Sequence Analysis, Computational Biology, Databases, Nucleic Acid, Databases, Protein, Proteomics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Modeling the language of life – Deep Learning Protein Sequences

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks