PreprintCited by 606Machine Learning in BioinformaticsProtein Structure and DynamicsFractal and DNA sequence analysis

The DOI matches its Crossref record; 103 of the 105 references it cites were found in a registry or an index.

Modeling the language of life – Deep Learning Protein Sequences

Open access

Heinzinger, Michael, Elnaggar, Ahmed, Wang, Yu, Dallago, Christian, Nechaev, Dmitrii, Matthes, Florian, Rost, Burkhard·2019·BMC Bioinformatics

DOI 10.1101/614313PMC PMC6918593

Preprint — published as 10.1186/s12859-019-3220-8 · Paper page

Open this paper in CoChat — free account Open PDF

Abstract

Background: One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results: We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion: We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability: SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1101/614313

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
The Protein Data Bank
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Untitled work
(2014) · Cited by 33,419
Untitled work
(2008) · Journal of Machine Learning Research · Cited by 33,051
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568

Flagged first, then by citations

Show all 105 references

2 of the 105 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

10.1007/s10479-011-0841-3. · DOI

Sorted by citations

Items 301–400 of 599

ELISL: early–late integrated synthetic lethality prediction in cancer
Tepeli, Y. I., Seale, C., & Gonçalves, J. P. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad764 · Cited by 11
LSTM-SAGDTA: Predicting Drug-target Binding Affinity with an Attention Graph Neural Network and LSTM Approach
Qiu, W., Liang, Q., Yu, L., Xiao, X., Qiu, W., & Lin, W. (2024) · Current Pharmaceutical Design · doi:10.2174/0113816128282837240130102817 · Cited by 12
Exploiting protein language models for the precise classification of ion channels and ion transporters
Ghazikhani, H., & Butler, G. (2024) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26694 · Cited by 10
Molecular sequence classification using efficient kernel based embedding
Ali, S., Ali, T. E., Murad, T., Mansoor, H., & Patterson, M. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121100 · Cited by 11
TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering
Shen, Y., Chen, Z., Mamalakis, M., Liu, Y., Li, T., Su, Y., He, J., Lió, P., & Wang, Y. G. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822695 · Cited by 10
Self-citation
How do I get the most out of my protein sequence using bioinformatics tools?
Pereira, J., & Alva, V. (2021) · Acta Crystallographica Section D Structural Biology · doi:10.1107/s2059798321007907 · Cited by 10
A framework for predicting variable-length epitopes of human-adapted viruses using machine learning methods
Yin, R., Zhu, X., Zeng, M., Wu, P., Li, M., & Kwoh, C. K. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac281 · Cited by 10
Seq2Mol: Automatic design of de novo molecules conditioned by the target protein sequences through deep neural networksPreprint
Ghanbarpour, A., & Lill, M. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15900 · Cited by 10
Neural Potts ModelPreprint
Sercu, T., Verkuil, R., Meier, J., Amos, B., Lin, Z., Chen, C., Liu, J. J., LeCun, Y., & Rives, A. W. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.04.08.439084 · Cited by 10
Rossmann-toolbox: a deep learning-based protocol for the prediction and design of cofactor specificity in Rossmann-fold proteinsPreprint
Kaminski, K., Ludwiczak, J., Jasiński, M., Bukala, A., Madaj, R., Szczepaniak, K., & Dunin-Horkawicz, S. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.05.05.440912 · Cited by 10
Machine learning driven acceleration of biopharmaceutical formulation development using Excipient Prediction Software (ExPreSo)
Vidal-Henriquez, E., Holder, T., Lee, N. F., Pompe, C., & Teese, M. G. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.10.026 · Cited by 9
Combining evolution and protein language models for an interpretable cancer driver mutation prediction with D2Deep
Tzavella, K., Díaz, A., Olsen, C., & Vranken, W. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae664 · Cited by 9
The genetic foundations of convergent traits
Allard, J. B., & Kumar, S. (2026) · Nature Reviews Genetics · doi:10.1038/s41576-026-00933-7 · Cited by 13
Res-Dom: predicting protein domain boundary from sequence using deep residual network and Bi-LSTM
Wang, L., Zhong, H., Xue, Z., & Wang, Y. (2022) · Bioinformatics Advances · doi:10.1093/bioadv/vbac060 · Cited by 9
Self-citation
An Overview of Protein Function Prediction Methods: A Deep Learning Perspective
Ispano, E., Bianca, F., Lavezzo, E., & Toppo, S. (2023) · Current Bioinformatics · doi:10.2174/1574893618666230505103556 · Cited by 9
Cell type matching across species using protein embeddings and transfer learning
Biharie, K., Michielsen, L., Reinders, M. J. T., & Mahfouz, A. E. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad248 · Cited by 9
PortPred: Exploiting deep learning embeddings of amino acid sequences for the identification of transporter proteins and their substrates
Anteghini, M., dos Santos, V. A. P. M., & Saccenti, E. (2023) · Journal of Cellular Biochemistry · doi:10.1002/jcb.30490 · Cited by 9
POOE: predicting oomycete effectors based on a pre-trained large protein language model
Zhao, M., Lei, C., Zhou, K., Huang, Y., Fu, C., Yang, S., & Zhang, Z. (2023) · mSystems · doi:10.1128/msystems.01004-23 · Cited by 9
Classifying alkaliphilic proteins using embeddings from protein language model
Susanty, M., Mursalim, M. K. N., Hertadi, R., Purwarianti, A., & Mengko, T. L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108385 · Cited by 10
Leveraging protein language model embeddings and logistic regression for efficient and accurate in-silico acidophilic proteins classification
Susanty, M., Mursalim, M. K. N., Hertadi, R., Purwarianti, A., & Rajab, T. L. (2024) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2024.108163 · Cited by 10
How did we get there? AI applications to biological networks and sequences
Anteghini, M., Gualdi, F., & Oliva, B. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110064 · Cited by 9
Predicting Secondary Structure of Protein Using Hybrid of Convolutional Neural Network and Support Vector Machine
Mada, U. G., Sutanto, V. M., Sukma, Z., Division, B. I. D. E., Afiahayati, A., & Mada, U. G. (2020) · International journal of intelligent engineering and systems · doi:10.22266/ijies2021.0228.23 · Cited by 10
LC-PLM: Long-context Protein Language Modeling Using Bidirectional Mamba with Shared Projection LayersPreprint
Wang, Y., Wang, Z., Sadeh, G., Zancato, L., Achille, A., Karypis, G., & Rangwala, H. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.29.620988 · Cited by 12
Self-citation
Intrinsic dataset features drive mutational effect prediction by protein language modelsPreprint
Vieira, L. C., Lin, S., & Wilke, C. O. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.08.710389 · Cited by 12
Scoring alignments by embedding vector similarity
(2024) · doi:10.1093/bib/bbae178 · Cited by 8
DeepPlantAllergy: deep learning for explainable prediction of allergenicity in plant proteins
Dhouib, W., Frikha, F., Rebaï, A., & Kharrat, N. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf605 · Cited by 8
Connecting the Dots in Self-Supervised Learning: A Brief Survey for Beginners
Fang, P., Li, X., Yang, Y., Zhang, S., Kang, Q.-Y., Li, X., & Lan, Z. (2022) · Journal of Computer Science and Technology · doi:10.1007/s11390-022-2158-x · Cited by 9
Exploration of protein sequence embeddings for protein-ligand binding site detection
Hoksza, D., & Gamouh, H. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995025 · Cited by 8
SCLpred-ECL: Subcellular Localization Prediction by Deep N-to-1 Convolutional Neural Networks
Gillani, M., & Pollastri, G. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25105440 · Cited by 8
SAFPred: synteny-aware gene function prediction for bacteria using protein embeddings
Urhan, A., Cosma, B.-M., Earl, A. M., Manson, A. L., & Abeel, T. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae328 · Cited by 8
Computational approaches for identifying neuropeptides: A comprehensive review
Rahmani, R., Kalankesh, L. R., & Ferdousi, R. (2024) · Molecular Therapy — Nucleic Acids · doi:10.1016/j.omtn.2024.102409 · Cited by 8
An explainable few-shot learning model for the directed evolution of antimicrobial peptides
Gao, Q., Ge, L., Wang, Y., Zhu, Y., Liu, Y., Zhang, H., Huang, J., & Qin, Z. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.138272 · Cited by 8
Self-citation
Machine learning approaches for predicting protein-ligand binding sites from sequence data
Vural, O., & Jololian, L. (2025) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2025.1520382 · Cited by 8
Genome language modeling (GLM): a beginner’s cheat sheet
Tyagi, N., Vahab, N., & Tyagi, S. (2025) · Biology Methods and Protocols · doi:10.1093/biomethods/bpaf022 · Cited by 8
The Power of Universal Contextualized Protein Embeddings in Cross-species Protein Function Prediction
van den Bent, I., Makrodimitris, S., & Reinders, M. J. T. (2021) · Evolutionary Bioinformatics · doi:10.1177/11769343211062608 · Cited by 9
OrganelX web server for sub-peroxisomal and sub-mitochondrial protein localization and peroxisomal target signal detection
Anteghini, M., Haja, A., dos Santos, V. A. P. M., Schomaker, L., & Saccenti, E. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.11.058 · Cited by 8
Deep learning enables the design of functional de novo antimicrobial proteinsPreprint
Caceres‐Delpiano, J., Ibañez, R., Alegre, P., Sanhueza, C., Paz-Fiblas, R., Correa, S., Retamal, P. A., Jiménez, J. C., & Álvarez, L. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.26.266940 · Cited by 8
fastMSA: Accelerating Multiple Sequence Alignment with Dense Retrieval on Protein LanguagePreprint
Hong, L., Sun, S., Zheng, L., Tan, Q., & Li, Y. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.12.20.473431 · Cited by 9
Context-Aware Amino Acid Embedding Advances Analysis of TCR-Epitope InteractionsPreprint
Zhang, P., Bang, S., Cai, M., & Lee, H. (2023) · doi:10.7554/elife.88837.1 · Cited by 8
BioStructNet: Structure-Based Network with Transfer Learning for Predicting Biocatalyst Functions
Wang, X., Zhou, J., Mueller, J., Quinn, D., Carvalho, A., Moody, T. S., & Huang, M. (2025) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c01391 · Cited by 7
Seq-RBPPred: Predicting RNA-Binding Proteins from Sequence
Yan, Y., Li, W., Wang, S., & Huang, T. (2024) · ACS Omega · doi:10.1021/acsomega.3c08381 · Cited by 8
Lightweight ProteinUnet2 network for protein secondary structure prediction: a step towards proper evaluation
Stąpor, K., Kotowski, K., Smolarczyk, T., & Roterman, I. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04623-z · Cited by 7
ACP-ESM2: The prediction of anticancer peptides based on pre-trained classifier
Song, H., Lin, X., Zhang, H., & Yin, H. (2024) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2024.108091 · Cited by 7
Advances in the Application of Protein Language Modeling for Nucleic Acid Protein Binding Site Prediction
Wang, B., & Li, W. (2024) · Genes · doi:10.3390/genes15081090 · Cited by 7
Advancing Protein-DNA Binding Site Prediction: Integrating Sequence Models and Machine Learning Classifiers
Murad, T., Chourasia, P., Ali, S., & Patterson, M. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-6960-8_25 · Cited by 7
Predicting binding affinities of emerging variants of SARS-CoV-2 using spike protein sequencing data: observations, caveats and recommendations
Zhang, R., Ghosh, S., & Pal, R. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac128 · Cited by 7
DrugFinder: Druggable Protein Identification Model Based on Pre-Trained Models and Evolutionary Information
Zhang, M., Wan, F., & Liu, T. (2023) · Algorithms · doi:10.3390/a16060263 · Cited by 7
Learning Complete Protein Representation by Deep Coupling of Sequence and StructurePreprint
Hu, B., Tan, C., Xia, J., Zheng, J., Huang, Y., Wu, L., Liu, Y., Xu, Y., & Li, S. Z. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.05.547769 · Cited by 7
Phage Host Prediction Using Deep Neural Network With Multi-Source Protein Language Models and Squeeze-and-Excitation Attention Mechanism
Gao, P., Xu, L., Bai, Y., Lin, Q., Ji, J., & Ma, L. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3582652 · Cited by 6
Large Language Model (LLM)-Based Advances in Prediction of Post-translational Modification Sites in Proteins
Pratyush, P., Pokharel, S., Schulze, S., Bramer, L., Newman, R. H., & KC, D. B. (2025) · Methods in molecular biology · doi:10.1007/978-1-0716-4623-6_19 · Cited by 6
A Comprehensive Review of Transformer-Based Language Models for Protein Sequence Analysis and Design
Ghosh, N., Santoni, D., Nawn, D., Ottaviani, E., & Felici, G. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3631454 · Cited by 7
A sweeter future: Using protein language models for exploring sweeter brazzein homologs
Chua, B. N., Guo, W. M., Wong, H. T., Ow, D. S., Ho, P. L., Koh, W. L. C., Koay, A. Z., & Wong, F. T. (2023) · Food Chemistry · doi:10.1016/j.foodchem.2023.136580 · Cited by 6
Representation Learning of Biological Concepts: A Systematic Review
Yang, Y., Zuo, X., Das, A., Xu, H., & Zheng, W. J. (2023) · Current Bioinformatics · doi:10.2174/1574893618666230612161210 · Cited by 6
ProSol-multi: Protein solubility prediction via amino acids multi-level correlation and discriminative distribution
Ghafoor, H., Asim, M. N., Ibrahim, M. A., & Dengel, A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e36041 · Cited by 6
Elliptic geometry-based kernel matrix for improved biological sequence classification
Ali, S., Shabbir, M., Mansoor, H., Chourasia, P., & Patterson, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112479 · Cited by 6
Trans-MoRFs: A Disordered Protein Predictor Based on the Transformer Architecture
Meng, C., Shi, Y., Fu, X., Zou, Q., & Han, W. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3539710 · Cited by 7
Evaluating sequence and structural similarity metrics for predicting shared paralog functions
Dennler, O., & Ryan, C. J. (2025) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaf051 · Cited by 6
Sequence Representations and Their Utility for Predicting Protein-Protein Interactions
Kimothi, D., Biyani, P., Hogan, J. M., & Davis, M. J. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3137325 · Cited by 6
λ -Scaled-attention: A novel fast attention mechanism for efficient modeling of protein sequences
Ranjan, A., Fahad, M. S., & Deepak, A. (2022) · Information Sciences · doi:10.1016/j.ins.2022.07.127 · Cited by 6
Accelerating functional protein discovery with GPT models: Antimicrobials and enzymes
Zeng, Z., Xu, R.-F., Guo, J., Jin, J., He, H., & Luo, X. (2025) · The Innovation Life · doi:10.59717/j.xinn-life.2025.100133 · Cited by 7
Convolutional ProteinUnetLM competitive with long short‐term memory‐based protein secondary structure predictors
Kotowski, K., Fabian, P., Roterman, I., & Stąpor, K. (2022) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26452 · Cited by 6
EGeRepDR: An enhanced genetic-based representation learning for drug repurposing using multiple biomedical sources
Muniyappan, S., Rayan, A. X. A., & Varrieth, G. T. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104528 · Cited by 6
Direct coupling analysis and the attention mechanism
(2025) · doi:10.1186/s12859-025-06062-y · Cited by 5
Protein interaction pattern recognition using heterogeneous semantics mining and hierarchical graph representation
Bi, X., Ma, W., Jiang, H., Lu, W., Nie, J., Wei, Z., & Zhang, S. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112563 · Cited by 5
Protein embeddings and local alignments
Malec, J., Golding, G. B., & Ilie, L. (2026) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.12.002 · Cited by 5
ImmUQBench: a benchmark on uncertainty quantification of protein immunogenicity prediction
Qayyum, A. B. A., Rahmati, A. H., Qian, X., & Yoon, B.-J. (2026) · Oxford Open Immunology · doi:10.1093/oxfimm/iqag003 · Cited by 5
Contemporary data-driven innovations in peptide-based therapeutic design
Priyadarsinee, L., Kungurtsev, V., Kumar, V., Chatterjee, B., Sastry, G. N., & Murugan, N. A. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag220 · Cited by 5
PseAAC2Vec protein encoding for TCR protein sequence classification
Tayebi, Z., Ali, S., Murad, T., Khan, I., & Patterson, M. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107956 · Cited by 6
ThermoFinder: A sequence-based thermophilic proteins prediction framework
Yu, H., & Luo, X. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.132469 · Cited by 5
SST-ResNet: A Sequence and Structure Information Integration Model for Protein Property Prediction
Zhou, G., Zhao, Y., He, S., & Bo, X. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26062783 · Cited by 5
Explainability of Protein Deep Learning Models
Fazel, Z., de Souza, C. P. E., Golding, G. B., & Ilie, L. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26115255 · Cited by 5
3D genome assisted protein–protein interaction prediction
Guo, Z., Liu, L., Feng, M., Su, K., Chi, R., Li, K., Lü, Q., Su, X., Da, L., Cao, S., et al. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.07.005 · Cited by 5
Circular Arc Length-Based Kernel Matrix For Protein Sequence Classification
Murad, T., Ali, S., Chourasia, P., Mansoor, H., & Patterson, M. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386477 · Cited by 5
Protein sequence analysis in the context of drug repurposing
Sánchez, N. G., Carro, E. U., Santamaría, L. P., & Rodríguez‐González, A. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02531-1 · Cited by 5
Foundation models in molecular biology
Si, Y., Zou, J., Gao, Y., Chuai, G., Liu, Q., & Chen, L. (2024) · Biophysics Reports · doi:10.52601/bpr.2024.240006 · Cited by 6
Ensemble learning model for Protein-Protein interaction prediction with multiple Machine learning techniques
Zhenghui, L., Li, M., Qianyong, C., Yunlong, G., Wang, N., & Guan, L. (2024) · Measurement · doi:10.1016/j.measurement.2024.116001 · Cited by 5
PepCARES: A Comprehensive Advanced Refinement and Evaluation System for Peptide Design and Affinity Screening
Xu, W., Wu, Z., Zhang, C., Zhu, C., & Duan, H. (2024) · ACS Omega · doi:10.1021/acsomega.4c07682 · Cited by 6
A comprehensive review of computational methods for Protein-DNA binding site prediction
Liu, Z., Qiu, W., Liu, Y., He, Y., Pei, W., Zhu, Y.-H., & Qiu, J. (2025) · Analytical Biochemistry · doi:10.1016/j.ab.2025.115862 · Cited by 5
CysPresso: a classification model utilizing deep learning protein representations to predict recombinant expression of cysteine-dense peptides
Ouellet, S., Ferguson, L., Lau, A. Z., & Lim, T. K. Y. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05327-8 · Cited by 5
Prediction of hot spots towards drug discovery by protein sequence embedding with 1D convolutional neural network
Zhang, Y., Yao, S., & Chen, P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0290899 · Cited by 5
Evaluating eUniRep and other protein feature representations for in silico directed evolutionPreprint
Favor, A. H., & Jayapurna, I. (2020) · doi:10.22541/au.159683529.96283070 · Cited by 5
PeTriBERT : Augmenting BERT with tridimensional encoding for inverse protein folding and designPreprint
Dumortier, B., Antoine Liutkus, A., Carré, C., & Krouk, G. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.08.10.503344 · Cited by 6
PhosHSGN: Deep Neural Networks Combining Sequence and Protein Spatial Information to Improve Protein Phosphorylation Site Prediction
Lu, J., Chen, H., & Qiu, J. (2024) · IEEE Access · doi:10.1109/access.2024.3427792 · Cited by 4
SENSE-PPI reconstructs interactomes within, across, and between species at the genome scale
Volzhenin, K., Bittner, L., & Carbone, A. (2024) · iScience · doi:10.1016/j.isci.2024.110371 · Cited by 4
GASIDN: identification of sub-Golgi proteins with multi-scale feature fusion
Sui, J., Chen, J., Chen, Y., Iwamori, N., & Sun, J. (2024) · BMC Genomics · doi:10.1186/s12864-024-10954-3 · Cited by 4
Pre-trained Language Models for Decoding Protein Language: a Survey
Amjad, M., Munir, A., Zia, U., & Paracha, R. Z. (2024) · International Conference on Digital Futures and Transformative Technologies (ICoDT2) · doi:10.1109/icodt262145.2024.10740205 · Cited by 4
Advances in Prediction of Posttranslational Modification Sites Known to Localize in Protein Supersecondary Structures
Pratyush, P., & KC, D. B. (2024) · Methods in molecular biology · doi:10.1007/978-1-0716-4213-9_8 · Cited by 4
PIPENN-EMB ensemble net and protein embeddings generalise protein interface prediction beyond homology
Thomas, D. P. G., Garcia Fernandez, C. M., Haydarlou, R., & Feenstra, K. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-88445-y · Cited by 4
DrugPred: An ensemble learning model based on ESM2 for predicting potential druggable proteins
Zhang, H.-Q., Liu, S.-H., Yu, J.-W., Li, R., Ye, D.-X., Jin, Y.-T., Huang, C.-B., & Deng, K.-J. (2025) · Future generations computer systems · doi:10.1016/j.future.2025.107801 · Cited by 4
A Benchmarking Platform for Assessing Protein Language Models on Function-Related Prediction Tasks
Çevrim, E., Yiğit, M. G., Ulusoy, E., Yılmaz, A., & Doğan, T. (2025) · Methods in molecular biology · doi:10.1007/978-1-0716-4662-5_14 · Cited by 7
Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs
Wu, W., Wang, C., Chen, L., Yin, M., Zhu, Y., Fu, K., Ye, J., Xiong, H., & Wang, Z. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737138 · Cited by 7
Multi-feature enhanced protein language models for accurate protein-RNA binding residue prediction
Feng, Z., Yu, H., Guan, X., Gu, L., Li, K., & Zhou, X. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02065-7 · Cited by 4
SurfSol: A Multimodal Surface-Based Deep Learning Framework for Protein Solubility Prediction
Fang, Y., Wei, L., Xing, B., & Wei, Z. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c00124 · Cited by 4
Compression and k -Mer Based Approach for Anticancer Peptide Analysis
Ali, S., Ali, T. E., Chourasia, P., & Patterson, M. (2026) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2026.3685197 · Cited by 4
Artificial Intelligence-Driven Antimicrobial Peptide Discovery: Prediction, Generation, Mining and Optimization
Lv, J., Liu, G., Zhou, X., & Chen, Y. (2026) · Probiotics and Antimicrobial Proteins · doi:10.1007/s12602-026-11013-x · Cited by 4
Progress and challenges for the application of machine learning for neglected tropical diseases
Khew, C., Akbar, R., & Mohd‐Assaad, N. (2023) · F1000Research · doi:10.12688/f1000research.129064.1 · Cited by 5
A pretrain-finetune approach for improving model generalizability in outcome prediction of acute respiratory distress syndrome patients
Lin, S., Yang, M., Liu, C., Wang, Z., & Long, X. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105397 · Cited by 4
A CNN-CBAM-BIGRU model for protein function prediction
Sharma, L., Deepak, A., Ranjan, A., & Krishnasamy, G. (2024) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2024-0004 · Cited by 4
Functional profiling of the sequence stockpile: a protein pair-based assessment of in silico prediction tools
Prabakaran, R., & Bromberg, Y. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf035 · Cited by 5
TMVisDB: Annotation and 3D-visualization of Transmembrane Proteins
Olenyi, T., Marquet, C., Grekova, A., Houri, L., Heinzinger, M., Dallago, C., & Rost, B. (2025) · Journal of Molecular Biology · doi:10.1016/j.jmb.2025.168997 · Cited by 4
Self-citation

Authors: Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B.
Affiliations: Technical University of Munich (DE)
Leibniz Supercomputing Centre (DE)
Corresponding author: Heinzinger, Michael
ORCID: Heinzinger, Michael
Elnaggar, Ahmed
Wang, Yu
Dallago, Christian
Matthes, Florian
Rost, Burkhard
Venue: BMC Bioinformatics
Publisher: openRxiv
Published: 2019-04-19
Volume: 20
Issue: 1
Pages: 723-723
ISSN: 1471-2105 (ISSN-L)
DOI: 10.1101/614313
PMC: PMC6918593
OpenAlex: W2995514860
Cited by: 606
References: 105
Citations by year: 2026: 53 · 2025: 103 · 2024: 115 · 2023: 117 · 2022: 99 · 2021: 83 · 2020: 29 · 2019: 7
FWCI: 18.03
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
Locations: BMC Bioinformatics (publishedVersion, cc-by)
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
Columbia Academic Commons (Columbia University)
Funders: Alexander von Humboldt-Stiftung
Nvidia
Deutsche Forschungsgemeinschaft (DFG-GZ: RO1320/4-1, RO1320/4‐1)
Bundesministerium für Bildung und Forschung
Leibniz-Gemeinschaft
Topics: Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
Keywords: protein sequence embeddings, ELMo, transfer learning, protein function prediction, protein structure prediction, secondary structure prediction, subcellular localization prediction, membrane proteins, dark proteome, metaproteomics, microbiome analysis
MeSH terms: Machine Learning, Amino Acid Sequence, Natural Language Processing, Proteins, Neural Networks, Computer, Sequence Analysis, Computational Biology, Databases, Nucleic Acid, Databases, Protein, Proteomics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Modeling the language of life – Deep Learning Protein Sequences

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks