PreprintCited by 606Machine Learning in BioinformaticsProtein Structure and DynamicsFractal and DNA sequence analysis

The DOI matches its Crossref record; 103 of the 105 references it cites were found in a registry or an index.

Modeling the language of life – Deep Learning Protein Sequences

Open access

Heinzinger, Michael, Elnaggar, Ahmed, Wang, Yu, Dallago, Christian, Nechaev, Dmitrii, Matthes, Florian, Rost, Burkhard·2019·BMC Bioinformatics

DOI 10.1101/614313PMC PMC6918593

Preprint — published as 10.1186/s12859-019-3220-8 · Paper page

Open this paper in CoChat — free account Open PDF

Abstract

Background: One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results: We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion: We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability: SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1101/614313

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
The Protein Data Bank
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Untitled work
(2014) · Cited by 33,419
Untitled work
(2008) · Journal of Machine Learning Research · Cited by 33,051
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568

Flagged first, then by citations

Show all 105 references

2 of the 105 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

10.1007/s10479-011-0841-3. · DOI

Sorted by citations

Items 501–599 of 599

NCSP-PLM: An ensemble learning framework for predicting non-classical secreted proteins based on protein language models and deep learning
Liu, T., Song, C., & Wang, C. (2023) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2024063 · Cited by 1
WITHDRAWN: ToxVec: Deep Language Model-Based Representation Learning for Venom Peptide ClassificationPreprint
Ahmadi, M., Jahed‐Motlagh, M. R., Asgari, E., Rahmani, A. T., & McHardy, A. C. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.29.319046 · Cited by 1
In silico protein science: from sequence to network
Marrero, M. C. (2021) · doi:10.18174/547092 · Cited by 1
Predicting SARS-CoV-2 epitope-specific TCR recognition using pre-trained protein embeddingsPreprint
Han, Y., & Lee, A. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.17.468929 · Cited by 1
P-PPI: accurate prediction of peroxisomal protein-protein interactions (P-PPI) using deep learning-based protein sequence embeddingsPreprint
Anteghini, M., dos Santos, V. A. P. M., & Saccenti, E. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.06.30.547177 · Cited by 1
Opinion piece/Book chapter: Artificial Intelligence learns protein predictionPreprint
Heinzinger, M., & Rost, B. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/n7wfp · Cited by 1
Self-citation
Context-Aware Amino Acid Embedding Advances Analysis of TCR-Epitope InteractionsPreprint
Zhang, P., Bang, S., Cai, M., & Lee, H. (2023) · eLife · doi:10.7554/elife.88837 · Cited by 1
Computational modeling of binding site for RNA-ligand complex by learning Multi-modal featuresPreprint
Wang, J., Quan, L., jin, Z., Wu, H., Ma, X., Wang, X., Xie, J., Pan, D., Chen, T., Wu, T., et al. (2023) · doi:10.36227/techrxiv.24475075 · Cited by 1
TooT-PLM-P2S: Incorporating Secondary Structure Information into Protein Language ModelsPreprint
Ghazikhani, H., & Butler, G. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.13.607781 · Cited by 1
Rational Design Inspired Application of Natural Language Processing Algorithms to Red Shift mNeptune684
Parkinson, S. (2021) · uO Research (University of Ottawa) · doi:10.20381/ruor-26150
Computational approaches to discover novel enzymes for fragrance and flavour
Durairaj, J. (2021) · doi:10.18174/551031
Peer Review #1 of "Navigating the amino acid sequence space between functional proteins using a deep learning framework (v0.1)"
(2021) · PeerJ Computer Science · doi:10.7287/peerj-cs.684v0.1/reviews/1
Protein matchmaking through representation learning
Heinzinger, M., Dallago, C., & Rost, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.09.007
Self-citation
Learning representations for molecular sequences
Kimothi, D. (2021) · Queensland University of Technology · doi:10.5204/thesis.eprints.226106
Peer Review #2 of "Navigating the amino acid sequence space between functional proteins using a deep learning framework (v0.1)"
(2021) · PeerJ Computer Science · doi:10.7287/peerj-cs.684v0.1/reviews/2
DNAVec: Pre-Trained Word Vector Representation of Genomic DNA Sequences
郎, 梅. (2021) · Hans Journal of Biomedicine · doi:10.12677/hjbm.2021.113016
Faculty Opinions recommendation of Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.
Koehl, P. (2021) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.739876259.793585293
Faculty Opinions recommendation of Highly accurate protein structure prediction with AlphaFold.
Arulanandam, A. J. (2021) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.740477161.793587548
Faculty Opinions recommendation of Highly accurate protein structure prediction with AlphaFold.
Chakraborty, S. (2021) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.740477161.793587538
SeqVec-GAT: A Golgi Classification Model Based on Multi-headed Graph Attention Network
Sui, J., Chen, Y., Chen, B., Cao, Y., Chen, J., & Cong, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13829-4_61
Discriminating Ion Channels from Non-Ion Channel Membrane Proteins Using Machine Learning and Deep Learning Classifiers with Protein Language Model RepresentationsPreprint
Ghazikhani, H., & Butler, G. (2023) · Preprints.org · doi:10.20944/preprints202301.0221.v1
PLANNING THE JOINT TRAJECTORY OF REDUNDANT KINEMATICS CHAIN
YANKOV, K. B. (2022) · Trakia Journal of Sciences · doi:10.15547/tjs.2022.03.011
Genome-scale enzymatic reaction prediction by variational graph autoencodersPreprint
Wang, C., Yuan, C., Wang, Y., Chen, R., Shi, Y., Patti, G. J., & Hou, Q. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.03.08.531729
Self-citation
Computational Approaches for Peroxisomal Protein Localization
Anteghini, M., & dos Santos, V. A. P. M. (2023) · Methods in molecular biology · doi:10.1007/978-1-0716-3048-8_29
TAPE-Pero: Using Deep Representation Learning Model to Identify and Localize Peroxisomal Proteins
Sui, J., Chen, Y., Cao, Y., & Zhao, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-4749-2_48
Identification of Active and Binding Sites with Multi-dimensional Feature Vectors and K-Nearest Neighbor Classification Algorithm
Zhang, B., Wang, Z., Bao, W., & Cheng, H. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-4749-2_51
Hist2Vec: Kernel-Based Embeddings for Biological Sequence Classification
Ali, S., Mansoor, H., Chourasia, P., & Patterson, M. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7074-2_30
Computational modeling of binding site for RNA-ligand complex by learning Multi-modal featuresPreprint
Wang, J., Quan, L., Jin, Z. X., Wu, H., Ma, X., Wang, X., Xie, J., Pan, D., Chen, T., Wu, T., et al. (2023) · doi:10.36227/techrxiv.24475075.v1
T Cell Receptor Protein Sequences and Sparse Coding: A Novel Approach to Cancer Classification
Tayebi, Z., Ali, S., Chourasia, P., Murad, T., & Patterson, M. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8141-0_17
Rendering protein mutation movies with MutAmore
Weissenow, K., & Rost, B. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05610-8
Self-citation
Deep-USIpred: identifying substrates of ubiquitin protein ligases E3 and deubiquitinases with pretrained protein embedding and bayesian neural network
Wang, J., Pan, G.-Q., Li, J., Shang, X., You, Z., & Huang, Y. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385619
Enhance the recognition of signal peptide domain and transmembrane domain based on transformer
Xu, J., Yang, F., Zhang, Z., & Zhu, S. (2023) · International Conference on Health Big Data and Intelligent Healthcare (ICHIH) · doi:10.1109/ichih60370.2023.10396285
A Protein Structure Enhanced Pre-training Model for Allosteric Site Detection
Lv, Q., Zhang, X., Jiang, X., Chen, M., Cao, Y., & He, W.-X. (2023) · ICAICE · doi:10.1145/3652628.3652822
Ensemble Learning Model Based on Convolutional Neural Network And Bi-Directional Long Short-Term Memory with Multi-Head Attention for Protein-Protein Interactions PredictionPreprint
Zhenghui, L., Li, M., Qianyong, C., Yunlong, G., Nan, W., & Guan, L. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4868685
Protein Design
Ramya, L., Ragunathan, P. T., & Yennamalli, R. M. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95502-7.00103-2
Prediction of Protein Localization
Nakai, K., Imai, K., & Wei, L. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95502-7.00042-7
Boosting Protein Language Models with Negative Sample Mining
Xu, Y., Zhao, X., Song, X., Wang, B., & Yu, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70381-2_13
Method for Recognizing the Characteristic Elements of Protein Secondary Structure From the Llm of Its Amino Acid Sequence
Teplyi, Y. (2024) · Vìsnik Nacìonalʹnogo unìversitetu Lʹvìvsʹka polìtehnìka Serìâ Ìnformacìjnì sistemi ta merežì · doi:10.23939/sisn2024.15.460
Transformer models for protein-guided drug compound generation: A comparison of amino acid sequences, pre-trained protein embeddings, SMILES, and SELFIES
Fossl, D. (2024) · doi:10.24124/2024/59584
Advances in Computational Pipelines and Workflows in Bioinformatics
Chandra, A. A., López, Y., Dehzangi, A., Shatabda, S., Sattar, A., Kamola, P. J., Sharma, R., Shigemizu, D., Tsunoda, T., & Sharma, A. K. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95502-7.00283-9
Prediction of Protein Secondary Structure Using a Hybrid Convolutional Blocks with GRU Units
Yang, S., & Chen, X. (2024) · Frontiers in Computing and Intelligent Systems · doi:10.54097/k8rmxp27
PPI-CoAttNet: A Web Server for Protein–Protein Interaction Tasks Using a Coattention Model
Bian, Q., Shen, Z., Gao, J., Shen, L., Lu, Y., Zhang, Q., Chen, R., Xu, D., Liu, T., Che, J., et al. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01365
Molecular structure and role of microbial proteins: a delimitation employing bioinformatics techniques
Saha, D., Borah, N. J., & Mamud, A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-31550-3.00014-7
Hash-binding: DNA-protein binding prediction using hashing-based embedding
Ali, S., Ali, T. E., & Patterson, M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11349-5
ClusterEmbed: Efficient Protein Structure Prediction with Clustering and Embeddings
Mu, B. (2025) · BIO Web of Conferences · doi:10.1051/bioconf/202518202013
ProtSEC: Ultrafast Protein Sequence Embedding in Complex Space Using Fast Fourier TransformPreprint
Raju, R. S., & Islam, R. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.17.670693
MicroEnvPPI: Microenvironment-Aware Optimization Enables Generalizable Protein–Protein Interaction Prediction
Yang, K., Chen, Y., Wei, Y., Xiang, M., Zhuo, L., Fu, X., Cao, D., & Zhang, W. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01907
Identification of Protein–Protein Interaction (PPI) Sites on the Influenza A (H1N1) Viral Genome Using Gradient Boosting and Artificial Neural Network (ANN) Models
Sajnani, V., Ali, O., Das, S., & Bandyopadhyay, D. (2025) · ACS Omega · doi:10.1021/acsomega.5c06314
Tamavaq™: A Hybrid Quantum–Classical Grover Pipeline for Precision Neoantigen Vaccination in GliomaPreprint
Grigoriadis, I., & Emmanouilides, C. E. (2025) · Research Square · doi:10.21203/rs.3.rs-7789121/v1
A Comprehensive Survey on Protein Function Prediction
Shukla, V., Pradhan, R., Sharma, D. K., & Nagaty, K. A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-9242-2_35
Additional file 1 of FoldHSphere: deep hyperspherical embeddings for protein fold recognition
Villegas-Morcillo, A., Sánchez, V., & Gomez, A. M. (2021) · Figshare · doi:10.6084/m9.figshare.16800161
Additional file 4 of SonicParanoid2: fast, accurate, and comprehensive orthology inference with machine learning and language models
Cosentino, S., Sriswasdi, S., & Iwasaki, W. (2024) · Figshare · doi:10.6084/m9.figshare.26753746
Additional file 1 of Organizing the bacterial annotation space with amino acid sequence embeddings
Grigson, S. R., McKerral, J. C., Mitchell, J. G., & Edwards, R. A. (2022) · Figshare · doi:10.6084/m9.figshare.21200740
Additional file 1 of Rendering protein mutation movies with MutAmore
Weißenow, K., & Rost, B. (2023) · Figshare · doi:10.6084/m9.figshare.24797848
Self-citation
Additional file 1 of Lightweight ProteinUnet2 network for protein secondary structure prediction: a step towards proper evaluation
Stąpor, K., Kotowski, K., Smolarczyk, T., & Roterman, I. (2022) · Figshare · doi:10.6084/m9.figshare.19402005
Additional file 1 of SonicParanoid2: fast, accurate, and comprehensive orthology inference with machine learning and language models
Cosentino, S., Sriswasdi, S., & Iwasaki, W. (2024) · Figshare · doi:10.6084/m9.figshare.26378001
Additional file 1 of ProtPlat: an efficient pre-training platform for protein classification based on FastText
Jin, Y., & Yang, Y. (2022) · Figshare · doi:10.6084/m9.figshare.19164572
Additional file 3 of SonicParanoid2: fast, accurate, and comprehensive orthology inference with machine learning and language models
Cosentino, S., Sriswasdi, S., & Iwasaki, W. (2024) · Figshare · doi:10.6084/m9.figshare.26753743
Additional file 2 of SonicParanoid2: fast, accurate, and comprehensive orthology inference with machine learning and language models
Cosentino, S., Sriswasdi, S., & Iwasaki, W. (2024) · Figshare · doi:10.6084/m9.figshare.26378136
Additional file 1 of How to approach machine learning-based prediction of drug/compound–target interactions
Ataş, H., & Doğan, T. (2024) · Figshare · doi:10.6084/m9.figshare.26566562
Additional file 1 of Protein sequence analysis in the context of drug repurposing
Sánchez, N. G., Carro, E. U., Prieto-Santamaría, L., & Rodríguez‐González, A. (2024) · Figshare · doi:10.6084/m9.figshare.26719884
Additional file 1 of Structure-aware protein solubility prediction from sequence through graph convolutional network and predicted contact map
Chen, J., Zheng, S., Zhao, H., & Yang, Y. (2021) · Figshare · doi:10.6084/m9.figshare.13788154
Additional file 1 of TMbed: transmembrane proteins predicted through language model embeddings
Bernhofer, M., & Rost, B. (2022) · Figshare · doi:10.6084/m9.figshare.20454488
Self-citation
Additional file 1 of CysPresso: a classification model utilizing deep learning protein representations to predict recombinant expression of cysteine-dense peptides
Ouellet, S., Ferguson, L., Lau, A. Z., & Lim, T. K. Y. (2024) · Figshare · doi:10.6084/m9.figshare.26589918
AI-Driven Insights into Protein Sequence Analysis Landscape
Asim, M. N., Ahmed, S., & Dengel, A. (2025) · Springer eBooks · doi:10.1007/978-3-031-90454-7_5
Design of Protein and Peptide Macromolecular Drugs
Li, H., & Li, S. (2025) · Springer eBooks · doi:10.1007/978-981-95-2525-6_45
CurvePotGCN – A graph neural network to predict protein–protein interactions using surface curvature and electrostatic potential as node-features
Kadam, V., Bhargav, P., & Mukherjee, A. (2026) · Journal of Chemical Sciences · doi:10.1007/s12039-025-02459-7
Protein-Protein Interaction Prediction Using Graph Neural Networks
Boumya, O., Hraiche, H., & Karboub, K. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-14964-0_27
IALA-LNN: Deep Learning for Peptide Retention Time Prediction Based on Improved Artificial Lemming Algorithm-Optimized Liquid Neural Networks
Chen, Y., Zhu, L., & Yang, S. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c02548
PCTPS: Accurate prediction of phase-separated proteins using protein language model embeddings and a CTN-KAN framework
Liu, T., Xia, Y., Sun, X., Song, S., Hu, X., & Zhang, L. (2026) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2026.109383
DualBAN: Unifying Intra- and Inter-Molecular Features for Compound-Protein Interaction Prediction
He, S., He, Z., Zhang, Y., & Ye, W. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3678303
Improved Interaction Prediction in PPI Networks Using Graph Neural Networks
Kumar, V., & Saikia, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-23241-0_14
Simple baselines rival protein language models in dense combinatorial protein landscapesPreprint
Talpir, I., & Fleishman, S. J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.01.722313 · Cited by 2
A Cross-Modal Hierarchical Contrastive Learning Framework for Protein-Protein Interaction Prediction
Zhang, R., Wang, Y., Wang, X., Long, Q., Zhao, J., & Xiao, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0369-7_2
Self-citation
RIMGOGraph: integrating AlphaFold-derived residue interaction graphs and protein language embeddings for structure-informed protein function prediction
Chang, T., Le, N. Q. K., & Chua, M. C. H. (2026) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2026.152509
ProtSATT: An Advanced Protein Solubility Predictor Based on Attention Mechanism
Deng, W., Chen, Z., Ji, C., Gao, J., Xu, H., & Huang, J. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c00821
Learning Protein-Protein Interaction Interfaces from Multimodal RepresentationsPreprint
Gardinazzi, Y., Garcia, E. N. V., Senci, S., Vora, D. D., Feltrin, A., & Cuturello, F. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.29.728739
Pseudoperplexity Probes Memorization in Protein Language ModelsPreprint
Alexander, P., PLONER, M., Sewald, Z., Senoner, T., Franz, S., Brenner, M., Heinzinger, M., & Rost, B. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.08.730685
Self-citation
An Overview and Benchmark Evaluation of RNA Representation Learning Methods
Zheng, Y., Gao, Y., Wang, F., & Yang, Y. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-4897-2_15
GatedGeoGO:Multi-Modal Geometry-Aware Network with Gated Fusion and GO Semantic Attention for Protein Function Prediction
Dong, M., Niu, D., Peng, Y., Li, H., Li, M., Wei, Z., & Li, Z. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c00882
Analyze the Sequence–Function Relationship of Umami Peptides in Shrimp Paste through Machine Learning
Sun, H.-T., Liang, Y., Wang, T., Zhao, X. L., & Wang, H. (2026) · Food Biophysics · doi:10.1007/s11483-026-10182-0
Identifying RNA ac4C Modification Sites via Pseudo-Nucleotide Fingerprint Encoding and Multi-Scale Feature Integration
Wang, Y., Mo, F., Sha, Y., Wen, Z., & Fang, C. (2026) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-026-00853-4
Self-citation
NeuroPpred-PSCG: A Multimodal Framework Using ProtT5 and Structural Features for Neuropeptide Prediction Based on Gated Cross-Attention Mechanism and Multiscale Gated Convolution
Zhang, S., & Zhang, X. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c01123
PLMGraph-Ace: Species-specific lysine acetylation-site prediction via fusion of a protein language model and a sequence-distance graph
Luo, W., Liu, Z., Wu, H., Xiao, X., Lin, W., & Qiu, W. (2026) · Current Genomics · doi:10.1016/j.curgen.2026.07.002
Cross-linking Mass Spectrometry-Constrained Protein Sequence Representation and Structural Consistency Validation Based on ProtFlash-XL
Shan, Y., & Wang, Y. (2026) · IEEE International Conference on Pattern Recognition, Machine Vision and Artificial Intelligence (PRMVAI) · doi:10.1109/prmvai70103.2026.11605881
Self-citation
Dual Cross-Attention Network for Hierarchical Feature Fusion in Protein-Protein Interaction Prediction
Lu, S., Li, Y., Tian, Z., Nan, X., Zhou, Q., & Zhang, S. (2026) · Computational and Structural Biotechnology Journal · doi:10.34133/csbj.0200
High-resolution dissection of concept acquisition in different families of protein language modelsPreprint
Whitfield, S., Marty, T., Vernon, R. M., LANGMEAD, C. J., Sridhar, D., & Fournier, Q. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.20.739599
Evaluating Protein Language Model Embeddings for Structural Similarity in the Protein-Sequence Twilight ZonePreprint
Danwada, S., Udomprasert, P., Raychawdhary, N., Seals, C. D., Wu, L., & Bhattacharya, S. (2026) · bioRxiv · doi:10.64898/2026.07.31.742099
CAHS: Context-Aware Homology Search
Bhattaram, S., Bhowmik, D., & Kannan, R. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3819006
Transformers in protein informatics: A survey of structure prediction, function annotation, interactions, and drug discovery
Zhang, Y., Ling, X., Li, Z., Song, L., Jia, Y., Wang, Y., & You, Z. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115976
Self-citation
EIHR-PROT: explicit modelling of homology reliability for protein function prediction
Jonathan, O., & Uwidia, O. E. (2026) · BMC Bioinformatics · doi:10.1186/s12859-026-06722-7
Prot-LAMBDA: Explicit Distance Learning Enhances Structural Reasoning in Protein Language ModelsPreprint
Ibtehaz, N., Zhang, Z., Kagaya, Y., Xu, M., Tomii, K., & Kihara, D. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.23.746565
Comprehensive Evaluation of Protein Language Model Embeddings for Drug–Target Affinity PredictionPreprint
Marijan, M., & Tanasijević, I. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.31.748056
Deep learning for protein folding
Lahiri, T., & Mondal, R. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40338-5.00015-9
Developing SCL2205 : a protein sequence-based spatial modelling dataset for the protein language model frontier
Ouso, D. O., & Pollastri, G. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag666
Language-Model-Based Detection of Genetic Editing in BacteriaPreprint
Gabay, E., & Burstein, D. E. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.17.751846
AdventML: Advanced Enzyme Temperature Prediction with Transformer-Based Embeddings and Resampling Strategies
François, J., Moor, B. D., & van Noort, V. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c02109
FAIRsharing record for: AlphaFold Protein Structure Database
FAIRsharing Team, & Liang, Y. (2025) · FAIRsharing.org · doi:10.25504/fairsharing.246086
FAIRsharing record for: Rhea
FAIRsharing Team, & Morgat, A. (2015) · FAIRsharing.org · doi:10.25504/fairsharing.pn1sr5

Authors: Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B.
Affiliations: Technical University of Munich (DE)
Leibniz Supercomputing Centre (DE)
Corresponding author: Heinzinger, Michael
ORCID: Heinzinger, Michael
Elnaggar, Ahmed
Wang, Yu
Dallago, Christian
Matthes, Florian
Rost, Burkhard
Venue: BMC Bioinformatics
Publisher: openRxiv
Published: 2019-04-19
Volume: 20
Issue: 1
Pages: 723-723
ISSN: 1471-2105 (ISSN-L)
DOI: 10.1101/614313
PMC: PMC6918593
OpenAlex: W2995514860
Cited by: 606
References: 105
Citations by year: 2026: 53 · 2025: 103 · 2024: 115 · 2023: 117 · 2022: 99 · 2021: 83 · 2020: 29 · 2019: 7
FWCI: 18.03
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
Locations: BMC Bioinformatics (publishedVersion, cc-by)
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
Columbia Academic Commons (Columbia University)
Funders: Alexander von Humboldt-Stiftung
Nvidia
Deutsche Forschungsgemeinschaft (DFG-GZ: RO1320/4-1, RO1320/4‐1)
Bundesministerium für Bildung und Forschung
Leibniz-Gemeinschaft
Topics: Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
Keywords: protein sequence embeddings, ELMo, transfer learning, protein function prediction, protein structure prediction, secondary structure prediction, subcellular localization prediction, membrane proteins, dark proteome, metaproteomics, microbiome analysis
MeSH terms: Machine Learning, Amino Acid Sequence, Natural Language Processing, Proteins, Neural Networks, Computer, Sequence Analysis, Computational Biology, Databases, Nucleic Acid, Databases, Protein, Proteomics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Modeling the language of life – Deep Learning Protein Sequences

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks