The DOI matches its Crossref record; 103 of the 105 references it cites were found in a registry or an index.
Modeling the language of life – Deep Learning Protein Sequences
Open accessPreprint — published as 10.1186/s12859-019-3220-8 · Paper page
Abstract
Background: One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results: We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion: We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability: SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2014) · Cited by 33,419
(2008) · Journal of Machine Learning Research · Cited by 33,051
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Flagged first, then by citations
2 of the 105 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- 10.1007/s10479-011-0841-3. · DOI
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329
Self-citationProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009
Simulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937
Evaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719
Single-sequence protein structure prediction using language models from deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., & AlQuraishi, M. (2021) · doi:10.1101/2021.08.02.454840 · Cited by 455
“Very large Transformer-based models trained on hundreds of millions and potentially billions of protein sequences are increasingly available 26,35,36 and the scaling previously observed in natural language applications 44 makes it likely that the performance of RGN2 and similar methods will…”
BackgroundSingle-sequence protein structure prediction using a language model and deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01432-w · Cited by 455
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 424
Self-citationRao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 365
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341
Bilingual language model for protein sequence and structure
Heinzinger, M., Weissenow, K., Sanchez, J. G., Henkel, A., Mirdita, M., Steinegger, M., & Rost, B. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae150 · Cited by 311
Self-citationElnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 364
Self-citationMachine learning for functional protein design
Notin, P., Rollins, N. J., Gal, Y., Sander, C., & Marks, D. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 304
Prediction of protein–protein interaction using graph neural networks
Jha, K., Saha, S., & Singh, H. (2022) · Scientific Reports · doi:10.1038/s41598-022-12201-9 · Cited by 254
Rhea, the reaction knowledgebase in 2022
Bansal, P., Morgat, A., Axelsen, K. B., Muthukrishnan, V., Coudert, E., Aimo, L., Hyka‐Nouspikel, N., Gasteiger, E., Kerhornou, A., Neto, T. B., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab1016 · Cited by 251
PredictProtein - Predicting Protein Structure and Function for 29 Years
Bernhofer, M., Dallago, C., Karl, T., Satagopam, V., Heinzinger, M., Littmann, M., Olenyi, T., Qiu, J., Schütze, K., Yachdav, G., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab354 · Cited by 246
Self-citationFine-tuning protein language models boosts predictions across diverse tasks
Schmirler, R., Heinzinger, M., & Rost, B. (2024) · Nature Communications · doi:10.1038/s41467-024-51844-2 · Cited by 229
Self-citationDeep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227
(2023) · doi:10.1101/2023.12.07.570727 · Cited by 207
Singh, R., Sledzieski, S., Bryson, B. D., Cowen, L., & Berger, B. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2220778120 · Cited by 216
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2023) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 238
Ankh ☥: Optimized Protein Language Model Unlocks General-Purpose ModellingPreprint
Elnaggar, A., Essam, H., Salah-Eldin, W., Moustafa, W., Elkerdawy, M., Rochereau, C., & Rost, B. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.16.524265 · Cited by 243
Self-citationLearning functional properties of proteins with language models
Ünsal, S., Ataş, H., Albayrak, M., Turhan, K., Acar, A. C., & Doğan, T. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00457-9 · Cited by 206
Transformer-based deep learning for predicting protein properties in the life sciences
Chandra, A. A., Tünnermann, L., Löfstedt, T., & Gratz, R. (2023) · eLife · doi:10.7554/elife.82819 · Cited by 202
Le, N. Q. K., Ho, Q., Nguyen, T., & Ou, Y. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab005 · Cited by 189
Language models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181
Machine Learning Applied to Predicting Microorganism Growth Temperatures and Enzyme Catalytic Optima
Li, G., Rabe, K. S., Nielsen, J., & Engqvist, M. K. M. (2019) · ACS Synthetic Biology · doi:10.1021/acssynbio.9b00099 · Cited by 185
Applications of transformer-based language models in bioinformatics: a survey
Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023) · Bioinformatics Advances · doi:10.1093/bioadv/vbad001 · Cited by 176
Embeddings from protein language models predict conservation and variant effects
(2021) · doi:10.1007/s00439-021-02411-y · Cited by 174
Self-citationEmbeddings from deep learning transfer GO annotations beyond homology
Littmann, M., Heinzinger, M., Dallago, C., Olenyi, T., & Rost, B. (2021) · Scientific Reports · doi:10.1038/s41598-020-80786-0 · Cited by 176
Self-citationDELPHI: accurate deep ensemble model for protein interaction sites prediction
Li, Y., Golding, G. B., & Ilie, L. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa750 · Cited by 169
Yang, X., Yang, S., Li, Q., Wuchty, S., & Zhang, Z. (2019) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2019.12.005 · Cited by 164
Learning meaningful representations of protein sequences
Detlefsen, N. S., Hauberg, S., & Boomsma, W. (2022) · Nature Communications · doi:10.1038/s41467-022-29443-w · Cited by 171
RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks
Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., & Šikić, M. (2025) · Nature Communications · doi:10.1038/s41467-025-60872-5 · Cited by 169
Artificial intelligence in early drug discovery enabling precision medicine
Boniolo, F., Dorigatti, E., Ohnmacht, A. J., Saur, D., Schubert, B., & Menden, M. P. (2021) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2021.1918096 · Cited by 147
Protein remote homology detection and structural alignment using deep learning
Hamamsy, T., Morton, J. T., Blackwell, R., Berenberg, D., Carriero, N. J., Gligorijević, V., Strauss, C. E. M., Leman, J. K., Cho, K., & Bonneau, R. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01917-2 · Cited by 157
Akiyama, M., & Sakakibara, Y. (2022) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqac012 · Cited by 148
(2022) · doi:10.1016/j.str.2022.05.001 · Cited by 139
Self-citationLearning spatial structures of proteins improves protein–protein interaction prediction
Song, B., Luo, X., Luo, X., Liu, Y., Niu, Z., & Zeng, X. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab558 · Cited by 138
Unifying the known and unknown microbial coding sequence space
Vanni, C., Schechter, M. S., Acinas, S. G., Barberán, A., Buttigieg, P. L., Casamayor, E. O., Delmont, T. O., Duarte, C. M., Eren, A. M., Finn, R. D., et al. (2022) · eLife · doi:10.7554/elife.67667 · Cited by 134
Yan, J., Cai, J., Zhang, B., Wang, Y., Wong, D. F., & Siu, S. W. I. (2022) · Antibiotics · doi:10.3390/antibiotics11101451 · Cited by 138
Self-citationAkbar, S., Hayat, M., Tahir, M., Khan, S. A., & Alarfaj, F. K. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102349 · Cited by 131
Light attention predicts protein location from the language of life
Stärk, H., Dallago, C., Heinzinger, M., & Rost, B. (2021) · Bioinformatics Advances · doi:10.1093/bioadv/vbab035 · Cited by 130
Self-citationFLIP: Benchmark tasks in fitness landscape inference for proteinsPreprint
Dallago, C., Mou, J., Johnston, K. E., Wittmann, B. J., Bhattacharya, N., Goldman, S., Madani, A., & Yang, K. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.09.467890 · Cited by 142
Self-citationFrom sequence to function through structure: Deep learning for protein design
(2022) · doi:10.1016/j.csbj.2022.11.014 · Cited by 124
Self-citationQiu, J., Bernhofer, M., Heinzinger, M., Kemper, S., Norambuena, T., Melo, F., & Rost, B. (2020) · Journal of Molecular Biology · doi:10.1016/j.jmb.2020.02.026 · Cited by 123
Self-citationWang, D., Feng, L., Ye, J., Zou, J., & Zheng, Y. (2023) · MedComm – Future Medicine · doi:10.1002/mef2.43 · Cited by 127
Machine learning modeling of family wide enzyme-substrate specificity screens
Goldman, S., Das, R., Yang, K. K., & Coley, C. W. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009853 · Cited by 120
Villegas-Morcillo, A., Makrodimitris, S., van Ham, R. C. H. J., Gomez, A. M., Sanchez, V., & Reinders, M. J. T. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa701 · Cited by 122
Chen, J., Zheng, S., Zhao, H., & Yang, Y. (2021) · Journal of Cheminformatics · doi:10.1186/s13321-021-00488-1 · Cited by 121
Raza, A., Uddin, J., Almuhaimeed, A., Akbar, S., Zou, Q., & Ahmad, A. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01563 · Cited by 117
A new age in protein design empowered by deep learning
Khakzad, H., Igashov, I., Schneuing, A., Goverde, C. A., Bronstein, M. M., & Correia, B. E. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.006 · Cited by 121
Akbar, S., Zou, Q., Raza, A., & Alarfaj, F. K. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102860 · Cited by 111
Machine learning to navigate fitness landscapes for protein engineering
Freschlin, C. R., Fahlberg, S. A., & Romero, P. A. (2022) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2022.102713 · Cited by 111
TMbed: transmembrane proteins predicted through language model embeddings
Bernhofer, M., & Rost, B. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04873-x · Cited by 109
Self-citationContrastive learning on protein embeddings enlightens midnight zone
(2022) · doi:10.1093/nargab/lqac043 · Cited by 105
Self-citationTaxonomic classification of DNA sequences beyond sequence similarity using deep neural networks
Mock, F., Kretschmer, F., Kriese, A., Böcker, S., & Marz, M. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2122636119 · Cited by 111
Learned Embeddings from Deep Learning to Visualize and Predict Protein Sets
Dallago, C., Schütze, K., Heinzinger, M., Olenyi, T., Littmann, M., Lu, A. X., Yang, K., Min, S., Yoon, S., Morton, J. T., et al. (2021) · Current Protocols · doi:10.1002/cpz1.113 · Cited by 109
Self-citationSignalP: The Evolution of a Web Server
Nielsen, H., Teufel, F., Brunak, S., & von HEIJNE, G. (2024) · Methods in molecular biology · doi:10.1007/978-1-0716-4007-4_17 · Cited by 107
EpiDope: a deep neural network for linear B-cell epitope prediction
Collatz, M., Mock, F., Barth, E., Hölzer, M., Sachse, K., & Marz, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa773 · Cited by 99
Transforming the Language of Life
Nambiar, A., Heflin, M., Liu, S., Maslov, S., Hopkins, M., & Ritz, A. (2020) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3388440.3412467 · Cited by 104
The Large Language Models on Biomedical Data Analysis: A Survey
Lan, W., Tang, Z., Liu, M., Chen, Q., Peng, W., Chen, Y. P., & Pan, Y. F. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3530794 · Cited by 100
Large language models improve annotation of prokaryotic viral proteins
Flamholz, Z. N., Biller, S. J., & Kelly, L. (2024) · Nature Microbiology · doi:10.1038/s41564-023-01584-8 · Cited by 92
Machine learning on protein–protein interaction prediction: models, challenges and trends
Tang, T., Zhang, X., Liu, Y., Peng, H., Zheng, B., Yin, Y., & Zeng, X. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad076 · Cited by 95
Le, N. Q. K., & Ho, Q. (2021) · Methods · doi:10.1016/j.ymeth.2021.12.004 · Cited by 86
Improving protein succinylation sites prediction using embeddings from protein language model
Pokharel, S., Pratyush, P., Heinzinger, M., Newman, R. H., & KC, D. B. (2022) · Scientific Reports · doi:10.1038/s41598-022-21366-2 · Cited by 90
Self-citationLindorff-Larsen, K., & Kragelund, B. B. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167196 · Cited by 85
Wang, Z., Combs, S. A., Brand, R. M., Calvo, M. R., Xu, P., Price, G., Golovach, N., Salawu, E. O., Wise, C., Ponnapalli, S. P., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-10775-y · Cited by 89
Representation learning applications in biological sequence analysis
Iuchi, H., Matsutani, T., Yamada, K., Iwano, N., Sumi, S., Hosoda, S., Zhao, S., Fukunaga, T., & Hamada, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.05.039 · Cited by 88
Cheng, J., Bendjama, K., Rittner, K., & Malone, B. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab422 · Cited by 87
Machine learning solutions for predicting protein–protein interactions
Casadio, R., Martelli, P. L., & Savojardo, C. (2022) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.1618 · Cited by 83
Cui, F., Zhang, Z., & Zou, Q. (2020) · Briefings in Functional Genomics · doi:10.1093/bfgp/elaa030 · Cited by 80
Porebski, B. T., Balmforth, M. R., Browne, G. J., Riley, A., Jamali, K., Fürst, M. J. L. J., Velic, M., Buchanan, A. G., Minter, R., Vaughan, T. J., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01093-3 · Cited by 74
Novel machine learning approaches revolutionize protein knowledge
Bordin, N., Dallago, C., Heinzinger, M., Kim, S., Littmann, M., Rauer, C., Steinegger, M., Rost, B., & Orengo, C. (2023) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2022.11.001 · Cited by 72
Self-citationConnecting chemistry and biology through molecular descriptors
Fernández‐Torras, A., Comajuncosa-Creus, A., Duran‐Frigola, M., & Aloy, P. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.09.001 · Cited by 74
Leveraging transformers‐based language models in proteome bioinformatics
Le, N. Q. K. (2023) · PROTEOMICS · doi:10.1002/pmic.202300011 · Cited by 75
Self-Supervised Contrastive Learning of Protein Representations By Mutual Information MaximizationPreprint
Lu, A. X., Zhang, H., Ghassemi, M., & Moses, A. M. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.04.283929 · Cited by 71
Moffat, L., & Jones, D. T. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab491 · Cited by 73
Structure‐Based Drug Discovery with Deep Learning**
Özçelik, R., van Tilborg, D., Jiménez-Luna, J., & Grisoni, F. (2023) · ChemBioChem · doi:10.1002/cbic.202200776 · Cited by 71
Sharma, R., Shrivastava, S., Singh, S. K., Kumar, A., Saxena, S., & Singh, R. K. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab422 · Cited by 69
Transformer Neural Networks for Protein Family and Interaction Prediction Tasks
Nambiar, A., Liu, S., Heflin, M., Forsyth, J. M., Maslov, S., Hopkins, M., & Ritz, A. (2022) · Journal of Computational Biology · doi:10.1089/cmb.2022.0132 · Cited by 68
Graph-BERT and language model-based framework for protein–protein interaction identification
Jha, K., Karmakar, S., & Saha, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-31612-w · Cited by 66
Theoretical and Data-Driven Approaches for Biomolecular Condensates
Saar, K. L., Qian, D., Good, L. L., Morgunov, A. S., Collepardo‐Guevara, R., Best, R. B., & Knowles, T. P. J. (2023) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00586 · Cited by 66
Yan, K., Lv, H., Shao, J., Chen, S., & Liu, B. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4147-8 · Cited by 66
Hoarfrost, A., Aptekmann, A. A., Farfañuk, G., & Bromberg, Y. (2022) · Nature Communications · doi:10.1038/s41467-022-30070-8 · Cited by 65
Integration of pre-trained protein language models into geometric deep learning networks
Wu, F., Wu, L., Radev, D., Xu, J., & Li, S. Z. (2023) · Communications Biology · doi:10.1038/s42003-023-05133-1 · Cited by 64
Kamiński, K., Ludwiczak, J., Jasiński, M., Bukala, A., Madaj, R., Szczepaniak, K., & Dunin-Horkawicz, S. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab371 · Cited by 60
Jiao, P., Wang, B., Wang, X., Liu, B., Wang, Y., & Li, J. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad637 · Cited by 65
Self-citationHou, Z., Yang, Y., Ma, Z., Wong, K., & Li, X. (2023) · Communications Biology · doi:10.1038/s42003-023-04462-5 · Cited by 62
Chen, J., Wang, J., Hu, Y., Li, X., Qian, Y., & Song, C.-L. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1506508 · Cited by 64
Deep learning frameworks for protein–protein interaction prediction
Hu, X., Feng, C., Ling, T., & Chen, M. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.06.025 · Cited by 61
Lin, B., Luo, X., Liu, Y., & Jin, X. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae289 · Cited by 63
Singh, J., Paliwal, K. K., Litfin, T., Singh, J., & Zhou, Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-11684-w · Cited by 59
Linguistically inspired roadmap for building biologically reliable protein language models
Vu, M. H., Akbar, R., Robert, P. A., Świątczak, B., Sandve, G. K., Greiff, V., & Haug, D. T. T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00637-1 · Cited by 59
Generative models for protein sequence modeling: recent advances and future directions
Mardikoraem, M., Wang, Z., Pascual, N., & Woldring, D. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad358 · Cited by 61
Chen, X., Li, C., Bernards, M. T., Shi, Y., Shao, Q., & He, Y. (2021) · Molecular Systems Design & Engineering · doi:10.1039/d0me00161a · Cited by 56
PhaLP: A Database for the Study of Phage Lytic Proteins and Their Evolution
Criel, B., Taelman, S., Criekinge, W. V., Stock, M., & Briers, Y. (2021) · Viruses · doi:10.3390/v13071240 · Cited by 55
Metadata
- Authors
- Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B.
- Affiliations
- Technical University of Munich (DE)
- Leibniz Supercomputing Centre (DE)
- Corresponding author
- Heinzinger, Michael
- Venue
- BMC Bioinformatics
- Publisher
- openRxiv
- Published
- 2019-04-19
- Volume
- 20
- Issue
- 1
- Pages
- 723-723
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC6918593
- OpenAlex
- W2995514860
- Cited by
- 606
- References
- 105
- Citations by year
- 2026: 53 · 2025: 103 · 2024: 115 · 2023: 117 · 2022: 99 · 2021: 83 · 2020: 29 · 2019: 7
- FWCI
- 18.03
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- Columbia Academic Commons (Columbia University)
- Funders
- Alexander von Humboldt-Stiftung
- Nvidia
- Deutsche Forschungsgemeinschaft (DFG-GZ: RO1320/4-1, RO1320/4‐1)
- Bundesministerium für Bildung und Forschung
- Leibniz-Gemeinschaft
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
- Keywords
- protein sequence embeddings, ELMo, transfer learning, protein function prediction, protein structure prediction, secondary structure prediction, subcellular localization prediction, membrane proteins, dark proteome, metaproteomics, microbiome analysis
- MeSH terms
- Machine Learning, Amino Acid Sequence, Natural Language Processing, Proteins, Neural Networks, Computer, Sequence Analysis, Computational Biology, Databases, Nucleic Acid, Databases, Protein, Proteomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
