Modeling the language of life – Deep Learning Protein Sequences
Open accessPreprint — published as 10.1186/s12859-019-3220-8 · Paper page
Abstract
Background: One common task in Computational Biology is the prediction of aspects of protein function and structure from their amino acid sequence. For 26 years, most state-of-the-art approaches toward this end have been marrying machine learning and evolutionary information. The retrieval of related proteins from ever growing sequence databases is becoming so time-consuming that the analysis of entire proteomes becomes challenging. On top, evolutionary information is less powerful for small families, e.g. for proteins from the Dark Proteome . Results: We introduce a novel way to represent protein sequences as continuous vectors ( embeddings ) by using the deep bi-directional model ELMo taken from natural language processing (NLP). The model has effectively captured the biophysical properties of protein sequences from unlabeled big data (UniRef50). After training, this knowledge is transferred to single protein sequences by predicting relevant sequence features. We refer to these new embeddings as SeqVec ( Seq uence-to- Vec tor) and demonstrate their effectiveness by training simple convolutional neural networks on existing data sets for two completely different prediction tasks. At the per-residue level, we significantly improved secondary structure (for NetSurfP-2.0 data set: Q3=79%±1, Q8=68%±1) and disorder predictions (MCC=0.59±0.03) over methods not using evolutionary information. At the per-protein level, we predicted subcellular localization in ten classes (for DeepLoc data set: Q10=68%±1) and distinguished membrane-bound from water-soluble proteins (Q2= 87%±1). All results built upon the embeddings gained from the new tool SeqVec neither explicitly nor implicitly using evolutionary information. Nevertheless, it improved over some methods using such information. Where the lightning-fast HHblits needed on average about two minutes to generate the evolutionary information for a target protein, SeqVec created the vector representation on average in 0.03 seconds. Conclusion: We have shown that transfer learning can be used to capture biochemical or biophysical properties of protein sequences from large unlabeled sequence databases. The effectiveness of the proposed approach was showcased for different prediction tasks using only single protein sequences. SeqVec embeddings enable predictions that outperform even some methods using evolutionary information. Thus, they prove to condense the underlying principles of protein sequences. This might be the first step towards competitive predictions based only on single protein sequences. Availability: SeqVec: https://github.com/mheinzinger/SeqVec Prediction server: https://embed.protein.properties
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302 · see citing works
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
(2014) · Cited by 33,419 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
A method and server for predicting damaging missense mutations.
Adzhubei, I. A., Schmidt, S., Peshkin, L., Ramensky, V. E., Gerasimova, A., Bork, P., Kondrashov, A. S., & Sunyaev, S. R. (2010) · Nature Methods · doi:10.1038/nmeth0410-248 · Cited by 13,194 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Cd-hit: Accelerated for clustering the next-generation sequencing data
Fu, L., Niu, B., Zhu, Z., Wu, S., & Li, W. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts565 · Cited by 11,175 · see citing works
Cd-hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences
Li, W.-Z., & Godzik, A. (2006) · Bioinform. · doi:10.1093/bioinformatics/btl158 · Cited by 10,990 · see citing works
MEME SUITE: tools for motif discovery and searching
Bailey, T. L., Boden, M., Buske, F. A., Frith, M., Grant, C. E., Clementi, L., Ren, J., Li, W. W., & Noble, W. S. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp335 · Cited by 10,648 · see citing works
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works
Principles that Govern the Folding of Protein Chains
Anfinsen, C. B. (1973) · Science · doi:10.1126/science.181.4096.223 · Cited by 7,096 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Amino acid substitution matrices from protein blocks.
Henikoff, S., & Henikoff, J. G. (1992) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.89.22.10915 · Cited by 6,218 · see citing works
Protein secondary structure prediction based on position-specific scoring matrices.
Jones, D. C. (1999) · Journal of Molecular Biology · doi:10.1006/jmbi.1999.3091 · Cited by 5,741 · see citing works
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1× · see citing works
“Predictions of intrinsic disorder were evaluated through the 10 Matthew’s correlation coefficient (MCC [94]) and the False-Positive Rate (FPR) representative 11 for tasks with high class imbalance.”
MethodMMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets
Steinegger, M., & Söding, J. (2017) · Nature Biotechnology · doi:10.1038/nbt.3988 · Cited by 4,840 · see citing works
WoLF PSORT: protein localization predictor
Horton, P., Park, K., Obayashi, T., Fujita, N., Harada, H., Adams-Collier, C. J., & Nakai, K. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm259 · Cited by 4,154 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
Prediction of Protein Secondary Structure at Better than 70% Accuracy
Rost, B., & Sander, C. (1993) · Journal of Molecular Biology · doi:10.1006/jmbi.1993.1413 · Cited by 2,927 · see citing works
Self-citationKnowledge‐based protein secondary structure assignment
Frishman, D., & Argos, P. (1995) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.340230412 · Cited by 2,524 · see citing works
HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment
Remmert, M., Biegert, A., Hauser, A., & Söding, J. (2012) · Nature Methods · doi:10.1038/nmeth.1818 · Cited by 2,288 · see citing works
Nakai, K., & Horton, P. (1999) · Trends in Biochemical Sciences · doi:10.1016/s0968-0004(98)01336-x · Cited by 2,181 · cited 1× · see citing works
“This concept resembles 24 that of using k-mers for database searches [33], clustering [74], motifs [75, 76], and prediction 25 methods [66, 70, 77-81].”
MethodUniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches
Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B., Wu, C. H., & the UniProt Consortium (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu739 · Cited by 1,928 · see citing works
Self-citationPrediction of protein subcellular localization
Yu, C., Chen, Y., Lu, C., & Hwang, J. (2006) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.21018 · Cited by 1,802 · see citing works
On the Convergence of Adam and Beyond
Reddi, S. J., Kale, S., & Kumar, S. (2019) · arXiv · doi:10.48550/arxiv.1904.09237 · Cited by 1,771 · see citing works
JPred4: a protein secondary structure prediction server
Drozdetskiy, A., Cole, C., Procter, J. B., & Barton, G. J. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv332 · Cited by 1,714 · see citing works
PISCES: a protein sequence culling server
Wang, G., & Dunbrack, R. L. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg224 · Cited by 1,664 · see citing works
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D. A., & Rush, A. M. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,546 · see citing works
Combining evolutionary information and neural networks to predict protein secondary structure
Rost, B., & Sander, C. (1994) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.340190108 · Cited by 1,452 · see citing works
Self-citationMorcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434 · see citing works
HH-suite3 for fast remote homology detection and deep protein annotation
Steinegger, M., Meier, M., Mirdita, M., Vöhringer, H., Haunsberger, S. J., & Söding, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3019-7 · Cited by 1,145 · see citing works
DeepLoc: prediction of protein subcellular localization using deep learning
Armenteros, J. J. A., Sønderby, C. K., Sønderby, S. K., Nielsen, H., & Winther, O. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx431 · Cited by 1,144 · see citing works
Protein 3D Structure Computed from Evolutionary Sequence Variation
Marks, D. S., Colwell, L. J., Sheridan, R., Hopf, T. A., Pagnani, A., Zecchina, R., & Sander, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028766 · Cited by 1,144 · see citing works
Bairoch, A. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.304 · Cited by 1,140 · see citing works
SNAP: predict effect of non-synonymous polymorphisms on function
Bromberg, Y., & Rost, B. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm238 · Cited by 863 · see citing works
Self-citationBoutet, E., Lieberherr, D., Tognolli, M., Schneider, M., Bansal, P., Bridge, A., Poux, S., Bougueleret, L., & Xenarios, I. (2016) · Methods in molecular biology · doi:10.1007/978-1-4939-3167-5_2 · Cited by 824 · see citing works
Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics
Asgari, E., & Mofrad, M. R. K. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141287 · Cited by 817 · see citing works
Fox, N. K., Brenner, S. E., & Chandonia, J.-M. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1240 · Cited by 762 · see citing works
Conservation and prediction of solvent accessibility in protein families
Rost, B., & Sander, C. (1994) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.340200303 · Cited by 666 · see citing works
Self-citationTransmembrane helices predicted at 95% accuracy
Rost, B., Sander, C., Casadio, R., & Fariselli, P. (1995) · Protein Science · doi:10.1002/pro.5560040318 · Cited by 664 · see citing works
Self-citationEvaluation and improvement of multiple sequence methods for protein secondary structure prediction
Cuff, J. A., & Barton, G. J. (1999) · Proteins Structure Function and Bioinformatics · doi:10.1002/(sici)1097-0134(19990301)34:4 · Cited by 660 · see citing works
Protein structure prediction from sequence variation
Marks, D. S., Hopf, T. A., & Sander, C. (2012) · Nature Biotechnology · doi:10.1038/nbt.2419 · Cited by 622 · see citing works
Rost, B., Fariselli, P., & Casadio, R. (1996) · Protein Science · doi:10.1002/pro.5560050824 · Cited by 606 · see citing works
Self-citationNetSurfP-2.0: improved prediction of protein structural features by integrated deep learning
Klausen, M., Jespersen, M. C., Nielsen, H., Jensen, K. K., Jurtz, V., Sønderby, C., Sommer, M., Winther, O., Nielsen, M., Petersen, B., et al. (2018) · bioRxiv · doi:10.1002/prot.25674 · Cited by 564 · see citing works
Mismatch string kernels for discriminative protein classification
Leslie, C. S., Eskin, E., Cohen, A., Weston, J., & Noble, W. S. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg431 · Cited by 559 · see citing works
Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields
Wang, S., Peng, J., Ma, J., & Xu, J. (2016) · Scientific Reports · doi:10.1038/srep18962 · Cited by 556 · see citing works
RaptorX-Property: a web server for protein structure property prediction
Wang, S., Li, W., Liu, S., & Xu, J. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw306 · Cited by 553 · see citing works
Comparing two K-category assignments by a K-category correlation coefficient
Gorodkin, J. (2004) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2004.09.006 · Cited by 545 · see citing works
Improved prediction of protein secondary structure by use of sequence profiles and neural networks.
Rost, B., & Sander, C. (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.16.7558 · Cited by 516 · see citing works
Self-citationOPTIMIZING LONG INTRINSIC DISORDER PREDICTORS WITH PROTEIN EVOLUTIONARY INFORMATION
PENG, K., Vučetić, S., Predrag Radivojac, A., Brown, C. J., Dunker, A. K., & Zoran Obradovic, A. (2005) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720005000886 · Cited by 497 · see citing works
Heffernan, R., Yang, Y., Paliwal, K. K., & Zhou, Y. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx218 · Cited by 358 · see citing works
Protein flexibility and intrinsic disorder
Predrag Radivojac, A., Zoran Obradovic, A., Smith, D. K., Zhu, G., Vučetić, S., Brown, C. J., Lawson, J. D., & Dunker, A. K. (2003) · Protein Science · doi:10.1110/ps.03128904 · Cited by 341 · see citing works
Chou, K., Wu, Z., & Xiao, X. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018258 · Cited by 319 · see citing works
YLoc—an interpretable web server for predicting subcellular localization
Briesemeister, S., hrer, J. ⁄. R. R. ⁄., & Kohlbacher, O. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq477 · Cited by 315 · see citing works
What’s in a name? Why these proteins are intrinsically disordered
Dunker, A. K., Babu, M. M., Barbar, E. J., Blackledge, M. J., Bondos, S. E., Dosztányi, Z., Dyson, H. J., Forman-Kay, J. D., Fuxreiter, M., Gsponer, J. A., et al. (2013) · Intrinsically Disordered Proteins · doi:10.4161/idp.24157 · Cited by 308 · see citing works
Self-citationBlum, T., Briesemeister, S., & Kohlbacher, O. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-274 · Cited by 305 · see citing works
Mimicking Cellular Sorting Improves Prediction of Subcellular Localization
Nair, R. P., & Rost, B. (2005) · Journal of Molecular Biology · doi:10.1016/j.jmb.2005.02.025 · Cited by 290 · see citing works
Self-citationSIFTS: Structure Integration with Function, Taxonomy and Sequences resource
Velankar, S. S., Dana, J. M., Jacobsen, J. O. B., van Ginkel, G., Gane, P. J., Luo, J., Oldfield, T. J., O’Donovan, C., Martin, M. J., & Kleywegt, G. J. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1258 · Cited by 277 · see citing works
ISIS: interaction sites identified from sequence
Ofran, Y., & Rost, B. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btl303 · Cited by 257 · see citing works
Self-citationProtein–Protein Interaction Hotspots Carved into Sequences
Ofran, Y., & Rost, B. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030119 · Cited by 255 · see citing works
Self-citationSixty-five years of the long march in protein secondary structure prediction: the final stretch?
Yang, Y., Gao, J., Wang, J., Heffernan, R., Hanson, J., Paliwal, K. K., & Zhou, Y. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw129 · Cited by 237 · see citing works
Unexpected features of the dark proteome
Perdigão, N., Heinrich, J., Stolte, C., Sabir, K. S., Buckley, M. J., Tabor, B., Signal, B., Gloss, B., Hammang, C. J., Rost, B., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1508380112 · Cited by 210 · see citing works
Self-citationPROFILE-BASED STRING KERNELS FOR REMOTE HOMOLOGY DETECTION AND MOTIF EXTRACTION
Kuang, R., Ie, E., WANG, K., Wang, K., SIDDIQI, M., Freund, Y., & Leslie, C. S. (2005) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s021972000500120x · Cited by 191 · see citing works
Protein flexibility and rigidity predicted from sequence
Schlessinger, A., & Rost, B. (2005) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.20587 · Cited by 186 · see citing works
Self-citationPROFcon: novel prediction of long-range contacts
Punta, M., & Rost, B. (2005) · Computer applications in the biosciences · doi:10.1093/bioinformatics/bti454 · Cited by 153 · see citing works
Self-citationPredicting transmembrane beta-barrels in proteomes
Bigelow, H. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh580 · Cited by 147 · see citing works
ProteinNet: a standardized data set for machine learning of protein structure
AlQuraishi, M. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2932-0 · Cited by 141 · see citing works
SherLoc2: A High-Accuracy Hybrid Method for Predicting Subcellular Localization of Proteins
Briesemeister, S., Blum, T., Brady, S., Lam, Y. P., Kohlbacher, O., & Shatkay, H. (2009) · Journal of Proteome Research · doi:10.1021/pr900665y · Cited by 138 · see citing works
Sequence‐similar, structure‐dissimilar protein pairs in the PDB
Kosloff, M., & Kolodny, R. (2008) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21770 · Cited by 124 · see citing works
Neural networks for secondary structure and structural class predictions
Chandonia, J., & Karplus, M. (1995) · Protein Science · doi:10.1002/pro.5560040214 · Cited by 121 · see citing works
Natively unstructured regions in proteins identified from contact predictions
Schlessinger, A., Punta, M., & Rost, B. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm349 · Cited by 120 · see citing works
Self-citationEvolutionary profiles improve protein–protein interaction prediction from sequence
Hamp, T., & Rost, B. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv077 · Cited by 117 · see citing works
Self-citationLocTree2 predicts localization for all domains of life
Goldberg, T., Hamp, T., & Rost, B. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts390 · Cited by 114 · see citing works
Self-citationDeepLoc: prediction of protein subcellular localization using deep learning
Armenteros, J. J. A., Sønderby, C. K., Sønderby, S. K., Nielsen, H., & Winther, O. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx548 · Cited by 101 · see citing works
Natively Unstructured Loops Differ from Other Loops
Schlessinger, A., Liu, J., & Rost, B. (2005) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030140 · Cited by 96 · see citing works
Self-citationJury returns on structure prediction
Rost, B., & Sander, C. (1992) · Nature · doi:10.1038/360540b0 · Cited by 93 · see citing works
Self-citationAbriata, L. A., Tamò, G. E., Monastyrskyy, B., Kryshtafovych, A., & Peraro, M. D. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25423 · Cited by 90 · see citing works
Mehta, P. K., Heringa, J., & Argos, P. (1995) · Protein Science · doi:10.1002/pro.5560041208 · Cited by 86 · see citing works
Protein secondary structure prediction
Barton, G. J. (1995) · Current Opinion in Structural Biology · doi:10.1016/0959-440x(95)80099-9 · Cited by 82 · see citing works
Better prediction of sub‐cellular localization by combining evolutionary and structural information
Nair, R. P., & Rost, B. (2003) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.10507 · Cited by 81 · see citing works
Self-citationAsgari, E., McHardy, A. C., & Mofrad, M. R. K. (2019) · Scientific Reports · doi:10.1038/s41598-019-38746-w · Cited by 77 · see citing works
Bernard, G., Chan, C. X., & Ragan, M. A. (2016) · Scientific Reports · doi:10.1038/srep28970 · Cited by 74 · see citing works
Marino‐Buslje, C., Teppa, E., Domenico, T. D., Delfino, J. M., & Nielsen, M. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000978 · Cited by 66 · see citing works
Predicting α-helix and β-strand segments of globular proteins
Solovyev, V. V., & Salamov, A. A. (1994) · Computer applications in the biosciences · doi:10.1093/bioinformatics/10.6.661 · Cited by 63 · see citing works
Improved protein contact predictions with the MetaPSICOV2 server in CASP12
Buchan, D. W. A., & Jones, D. T. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25379 · Cited by 53 · see citing works
All-atom 3D structure prediction of transmembrane β-barrel proteins from sequences
Hayat, S., Sander, C., Marks, D. S., & Elofsson, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1419956112 · Cited by 51 · see citing works
Mut2Vec: distributed representation of cancerous mutations
Kim, S., Lee, H., Kim, K., & Kang, J. (2018) · BMC Medical Genomics · doi:10.1186/s12920-018-0349-7 · Cited by 51 · see citing works
Prediction of Intrinsic Disorder and Its Use in Functional Proteomics
Uversky, V. N., Predrag Radivojac, A., Iakoucheva, L. M., Zoran Obradovic, A., & Dunker, A. K. (2007) · Methods in molecular biology · doi:10.1007/978-1-59745-547-3_5 · Cited by 44 · see citing works
Xu, Y., Song, J., Wilson, C., & Whisstock, J. C. (2018) · Scientific Reports · doi:10.1038/s41598-018-26392-7 · Cited by 36 · see citing works
Single cell RNA-seq data clustering using TF-IDF based methods
Moussa, M. R., & Măndoiu, I. (2018) · BMC Genomics · doi:10.1186/s12864-018-4922-4 · Cited by 35 · see citing works
Identifying remote protein homologs by network propagation
Noble, W. S., Kuang, R., Leslie, C. S., & Weston, J. (2005) · FEBS Journal · doi:10.1111/j.1742-4658.2005.04947.x · Cited by 31 · see citing works
Molecular and clinical analysis of Ellis-van Creveld syndrome in the United Arab Emirates
Ali, B. R., Akawi, N. A., Chedid, F., Bakir, M., Rehman, M. U., Rahmani, A., & Al‐Gazali, L. (2010) · BMC Medical Genetics · doi:10.1186/1471-2350-11-33 · Cited by 27 · see citing works
Dark Proteins Important for Cellular Function
Schafferhans, A., O’Donoghue, S. I., Heinzinger, M., & Rost, B. (2018) · PROTEOMICS · doi:10.1002/pmic.201800227 · Cited by 9 · see citing works
Self-citationCharacteristics of Sentence Length in Running Text
Schils, E. (1993) · Literary and Linguistic Computing · doi:10.1093/llc/8.1.20 · Cited by 9 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
2 of the 105 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Rost B, Casadio R, Fariselli P, Sander C. Transmembrane helix prediction at 95% accuracy. Protein Sci. 1995;4:521-33.
- Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, Qin C, Zidek A, Nelson A, Bridgland A, Penedones H. De novo structure prediction with deeplearning based scoring. Annu Rev Biochem. 2018;77(363-382):6.
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · see citing works
Self-citationProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009 · see citing works
Simulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937 · see citing works
Evaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719 · see citing works
Single-sequence protein structure prediction using language models from deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., & AlQuraishi, M. (2021) · doi:10.1101/2021.08.02.454840 · Cited by 455 · see citing works
“Very large Transformer-based models trained on hundreds of millions and potentially billions of protein sequences are increasingly available 26,35,36 and the scaling previously observed in natural language applications 44 makes it likely that the performance of RGN2 and similar methods will…”
BackgroundSingle-sequence protein structure prediction using a language model and deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01432-w · Cited by 455 · see citing works
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 409 · see citing works
Self-citationRao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350 · see citing works
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 345 · see citing works
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341 · see citing works
Bilingual language model for protein sequence and structure
Heinzinger, M., Weissenow, K., Sanchez, J. G., Henkel, A., Mirdita, M., Steinegger, M., & Rost, B. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae150 · Cited by 311 · see citing works
Self-citationElnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 305 · see citing works
Self-citationMachine learning for functional protein design
Notin, P., Rollins, N. J., Gal, Y., Sander, C., & Marks, D. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 278 · see citing works
Prediction of protein–protein interaction using graph neural networks
Jha, K., Saha, S., & Singh, H. (2022) · Scientific Reports · doi:10.1038/s41598-022-12201-9 · Cited by 254 · see citing works
Rhea, the reaction knowledgebase in 2022
Bansal, P., Morgat, A., Axelsen, K. B., Muthukrishnan, V., Coudert, E., Aimo, L., Hyka‐Nouspikel, N., Gasteiger, E., Kerhornou, A., Neto, T. B., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab1016 · Cited by 251 · see citing works
PredictProtein - Predicting Protein Structure and Function for 29 Years
Bernhofer, M., Dallago, C., Karl, T., Satagopam, V., Heinzinger, M., Littmann, M., Olenyi, T., Qiu, J., Schütze, K., Yachdav, G., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab354 · Cited by 239 · see citing works
Self-citationFine-tuning protein language models boosts predictions across diverse tasks
Schmirler, R., Heinzinger, M., & Rost, B. (2024) · Nature Communications · doi:10.1038/s41467-024-51844-2 · Cited by 229 · see citing works
Self-citationDeep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227 · see citing works
(2023) · doi:10.1101/2023.12.07.570727 · Cited by 207 · see citing works
Singh, R., Sledzieski, S., Bryson, B. D., Cowen, L., & Berger, B. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2220778120 · Cited by 207 · see citing works
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2023) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 206 · see citing works
Ankh ☥: Optimized Protein Language Model Unlocks General-Purpose ModellingPreprint
Elnaggar, A., Essam, H., Salah-Eldin, W., Moustafa, W., Elkerdawy, M., Rochereau, C., & Rost, B. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.16.524265 · Cited by 201 · see citing works
Self-citationLearning functional properties of proteins with language models
Ünsal, S., Ataş, H., Albayrak, M., Turhan, K., Acar, A. C., & Doğan, T. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00457-9 · Cited by 196 · see citing works
Transformer-based deep learning for predicting protein properties in the life sciences
Chandra, A. A., Tünnermann, L., Löfstedt, T., & Gratz, R. (2023) · eLife · doi:10.7554/elife.82819 · Cited by 192 · see citing works
Le, N. Q. K., Ho, Q., Nguyen, T., & Ou, Y. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab005 · Cited by 188 · see citing works
Language models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181 · see citing works
Machine Learning Applied to Predicting Microorganism Growth Temperatures and Enzyme Catalytic Optima
Li, G., Rabe, K. S., Nielsen, J., & Engqvist, M. K. M. (2019) · ACS Synthetic Biology · doi:10.1021/acssynbio.9b00099 · Cited by 178 · see citing works
Applications of transformer-based language models in bioinformatics: a survey
Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023) · Bioinformatics Advances · doi:10.1093/bioadv/vbad001 · Cited by 176 · see citing works
Embeddings from protein language models predict conservation and variant effects
(2021) · doi:10.1007/s00439-021-02411-y · Cited by 174 · see citing works
Self-citationEmbeddings from deep learning transfer GO annotations beyond homology
Littmann, M., Heinzinger, M., Dallago, C., Olenyi, T., & Rost, B. (2021) · Scientific Reports · doi:10.1038/s41598-020-80786-0 · Cited by 169 · see citing works
Self-citationDELPHI: accurate deep ensemble model for protein interaction sites prediction
Li, Y., Golding, G. B., & Ilie, L. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa750 · Cited by 164 · see citing works
Yang, X., Yang, S., Li, Q., Wuchty, S., & Zhang, Z. (2019) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2019.12.005 · Cited by 163 · see citing works
Learning meaningful representations of protein sequences
Detlefsen, N. S., Hauberg, S., & Boomsma, W. (2022) · Nature Communications · doi:10.1038/s41467-022-29443-w · Cited by 163 · see citing works
RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks
Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., & Šikić, M. (2025) · Nature Communications · doi:10.1038/s41467-025-60872-5 · Cited by 146 · see citing works
Artificial intelligence in early drug discovery enabling precision medicine
Boniolo, F., Dorigatti, E., Ohnmacht, A. J., Saur, D., Schubert, B., & Menden, M. P. (2021) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2021.1918096 · Cited by 145 · see citing works
Protein remote homology detection and structural alignment using deep learning
Hamamsy, T., Morton, J. T., Blackwell, R., Berenberg, D., Carriero, N. J., Gligorijević, V., Strauss, C. E. M., Leman, J. K., Cho, K., & Bonneau, R. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01917-2 · Cited by 145 · see citing works
Akiyama, M., & Sakakibara, Y. (2022) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqac012 · Cited by 144 · see citing works
(2022) · doi:10.1016/j.str.2022.05.001 · Cited by 139 · see citing works
Self-citationLearning spatial structures of proteins improves protein–protein interaction prediction
Song, B., Luo, X., Luo, X., Liu, Y., Niu, Z., & Zeng, X. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab558 · Cited by 134 · see citing works
Unifying the known and unknown microbial coding sequence space
Vanni, C., Schechter, M. S., Acinas, S. G., Barberán, A., Buttigieg, P. L., Casamayor, E. O., Delmont, T. O., Duarte, C. M., Eren, A. M., Finn, R. D., et al. (2022) · eLife · doi:10.7554/elife.67667 · Cited by 134 · see citing works
Yan, J., Cai, J., Zhang, B., Wang, Y., Wong, D. F., & Siu, S. W. I. (2022) · Antibiotics · doi:10.3390/antibiotics11101451 · Cited by 132 · see citing works
Self-citationAkbar, S., Hayat, M., Tahir, M., Khan, S. A., & Alarfaj, F. K. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102349 · Cited by 131 · see citing works
Light attention predicts protein location from the language of life
Stärk, H., Dallago, C., Heinzinger, M., & Rost, B. (2021) · Bioinformatics Advances · doi:10.1093/bioadv/vbab035 · Cited by 130 · see citing works
Self-citationFLIP: Benchmark tasks in fitness landscape inference for proteinsPreprint
Dallago, C., Mou, J., Johnston, K. E., Wittmann, B. J., Bhattacharya, N., Goldman, S., Madani, A., & Yang, K. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.09.467890 · Cited by 127 · see citing works
Self-citationFrom sequence to function through structure: Deep learning for protein design
(2022) · doi:10.1016/j.csbj.2022.11.014 · Cited by 124 · see citing works
Self-citationQiu, J., Bernhofer, M., Heinzinger, M., Kemper, S., Norambuena, T., Melo, F., & Rost, B. (2020) · Journal of Molecular Biology · doi:10.1016/j.jmb.2020.02.026 · Cited by 123 · see citing works
Self-citationWang, D., Feng, L., Ye, J., Zou, J., & Zheng, Y. (2023) · MedComm – Future Medicine · doi:10.1002/mef2.43 · Cited by 123 · see citing works
Machine learning modeling of family wide enzyme-substrate specificity screens
Goldman, S., Das, R., Yang, K. K., & Coley, C. W. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009853 · Cited by 120 · see citing works
Villegas-Morcillo, A., Makrodimitris, S., van Ham, R. C. H. J., Gomez, A. M., Sanchez, V., & Reinders, M. J. T. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa701 · Cited by 118 · see citing works
Chen, J., Zheng, S., Zhao, H., & Yang, Y. (2021) · Journal of Cheminformatics · doi:10.1186/s13321-021-00488-1 · Cited by 118 · see citing works
Raza, A., Uddin, J., Almuhaimeed, A., Akbar, S., Zou, Q., & Ahmad, A. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01563 · Cited by 116 · see citing works
A new age in protein design empowered by deep learning
Khakzad, H., Igashov, I., Schneuing, A., Goverde, C. A., Bronstein, M. M., & Correia, B. E. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.006 · Cited by 113 · see citing works
Akbar, S., Zou, Q., Raza, A., & Alarfaj, F. K. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102860 · Cited by 110 · see citing works
Machine learning to navigate fitness landscapes for protein engineering
Freschlin, C. R., Fahlberg, S. A., & Romero, P. A. (2022) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2022.102713 · Cited by 105 · see citing works
TMbed: transmembrane proteins predicted through language model embeddings
Bernhofer, M., & Rost, B. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04873-x · Cited by 105 · see citing works
Self-citationContrastive learning on protein embeddings enlightens midnight zone
(2022) · doi:10.1093/nargab/lqac043 · Cited by 105 · see citing works
Self-citationTaxonomic classification of DNA sequences beyond sequence similarity using deep neural networks
Mock, F., Kretschmer, F., Kriese, A., Böcker, S., & Marz, M. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2122636119 · Cited by 104 · see citing works
Learned Embeddings from Deep Learning to Visualize and Predict Protein Sets
Dallago, C., Schütze, K., Heinzinger, M., Olenyi, T., Littmann, M., Lu, A. X., Yang, K., Min, S., Yoon, S., Morton, J. T., et al. (2021) · Current Protocols · doi:10.1002/cpz1.113 · Cited by 101 · see citing works
Self-citationSignalP: The Evolution of a Web Server
Nielsen, H., Teufel, F., Brunak, S., & von HEIJNE, G. (2024) · Methods in molecular biology · doi:10.1007/978-1-0716-4007-4_17 · Cited by 100 · see citing works
EpiDope: a deep neural network for linear B-cell epitope prediction
Collatz, M., Mock, F., Barth, E., Hölzer, M., Sachse, K., & Marz, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa773 · Cited by 97 · see citing works
Transforming the Language of Life
Nambiar, A., Heflin, M., Liu, S., Maslov, S., Hopkins, M., & Ritz, A. (2020) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3388440.3412467 · Cited by 97 · see citing works
The Large Language Models on Biomedical Data Analysis: A Survey
Lan, W., Tang, Z., Liu, M., Chen, Q., Peng, W., Chen, Y. P., & Pan, Y. F. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3530794 · Cited by 94 · see citing works
Large language models improve annotation of prokaryotic viral proteins
Flamholz, Z. N., Biller, S. J., & Kelly, L. (2024) · Nature Microbiology · doi:10.1038/s41564-023-01584-8 · Cited by 92 · see citing works
Machine learning on protein–protein interaction prediction: models, challenges and trends
Tang, T., Zhang, X., Liu, Y., Peng, H., Zheng, B., Yin, Y., & Zeng, X. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad076 · Cited by 89 · see citing works
Le, N. Q. K., & Ho, Q. (2021) · Methods · doi:10.1016/j.ymeth.2021.12.004 · Cited by 86 · see citing works
Improving protein succinylation sites prediction using embeddings from protein language model
Pokharel, S., Pratyush, P., Heinzinger, M., Newman, R. H., & KC, D. B. (2022) · Scientific Reports · doi:10.1038/s41598-022-21366-2 · Cited by 86 · see citing works
Self-citationLindorff-Larsen, K., & Kragelund, B. B. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167196 · Cited by 85 · see citing works
Wang, Z., Combs, S. A., Brand, R. M., Calvo, M. R., Xu, P., Price, G., Golovach, N., Salawu, E. O., Wise, C., Ponnapalli, S. P., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-10775-y · Cited by 84 · see citing works
Representation learning applications in biological sequence analysis
Iuchi, H., Matsutani, T., Yamada, K., Iwano, N., Sumi, S., Hosoda, S., Zhao, S., Fukunaga, T., & Hamada, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.05.039 · Cited by 83 · see citing works
Cheng, J., Bendjama, K., Rittner, K., & Malone, B. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab422 · Cited by 83 · see citing works
Machine learning solutions for predicting protein–protein interactions
Casadio, R., Martelli, P. L., & Savojardo, C. (2022) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.1618 · Cited by 80 · see citing works
Cui, F., Zhang, Z., & Zou, Q. (2020) · Briefings in Functional Genomics · doi:10.1093/bfgp/elaa030 · Cited by 79 · see citing works
Porebski, B. T., Balmforth, M. R., Browne, G. J., Riley, A., Jamali, K., Fürst, M. J. L. J., Velic, M., Buchanan, A. G., Minter, R., Vaughan, T. J., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01093-3 · Cited by 72 · see citing works
Novel machine learning approaches revolutionize protein knowledge
Bordin, N., Dallago, C., Heinzinger, M., Kim, S., Littmann, M., Rauer, C., Steinegger, M., Rost, B., & Orengo, C. (2023) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2022.11.001 · Cited by 72 · see citing works
Self-citationConnecting chemistry and biology through molecular descriptors
Fernández‐Torras, A., Comajuncosa-Creus, A., Duran‐Frigola, M., & Aloy, P. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.09.001 · Cited by 70 · see citing works
Leveraging transformers‐based language models in proteome bioinformatics
Le, N. Q. K. (2023) · PROTEOMICS · doi:10.1002/pmic.202300011 · Cited by 70 · see citing works
Self-Supervised Contrastive Learning of Protein Representations By Mutual Information MaximizationPreprint
Lu, A. X., Zhang, H., Ghassemi, M., & Moses, A. M. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.04.283929 · Cited by 70 · see citing works
Moffat, L., & Jones, D. T. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab491 · Cited by 69 · see citing works
Structure‐Based Drug Discovery with Deep Learning**
Özçelik, R., van Tilborg, D., Jiménez-Luna, J., & Grisoni, F. (2023) · ChemBioChem · doi:10.1002/cbic.202200776 · Cited by 69 · see citing works
Sharma, R., Shrivastava, S., Singh, S. K., Kumar, A., Saxena, S., & Singh, R. K. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab422 · Cited by 67 · see citing works
Transformer Neural Networks for Protein Family and Interaction Prediction Tasks
Nambiar, A., Liu, S., Heflin, M., Forsyth, J. M., Maslov, S., Hopkins, M., & Ritz, A. (2022) · Journal of Computational Biology · doi:10.1089/cmb.2022.0132 · Cited by 67 · see citing works
Graph-BERT and language model-based framework for protein–protein interaction identification
Jha, K., Karmakar, S., & Saha, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-31612-w · Cited by 66 · see citing works
Theoretical and Data-Driven Approaches for Biomolecular Condensates
Saar, K. L., Qian, D., Good, L. L., Morgunov, A. S., Collepardo‐Guevara, R., Best, R. B., & Knowles, T. P. J. (2023) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00586 · Cited by 66 · see citing works
Yan, K., Lv, H., Shao, J., Chen, S., & Liu, B. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4147-8 · Cited by 66 · see citing works
Hoarfrost, A., Aptekmann, A. A., Farfañuk, G., & Bromberg, Y. (2022) · Nature Communications · doi:10.1038/s41467-022-30070-8 · Cited by 62 · see citing works
Integration of pre-trained protein language models into geometric deep learning networks
Wu, F., Wu, L., Radev, D., Xu, J., & Li, S. Z. (2023) · Communications Biology · doi:10.1038/s42003-023-05133-1 · Cited by 61 · see citing works
Kamiński, K., Ludwiczak, J., Jasiński, M., Bukala, A., Madaj, R., Szczepaniak, K., & Dunin-Horkawicz, S. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab371 · Cited by 60 · see citing works
Jiao, P., Wang, B., Wang, X., Liu, B., Wang, Y., & Li, J. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad637 · Cited by 60 · see citing works
Self-citationHou, Z., Yang, Y., Ma, Z., Wong, K., & Li, X. (2023) · Communications Biology · doi:10.1038/s42003-023-04462-5 · Cited by 59 · see citing works
Chen, J., Wang, J., Hu, Y., Li, X., Qian, Y., & Song, C.-L. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1506508 · Cited by 59 · see citing works
Deep learning frameworks for protein–protein interaction prediction
Hu, X., Feng, C., Ling, T., & Chen, M. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.06.025 · Cited by 58 · see citing works
Lin, B., Luo, X., Liu, Y., & Jin, X. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae289 · Cited by 58 · see citing works
Singh, J., Paliwal, K. K., Litfin, T., Singh, J., & Zhou, Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-11684-w · Cited by 57 · see citing works
Linguistically inspired roadmap for building biologically reliable protein language models
Vu, M. H., Akbar, R., Robert, P. A., Świątczak, B., Sandve, G. K., Greiff, V., & Haug, D. T. T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00637-1 · Cited by 57 · see citing works
Generative models for protein sequence modeling: recent advances and future directions
Mardikoraem, M., Wang, Z., Pascual, N., & Woldring, D. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad358 · Cited by 57 · see citing works
Chen, X., Li, C., Bernards, M. T., Shi, Y., Shao, Q., & He, Y. (2021) · Molecular Systems Design & Engineering · doi:10.1039/d0me00161a · Cited by 56 · see citing works
PhaLP: A Database for the Study of Phage Lytic Proteins and Their Evolution
Criel, B., Taelman, S., Criekinge, W. V., Stock, M., & Briers, Y. (2021) · Viruses · doi:10.3390/v13071240 · Cited by 55 · see citing works
- Authors
- Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B.
- Affiliations
- Technical University of Munich (DE)
- Leibniz Supercomputing Centre (DE)
- Corresponding author
- Heinzinger, Michael
- Venue
- BMC Bioinformatics
- Publisher
- openRxiv
- Published
- 2019-04-19
- Volume
- 20
- Issue
- 1
- Pages
- 723-723
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC6918593
- OpenAlex
- W2995514860
- Cited by
- 606
- References
- 105
- Citations by year
- 2026: 53 · 2025: 103 · 2024: 115 · 2023: 117 · 2022: 99 · 2021: 83 · 2020: 29 · 2019: 7
- FWCI
- 18.03
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- Columbia Academic Commons (Columbia University)
- Funders
- Alexander von Humboldt-Stiftung
- Nvidia
- Deutsche Forschungsgemeinschaft (DFG-GZ: RO1320/4-1, RO1320/4‐1)
- Bundesministerium für Bildung und Forschung
- Leibniz-Gemeinschaft
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
- Keywords
- protein sequence embeddings, ELMo, transfer learning, protein function prediction, protein structure prediction, secondary structure prediction, subcellular localization prediction, membrane proteins, dark proteome, metaproteomics, microbiome analysis
- MeSH terms
- Machine Learning, Amino Acid Sequence, Natural Language Processing, Proteins, Neural Networks, Computer, Sequence Analysis, Computational Biology, Databases, Nucleic Acid, Databases, Protein, Proteomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
