The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Transformer-based deep learning for predicting protein properties in the life sciences
Open accessAbstract
Recent developments in deep learning, coupled with an increasing number of sequenced proteins, have led to a breakthrough in life science applications, in particular in protein property prediction. There is hope that deep learning can close the gap between the number of sequenced proteins and proteins with known properties based on lab experiments. Language models from the field of natural language processing have gained popularity for protein property predictions and have led to a new computational revolution in biology, where old prediction results are being improved regularly. Such models can learn useful multipurpose representations of proteins from large open repositories of protein sequences and can be used, for instance, to predict protein properties. The field of natural language processing is growing quickly because of developments in a class of models based on a particular model—the Transformer model. We review recent developments and the use of large-scale Transformer models in applications for predicting protein characteristics and how such models can be used to predict, for example, post-translational modifications. We review shortcomings of other deep learning models and explain how the Transformer models have quickly proven to be a very promising way to unravel information hidden in the sequences of amino acids.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Principal component analysis: A review and recent developments
Jolliffe, I. T., & Cadima, J. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2015.0202 · Cited by 10,117
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391
Research works printed but not linked: 121
- Alanis-Lobato, 2016, HIPPIE v2. 0: enhancing meaningfulness and reliability of protein–protein interaction networks, Nucleic Acids Research, 45, D408, 10.1093/nar/gkw985 · DOI · Paper page
- Albawi S, Mohammed TA, Al-Zawi S. 2017. Understanding of a convolutional neural network. 2017 International Conference on Engineering and Technology (ICET. · DOI · Paper page
- Altschul, 1997, Gapped blast and PSI-BLAST: a new generation of protein database search programs, Nucleic Acids Research, 25, 3389, 10.1093/nar/25.17.3389 · DOI · Paper page
- Ammari MG, Gresham CR, McCarthy FM, Nanduri B. 2016. HPIDB 2.0: a curated database for host–pathogen interactions. Database.
- Behjati, 2022, Protein sequence profile prediction using protalbert transformer, Computational Biology and Chemistry, 99, 10.1016/j.compbiolchem.2022.107717 · DOI
- Bergstra J, Yamins D, Cox D. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. International conference on machine learning; 2013: PMLR.
- Brandes, 2021, ProteinBERT: A Universal Deep-Learning Model of Protein Sequence and Function, 10.1101/2021.05.24.445464 · doi:10.1101/2021.05.24.445464
- Britz D, Goldie A, Luong MT, Le Q. 2017. Massive Exploration of Neural Machine Translation Architectures. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. · DOI · Paper page
- Bulcun, 2012, Disorders of glucose metabolism and insulin resistance in patients with obstructive sleep apnoea syndrome, International Journal of Clinical Practice, 66, 91, 10.1111/j.1742-1241.2011.02795.x · DOI
- Cai, 2021, MSA-regularized protein sequence transformer toward predicting genome-wide chemical-protein interactions: application to gpcrome deorphanization, Journal of Chemical Information and Modeling, 61, 1570, 10.1021/acs.jcim.0c01285 · DOI
- Chan, 2015, Glass: a comprehensive database for experimentally validated GPCR-ligand associations, Bioinformatics, 31, 3035, 10.1093/bioinformatics/btv302 · DOI
- Chandra, 2019, Bigram-PGK: phosphoglycerylation prediction using the technique of bigram probabilities of position specific scoring matrix, BMC Molecular and Cell Biology, 20, 10.1186/s12860-019-0240-1 · DOI
- Chandra, 2020, RAM-PGK: prediction of lysine phosphoglycerylation based on residue adjacency matrix, Genes, 11, 10.3390/genes11121524 · DOI
- Charte, 2019, A snapshot on nonstandard supervised learning problems: taxonomy, relationships, problem transformations and algorithm adaptations, Progress in Artificial Intelligence, 8, 1, 10.1007/s13748-018-00167-7 · DOI
- Chen, 2002, The binding database: data management and interface design, Bioinformatics, 18, 130, 10.1093/bioinformatics/18.1.130 · DOI · Paper page
- Chen, 2020, IStable 2.0: predicting protein thermal stability changes by integrating various characteristic modules, Computational and Structural Biotechnology Journal, 18, 622, 10.1016/j.csbj.2020.02.021 · DOI
- Cheng, 2021, BERTMHC: improved MHC-peptide class II interaction prediction with transformer and multiple instance learning, Bioinformatics, 37, 4172, 10.1093/bioinformatics/btab422 · DOI
- Choromanski K, Likhosherstov V, Dohan D, Song X, Gane A, Sarlos T. 2020. Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers. arXiv. https://arxiv.org/abs/2006.03555. · arXiv
- Chou, 1995, Prediction of protein structural classes, Critical Reviews in Biochemistry and Molecular Biology, 30, 275, 10.3109/10409239509083488 · DOI · Paper page
- Chou, 2020, Progresses in predicting post-translational modification, International Journal of Peptide Research and Therapeutics, 26, 873, 10.1007/s10989-019-09893-5 · DOI
- Clark K, Luong MT, Le QV. 2020. ELECTRA: Pre-Training Text Encoders as Discriminators Rather Than Generators. arXiv. https://arxiv.org/abs/2003.10555. · arXiv
- Cortés, 2011, Predicting protein distance maps according to physicochemical properties, Journal of Integrative Bioinformatics, 8, 158, 10.1515/jib-2011-181 · doi:10.1515/jib-2011-181
- Dai Z, Yang Z, Yang Y, Cohen WW, Carbonell J, Le QV. 2018. Transformer-xl: language modeling with longer-term dependency. Transformer-Xl.
- Dai Z, Yang Z, Yang Y, Carbonell J, Le QV, Salakhutdinov R. 2019. Transformer-Xl: Attentive Language Models beyond a Fixed-Length Context. arXiv. https://arxiv.org/abs/1901.02860. · arXiv
- Dehghani M, Gouws S, Vinyals O, Uszkoreit J, Kaiser Ł. 2018. Universal Transformers. arXiv. https://arxiv.org/abs/1807.03819. · arXiv
- Dick, 2018, Reciprocal perspective for improved protein-protein interaction prediction, Scientific Reports, 8, 1, 10.1038/s41598-018-30044-1 · DOI
- Dodge, 1998, The HSSP database of protein structure-sequence alignments and family profiles, Nucleic Acids Research, 26, 313, 10.1093/nar/26.1.313 · DOI · Paper page
- Du, 2021, The trrosetta server for fast and accurate protein structure prediction, Nature Protocols, 16, 5634, 10.1038/s41596-021-00628-9 · DOI · Paper page
- Ehrenberger, 2015, Computational prediction of protein-protein interactions, Methods in Molecular Biology, 1278, 57, 10.1007/978-1-4939-2425-7_4 · DOI
- ElAbd, 2020, Amino acid encoding for deep learning applications, BMC Bioinformatics, 21, 10.1186/s12859-020-03546-x · DOI
- Elman, 1990, Finding structure in time, Cognitive Science, 14, 179, 10.1207/s15516709cog1402_1 · DOI · Paper page
- Elnaggar A, Heinzinger M, Dallago C, Rehawi G, Wang Y, Jones L, Gibbs T, Feher T, Angerer C, Steinegger M, Bhowmik D, Rost B. 2020b. ProtTrans: Towards Cracking the Language of Life’s Code through Self-Supervised Deep Learning and High Performance Computing. arXiv. https://arxiv.org/abs/2007.06225. · arXiv
- Fang, 2020, A critical review of five machine learning-based algorithms for predicting protein stability changes upon mutation, Briefings in Bioinformatics, 21, 1285, 10.1093/bib/bbz071 · DOI
- Fedus W, Zoph B, Shazeer N. 2021. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv. https://arxiv.org/abs/2101.03961. · arXiv
- Finn, 2011, HMMER web server: interactive sequence similarity searching, Nucleic Acids Research, 39, W29, 10.1093/nar/gkr367 · DOI · Paper page
- Golubchik, 2007, Mind the gaps: evidence of bias in estimates of multiple sequence alignments, Molecular Biology and Evolution, 24, 2433, 10.1093/molbev/msm176 · DOI
- Google Scholar. 2022. Google Scholar. https://scholar.google.com/scholar?q=transformer+language+model+transformer+language+model&hl=en&as_sdt=0,5 [Accessed July 1, 2022].
- Gromiha MM, Nagarajan R, Selvaraj S. 2019. Protein structural bioinformatics: an overview. Protein Structural Bioinformatics.
- Guerler, 2013, Mapping monomeric threading to protein-protein structure prediction, Journal of Chemical Information and Modeling, 53, 717, 10.1021/ci300579r · DOI · Paper page
- Hanin B. 2018. Which neural net architectures give rise to exploding and vanishing gradients?. Advances in Neural Information Processing Systems.
- Hao Y, Dong L, Wei F, Xu K. 2021. Self-attention attribution: interpreting information interactions inside transformer. Proceedings of the AAAI Conference on Artificial Intelligence. · DOI · Paper page
- He L, Zhang S, Wu L, Xia H, Ju F, Zhang H. 2021. Pre-Training Co-Evolutionary Protein Representation via A Pairwise Masked Language Model. arXiv. https://arxiv.org/abs/2110.15527. · arXiv
- Heinzinger, 2021, Contrastive Learning on Protein Embeddings Enlightens Midnight Zone at Lightning Speed, 10.1101/2021.11.14.468528 · DOI · Paper page
- Heinzinger, 2022, Contrastive learning on protein embeddings enlightens midnight zone, NAR Genomics and Bioinformatics, 4, 10.1093/nargab/lqac043 · doi:10.1093/nargab/lqac043
- Hong, 2021, FastMSA: Accelerating Multiple Sequence Alignment with Dense Retrieval on Protein Language, 10.1101/2021.12.20.473431 · DOI
- Huang Z, Xu W, Yu K. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv. https://arxiv.org/abs/1508.01991. · arXiv
- Jia, 2016, ISuc-pseopt: identifying lysine succinylation sites in proteins by incorporating sequence-coupling effects into pseudo components and optimizing imbalanced training dataset, Analytical Biochemistry, 497, 48, 10.1016/j.ab.2015.12.009 · DOI · Paper page
- Jiang, 2017, Protein secondary structure prediction: A survey of the state of the art, Journal of Molecular Graphics & Modelling, 76, 379, 10.1016/j.jmgm.2017.07.015 · DOI
- Jiang T, Fang L, Wang K. 2021. MutFormer: A Context-Dependent Transformer-Based Model to Predict Pathogenic Missense Mutations. arXiv. https://arxiv.org/abs/2110.14746. · arXiv
- Jones, 1999, Protein secondary structure prediction based on position-specific scoring matrices, Journal of Molecular Biology, 292, 195, 10.1006/jmbi.1999.3091 · DOI · Paper page
- Katchalski-Katzir, 1992, Molecular surface recognition: determination of geometric fit between proteins and their ligands by correlation techniques, PNAS, 89, 2195, 10.1073/pnas.89.6.2195 · DOI · Paper page
- Khalili, 2022, Predicting protein phosphorylation sites in soybean using interpretable deep tabular learning network, Briefings in Bioinformatics, 23, 10.1093/bib/bbac015 · DOI
- Khan M, Jan B, Farman H. 2019. Deep learning: convergence to big data analytics. Deep Learning Methods and Applications. · DOI · Paper page
- Kitaev N, Kaiser Ł, Levskaya A. 2020. Reformer: The Efficient Transformer. arXiv. https://arxiv.org/abs/2001.04451. · arXiv
- Koumakis, 2020, Deep learning models in genomics; are we there yet?, Computational and Structural Biotechnology Journal, 18, 1466, 10.1016/j.csbj.2020.06.017 · DOI · Paper page
- Kryshtafovych, 2019, Critical assessment of methods of protein structure prediction (CASP)-round XIII, Proteins, 87, 1011, 10.1002/prot.25823 · DOI · Paper page
- Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. 2019. Albert: A Lite Bert for Self-Supervised Learning of Language Representations. arXiv. https://arxiv.org/abs/1909.11942. · arXiv
- Lanchantin J, Weingarten T, Sekhon A, Miller C, Qi Y. 2021. Transfer learning for predicting virus-host protein interactions for novel virus sequences. BCB ’21: Proceedings of the 12th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics. · DOI
- Laskar MTR, Huang X, Hoque E. 2020. Contextualized embeddings based transformer encoder for sentence similarity modeling in answer selection task. Proceedings of The 12th Language Resources and Evaluation Conference.
- Levy O, Goldberg Y. 2014. Dependency-based word embeddings. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.
- Li, 2022, Positive-unlabeled learning in bioinformatics and computational biology: a brief review, Briefings in Bioinformatics, 23, 10.1093/bib/bbab461 · DOI
- Liu, 2007, BindingDB: a web-accessible database of experimentally determined protein-ligand binding affinities, Nucleic Acids Research, 35, D198, 10.1093/nar/gkl999 · DOI · Paper page
- Liu, 2012, Using amino acid physicochemical distance transformation for fast protein remote homology detection, PLOS ONE, 7, 10.1371/journal.pone.0046633 · doi:10.1371/journal.pone.0046633
- Liu, 2020, DeepSVM-fold: protein fold recognition by combining support vector machines and pairwise sequence similarity scores generated by deep learning networks, Briefings in Bioinformatics, 21, 1733, 10.1093/bib/bbz098 · DOI
- López, 2017, SucStruct: prediction of succinylated lysine residues by using structural properties of amino acids, Analytical Biochemistry, 527, 24, 10.1016/j.ab.2017.03.021 · DOI
- Lopez MJ, Mohiuddin SS. 2020. Biochemistry, essential amino acids. Biochemistry.
- Lu T, Lu AX, Moses AM. 2021. Random Embeddings and Linear Regression Can Predict Protein Function. arXiv. https://arxiv.org/abs/2104.14661. · arXiv
- McDowall, 2009, Pips: human protein-protein interaction prediction database, Nucleic Acids Research, 37, D651, 10.1093/nar/gkn870 · DOI · Paper page
- Mikolov T, Kombrink S, Burget L, Černocký J. 2011. Extensions of recurrent neural network language model. 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP); 2011: IEEE. · DOI · Paper page
- Mikolov T, Chen K, Corrado GS. 2013a. International Conference on Learning Representations. Efficient Estimation of Word Representations in Vector Space.
- Mistry, 2021, Pfam: the protein families database in 2021, Nucleic Acids Research, 49, D412, 10.1093/nar/gkaa913 · DOI · Paper page
- Nambiar A, Heflin M, Liu S, Maslov S, Hopkins M, Ritz A. 2020. Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks. BCB ’20: Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. · DOI
- Nauman, 2019, Beyond homology transfer: deep learning for automated annotation of proteins, Journal of Grid Computing, 17, 225, 10.1007/s10723-018-9450-6 · DOI
- Ofer, 2021, The language of proteins: Nlp, machine learning & protein sequences, Computational and Structural Biotechnology Journal, 19, 1750, 10.1016/j.csbj.2021.03.022 · DOI · Paper page
- Öztürk, 2018, DeepDTA: deep drug-target binding affinity prediction, Bioinformatics, 34, i821, 10.1093/bioinformatics/bty593 · DOI · Paper page
- Pan, 2022, Systematic evaluation of computational tools to predict the effects of mutations on protein stability in the absence of experimental structures, Briefings in Bioinformatics, 23, 10.1093/bib/bbac025 · DOI
- Pascanu R, Mikolov T, Bengio Y. 2013. On the difficulty of training recurrent neural networks. International conference on machine learning; 2013: PMLR.
- Phuong, 2006, Multiple alignment of protein sequences with repeats and rearrangements, Nucleic Acids Research, 34, 5932, 10.1093/nar/gkl511 · DOI
- Qiao, 2022, BERT-kcr: prediction of lysine crotonylation sites by a transfer learning method with pre-trained BERT models, Bioinformatics, 38, 648, 10.1093/bioinformatics/btab712 · DOI
- Raghu M, Unterthiner T, Kornblith S, Zhang C, Dosovitskiy A. 2021. Do vision transformers see like convolutional neural networks. Advances in Neural Information Processing Systems.
- Ramazi, 2021, Posttranslational modifications in proteins: resources, tools and prediction methods, Database, 2021, 10.1093/database/baab012 · DOI · Paper page
- Rao RM, Liu J, Verkuil R, Meier J, Canny J. 2021. MSA transformer. International Conference on Machine Learning; 2021: PMLR.
- Saethang, 2016, A machine learning strategy for predicting localization of post-translational modification sites in protein-protein interacting regions, BMC Bioinformatics, 17, 10.1186/s12859-016-1165-8 · DOI
- Schmiedel, 2019, Determining protein structures using deep mutagenesis, Nature Genetics, 51, 1177, 10.1038/s41588-019-0431-x · DOI
- Sharma, 2013, A feature extraction technique using bi-gram probabilities of position specific scoring matrix for protein fold recognition, Journal of Theoretical Biology, 320, 41, 10.1016/j.jtbi.2012.12.008 · DOI
- Shi, 2021, Deep learning for mining protein data, Briefings in Bioinformatics, 22, 194, 10.1093/bib/bbz156 · DOI
- Singh, 2021, SPOT-1D-single: improving the single-sequence-based prediction of protein secondary structure, backbone angles, solvent accessibility and half-sphere exposures using a large training set and ensembled deep learning, Bioinformatics, 37, 3464, 10.1093/bioinformatics/btab316 · DOI
- Singh, 2022, SPOT-contact-LM: improving single-sequence-based prediction of protein contact MAP using a transformer language model, Bioinformatics, 38, 1888, 10.1093/bioinformatics/btac053 · DOI
- Smith S, Patwary M, Norick B, LeGresley P, Rajbhandari S, Casper J. 2022. Using Deepspeed and Megatron to Train Megatron-Turing Nlg 530b, a Large-Scale Generative Language Model. arXiv. https://arxiv.org/abs/2201.11990. · arXiv
- Sourkov V. 2018. Igloo: Slicing the Features Space to Represent Sequences. arXiv. https://arxiv.org/abs/1807.03402. · arXiv
- Sturmfels P, Vig J, Madani A, Rajani NF. 2020. Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models. arXiv. https://arxiv.org/abs/2012.00195. · arXiv
- Sułkowska, 2012, Genomics-aided structure prediction, PNAS, 109, 10340, 10.1073/pnas.1207864109 · DOI · Paper page
- Tang, 2014, Making sense of large-scale kinase inhibitor bioactivity data sets: a comparative and integrative analysis, Journal of Chemical Information and Modeling, 54, 735, 10.1021/ci400709d · DOI · Paper page
- Tavares, 2013, Strategies and molecular tools to fight antimicrobial resistance: resistome, transcriptome, and antimicrobial peptides, Frontiers in Microbiology, 4, 10.3389/fmicb.2013.00412 · DOI
- Tran K, Bisazza A, Monz C. 2018. The Importance of Being Recurrent for Modeling Hierarchical Structure. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. · DOI · Paper page
- Trost, 2013, Computational phosphorylation site prediction in plants using random forests and organism-specific instance weights, Bioinformatics, 29, 686, 10.1093/bioinformatics/btt031 · DOI
- Turian J, Ratinov L. 2010. Word representations: a simple and general method for semi-supervised learning. Proceedings of the 48th annual meeting of the association for computational linguistics.
- UniProt, 2021, UniProt: the universal protein knowledgebase in 2021, Nucleic Acids Research, 49, D480, 10.1093/nar/gkaa1100 · DOI · Paper page
- Van der Maaten, 2008, Visualizing data using t-SNE, Journal of Machine Learning Research, 9
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN. 2017. Attention is all you need. Advances in Neural Information Processing Systems.
- Väth, 2022, PROVAL: A framework for comparison of protein sequence embeddings, Journal of Computational Mathematics and Data Science, 2022, 10.1016/j.jcmds.2022.100044 · DOI
- Vig J. 2019a. BertViz: A tool for visualizing multihead self-attention in the BERT model. ICLR Workshop: Debugging Machine Learning Models.
- Vig J, Madani A, Varshney LR, Xiong C, Socher R, Rajani NF. 2020. Bertology Meets Biology: Interpreting Attention in Protein Language Models. arXiv. https://arxiv.org/abs/2006.15222. · arXiv
- Walls, 2017, Protein Stability: Enhancement and Measurement, 101, 10.1007/978-1-4939-6412-3 · doi:10.1007/978-1-4939-6412-3
- Wang, 2017, Prediction of phosphorylation sites based on krawtchouk image moments, Proteins, 85, 2231, 10.1002/prot.25388 · DOI
- Wang, 2020, MusiteDeep: a deep-learning based Webserver for protein post-translational modification site prediction and visualization, Nucleic Acids Research, 48, W140, 10.1093/nar/gkaa275 · DOI · Paper page
- wang, 2021, ELECTRA-DTA: A new compound-protein binding affinity prediction model based on the contextualized sequence encoding, Journal of Cheminformatics, 14, 10.1186/s13321-022-00591-x · DOI
- Wang, 2021, There once was a really bad poet, it was automated but you did ’'t know it, Transactions of the Association for Computational Linguistics, 9, 605, 10.1162/tacl_a_00387 · DOI
- Wang, 2022, Prior knowledge facilitates low homologous protein secondary structure prediction with DSM distillation, Bioinformatics, 38, 3574, 10.1093/bioinformatics/btac351 · DOI
- Wettig A, Gao T, Zhong Z, Mask C. 2022. Should You Mask 15% in Masked Language Modeling?. arXiv. https://arxiv.org/abs/2202.08005. · arXiv
- Wilburn, 2020, Remote homology search with hidden Potts models, PLOS Computational Biology, 16, 10.1371/journal.pcbi.1008085 · DOI
- Xu, 2018, PhosContext2vec: a distributed representation of residue-level sequence contexts and its application to general and kinase-specific phosphorylation site prediction, Scientific Reports, 8, 10.1038/s41598-018-26392-7 · DOI
- Xue Y, Liu Z, Fang X. 2022. Multimodal Pre-Training Model for Sequence-based Prediction of Protein-Protein Interaction. Machine Learning in Computational Biology; 2022: PMLR.
- Yamaguchi, 2021, Evotuning protocols for transformer-based variant effect prediction on multi-domain proteins, Briefings in Bioinformatics, 22, 10.1093/bib/bbab234 · DOI
- Yin W, Kann K, Yu M, Schütze H. 2017. Comparative Study of CNN and RNN for Natural Language Processing. arXiv. https://arxiv.org/abs/1702.01923. · arXiv
- Yu, 2017, Protein sequence comparison based on physicochemical properties and the position-feature energy matrix, Scientific Reports, 7, 10.1038/srep46237 · DOI
- Zaheer M, Guruganesh G, Dubey KA, Ainslie J, Alberti C, Ontanon S. 2020. Big bird: transformers for longer sequences. Advances in Neural Information Processing Systems.
- Zare-Mirakabad, 2021, Protein Sequence Profile Prediction Using Protalbert Transformer1, 10.1101/2021.09.23.461475 · DOI
- Zeng, 2016, Convolutional neural network architectures for predicting DNA-protein binding, Bioinformatics, 32, i121, 10.1093/bioinformatics/btw255 · DOI · Paper page
- Zhang H, Ju F, Zhu J, He L, Shao B, Zheng N. 2021. Co-evolution Transformer for Protein Contact Prediction. Advances in Neural Information Processing Systems.
- Zhao Q, Ma J, Wang Y, Xie F, Lv Z, Xu Y. 2021. Mul-SNO: A novel prediction tool for S-nitrosylation sites based on deep learning methods. IEEE Journal of Biomedical and Health Informatics. · DOI
Cited by
Protein codes promote selective subcellular compartmentalization
Kilgore, H. R., Chinn, I., Mikhael, P. G., Mitnikov, I., Van Dongen, C., Zylberberg, G., Afeyan, L., Banani, S. F., Wilson-Hawken, S., Lee, T. I., et al. (2025) · Science · doi:10.1126/science.adq2634 · Cited by 99
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75
Medium-sized protein language models perform well at transfer learning on realistic datasets
Vieira, L. C., Handojo, M. L., & Wilke, C. O. (2025) · Scientific Reports · doi:10.1038/s41598-025-05674-x · Cited by 41
ESM ‐scan—A tool to guide amino acid substitutions
Totaro, M. G., Vide, U., Zausinger, R., Winkler, A., & Oberdorfer, G. (2024) · Protein Science · doi:10.1002/pro.5221 · Cited by 19
Kong, S. W., Lee, I.-H., Collen, L. V., Field, M., Manrai, A. K., Snapper, S. B., & Mandl, K. D. (2025) · npj Genomic Medicine · doi:10.1038/s41525-025-00480-w · Cited by 18
Metadata
- Authors
- Chandra, A., Tünnermann, L., Löfstedt, T., & Gratz, R.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2023-01-18
- Volume
- 12
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W4317212783
- Cited by
- 184
- References
- 42
- FWCI
- 13.75
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- creativecommons.org/licenses/by/4.0/ (am)
- creativecommons.org/licenses/by/4.0/ (tdm)
- Funders
- Kempestiftelserna (JCK-2144, JCK-2015.1)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Transformer-based deep learning for predicting protein properties in the life sciences. https://app.cochat.ai/papers/10.7554/elife.82819
- The paper
- Chandra, A., Tünnermann, L., Löfstedt, T., & Gratz, R. (2023). Transformer-based deep learning for predicting protein properties in the life sciences. eLife. https://doi.org/10.7554/elife.82819
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.