Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 184

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Transformer-based deep learning for predicting protein properties in the life sciences

Open access
Chandra, Abel, Tünnermann, Laura, Löfstedt, Tommy, Gratz, Regina·2023·eLife

Abstract

Recent developments in deep learning, coupled with an increasing number of sequenced proteins, have led to a breakthrough in life science applications, in particular in protein property prediction. There is hope that deep learning can close the gap between the number of sequenced proteins and proteins with known properties based on lab experiments. Language models from the field of natural language processing have gained popularity for protein property predictions and have led to a new computational revolution in biology, where old prediction results are being improved regularly. Such models can learn useful multipurpose representations of proteins from large open repositories of protein sequences and can be used, for instance, to predict protein properties. The field of natural language processing is growing quickly because of developments in a class of models based on a particular model—the Transformer model. We review recent developments and the use of large-scale Transformer models in applications for predicting protein characteristics and how such models can be used to predict, for example, post-translational modifications. We review shortcomings of other deep learning models and explain how the Transformer models have quickly proven to be a very promising way to unravel information hidden in the sequences of amino acids.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.82819

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 42 references

Research works printed but not linked: 121

  1. Alanis-Lobato, 2016, HIPPIE v2. 0: enhancing meaningfulness and reliability of protein–protein interaction networks, Nucleic Acids Research, 45, D408, 10.1093/nar/gkw985 · DOI · Paper page
  2. Albawi S, Mohammed TA, Al-Zawi S. 2017. Understanding of a convolutional neural network. 2017 International Conference on Engineering and Technology (ICET. · DOI · Paper page
  3. Altschul, 1997, Gapped blast and PSI-BLAST: a new generation of protein database search programs, Nucleic Acids Research, 25, 3389, 10.1093/nar/25.17.3389 · DOI · Paper page
  4. Ammari MG, Gresham CR, McCarthy FM, Nanduri B. 2016. HPIDB 2.0: a curated database for host–pathogen interactions. Database.
  5. Behjati, 2022, Protein sequence profile prediction using protalbert transformer, Computational Biology and Chemistry, 99, 10.1016/j.compbiolchem.2022.107717 · DOI
  6. Bergstra J, Yamins D, Cox D. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. International conference on machine learning; 2013: PMLR.
  7. Brandes, 2021, ProteinBERT: A Universal Deep-Learning Model of Protein Sequence and Function, 10.1101/2021.05.24.445464 · doi:10.1101/2021.05.24.445464
  8. Britz D, Goldie A, Luong MT, Le Q. 2017. Massive Exploration of Neural Machine Translation Architectures. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. · DOI · Paper page
  9. Bulcun, 2012, Disorders of glucose metabolism and insulin resistance in patients with obstructive sleep apnoea syndrome, International Journal of Clinical Practice, 66, 91, 10.1111/j.1742-1241.2011.02795.x · DOI
  10. Cai, 2021, MSA-regularized protein sequence transformer toward predicting genome-wide chemical-protein interactions: application to gpcrome deorphanization, Journal of Chemical Information and Modeling, 61, 1570, 10.1021/acs.jcim.0c01285 · DOI
  11. Chan, 2015, Glass: a comprehensive database for experimentally validated GPCR-ligand associations, Bioinformatics, 31, 3035, 10.1093/bioinformatics/btv302 · DOI
  12. Chandra, 2019, Bigram-PGK: phosphoglycerylation prediction using the technique of bigram probabilities of position specific scoring matrix, BMC Molecular and Cell Biology, 20, 10.1186/s12860-019-0240-1 · DOI
  13. Chandra, 2020, RAM-PGK: prediction of lysine phosphoglycerylation based on residue adjacency matrix, Genes, 11, 10.3390/genes11121524 · DOI
  14. Charte, 2019, A snapshot on nonstandard supervised learning problems: taxonomy, relationships, problem transformations and algorithm adaptations, Progress in Artificial Intelligence, 8, 1, 10.1007/s13748-018-00167-7 · DOI
  15. Chen, 2002, The binding database: data management and interface design, Bioinformatics, 18, 130, 10.1093/bioinformatics/18.1.130 · DOI · Paper page
  16. Chen, 2020, IStable 2.0: predicting protein thermal stability changes by integrating various characteristic modules, Computational and Structural Biotechnology Journal, 18, 622, 10.1016/j.csbj.2020.02.021 · DOI
  17. Cheng, 2021, BERTMHC: improved MHC-peptide class II interaction prediction with transformer and multiple instance learning, Bioinformatics, 37, 4172, 10.1093/bioinformatics/btab422 · DOI
  18. Choromanski K, Likhosherstov V, Dohan D, Song X, Gane A, Sarlos T. 2020. Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers. arXiv. https://arxiv.org/abs/2006.03555. · arXiv
  19. Chou, 1995, Prediction of protein structural classes, Critical Reviews in Biochemistry and Molecular Biology, 30, 275, 10.3109/10409239509083488 · DOI · Paper page
  20. Chou, 2020, Progresses in predicting post-translational modification, International Journal of Peptide Research and Therapeutics, 26, 873, 10.1007/s10989-019-09893-5 · DOI
  21. Clark K, Luong MT, Le QV. 2020. ELECTRA: Pre-Training Text Encoders as Discriminators Rather Than Generators. arXiv. https://arxiv.org/abs/2003.10555. · arXiv
  22. Cortés, 2011, Predicting protein distance maps according to physicochemical properties, Journal of Integrative Bioinformatics, 8, 158, 10.1515/jib-2011-181 · doi:10.1515/jib-2011-181
  23. Dai Z, Yang Z, Yang Y, Cohen WW, Carbonell J, Le QV. 2018. Transformer-xl: language modeling with longer-term dependency. Transformer-Xl.
  24. Dai Z, Yang Z, Yang Y, Carbonell J, Le QV, Salakhutdinov R. 2019. Transformer-Xl: Attentive Language Models beyond a Fixed-Length Context. arXiv. https://arxiv.org/abs/1901.02860. · arXiv
  25. Dehghani M, Gouws S, Vinyals O, Uszkoreit J, Kaiser Ł. 2018. Universal Transformers. arXiv. https://arxiv.org/abs/1807.03819. · arXiv
  26. Dick, 2018, Reciprocal perspective for improved protein-protein interaction prediction, Scientific Reports, 8, 1, 10.1038/s41598-018-30044-1 · DOI
  27. Dodge, 1998, The HSSP database of protein structure-sequence alignments and family profiles, Nucleic Acids Research, 26, 313, 10.1093/nar/26.1.313 · DOI · Paper page
  28. Du, 2021, The trrosetta server for fast and accurate protein structure prediction, Nature Protocols, 16, 5634, 10.1038/s41596-021-00628-9 · DOI · Paper page
  29. Ehrenberger, 2015, Computational prediction of protein-protein interactions, Methods in Molecular Biology, 1278, 57, 10.1007/978-1-4939-2425-7_4 · DOI
  30. ElAbd, 2020, Amino acid encoding for deep learning applications, BMC Bioinformatics, 21, 10.1186/s12859-020-03546-x · DOI
  31. Elman, 1990, Finding structure in time, Cognitive Science, 14, 179, 10.1207/s15516709cog1402_1 · DOI · Paper page
  32. Elnaggar A, Heinzinger M, Dallago C, Rehawi G, Wang Y, Jones L, Gibbs T, Feher T, Angerer C, Steinegger M, Bhowmik D, Rost B. 2020b. ProtTrans: Towards Cracking the Language of Life’s Code through Self-Supervised Deep Learning and High Performance Computing. arXiv. https://arxiv.org/abs/2007.06225. · arXiv
  33. Fang, 2020, A critical review of five machine learning-based algorithms for predicting protein stability changes upon mutation, Briefings in Bioinformatics, 21, 1285, 10.1093/bib/bbz071 · DOI
  34. Fedus W, Zoph B, Shazeer N. 2021. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv. https://arxiv.org/abs/2101.03961. · arXiv
  35. Finn, 2011, HMMER web server: interactive sequence similarity searching, Nucleic Acids Research, 39, W29, 10.1093/nar/gkr367 · DOI · Paper page
  36. Golubchik, 2007, Mind the gaps: evidence of bias in estimates of multiple sequence alignments, Molecular Biology and Evolution, 24, 2433, 10.1093/molbev/msm176 · DOI
  37. Google Scholar. 2022. Google Scholar. https://scholar.google.com/scholar?q=transformer+language+model+transformer+language+model&hl=en&as_sdt=0,5 [Accessed July 1, 2022].
  38. Gromiha MM, Nagarajan R, Selvaraj S. 2019. Protein structural bioinformatics: an overview. Protein Structural Bioinformatics.
  39. Guerler, 2013, Mapping monomeric threading to protein-protein structure prediction, Journal of Chemical Information and Modeling, 53, 717, 10.1021/ci300579r · DOI · Paper page
  40. Hanin B. 2018. Which neural net architectures give rise to exploding and vanishing gradients?. Advances in Neural Information Processing Systems.
  41. Hao Y, Dong L, Wei F, Xu K. 2021. Self-attention attribution: interpreting information interactions inside transformer. Proceedings of the AAAI Conference on Artificial Intelligence. · DOI · Paper page
  42. He L, Zhang S, Wu L, Xia H, Ju F, Zhang H. 2021. Pre-Training Co-Evolutionary Protein Representation via A Pairwise Masked Language Model. arXiv. https://arxiv.org/abs/2110.15527. · arXiv
  43. Heinzinger, 2021, Contrastive Learning on Protein Embeddings Enlightens Midnight Zone at Lightning Speed, 10.1101/2021.11.14.468528 · DOI · Paper page
  44. Heinzinger, 2022, Contrastive learning on protein embeddings enlightens midnight zone, NAR Genomics and Bioinformatics, 4, 10.1093/nargab/lqac043 · doi:10.1093/nargab/lqac043
  45. Hong, 2021, FastMSA: Accelerating Multiple Sequence Alignment with Dense Retrieval on Protein Language, 10.1101/2021.12.20.473431 · DOI
  46. Huang Z, Xu W, Yu K. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv. https://arxiv.org/abs/1508.01991. · arXiv
  47. Jia, 2016, ISuc-pseopt: identifying lysine succinylation sites in proteins by incorporating sequence-coupling effects into pseudo components and optimizing imbalanced training dataset, Analytical Biochemistry, 497, 48, 10.1016/j.ab.2015.12.009 · DOI · Paper page
  48. Jiang, 2017, Protein secondary structure prediction: A survey of the state of the art, Journal of Molecular Graphics & Modelling, 76, 379, 10.1016/j.jmgm.2017.07.015 · DOI
  49. Jiang T, Fang L, Wang K. 2021. MutFormer: A Context-Dependent Transformer-Based Model to Predict Pathogenic Missense Mutations. arXiv. https://arxiv.org/abs/2110.14746. · arXiv
  50. Jones, 1999, Protein secondary structure prediction based on position-specific scoring matrices, Journal of Molecular Biology, 292, 195, 10.1006/jmbi.1999.3091 · DOI · Paper page
  51. Katchalski-Katzir, 1992, Molecular surface recognition: determination of geometric fit between proteins and their ligands by correlation techniques, PNAS, 89, 2195, 10.1073/pnas.89.6.2195 · DOI · Paper page
  52. Khalili, 2022, Predicting protein phosphorylation sites in soybean using interpretable deep tabular learning network, Briefings in Bioinformatics, 23, 10.1093/bib/bbac015 · DOI
  53. Khan M, Jan B, Farman H. 2019. Deep learning: convergence to big data analytics. Deep Learning Methods and Applications. · DOI · Paper page
  54. Kitaev N, Kaiser Ł, Levskaya A. 2020. Reformer: The Efficient Transformer. arXiv. https://arxiv.org/abs/2001.04451. · arXiv
  55. Koumakis, 2020, Deep learning models in genomics; are we there yet?, Computational and Structural Biotechnology Journal, 18, 1466, 10.1016/j.csbj.2020.06.017 · DOI · Paper page
  56. Kryshtafovych, 2019, Critical assessment of methods of protein structure prediction (CASP)-round XIII, Proteins, 87, 1011, 10.1002/prot.25823 · DOI · Paper page
  57. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. 2019. Albert: A Lite Bert for Self-Supervised Learning of Language Representations. arXiv. https://arxiv.org/abs/1909.11942. · arXiv
  58. Lanchantin J, Weingarten T, Sekhon A, Miller C, Qi Y. 2021. Transfer learning for predicting virus-host protein interactions for novel virus sequences. BCB ’21: Proceedings of the 12th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics. · DOI
  59. Laskar MTR, Huang X, Hoque E. 2020. Contextualized embeddings based transformer encoder for sentence similarity modeling in answer selection task. Proceedings of The 12th Language Resources and Evaluation Conference.
  60. Levy O, Goldberg Y. 2014. Dependency-based word embeddings. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.
  61. Li, 2022, Positive-unlabeled learning in bioinformatics and computational biology: a brief review, Briefings in Bioinformatics, 23, 10.1093/bib/bbab461 · DOI
  62. Liu, 2007, BindingDB: a web-accessible database of experimentally determined protein-ligand binding affinities, Nucleic Acids Research, 35, D198, 10.1093/nar/gkl999 · DOI · Paper page
  63. Liu, 2012, Using amino acid physicochemical distance transformation for fast protein remote homology detection, PLOS ONE, 7, 10.1371/journal.pone.0046633 · doi:10.1371/journal.pone.0046633
  64. Liu, 2020, DeepSVM-fold: protein fold recognition by combining support vector machines and pairwise sequence similarity scores generated by deep learning networks, Briefings in Bioinformatics, 21, 1733, 10.1093/bib/bbz098 · DOI
  65. López, 2017, SucStruct: prediction of succinylated lysine residues by using structural properties of amino acids, Analytical Biochemistry, 527, 24, 10.1016/j.ab.2017.03.021 · DOI
  66. Lopez MJ, Mohiuddin SS. 2020. Biochemistry, essential amino acids. Biochemistry.
  67. Lu T, Lu AX, Moses AM. 2021. Random Embeddings and Linear Regression Can Predict Protein Function. arXiv. https://arxiv.org/abs/2104.14661. · arXiv
  68. McDowall, 2009, Pips: human protein-protein interaction prediction database, Nucleic Acids Research, 37, D651, 10.1093/nar/gkn870 · DOI · Paper page
  69. Mikolov T, Kombrink S, Burget L, Černocký J. 2011. Extensions of recurrent neural network language model. 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP); 2011: IEEE. · DOI · Paper page
  70. Mikolov T, Chen K, Corrado GS. 2013a. International Conference on Learning Representations. Efficient Estimation of Word Representations in Vector Space.
  71. Mistry, 2021, Pfam: the protein families database in 2021, Nucleic Acids Research, 49, D412, 10.1093/nar/gkaa913 · DOI · Paper page
  72. Nambiar A, Heflin M, Liu S, Maslov S, Hopkins M, Ritz A. 2020. Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks. BCB ’20: Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. · DOI
  73. Nauman, 2019, Beyond homology transfer: deep learning for automated annotation of proteins, Journal of Grid Computing, 17, 225, 10.1007/s10723-018-9450-6 · DOI
  74. Ofer, 2021, The language of proteins: Nlp, machine learning & protein sequences, Computational and Structural Biotechnology Journal, 19, 1750, 10.1016/j.csbj.2021.03.022 · DOI · Paper page
  75. Öztürk, 2018, DeepDTA: deep drug-target binding affinity prediction, Bioinformatics, 34, i821, 10.1093/bioinformatics/bty593 · DOI · Paper page
  76. Pan, 2022, Systematic evaluation of computational tools to predict the effects of mutations on protein stability in the absence of experimental structures, Briefings in Bioinformatics, 23, 10.1093/bib/bbac025 · DOI
  77. Pascanu R, Mikolov T, Bengio Y. 2013. On the difficulty of training recurrent neural networks. International conference on machine learning; 2013: PMLR.
  78. Phuong, 2006, Multiple alignment of protein sequences with repeats and rearrangements, Nucleic Acids Research, 34, 5932, 10.1093/nar/gkl511 · DOI
  79. Qiao, 2022, BERT-kcr: prediction of lysine crotonylation sites by a transfer learning method with pre-trained BERT models, Bioinformatics, 38, 648, 10.1093/bioinformatics/btab712 · DOI
  80. Raghu M, Unterthiner T, Kornblith S, Zhang C, Dosovitskiy A. 2021. Do vision transformers see like convolutional neural networks. Advances in Neural Information Processing Systems.
  81. Ramazi, 2021, Posttranslational modifications in proteins: resources, tools and prediction methods, Database, 2021, 10.1093/database/baab012 · DOI · Paper page
  82. Rao RM, Liu J, Verkuil R, Meier J, Canny J. 2021. MSA transformer. International Conference on Machine Learning; 2021: PMLR.
  83. Saethang, 2016, A machine learning strategy for predicting localization of post-translational modification sites in protein-protein interacting regions, BMC Bioinformatics, 17, 10.1186/s12859-016-1165-8 · DOI
  84. Schmiedel, 2019, Determining protein structures using deep mutagenesis, Nature Genetics, 51, 1177, 10.1038/s41588-019-0431-x · DOI
  85. Sharma, 2013, A feature extraction technique using bi-gram probabilities of position specific scoring matrix for protein fold recognition, Journal of Theoretical Biology, 320, 41, 10.1016/j.jtbi.2012.12.008 · DOI
  86. Shi, 2021, Deep learning for mining protein data, Briefings in Bioinformatics, 22, 194, 10.1093/bib/bbz156 · DOI
  87. Singh, 2021, SPOT-1D-single: improving the single-sequence-based prediction of protein secondary structure, backbone angles, solvent accessibility and half-sphere exposures using a large training set and ensembled deep learning, Bioinformatics, 37, 3464, 10.1093/bioinformatics/btab316 · DOI
  88. Singh, 2022, SPOT-contact-LM: improving single-sequence-based prediction of protein contact MAP using a transformer language model, Bioinformatics, 38, 1888, 10.1093/bioinformatics/btac053 · DOI
  89. Smith S, Patwary M, Norick B, LeGresley P, Rajbhandari S, Casper J. 2022. Using Deepspeed and Megatron to Train Megatron-Turing Nlg 530b, a Large-Scale Generative Language Model. arXiv. https://arxiv.org/abs/2201.11990. · arXiv
  90. Sourkov V. 2018. Igloo: Slicing the Features Space to Represent Sequences. arXiv. https://arxiv.org/abs/1807.03402. · arXiv
  91. Sturmfels P, Vig J, Madani A, Rajani NF. 2020. Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models. arXiv. https://arxiv.org/abs/2012.00195. · arXiv
  92. Sułkowska, 2012, Genomics-aided structure prediction, PNAS, 109, 10340, 10.1073/pnas.1207864109 · DOI · Paper page
  93. Tang, 2014, Making sense of large-scale kinase inhibitor bioactivity data sets: a comparative and integrative analysis, Journal of Chemical Information and Modeling, 54, 735, 10.1021/ci400709d · DOI · Paper page
  94. Tavares, 2013, Strategies and molecular tools to fight antimicrobial resistance: resistome, transcriptome, and antimicrobial peptides, Frontiers in Microbiology, 4, 10.3389/fmicb.2013.00412 · DOI
  95. Tran K, Bisazza A, Monz C. 2018. The Importance of Being Recurrent for Modeling Hierarchical Structure. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. · DOI · Paper page
  96. Trost, 2013, Computational phosphorylation site prediction in plants using random forests and organism-specific instance weights, Bioinformatics, 29, 686, 10.1093/bioinformatics/btt031 · DOI
  97. Turian J, Ratinov L. 2010. Word representations: a simple and general method for semi-supervised learning. Proceedings of the 48th annual meeting of the association for computational linguistics.
  98. UniProt, 2021, UniProt: the universal protein knowledgebase in 2021, Nucleic Acids Research, 49, D480, 10.1093/nar/gkaa1100 · DOI · Paper page
  99. Van der Maaten, 2008, Visualizing data using t-SNE, Journal of Machine Learning Research, 9
  100. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN. 2017. Attention is all you need. Advances in Neural Information Processing Systems.
  101. Väth, 2022, PROVAL: A framework for comparison of protein sequence embeddings, Journal of Computational Mathematics and Data Science, 2022, 10.1016/j.jcmds.2022.100044 · DOI
  102. Vig J. 2019a. BertViz: A tool for visualizing multihead self-attention in the BERT model. ICLR Workshop: Debugging Machine Learning Models.
  103. Vig J, Madani A, Varshney LR, Xiong C, Socher R, Rajani NF. 2020. Bertology Meets Biology: Interpreting Attention in Protein Language Models. arXiv. https://arxiv.org/abs/2006.15222. · arXiv
  104. Walls, 2017, Protein Stability: Enhancement and Measurement, 101, 10.1007/978-1-4939-6412-3 · doi:10.1007/978-1-4939-6412-3
  105. Wang, 2017, Prediction of phosphorylation sites based on krawtchouk image moments, Proteins, 85, 2231, 10.1002/prot.25388 · DOI
  106. Wang, 2020, MusiteDeep: a deep-learning based Webserver for protein post-translational modification site prediction and visualization, Nucleic Acids Research, 48, W140, 10.1093/nar/gkaa275 · DOI · Paper page
  107. wang, 2021, ELECTRA-DTA: A new compound-protein binding affinity prediction model based on the contextualized sequence encoding, Journal of Cheminformatics, 14, 10.1186/s13321-022-00591-x · DOI
  108. Wang, 2021, There once was a really bad poet, it was automated but you did ’'t know it, Transactions of the Association for Computational Linguistics, 9, 605, 10.1162/tacl_a_00387 · DOI
  109. Wang, 2022, Prior knowledge facilitates low homologous protein secondary structure prediction with DSM distillation, Bioinformatics, 38, 3574, 10.1093/bioinformatics/btac351 · DOI
  110. Wettig A, Gao T, Zhong Z, Mask C. 2022. Should You Mask 15% in Masked Language Modeling?. arXiv. https://arxiv.org/abs/2202.08005. · arXiv
  111. Wilburn, 2020, Remote homology search with hidden Potts models, PLOS Computational Biology, 16, 10.1371/journal.pcbi.1008085 · DOI
  112. Xu, 2018, PhosContext2vec: a distributed representation of residue-level sequence contexts and its application to general and kinase-specific phosphorylation site prediction, Scientific Reports, 8, 10.1038/s41598-018-26392-7 · DOI
  113. Xue Y, Liu Z, Fang X. 2022. Multimodal Pre-Training Model for Sequence-based Prediction of Protein-Protein Interaction. Machine Learning in Computational Biology; 2022: PMLR.
  114. Yamaguchi, 2021, Evotuning protocols for transformer-based variant effect prediction on multi-domain proteins, Briefings in Bioinformatics, 22, 10.1093/bib/bbab234 · DOI
  115. Yin W, Kann K, Yu M, Schütze H. 2017. Comparative Study of CNN and RNN for Natural Language Processing. arXiv. https://arxiv.org/abs/1702.01923. · arXiv
  116. Yu, 2017, Protein sequence comparison based on physicochemical properties and the position-feature energy matrix, Scientific Reports, 7, 10.1038/srep46237 · DOI
  117. Zaheer M, Guruganesh G, Dubey KA, Ainslie J, Alberti C, Ontanon S. 2020. Big bird: transformers for longer sequences. Advances in Neural Information Processing Systems.
  118. Zare-Mirakabad, 2021, Protein Sequence Profile Prediction Using Protalbert Transformer1, 10.1101/2021.09.23.461475 · DOI
  119. Zeng, 2016, Convolutional neural network architectures for predicting DNA-protein binding, Bioinformatics, 32, i121, 10.1093/bioinformatics/btw255 · DOI · Paper page
  120. Zhang H, Ju F, Zhu J, He L, Shao B, Zheng N. 2021. Co-evolution Transformer for Protein Contact Prediction. Advances in Neural Information Processing Systems.
  121. Zhao Q, Ma J, Wang Y, Xie F, Lv Z, Xu Y. 2021. Mul-SNO: A novel prediction tool for S-nitrosylation sites based on deep learning methods. IEEE Journal of Biomedical and Health Informatics. · DOI

Cited by

See all 184

Metadata

Authors
Chandra, A., Tünnermann, L., Löfstedt, T., & Gratz, R.
Venue
eLife
Published
2023-01-18
Volume
12
ISSN
2050-084X (ISSN-L)
OpenAlex
W4317212783
Cited by
184
References
42
FWCI
13.75
Type
journal article
Language
en
Open access
gold
Funders
Kempestiftelserna (JCK-2144, JCK-2015.1)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Transformer-based deep learning for predicting protein properties in the life sciences. https://app.cochat.ai/papers/10.7554/elife.82819
The paper
Chandra, A., Tünnermann, L., Löfstedt, T., & Gratz, R. (2023). Transformer-based deep learning for predicting protein properties in the life sciences. eLife. https://doi.org/10.7554/elife.82819

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.