Exploring the Knowledge of an Outstanding Protein to Protein Interaction Transformer
Open accessPublished version of a preprint
Abstract
Protein-to-protein interaction (PPI) prediction aims to predict whether two given proteins interact or not. Compared with traditional experimental methods of high cost and low efficiency, the current deep learning based approach makes it possible to discover massive potential PPIs from large-scale databases. However, deep PPI prediction models perform poorly on unseen species, as their proteins are not in the training set. Targetting on this issue, the paper first proposes PPITrans, a Transformer based PPI prediction model that exploits a language model pre-trained on proteins to conduct binary PPI prediction. To validate the effectiveness on unseen species, PPITrans is trained with Human PPIs and tested on PPIs of other species. Experimental results show that PPITrans significantly outperforms the previous state-of-the-art on various metrics, especially on PPIs of unseen species. For example, the AUPR improves 0.339 absolutely on Fly PPIs. Aiming to explore the knowledge learned by PPITrans from PPI data, this paper also designs a series of probes belonging to three categories. Their results reveal several interesting findings, like that although PPITrans cannot capture the spatial structure of proteins, it can obtain knowledge of PPI type and binding affinity, learning more than binary PPI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Szklarczyk, D., Gable, A. L., Nastou, K. C., Lyon, D., Kirsch, R., Pyysalo, S., Doncheva, N. T., Legeay, M., Fang, T., Bork, P., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1074 · Cited by 8,056 · see citing works
Principles that Govern the Folding of Protein Chains
Anfinsen, C. B. (1973) · Science · doi:10.1126/science.181.4096.223 · Cited by 7,096 · see citing works
UniProt: the universal protein knowledgebase in 2021
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Agivetova, R., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1100 · Cited by 6,372 · see citing works
A novel genetic system to detect protein–protein interactions
Fields, S., & Song, O.-K. (1989) · Nature · doi:10.1038/340245a0 · Cited by 6,152 · see citing works
Regulation of cancer cell metabolism
Cairns, R. A., Harris, I. S., & Mak, T. W. (2011) · Nature Reviews Cancer · doi:10.1038/nrc2981 · Cited by 4,744 · see citing works
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036 · see citing works
The SWISS-PROT protein knowledgebase and its supplement TrEMBL in 2003
Boeckmann, B. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg095 · Cited by 3,894 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · cited 5× · see citing works
“Prot-Trans [16] pre-trained many language models commonly used in the NLP field and compared their performance comprehensively.”
BackgroundMethodInfluentialUniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches
Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B., Wu, C. H., & the UniProt Consortium (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu739 · Cited by 1,928 · see citing works
Global Analysis of Protein Activities Using Proteome Chips
Zhu, H., Bilgin, M., Bangham, R., Hall, D., Casamayor, A., Bertone, P., Lan, N., Jansen, R., Bidlingmaier, S., Houfek, T., et al. (2001) · Science · doi:10.1126/science.1062191 · Cited by 1,884 · see citing works
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131 · cited 1× · see citing works
“Unirep [37] explores the pre-training and finetuning paradigm.”
BackgroundPredicting protein–protein interactions based only on sequences information
Shen, J., Zhang, J., Luo, X., Zhu, W., Yu, K., Chen, K., Li, Y., & Jiang, H. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0607879104 · Cited by 996 · see citing works
Lu, H., Zhou, Q., He, J., Jiang, Z., Peng, C., Tong, R., & Shi, J. (2020) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-020-00315-3 · Cited by 891 · see citing works
Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics
Asgari, E., & Mofrad, M. R. K. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141287 · Cited by 817 · see citing works
Network-based prediction of protein interactions
Kovács, I. A., Luck, K., Spirohn, K., Wang, Y., Pollis, C., Schlabach, S., Bian, W., Kim, D., Kishore, N., Hao, T., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-09177-y · Cited by 474 · see citing works
Szklarczyk, D., Gable, A. L., Nastou, K. C., Lyon, D., Kirsch, R. L., Pyysalo, S., Doncheva, N. T., Legeay, M., Fang, T., Peer Bork, B., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab835 · Cited by 450 · see citing works
Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold
Steinegger, M., Mirdita, M., & Söding, J. (2019) · Nature Methods · doi:10.1038/s41592-019-0437-4 · Cited by 445 · see citing works
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 409 · see citing works
Multifaceted protein–protein interaction prediction based on Siamese residual RCNN
Chen, M., Ju, C. J. ‐., Zhou, G., Chen, X., Zhang, T., Chang, K.-W., Zaniolo, C., & Wang, W. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz328 · Cited by 375 · see citing works
Predicting protein–protein interactions through sequence-based deep learning
Hashemifar, S. S., Neyshabur, B., Khan, A. A., & Xu, J. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty573 · Cited by 370 · see citing works
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 345 · see citing works
Learned protein embeddings for machine learning
Yang, K., Wu, Z., Bedbrook, C. N., & Arnold, F. H. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty178 · Cited by 316 · see citing works
The PIR-International Protein Sequence Database
Barker, W. C., Garavelli, J. S., McGarvey, P. B., Marzec, C. R., Orcutt, B. C., Srinivasarao, G. Y., Yeh, L.-S. L., Ledley, R. S., Mewes, H., Pfeiffer, F., et al. (1999) · Nucleic Acids Research · doi:10.1093/nar/27.1.39 · Cited by 245 · see citing works
Sledzieski, S., Singh, R., Cowen, L., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.08.010 · Cited by 208 · see citing works
Lei, C., & Ruan, J. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts688 · Cited by 196 · see citing works
A survey on computational models for predicting protein–protein interactions
Hu, L., Wang, X., Huang, Y., Hu, P., & You, Z. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab036 · Cited by 187 · see citing works
Deep Neural Network Based Predictions of Protein Interactions Using Primary Sequences
Li, H., Gong, X., Yu, H., & Zhou, C. (2018) · Molecules · doi:10.3390/molecules23081923 · Cited by 152 · see citing works
Learning spatial structures of proteins improves protein–protein interaction prediction
Song, B., Luo, X., Luo, X., Liu, Y., Niu, Z., & Zeng, X. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab558 · Cited by 134 · see citing works
Self-citationLv, G., Hu, Z., Bi, Y., & Zhang, S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/506 · Cited by 87 · see citing works
Xiao, Z., & Deng, Y. (2020) · PLoS ONE · doi:10.1371/journal.pone.0238915 · Cited by 50 · see citing works
GWIDD: Genome-wide protein docking database
Kundrotas, P. J., Zhu, Z., & Vakser, I. A. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp944 · Cited by 47 · see citing works
Multimodal Pre-Training Model for Sequence-based Prediction of Protein-Protein InteractionPreprint
Xue, Y., Liu, Z., Fang, X., & Wang, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.04814 · Cited by 10 · see citing works
A Survey on Prediction of Protein-Protein Interactions
Nivedha, S., & Bhavani, S. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1937/1/012011 · Cited by 7 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 47 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Will, I., Beckerson, W. C., & de Bekker, C. (2023) · Scientific Reports · doi:10.1038/s41598-023-40764-8 · Cited by 12 · see citing works
The Role of AI and ML in Shaping the Future of the Food Industry
Vijayvargiya, S., Sinha, S., & Sivakumar, V. (2025) · Current Nutrition & Food Science · doi:10.2174/0115734013371588250815061734 · Cited by 4 · see citing works
Protein-protein interaction prediction using bidirectional GRUs with explicit ensemble
Lan, Q., Zheng, Z., Tang, Z., Qiu, X., & Yin, Z. (2025) · PLoS ONE · doi:10.1371/journal.pone.0326960 · Cited by 2 · see citing works
Guo, X., Jiang, L., Li, J., Yuan, M., Li, D., Shi, W., Zhang, Z., & Wuchty, S. (2026) · Genome biology · doi:10.1186/s13059-026-04057-3 · Cited by 1 · see citing works
Nozawa, S., Tomii, K., & Fukasawa, Y. (2026) · Protein Science · doi:10.1002/pro.70790
AASIA: A comprehensive protein structural interactome database for agricultural animals
Li, J., Yuan, M., Jiang, L., Li, D., Yin, Z., Zhu, F., Shi, W., Hou, Z., & Zhang, Z. (2026) · Journal of Advanced Research · doi:10.1016/j.jare.2026.09.015
- Authors
- Yang, S., Cheng, P., Liu, Y., Feng, D., & Wang, S.
- Affiliations
- Beijing Institute of Microbiology and Epidemiology (CN)
- National University of Defense Technology (CN)
- Chinese PLA General Hospital (CN)
- Corresponding author
- Yang, Sen
- Venue
- IEEE/ACM Transactions on Computational Biology and Bioinformatics
- Published
- 2024-09
- Volume
- 21
- Issue
- 5
- Pages
- 1287-1298
- ISSN
- 1545-5963 (ISSN-L)
- 1557-9964
- 2374-0043
- OpenAlex
- W4393217153
- Cited by
- 6
- References
- 47
- Citations by year
- 2026: 3 · 2025: 2 · 2023: 1
- FWCI
- 0.68
- Type
- journal article
- Language
- en
- Open access
- green
- dx.doi.org
- Licence
- Locations
- IEEE Transactions on Computational Biology and Bioinformatics (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (62306334, 81830101)
- Open Fund of PDL (WDZC20245250107)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Keywords
- protein-protein interaction prediction, transformer models, protein language models, cross-species generalization, binding affinity, protein spatial structure
- MeSH terms
- Deep Learning, Algorithms, Animals, Humans, Proteins, Computational Biology, Protein Interaction Mapping, Databases, Protein, Protein Interaction Maps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
