CoChat

PIPENN-EMB: ensemble net and protein embeddings generalise protein interface prediction beyond homology

Open access
Thomas, David P. G., Garcia Fernandez, Carlos M., Haydarlou, Reza, Feenstra, K. Anton · 2024 · Scientific Reports

Preprint — published as 10.1038/s41598-025-88445-y · Paper page

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

Protein interactions are crucial for understanding biological functions and disease mechanisms, but predicting these remains a complex task in computational biology. Increasingly, Deep Learning models are having success in interface prediction. This study presents PIPENN-EMB which explores the added value of using embeddings from the ProtT5-XL protein language model. Our results show substantial improvement over the previously published PIPENN model for protein interaction interface prediction, reaching an MCC of 0.313 vs. 0.249, and AUC-ROC 0.800 vs. 0.755 on the BIO_DL_TE test set. We furthermore show that these embeddings cover a broad range of ‘hand-crafted’ protein features in ablation studies. PIPENN-EMB reaches state-of-the-art performance on the ZK448 dataset for protein-protein interface prediction. We showcase predictions on 25 resistance-related proteins from Mycobacterium tuberculosis . Furthermore, whereas other state-of-the-art sequence-based methods perform worse for proteins that have little recognisable homology in their training data, PIPENN-EMB generalises to remote homologs, yielding stable AUC-ROC across all three test sets with less than 30% sequence identity to the training dataset, and even to proteins with less than 15% sequence identity. Availability: Webserver, source code and datasets at www.ibi.vu.nl/programs/pipennemb/

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/2024.10.31.621117

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Neural network architecture of the (A) fully connected feed forward (ann); (B) residual convolutional network combined with the recurrent network (cnet); (C) residual convolutional network (rnet); (D) dilated convolutional network (dnet); (E) recurrent network (rnn); (F) U-shape convolutional network (unet); (G) ensemble network (ensnet).
Fig. 1 Neural network architecture of the (A) fully connected feed forward (ann); (B) residual convolutional network combined with the recurrent network (cnet); (C) residual convolutional network (rnet); (D) dilated convolutional network (dnet); (E) recurrent network (rnn); (F) U-shape convolutional network (unet); (G) ensemble network (ensnet).CC BY-NC-ND · PubMed Central
Fig. 2 Model performance of state of the art sequence based prediction methods. (A) AUROC and (B) Precision-Recall plots for comparison of models tested on the BioDL_P_TE test set, the MTB use case, and the ZK448 benchmark dataset. Curves correspond to the interface prediction performance for the sequence-based PIPENN-EMB and Seq-InSite. Legends report AUROC and AUPR values as measure of overall prediction capabilities.
Fig. 2 Model performance of state of the art sequence based prediction methods. (A) AUROC and (B) Precision-Recall plots for comparison of models tested on the BioDL_P_TE test set, the MTB use case, and the ZK448 benchmark dataset. Curves correspond to the interface prediction performance for the sequence-based PIPENN-EMB and Seq-InSite. Legends report AUROC and AUPR values as measure of overall prediction capabilities.CC BY-NC-ND · PubMed Central
Fig. 3 Correlation between model performance and sequence homology for different test datasets. Data points represent individual proteins from the different datasets, showing the method prediction performance in AUROC versus the (the product of) sequence similarity and query coverage for hits in the method’s training set. The linear regression lines indicate trends of AUROC with increasing sequence similarity and coverage. Shaded areas show the 95% confidence intervals. Correlation coefficients (R²) are annotated in the legends for each model, with an asterisk (*) for a p-value < 0.05. Results are shown for three datasets: (A) BioDL_P_TE, (B) ZK448, and (C) MTB.
Fig. 3 Correlation between model performance and sequence homology for different test datasets. Data points represent individual proteins from the different datasets, showing the method prediction performance in AUROC versus the (the product of) sequence similarity and query coverage for hits in the method’s training set. The linear regression lines indicate trends of AUROC with increasing sequence similarity and coverage. Shaded areas show the 95% confidence intervals. Correlation coefficients (R²) are annotated in the legends for each model, with an asterisk (*) for a p-value < 0.05. Results are shown for three datasets: (A) BioDL_P_TE, (B) ZK448, and (C) MTB.CC BY-NC-ND · PubMed Central
Fig. 4 Performance of PIPENN-EMB and Seq-Insite for distant homologs from their respective training data. Boxplots illustrate the differences in the distribution of AUROC for different homology cut-offs to the respective training data of the prediction methods. Three thresholds based on sequence identity times query coverage were applied: proteins with 0–15% identity times coverage are considered remote homologs, 16–30% are considered distant homologs, and proteins between 31–45% are considered close homologs to the respective training data for the model. The number of proteins used for each distribution is indicated above the boxes. Results are shown for three datasets: (A) BioDL_P_TE, (B) ZK448, and MTB.
Fig. 4 Performance of PIPENN-EMB and Seq-Insite for distant homologs from their respective training data. Boxplots illustrate the differences in the distribution of AUROC for different homology cut-offs to the respective training data of the prediction methods. Three thresholds based on sequence identity times query coverage were applied: proteins with 0–15% identity times coverage are considered remote homologs, 16–30% are considered distant homologs, and proteins between 31–45% are considered close homologs to the respective training data for the model. The number of proteins used for each distribution is indicated above the boxes. Results are shown for three datasets: (A) BioDL_P_TE, (B) ZK448, and MTB.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 51

  1. 10.1073/pnas.93.1.13 · DOI · Paper page
  2. 10.1074/mcp.R113.027771 · DOI · Paper page
  3. 10.1038/s41588-020-00774-y · DOI · Paper page
  4. 10.2174/138161212802651661 · DOI
  5. 10.1038/nrd.2016.29 · DOI · Paper page
  6. 10.1016/J.CEB.2023.102291 · DOI
  7. 10.1038/s41598-021-81132-8 · DOI · Paper page
  8. 10.1093/bib/bbp021 · DOI · Paper page
  9. 10.1016/j.febslet.2015.10.003 · DOI · Paper page
  10. 10.1093/bib/bbx022 · DOI · Paper page
  11. 10.1093/bioinformatics/005 · doi:10.1093/bioinformatics/005
  12. 10.1093/bioinformatics/btac071 · DOI
  13. 10.1002/prot.26609 · DOI · Paper page
  14. 10.1038/s41586-021-03828-1 · DOI · Paper page
  15. 10.1002/ADVS.202102592 · DOI · Paper page
  16. 10.1093/bioinformatics/btab761 · DOI · Paper page
  17. 10.1038/s41591-021-01533-0 · DOI · Paper page
  18. 10.1038/s41592-021-01365-3 · DOI · Paper page
  19. 10.1101/864405 · DOI
  20. 10.1038/s41598-020-80786-0 · DOI · Paper page
  21. 10.1038/s41598-021-00311-9 · DOI
  22. 10.1038/s41598-022-13951-2 · DOI
  23. 10.1038/s41598-022-19608-4 · DOI · Paper page
  24. 10.1093/bioinformatics/btac020 · DOI · Paper page
  25. 10.1093/bioinformatics/btaa701 · DOI · Paper page
  26. 10.1109/TPAMI.2021.3095381 · DOI · Paper page
  27. 10.1073/pnas.2016239118 · DOI · Paper page
  28. 10.34133/research.0240 · DOI
  29. Cui, Y. , Dong, Q. & Hong, D. Predicting protein-ligand binding residues with deep convolutional neural networks. BMC Bioinforma. 20 (2019). · DOI · Paper page
  30. 10.1016/j.engappai.2022.105151 · DOI · Paper page
  31. 10.1016/J.CSBJ.2020.10.017 · DOI
  32. 10.3389/FMOLB.2021.619403/BIBTEX · doi:10.3389/fmolb.2021.619403/bibtex
  33. 10.1093/bioinformatics/btm098 · DOI · Paper page
  34. 10.1038/s41467-018-02974-x · DOI · Paper page
  35. 10.1186/s13073-018-0611-9 · DOI · Paper page
  36. 10.1093/nar/gkab988 · DOI · Paper page
  37. 10.1093/BIOINFORMATICS/BTAD738 · DOI · Paper page
  38. 10.1002/prot.26615 · DOI
  39. 10.1038/s41467-022-35677-5 · DOI · Paper page
  40. 10.1093/bioinformatics/btae012 · DOI
  41. 10.1371/journal.pcbi.1010669 · DOI
  42. 10.1006/jmbi.1990.9999 · doi:10.1006/jmbi.1990.9999 · Paper page
  43. 10.1186/1471-2105-10-421 · DOI · Paper page
  44. 10.1093/bioinformatics/btaa750 · DOI · Paper page
  45. 10.1093/bioinformatics/btz324 · DOI · Paper page
  46. 2014, SPRINGS: Prediction of Protein-Protein Interaction Sites Using Artificial Neural Networks, J Proteom & Comput. Biol, 1, 7
  47. 10.1016/j.csbj.2021.04.034 · DOI
  48. 10.1093/NARGAB/LQAC043 · doi:10.1093/nargab/lqac043
  49. 10.1101/2022.12.15.519894 · DOI
  50. 10.1038/s41586-021-03819-2 · DOI · Paper page
  51. 10.1038/s41598-020-75467-x · DOI · Paper page

Metadata

Authors
Thomas, D. P. G., Garcia Fernandez, C. M., Haydarlou, R., & Feenstra, K. A.
Affiliations
Vrije Universiteit Amsterdam (NL)
Corresponding author
  • Haydarlou, Reza
  • Feenstra, K. Anton
Venue
Scientific Reports
Publisher
openRxiv
Published
2024-11-02
Volume
15
Issue
1
Pages
4391-4391
ISSN
2045-2322 (ISSN-L)
OpenAlex
W4407153268
Cited by
4
References
2
Citations by year
2026: 4
FWCI
1.20
Type
journal article
Language
en
Open access
Locations
  • Scientific Reports (publishedVersion, cc-by-nc-nd)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • VU Research Portal (publishedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
Vrije Universiteit Amsterdam
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
Keywords
protein-protein interface prediction, protein language models, protein embeddings, protein-protein interaction, ablation study
MeSH terms
Deep Learning, Bacterial Proteins, Mycobacterium tuberculosis, Proteins, Computational Biology, Protein Interaction Mapping, Databases, Protein

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: PIPENN-EMB: ensemble net and protein embeddings generalise protein interface prediction beyond homology. https://app.cochat.ai/papers/10.1101/2024.10.31.621117
The paper
Thomas, D. P. G., Garcia Fernandez, C. M., Haydarlou, R., & Feenstra, K. A. (2024). PIPENN-EMB: ensemble net and protein embeddings generalise protein interface prediction beyond homology. Scientific Reports. https://doi.org/10.1101/2024.10.31.621117

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.