Benchmark Evaluation of Protein–Protein Interaction Prediction Algorithms
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Protein–protein interactions (PPIs) perform various functions and regulate processes throughout cells. Knowledge of the full network of PPIs is vital to biomedical research, but most of the PPIs are still unknown. As it is infeasible to discover all of them experimentally due to technical and resource limitations, computational prediction of PPIs is essential and accurately assessing the performance of algorithms is required before further application or translation. However, many published methods compose their evaluation datasets incorrectly, using a higher proportion of positive class data than occuring naturally, leading to exaggerated performance. We re-implemented various published algorithms and evaluated them on datasets with realistic data compositions and found that their performance is overstated in original publications; with several methods outperformed by our control models built on ‘illogical’ and random number features. We conclude that these methods are influenced by an over-characterization of some proteins in the literature and due to scale-free nature of PPI network and that they fail when tested on all possible protein pairs. Additionally, we found that sequence-only-based algorithms performed worse than those that employ functional and expression features. We present a benchmark evaluation of many published algorithms for PPI prediction. The source code of our implementations and the benchmark datasets created here are made available in open source.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
UniProt: the universal protein knowledgebase in 2021
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Agivetova, R., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1100 · Cited by 6,372
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
BioGRID: a general repository for interaction datasets
Stark, C. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkj109 · Cited by 3,960 · cited 1×
“For example, The Biological General Repository for Interaction Datasets (BioGRID) contains ~125,000 unique, non-self-interactions among ~14,500 proteins (represented by their gene names/symbols and not distinguishing isoforms; this aspect that proteins are referred to by their genes is applicable throughout this paper…”
BackgroundGene Ontology Consortium: going forward
The Gene Ontology Consortium (2015) · Nucleic Acids Research · doi:10.1093/nar/gku1179 · Cited by 3,492
A reference map of the human binary protein interactome
Luck, K., Kim, D.-K., Lambourne, L., Spirohn, K., Begg, B. E., Bian, W., Brignall, R., Cafarelli, T., Campos-Laborie, F. J., Charloteaux, B., et al. (2020) · Nature · doi:10.1038/s41586-020-2188-x · Cited by 1,197 · cited 2×
“In one recent approach, a high quality protein interaction network of 53,000 PPIs was produced using such an approach [4].”
BackgroundMethodWhere Have All the Interactions Gone? Estimating the Coverage of Two-Hybrid Protein Interaction Maps
Huang, H., Jedynak, B. M., & Bader, J. S. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030214 · Cited by 182
Cited by
Predictomes, a classifier-curated database of AlphaFold-modeled protein-protein interactions
Schmid, E. W., & Walter, J. C. (2025) · Molecular Cell · doi:10.1016/j.molcel.2025.01.034 · Cited by 120
Shor, B., & Schneidman-Duhovny, D. (2024) · Nature Methods · doi:10.1038/s41592-024-02174-0 · Cited by 85
SpatialPPI: Three-dimensional space protein-protein interaction prediction with AlphaFold Multimer
(2024) · doi:10.1016/j.csbj.2024.03.009 · Cited by 29
Metadata
- Authors
- Dunham, B., & Ganapathiraju, M. K.
- Venue
- Molecules
- Publisher
- MDPI AG
- Published
- 2021-12-22
- Volume
- 27
- Issue
- 1
- Pages
- 41
- ISSN
- 1420-3049 (ISSN-L)
- OpenAlex
- W4200461926
- Cited by
- 47
- References
- 6
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.