The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
A Comprehensive Benchmark of Kernel Methods to Extract Protein–Protein Interactions from Literature
Open accessAbstract
The most important way of conveying new findings in biomedical research is scientific publication. Extraction of protein-protein interactions (PPIs) reported in scientific publications is one of the core topics of text mining in the life sciences. Recently, a new class of such methods has been proposed - convolution kernels that identify PPIs using deep parses of sentences. However, comparing published results of different PPI extraction methods is impossible due to the use of different evaluation corpora, different evaluation metrics, different tuning procedures, etc. In this paper, we study whether the reported performance metrics are robust across different corpora and learning settings and whether the use of deep parsing actually leads to an increase in extraction quality. Our ultimate goal is to identify the one method that performs best in real-life scenarios, where information extraction is performed on unseen text and not on specifically prepared evaluation data. We performed a comprehensive benchmarking of nine different methods for PPI extraction that use convolution kernels on rich linguistic information. Methods were evaluated on five different public corpora using cross-validation, cross-learning, and cross-corpus evaluation. Our study confirms that kernels using dependency trees generally outperform kernels based on syntax trees. However, our study also shows that only the best kernel methods can compete with a simple rule-based approach when the evaluation prevents information leakage between training and test corpora. Our results further reveal that the F-score of many approaches drops significantly if no corpus-specific parameter optimization is applied and that methods reaching a good AUC score often perform much worse in terms of F-score. We conclude that for most kernels no sensible estimation of PPI extraction performance on new text is possible, given the current heterogeneity in evaluation data. Nevertheless, our study shows that three kernels are clearly superior to the other methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ideker, T., & Sharan, R. (2008) · Genome Research · doi:10.1101/gr.071852.107 · Cited by 817
Text Mining for Metabolic Pathways, Signaling Cascades, and Protein Networks
Hoffmann, R., Krallinger, M., Andres, E., Tamames, J., Blaschke, C., & Valencia, A. (2005) · Science's STKE · doi:10.1126/stke.2832005pe21 · Cited by 88 · cited 1×
“Collecting data on individual interactions is crucial for understanding these processes at a systems biology level [1].”
Background
Cited by
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Xu, J., Yang, P., Xue, S., Sharma, B., Sanchez-Martin, M., Wang, F., Beaty, K. A., Dehan, E., & Parikh, B. (2019) · Human Genetics · doi:10.1007/s00439-019-01970-5 · Cited by 305
ChemSpot: a hybrid system for chemical named entity recognition
Rocktäschel, T., Weidlich, M., & Leser, U. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts183 · Cited by 269
Self-citationYang, X., Bian, J., Gong, Y., Hogan, W. R., & Wu, Y. (2019) · Drug Safety · doi:10.1007/s40264-018-0761-0 · Cited by 66
Metadata
- Authors
- Tikk, D., Thomas, P., Palaga, P., Hakenberg, J., & Leser, U.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-07-01
- Volume
- 6
- Issue
- 7
- Pages
- e1000837
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2048059249
- Cited by
- 171
- References
- 2
- FWCI
- 7.21
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Comprehensive Benchmark of Kernel Methods to Extract Protein–Protein Interactions from Literature. https://app.cochat.ai/papers/10.1371/journal.pcbi.1000837
- The paper
- Tikk, D., Thomas, P., Palaga, P., Hakenberg, J., & Leser, U. (2010). A Comprehensive Benchmark of Kernel Methods to Extract Protein–Protein Interactions from Literature. PLoS Computational Biology. https://doi.org/10.1371/journal.pcbi.1000837
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.