New benchmark metrics for protein‐protein docking methods
Open accessAbstract
With the development of many computational methods that predict the structural models of protein‐protein complexes, there is a pressing need to benchmark their performance. As was the case for protein monomers, assessing the quality of models of protein complexes is not straightforward. An effective scoring scheme should be able to detect substructure similarity and estimate its statistical significance. Here, we focus on characterizing the similarity of the interfaces of the complex and introduce two scoring functions. The first, the i nterfacial Template Modeling score ( i TM‐score), measures the geometric distance between the interfaces, while the second, the Interface Similarity score (IS‐score), evaluates their residue‐residue contact similarity in addition to their geometric similarity. We first demonstrate that the IS‐score is more suitable for assessing docking models than the i TM‐score. The IS‐score is then validated in a large‐scale benchmark test on 1562 dimeric complexes. Finally, the scoring function is applied to evaluate docking models submitted to the Critical Assessment of Prediction of Interactions (CAPRI) experiments. While the results according to the new scoring scheme are generally consistent with the original CAPRI assessment, the IS‐score identifies models whose significance was previously underestimated. Proteins 2011. © 2011 Wiley‐Liss, Inc.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The interface similarity score (IS-score) for H1065 has a high value at 0.6020.
“The interface similarity score (IS-score), which was designed to evaluate dimeric protein-protein interfaces40, has a high value at 0.6020.”
From AF2Complex predicts direct physical interactions in multimeric proteins with deep learning (2022) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
VMD: visual molecular dynamics
Humphrey, W., Dalke, A., & Schulten, K. (1996) · Journal of Molecular Graphics · doi:10.1016/0263-7855(96)00018-5 · Cited by 66,952 · see citing works
Mass spectrometry-based proteomics
Aebersold, R., & Mann, M. (2003) · Nature · doi:10.1038/nature01511 · Cited by 6,955 · see citing works
A comprehensive analysis of protein–protein interactions in Saccharomyces cerevisiae
Uetz, P., Giot, L., Cagney, G., Mansfield, T. A., Judson, R. S., Knight, J. R., Lockshon, D., Narayan, V., Srinivasan, M., Pochart, P., et al. (2000) · Nature · doi:10.1038/35001009 · Cited by 4,706 · see citing works
A solution for the best rotation to relate two sets of vectors
Kabsch, W. (1976) · Acta Crystallographica Section A · doi:10.1107/s0567739476001873 · Cited by 3,462 · see citing works
HADDOCK: A Protein−Protein Docking Approach Based on Biochemical or Biophysical Information
Dominguez, C., Boelens, R., & Bonvin, A. M. J. J. (2003) · Journal of the American Chemical Society · doi:10.1021/ja026939x · Cited by 3,187 · see citing works
Global landscape of protein complexes in the yeast Saccharomyces cerevisiae
Krogan, N. J., Cagney, G., Yu, H., Zhong, G., Guo, X., Ignatchenko, A., Li, J., Pu, S., Datta, N., Tikuisis, A., et al. (2006) · Nature · doi:10.1038/nature04670 · Cited by 2,915 · see citing works
A generic protein purification method for protein complex characterization and proteome exploration
Rigaut, G., Shevchenko, A., Rutz, B., Wilm, M., Mann, M., & Séraphin, B. (1999) · Nature Biotechnology · doi:10.1038/13732 · Cited by 2,707 · see citing works
Proteome survey reveals modularity of the yeast cell machinery
Gavin, A., Aloy, P., Grandi, P., Krause, R., Böesche, M., Marzioch, M., Rau, C., Jensen, L. J., Bastuck, S., Dümpelfeld, B., et al. (2006) · Nature · doi:10.1038/nature04532 · Cited by 2,544 · see citing works
Scoring function for automated assessment of protein structure template quality
Zhang, Y., & Skolnick, J. (2004) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.20264 · Cited by 2,392 · see citing works
Self-citationA Protein Interaction Map of Drosophila melanogaster
Giot, L., Bader, J. S., Brouwer, C., Chaudhuri, A., Kuang, B., Li, Y., Hao, Y. L., Ooi, C., Godwin, B., Vitols, E., et al. (2003) · Science · doi:10.1126/science.1090289 · Cited by 2,266 · see citing works
A Map of the Interactome Network of the Metazoan C. elegans
Li, S., Armstrong, C. M., Bertin, N., Ge, H., Milstein, S., Boxem, M., Vidalain, P., Han, J.-D. J., Chesneau, A., Hao, T., et al. (2004) · Science · doi:10.1126/science.1091403 · Cited by 1,628 · see citing works
ZDOCK: An initial‐stage protein‐docking algorithm
Chen, R., Li, L., & Weng, Z. (2003) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.10389 · Cited by 1,393 · see citing works
LGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069 · see citing works
PIPER: An FFT‐based protein docking program with pairwise potentials
Kozakov, D., Brenke, R., Comeau, S. R., & Vajda, S. (2006) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21117 · Cited by 865 · see citing works
Gabb, H. A., Jackson, R. M., & Sternberg, M. J. E. (1997) · Journal of Molecular Biology · doi:10.1006/jmbi.1997.1203 · Cited by 820 · see citing works
MAMMOTH (Matching molecular models obtained from theory): An automated method for model comparison
Ortíz, Á. R., Strauss, C. E. M., & Olmea, O. (2002) · Protein Science · doi:10.1110/ps.0215902 · Cited by 492 · see citing works
Assessment of blind predictions of protein–protein interactions: Current status of docking methods
Mendez, R., Leplae, R., Maria, L. D., & Wodak, S. (2003) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.10393 · Cited by 438 · see citing works
Protein–Protein Docking with Backbone Flexibility
Wang, C., Bradley, P. H., & Baker, D. A. (2007) · Journal of Molecular Biology · doi:10.1016/j.jmb.2007.07.050 · Cited by 413 · see citing works
MaxSub: an automated measure for the assessment of protein structure prediction quality
Siew, N., Elofsson, A., Rychlewski, L., & Fischer, D. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.9.776 · Cited by 401 · see citing works
Docking and scoring protein complexes: CAPRI 3rd Edition
Lensink, M., Mendez, R., & Wodak, S. (2007) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21804 · Cited by 351 · see citing works
Interrogating protein interaction networks through structural biology
Aloy, P., & Russell, R. B. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.092147999 · Cited by 342 · see citing works
Flexible protein–protein docking
Bonvin, A. M. J. J. (2006) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2006.02.002 · Cited by 310 · see citing works
A unified statistical framework for sequence comparison and structure comparison
Levitt, M., & Gerstein, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.11.5913 · Cited by 300 · see citing works
A structural perspective on protein?protein interactions
Russell, R. B., Alber, F., Aloy, P., Davis, F. P., Korkin, D., Pichaud, M., Topf, M., & Šali, A. (2004) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2004.04.006 · Cited by 263 · see citing works
What is the probability of a chance prediction of a protein structure with an rmsd of 6 A?
Reva, B., Finkelstein, A., & Skolnick, J. (1998) · Folding & design · doi:10.1016/s1359-0278(98)00019-4 · Cited by 261 · see citing works
Self-citationProtein docking for low-resolution structures
Vakser, I. A. (1995) · Protein Engineering Design and Selection · doi:10.1093/protein/8.4.371 · Cited by 246 · see citing works
Kastritis, P. L., & Bonvin, A. M. J. J. (2010) · Journal of Proteome Research · doi:10.1021/pr9009854 · Cited by 246 · see citing works
Lu, L. J., Lu, H. J., & Skolnick, J. (2002) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.10222 · Cited by 240 · see citing works
Self-citationSoft protein–protein docking in internal coordinates
Fernández‐Recio, J., Totrov, M. M., & Abagyan, R. A. (2002) · Protein Science · doi:10.1110/ps.19202 · Cited by 197 · see citing works
Assessing predictions of protein–protein interaction: The CAPRI experiment
Janin, J. (2005) · Protein Science · doi:10.1110/ps.041081905 · Cited by 179 · see citing works
Gao, M., & Skolnick, J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012820107 · Cited by 139 · see citing works
Self-citationSandak, B., Wolfson, H. J., & Nussinov, R. (1998) · Proteins Structure Function and Bioinformatics · doi:10.1002/(sici)1097-0134(19980801)32:2 · Cited by 120 · see citing works
Architectures and Functional Coverage of Protein–Protein Interfaces
Tunçbağ, N., Gürsoy, A., Güney, E., Nussinov, R., & Keskin, Ö. (2008) · Journal of Molecular Biology · doi:10.1016/j.jmb.2008.04.071 · Cited by 109 · see citing works
iAlign: a method for the structural comparison of protein–protein interfaces
Gao, M., & Skolnick, J. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq404 · Cited by 98 · see citing works
Self-citationDocking by structural similarity at protein‐protein interfaces
Sinha, R., Kundrotas, P. J., & Vakser, I. A. (2010) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.22812 · Cited by 85 · see citing works
Prism: Protein-Protein Interaction Prediction by Structural Matching
Keskin, O., Nussinov, R., & Gursoy, A. (2008) · Methods in Molecular Biology · doi:10.1007/978-1-59745-398-1_30 · Cited by 80 · see citing works
M-TASSER: An Algorithm for Protein Quaternary Structure Prediction
Chen, H., & Skolnick, J. (2008) · Biophysical Journal · doi:10.1529/biophysj.107.114280 · Cited by 68 · see citing works
Self-citationD ockground protein–protein docking decoy set
Liu, S., Gao, Y. D., & Vakser, I. A. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn497 · Cited by 68 · see citing works
Docking without docking: ISEARCH—prediction of interactions using known interfaces
Günther, S., May, P., Hoppe, A., Frömmel, C., & Preißner, R. (2007) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.21746 · Cited by 53 · see citing works
Docking by Structural Similarity at Protein-Protein Interfaces
Kundrotas, P. J., Sinha, R., & Vakser, I. A. (2010) · Biophysical Journal · doi:10.1016/j.bpj.2009.12.1043 · Cited by 29 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
DockQ: A Quality Measure for Protein-Protein Docking Models
Basu, S., & Wallner, B. (2016) · PLOS ONE · doi:10.1371/journal.pone.0161879 · Cited by 460 · see citing works
AF2Complex predicts direct physical interactions in multimeric proteins with deep learning
Gao, M., Nakajima An, D., Parks, J. M., & Skolnick, J. (2022) · Nature Communications · doi:10.1038/s41467-022-29394-2 · Cited by 252 · see citing works
Self-citationMapping Monomeric Threading to Protein–Protein Structure Prediction
Guerler, A., Govindarajoo, B., & Zhang, Y. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci300579r · Cited by 73 · see citing works
Exploiting Conformational Ensembles in Modeling Protein–Protein Interactions on the Proteome Scale
Kuzu, G., Gursoy, A., Nussinov, R., & Keskin, O. (2013) · Journal of Proteome Research · doi:10.1021/pr400006k · Cited by 47 · see citing works
- Authors
- Gao, M., & Skolnick, J.
- Affiliations
- Georgia Institute of Technology (US)
- Corresponding author
- Skolnick, Jeffrey
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2011-03
- Volume
- 79
- Issue
- 5
- Pages
- 1623-1634
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- PMC
- PMC3076516
- OpenAlex
- W2017226419
- Cited by
- 42
- References
- 40
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 3 · 2023: 3 · 2022: 2 · 2021: 2 · 2020: 1 · 2019: 1 · 2018: 1 · 2017: 4 · 2016: 7 · 2014: 2 · 2013: 4 · 2012: 3
- FWCI
- 1.43
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proteins Structure Function and Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (GM-48835)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Keywords
- protein-protein docking, benchmarking, scoring functions, protein-protein complex, residue-residue contacts, Capri, dimeric complexes, model quality assessment
- MeSH terms
- Algorithms, Bacteria, Bacterial Proteins, Models, Molecular, Protein Binding, Proteins, Computational Biology, Protein Interaction Mapping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
