D3R grand challenge 2015: evaluation of protein-ligand pose and affinity predictions.
Open accessAbstract
The Drug Design Data Resource (D3R) ran Grand Challenge 2015 between September 2015 and February 2016. Two targets served as the framework to test community docking and scoring methods: (1) HSP90, donated by AbbVie and the Community Structure Activity Resource (CSAR), and (2) MAP4K4, donated by Genentech. The challenges for both target datasets were conducted in two stages, with the first stage testing pose predictions and the capacity to rank compounds by affinity with minimal structural data; and the second stage testing methods for ranking compounds with knowledge of at least a subset of the ligand-protein poses. An additional sub-challenge provided small groups of chemically similar HSP90 compounds amenable to alchemical calculations of relative binding free energy. Unlike previous blinded Challenges, we did not provide cognate receptors or receptors prepared with hydrogens and likewise did not require a specified crystal structure to be used for pose or affinity prediction in Stage 1. Given the freedom to select from over 200 crystal structures of HSP90 in the PDB, participants employed workflows that tested not only core docking and scoring technologies, but also methods for addressing water-mediated ligand-protein interactions, binding pocket flexibility, and the optimal selection of protein structures for use in docking calculations. Nearly 40 participating groups submitted over 350 prediction sets for Grand Challenge 2015. This overview describes the datasets and the organization of the challenge components, summarizes the results across all submitted predictions, and considers broad conclusions that may be drawn from this collaborative community endeavor.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
D3R periodically provides pharmaceutical-related benchmark datasets and a Grand Challenge as a blinded community challenge with unpublished data.
“Drug Design Data Resource (D3R10) periodically provide pharmaceutical-related benchmark datasets and a Grand Challenge as a blinded community challenge with unpublished data (Gathiaka et al., 2016).”
From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Gathiaka et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Yung-Chi, C., & Prusoff, W. H. (1973) · Biochemical Pharmacology · doi:10.1016/0006-2952(73)90196-2 · Cited by 12,922 · see citing works
PubChem Substance and Compound databases
Kim, S., Thiessen, P. A., Bolton, E. E., Chen, J., Fu, G., Gindulyte, A., Han, L., He, J., He, S., Shoemaker, B. A., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv951 · Cited by 5,436 · see citing works
ChEMBL: a large-scale bioactivity database for drug discovery
Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr777 · Cited by 4,382 · see citing works
BindingDB: a web-accessible database of experimentally determined protein-ligand binding affinities
Liu, T., Lin, Y., Wen, X., Jorissen, R. N., & Gilson, M. K. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkl999 · Cited by 2,145 · see citing works
Self-citationHawkins, P. C. D., Skillman, A. G., Warren, G. L., Ellingson, B. A., & Stahl, M. T. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100031x · Cited by 1,635 · see citing works
Structure and mechanism of the Hsp90 molecular chaperone machinery
Pearl, L. H., & Prodromou, C. (2006) · Annual Review of Biochemistry · doi:10.1146/annurev.biochem.75.103004.142738 · Cited by 1,103 · see citing works
Prediction of Ph-dependent Properties of Proteins
Antosiewicz, J. M., McCammon, J. A., & Gilson, M. K. (1994) · Journal of Molecular Biology · doi:10.1006/jmbi.1994.1301 · Cited by 818 · see citing works
Self-citationRush, T. S., Grant, J., Mosyak, L., & Nicholls, A. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm040163o · Cited by 601 · see citing works
A New Generation of Crystallographic Validation Tools for the Protein Data Bank
Read, R. J., Adams, P. D., Arendall, W. B., Brünger, A. T., Emsley, P., Joosten, R. P., Kleywegt, G. J., Krissinel, E., Lütteke, T., Otwinowski, Z., et al. (2011) · Structure · doi:10.1016/j.str.2011.08.006 · Cited by 468 · see citing works
Tembre, B. L., & McCammon, J. A. (1984) · Computers & Chemistry · doi:10.1016/0097-8485(84)85020-2 · Cited by 459 · see citing works
Standard Binding Free Energies from Computer Simulations: What Is the Best Strategy?
Gumbart, J. C., Roux, B., & Chipot, C. (2013) · Journal of Chemical Theory and Computation · doi:10.1021/ct3008099 · Cited by 343 · see citing works
Computer Automated log P Calculations Based on an Extended Group Contribution Approach
Klopman, G., Li, J.-Y., Wang, S., & Dimayuga, M. (1994) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00020a009 · Cited by 286 · see citing works
Efficient Determination of Protein–Protein Standard Binding Free Energies from First Principles
Gumbart, J. C., Roux, B., & Chipot, C. (2013) · Journal of Chemical Theory and Computation · doi:10.1021/ct400273t · Cited by 211 · see citing works
The SAMPL4 host–guest blind prediction challenge: an overview
Muddana, H. S., Fenley, A. T., Mobley, D. L., et al. (2014) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-014-9735-1 · Cited by 183 · see citing works
Hsp90: A Novel Target for Cancer Therapy
Solit, D. B., & Rosen, N. X. (2006) · Current Topics in Medicinal Chemistry · doi:10.2174/156802606777812068 · Cited by 171 · see citing works
Activity, assay and target data curation and quality in the ChEMBL database
Papadatos, G., Gaulton, A., Hersey, A., & Overington, J. P. (2015) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-015-9860-5 · Cited by 169 · see citing works
POSIT: Flexible Shape-Guided Docking For Pose Prediction
Kelley, B. P., Brown, S. P., Warren, G. L., & Muchmore, S. W. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00142 · Cited by 159 · see citing works
Assessment of programs for ligand binding affinity prediction
Kim, R., & Skolnick, J. (2008) · Journal of Computational Chemistry · doi:10.1002/jcc.20893 · Cited by 147 · see citing works
CSAR Benchmark Exercise of 2010: Combined Evaluation Across All Submitted Scoring Functions
Smith, R. D., Dunbar, J. B., Ung, P. M. U., Esposito, E. X., Yang, C., Wang, S., & Carlson, H. A. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200269q · Cited by 142 · see citing works
Self-citationDamm-Ganamet, K. L., Smith, R. D., Dunbar, J. B., Stuckey, J. A., & Carlson, H. A. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci400025f · Cited by 133 · see citing works
Self-citationHealthy skepticism: assessing realistic model performance
Brown, S. P., Muchmore, S. W., & Hajduk, P. J. (2009) · Drug Discovery Today · doi:10.1016/j.drudis.2009.01.012 · Cited by 126 · see citing works
Understanding the Impact of the P-loop Conformation on Kinase Selectivity
Guimarães, C. R. W., Rai, B. K., Munchhof, M. J., et al. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200153c · Cited by 119 · see citing works
CSAR Data Set Release 2012: Ligands, Affinities, Complexes, and Docking Decoys
Dunbar, J. B., Smith, R. D., Damm‐Ganamet, K. L., Ahmed, A., Esposito, E. X., Delproposto, J., Chinnaswamy, K., Kang, Y.-N., Kubish, G. M., Gestwicki, J. E., et al. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci4000486 · Cited by 119 · see citing works
Self-citationDiscovery and Design of Novel HSP90 Inhibitors Using Multiple Fragment‐based Design Strategies
Huth, J. R., Park, C., Petros, A. M., Kunzer, A. R., Wendt, M. D., Wang, X., Lynch, C. L., Mack, J. C., Swift, K. M., Judge, R. A., et al. (2007) · Chemical Biology & Drug Design · doi:10.1111/j.1747-0285.2007.00535.x · Cited by 116 · see citing works
Gooljarsingh, L. T., Fernandes, C., Yan, K., Zhang, H., Grooms, M. H., Johanson, K., Sinnamon, R. H., Kirkpatrick, R. B., Kerrigan, J. J., Lewis, T. S., et al. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0602650103 · Cited by 116 · see citing works
HSP90 inhibition: two-pronged exploitation of cancer dependencies
Travers, J., Sharp, S. Y., & Workman, P. (2011) · Drug Discovery Today · doi:10.1016/j.drudis.2011.12.021 · Cited by 105 · see citing works
CSAR 2014: A Benchmark Exercise Using Unpublished Data from Pharma
Carlson, H. A., Smith, R. D., Damm‐Ganamet, K. L., Stuckey, J. A., Ahmed, A., Convery, M. A., Somers, D. O., Kranz, M., Elkins, P. A., Cui, G., et al. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00523 · Cited by 102 · see citing works
Self-citationBlind prediction of HIV integrase binding from the SAMPL4 challenge
Mobley, D. L., Liu, S., Lim, N. M., Wymer, K. L., Perryman, A. L., Forli, S., Deng, N., Su, J. M., Branson, K., & Olson, A. J. (2014) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-014-9723-5 · Cited by 58 · see citing works
Self-citationNeuritogenic Militarinone‐Inspired 4‐Hydroxypyridones Target the Stress Pathway Kinase MAP4K4
Schröder, P., Förster, T., Kleine, S., Becker, C. J., Richters, A., Ziegler, S., Rauh, D., Kumar, K., & Waldmann, H. (2015) · Angewandte Chemie International Edition · doi:10.1002/anie.201501515 · Cited by 54 · see citing works
Smith, R. D., Damm-Ganamet, K., Dunbar, J., Ahmed, A., Chinnaswamy, K., Delproposto, J., Kubish, G., Tinberg, C., Khare, S., Dou, J., et al. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00387 · Cited by 54 · see citing works
Self-citationRelationship between multiple sequence alignments and quality of protein comparative models
Cozzetto, D., & Tramontano, A. L. (2004) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.20284 · Cited by 51 · see citing works
Ndubaku, C. O., Crawford, T. D., Chen, H., et al. (2015) · ACS Medicinal Chemistry Letters · doi:10.1021/acsmedchemlett.5b00174 · Cited by 49 · see citing works
Crawford, T. D., Ndubaku, C. O., Chen, H., Boggs, J. W., Bravo, B. J., Delatorre, K. J., Giannetti, A. M., Gould, S. E., Harris, S. F., Magnuson, S. R., et al. (2014) · Journal of Medicinal Chemistry · doi:10.1021/jm500155b · Cited by 48 · see citing works
N-Aryl-benzimidazolones as novel small molecule HSP90 inhibitors
Bruncko, M., Tahir, S. K., Song, X., et al. (2010) · Bioorganic & Medicinal Chemistry Letters · doi:10.1016/j.bmcl.2010.10.010 · Cited by 37 · see citing works
The Collaborative Drug Discovery (CDD) Database
Ekins, S., & Bunin, B. A. (2013) · Methods in molecular biology · doi:10.1007/978-1-62703-342-8_10 · Cited by 30 · see citing works
Wang, L., Stanley, M. S., Boggs, J. W., et al. (2014) · Bioorganic & Medicinal Chemistry Letters · doi:10.1016/j.bmcl.2014.07.071 · Cited by 28 · see citing works
DCC: a Swiss army knife for structure factor analysis and validation
Yang, H., Peisach, E., Westbrook, J., et al. (2016) · Journal of Applied Crystallography · doi:10.1107/s1600576716004428 · Cited by 23 · see citing works
Self-citationA pose prediction approach based on ligand 3D shape similarity
Kumar, A., & Zhang, K. Y. J. (2016) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-9923-2 · Cited by 22 · see citing works
Prospective evaluation of shape similarity based pose prediction method in D3R Grand Challenge 2015
Kumar, A., & Zhang, K. Y. J. (2016) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-9931-2 · Cited by 15 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [17]Wang L Stanley M Boggs JW 2014 Fragment-based identification and optimization of a class of potent pyrrolo[2,1-f][1,2,4]triazine MAP4K4 inhibitors Bioorg Med Chem Lett 24 4546 52 10.1016/j.bmcl.2014.07.071 25139565 · DOI
- [21]Tembre BL Mc Cammon JA 1984 Ligand-receptor interactions Comput Chem 8 281 283 http://dx.doi.org/10.1016/0097-8485(84)85020-2 · DOI
Cited by
Sorted by citations
Guedes, I. A., Pereira, F. S. S., & Dardenne, L. E. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.01089 · Cited by 331 · see citing works
Gaieb, Z., Liu, S., Gathiaka, S., Chiu, M., Yang, H., Shao, C., Feher, V. A., Walters, W. P., Kuhn, B., Rudolph, M. G., et al. (2018) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-017-0088-4 · Cited by 193 · see citing works
Self-citationAckloo, S., Al-awar, R., Amaro, R. E., Arrowsmith, C. H., Azevedo, H., Batey, R. A., Bengio, Y., Betz, U. A. K., Bologa, C. G., Chodera, J. D., et al. (2022) · Nature Reviews Chemistry · doi:10.1038/s41570-022-00363-z · Cited by 112 · see citing works
Self-citationExplainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts
Karimi, M., Wu, D., Wang, Z., & Shen, Y. (2019) · doi:10.1101/2019.12.28.890103 · Cited by 64 · see citing works
New prediction categories in <scp>CASP15</scp>
Kryshtafovych, A., Antczak, M., Szachniuk, M., Zok, T., Kretsch, R. C., Rangan, R., Pham, P., Das, R., Robin, X., Studer, G., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26515 · Cited by 54 · see citing works
“Previous ligand docking challenges like Teach Discover Treat (TDT), 39 Continuous Evaluation of Ligand Prediction Performance (CELPP) 40 and Drug Discovery Data Resource (D3R) 34 – 37 have used two main types of metrics to assess how well participants can model receptor – ligand complexes.”
MethodA membrane protein complex docking benchmark
Koukos, P. I., Faro, I., van Noort, C. W., & Bonvin, A. M. J. J. (2018) · Journal of Molecular Biology · doi:10.1016/j.jmb.2018.11.005 · Cited by 23 · see citing works
A fully automated benchmarking suite to compare macromolecular complexes
Studer, G., Robin, X., Bienert, S., Durairaj, J., Škrinjar, P., Tauriello, G., Waterhouse, A. M., & Schwede, T. (2026) · Nature Methods · doi:10.1038/s41592-025-02973-z · Cited by 10 · see citing works
“A local superposition was needed as the reference structures in CASP were larger and more flexible than those in GPCR dock 15,16 and the first stage of the D3R Grand Challenge 3 9 .”
Influential
- Authors
- Gathiaka, S., Liu, S., Chiu, M., Yang, H., Stuckey, J. A., Kang, Y. N., Delproposto, J., Kubish, G., Dunbar, J. B., Carlson, H. A., Burley, S. K., Walters, W. P., Amaro, R. E., Feher, V. A., & Gilson, M. K.
- Affiliations
- University of California San Diego (US)
- Rutgers, The State University of New Jersey (US)
- University of Michigan (US)
- Relay Therapeutics (United States) (US)
- Schrodinger (United States) (US)
- Corresponding author
- Amaro, Rommie E.
- Feher, Victoria A.
- Gilson, Michael K.
- ORCID
- Venue
- Journal of Computer-Aided Molecular Design
- Published
- 2016-09
- Volume
- 30
- Issue
- 9
- Pages
- 651-668
- ISSN
- 0920-654X (ISSN-L)
- 1573-4951
- PMC
- PMC5562487
- OpenAlex
- W2527541313
- Cited by
- 207
- References
- 39
- Citations by year
- 2026: 13 · 2025: 11 · 2024: 11 · 2023: 17 · 2022: 17 · 2021: 22 · 2020: 30 · 2019: 29 · 2018: 24 · 2017: 32 · 2016: 1
- FWCI
- 26.68
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Computer-Aided Molecular Design (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Rutgers University Community Repository (Rutgers University) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institute of General Medical Sciences (1U01GM111528)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › vaccines and immunoinformatics approaches
- Keywords
- protein-ligand docking, binding affinity prediction, HSP90, MAP4K4, pose prediction, compound ranking, alchemical free energy calculations, water-mediated interactions
- MeSH terms
- Binding Sites, Ligands, Protein Binding, Protein Conformation, Drug Design, Crystallography, X-Ray, HSP90 Heat-Shock Proteins, Quantitative Structure-Activity Relationship, Molecular Docking Simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
