Cited by 133Computational Drug Discovery MethodsProtein Structure and Dynamicsvaccines and immunoinformatics approaches

CSAR benchmark exercise 2011–2012: evaluation of results from docking and relative ranking of blinded congeneric series.

Open access
Damm-Ganamet, Kelly L., Smith, Richard D., Dunbar, James B., Stuckey, Jeanne A., Carlson, Heather A.·2013·Journal of Chemical Information and Modeling

Abstract

The Community Structure-Activity Resource (CSAR) recently held its first blinded exercise based on data provided by Abbott, Vertex, and colleagues at the University of Michigan, Ann Arbor. A total of 20 research groups submitted results for the benchmark exercise where the goal was to compare different improvements for pose prediction, enrichment, and relative ranking of congeneric series of compounds. The exercise was built around blinded high-quality experimental data from four protein targets: LpxC, Urokinase, Chk1, and Erk2. Pose prediction proved to be the most straightforward task, and most methods were able to successfully reproduce binding poses when the crystal structure employed was co-crystallized with a ligand from the same chemical series. Multiple evaluation metrics were examined, and we found that RMSD and native contact metrics together provide a robust evaluation of the predicted poses. It was notable that most scoring functions underpredicted contacts between the hetero atoms (i.e., N, O, S, etc.) of the protein and ligand. Relative ranking was found to be the most difficult area for the methods, but many of the scoring functions were able to properly identify Urokinase actives from the inactives in the series. Lastly, we found that minimizing the protein and correcting histidine tautomeric states positively trended with low RMSD for pose prediction but minimizing the ligand negatively trended. Pregenerated ligand conformations performed better than those that were generated on the fly. Optimizing docking parameters and pretraining with the native ligand had a positive effect on the docking performance as did using restraints, substructure fitting, and shape fitting. Lastly, for both sampling and ranking scoring functions, the use of the empirical scoring function appeared to trend positively with the RMSD. Here, by combining the results of many methods, we hope to provide a statistically relevant evaluation and elucidate specific shortcomings of docking methodology for the community.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1021/ci400025f

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Detection of active compounds among decoys and accurate prediction of binding affinity remain challenging tasks, even when induced fit and entropy effects are not important for binding.

    “However, the detection of active compounds among a set of decoy compounds and the accurate prediction of binding affinity remain challenging tasks, even when induced fit and entropy effects are not important for binding (Gohlke and Klebe, 2002; Damm-Ganamet et al., 2013; Yuriev and Ramsland, 2013; Grinter and Zou, 2014; Smith et al., 2016).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Warren G. M.; McGaughey G. B.; Nevins N. Editorial. J. Comput.-Aided Mol. Des. 2012, 26, 674.

Cited by

Sorted by citations

Authors
Damm-Ganamet, K. L., Smith, R. D., Dunbar, J. B., Stuckey, J. A., & Carlson, H. A.
Affiliations
  • Department of Medicinal Chemistry, University of Michigan, Ann Arbor, Michigan 48109-1065, United States
  • University of Michigan (US)
Corresponding author
Carlson, Heather A.
Venue
Journal of Chemical Information and Modeling
Published
2013-05-10
Volume
53
Issue
8
Pages
1853-1870
ISSN
  • 1549-9596 (ISSN-L)
  • 1549-960X
OpenAlex
W2028609644
Cited by
133
References
64
Citations by year
2026: 5 · 2025: 4 · 2024: 5 · 2023: 6 · 2022: 4 · 2021: 7 · 2020: 8 · 2019: 8 · 2018: 10 · 2017: 19 · 2016: 14 · 2015: 24 · 2014: 13 · 2013: 6
FWCI
20.77
Type
journal article
Language
en
Open access
Locations
  • Journal of Chemical Information and Modeling (publishedVersion, other-oa)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Institute of General Medical Sciences (U01GM086873)
Topics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › vaccines and immunoinformatics approaches
Keywords
molecular docking, benchmark exercise, congeneric series, relative ranking, scoring functions, pose prediction, protein-ligand docking, RMSD, urokinase, LpxC, Chk1, ERK2, protein-ligand binding, ligand conformation, empirical scoring function, structure-activity relationship
MeSH terms
Ligands, Protein Conformation, Proteins, Structure-Activity Relationship, Drug Design, Benchmarking, Molecular Docking Simulation, Databases, Pharmaceutical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review