The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Comparative Assessment of Scoring Functions on a Diverse Test Set
Abstract
Scoring functions are widely applied to the evaluation of protein-ligand binding in structure-based drug design. We have conducted a comparative assessment of 16 popular scoring functions implemented in main-stream commercial software or released by academic research groups. A set of 195 diverse protein-ligand complexes with high-resolution crystal structures and reliable binding constants were selected through a systematic nonredundant sampling of the PDBbind database and used as the primary test set in our study. All scoring functions were evaluated in three aspects, that is, "docking power", "ranking power", and "scoring power", and all evaluations were independent from the context of molecular docking or virtual screening. As for "docking power", six scoring functions, including GOLD::ASP, DS::PLP1, DrugScore(PDB), GlideScore-SP, DS::LigScore, and GOLD::ChemScore, achieved success rates over 70% when the acceptance cutoff was root-mean-square deviation < 2.0 A. Combining these scoring functions into consensus scoring schemes improved the success rates to 80% or even higher. As for "ranking power" and "scoring power", the top four scoring functions on the primary test set were X-Score, DrugScore(CSD), DS::PLP, and SYBYL::ChemScore. They were able to correctly rank the protein-ligand complexes containing the same type of protein with success rates around 50%. Correlation coefficients between the experimental binding constants and the binding scores computed by these scoring functions ranged from 0.545 to 0.644. Besides the primary test set, each scoring function was also tested on four additional test sets, each consisting of a certain number of protein-ligand complexes containing one particular type of protein. Our study serves as an updated benchmark for evaluating the general performance of today's scoring functions. Our results indicate that no single scoring function consistently outperforms others in all three aspects. Thus, it is important in practice to choose the appropriate scoring functions for different purposes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Friesner, R. A., Banks, J. L., Murphy, R. B., Halgren, T. A., Klicic, J. J., Mainz, D. T., Repasky, M. P., Knoll, E. H., Shelley, M., Perry, J. K., et al. (2004) · Journal of Medicinal Chemistry · doi:10.1021/jm0306430 · Cited by 10,057
Development and validation of a genetic algorithm for flexible docking 1 1Edited by F. E. Cohen
Jones, G., Willett, P., Glen, R. C., Leach, A. R., & Taylor, R. (1997) · Journal of Molecular Biology · doi:10.1006/jmbi.1996.0897 · Cited by 6,763
ZINC − A Free Database of Commercially Available Compounds for Virtual Screening
Irwin, J. J., & Shoichet, B. K. (2005) · Journal of Chemical Information and Modeling · doi:10.1021/ci049714+ · Cited by 4,058
Benchmarking sets for molecular docking.
Huang, N., Shoichet, B. K., & Irwin, J. J. (2006) · Journal of Medicinal Chemistry · doi:10.1021/jm0608356 · Cited by 1,318
Massova, I., & Kollman, P. A. (2000) · Perspectives in Drug Discovery and Design · doi:10.1023/a:1008763014207 · Cited by 1,015
The PDBbind database: methodologies and updates
Wang, R., Fang, X., Lu, Y., Yang, C.-Y., & Wang, S. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm048957q · Cited by 933
Self-citationComparative evaluation of 11 scoring functions for molecular docking.
Wang, R., Lu, Y., & Wang, S. (2003) · Journal of Medicinal Chemistry · doi:10.1021/jm0203783 · Cited by 849
Self-citationBissantz, C., Folkers, G., & Rognan, D. (2000) · Journal of Medicinal Chemistry · doi:10.1021/jm001044l · Cited by 740
Charifson, P. S., Corkery, J. J., Murcko, M. A., & Walters, W. P. (1999) · Journal of Medicinal Chemistry · doi:10.1021/jm990352k · Cited by 698
Comparative evaluation of eight docking tools for docking and virtual screening accuracy
Kellenberger, E., Rodrigo, J., Muller, P., & Rognan, D. (2004) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.20149 · Cited by 577
Cited by
Ballester, P. J., & Mitchell, J. B. O. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq112 · Cited by 989
KDEEP: Protein–Ligand Absolute Binding Affinity Prediction via 3D-Convolutional Neural Networks
Jiménez, J., Škalič, M., Martínez-Rosell, G., & De Fabritiis, G. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00650 · Cited by 926
Comparative Assessment of Scoring Functions: The CASF-2016 Update
Su, M., Yang, Q., Du, Y., Feng, G., Liu, Z., Li, Y., & Wang, R. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00545 · Cited by 824
Self-citationRole of computer-aided drug design in modern drug discovery
Macalino, S. J. Y., Gosu, V., Hong, S., & Choi, S. (2015) · Archives of Pharmacal Research · doi:10.1007/s12272-015-0640-5 · Cited by 762
A Structure-Based Drug Discovery Paradigm
Batool, M., Ahmad, B., & Choi, S. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20112783 · Cited by 744
Sun, H., Li, Y., Tian, S., Xu, L., & Hou, T. (2014) · Phys. Chem. Chem. Phys. · doi:10.1039/c4cp01388c · Cited by 738
Self-citationHalogen atoms in the modern medicinal chemistry: hints for the drug design
Hernandes, M., Cavalcanti, S. M., Moreira, D. R., de Azevedo Junior, W., & Leite, A. C. (2010) · Current Drug Targets · doi:10.2174/138945010790711996 · Cited by 723
Use of molecular docking computational tools in drug discovery
(2021) · Progress in Medicinal Chemistry · doi:10.1016/bs.pmch.2021.01.004 · Cited by 621
Hsin, K.-Y., Ghosh, S., & Kitano, H. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083922 · Cited by 606
Receptor–ligand molecular docking
Guedes, I. A., de Magalhães, C. S., & Dardenne, L. E. (2014) · Biophysical Reviews · doi:10.1007/s12551-013-0130-2 · Cited by 606
Molecular Docking: Challenges, Advances and its Use in Drug Discovery Perspective
Saikia, S., & Bordoloi, M. (2019) · Current Drug Targets · doi:10.2174/1389450119666181022153016 · Cited by 571
Huang, S.-Y., Grinter, S. Z., & Zou, X. (2010) · Physical Chemistry Chemical Physics · doi:10.1039/c0cp00151a · Cited by 508
Forging the basis for developing protein-ligand interaction scoring functions.
Liu, Z., Su, M., Han, L., Liu, J., Yang, Q., Li, Y., & Wang, R. (2017) · Accounts of Chemical Research · doi:10.1021/acs.accounts.6b00491 · Cited by 493
Self-citationEvaluation of AutoDock and AutoDock Vina on the CASF-2013 Benchmark
Gaillard, T. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00312 · Cited by 473
Fast, accurate, and reliable molecular docking with QuickVina 2
Alhossary, A., Handoko, S. D., Mu, Y., & Kwoh, C.-K. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv082 · Cited by 375
Ain, Q. U., Aleksandrova, A., Roessler, F. D., & Ballester, P. J. (2015) · WIREs Computational Molecular Science · doi:10.1002/wcms.1225 · Cited by 357
Guedes, I. A., Pereira, F. S. S., & Dardenne, L. E. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.01089 · Cited by 331
Improving scoring-docking-screening powers of protein-ligand scoring functions using random forest
Wang, C., & Zhang, Y. (2017) · Journal of Computational Chemistry · doi:10.1002/jcc.24667 · Cited by 334
Schneider, N., Lange, G., Hindle, S., Klein, R., & Rarey, M. (2013) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-012-9626-2 · Cited by 306
Li, X., Li, Y., Cheng, T., Liu, Z., & Wang, R. (2010) · Journal of Computational Chemistry · doi:10.1002/jcc.21498 · Cited by 303
Self-citationClassification of Current Scoring Functions
Liu, J., & Wang, R. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/ci500731a · Cited by 303
Self-citationDiscovery of selective bioactive small molecules by targeting an RNA dynamic ensemble
Stelzer, A. C., Frank, A. T., Kratz, J. D., Swanson, M. D., Gonzalez-Hernandez, M. J., Lee, J., Andricioaei, I., Markovitz, D. M., & Al-Hashimi, H. M. (2011) · Nature Chemical Biology · doi:10.1038/nchembio.596 · Cited by 291
DSX: A Knowledge-Based Scoring Function for the Assessment of Protein–Ligand Complexes
Neudert, G., & Klebe, G. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200274q · Cited by 264
Exploring the stability of ligand binding modes to proteins by molecular dynamics simulations
Liu, K., Watanabe, E., & Kokubo, H. (2017) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-0005-2 · Cited by 262
Cang, Z., Mu, L., & Wei, G.-W. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005929 · Cited by 257
Li, H., Leung, K., Wong, M., & Ballester, P. J. (2015) · Molecular Informatics · doi:10.1002/minf.201400132 · Cited by 252
Cang, Z., & Wei, G. (2018) · International Journal for Numerical Methods in Biomedical Engineering · doi:10.1002/cnm.2914 · Cited by 172
Beware of machine learning-based scoring functions-on the danger of developing black boxes.
Gabel, J., Desaphy, J., & Rognan, D. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500406k · Cited by 143
Ouyang, X., Zhou, S., Su, C. T. T., Ge, Z., Li, R., & Kwoh, C. K. (2013) · Journal of Computational Chemistry · doi:10.1002/jcc.23136 · Cited by 134
Damm-Ganamet, K. L., Smith, R. D., Dunbar, J. B., Stuckey, J. A., & Carlson, H. A. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci400025f · Cited by 133
Rigidity Strengthening: A Mechanism for Protein–Ligand Binding
Nguyen, D. D., Xiao, T., Wang, M., & Wei, G.-W. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00226 · Cited by 105
Systematic exploitation of multiple receptor conformations for virtual ligand screening.
Bottegoni, G., Rocchia, W., Rueda, M., Abagyan, R., & Cavalli, A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018845 · Cited by 96
Wang, Y., Guo, Y., Kuang, Q., Pu, X., Ji, Y., Zhang, Z., & Li, M. (2015) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-014-9827-y · Cited by 69
(2024) · doi:10.1038/s42256-024-00855-1 · Cited by 57
Robust scoring functions for protein-ligand interactions with quantum chemical charge models.
Wang, J.-C., Lin, J.-H., Chen, C.-M., Perryman, A. L., & Olson, A. J. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200220v · Cited by 54
Ding, B., Wang, J., Li, N., & Wang, W. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci300508m · Cited by 45
A quantum mechanics-based halogen bonding scoring function for protein-ligand interactions.
Yang, Z., Liu, Y., Chen, Z., Xu, Z., Shi, J., Chen, K., & Zhu, W. (2015) · Journal of Molecular Modeling · doi:10.1007/s00894-015-2681-6 · Cited by 23
Scoring Functions for Protein–Ligand Interactions
Sotriffer, C. (2012) · Protein‐Ligand Interactions · doi:10.1002/9783527645947.ch12 · Cited by 16
Santos-Martins, D. (2016) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-9943-y · Cited by 5
Hu, X., Shinn, P., Itkin, Z., Ye, L., Zhang, Y.-Q., Shen, M., Ford-Scheimer, S., & Hall, M. D. (2024) · ACS Pharmacology & Translational Science · doi:10.1021/acsptsci.4c00256 · Cited by 3
Metadata
- Authors
- Cheng, T., Li, X., Li, Y., Liu, Z., & Wang, R.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2009-04-09
- Volume
- 49
- Issue
- 4
- Pages
- 1079-1093
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W1993403967
- Cited by
- 513
- References
- 10
- FWCI
- 27.04
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.