Machine learning consensus scoring improves performance across targets in structure-based virtual screening.
Open accessAbstract
In structure-based virtual screening, compound ranking through a consensus of scores from a variety of docking programs or scoring functions, rather than ranking by scores from a single program, provides better predictive performance and reduces target performance variability. Here we compare traditional consensus scoring methods with a novel, unsupervised gradient boosting approach. We also observed increased score variation among active ligands and developed a statistical mixture model consensus score based on combining score means and variances. To evaluate performance, we used the common performance metrics ROCAUC and EF1 on 21 benchmark targets from DUD-E. Traditional consensus methods, such as taking the mean of quantile normalized docking scores, outperformed individual docking methods and are more robust to target variation. The mixture model and gradient boosting provided further improvements over the traditional consensus methods. These methods are readily applicable to new targets in academic research and overcome the potentially poor performance of using a single docking method on a new target.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ericksen et al. (2017) developed machine-learning consensus scoring using discrete mixture models and gradient boosting to combine scores from eight docking programs and obtained improved performance over individual scoring functions on 21 targets from the DUD-E dataset.
“For example, Ericksen et al. (2017) developed machine-learning CS using discrete mixture models and gradient boosting to combine the scores from eight docking programs and obtained improved performances than individual scoring functions on 21 targets from DUD-E dataset.”
From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Ericksen et al., 2017
Consensus scoring excels over individual scoring functions in docking-based VS and is less sensitive to protein target variation.
“In addition, they compared their machine-learning-based CS with individual scoring functions and traditional CS schemed, confirming that CS excel individual scoring functions performances in docking-based VS, being less sensitive to protein target variation.”
From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Ericksen et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · see citing works
UCSF Chimera—A visualization system for exploratory research and analysis
Pettersen, E. F., Goddard, T. D., Huang, C. C., Couch, G. S., Greenblatt, D. M., Meng, E. C., & Ferrin, T. E. (2004) · Journal of Computational Chemistry · doi:10.1002/jcc.20084 · Cited by 45,260 · see citing works
Matplotlib: a 2D graphics environment
Hunter, J. D. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.55 · Cited by 39,249 · see citing works
Trott, O., & Olson, A. J. (2010) · Journal of Computational Chemistry · doi:10.1002/jcc.21334 · Cited by 38,412 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
AutoDock4 and AutoDockTools4: Automated docking with selective receptor flexibility
Morris, G. M., Huey, R., Lindstrom, W., Sanner, M. F., Belew, R. K., Goodsell, D. S., & Olson, A. J. (2009) · Journal of Computational Chemistry · doi:10.1002/jcc.21256 · Cited by 25,745 · see citing works
McLachlan, G., & Peel, D. (2000) · Wiley Series in Probability and Statistics · doi:10.1002/0471721182 · Cited by 10,991 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104 · see citing works
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · see citing works
Automatic atom type and bond type perception in molecular mechanical calculations
Wang, J., Wang, W., Kollman, P. A., & Case, D. A. (2006) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2005.12.005 · Cited by 5,517 · see citing works
Docking and scoring in virtual screening for drug discovery: methods and applications
Kitchen, D. B., Decornez, H. Y., Furr, J. R., & Bajorath, J. (2004) · Nature Reviews Drug Discovery · doi:10.1038/nrd1549 · Cited by 4,245 · see citing works
An empirical comparison of supervised learning algorithms
Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143865 · Cited by 2,780 · see citing works
Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking.
Mysinger, M. M., Carchia, M., Irwin, J. J., & Shoichet, B. K. (2012) · Journal of Medicinal Chemistry · doi:10.1021/jm300687e · Cited by 2,412 · see citing works
Dolinsky, T. J., Czodrowski, P., Li, H., Nielsen, J. E., Jensen, J. H., Klebe, G., & Baker, N. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm276 · Cited by 2,015 · see citing works
Distributed computing in practice: the Condor experience
Thain, D., Tannenbaum, T., & Livny, M. (2005) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.938 · Cited by 1,915 · see citing works
A Critical Assessment of Docking Programs and Scoring Functions
Warren, G. L., Andrews, C. W., Capelli, A., Clarke, B., LaLonde, J. M., Lambert, M. H., Lindvall, M., Nevins, N., Semus, S. F., Senger, S., et al. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm050362n · Cited by 1,840 · see citing works
Hawkins, P. C. D., Skillman, A. G., Warren, G. L., Ellingson, B. A., & Stahl, M. T. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100031x · Cited by 1,635 · see citing works
Empirical Scoring Functions for Advanced Protein−Ligand Docking with PLANTS
Korb, O., Stützle, T., & Exner, T. E. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci800298z · Cited by 1,358 · see citing works
Structure-Based Virtual Screening for Drug Discovery: Principles, Applications and Recent Advances
Lionta, E., Spyrou, G. M., Vassilatis, D. K., & Cournia, Z. (2014) · Current Topics in Medicinal Chemistry · doi:10.2174/1568026614666140929124445 · Cited by 1,220 · see citing works
Lessons Learned in Empirical Scoring with smina from the CSAR 2011 Benchmarking Exercise
Koes, D. R., Baumgartner, M. P., & Camacho, C. J. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci300604z · Cited by 1,150 · see citing works
Pordes, R., Petravick, D., Kramer, B., Olson, D., Livny, M., Roy, A., Avery, P., Blackburn, K., Wenaus, T. J., Würthwein, F., et al. (2007) · Journal of Physics Conference Series · doi:10.1088/1742-6596/78/1/012057 · Cited by 750 · see citing works
DOCK 6: Impact of New Features and Current Docking Performance
Allen, W. J., Balius, T. E., Mukherjee, S., Brozell, S., Moustakas, D., Lang, P. T., Case, D., Kuntz, I., & Rizzo, R. C. (2015) · Journal of Computational Chemistry · doi:10.1002/jcc.23905 · Cited by 731 · see citing works
Charifson, P. S., Corkery, J. J., Murcko, M. A., & Walters, W. P. (1999) · Journal of Medicinal Chemistry · doi:10.1021/jm990352k · Cited by 698 · see citing works
rDock: A Fast, Versatile and Open Source Program for Docking Ligands to Proteins and Nucleic Acids
Carmona, S. R., Álvarez-García, D., Foloppe, N., Garmendia-Doval, A. B., Juhos, S., Schmidtke, P., Barril, X., Hubbard, R. E., & Morley, S. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003571 · Cited by 645 · see citing works
Moitessier, N., Englebienne, P., Lee, D., Lawandi, J., & Corbeil, C. R. (2008) · British Journal of Pharmacology · doi:10.1038/sj.bjp.0707515 · Cited by 514 · see citing works
FRED and HYBRID docking performance on standardized datasets
McGann, M. R. (2012) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-012-9584-8 · Cited by 507 · see citing works
Detailed Analysis of Scoring Functions for Virtual Screening
Ståhl, M., & Rarey, M. (2001) · Journal of Medicinal Chemistry · doi:10.1021/jm0003992 · Cited by 490 · see citing works
Comparison of Several Molecular Docking Programs: Pose Prediction and Virtual Screening Accuracy
Cross, J. B., Thompson, D. C., Rai, B. K., Baber, J. C., Fan, K. Y., Hu, Y., & Humblet, C. C. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900056c · Cited by 449 · see citing works
Recognizing Pitfalls in Virtual Screening: A Critical Review
Scior, T., Bender, A., Tresadern, G., Medina-Franco, J. L., Martínez-Mayorga, K., Langer, T., Cuanalo-Contreras, K., & Agrafiotis, D. K. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci200528d · Cited by 438 · see citing works
Consensus scoring for ligand/protein interactions
Clark, R. D., Strizhev, A., Leonard, J. M., Blake, J. F., & Matthew, J. B. (2002) · Journal of Molecular Graphics and Modelling · doi:10.1016/s1093-3263(01)00125-5 · Cited by 401 · see citing works
Boosting docking-based virtual screening with deep learning.
Pereira, J. C., Caffarena, E. R., & dos Santos, C. N. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00355 · Cited by 395 · see citing works
Virtual Screening Using Protein−Ligand Docking: Avoiding Artificial Enrichment
Verdonk, M. L., Berdini, V., Hartshorn, M. J., Mooij, W. T. M., Murray, C. W., Taylor, R. D., & Watson, P. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci034289q · Cited by 379 · see citing works
Recommendations for evaluation of computational methods
Jain, A. N., & Nicholls, A. O. (2008) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-008-9196-5 · Cited by 350 · see citing works
How Does Consensus Scoring Work for Virtual Library Screening? An Idealized Computer Experiment
Wang, R., & Wang, S. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci010025x · Cited by 312 · see citing works
Consensus Docking: Improving the Reliability of Docking in a Virtual Screening Context
Houston, D. R., & Walkinshaw, M. D. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci300399w · Cited by 265 · see citing works
Consensus scoring for protein–ligand interactions
Fehér, M. (2006) · Drug Discovery Today · doi:10.1016/j.drudis.2006.03.009 · Cited by 232 · see citing works
Comparison of Automated Docking Programs as Virtual Screening Tools
Cummings, M. D., DesJarlais, R. L., Gibbs, A. C., Mohan, V., & Jaeger, E. P. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm049798d · Cited by 208 · see citing works
What do we know and when do we know it?
Nicholls, A. O. (2008) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-008-9170-2 · Cited by 203 · see citing works
Large-scale inference: empirical Bayes methods for estimation, testing, and prediction
Dialsingh, I. (2012) · Journal of Applied Statistics · doi:10.1080/02664763.2012.692543 · Cited by 190 · see citing works
Docking Screens: Right for the Right Reasons?
Kolb, P., & Irwin, J. J. (2009) · Current Topics in Medicinal Chemistry · doi:10.2174/156802609789207091 · Cited by 132 · see citing works
ConsDock: A new program for the consensus analysis of protein–ligand interactions
Paul, N., & Rognan, D. (2002) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.10119 · Cited by 119 · see citing works
Fast Structure-Based Virtual Ligand Screening Combining FRED, DOCK, and Surflex
Miteva, M. A., Lee, W. H., Montès, M., & Villoutreix, B. O. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm050262h · Cited by 111 · see citing works
Extensive Consensus Docking Evaluation for Ligand Pose Prediction and Virtual Screening Studies
Tuccinardi, T., Poli, G., Romboli, V., Giordano, A., & Martinelli, A. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500424n · Cited by 104 · see citing works
Zhang, X., Wong, S., & Lightstone, F. C. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci4005145 · Cited by 89 · see citing works
Poli, G., Martinelli, A., & Tuccinardi, T. (2016) · Journal of Enzyme Inhibition and Medicinal Chemistry · doi:10.1080/14756366.2016.1193736 · Cited by 81 · see citing works
Case-specific performance of MM-PBSA, MM-GBSA, and SIE in virtual screening
Virtanen, S. I., Niinivehmas, S. P., & Pentikäinen, O. T. (2015) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2015.10.012 · Cited by 65 · see citing works
Cleves, A. E., & Jain, A. N. (2015) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-015-9846-3 · Cited by 57 · see citing works
The statistics of virtual screening and lead optimization
McGann, M. R., Nicholls, A. O., & Enyedy, I. J. (2015) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-015-9861-4 · Cited by 16 · see citing works
Approaches to Virtual Screening and Screening Library Selection
Wildman, S. A. (2013) · Current Pharmaceutical Design · doi:10.2174/1381612811319260009 · Cited by 8 · see citing works
Self-citationConsDock: A new program for the consensus analysis of protein–ligand interactions
Paul, N., & Rognan, D. (2002) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.10119.abs · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Guedes, I. A., Pereira, F. S. S., & Dardenne, L. E. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.01089 · Cited by 331 · see citing works
A marine microbiome antifungal targets urgent-threat drug-resistant fungi
Zhang, F., Zhao, M., Braun, D. R., Ericksen, S. S., Piotrowski, J. S., Nelson, J., Peng, J., Ananiev, G. E., Chanana, S., Barns, K., et al. (2020) · Science · doi:10.1126/science.abd6919 · Cited by 177 · see citing works
Self-citation
- Authors
- Ericksen, S. S., Wu, H., Zhang, H., Michael, L. A., Newton, M. A., Hoffmann, F. M., & Wildman, S. A.
- Affiliations
- University of Wisconsin–Madison (US)
- Corresponding author
- Wildman, Scott A.
- ORCID
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2017-07-12
- Volume
- 57
- Issue
- 7
- Pages
- 1579-1590
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- PMC
- PMC5872818
- OpenAlex
- W2729949860
- Cited by
- 107
- References
- 53
- Citations by year
- 2026: 11 · 2025: 5 · 2024: 7 · 2023: 5 · 2022: 13 · 2021: 23 · 2020: 11 · 2019: 19 · 2018: 12 · 2017: 1
- FWCI
- 8.82
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Chemical Information and Modeling (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Center for Advancing Translational Sciences (UL1TR000427)
- National Cancer Institute (P30 CA014520)
- Wisconsin Alumni Research Foundation
- National Institute of Allergy and Infectious Diseases (1U54AI117924)
- Division of Advanced Cyberinfrastructure (1321762)
- Division of Physics (1148698)
- Office of the Vice Chancellor for Research and Graduate Education, University of Wisconsin-Madison
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › vaccines and immunoinformatics approaches
- Keywords
- structure-based virtual screening, consensus scoring, gradient boosting, compound ranking, unsupervised learning
- MeSH terms
- Machine Learning, Drug Evaluation, Preclinical, Proteins, User-Computer Interface, Benchmarking, Molecular Targeted Therapy, Molecular Docking Simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
