The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Three-Dimensional Convolutional Neural Networks and a Cross-Docked Data Set for Structure-Based Drug Design
Open accessAbstract
One of the main challenges in drug discovery is predicting protein-ligand binding affinity. Recently, machine learning approaches have made substantial progress on this task. However, current methods of model evaluation are overly optimistic in measuring generalization to new targets, and there does not exist a standard dataset of sufficient size to compare performance between models. We present a new dataset for structure-based machine learning, the CrossDocked2020 set, with 22.5 million poses of ligands docked into multiple similar binding pockets across the Protein Data Bank and perform a comprehensive evaluation of grid-based convolutional neural network models on this dataset. We also demonstrate how the partitioning of the training data and test data can impact the results of models trained with the PDBbind dataset, how performance improves by adding more, lower-quality training data, and how training with docked poses imparts pose sensitivity to the predicted affinity of a complex. Our best performing model, an ensemble of 5 densely connected convolutional newtworks, achieves a root mean squared error of 1.42 and Pearson R of 0.612 on the affinity prediction task, an AUC of 0.956 at binding pose classification, and a 68.4% accuracy at pose selection on the CrossDocked2020 set. By providing data splits for clustered cross-validation and the raw data for the CrossDocked2020 set, we establish the first standardized dataset for training machine learning models to recognize ligands in non-cognate target structures while also greatly expanding the number of poses available for training. In order to facilitate community adoption of this dataset for benchmarking protein-ligand binding affinity prediction, we provide our models, weights, and the CrossDocked2020 set at https://github.com/gnina/models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Trott, O., & Olson, A. J. (2010) · Journal of Computational Chemistry · doi:10.1002/jcc.21334 · Cited by 38,412
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
All-Atom Empirical Potential for Molecular Modeling and Dynamics Studies of Proteins
MacKerell, A. D., Bashford, D., Bellott, M., Dunbrack, R. L., Evanseck, J. D., Field, M. J., Fischer, S., Gao, J., Guo, H., Ha, S., et al. (1998) · The Journal of Physical Chemistry B · doi:10.1021/jp973084f · Cited by 14,311
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
A Second Generation Force Field for the Simulation of Proteins, Nucleic Acids, and Organic Molecules
Cornell, W. D., Cieplak, P., Bayly, C. I., Gould, I. R., Merz, K. M., Ferguson, D. M., Spellmeyer, D. C., Fox, T., Caldwell, J. W., & Kollman, P. A. (1995) · Journal of the American Chemical Society · doi:10.1021/ja00124a002 · Cited by 13,103
Friesner, R. A., Banks, J. L., Murphy, R. B., Halgren, T. A., Klicic, J. J., Mainz, D. T., Repasky, M. P., Knoll, E. H., Shelley, M., Perry, J. K., et al. (2004) · Journal of Medicinal Chemistry · doi:10.1021/jm0306430 · Cited by 10,057
Docking and scoring in virtual screening for drug discovery: methods and applications
Kitchen, D. B., Decornez, H., Furr, J. R., & Bajorath, J. (2004) · Nature Reviews Drug Discovery · doi:10.1038/nrd1549 · Cited by 4,150
Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking.
Mysinger, M. M., Carchia, M., Irwin, J. J., & Shoichet, B. K. (2012) · Journal of Medicinal Chemistry · doi:10.1021/jm300687e · Cited by 2,412
DeepDTA: deep drug–target binding affinity prediction
Öztürk, H., Özgür, A., & Ozkirimli, E. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty593 · Cited by 1,696
JOURNAL OF CHEMICAL INFORMATION AND MODELING
(2005) · Chemical & Engineering News Archive · doi:10.1021/cen-v083n019.p031 · Cited by 1,553
Wang, R., Lai, L., & Wang, S. (2002) · Journal of Computer-Aided Molecular Design · doi:10.1023/a:1016357811882 · Cited by 1,191
Knowledge-based scoring function to predict protein-ligand interactions
Gohlke, H., Hendlich, M., & Klebe, G. (2000) · Journal of Molecular Biology · doi:10.1006/jmbi.1999.3371 · Cited by 1,157
Lessons Learned in Empirical Scoring with smina from the CSAR 2011 Benchmarking Exercise
Koes, D. R., Baumgartner, M. P., & Camacho, C. J. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci300604z · Cited by 1,136
Self-citationBallester, P. J., & Mitchell, J. B. O. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq112 · Cited by 989
Proteins: Structure, Function, and Bioinformatics
(2006) · Proteins Structure Function and Bioinformatics · doi:10.1002/(issn)1097-0134 · Cited by 943
KDEEP: Protein–Ligand Absolute Binding Affinity Prediction via 3D-Convolutional Neural Networks
Jiménez, J., Škalič, M., Martínez-Rosell, G., & De Fabritiis, G. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00650 · Cited by 926
Protein–Ligand Scoring with Convolutional Neural Networks
Ragoza, M., Hochuli, J., Idrobo, E., Sunseri, J., & Koes, D. R. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00740 · Cited by 774
Self-citationDevelopment and evaluation of a deep learning model for protein–ligand binding affinity prediction
Stepniewska-Dziubinska, M. M., Zielenkiewicz, P., & Siedlecki, P. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty374 · Cited by 663
Current Topics in Medicinal Chemistry
(2019) · Current Topics in Medicinal Chemistry · doi:10.2174/ctmc-15680266-615 · Cited by 629
Structure-Based Virtual Screening for Drug Discovery: a Problem-Centric Review
Cheng, T., Li, Q., Zhou, Z., Wang, Y., & Bryant, S. H. (2012) · The AAPS Journal · doi:10.1208/s12248-012-9322-0 · Cited by 574
Forging the basis for developing protein-ligand interaction scoring functions.
Liu, Z., Su, M., Han, L., Liu, J., Yang, Q., Li, Y., & Wang, R. (2017) · Accounts of Chemical Research · doi:10.1021/acs.accounts.6b00491 · Cited by 493
Lim, J., Ryu, S., Park, K., Choe, Y. J., Ham, J., & Kim, W. Y. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00387 · Cited by 402
Huang, S., & Zou, X. (2007) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21214 · Cited by 401
Journal of Computer-Aided Molecular Design
(2013) · Journal of Computer-Aided Molecular Design · doi:10.1007/10822.1573-4951 · Cited by 401
Free Energy Simulations Come of Age: Protein−Ligand Recognition
Simonson, T., Archontis, G., & Karplus, M. (2002) · Accounts of Chemical Research · doi:10.1021/ar010030m · Cited by 378
NNScore 2.0: A Neural-Network Receptor–Ligand Scoring Function
Durrant, J. D., & McCammon, J. A. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci2003889 · Cited by 361
Latest developments in molecular docking: 2010–2011 in review
Yuriev, E., & Ramsland, P. A. (2013) · Journal of Molecular Recognition · doi:10.1002/jmr.2266 · Cited by 344
Improving scoring-docking-screening powers of protein-ligand scoring functions using random forest
Wang, C., & Zhang, Y. (2017) · Journal of Computational Chemistry · doi:10.1002/jcc.24667 · Cited by 334
Sieg, J., Flachsenberg, F., & Rarey, M. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00712 · Cited by 321
Extension of the Universal Force Field to Metal–Organic Frameworks
Addicoat, M. A., Vankova, N., Akter, I. F., & Heine, T. (2014) · Journal of Chemical Theory and Computation · doi:10.1021/ct400952t · Cited by 293
Chen, L., Cruz, A., Ramsey, S., Dickson, C. J., Duca, J. S., Hornak, V., Koes, D. R., & Kurtzman, T. (2019) · PLOS ONE · doi:10.1371/journal.pone.0220113 · Cited by 283
Self-citationKonc, J., & Janežič, D. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq100 · Cited by 273
Quo Vadis, Virtual Screening? A Comprehensive Survey of Prospective Applications
Ripphausen, P., Nisius, B., Peltason, L., & Bajorath, J. (2010) · Journal of Medicinal Chemistry · doi:10.1021/jm101020z · Cited by 262
Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization
Wallach, I., & Heifets, A. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00403 · Cited by 260
LIT-PCBA: An Unbiased Data Set for Machine Learning and Virtual Screening
Tran-Nguyen, V.-K., Jacquemard, C., & Rognan, D. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00155 · Cited by 259
Cang, Z., Mu, L., & Wei, G.-W. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005929 · Cited by 257
From machine learning to deep learning: Advances in scoring functions for protein–ligand docking
Shen, C., Ding, J., Wang, Z., Cao, D., Ding, X., & Hou, T. (2020) · WIREs Computational Molecular Science · doi:10.1002/wcms.1429 · Cited by 244
Extension of the Universal Force Field for Metal–Organic Frameworks
Coupry, D. E., Addicoat, M. A., & Heine, T. (2016) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.6b00664 · Cited by 242
Evol and ProDy for bridging protein sequence evolution and structural dynamics
Bakan, A., Dutta, A., Mao, W., Liu, Y., Chennubhotla, C., Lezon, T. R., & Bahar, I. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu336 · Cited by 216
Comparative Assessment of Scoring Functions on an Updated Benchmark: 1. Compilation of the Test Set
Li, Y., Liu, Z., Li, J., Han, L., Liu, J., Zhao, Z., & Wang, R. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500080q · Cited by 215
Pocketome: an encyclopedia of small-molecule binding sites in 4D
Kufareva, I., Ilatovskiy, A. V., & Abagyan, R. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr825 · Cited by 175
Wójcikowski, M., Kukiełka, M., Stepniewska-Dziubinska, M. M., & Siedlecki, P. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty757 · Cited by 169
Machine‐learning scoring functions for structure‐based drug lead optimization
Li, H., Sze, K., Lü, G., & Ballester, P. J. (2020) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.1465 · Cited by 150
Damm-Ganamet, K. L., Smith, R. D., Dunbar, J. B., Stuckey, J. A., & Carlson, H. A. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci400025f · Cited by 133
A knowledge-based scoring function for protein-ligand interactions: Probing the reference state
Muegge, I. (2000) · Perspectives in Drug Discovery and Design · doi:10.1023/a:1008729005958 · Cited by 120
Krämer, C., & Gedeck, P. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100264e · Cited by 114
Learning from the ligand: using ligand-based features to improve binding affinity prediction
Boyles, F., Deane, C. M., & Morris, G. M. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz665 · Cited by 112
Imrie, F., Bradley, A., van der Schaar, M., & Deane, C. M. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00350 · Cited by 101
Morrone, J. A., Weber, J. K., Huynh, T., Luo, H., & Cornell, W. D. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00927 · Cited by 97
Low-Quality Structural and Interaction Data Improves Binding Affinity Prediction via Random Forest
Li, H., Leung, K., Wong, M., & Ballester, P. J. (2015) · Molecules · doi:10.3390/molecules200610947 · Cited by 92
Ashtawy, H. M., & Mahapatra, N. R. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00309 · Cited by 91
Structure-based Virtual Screening Approaches in Kinase-directed Drug Discovery
Bajusz, D., Ferenczy, G. G., & Keserű, G. M. (2017) · Current Topics in Medicinal Chemistry · doi:10.2174/1568026617666170224121313 · Cited by 88
Cleves, A. E., & Jain, A. N. (2007) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-007-9150-y · Cited by 73
Cross‐docking benchmark for automated pose and ranking prediction of ligand binding
Wierbowski, S. D., Wingert, B. M., Zheng, J., & Camacho, C. J. (2019) · Protein Science · doi:10.1002/pro.3784 · Cited by 69
Correcting the impact of docking pose generation error on binding affinity prediction
Li, H., Leung, K., Wong, M., & Ballester, P. J. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1169-4 · Cited by 68
libmolgrid: Graphics Processing Unit Accelerated Molecular Gridding for Deep Learning Applications
Sunseri, J., & Koes, D. R. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b01145 · Cited by 61
Self-citationCounterexample-Guided Data Augmentation
Dreossi, T., Ghosh, S., Yue, X., Keutzer, K., Sangiovanni-Vincentelli, A., & Seshia, S. A. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/286 · Cited by 60
Overview of Methods and Strategies for Conducting Virtual Small Molecule Screening
Fradera, X., & Babaoglu, K. (2017) · Current Protocols in Chemical Biology · doi:10.1002/cpch.27 · Cited by 57
Benchmarking methods and data sets for ligand enrichment assessment in virtual screening
Xia, J., Tilahun, E. L., Reid, T.-E. R., Zhang, L., & Wang, X. S. (2014) · Methods · doi:10.1016/j.ymeth.2014.11.015 · Cited by 51
Ballester, P. J., & Mitchell, J. B. O. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200057e · Cited by 49
Evaluation of Cross-Validation Strategies in Sequence-Based Binding Prediction Using Deep Learning
Rio, A. L.-D., Nonell‐Canals, A., Vidal, D., & Perera-Lluna, A. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00663 · Cited by 39
Ye, Z., Baumgartner, M. P., Wingert, B. M., & Camacho, C. J. (2016) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-9941-0 · Cited by 22
Cited by
GNINA 1.0: molecular docking with deep learning
McNutt, A. T., Francoeur, P., Aggarwal, R., Masuda, T., Meli, R., Ragoza, M., Sunseri, J., & Koes, D. R. (2021) · Journal of Cheminformatics · doi:10.1186/s13321-021-00522-2 · Cited by 861
Self-citationThe Art and Science of Molecular Docking
Paggi, J. M., Pandit, A., & Dror, R. O. (2024) · Annual Review of Biochemistry · doi:10.1146/annurev-biochem-030222-120000 · Cited by 459
Machine-learning methods for ligand–protein molecular docking
Crampon, K., Giorkallos, A., Deldossi, M., Baud, S., & Steffenel, L. A. (2022) · Drug Discovery Today · doi:10.1016/j.drudis.2021.09.007 · Cited by 388
A Comprehensive Survey on Deep Graph Representation Learning
Ju, W., Fang, Z., Gu, Y., Liu, Z., Long, Q., Qiao, Z., Qin, Y., Shen, J., Sun, F., Xiao, Z., et al. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106207 · Cited by 341
Structure-based drug design with equivariant diffusion models
Schneuing, A., Harris, C., Du, Y., Didi, K., Jamasb, A., Igashov, I., Du, W., Gomes, C., Blundell, T. L., Lio, P., et al. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00737-x · Cited by 316
Metadata
- Authors
- Francoeur, P. G., Masuda, T., Sunseri, J., Jia, A., Iovanisci, R. B., Snyder, I., & Koes, D. R.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2020-08-31
- Volume
- 60
- Issue
- 9
- Pages
- 4200-4215
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W3082411326
- Cited by
- 343
- References
- 64
- FWCI
- 10.84
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.