The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
LIT-PCBA: An Unbiased Data Set for Machine Learning and Virtual Screening
Open accessAbstract
Comparative evaluation of virtual screening methods requires a rigorous benchmarking procedure on diverse, realistic, and unbiased data sets. Recent investigations from numerous research groups unambiguously demonstrate that artificially constructed ligand sets classically used by the community (e.g., DUD, DUD-E, MUV) are unfortunately biased by both obvious and hidden chemical biases, therefore overestimating the true accuracy of virtual screening methods. We herewith present a novel data set (LIT-PCBA) specifically designed for virtual screening and machine learning. LIT-PCBA relies on 149 dose-response PubChem bioassays that were additionally processed to remove false positives and assay artifacts and keep active and inactive compounds within similar molecular property ranges. To ascertain that the data set is suited to both ligand-based and structure-based virtual screening, target sets were restricted to single protein targets for which at least one X-ray structure is available in complex with ligands of the same phenotype (e.g., inhibitor, inverse agonist) as that of the PubChem active compounds. Preliminary virtual screening on the 21 remaining target sets with state-of-the-art orthogonal methods (2D fingerprint similarity, 3D shape similarity, molecular docking) enabled us to select 15 target sets for which at least one of the three screening methods is able to enrich the top 1%-ranked compounds in true actives by at least a factor of 2. The corresponding ligand sets (training, validation) were finally unbiased by the recently described asymmetric validation embedding (AVE) procedure to afford the LIT-PCBA data set, consisting of 15 targets and 7844 confirmed active and 407,381 confirmed inactive compounds. The data set mimics experimental screening decks in terms of hit rate (ratio of active to inactive compounds) and potency distribution. It is available online at http://drugdesign.unistra.fr/LIT-PCBA for download and for benchmarking novel virtual screening methods, notably those relying on machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196
Extended-Connectivity Fingerprints
Rogers, D., & Hahn, M. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100050t · Cited by 7,524
ChEMBL: a large-scale bioactivity database for drug discovery
Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr777 · Cited by 4,382
ZINC − A Free Database of Commercially Available Compounds for Virtual Screening
Irwin, J. J., & Shoichet, B. K. (2005) · Journal of Chemical Information and Modeling · doi:10.1021/ci049714+ · Cited by 4,058
The Properties of Known Drugs. 1. Molecular Frameworks
Bemis, G. W., & Murcko, M. A. (1996) · Journal of Medicinal Chemistry · doi:10.1021/jm9602928 · Cited by 2,482
Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking.
Mysinger, M. M., Carchia, M., Irwin, J. J., & Shoichet, B. K. (2012) · Journal of Medicinal Chemistry · doi:10.1021/jm300687e · Cited by 2,412
UniProt: the universal protein knowledgebase
UniProt Consortium, T. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky092 · Cited by 2,020
Hawkins, P. C. D., Skillman, A. G., Warren, G. L., Ellingson, B. A., & Stahl, M. T. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100031x · Cited by 1,607
Benchmarking sets for molecular docking.
Huang, N., Shoichet, B. K., & Irwin, J. J. (2006) · Journal of Medicinal Chemistry · doi:10.1021/jm0608356 · Cited by 1,318
Concepts of Artificial Intelligence for Computer-Assisted Drug Discovery
Yang, X., Wang, Y., Byrne, R., Schneider, G., & Yang, S. (2019) · Chemical Reviews · doi:10.1021/acs.chemrev.8b00728 · Cited by 1,022
Comparison of Shape-Matching and Docking as Virtual Screening Tools
Hawkins, P. C. D., Skillman, A. G., & Nicholls, A. (2007) · Journal of Medicinal Chemistry · doi:10.1021/jm0603365 · Cited by 993
Evaluating Virtual Screening Methods: Good and Bad Metrics for the “Early Recognition” Problem
Truchon, J.-F., & Bayly, C. I. (2007) · Journal of Chemical Information and Modeling · doi:10.1021/ci600426e · Cited by 874
Bissantz, C., Folkers, G., & Rognan, D. (2000) · Journal of Medicinal Chemistry · doi:10.1021/jm001044l · Cited by 740
Self-citationWang, Y., Bryant, S. H., Cheng, T., Wang, J., Gindulyte, A., Shoemaker, B. A., Thiessen, P. A., He, S., & Zhang, J. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw1118 · Cited by 671
Triballeau, N., Acher, F., Brabet, I., Pin, J.-P., & Bertrand, H.-O. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm049092j · Cited by 631
Jain, A. N. (2007) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-007-9114-2 · Cited by 587
Maximum Unbiased Validation (MUV) Data Sets for Virtual Screening Based on PubChem Bioactivity Data
Rohrer, S. G., & Baumann, K. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci8002649 · Cited by 402
Boosting docking-based virtual screening with deep learning.
Pereira, J. C., Caffarena, E. R., & dos Santos, C. N. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00355 · Cited by 395
Virtual Screening Using Protein−Ligand Docking: Avoiding Artificial Enrichment
Verdonk, M. L., Berdini, V., Hartshorn, M. J., Mooij, W. T. M., Murray, C. W., Taylor, R. D., & Watson, P. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci034289q · Cited by 380
Sieg, J., Flachsenberg, F., & Rarey, M. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00712 · Cited by 321
Hill coefficients, dose–response curves and allosteric mechanisms
Prinz, H. (2010) · Journal of Chemical Biology · doi:10.1007/s12154-009-0029-3 · Cited by 284
Chen, L., Cruz, A., Ramsey, S., Dickson, C. J., Duca, J. S., Hornak, V., Koes, D. R., & Kurtzman, T. (2019) · PLOS ONE · doi:10.1371/journal.pone.0220113 · Cited by 283
McGovern, S. L., & Shoichet, B. K. (2003) · Journal of Medicinal Chemistry · doi:10.1021/jm0300330 · Cited by 276
The Light and Dark Sides of Virtual Screening: What Is There to Know?
Gimeno, A., Ojeda-Montes, M. J., Tomás-Hernández, S., Cereto-Massagué, A., Beltrán-Debón, R., Mulero, M., Pujadas, G., & Garcia-Vallvé, S. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20061375 · Cited by 267
Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization
Wallach, I., & Heifets, A. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00403 · Cited by 260
Protoss: a holistic approach to predict tautomers and protonation states in protein-ligand complexes
Bietz, S., Urbaczek, S., Schulz, B., & Rarey, M. (2014) · Journal of Cheminformatics · doi:10.1186/1758-2946-6-12 · Cited by 214
Getting the most out of PubChem for virtual screening
Kim, S. (2016) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2016.1216967 · Cited by 213
Good, A. C., & Oprea, T. I. (2008) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-007-9167-2 · Cited by 180
How to do an evaluation: pitfalls and traps
Hawkins, P. C. D., Warren, G. L., Skillman, A. G., & Nicholls, A. (2008) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-007-9166-3 · Cited by 166
Encoding Protein–Ligand Interaction Patterns in Fingerprints and Graphs
Desaphy, J., Raimbaud, E., Ducrot, P., & Rognan, D. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci300566n · Cited by 159
Self-citationParameter Estimation for Scoring Protein−Ligand Interactions Using Negative Training Data
Pham, T. A., & Jain, A. N. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm050040j · Cited by 147
Krüger, D. M., & Evers, A. (2009) · ChemMedChem · doi:10.1002/cmdc.200900314 · Cited by 125
Predictiveness curves in virtual screening
Empereur‐mot, C., Guillemain, H., Latouche, A., Zagury, J., Viallon, V., & Montès, M. (2015) · Journal of Cheminformatics · doi:10.1186/s13321-015-0100-8 · Cited by 125
Chaput, L., Martinez‐Sanz, J., Saettel, N., & Mouawad, L. (2016) · Journal of Cheminformatics · doi:10.1186/s13321-016-0167-x · Cited by 122
Decoys Selection in Benchmarking Datasets: Overview and Perspectives
Réau, M. F., Langenfeld, F., Zagury, J., Lagarde, N., & Montès, M. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.00011 · Cited by 118
The impact of in silico screening in the discovery of novel and safer drug candidates
Rognan, D. (2017) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2017.02.034 · Cited by 106
Self-citationImrie, F., Bradley, A., van der Schaar, M., & Deane, C. M. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00350 · Cited by 101
IChem: A Versatile Toolkit for Detecting, Comparing, and Predicting Protein–Ligand Interactions
Silva, F. D., Desaphy, J., & Rognan, D. (2017) · ChemMedChem · doi:10.1002/cmdc.201700505 · Cited by 98
Self-citationVogel, S., Bauer, M. R., & Boeckler, F. M. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci2001549 · Cited by 97
Tian, S., Sun, H., Li, Y., Pan, P., Li, D., & Hou, T. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci400382r · Cited by 74
Virtual Decoy Sets for Molecular Docking Benchmarks
Wallach, I., & Lilien, R. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci100374f · Cited by 60
Enhanced HTS Hit Selection via a Local Hit Rate Analysis
Posner, B. A., Xi, H. S., & Mills, J. E. J. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900113d · Cited by 48
Critical Comparison of Virtual Screening Methods against the MUV Data Set
Tiikkainen, P., Markt, P., Wolber, G., Kirchmair, J., Distinto, S., Poso, A., & Kallioniemi, O. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900249b · Cited by 45
Improving small molecule virtual screening strategies for the next generation of therapeutics
Wingert, B. M., & Camacho, C. J. (2018) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2018.06.006 · Cited by 39
Virtual Screening Meets Deep Learning
Pérez-Sianes, J., Pérez‐Sánchez, H., & Díaz, F. (2018) · Current Computer - Aided Drug Design · doi:10.2174/1573409914666181018141602 · Cited by 39
Tran‐Nguyen, V., Silva, F. D., Bret, G., & Rognan, D. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00684 · Cited by 36
Self-citation
Cited by
Kim, S., Chen, J., Cheng, T., Gindulyte, A., He, J., He, S., Li, Q., Shoemaker, B. A., Thiessen, P. A., Yu, B., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkac956 · Cited by 3,443
Francoeur, P. G., Masuda, T., Sunseri, J., Jia, A., Iovanisci, R. B., Snyder, I., & Koes, D. R. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00411 · Cited by 343
Jiang, D., Hsieh, C.-Y., Wu, Z., Kang, Y., Wang, J., Wang, E., Liao, B., Shen, C., Xu, L., Wu, J., et al. (2021) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.1c01830 · Cited by 262
AI-Driven Drug Discovery: A Comprehensive Review
Ferreira, F. J. N., & Carneiro, A. S. (2025) · ACS Omega · doi:10.1021/acsomega.5c00549 · Cited by 202
Deep contrastive learning enables genome-wide virtual screening
Jia, Y., Gao, B., Tan, J., Zheng, J., Hong, X., Zhu, W., Tan, H., Xiao, Y., Tan, L., Cai, H., et al. (2026) · Science · doi:10.1126/science.ads9530 · Cited by 66
Metadata
- Authors
- Tran-Nguyen, V.-K., Jacquemard, C., & Rognan, D.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2020-04-13
- Volume
- 60
- Issue
- 9
- Pages
- 4263-4273
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W3015572666
- Cited by
- 259
- References
- 46
- FWCI
- 10.96
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.