The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization
Open accessAbstract
Undetected overfitting can occur when there are significant redundancies between training and validation data. We describe AVE, a new measure of training-validation redundancy for ligand-based classification problems, that accounts for the similarity among inactive molecules as well as active ones. We investigated seven widely used benchmarks for virtual screening and classification, and we show that the amount of AVE bias strongly correlates with the performance of ligand-based predictive methods irrespective of the predicted property, chemical fingerprint, similarity measure, or previously applied unbiasing techniques. Therefore, it may be the case that the previously reported performance of most ligand-based methods can be explained by overfitting to benchmarks rather than good prospective accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Extended-Connectivity Fingerprints
Rogers, D., & Hahn, M. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100050t · Cited by 7,524
The Properties of Known Drugs. 1. Molecular Frameworks
Bemis, G. W., & Murcko, M. A. (1996) · Journal of Medicinal Chemistry · doi:10.1021/jm9602928 · Cited by 2,482
Directory of useful decoys, enhanced (DUD-E): better ligands and decoys for better benchmarking.
Mysinger, M. M., Carchia, M., Irwin, J. J., & Shoichet, B. K. (2012) · Journal of Medicinal Chemistry · doi:10.1021/jm300687e · Cited by 2,412
Algorithms for hierarchical clustering: an overview
Murtagh, F., & Contreras, P. (2012) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.53 · Cited by 1,874
Reoptimization of MDL Keys for Use in Drug Discovery
Durant, J. L., Leland, B. A., Henry, D. R., & Nourse, J. G. (2002) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci010132r · Cited by 1,862
Molecular graph convolutions: moving beyond fingerprints
Kearnes, S., McCloskey, K., Berndl, M., Pande, V., & Riley, P. (2016) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-9938-8 · Cited by 1,631
Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17
Ruddigkeit, L., van Deursen, R., Blum, L. C., & Reymond, J.-L. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci300415d · Cited by 1,505
The ChEMBL bioactivity database: an update
Bento, A. P., Gaulton, A., Hersey, A., Bellis, L. J., Chambers, J., Davies, M., Krüger, F. A., Light, Y., Mak, L., McGlinchey, S., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1031 · Cited by 1,492
The SIDER database of drugs and side effects
Kuhn, M., Letunic, I., Jensen, L. J., & Bork, P. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1075 · Cited by 1,451
Cited by
Sieg, J., Flachsenberg, F., & Rarey, M. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00712 · Cited by 321
Synthetic organic chemistry driven by artificial intelligence
de Almeida, A. F., Moreira, R., & Rodrigues, T. (2019) · Nature Reviews Chemistry · doi:10.1038/s41570-019-0124-0 · Cited by 303
Chen, L., Cruz, A., Ramsey, S., Dickson, C. J., Duca, J. S., Hornak, V., Koes, D. R., & Kurtzman, T. (2019) · PLOS ONE · doi:10.1371/journal.pone.0220113 · Cited by 283
LIT-PCBA: An Unbiased Data Set for Machine Learning and Virtual Screening
Tran-Nguyen, V.-K., Jacquemard, C., & Rognan, D. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00155 · Cited by 259
Janet, J. P., Ramesh, S., Duan, C., & Kulik, H. J. (2020) · ACS Central Science · doi:10.1021/acscentsci.0c00026 · Cited by 200
Metadata
- Authors
- Wallach, I., & Heifets, A.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2018-04-26
- Volume
- 58
- Issue
- 5
- Pages
- 916-932
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W2803094965
- Cited by
- 260
- References
- 32
- FWCI
- 14.03
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization. https://app.cochat.ai/papers/10.1021/acs.jcim.7b00403
- The paper
- Wallach, I., & Heifets, A. (2018). Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization. Journal of Chemical Information and Modeling. https://doi.org/10.1021/acs.jcim.7b00403
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.