On failure modes in molecule generation and optimization
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
There has been a wave of generative models for molecules triggered by advances in the field of Deep Learning. These generative models are often used to optimize chemical compounds towards particular properties or a desired biological activity. The evaluation of generative models remains challenging and suggested performance metrics or scoring functions often do not cover all relevant aspects of drug design projects. In this work, we highlight some unintended failure modes in molecular generation and optimization and how these evade detection by current performance metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Self-citationLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Weininger, D. (1988) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00057a005 · Cited by 7,436 · cited 1×
“The first wave of Deep Learning methods for molecule generation [1,2] used the SMILES [13] representation of compounds in combination with recurrent neural networks (RNNs) [14,15], which are well suited to process sequences.”
MethodDeep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00572 · Cited by 3,106
Deep learning enables rapid identification of potent DDR1 kinase inhibitors
Zhavoronkov, A., Ivanenkov, Y. A., Aliper, A., Veselov, M. S., Aladinskiy, V. A., Aladinskaya, A. V., Terentiev, V. A., Polykovskiy, D. A., Kuznetsov, M. D., Asadulaev, A., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0224-x · Cited by 1,573 · cited 1×
“Zhavoronkov et al. [31] identified a DDR1 kinase inhibitor using generative Deep Learning models.”
MethodGenerating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks
Segler, M. H. S., Kogej, T., Tyrchan, C., & Waller, M. P. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00512 · Cited by 1,537
Cited by
Boltz-2: Towards Accurate and Efficient Binding Affinity Prediction
Passaro, S., Corso, G., Wohlwend, J., Reveiz, M., Thaler, S., Somnath, V. R., Getz, N., Portnoi, T., Roy, J., Stark, H., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.14.659707 · Cited by 665
Artificial intelligence in drug development
Zhang, K., Yang, X., Wang, Y., Yu, Y., Huang, N., Li, G., Li, X., Wu, J. C., & Yang, S. (2025) · Nature Medicine · doi:10.1038/s41591-024-03434-4 · Cited by 498
REINVENT 2.0: An AI Tool for De Novo Drug Design
Blaschke, T., Arús-Pous, J., Chen, H., Margreitter, C., Tyrchan, C., Engkvist, O., Papadopoulos, K., & Patronov, A. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00915 · Cited by 496
Generative models for molecular discovery: Recent advances and challenges
Bilodeau, C., Jin, W., Jaakkola, T., Barzilay, R., & Jensen, K. F. (2022) · WIREs Computational Molecular Science · doi:10.1002/wcms.1608 · Cited by 318
Generative Models as an Emerging Paradigm in the Chemical Sciences
Anstine, D. M., & Isayev, O. (2023) · Journal of the American Chemical Society · doi:10.1021/jacs.2c13467 · Cited by 313
De novo molecular design and generative models
Meyers, J., Fabian, B., & Brown, N. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2021.05.019 · Cited by 292
SELFIES and the future of molecular string representations
Krenn, M., Ai, Q., Barthel, S., Carson, N., Frei, A., Frey, N. C., Friederich, P., Gaudin, T., Gayle, A. A., Jablonka, K. M., et al. (2022) · Patterns · doi:10.1016/j.patter.2022.100588 · Cited by 261
Artificial Intelligence for Drug Discovery: Are We There Yet?
Hasselgren, C., & Oprea, T. I. (2024) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-040323-040828 · Cited by 236
“and compounds that fall outside the desired property space (135).”
BackgroundGenerative machine learning for de novo drug discovery: A systematic review
Martinelli, D. D. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105403 · Cited by 177
In silico proof of principle of machine learning-based antibody design at unconstrained scale
Akbar, R., Robert, P. A., Weber, C. R., Widrich, M., Frank, R., Pavlović, M., Scheffer, L., Chernigovskaya, M., Snapkov, I., Slabodkin, A., et al. (2022) · mAbs · doi:10.1080/19420862.2022.2031482 · Cited by 94
Thomas, M., Smith, R. T., O’Boyle, N. M., de Graaf, C., & Bender, A. (2021) · Journal of Cheminformatics · doi:10.1186/s13321-021-00516-0 · Cited by 78
“observed different chemical space occupation for independent runs with similar models [27].”
Deep learning for low-data drug discovery: Hurdles and opportunities
van Tilborg, D., Brinkmann, H., Criscuolo, E., Rossen, L., Özçelik, R., & Grisoni, F. (2024) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2024.102818 · Cited by 61
Systemic evolutionary chemical space exploration for drug discovery
Lu, C., Liu, S., Shi, W., Yu, J., Zhou, Z., Zhang, X., Lu, X., Cai, F., Xia, N., & Wang, Y. (2022) · Journal of Cheminformatics · doi:10.1186/s13321-022-00598-4 · Cited by 30
Sc2Mol: a scaffold-based two-step molecule generator with variational autoencoder and transformer
Liao, Z., Xie, L., Mamitsuka, H., & Zhu, S. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btac814 · Cited by 24
Metadata
- Authors
- Renz, P., Van Rompaey, D., Wegner, J. K., Hochreiter, S., & Klambauer, G.
- Venue
- Drug Discovery Today: Technologies
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 32-33
- Pages
- 55-63
- ISSN
- 1740-6749 (ISSN-L)
- OpenAlex
- W3094553402
- Cited by
- 154
- References
- 18
- FWCI
- 9.99
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.