Cited by 895Computational Drug Discovery MethodsMachine Learning in Materials ScienceAdvanced Chemical Physics Studies

GuacaMol: Benchmarking Models for de Novo Molecular Design

Open access
Brown, Nathan, Fiscato, Marco, Segler, Marwin H.S., Vaucher, Alain C.·2019·Journal of Chemical Information and Modeling

Abstract

De novo design seeks to generate molecules with required property profiles by virtual design-make-test cycles. With the emergence of deep learning and neural generative models in many application areas, models for molecular design based on neural networks appeared recently and show promising results. However, the new models have not been profiled on consistent tasks, and comparative studies to well-established algorithms have only seldom been performed. To standardize the assessment of both classical and neural models for de novo molecular design, we propose an evaluation framework, GuacaMol, based on a suite of standardized benchmarks. The benchmark tasks encompass measuring the fidelity of the models to reproduce the property distribution of the training sets, the ability to generate novel molecules, the exploration and exploitation of chemical space, and a variety of single and multiobjective optimization tasks. The benchmarking open-source Python code and a leaderboard can be found on https://benevolent.ai/guacamol .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1021/acs.jcim.8b00839

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Mature software tools now exist to run generative models on conventional computers, and community-accepted benchmarks enable systematic assessment of generated small molecules considering predicted activity, novelty, validity, and diversity.

    “Mature software tools now exist to run generative models on conventional computers,154,157 and community-accepted benchmarks enable the systematic assessment of generated small-molecules, considering not only their predicted activity but also their novelty, validity, and diversity.158,159”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [4](4) van Hilten, N.; Chevillard, F.; Kolb, P. Virtual Compound Libraries in ComputerAssisted Drug Discovery. J. Chem. Inf. Model. in press. -
  2. [10](10) Lauck, F.; Rarey, M. In De novo Molecular Design ; Schneider, G., Ed.; Wiley Online Library, 2013; pp 325-347. -
  3. [33](33) Baskin, I. I.; Gordeeva, E. V.; Devdariani, R. O.; Zefirov, N. S.; Paliulin, V. A.; Stankevich, M. I. Methodology of the Inverse Problem Solution for the Structure Property Relation in Case of Topological Indexes. Dokl. Akad. Nauk SSSR 1989 , 307 , 613-617. -
  4. [36](36) Janz, D.; Westhuizen, J. v. d.; Paige, B.; Kusner, M.; Hern´ andez-Lobato, J. M. Learning a Generative Model for Validity in Complex Discrete Structures. ICLR 2018 Conference. 2018. -
  5. [42](42) Samanta, B.; De, A.; Ganguly, N.; Gomez-Rodriguez, M. Designing Random Graph Models Using Variational Autoencoders With Applications to Chemical Design. 2018 , arXiv:1802.05283. - · arXiv
  6. [49](49) Neil, D.; Segler, M.; Guasch, L.; Ahmed, M.; Plumbley, D.; Sellwood, M.; Brown, N. Exploring Deep Recurrent Models with Reinforcement Learning for Molecule Design. ICLR 2018 Conference. 2018. -
  7. [74](74) Gasteiger, J.; Engel, T. Chemoinformatics: a textbook ; John Wiley & Sons, 2006. -

Cited by

Sorted by citations

Authors
Brown, N., Fiscato, M., Segler, M. H. S., & Vaucher, A. C.
Affiliations
BenevolentAI (United Kingdom) (GB)
Corresponding author
  • Fiscato, Marco
  • Segler, Marwin H.S.
  • Vaucher, Alain C.
Venue
Journal of Chemical Information and Modeling
Published
2019-03-19
Volume
59
Issue
3
Pages
1096-1108
ISSN
  • 1549-9596 (ISSN-L)
  • 1549-960X
OpenAlex
W2900694120
Cited by
895
References
83
Citations by year
2026: 147 · 2025: 149 · 2024: 131 · 2023: 125 · 2022: 100 · 2021: 135 · 2020: 80 · 2019: 27 · 2017: 1
FWCI
57.68
Type
journal article
Language
en
Open access
Locations
  • Journal of Chemical Information and Modeling (publishedVersion, other-oa)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Advanced Chemical Physics Studies
Keywords
de novo molecular design, benchmarking framework, chemical space exploration, multi-objective optimization, model evaluation
MeSH terms
Deep Learning, Pharmaceutical Preparations, Isomerism, Models, Molecular, Monte Carlo Method, Drug Design, Molecular Structure, Benchmarking, Quantitative Structure-Activity Relationship

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review