Improving Chemical Autoencoder Latent Space and Molecular De Novo Generation Diversity with Heteroencoders
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Chemical autoencoders are attractive models as they combine chemical space navigation with possibilities for de novo molecule generation in areas of interest. This enables them to produce focused chemical libraries around a single lead compound for employment early in a drug discovery project. Here, it is shown that the choice of chemical representation, such as strings from the simplified molecular-input line-entry system (SMILES), has a large influence on the properties of the latent space. It is further explored to what extent translating between different chemical representations influences the latent space similarity to the SMILES strings or circular fingerprints. By employing SMILES enumeration for either the encoder or decoder, it is found that the decoder has the largest influence on the properties of the latent space. Training a sequence to sequence heteroencoder based on recurrent neural networks (RNNs) with long short-term memory cells (LSTM) to predict different enumerated SMILES strings from the same canonical SMILES string gives the largest similarity between latent space distance and molecular similarity measured as circular fingerprints similarity. Using the output from the code layer in quantitative structure activity relationship (QSAR) of five molecular datasets shows that heteroencoder derived vectors markedly outperforms autoencoder derived vectors as well as models built using ECFP4 fingerprints, underlining the increased chemical relevance of the latent space. However, the use of enumeration during training of the decoder leads to a marked increase in the rate of decoding to different molecules than encoded, a tendency that can be counteracted with more complex network architectures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“…neural networks (CNNs), as well as recurrent neural networks (RNNs) have been used for the encoder part [1–3], whereas the decoder part has mostly been based on RNNs with either gated recurrent units (GRU) [4] or long short-term memory cells (LSTM) [5] to enable longer range sequence memory.”
MethodScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 2×
“PCA and scaling were performed with the Scikit-Learn python package [20].”
MethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2010) · International Conference on Machine Learning · Cited by 13,995
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 2×
“Both convolutional neural networks (CNNs), as well as recurrent neural networks (RNNs) have been used for the encoder part [1–3], whereas the decoder part has mostly been based on RNNs with either gated recurrent units (GRU) [4] or long short-term memory cells (LSTM) [5] to enable longer range sequence memory.”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
The NumPy Array: A Structure for Efficient Numerical Computation
van der Walt, S., Colbert, S. C., & Varoquaux, G. (2011) · Computing in Science & Engineering · doi:10.1109/mcse.2011.37 · Cited by 9,381 · cited 1×
“The 2D coordinates of the RDKit molecule were rotated randomly up to +/- 180° around the center of coordinates before discretization into numpy [22] floating point arrays.”
MethodWeininger, D. (1988) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00057a005 · Cited by 7,436
Biopython: freely available python tools for computational molecular biology and bioinformatics
Cock, P. J. A., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., Friedberg, I., Hamelryck, T., Kauff, F., Wilczynski, B., et al. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp163 · Cited by 6,080
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 1×
“The decoder consisted of a single layer of 64 LSTM cells trained with teacher forcing [26] in batch mode.”
Method
Cited by
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models
Polykovskiy, D., Zhebrak, A., Sanchez-Lengeling, B., Golovanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Artamonov, A., Aladinskiy, V., Veselov, M., et al. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.565644 · Cited by 849
“ES structures to latent vectors. They then trained a Wasserstein GAN with gradient penalty (WGAN-GP) [97] on latent codes of an autoencoder (Figure 5). The autoencoder architecture is a heteroencoder [98] containing a two-layer bidirectional encoder with 512 LSTM units per layer. Authors added a Gaussian noise with a…”
Deep learning for molecular design—a review of the state of the art
Elton, D. C., Boukouvalas, Z., Fuge, M. D., & Chung, P. W. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c9me00039a · Cited by 683
Molecular representations in AI-driven drug discovery: a review and practical guide
David, L., Thakkar, A., Mercado, R., & Engkvist, O. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00460-5 · Cited by 587
“SMILES can thus be enumerated for data augmentation [50].”
BackgroundMolGPT: Molecular Generation Using a Transformer-Decoder Model
Bagal, V., Aggarwal, R., Vinod, P. K., & Priyakumar, U. D. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.1c00600 · Cited by 543
Artificial intelligence for natural product drug discovery
Mullowney, M. W., Duncan, K. R., Elsayed, S. S., Garg, N., van der Hooft, J. J. J., Martin, N. I., Meijer, D., Terlouw, B. R., Biermann, F., Blin, K., et al. (2023) · Nature Reviews Drug Discovery · doi:10.1038/s41573-023-00774-7 · Cited by 460
A de novo molecular generation method using latent vector based generative adversarial network
Prykhodko, O., Johansson, S. V., Kotsias, P.-C., Arús-Pous, J., Bjerrum, E. J., Engkvist, O., & Chen, H. (2019) · Journal of Cheminformatics · doi:10.1186/s13321-019-0397-9 · Cited by 402
Self-citationRandomized SMILES strings improve the quality of molecular generative models
Arús-Pous, J., Johansson, S. V., Prykhodko, O., Bjerrum, E. J., Tyrchan, C., Reymond, J.-L., Chen, H., & Engkvist, O. (2019) · Journal of Cheminformatics · doi:10.1186/s13321-019-0393-0 · Cited by 385
Self-citationAdvances in De Novo Drug Design: From Conventional to Machine Learning Methods
Mouchlis, V. D., Afantitis, A., Serra, A., Fratello, M., Papadiamantis, A. G., Aidinis, V., Lynch, I., Greco, D., & Melagraki, G. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22041676 · Cited by 353
Chemformer: a pre-trained transformer for computational chemistry
Irwin, R., Dimitriadis, S., He, J., & Bjerrum, E. J. (2022) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/ac3ffb · Cited by 343
Self-citationConstrained Bayesian optimization for automatic chemical design using variational autoencoders
Griffiths, R.-R., & Hernández-Lobato, J. M. (2020) · Chemical Science · doi:10.1039/c9sc04026a · Cited by 339
De novo molecular design and generative models
Meyers, J., Fabian, B., & Brown, N. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2021.05.019 · Cited by 292
Efficient multi-objective molecular optimization in a continuous latent space
Winter, R., Montanari, F., Steffen, A., Briem, H., Noé, F., & Clevert, D.-A. (2019) · Chemical Science · doi:10.1039/c9sc01928f · Cited by 260
Generative Models for De Novo Drug Design
Tong, X., Liu, X., Tan, X., Li, X., Jiang, J., Xiong, Z., Xu, T., Jiang, H., Qiao, N., & Zheng, M. (2021) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.1c00927 · Cited by 233
Generative molecular design in low data regimes
Moret, M., Friedrich, L., Grisoni, F., Merk, D., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0160-y · Cited by 219
“This result extends previous studies using data augmentation for generative SMILES modelling 33–35 .”
ResultGenerative machine learning for de novo drug discovery: A systematic review
Martinelli, D. D. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105403 · Cited by 177
Metadata
- Authors
- Bjerrum, E. J., & Sattarov, B.
- Venue
- Biomolecules
- Publisher
- MDPI AG
- Published
- 2018-10-30
- Volume
- 8
- Issue
- 4
- Pages
- 131
- ISSN
- 2218-273X (ISSN-L)
- OpenAlex
- W2886791556
- Cited by
- 178
- References
- 32
- FWCI
- 14.74
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.