The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
MolGPT: Molecular Generation Using a Transformer-Decoder Model
Open accessAbstract
generation of molecules, termed as inverse molecular design, has been gaining enormous traction in drug design. The representation of molecules in SMILES notation as a string of characters enables the usage of state of the art models in natural language processing, such as Transformers, for molecular design in general. Inspired by generative pre-training (GPT) models that have been shown to be successful in generating meaningful text, we train a transformer-decoder on the next token prediction task using masked self-attention for the generation of druglike molecules in this study. We show that our model, MolGPT, performs on par with other previously proposed modern machine learning frameworks for molecular generation in terms of generating valid, unique, and novel molecules. Furthermore, we demonstrate that the model can be trained conditionally to control multiple properties of the generated molecules. We also show that the model can be used to generate molecules with desired scaffolds as well as desired molecular properties by conditioning the generation on scaffold SMILES strings of desired scaffolds and property values. Using saliency maps, we highlight the interpretability of the generative process of the model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,886
Weininger, D. (1988) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00057a005 · Cited by 7,436
PubChem Substance and Compound databases
Kim, S., Thiessen, P. A., Bolton, E. E., Chen, J., Fu, G., Gindulyte, A., Han, L., He, J., He, S., Shoemaker, B. A., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv951 · Cited by 5,436
ZINC − A Free Database of Commercially Available Compounds for Virtual Screening
Irwin, J. J., & Shoichet, B. K. (2005) · Journal of Chemical Information and Modeling · doi:10.1021/ci049714+ · Cited by 4,058
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00572 · Cited by 3,106
The Properties of Known Drugs. 1. Molecular Frameworks
Bemis, G. W., & Murcko, M. A. (1996) · Journal of Medicinal Chemistry · doi:10.1021/jm9602928 · Cited by 2,482
Gaulton, A., Hersey, A., Nowotka, M., Bento, A. P., Chambers, J., Mendez, D., Mutowo, P., Atkinson, F., Bellis, L. J., Cibrián-Uhalte, E., et al. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw1074 · Cited by 2,330
Quantifying the chemical beauty of drugs
Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S., & Hopkins, A. L. (2012) · Nature Chemistry · doi:10.1038/nchem.1243 · Cited by 2,242
Inverse molecular design using machine learning: Generative models for matter engineering
Sanchez-Lengeling, B., & Aspuru-Guzik, A. (2018) · Science · doi:10.1126/science.aat2663 · Cited by 2,050
Ertl, P., & Schuffenhauer, A. (2009) · Journal of Cheminformatics · doi:10.1186/1758-2946-1-8 · Cited by 1,939
The rise of deep learning in drug discovery
Chen, H., Engkvist, O., Wang, Y., Olivecrona, M., & Blaschke, T. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.01.039 · Cited by 1,770
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules.
Gómez‐Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera‐Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru‐Guzik, A. (2018) · Cambridge University Engineering Department Publications Database · doi:10.17863/cam.34134 · Cited by 1,703
Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks
Segler, M. H. S., Kogej, T., Tyrchan, C., & Waller, M. P. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00512 · Cited by 1,537
Molecular de-novo design through deep reinforcement learning
Olivecrona, M., Blaschke, T., Engkvist, O., & Chen, H. (2017) · Journal of Cheminformatics · doi:10.1186/s13321-017-0235-x · Cited by 1,324
Deep reinforcement learning for de novo drug design
Popova, M., Isayev, O., & Tropsha, A. (2018) · Science Advances · doi:10.1126/sciadv.aap7885 · Cited by 1,182
Molecular Transformer: A Model for Uncertainty-Calibrated Chemical Reaction Prediction
Schwaller, P., Laino, T., Gaudin, T., Bolgar, P., Hunter, C. A., Bekas, C., & Lee, A. A. (2019) · ACS Central Science · doi:10.1021/acscentsci.9b00576 · Cited by 928
GuacaMol: Benchmarking Models for de Novo Molecular Design
Brown, N., Fiscato, M., Segler, M. H. S., & Vaucher, A. C. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00839 · Cited by 895
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models
Polykovskiy, D., Zhebrak, A., Sanchez-Lengeling, B., Golovanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Artamonov, A., Aladinskiy, V., Veselov, M., et al. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.565644 · Cited by 849
Clark, D. E. (1999) · Journal of Pharmaceutical Sciences · doi:10.1021/js9804011 · Cited by 627
Estimation of the size of drug-like chemical space based on GDB-17 data
Polishchuk, P. G., Madzhidov, T. I., & Varnek, A. (2013) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-013-9672-4 · Cited by 619
Kadurin, A., Nikolenko, S., Khrabrov, K., Aliper, A., & Zhavoronkov, A. (2017) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.7b00346 · Cited by 587
GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders
Simonovsky, M., & Komodakis, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01418-6_41 · Cited by 563
Generative Recurrent Networks for De Novo Drug Design
Gupta, A., Müller, A. T., Huisman, B. J. H., Fuchs, J. A., Schneider, P., & Schneider, G. (2018) · Molecular Informatics · doi:10.1002/minf.201700111 · Cited by 441
Molecular generative model based on conditional variational autoencoder for de novo molecular design
Lim, J., Ryu, S., Kim, J. W., & Kim, W. Y. (2018) · Journal of Cheminformatics · doi:10.1186/s13321-018-0286-7 · Cited by 421
A de novo molecular generation method using latent vector based generative adversarial network
Prykhodko, O., Johansson, S. V., Kotsias, P.-C., Arús-Pous, J., Bjerrum, E. J., Engkvist, O., & Chen, H. (2019) · Journal of Cheminformatics · doi:10.1186/s13321-019-0397-9 · Cited by 402
Randomized SMILES strings improve the quality of molecular generative models
Arús-Pous, J., Johansson, S. V., Prykhodko, O., Bjerrum, E. J., Tyrchan, C., Reymond, J.-L., Chen, H., & Engkvist, O. (2019) · Journal of Cheminformatics · doi:10.1186/s13321-019-0393-0 · Cited by 385
Reinforced Adversarial Neural Computer for de Novo Molecular Design
Putin, E., Asadulaev, A., Ivanenkov, Y., Aladinskiy, V., Sanchez-Lengeling, B., Aspuru-Guzik, A., & Zhavoronkov, A. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00690 · Cited by 372
Fréchet ChemNet Distance: A Metric for Generative Models for Molecules in Drug Discovery
Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S., & Klambauer, G. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00234 · Cited by 343
Entangled Conditional Adversarial Autoencoder for de Novo Drug Discovery
Polykovskiy, D., Zhebrak, A., Vetrov, D., Ivanenkov, Y., Aladinskiy, V., Mamoshina, P., Bozdaganyan, M., Aliper, A., Zhavoronkov, A., & Kadurin, A. (2018) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.8b00839 · Cited by 278
Mol-CycleGAN: a generative model for molecular optimization
Maziarka, Ł., Pocha, A., Kaczmarczyk, J., Rataj, K., Danel, T., & Warchoł, M. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-019-0404-1 · Cited by 275
Efficient multi-objective molecular optimization in a continuous latent space
Winter, R., Montanari, F., Steffen, A., Briem, H., Noé, F., & Clevert, D.-A. (2019) · Chemical Science · doi:10.1039/c9sc01928f · Cited by 260
Kotsias, P.-C., Arús-Pous, J., Chen, H., Engkvist, O., Tyrchan, C., & Bjerrum, E. J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0174-5 · Cited by 234
Adversarial Threshold Neural Computer for Molecular de Novo Design
Putin, E., Asadulaev, A., Vanhaelen, Q., Ivanenkov, Y., Aladinskaya, A. V., Aliper, A., & Zhavoronkov, A. (2018) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.7b01137 · Cited by 211
SMILES-based deep generative scaffold decorator for de-novo drug design
Arús-Pous, J., Patronov, A., Bjerrum, E. J., Tyrchan, C., Reymond, J.-L., Chen, H., & Engkvist, O. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00441-8 · Cited by 200
Bjerrum, E. J., & Sattarov, B. (2018) · Biomolecules · doi:10.3390/biom8040131 · Cited by 178
Deep-learning-based inverse design model for intelligent discovery of organic molecules
Kim, K., Kang, S., Yoo, J., Kwon, Y., Nam, Y., Lee, D., Kim, I., Choi, Y.-S., Jung, Y., Kim, S., et al. (2018) · npj Computational Materials · doi:10.1038/s41524-018-0128-1 · Cited by 163
Molecular Generative Model Based on an Adversarially Regularized Autoencoder
Hong, S. H., Ryu, S., Lim, J., & Kim, W. Y. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00694 · Cited by 95
Deep learning enabled inorganic material generator
Pathak, Y., Juneja, K. S., Varma, G., Ehara, M., & Priyakumar, U. D. (2020) · Physical Chemistry Chemical Physics · doi:10.1039/d0cp03508d · Cited by 62
Self-citationMEMES: Machine learning framework for Enhanced MolEcular Screening
Mehta, S., Laghuvarapu, S., Pathak, Y., Sethi, A., Alvala, M., & Priyakumar, U. D. (2021) · Chemical Science · doi:10.1039/d1sc02783b · Cited by 45
Self-citationPathak, Y., Laghuvarapu, S., Mehta, S., & Priyakumar, U. D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5433 · Cited by 43
Self-citation
Cited by
Chemprop: A Machine Learning Package for Chemical Property Prediction
Heid, E., Greenman, K. P., Chung, Y., Li, S.-C., Graff, D. E., Vermeire, F. H., Wu, H., Green, W. H., & McGill, C. J. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01250 · Cited by 663
Attention is all you need: utilizing attention in AI-enabled drug discovery
Zhang, Y., Liu, C., Liu, M., Liu, T., Lin, H., Huang, C.-B., & Ning, L. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad467 · Cited by 377
A Comprehensive Survey on Deep Graph Representation Learning
Ju, W., Fang, Z., Gu, Y., Liu, Z., Long, Q., Qiao, Z., Qin, Y., Shen, J., Sun, F., Xiao, Z., et al. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106207 · Cited by 341
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
Sai, S., Gaur, A., Sai, R., Chamola, V., Guizani, M., & Rodrigues, J. J. P. C. (2024) · IEEE Access · doi:10.1109/access.2024.3367715 · Cited by 268
Metadata
- Authors
- Bagal, V., Aggarwal, R., Vinod, P. K., & Priyakumar, U. D.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2021-10-25
- Volume
- 62
- Issue
- 9
- Pages
- 2064-2076
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W3209056694
- Cited by
- 543
- References
- 45
- FWCI
- 28.74
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.