CoChat
Sign Up
Cited by 25Computational Drug Discovery MethodsMachine Learning in Materials ScienceHistory and advancements in chemistry

FragNet, a Contrastive Learning-Based Transformer Model for Clustering, Interpreting, Visualizing, and Navigating Chemical Space

Open access
Shrivastava, Aditya Divyakant, Kell, Douglas B.·2021·Molecules
DOI 10.3390/molecules26072065PMC PMC8038408
Open this paper in CoChat — free accountOpen PDF

Abstract

The question of molecular similarity is core in cheminformatics and is usually assessed via a pairwise comparison based on vectors of properties or molecular fingerprints. We recently exploited variational autoencoders to embed 6M molecules in a chemical space, such that their (Euclidean) distance within the latent space so formed could be assessed within the framework of the entire molecular set. However, the standard objective function used did not seek to manipulate the latent space so as to cluster the molecules based on any perceived similarity. Using a set of some 160,000 molecules of biological relevance, we here bring together three modern elements of deep learning to create a novel and disentangled latent space, viz transformers, contrastive learning, and an embedded autoencoder. The effective dimensionality of the latent space was varied such that clear separation of individual types of molecules could be observed within individual dimensions of the latent space. The capacity of the network was such that many dimensions were not populated at all. As before, we assessed the utility of the representation by comparing clozapine with its near neighbors, and we also did the same for various antibiotics related to flucloxacillin. Transformers, especially when as here coupled with contrastive learning, effectively provide one-shot learning and lead to a successful and disentangled representation of molecular latent spaces that at once uses the entire training set in their construction while allowing “similar” molecules to cluster together in an effective and interpretable way.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/molecules26072065

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Deep learning has been applied and shown great performance in modelling chemical spaces, gene expression, enzyme-related parameters such as enzyme affinity and EC numbers.

    “Deep learning has been applied and shown great performance in modelling chemical spaces 18 , gene expression 19 , enzyme-related parameters such as enzyme affinity 20 and EC numbers 21 .”

    From Deep learning-based kcat prediction enables improved enzyme-constrained model reconstruction (2022) · cites this paper as [18]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 167

  1. [1]LeCun Y. Bengio Y. Hinton G. Deep learning Nature 2015 521 436 444 10.1038/nature14539 26017442 · DOI · Paper page
  2. [2]Schmidhuber J. Deep learning in neural networks: An overview Neural Netw. 2015 61 85 117 10.1016/j.neunet.2014.09.003 25462637 · DOI · Paper page
  3. [3]Brown T.B. Mann B. Ryder N. Subbiah M. Kaplan J. Dhariwal P. Neelakantan A. Shyam P. Sastry G. Askell A. Language models are few-shot learners arXiv 2020 2005.14165
  4. [4]Senior A.W. Evans R. Jumper J. Kirkpatrick J. Sifre L. Green T. Qin C. Zidek A. Nelson A.W.R. Bridgland A. Improved protein structure prediction using potentials from deep learning Nature 2020 577 706 710 10.1038/s41586-019-1923-7 31942072 · DOI · Paper page
  5. [5]Samanta S. O’Hagan S. Swainston N. Roberts T.J. Kell D.B. VAE-Sim: A novel molecular similarity measure based on a variational autoencoder Molecules 2020 25 3446 10.3390/molecules25153446 PMC7435890 32751155 · DOI
  6. [6]Kingma D. Welling M. Auto-encoding variational Bayes arXiv 2014 1312.6114v1310
  7. [8]Wei R. Mahmood A. Recent advances in variational autoencoders with representation learning for biomedical informatics: A survey IEEE Access 2021 9 4939 4956 10.1109/ACCESS.2020.3048309 · DOI
  8. [9]Wei R. Garcia C. El-Sayed A. Peterson V. Mahmood A. Variations in variational autoencoders—A comparative evaluation IEEE Access 2020 8 153651 153670 10.1109/ACCESS.2020.3018151 · DOI
  9. [10]Van Deursen R. Tetko I.V. Godin G. Beyond chemical 1d knowledge using transformers arXiv 2020 2010.01027
  10. [11]Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A.N. Kaiser L. Polosukhin I. Attention is all you need arXiv 2017 1706.03762
  11. [12]Chithrananda S. Grand G. Ramsundar B. Chemberta: Large-scale self-supervised pretraining for molecular property prediction arXiv 2020 2010.09885
  12. [13]Sundararajan M. Taly A. Yan Q. Axiomatic attribution for deep networks arXiv 2017 1703.01365
  13. [14]Simonyan K. Vedaldi A. Zisserman A. Deep inside convolutional networks: Visualising image classification models and saliency maps arXiv 2013 1312.6034
  14. [15]Azodi C.B. Tang J. Shiu S.H. Opening the black box: Interpretable machine learning for geneticists Trends Genet. 2020 36 442 455 10.1016/j.tig.2020.03.005 32396837 · DOI · Paper page
  15. [16]Core M.G. Lane H.C. van Lent M. Gomboc D. Solomon S. Rosenberg M. Building explainable artificial intelligence systems AAAI 2006 1766 1773 10.21236/ada459166 · DOI
  16. [17]Holzinger A. Biemann C. Pattichis C.S. Kell D.B. What do we need to build explainable AI systems for the medical domain? arXiv 2017 1712.09923v09921
  17. [18]Samek W. Montavon G. Vedaldi A. Hansen L.K. Müller K.-R. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning Springer Berlin, Germany 2019
  18. [19]Singh A. Sengupta S. Lakshminarayanan V. Explainable deep learning models in medical image analysis arXiv 2020 2005.13799 PMC8321083 34460598
  19. [22]Gunning D. Stefik M. Choi J. Miller T. Stumpf S. Yang G.Z. XAI-explainable artificial intelligence Sci. Robot. 2019 4 eaay7120 10.1126/scirobotics.aay7120 33137719 · DOI · Paper page
  20. [23]Parmar G. Li D. Lee K. Tu Z. Dual contradistinctive generative autoencoder arXiv 2020 2011.10063
  21. [24]Peis I. Olmos P.M. Artés-Rodríguez A. Unsupervised learning of global factors in deep generative models arXiv 2020 2012.08234
  22. [25]Klys J. Snell J. Zemel R. Learning latent subspaces in variational autoencoders arXiv 2018 1812.06190
  23. [26]He Z. Kan M. Zhang J. Shan S. PA-GAN: Progressive attention generative adversarial network for facial attribute editing arXiv 2020 2007.05892
  24. [27]Shen X. Liu F. Dong H. Lian Q. Chen Z. Zhang T. Disentangled generative causal representation learning arXiv 2020 2010.02637
  25. [28]Esser P. Rombach R. Ommer B. A note on data biases in generative models arXiv 2020 2012.02516
  26. [29]Kumar A. Sattigeri P. Balakrishnan A. Variational inference of disentangled latent concepts from unlabeled observations arXiv 2017 1711.00848
  27. [30]Kim H. Mnih A. Disentangling by factorising arXiv 2018 1802.05983
  28. [31]Locatello F. Bauer S. Lucic M. Rätsch G. Gelly S. Schölkopf B. Bachem O. Challenging common assumptions in the unsupervised learning of disentangled representations arXiv 2018 1811.12359
  29. [32]Locatello F. Tschannen M. Bauer S. Rätsch G. Schölkopf B. Bachem O. Disentangling factors of variation using few labels arXiv 2019 1905.01258v01251
  30. [33]Locatello F. Poole B. Rätsch G. Schölkopf B. Bachem O. Tschannen M. Weakly-supervised disentanglement without compromises arXiv 2020 2002.02886
  31. [34]Oldfield J. Panagakis Y. Nicolaou M.A. Adversarial learning of disentangled and generalizable representations of visual attributes IEEE Trans. Neural Netw. Learn. Syst. 2021 10.1109/TNNLS.2021.3053205 33531308 · DOI
  32. [35]Pandey A. Schreurs J. Suykens J.A.K. Generative restricted kernel machines: A framework for multi-view generation and disentangled feature learning Neural Netw. 2021 135 177 191 10.1016/j.neunet.2020.12.010 33395588 · DOI
  33. [36]Hao Z. Lv D. Li Z. Cai R. Wen W. Xu B. Semi-supervised disentangled framework for transferable named entity recognition Neural Netw. 2021 135 127 138 10.1016/j.neunet.2020.11.017 33383527 · DOI
  34. [37]Shen Y. Yang C. Tang X. Zhou B. Interfacegan: Interpreting the disentangled face representation learned by gans IEEE Trans. Pattern Anal. Mach. Intell. 2020 10.1109/TPAMI.2020.3034267 33108282 · DOI
  35. [38]Tang Y. Tang Y. Zhu Y. Xiao J. Summers R.M. A disentangled generative model for disease decomposition in chest x-rays via normal image synthesis Med. Image Anal. 2021 67 101839 10.1016/j.media.2020.101839 33080508 · DOI
  36. [39]Cootes T.F. Edwards G.J. Taylor C.J. Active appearance models IEEE Trans. Pattern Anal. Mach. Intell. 2001 23 681 685 10.1109/34.927467 · DOI · Paper page
  37. [40]Cootes T.F. Taylor C.J. Cooper D.H. Graham J. Active shape models—Their training and application Comput. Vis. Image Underst. 1995 61 38 59 10.1006/cviu.1995.1004 · DOI · Paper page
  38. [41]Hill A. Cootes T.F. Taylor C.J. Active shape models and the shape approximation problem Image Vis. Comput. 1996 14 601 607 10.1016/0262-8856(96)01097-9 · DOI
  39. [42]Salam H. Seguier R. A survey on face modeling: Building a bridge between face analysis and synthesis Vis. Comput. 2018 34 289 319 10.1007/s00371-016-1332-y · DOI
  40. [43]Bozkurt A. Esmaeili B. Brooks D.H. Dy J.G. van de Meent J.-W. Evaluating combinatorial generalization in variational autoencoders arXiv 2019 1911.04594v04591
  41. [44]Alemi A.A. Poole B. Fischer I. Dillon J.V. Saurous R.A. Murphy K. Fixing a broken ELBO arXiv 2019 1711.00464
  42. [45]Zhao S. Song J. Ermon S. InfoVAE: Balancing learning and inference in variational autoencoders arXiv 2017 1706.02262v02263 10.1609/aaai.v33i01.33015885 · DOI
  43. [46]Leibfried F. Dutordoir V. John S.T. Durrande N. A tutorial on sparse Gaussian processes and variational inference arXiv 2020 2012.13962
  44. [47]Rezende D.J. Viola F. Taming VAEs arXiv 2018 1810.00597v00591
  45. [48]Dai B. Wipf D. Diagnosing and enhancing VAE models arXiv 2019 1903.05789v05782
  46. [49]Li Y. Yu S. Principe J.C. Li X. Wu D. PRI-VAE: Principle-of-relevant-information variational autoencoders arXiv 2020 2007.06503
  47. [50]Higgins I. Matthey L. Pal A. Burgess C. Glorot X. Botvinick M. Mohamed S. Lerchner A. β-VAE: Learning basic visual concepts with a constrained variational framework Proceedings of the ICLR 2017 Toulon, France 24–26 April 2017
  48. [51]Burgess C.P. Higgins I. Pal A. Matthey L. Watters N. Desjardins G. Lerchner A. Understanding disentangling in β-VAE arXiv 2018 1804.03599
  49. [52]Havtorn J.D. Frellsen J. Hauberg S. Maaløe L. Hierarchical vaes know what they don’t know arXiv 2021 2102.08248
  50. [53]Kumar A. Poole B. On implicit regularization in β-VAEs arXiv 2021 2002.00041
  51. [54]Yang T. Ren X. Wang Y. Zeng W. Zheng N. Ren P. GroupifyVAE: From group-based definition to VAE-based unsupervised representation disentanglement arXiv 2021 2102.10303
  52. [55]Gatopoulos I. Tomczak J.M. Self-supervised variational auto-encoders arXiv 2020 2010.02014 PMC8231894 34198552
  53. [56]Rong Y. Bian Y. Xu T. Xie W. Wei Y. Huang W. Huang J. Self-supervised graph transformer on large-scale molecular data arXiv 2020 2007.02835
  54. [57]Saeed A. Grangier D. Zeghidour N. Contrastive learning of general-purpose audio representations arXiv 2020 2010.10915
  55. [58]Aneja J. Schwing A. Kautz J. Vahdat A. NCP-VAE: Variational autoencoders with noise contrastive priors arXiv 2020 2010.02917
  56. [59]Artelt A. Hammer B. Efficient computation of contrastive explanations arXiv 2020 2010.02647
  57. [60]Ciga O. Martel A.L. Xu T. Self supervised contrastive learning for digital histopathology arXiv 2020 2011.13971
  58. [61]Chen T. Kornblith S. Norouzi M. Hinton G. A simple framework for contrastive learning of visual representations arXiv 2020 2002.05709
  59. [62]Jaiswal A. Babu A.R. Zadeh M.Z. Banerjee D. Makedon F. A survey on contrastive self-supervised learning arXiv 2020 2011.00362
  60. [63]Purushwalkam S. Gupta A. Demystifying contrastive self-supervised learning: Invariances, augmentations and dataset biases arXiv 2020 2007.13916
  61. [64]Van den Oord A. Li Y. Vinyals O. Representation learning with contrastive predictive coding arXiv 2018 1807.03748v03742
  62. [65]Verma V. Luong M.-T. Kawaguchi K. Pham H. Le Q.V. Towards domain-agnostic contrastive learning arXiv 2020 2011.04419
  63. [66]Le-Khac P.H. Healy G. Smeaton A.F. Contrastive representation learning: A framework and review arXiv 2020 2010.05113 10.1109/ACCESS.2020.3031549 · DOI · Paper page
  64. [67]Wang Q. Meng F. Breckon T.P. Data augmentation with norm-VAE for unsupervised domain adaptation arXiv 2020 2012.00848 36827959
  65. [68]Li H. Zhang X. Sun R. Xiong H. Tian Q. Center-wise local image mixture for contrastive representation learning arXiv 2020 2011.02697
  66. [69]You Y. Chen T. Sui Y. Chen T. Wang Z. Shen Y. Graph contrastive learning with augmentations arXiv 2020 2010.13902
  67. [70]Willett P. Similarity-based data mining in files of two-dimensional chemical structures using fingerprint measures of molecular resemblance Wires Data Min. Knowl. 2011 1 241 251 10.1002/widm.26 · DOI
  68. [71]Stumpfe D. Bajorath J. Similarity searching Wires Comput. Mol. Sci. 2011 1 260 282 10.1002/wcms.23 · DOI
  69. [72]Maggiora G. Vogt M. Stumpfe D. Bajorath J. Molecular similarity in medicinal chemistry J. Med. Chem. 2014 57 3186 3204 10.1021/jm401411z 24151987 · DOI
  70. [73]Irwin J.J. Shoichet B.K. ZINC--a free database of commercially available compounds for virtual screening J. Chem. Inf. Model. 2005 45 177 182 10.1021/ci049714+ 15667143 PMC1360656 · DOI · Paper page
  71. [74]Ertl P. Schuffenhauer A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions J. Cheminform. 2009 1 8 10.1186/1758-2946-1-8 20298526 PMC3225829 · DOI · Paper page
  72. [75]Patel H. Ihlenfeldt W.D. Judson P.N. Moroz Y.S. Pevzner Y. Peach M.L. Delannee V. Tarasova N.I. Nicklaus M.C. Savi, in silico generation of billions of easily synthesizable compounds through expert-system type rules Sci. Data 2020 7 384 10.1038/s41597-020-00727-4 33177514 PMC7658252 · DOI
  73. [76]Bickerton G.R. Paolini G.V. Besnard J. Muresan S. Hopkins A.L. Quantifying the chemical beauty of drugs Nat. Chem. 2012 4 90 98 10.1038/nchem.1243 22270643 PMC3524573 · DOI
  74. [77]Cernak T. Dykstra K.D. Tyagarajan S. Vachal P. Krska S.W. The medicinal chemist’s toolbox for late stage functionalization of drug-like molecules Chem. Soc. Rev. 2016 45 546 576 10.1039/C5CS00628G 26507237 · DOI
  75. [78]Lovrić M. Molero J.M. Kern R. PySpark and RDKit: Moving towards big data in cheminformatics Mol. Inform. 2019 38 e1800082 10.1002/minf.201800082 30844132 · DOI
  76. [79]Clyde A. Ramanathan A. Stevens R. Scaffold embeddings: Learning the structure spanned by chemical fragments, scaffolds and compounds arXiv 2021 2103.06867
  77. [80]Arús-Pous J. Awale M. Probst D. Reymond J.L. Exploring chemical space with machine learning Chem. Int. J. Chem. 2019 73 1018 1023 10.2533/chimia.2019.1018 31883554 · DOI
  78. [81]Awale M. Probst D. Reymond J.L. WebMolCS: A web-based interface for visualizing molecules in three-dimensional chemical spaces J. Chem. Inf. Model. 2017 57 643 649 10.1021/acs.jcim.6b00690 28316236 · DOI
  79. [82]Baldi P. Muller K.R. Schneider G. Charting chemical space: Challenges and opportunities for artificial intelligence and machine learning Mol. Inform. 2011 30 751 752 10.1002/minf.201180003 27467407 · DOI
  80. [83]Chen Y. Garcia de Lomana M. Friedrich N.O. Kirchmair J. Characterization of the chemical space of known and readily obtainable natural products J. Chem. Inf. Model. 2018 58 1518 1532 10.1021/acs.jcim.8b00302 30010333 · DOI
  81. [84]Drew K.L.M. Baiman H. Khwaounjoo P. Yu B. Reynisson J. Size estimation of chemical space: How big is it? J. Pharm. Pharmacol. 2012 64 490 495 10.1111/j.2042-7158.2011.01424.x 22420655 · DOI
  82. [85]Ertl P. Visualization of chemical space for medicinal chemists J. Cheminform. 2014 6 O4 10.1186/1758-2946-6-S1-O4 24765127 PMC3980134 · DOI
  83. [86]Gonzalez-Medina M. Prieto-Martinez F.D. Naveja J.J. Mendez-Lucio O. El-Elimat T. Pearce C.J. Oberlies N.H. Figueroa M. Medina-Franco J.L. Chemoinformatic expedition of the chemical space of fungal products Future Med. Chem. 2016 8 1399 1412 10.4155/fmc-2016-0079 27485744 PMC5558535 · DOI
  84. [87]Klimenko K. Marcou G. Horvath D. Varnek A. Chemical space mapping and structure-activity analysis of the chembl antiviral compound set J. Chem. Inf. Model. 2016 56 1438 1454 10.1021/acs.jcim.6b00192 27410486 · DOI
  85. [88]Lin A. Horvath D. Afonina V. Marcou G. Reymond J.L. Varnek A. Mapping of the available chemical space versus the chemical universe of lead-like compounds ChemMedChem 2018 13 540 554 10.1002/cmdc.201700561 29154440 · DOI
  86. [89]Lucas X. Gruning B.A. Bleher S. Günther S. The purchasable chemical space: A detailed picture J. Chem. Inf. Model. 2015 55 915 924 10.1021/acs.jcim.5b00116 25894297 · DOI
  87. [90]Nigam A. Friederich P. Krenn M. Aspuru-Guzik A. Augmenting genetic algorithms with deep neural networks for exploring the chemical space arXiv 2019 1909.11655
  88. [91]O’Hagan S. Kell D.B. Generation of a small library of natural products designed to cover chemical space inexpensively Pharm. Front. 2019 1 e190005 31485581 PMC6726486
  89. [92]Polishchuk P.G. Madzhidov T.I. Varnek A. Estimation of the size of drug-like chemical space based on GDB-17 data J. Comput. Aided Mol. Des. 2013 27 675 679 10.1007/s10822-013-9672-4 23963658 · DOI · Paper page
  90. [94]Rosén J. Gottfries J. Muresan S. Backlund A. Oprea T.I. Novel chemical space exploration via natural products J. Med. Chem. 2009 52 1953 1962 10.1021/jm801514w 19265440 PMC2696019 · DOI
  91. [95]Thakkar A. Selmi N. Reymond J.L. Engkvist O. Bjerrum E. ‘Ring breaker’: Neural network driven synthesis prediction of the ring system chemical space J. Med. Chem. 2020 63 8791 8808 10.1021/acs.jmedchem.9b01919 32352286 · DOI
  92. [96]Thiede L.A. Krenn M. Nigam A. Aspuru-Guzik A. Curiosity in exploring chemical space: Intrinsic rewards for deep molecular reinforcement learning arXiv 2020 2012.11293
  93. [97]Coley C.W. Defining and exploring chemical spaces Trends Chem. 2021 3 133 145 10.1016/j.trechm.2020.11.004 · DOI
  94. [98]Bender A. Glen R.C. Molecular similarity: A key technique in molecular informatics Org. Biomol. Chem. 2004 2 3204 3218 10.1039/b409813g 15534697 · DOI
  95. [100]Sterling T. Irwin J.J. ZINC 15—Ligand discovery for everyone J. Chem. Inf. Model. 2015 55 2324 2337 10.1021/acs.jcim.5b00559 26479676 PMC4658288 · DOI · Paper page
  96. [103]So D.R. Liang C. Le Q.V. The evolved transformer arXiv 2019 1901.11117
  97. [104]Grechishnikova D. Transformer neural network for protein specific de novo drug generation as machine translation problem bioRxiv 2020 10.1101/863415v863411.full PMC7801439 33432013 · DOI
  98. [106]Yun C. Bhojanapalli S. Rawat A.S. Reddi S.J. Kumar S. Are transformers universal approximators of sequence-to-sequence functions? arXiv 2019 1912.10077
  99. [108]Fedus W. Zoph B. Shazeer N. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity arXiv 2021 2101.03961
  100. [110]Goyal P. Caron M. Lefaudeux B. Xu M. Wang P. Pai V. Singh M. Liptchinsky V. Misra I. Joulin A. Self-supervised pretraining of visual features in the wild arXiv 2021 2103.01988v01981
  101. [111]Wang Y. Wang J. Cao Z. Farimani A.B. MolCLR: Molecular contrastive learning of representations via graph neural networks arXiv 2021 2102.10056
  102. [112]Chen T. Kornblith S. Swersky K. Norouzi M. Hinton G. Big self-supervised models are strong semi-supervised learners arXiv 2020 2006.10029
  103. [113]O’Hagan S. Kell D.B. Structural similarities between some common fluorophores used in biology, marketed drugs, endogenous metabolites, and natural products Mar. Drugs 2020 18 582 10.3390/md18110582 33238416 PMC7700180 · DOI
  104. [114]Ji Z. Zou X. Huang T. Wu S. Unsupervised few-shot feature learning via self-supervised training Front. Comput. Neurosci. 2020 14 83 10.3389/fncom.2020.00083 33178000 PMC7592391 · DOI
  105. [115]Wang Y. Yao Q. Kwok J. Ni L.M. Generalizing from a few examples: A survey on few-shot learning arXiv 2019 1904.05046
  106. [117]Li F.-F. Fergus R. Perona P. One-shot learning of object categories IEEE Trans. Pattern Anal. Mach. Intell. 2006 28 594 611 16566508
  107. [118]Rezende D.J. Mohamed S. Danihelka I. Gregor K. Wierstra D. One-shot generalization in deep generative models arXiv 2016 1603.05106v05101
  108. [119]Altae-Tran H. Ramsundar B. Pappu A.S. Pande V. Low data drug discovery with one-shot learning ACS Cent. Sci. 2017 3 283 293 10.1021/acscentsci.6b00367 28470045 PMC5408335 · DOI · Paper page
  109. [120]Baskin I.I. Is one-shot learning a viable option in drug discovery? Expert Opin. Drug Discov. 2019 14 601 603 10.1080/17460441.2019.1593368 30882252 · DOI
  110. [121]He X. Zhao K.Y. Chu X.W. AutoML: A survey of the state-of-the-art Knowl. Based Syst. 2021 212 106622 10.1016/j.knosys.2020.106622 · DOI
  111. [122]Chochlakis G. Georgiou E. Potamianos A. End-to-end generative zero-shot learning via few-shot learning arXiv 2021 2102.04379
  112. [123]Majumder O. Ravichandran A. Maji S. Polito M. Bhotika R. Soatto S. Revisiting contrastive learning for few-shot classification arXiv 2021 2101.11058
  113. [124]Dasari S. Gupta A. Transformers for one-shot visual imitation arXiv 2020 2011.05970
  114. [125]Logeswaran L. Lee A. Ott M. Lee H. Ranzato M.A. Szlam A. Few-shot sequence learning with transformers arXiv 2020 2012.09543
  115. [126]Belkin M. Hsu D. Ma S. Mandal S. Reconciling modern machine-learning practice and the classical bias-variance trade-off Proc. Natl. Acad. Sci. USA 2019 116 15849 15854 10.1073/pnas.1903070116 31341078 PMC6689936 · DOI · Paper page
  116. [128]Van der Maaten L. Learning a parametric embedding by preserving local structure Proc. AISTATS 2009 384 391
  117. [129]McInnes L. Healy J. Melville J. UMAP: Uniform manifold approximation and projection for dimension reduction arXiv 2018 1802.03426v03422
  118. [130]McInnes L. Healy J. Saul N. Großberger L. UMAP: Uniform manifold approximation and projection J. Open Source Softw. 2018 10.21105/joss.00861 · DOI · Paper page
  119. [131]Dickens D. Rädisch S. Chiduza G.N. Giannoudis A. Cross M.J. Malik H. Schaeffeler E. Sison-Young R.L. Wilkinson E.L. Goldring C.E. Cellular uptake of the atypical antipsychotic clozapine is a carrier-mediated process Mol. Pharm. 2018 15 3557 3572 10.1021/acs.molpharmaceut.8b00547 29944835 · DOI
  120. [132]Horvath D. Jeandenans C. Neighborhood behavior of in silico structural spaces with respect to in vitro activity spaces-a novel understanding of the molecular similarity principle in the context of multiple receptor binding profiles J. Chem. Inf. Comput. Sci. 2003 43 680 690 10.1021/ci025634z 12653538 · DOI
  121. [133]Bender A. Jenkins J.L. Li Q.L. Adams S.E. Cannon E.O. Glen R.C. Molecular similarity: Advances in methods, applications and validations in virtual screening and qsar Annu. Rep. Comput. Chem. 2006 2 141 168 32362803 PMC7185533
  122. [134]Horvath D. Koch C. Schneider G. Marcou G. Varnek A. Local neighborhood behavior in a combinatorial library context J. Comput. Aid. Mol. Des. 2011 25 237 252 10.1007/s10822-011-9416-2 21318427 · DOI
  123. [135]Gasteiger J. Handbook of Chemoinformatics: From Data to Knowledge Wiley/VCH Weinheim, Germany 2003
  124. [136]Bajorath J. Chemoinformatics: Concepts, Methods and Tools for Drug Discovery Humana Press Totowa, NJ, USA 2004
  125. [137]Sutherland J.J. Raymond J.W. Stevens J.L. Baker T.K. Watson D.E. Relating molecular properties and in vitro assay results to in vivo drug disposition and toxicity outcomes J. Med. Chem. 2012 55 6455 6466 10.1021/jm300684u 22716080 · DOI
  126. [138]Capecchi A. Probst D. Reymond J.L. One molecular fingerprint to rule them all: Drugs, biomolecules, and the metabolome J. Cheminform. 2020 12 43 10.1186/s13321-020-00445-4 33431010 PMC7291580 · DOI
  127. [139]Muegge I. Mukherjee P. An overview of molecular fingerprint similarity search in virtual screening Expert Opin. Drug Discov. 2016 11 137 148 10.1517/17460441.2016.1117070 26558489 · DOI
  128. [140]Nisius B. Bajorath J. Rendering conventional molecular fingerprints for virtual screening independent of molecular complexity and size effects ChemMedChem 2010 5 859 868 10.1002/cmdc.201000089 20425878 · DOI
  129. [141]Riniker S. Landrum G.A. Similarity maps—A visualization strategy for molecular fingerprints and machine-learning methods J. Cheminform. 2013 5 43 10.1186/1758-2946-5-43 24063533 PMC3852750 · DOI
  130. [142]Vogt I. Stumpfe D. Ahmed H.E. Bajorath J. Methods for computer-aided chemical biology. Part 2: Evaluation of compound selectivity using 2d molecular fingerprints Chem. Biol. Drug Des. 2007 70 195 205 10.1111/j.1747-0285.2007.00555.x 17718714 · DOI
  131. [143]O’Hagan S. Swainston N. Handl J. Kell D.B. A ‘rule of 0.5′ for the metabolite-likeness of approved pharmaceutical drugs Metabolomics 2015 11 323 339 10.1007/s11306-014-0733-z 25750602 PMC4342520 · DOI
  132. [144]O’Hagan S. Kell D.B. Understanding the foundations of the structural similarities between marketed drugs and endogenous human metabolites Front. Pharm. 2015 6 105 10.3389/fphar.2015.00105 PMC4429554 26029108 · DOI
  133. [145]O’Hagan S. Kell D.B. The apparent permeabilities of Caco-2 cells to marketed drugs: Magnitude, and independence from both biophysical properties and endogenite similarities Peer J. 2015 3 e1405 10.7717/peerj.1405 26618081 PMC4655101 · DOI
  134. [146]O’Hagan S. Kell D.B. MetMaxStruct: A Tversky-similarity-based strategy for analysing the (sub)structural similarities of drugs and endogenous metabolites Front. Pharm. 2016 7 266 10.3389/fphar.2016.00266 PMC4992690 27597830 · DOI
  135. [147]O’Hagan S. Kell D.B. Analysis of drug-endogenous human metabolite similarities in terms of their maximum common substructures J. Cheminform. 2017 9 18 10.1186/s13321-017-0198-y 28316656 PMC5344883 · DOI
  136. [148]O’Hagan S. Kell D.B. Analysing and navigating natural products space for generating small, diverse, but representative chemical libraries Biotechnol. J. 2018 13 1700503 10.1002/biot.201700503 29168302 · DOI
  137. [149]Gawehn E. Hiss J.A. Schneider G. Deep learning in drug discovery Mol. Inform. 2016 35 3 14 10.1002/minf.201501008 27491648 · DOI · Paper page
  138. [150]Gómez-Bombarelli R. Wei J.N. Duvenaud D. Hernández-Lobato J.M. Sánchez-Lengeling B. Sheberla D. Aguilera-Iparraguirre J. Hirzel T.D. Adams R.P. Aspuru-Guzik A. Automatic chemical design using a data-driven continuous representation of molecules ACS Cent. Sci. 2018 4 268 276 10.1021/acscentsci.7b00572 29532027 PMC5833007 · DOI · Paper page
  139. [151]Sanchez-Lengeling B. Aspuru-Guzik A. Inverse molecular design using machine learning: Generative models for matter engineering Science 2018 361 360 365 10.1126/science.aat2663 30049875 · DOI
  140. [152]Arús-Pous J. Probst D. Reymond J.L. Deep learning invades drug design and synthesis Chimia 2018 72 70 71 10.2533/chimia.2018.70 29490798 · DOI
  141. [153]Yang K. Swanson K. Jin W. Coley C. Eiden P. Gao H. Guzman-Perez A. Hopper T. Kelley B. Mathea M. Analyzing learned molecular representations for property prediction J. Chem. Inf. Model. 2019 59 3370 3388 10.1021/acs.jcim.9b00237 31361484 PMC6727618 · DOI · Paper page
  142. [155]Khemchandani Y. O’Hagan S. Samanta S. Swainston N. Roberts T.J. Bollegala D. Kell D.B. DeepGraphMolGen, a multiobjective, computational strategy for generating molecules with desirable properties: A graph convolution and reinforcement learning approach J. Cheminform. 2020 12 53 10.1186/s13321-020-00454-3 33431037 PMC7487898 · DOI
  143. [156]Shen C. Krenn M. Eppel S. Aspuru-Guzik A. Deep molecular dreaming: Inverse machine learning for de-novo molecular design and interpretability with surjective representations arXiv 2020 2012.09712
  144. [157]Moret M. Friedrich L. Grisoni F. Merk D. Schneider G. Generative molecular design in low data regimes Nat. Mach. Intell. 2020 2 171 180 10.1038/s42256-020-0160-y · DOI · Paper page
  145. [158]Kell D.B. Samanta S. Swainston N. Deep learning and generative methods in cheminformatics and chemical biology: Navigating small molecule space intelligently Biochem. J. 2020 477 4559 4580 10.1042/BCJ20200781 33290527 PMC7733676 · DOI
  146. [159]Walters W.P. Barzilay R. Applications of deep learning in molecule generation and molecular property prediction Acc. Chem. Res. 2021 54 263 270 10.1021/acs.accounts.0c00699 33370107 · DOI
  147. [160]Zaheer M. Guruganesh G. Dubey A. Ainslie J. Alberti C. Ontanon S. Pham P. Ravula A. Wang Q. Yang L. Big bird: Transformers for longer sequences arXiv 2020 2007.14062
  148. [161]Hutson M. The language machines Nature 2021 591 22 25 10.1038/d41586-021-00530-0 33658699 · DOI
  149. [162]Topal M.O. Bas A. van Heerden I. Exploring transformers in natural language generation: GPT, BERT, and XLNET arXiv 2021 2102.08036
  150. [163]Zandie R. Mahoor M.H. Topical language generation using transformers arXiv 2021 2103.06434
  151. [164]Weininger D. Smiles, a chemical language and information system.1. Introduction to methodology and encoding rules J. Chem. Inf. Comput. Sci. 1988 28 31 36 10.1021/ci00057a005 · DOI · Paper page
  152. [165]Tetko I.V. Karpov P. Van Deursen R. Godin G. State-of-the-art augmented NLP transformer models for direct and single-step retrosynthesis Nat. Commun. 2020 11 5575 10.1038/s41467-020-19266-y 33149154 PMC7643129 · DOI
  153. [166]Lim S. Lee Y.O. Predicting chemical properties using self-attention multi-task learning based on SMILES representation arXiv 2020 2010.11272
  154. [167]Pflüger P.M. Glorius F. Molecular machine learning: The future of synthetic chemistry? Angew. Chem. Int. Ed. Engl. 2020 10.1002/anie.202008366 32931084 · DOI
  155. [168]Shin B. Park S. Bak J. Ho J.C. Controlled molecule generator for optimizing multiple chemical properties arXiv 2020 2010.13908 PMC8860388 35194593
  156. [169]Liu X. Zhang F. Hou Z. Mian L. Wang Z. Zhang J. Tang J. Self-supervised learning: Generative or contrastive arXiv 2020 2006.08218v08214
  157. [170]Wanyan T. Honarvar H. Jaladanki S.K. Zang C. Naik N. Somani S. Freitas J.K.D. Paranjpe I. Vaid A. Miotto R. Contrastive learning improves critical event prediction in COVID-19 patients arXiv 2021 2101.04013 PMC8542449 34723227
  158. [171]Kostas D. Aroca-Ouellette S. Rudzicz F. Bendr: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data arXiv 2021 2101.12037 PMC8261053 34248521
  159. [173]Botvinick M. Barrett D.G.T. Battaglia P. de Freitas N. Kumaran D. Leibo J.Z. Lillicrap T. Modayil J. Mohamed S. Rabinowitz N.C. Building machines that learn and think for themselves Behav. Brain Sci. 2017 40 e255 10.1017/S0140525X17000048 29342685 · DOI
  160. [175]Shevlin H. Vold K. Crosby M. Halina M. The limits of machine intelligence despite progress in machine intelligence, artificial general intelligence is still a major challenge EMBO Rep. 2019 20 10.15252/embr.201949177 PMC6776890 31531926 · DOI
  161. [176]Pei J. Deng L. Song S. Zhao M. Zhang Y. Wu S. Wang G. Zou Z. Wu Z. He W. Towards artificial general intelligence with hybrid Tianjic chip architecture Nature 2019 572 106 111 10.1038/s41586-019-1424-8 31367028 · DOI · Paper page
  162. [177]Stanley K.O. Clune J. Lehman J. Miikkulainen R. Designing neural networks through neuroevolution Nat. Mach. Intell. 2019 1 24 35 10.1038/s42256-018-0006-z · DOI
  163. [178]Zhang Y. Qu P. Ji Y. Zhang W. Gao G. Wang G. Song S. Li G. Chen W. Zheng W. A system hierarchy for brain-inspired computing Nature 2020 586 378 384 10.1038/s41586-020-2782-y 33057220 · DOI
  164. [179]Nadji-Tehrani M. Eslami A. A brain-inspired framework for evolutionary artificial general intelligence IEEE Trans. Neural Netw. Learn. Syst. 2020 31 5257 5271 10.1109/TNNLS.2020.2965567 32175876 · DOI
  165. [180]Bjerrum E.J. SMILES enumeration as data augmentation for neural network modeling of molecules arXiv 2017 1703.07076
  166. [181]Sohn K. Improved deep metric learning with multi-class n-pair loss objective NIPS 2016 30 1857 1865
  167. [183]Kingma D.P. Ba J.L. Adam: A method for stochastic optimization arXiv 2015 1412.6980v1418

Cited by

Sorted by citations

Authors
Shrivastava, A. D., & Kell, D. B.
Affiliations
University of Liverpool (GB)
Corresponding author
Kell, Douglas B.
ORCID
Kell, Douglas B.
Venue
Molecules
Publisher
MDPI AG
Published
2021-04-03
Volume
26
Issue
7
Pages
2065
ISSN
1420-3049 (ISSN-L)
DOI
10.3390/molecules26072065
PMC
PMC8038408
OpenAlex
W3147221285
Cited by
25
References
14
Citations by year
2026: 2 · 2025: 1 · 2024: 7 · 2023: 6 · 2022: 4 · 2021: 5
FWCI
2.90
Type
journal article
Language
en
Open access
Licence
Locations
  • Molecules (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Technical University of Denmark, DTU Orbit (Technical University of Denmark, DTU) (submittedVersion, other-oa)
Funders
Novo Nordisk Fonden (NNF10CC1016517)
Topics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
  • Physical Sciences › Chemistry › Physical and Theoretical Chemistry › History and advancements in chemistry
Keywords
chemical space, molecular embeddings, contrastive learning, transformer models, variational autoencoder, disentangled latent space, molecular clustering, one-shot learning, molecular fingerprints, clozapine, flucloxacillin, antibiotics, interpretability
MeSH terms
Deep Learning, Cheminformatics, Clozapine, Floxacillin, Software, Temperature, Cluster Analysis, Learning Curve

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review