MassGenie: A Transformer-Based Deep Learning Method for Identifying Small Molecules from Their Mass Spectra
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
The ‘inverse problem’ of mass spectrometric molecular identification (‘given a mass spectrum, calculate/predict the 2D structure of the molecule whence it came’) is largely unsolved, and is especially acute in metabolomics where many small molecules remain unidentified. This is largely because the number of experimentally available electrospray mass spectra of small molecules is quite limited. However, the forward problem (‘calculate a small molecule’s likely fragmentation and hence at least some of its mass spectrum from its structure alone’) is much more tractable, because the strengths of different chemical bonds are roughly known. This kind of molecular identification problem may be cast as a language translation problem in which the source language is a list of high-resolution mass spectral peaks and the ‘translation’ a representation (for instance in SMILES) of the molecule. It is thus suitable for attack using the deep neural networks known as transformers. We here present MassGenie, a method that uses a transformer-based deep neural network, trained on ~6 million chemical structures with augmented SMILES encoding and their paired molecular fragments as generated in silico, explicitly including the protonated molecular ion. This architecture (containing some 400 million elements) is used to predict the structure of a molecule from the various fragments that may be expected to be observed when some of its bonds are broken. Despite being given essentially no detailed nor explicit rules about molecular fragmentation methods, isotope patterns, rearrangements, neutral losses, and the like, MassGenie learns the effective properties of the mass spectral fragment and valency space, and can generate candidate molecular structures that are very close or identical to those of the ‘true’ molecules. We also use VAE-Sim, a previously published variational autoencoder, to generate candidate molecules that are ‘similar’ to the top hit. In addition to using the ‘top hits’ directly, we can produce a rank order of these by ‘round-tripping’ candidate molecules and comparing them with the true molecules, where known. As a proof of principle, we confine ourselves to positive electrospray mass spectra from molecules with a molecular mass of 500Da or lower, including those in the last CASMI challenge (for which the results are known), getting 49/93 (53%) precisely correct. The transformer method, applied here for the first time to mass spectral interpretation, works extremely effectively both for mass spectra generated in silico and on experimentally obtained mass spectra from pure compounds. It seems to act as a Las Vegas algorithm, in that it either gives the correct answer or simply states that it cannot find one. The ability to create and to ‘learn’ millions of fragmentation patterns in silico, and therefrom generate candidate structures (that do not have to be in existing libraries) directly, thus opens up entirely the field of de novo small molecule structure prediction from experimental mass spectra.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“) It is clear that the effectiveness of such networks will continue 143 to increase, since how this potency scales with the size of the training dataset and of the 144 network is broadly predictable [107-111]).”
BackgroundSumner, L. W., Amberg, A., Barrett, D., Beale, M. H., Beger, R., Daykin, C. A., Fan, T. W.-M., Fiehn, O., Goodacre, R., Griffin, J. L., et al. (2007) · Metabolomics · doi:10.1007/s11306-007-0082-2 · Cited by 5,368
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
The Human Serum Metabolome (HUSERMET) Consortium, Dunn, W. B., Broadhurst, D., Begley, P., Zelena, E., Francis-McIntyre, S., Anderson, N., Brown, M., Knowles, J. D., Halsall, A., et al. (2011) · Nature Protocols · doi:10.1038/nprot.2011.335 · Cited by 3,294 · cited 1×
“(3) We can use VAE-Sim to generate further candidate molecules that are ‘close’ in chemical space (and possess the correct molecular formula) and rank those as in (2).”
MethodSelf-citationZINC 15 – Ligand Discovery for Everyone
Sterling, T., & Irwin, J. J. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00559 · Cited by 3,224 · cited 1×
“The problem with all of the above is that the number of small molecules of potential interest (‘chemical space’) is vast [5] (maybe 1060 [6,7]) whereas the number of synthesised and purchasable molecules as recorded at the ZINC database [8] (http://zinc15.”
BackgroundAutomatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00572 · Cited by 3,106
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Self-citationAccurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · cited 2×
“Secondly, we apply scaling (as in [89]), to the overall learning rate, dependent upon the batch size considered for training.”
MethodNeural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks
Segler, M. H. S., Kogej, T., Tyrchan, C., & Waller, M. P. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00512 · Cited by 1,537
Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17
Ruddigkeit, L., van Deursen, R., Blum, L. C., & Reymond, J.-L. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci300415d · Cited by 1,505 · cited 1×
“already been done by Reymond and colleagues [45]) we can generate as many more (in 149 both type and number) as we like in silico; using these in further training, probably with 150 larger encoding networks, will require more computer power, but this is evidently going 151 to become available (as noted above, e.”
BackgroundReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435
Lin, T., Wang, Y., Liu, X., & Qiu, X. (2021) · arXiv · doi:10.48550/arxiv.2106.04554 · Cited by 1,420
Efficient transformers: A survey
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267
Efficient Transformers: A Survey
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2020) · arXiv · doi:10.48550/arxiv.2009.06732 · Cited by 1,267
Kind, T., & Fiehn, O. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-105 · Cited by 1,201
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867 · cited 1×
“[77, 122-132]), as we use in the 166 spectrum-to-structure problem for the first time here, do presently seem particularly 167 promising.”
MethodElsken, T., Metzen, J. H., & Hutter, F. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_3 · Cited by 785
Explainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718 · cited 1×
“The existence of recognisable ‘features’ that may be extracted and combined to create meaningful images is what lies at the heart of the kinds of deep learning systems applied in modern, generative image processing [99].”
BackgroundDeep learning for molecular design—a review of the state of the art
Elton, D. C., Boukouvalas, Z., Fuge, M. D., & Chung, P. W. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c9me00039a · Cited by 683
Estimation of the size of drug-like chemical space based on GDB-17 data
Polishchuk, P. G., Madzhidov, T. I., & Varnek, A. (2013) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-013-9672-4 · Cited by 619
Molecular representations in AI-driven drug discovery: a review and practical guide
David, L., Thakkar, A., Mercado, R., & Engkvist, O. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00460-5 · Cited by 587
Kell, D. B., & Oliver, S. G. (2004) · BioEssays · doi:10.1002/bies.10385 · Cited by 535
Self-citationLearning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483
High-confidence structural annotation of metabolites absent from spectral libraries
Hoffmann, M. A., Nothias, L.-F., Ludwig, M., Fleischauer, M., Gentry, E. C., Witting, M., Dorrestein, P. C., Dührkop, K., & Böcker, S. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01045-9 · Cited by 388
Kaiser, L., Gomez, A. N., Shazeer, N., Vaswani, A., Parmar, N., Jones, L., & Uszkoreit, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05137 · Cited by 253
New software tools, databases, and resources in metabolomics: updates from 2020
Misra, B. B. (2021) · Metabolomics · doi:10.1007/s11306-021-01796-1 · Cited by 227
MSNovelist: de novo structure generation from mass spectra
Stravs, M. A., Dührkop, K., Böcker, S., & Zamboni, N. (2022) · Nature Methods · doi:10.1038/s41592-022-01486-3 · Cited by 226
Scaling Laws for Autoregressive Generative Modeling
Henighan, T., Kaplan, J., Katz, M., Chen, M., Hesse, C., Jackson, J., Jun, H., Brown, T. B., Dhariwal, P., Gray, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.14701 · Cited by 150
Frozen Pretrained Transformers as Universal Computation Engines
Lu, K., Grover, A., Abbeel, P., & Mordatch, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20729 · Cited by 113
NormFormer: Improved Transformer Pretraining with Extra Normalization
Shleifer, S., Weston, J., & Ott, M. (2021) · arXiv · doi:10.48550/arxiv.2110.09456 · Cited by 24
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers
Irie, K., Schlag, I., Csordás, R., & Schmidhuber, J. (2021) · arXiv · doi:10.48550/arxiv.2106.06295 · Cited by 14 · cited 1×
“[77, 122-132]), as we use in the 166 spectrum-to-structure problem for the first time here, do presently seem particularly 167 promising.”
Method
Cited by
Annotating metabolite mass spectra with domain-inspired chemical formula transformers
Goldman, S., Wohlwend, J., Stražar, M., Haroush, G., Xavier, R. J., & Coley, C. W. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00708-3 · Cited by 87
Decoding the Foodome: Molecular Networks Connecting Diet and Health
Menichetti, G., Barabási, A.-L., & Loscalzo, J. (2024) · Annual Review of Nutrition · doi:10.1146/annurev-nutr-062322-030557 · Cited by 32
“MassGenie has showcased the validity of this approach by generating a candidate list of chemical structures through peak translation for a subset of chemicals in ZINC (66, 132).”
MethodVoronov, G., Lightheart, R., Frandsen, A., Bargh, B., Haynes, S. E., Spencer, E., Schoenhardt, K. E., Davidson, C., Schaum, A., Macherla, V. R., et al. (2022) · doi:10.1101/2022.10.09.511482 · Cited by 18
Metadata
- Authors
- Shrivastava, A. D., Swainston, N., Samanta, S., Roberts, I., Wright Muelas, M., & Kell, D. B.
- Venue
- Biomolecules
- Publisher
- MDPI AG
- Published
- 2021-11-30
- Volume
- 11
- Issue
- 12
- Pages
- 1793
- ISSN
- 2218-273X (ISSN-L)
- OpenAlex
- W3217248762
- Cited by
- 87
- References
- 40
- FWCI
- 4.62
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.