Multimodal Image Fusion Offers Better Spatial Resolution for Mass Spectrometry Imaging
Open accessPublished version of a preprint
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
High-resolution reconstruction has attracted increasing research interest in mass spectrometry imaging (MSI), but it remains a challenging ill-posed problem. In the present study, we proposed a deep learning model to fuse multimodal images to enhance the spatial resolution of MSI data, namely, DeepFERE. Hematoxylin and eosin (H&E) stain microscopy imaging was used to pose constraints in the process of high-resolution reconstruction to alleviate the ill-posedness. A novel model architecture was designed to achieve multi-task optimization by incorporating multi-modal image registration and fusion in a mutually reinforced framework. Experimental results demonstrated that the proposed DeepFERE model is able to produce high-resolution reconstruction images with rich chemical information and a detailed structure on both visual inspection and quantitative evaluation. In addition, our method was found to be able to improve the delimitation of the boundary between cancerous and para-cancerous regions in the MSI image. Furthermore, the reconstruction of low-resolution spatial transcriptomics data demonstrated that the developed DeepFERE model may find wider applications in biomedical fields.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similarly, the DeepFERE model fuses mass spectrometry imaging (MSI) with hematoxylin and eosin (H&E) stain microscopy to enhance spatial resolution, improving the delimitation of cancerous and para-cancerous regions (Guo et al., 2023).”
From Spatial pharmacology of anti-cancer immunomodulators: a comprehensive review (2026) · cites this paper as Guo et al., 2023
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Textural Features for Image Classification
Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1973.4309314 · Cited by 22,886
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
(2018) · doi:10.1109/cvpr40276.2018 · Cited by 4,737
Cubic convolution interpolation for digital image processing
Keys, R. (1981) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1981.1163711 · Cited by 3,255
Deep Learning for Image Super-Resolution: A Survey
Wang, Z., Chen, J., & Hoi, S. C. H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2982166 · Cited by 1,969
MALDIquant: a versatile R package for the analysis of mass spectrometry data
Gibb, S., & Strimmer, K. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts447 · Cited by 695
INet: Convolutional Networks for Biomedical Image Segmentation
Weng, W., & Zhu, X. (2021) · IEEE Access · doi:10.1109/access.2021.3053408 · Cited by 509
Transmission-mode MALDI-2 mass spectrometry imaging of cells and tissues at subcellular resolution
Niehaus, M., Soltwisch, J., Belov, M. E., & Dreisewerd, K. (2019) · Nature Methods · doi:10.1038/s41592-019-0536-2 · Cited by 429
Van de Plas, R., Yang, J., Spraggins, J., & Caprioli, R. M. (2015) · Nature Methods · doi:10.1038/nmeth.3296 · Cited by 312
Unsupervised machine learning for exploratory data analysis in imaging mass spectrometry
Verbeeck, N., Caprioli, R. M., & Van de Plas, R. (2020) · Mass Spectrometry Reviews · doi:10.1002/mas.21602 · Cited by 281
Metabolic Imaging at the Single-Cell Scale: Recent Advances in Mass Spectrometry Imaging
Gilmore, I. S., Heiles, S., & Pieterse, C. L. (2019) · Annual Review of Analytical Chemistry · doi:10.1146/annurev-anchem-061318-115516 · Cited by 192
Abdelmoula, W. M., Balluff, B., Englert, S., Dijkstra, J., Reinders, M. J. T., Walch, A., McDonnell, L. A., & Lelieveldt, B. P. F. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510227113 · Cited by 183
KRIEGSMANN, J., KRIEGSMANN, M., & CASADONTE, R. (2015) · International Journal of Oncology · doi:10.3892/ijo.2014.2788 · Cited by 165
Wang, J., Qiu, S., Chen, S., Xiong, C., Liu, H., Wang, J., Zhang, N., Hou, J., He, Q., & Nie, Z. (2015) · Analytical Chemistry · doi:10.1021/ac504294s · Cited by 158
Self-citationNeumann, E. K., Comi, T. J., Rubakhin, S. S., & Sweedler, J. V. (2019) · Angewandte Chemie International Edition · doi:10.1002/anie.201812892 · Cited by 149
Deep learning and 3D-DESI imaging reveal the hidden metabolic heterogeneity of cancer
Inglese, P., McKenzie, J. S., Mroz, A., Kinross, J., Veselkov, K., Holmes, E., Takats, Z., Nicholson, J. K., & Glen, R. C. (2017) · Chemical Science · doi:10.1039/c6sc03738k · Cited by 139
Zhang, A., Sun, H. J., Yan, G., Han, Y., Yuan, Y., & Wang, X. (2013) · Clinica Chimica Acta · doi:10.1016/j.cca.2012.12.024 · Cited by 103
Neumann, E. K., Comi, T. J., Spegazzini, N., Mitchell, J. W., Rubakhin, S. S., Gillette, M. U., Bhargava, R., & Sweedler, J. V. (2018) · Analytical Chemistry · doi:10.1021/acs.analchem.8b02913 · Cited by 76
Ahlf, D. R., Masyuko, R. N., Hummon, A. B., & Bohn, P. W. (2014) · The Analyst · doi:10.1039/c4an00826j · Cited by 65
Recent Advances in Mass Spectrometry-Based Structural Elucidation Techniques
Ma, X. (2022) · Molecules · doi:10.3390/molecules27196466 · Cited by 61
Zhang, J. C., Du, Q., Song, X., Gao, S., Pang, X., Li, Y., Zhang, R., Abliz, Z., & He, J. (2020) · Theranostics · doi:10.7150/thno.41763 · Cited by 58
Lin, L., Chen, C., Huang, Y.-C., Chung, H., Lin, C.-W., Chen, K., Peng, Y., Ding, S., Wang, M., Shen, T., et al. (2019) · Analytica Chimica Acta · doi:10.1016/j.aca.2019.11.014 · Cited by 44
Self-citationAnalytical challenges of shotgun lipidomics at different resolution of measurements
Wang, J., & Han, X. (2019) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2019.115697 · Cited by 43
Self-citationMass spectrometry imaging takes off
Doerr, A. J. (2018) · Nature Methods · doi:10.1038/nmeth.4546 · Cited by 40
Müller, M. A., Kompauer, M., Strupat, K., Heiles, S., & Spengler, B. (2020) · Journal of the American Society for Mass Spectrometry · doi:10.1021/jasms.0c00368 · Cited by 27
Correlative Hyperspectral Imaging Using a Dimensionality-Reduction-Based Image Fusion Method
Race, A. M., Rae, A., Vorng, J., Havelund, R., Dexter, A., Kumar, N., Steven, R. T., Passarelli, M. K., Tyler, B. J., Bunch, J., et al. (2020) · Analytical Chemistry · doi:10.1021/acs.analchem.9b05055 · Cited by 26
Smets, T., Keyser, T. D., Tousseyn, T., Waelkens, E., & Moor, B. D. (2021) · Analytical Chemistry · doi:10.1021/acs.analchem.0c04759 · Cited by 23
Application of super‐resolution reconstruction of sparse representation in mass spectrometry imaging
Tang, F., Bi, Y., He, J., Li, T., Abliz, Z., & Wang, X. (2015) · Rapid Communications in Mass Spectrometry · doi:10.1002/rcm.7205 · Cited by 18
Magnification of Label Maps With a Topology-Preserving Level-Set Method
Trede, D., Alexandrov, T., Sagiv, C., & Maaß, P. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2199325 · Cited by 6
Ambient Mass Spectrometry for the Intraoperative Molecular Diagnosis of Human Brain Tumors
Parry, P. V., & Engh, J. A. (2013) · Neurosurgery · doi:10.1227/01.neu.0000428422.82081.62 · Cited by 6
Research works printed but not linked: 1
- Helminiak, D.; Hu, H.; Laskin, J.; Ye, D. IS&T International Symposium on Electronic Imaging , 2021; Vol. 33, pp 2901–2907.
Cited by
Zhang, H., Zhang, J., Yuan, C., Zhang, D., Lu, D., Chen, S., Wu, Y., & Wu, C. (2024) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2024.117834 · Cited by 37
High-resolution imaging mass cytometry to map subcellular structures
Bollhagen, A., Whipman, J., Coelho, R., Heinzelmann-Schwarz, V. A., Jacob, F., & Bodenmiller, B. (2025) · Nature Methods · doi:10.1038/s41592-025-02889-8 · Cited by 34
“We compared nuclear blind deconvolution (for example, SpiDe-SR 11 ) and cross-modality deep learning 12 , have shown promise but rely on strong prior assumptions or high-resolution training data.”
Liang, Z., Guo, Y., Sharma, A., McCurdy, C. R., & Prentice, B. M. (2024) · Analytical Chemistry · doi:10.1021/acs.analchem.4c01553 · Cited by 24
Advances in multimodal mass spectrometry for single‐cell analysis and imaging enhancement
Croslow, S. W., Trinklein, T. J., & Sweedler, J. V. (2024) · FEBS Letters · doi:10.1002/1873-3468.14798 · Cited by 21
Guang, X., Gan, S., Guo, B., & Li, Y. (2024) · Rapid Communications in Mass Spectrometry · doi:10.1002/rcm.9717 · Cited by 8
Metadata
- Authors
- Guo, L., Zhu, J., Wang, K., Cheng, K.-K., Xu, J., Dong, L., Xu, X., Chen, C., Shah, M., Peng, Z., Wang, J., Cai, Z., & Dong, J.
- Affiliations
- Xiamen University (CN)
- Shanxi University (CN)
- University of Technology Malaysia (MY)
- Xiamen University Malaysia (MY)
- The University of Sydney (AU)
- Fujian Medical University (CN)
- Second Military Medical University (CN)
- Hong Kong Baptist University (HK)
- Corresponding author
- Dong, Jiyang
- ORCID
- Venue
- Analytical Chemistry
- Publisher
- American Chemical Society (ACS)
- Published
- 2023-06-09
- Volume
- 95
- Issue
- 25
- Pages
- 9714-9721
- ISSN
- 0003-2700 (ISSN-L)
- 1520-6882
- OpenAlex
- W4380202397
- Cited by
- 36
- References
- 32
- Citations by year
- 2026: 14 · 2025: 9 · 2024: 13
- FWCI
- 3.30
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Analytical Chemistry (publishedVersion)
- ChemRxiv (acceptedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Key Research and Development Program of China (2017YFC1600505, 2017YFE0191000)
- Ministry of Higher Education, Malaysia (FRGS/1/2020/WAB13/UTM/02/1)
- Natural Science Foundation of Fujian Province (2022Y0003)
- National Natural Science Foundation of China (81871445)
- Topics
- Physical Sciences › Chemistry › Spectroscopy › Mass Spectrometry Techniques and Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Keywords
- mass spectrometry imaging, multimodal image fusion, spatial resolution enhancement, high-resolution reconstruction, H&E staining, multi-task optimization, image registration, spatial transcriptomics
- MeSH terms
- Image Processing, Computer-Assisted, Microscopy, Mass Spectrometry
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.