Bridging the gap between structural and metabolic neuroimaging via MRI-to-PET synthesis: a tri-attention enhanced GAN approach
Abstract
Magnetic resonance imaging (MRI) and positron emission tomography (PET) are two essential neuroimaging modalities that provide complementary structural and metabolic information about the brain, thereby enhancing diagnostic precision for brain disorders such as Alzheimer's disease (AD). To address the limitations of missing modality data, we propose a novel 3D GAN-based framework for MRI-to-PET neuroimage synthesis, incorporating a Tri-Attention module to integrate spatial, channel, and frequency attention across multiple scales. The proposed method enables the generation of complementary metabolism information by synthesizing PET scans, effectively bridging the modality gap. The effectiveness of the proposed method is evaluated on a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Experimental results demonstrate the superiority of our approach, achieving significant improvements in image quality metrics (SSIM: 0.882, PSNR: 26.508) and clinical metrics (SUVR), outperforming state-of-the-art methods. These findings underscore the potential of our framework to bridge the gap between structural and metabolic information, offering a promising tool for cross-modality neuroimage synthesis and clinical applications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tri-Attention enhanced 3D-GAN successfully generates high-quality PET images from MRI.
“innovative GAN architectures like the Tri-Attention enhanced 3D-GAN successfully generate high-quality PET images from MRI”
From Multimodal neuroimaging and AI integration in cognitive disorders: advances, challenges, and future directions for precision medicine (2026) · cites this paper as Sheng et al., 2025
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
Fischl, B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.021 · Cited by 9,526 · see citing works
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00917 · Cited by 4,514 · see citing works
Jack, C. R., Knopman, D. S., Jagust, W. J., Petersen, R. C., Weiner, M. W., Aisen, P. S., Shaw, L. M., Vemuri, P., Wiste, H. J., Weigand, S. D., et al. (2013) · The Lancet Neurology · doi:10.1016/s1474-4422(12)70291-0 · Cited by 4,396 · see citing works
Jessen, F., Amariglio, R. E., van Boxtel, M., Breteler, M., Ceccaldi, M., Chételat, G., Dubois, B., Dufouil, C., Ellis, K. A., van der Flier, W. M., et al. (2014) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2014.01.001 · Cited by 3,014 · see citing works
A Century of Alzheimer's Disease
Goedert, M., & Spillantini, M. G. (2006) · Science · doi:10.1126/science.1132814 · Cited by 2,106 · see citing works
Automated anatomical labelling atlas 3
Rolls, E. T., Huang, C., Lin, C., Feng, J., & Joliot, M. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116189 · Cited by 2,035 · see citing works
The Alzheimer's Disease Neuroimaging Initiative
Mueller, S. G., Weiner, M. W., Thal, L. J., Petersen, R. C., Jack, C., Jagust, W., Trojanowski, J. Q., Toga, A. W., & Beckett, L. (2005) · Neuroimaging Clinics of North America · doi:10.1016/j.nic.2005.09.008 · Cited by 1,525 · see citing works
FcaNet: Frequency Channel Attention Networks
Qin, Z., Zhang, P., Wu, F. Q., & Li, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00082 · Cited by 1,513 · see citing works
Advanced Normalization Tools (ANTs)
Avants, B. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5138159 · Cited by 982 · see citing works
Palmqvist, S., Schöll, M., Strandberg, O., Mattsson, N., Stomrud, E., Zetterberg, H., Blennow, K., Landau, S., Jagust, W., & Hansson, O. (2017) · Nature Communications · doi:10.1038/s41467-017-01150-x · Cited by 911 · see citing works
Focal Frequency Loss for Image Reconstruction and Synthesis
Jiang, L., Dai, B., Wu, W., & Loy, C. C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01366 · Cited by 502 · see citing works
SynthStrip: skull stripping for any brain image
Hoopes, A., Mora, J., Dalca, A. V., Fischl, B. R., & Hoffmann, M. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119474 · Cited by 502 · see citing works
The Alzheimer's Disease Neuroimaging Initiative: A review of papers published since its inception
Weiner, M. W., Veitch, D. P., Aisen, P. S. S., Beckett, L. A., Cairns, N. J., Green, R. C., Harvey, D. J., Jack, C. R., Jagust, W. J., Liu, E., et al. (2011) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2011.09.172 · Cited by 477 · see citing works
In vivo staging of regional amyloid deposition
Grothe, M. J., Barthel, H., Sepulcre, J., Dyrba, M., Sabri, O., Teipel, S. J., Initiative, F. T. A. D. N., Initiative, A. D. N., Weiner, M., Aisen, P., et al. (2017) · Neurology · doi:10.1212/wnl.0000000000004643 · Cited by 440 · see citing works
Khodatars, M., Shoeibi, A., Sadeghi, D., Ghaasemi, N., Jafari, M., Moridian, P., Khadem, A. A., Alizadehsani, R., Zare, A., Kong, Y., et al. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104949 · Cited by 294 · see citing works
Deep learning based synthesis of MRI, CT and PET: Review and analysis
Dayarathna, S., Islam, K. T., Uribe, S. A., Yang, G., Hayat, M., & Chen, Z. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103046 · Cited by 285 · see citing works
Neuroimaging advances regarding subjective cognitive decline in preclinical Alzheimer's disease
Wang, X., Huang, W., Su, L., Xing, Y., Jessen, F., Sun, Y., Shu, N., & Han, Y. (2020) · Molecular Neurodegeneration · doi:10.1186/s13024-020-00395-3 · Cited by 265 · see citing works
Quantification of amyloid PET for future clinical use: a state-of-the-art review
Pemberton, H. G., Collij, L. E., Heeman, F., Bollack, A., Shekari, M., Salvadó, G., Alves, I. L., Garcia, D. V., Battle, M., Buckley, C., et al. (2022) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-022-05784-y · Cited by 226 · see citing works
A Survey on Deep Learning for Neuroimaging-Based Brain Disorder Analysis
Zhang, L., Wang, M., Liu, M., & Zhang, D. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00779 · Cited by 221 · see citing works
Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., & Shen, D. G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00931-1_52 · Cited by 205 · see citing works
Pan, Y., Liu, M., Xia, Y., & Shen, D. G. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3091214 · Cited by 138 · see citing works
Zhang, J., He, X., Qing, L., Gao, F., & Wang, B. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106676 · Cited by 117 · see citing works
Deep learning-based PET image denoising and reconstruction: a review
Hashimoto, F., Onishi, Y., Ote, K., Tashima, H., Reader, A. J., & Yamaya, T. (2024) · Radiological Physics and Technology · doi:10.1007/s12194-024-00780-3 · Cited by 103 · see citing works
Lan, H., Initiative, T. A. D. N., Toga, A. W., & Sepehrband, F. (2021) · Magnetic Resonance in Medicine · doi:10.1002/mrm.28819 · Cited by 99 · see citing works
El–Sappagh, S., Ali, F., Abuhmed, T., Singh, J., & Alonso, J. M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.009 · Cited by 87 · see citing works
Conditional GAN with 3D discriminator for MRI generation of Alzheimer’s disease progression
Jung, E., Luna, M., & Park, S. H. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.109061 · Cited by 82 · see citing works
Optimal Reference Region to Measure Longitudinal Amyloid-β Change with 18F-Florbetaben PET
Bullich, S., Villemagne, V. L., Catafau, A. M., Jovalekic, A., Koglin, N., Rowe, C. C., & Santi, S. D. (2017) · Journal of Nuclear Medicine · doi:10.2967/jnumed.116.187351 · Cited by 66 · see citing works
Dual-Domain Attention for Image Deblurring
Cui, Y., Tao, Y., Ren, W., & Knoll, A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25122 · Cited by 64 · see citing works
Liu, Y., Yue, L., Xiao, S., Yang, W., Shen, D. G., & Liu, M. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102266 · Cited by 59 · see citing works
[11C]PIB amyloid quantification: effect of reference region selection
Heeman, F., Hendriks, J., Alves, I. L., Ossenkoppele, R., Tolboom, N., van Berckel, B. N. M., Lammertsma, A. A., & Yaqub, M. M. (2020) · EJNMMI Research · doi:10.1186/s13550-020-00714-1 · Cited by 37 · see citing works
Parker, A. F., Ohlhauser, L., Scarapicchia, V., Smart, C. M., Szoeke, C., & Gawryluk, J. R. (2022) · Journal of Alzheimer s Disease · doi:10.3233/jad-215249 · Cited by 21 · see citing works
Flagged first, then by citations
1 of the 34 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Dang, M., Liu, B., Chen, Y., & Zhang, Z. (2026) · Psychoradiology · doi:10.1093/psyrad/kkag007 · Cited by 7 · see citing works
Palani, D., Ramu, V., Selvam, P., & Arunachalam, K. P. (2025) · Biomedical Materials & Devices · doi:10.1007/s44174-025-00462-7 · Cited by 1 · see citing works
Huang, F., Xiang, L., Ye, X., Lin, Z., Huang, S., Liang, Y., Yang, H., Wei, L., Liao, X., & Gao, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133397
Comprehensive review of GANs and diffusion models in multimodal 3D medical image conversion
Jiang, N., 李皓塱, Teng, X., & Hu, Z. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.115079
- Authors
- Sheng, J., Zhu, H., Zhou, R., Zhang, Q., Wang, J., & Ying, Z.
- Affiliations
- Hangzhou Dianzi University (CN)
- Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
- Corresponding author
- Sheng, Jinhua
- Zhou, Rougang
- Venue
- Brain Research
- Publisher
- Elsevier BV
- Published
- 2025-09
- Volume
- 1862
- Pages
- 149691
- ISSN
- 0006-8993 (ISSN-L)
- OpenAlex
- W4410432015
- Cited by
- 4
- References
- 34
- Citations by year
- 2026: 3 · 2025: 1
- FWCI
- 1.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Brain Research (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Zhejiang Province Natural Science Foundation (LZ24F010007)
- National Natural Science Foundation of China (62271177)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Keywords
- generative adversarial networks, brain glucose metabolism, Alzheimer's disease, ADNI, standardized uptake value ratio
- MeSH terms
- Alzheimer Disease, Brain, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Male, Positron-Emission Tomography, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
