Missing MRI Pulse Sequence Synthesis Using Multi-Modal Generative Adversarial Network
Open accessAbstract
Magnetic resonance imaging (MRI) is being increasingly utilized to assess, diagnose, and plan treatment for a variety of diseases. The ability to visualize tissue in varied contrasts in the form of MR pulse sequences in a single scan provides valuable insights to physicians, as well as enabling automated systems performing downstream analysis. However, many issues like prohibitive scan time, image corruption, different acquisition protocols, or allergies to certain contrast materials may hinder the process of acquiring multiple sequences for a patient. This poses challenges to both physicians and automated systems since complementary information provided by the missing sequences is lost. In this paper, we propose a variant of generative adversarial network (GAN) capable of leveraging redundant information contained within multiple available sequences in order to generate one or more missing sequences for a patient scan. The proposed network is designed as a multi-input, multi-output network which combines information from all the available pulse sequences and synthesizes the missing ones in a single forward pass. We demonstrate and validate our method on two brain MRI datasets each with four sequences, and show the applicability of the proposed method in simultaneously synthesizing all missing sequences in any possible scenario where either one, two, or three of the four sequences may be missing. We compare our approach with competing unimodal and multi-modal methods, and show that we outperform both quantitatively and qualitatively.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The performance of imputation-based methods for missing modalities depends on the generation model that imputes the missing modalities.
“Performance of these methods depend on the generation model that imputes the missing modalities.”
From Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation (2025) · cites this paper as [25]
In “Imputation Value and Curriculum Learning” (page unavailable), the paper reports different synthesis results for different values used to fill missing MRI sequences, and its benchmark compares the performance of different synthesis models. This supports the general idea that synthesis performance depends on how missing sequences are generated, but the paper does not directly evaluate the performance of downstream imputation-based methods. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863 · see citing works
The multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781 · see citing works
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1× · see citing works
“4) Curriculum learning: In order to train our proposed method we use a curriculum learning (CL) [45]”
Elastix: a toolbox for intensity-based medical image registration
Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2035616 · Cited by 4,311 · see citing works
Instance Normalization: The Missing Ingredient for Fast StylizationPreprint
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv.org · doi:10.48550/arxiv.1607.08022 · Cited by 4,239 · see citing works
Brain tumor segmentation with Deep Neural Networks
Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., & Larochelle, H. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.05.004 · Cited by 3,278 · see citing works
Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images
Pereira, S., Pinto, A., Alves, V., & Silva, C. A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2538465 · Cited by 2,735 · see citing works
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494 · see citing works
Deep multi-scale video prediction beyond mean square errorPreprint
Mathieu, M., Couprie, C., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05440 · Cited by 1,182 · see citing works
VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images
Chen, H., Dou, Q., Yu, L., Qin, J., & Heng, P. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.041 · Cited by 767 · see citing works
Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks
Dar, S. U. H., Yurt, M., Karacan, L., Erdem, A., Erdem, E., & Çukur, T. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2901750 · Cited by 614 · see citing works
SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation
Xue, Y., Xu, T., Zhang, H., Long, L. R., & Huang, X. (2018) · Neuroinformatics · doi:10.1007/s12021-018-9377-x · Cited by 608 · see citing works
HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation
Dolz, J., Gopinath, K. K., Yuan, J. Y., Lombaert, H., Desrosiers, C., & Ayed, I. B. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2878669 · Cited by 592 · see citing works
Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks
Wang, G., Li, W., Sebastien Ourselin, A., & Vercauteren, T. K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75238-9_16 · Cited by 579 · see citing works
GANs for medical image analysis
Kazeminia, S., Baur, C., Kuijper, A., van Ginneken, B., Navab, N., Albarqouni, S., & Mukhopadhyay, A. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101938 · Cited by 548 · see citing works
Maier, O., Menze, B. H., von der Gablentz, J., Häni, L., Heinrich, M. P., Liebrand, M., Winzeck, S., Basit, A., Bentley, P., Chen, L., et al. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.07.009 · Cited by 528 · see citing works
Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge
Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., & Maier-Hein, K. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75238-9_25 · Cited by 528 · see citing works
BET2 : MR-Based Estimation of Brain, Skull and Scalp Surfaces
Jenkinson, M. (2005) · Medical Entomology and Zoology · Cited by 348 · see citing works
Multimodal MR Synthesis via Modality-Invariant Latent Representation
Chartsias, A., Joyce, T., Giuffrida, M. V., & Tsaftaris, S. A. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2764326 · Cited by 343 · see citing works
HeMIS: Hetero-Modal Image Segmentation
Havaei, M., Guizard, N., Chapados, N., & Bengio, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_54 · Cited by 332 · cited 2× · see citing works
“Most notably, Havaei et al. [3] and Varsavsky et al. [4] provide methods for tumor segmentation using brain MRI that are robust to missing sequences [3], or to missing sequence labels [4].”
MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music GenerationPreprint
Yang, L.-C., Chou, S.-Y., & Yang, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10847 · Cited by 308 · see citing works
Ea-GANs: Edge-Aware Generative Adversarial Networks for Cross-Modality MR Image Synthesis
Yu, B., Zhou, L., Wang, L., Shi, Y., Fripp, J., & Bourgeat, P. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2895894 · Cited by 301 · see citing works
Adversarial Stain Transfer for Histopathology Image Analysis
BenTaieb, A., & Hamarneh, G. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2781228 · Cited by 231 · see citing works
Self-citationRandom forest regression for magnetic resonance image synthesis
Jog, A., Carass, A., Roy, S., Pham, D. L., & Prince, J. L. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.08.009 · Cited by 212 · see citing works
Learning to Generate Images of Outdoor Scenes from Attributes and Semantic LayoutsPreprint
Karacan, L., Akata, Z., Erdem, A., & Erdem, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00215 · Cited by 143 · see citing works
Is Synthesizing MRI Contrast Useful for Inter-modality Analysis?
Iglesias, J. E., Konukoğlu, E., Zikic, D., Glocker, B., Leemput, K. V., & Fischl, B. R. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40811-3_79 · Cited by 142 · see citing works
Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network
Nguyen, H. V., Zhou, K., & Vemulapalli, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24553-9_83 · Cited by 130 · see citing works
Adversarial Training and Dilated Convolutions for Brain MRI Segmentation
Moeskops, P., Veta, M., Lafarge, M. W., Eppenhof, K. A. J., & Pluim, J. P. W. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67558-9_7 · Cited by 126 · see citing works
Modality Propagation: Coherent Synthesis of Subject-Specific Scans with Data-Driven Regularization
Ye, D. H., Zikic, D., Glocker, B., Criminisi, A., & Ender Konukoglu, A. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40811-3_76 · Cited by 108 · see citing works
Why Does Synthesized Data Improve Multi-sequence Classification?
van Tulder, G., & de Bruijne, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24553-9_65 · Cited by 89 · see citing works
Magnetic resonance image synthesis through patch regression
Jog, A., Roy, S., Carass, A., & Prince, J. L. (2013) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2013.6556484 · Cited by 84 · see citing works
Multiple Sclerosis Lesion Synthesis in MRI Using an Encoder-Decoder U-NET
Salem, M., Valverde, S., Cabezas, M., Pareto, D., Oliver, A., Salví, J., Rovira, À., & Lladó, X. (2019) · IEEE Access · doi:10.1109/access.2019.2900198 · Cited by 79 · see citing works
Generative Adversarial Training for MRA Image Synthesis Using Multi-contrast MRI
Olut, Ş., Şahin, Y. H., Demir, U., & Ünal, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00320-3_18 · Cited by 75 · see citing works
Whole Image Synthesis Using a Deep Encoder-Decoder Network
Sevetlidis, V., Giuffrida, M. V., & Tsaftaris, S. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46630-9_13 · Cited by 68 · see citing works
Generative adversarial networks to segment skin lesions
Izadi, S., Mirikharaji, Z., Kawahara, J., & Hamarneh, G. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363712 · Cited by 58 · see citing works
Self-citationA review of MRI pulse sequences and techniques in neuroimaging
Jackson, E. F., Ginsberg, L. E., Schomer, D. F., & Leeds, N. E. (1997) · Surgical Neurology · doi:10.1016/s0090-3019(96)00375-8 · Cited by 55 · see citing works
Pseudo-healthy Image Synthesis for White Matter Lesion Segmentation
Bowles, C. T., Qin, C., Ledig, C., Guerrero, R., Gunn, R. N., Hammers, A., Sakka, E., Dickie, D. A., Hernández, M. D. C. V., Royle, N. A., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46630-9_9 · Cited by 49 · see citing works
Generative Adversarial Networks for MR-CT Deformable Image RegistrationPreprint
Tanner, C., Özdemir, F., Profanter, R., Vishnevskiy, V., Ender Konukoglu, A., & Göksel, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.07349 · Cited by 45 · see citing works
Huang, Y., Beltrachini, L., Shao, L., & Frangi, A. F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46630-9_12 · Cited by 24 · see citing works
Mehta, R., & Arbel, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00536-8_13 · Cited by 24 · see citing works
PIMMS: Permutation Invariant Multi-modal Segmentation
Varsavsky, T., Eaton-Rosen, Z., Sudre, C. H., Nachev, P., & Cardoso, M. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00889-5_23 · Cited by 22 · see citing works
Patch Based Synthesis of Whole Head MR Images: Application To EPI Distortion Correction
Roy, S., Chou, Y., Jog, A., Butman, J. A., & Pham, D. L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46630-9_15 · Cited by 20 · see citing works
Random forest FLAIR reconstruction from T1, T2, and PD-weighted MRI
Jog, A., Carass, A., Pham, D. L., & Prince, J. L. (2014) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2014.6868061 · Cited by 18 · see citing works
Tree-Encoded Conditional Random Fields for Image Synthesis
Jog, A., Carass, A., Pham, D. L., & Prince, J. L. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19992-4_58 · Cited by 8 · see citing works
Flagged first, then by citations
2 of the 51 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- S. Bakas et al., "2017 International MICCAI BraTS Challenge," 2017.
- I. Goodfellow et al., "Generative adversarial nets," in Advances in Neural Information Processing Systems, 2014, pp. 2672-2680.
- J. Gauthier, "Conditional generative adversarial nets for convolutional face generation," Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter semester, vol. 2014, no. 5, p. 2, 2014.
Cited by
Sorted by citations
Diffusion models in medical imaging: A comprehensive survey
Kazerouni, A., Aghdam, E. K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102846 · Cited by 850 · see citing works
ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis
Dalmaz, O., Yurt, M., & Çukur, T. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3167808 · Cited by 613 · see citing works
Unsupervised Medical Image Translation With Adversarial Diffusion Models
Özbey, M., Dalmaz, O., Dar, S. U. H., Bedel, H. A., Öztürk, Ş., Güngör, A., & Çukur, T. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3290149 · Cited by 542 · see citing works
Data augmentation for medical imaging: A systematic literature review
Garcea, F., Serra, A., Lamberti, F., & Morra, L. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106391 · Cited by 423 · see citing works
Deep learning based synthesis of MRI, CT and PET: Review and analysis
Dayarathna, S., Islam, K. T., Uribe, S. A., Yang, G., Hayat, M., & Chen, Z. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103046 · Cited by 285 · see citing works
Deep learning methods to generate synthetic CT from MRI in radiotherapy: A literature review
Boulanger, M., Nunes, J., Chourak, H., Largent, A., Tahri, S., Acosta, O., de Crevoisier, R., Lafond, C., & Barateau, A. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.07.027 · Cited by 210 · see citing works
mustGAN: multi-stream Generative Adversarial Networks for MR Image Synthesis
Yurt, M., Dar, S. U. H., Erdem, A., Erdem, E., Oguz, K. K., & Çukur, T. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101944 · Cited by 177 · see citing works
Dar, S. U. H., Yurt, M., Shahdloo, M., Ildiz, M. E., Tınaz, B., & Çukur, T. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3001737 · Cited by 175 · see citing works
Generative-AI-Driven Human Digital Twin in IoT Healthcare: A Comprehensive Survey
Chen, J., Shi, Y., Yi, C., Du, H., Kang, J., & Niyato, D. T. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3421918 · Cited by 155 · see citing works
Conte, G. M., Weston, A. D., Vogelsang, D. C., Philbrick, K. A., Cai, J. C., Barbera, M., Sanvito, F., Lachance, D. H., Jenkins, R. B., Tobin, W. O., et al. (2021) · Radiology · doi:10.1148/radiol.2021203786 · Cited by 131 · see citing works
One Model to Synthesize Them All: Multi-Contrast Multi-Scale Transformer for Missing Data Imputation
Liu, J., Pasumarthi, S., Duffy, B. A., Gong, E., Datta, K., & Zaharchuk, G. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3261707 · Cited by 113 · see citing works
Generative AI for brain image computing and brain network computing: a review
Gong, C., Jing, C., Chen, X., Pun, C. M., Huang, G., Saha, A., Nieuwoudt, M., Li, H.-X., Hu, Y., & Wang, S. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1203104 · Cited by 111 · see citing works
Learning With Privileged Multimodal Knowledge for Unimodal Segmentation
Chen, C., Dou, Q., Jin, Y., Liu, Q., & Heng, P. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3119385 · Cited by 108 · see citing works
CoLa-Diff: Conditional Latent Diffusion Model for Multi-modal MRI Synthesis
Jiang, L., Mao, Y., Wang, X., Chen, X., & Li, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43999-5_38 · Cited by 108 · see citing works
Autoencoder based self-supervised test-time adaptation for medical image analysis
He, Y., Carass, A., Zuo, L., Dewey, B. E., & Prince, J. L. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102136 · Cited by 103 · see citing works
ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities
Wang, Y., Zhang, Y., Liu, Y., Lin, Z., Tian, J., Zhong, C., Shi, Z., Jianping Fan, B., & He, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87234-2_39 · Cited by 102 · see citing works
Multi-Domain Image Completion for Random Missing Input Data
Shen, L., Zhu, W., Wang, X., Xing, L., Pauly, J. M., Türkbey, B., Harmon, S. A., Sanford, T., Mehralivand, S., Choyke, P. L., et al. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3046444 · Cited by 99 · see citing works
The role of generative adversarial networks in brain MRI: a scoping review
Ali, H., Biswas, M. R., Mohsen, F., Shah, U., Alamgir, A., Mousa, O., & Shah, Z. (2022) · Insights into Imaging · doi:10.1186/s13244-022-01237-0 · Cited by 96 · see citing works
Multimodal MRI synthesis using unified generative adversarial networks
Dai, X., Lei, Y., Fu, Y., Curran, W. J., Liu, T., Mao, H. Z., & Yang, X. (2020) · Medical Physics · doi:10.1002/mp.14539 · Cited by 95 · see citing works
DiamondGAN: Unified Multi-modal Generative Adversarial Networks for MRI Sequences Synthesis
Li, H., Paetzold, J. C., Sekuboyina, A., Kofler, F., Zhang, J., Kirschke, J. S., Wiestler, B., & Menze, B. H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32251-9_87 · Cited by 91 · see citing works
Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing
Meng, X., Sun, K., Xu, J., He, X., & Shen, D. G. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3368664 · Cited by 84 · see citing works
Brain stroke lesion segmentation using consistent perception generative adversarial network
Wang, S., Chen, Z., You, S., Wang, B.-C., Shen, Y., & Lei, B. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06816-8 · Cited by 77 · see citing works
Similarity and quality metrics for MR image-to-image translation
Dohmen, M., Klemens, M., Baltruschat, I. M., Truong, T., & Lenga, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-87358-0 · Cited by 74 · see citing works
Unified Multi-Modal Image Synthesis for Missing Modality Imputation
Zhang, Y., Peng, C., Wang, Q., Song, D., Li, K., & Zhou, S. K. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3424785 · Cited by 71 · see citing works
Self-consistent recursive diffusion bridge for medical image translation
Arslan, F., Kabaş, B., Dalmaz, O., Özbey, M., & Çukur, T. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103747 · Cited by 74 · see citing works
FedMed-GAN: Federated domain translation on unsupervised cross-modality brain image synthesis
Wang, J., Xie, G., Huang, Y., Lyu, J., Zheng, F., Zheng, Y., & Jin, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126282 · Cited by 67 · see citing works
A literature survey of MR-based brain tumor segmentation with missing modalities
Zhou, T., Ruan, S., & Hu, H. (2022) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2022.102167 · Cited by 66 · see citing works
MFTrans: Modality-Masked Fusion Transformer for Incomplete Multi-Modality Brain Tumor Segmentation
Shi, J., Yu, L. C., Cheng, Q., Yang, X. Y., Cheng, K.-T. T., & Yan, Z. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3326151 · Cited by 65 · see citing works
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation
Reza, M. K., Prater-Bennette, A., & Asif, M. S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3476487 · Cited by 61 · see citing works
“Knowledge distillation and generation methods have also become popular for robust MML. Studies by [47] and [64] used GAN based generative models while [11] used VAE based generative models for imputing missing modalities from available input modalities for unlerlying multimodal tasks.”
MethodYao, W., Yin, K., Cheung, W. K., Liu, J., & Qin, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i15.29578 · Cited by 61 · see citing works
Liu, Y., Yue, L., Xiao, S., Yang, W., Shen, D. G., & Liu, M. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102266 · Cited by 59 · see citing works
Wang, G., Gong, E., Banerjee, S., Martin, D., Tong, E., Choi, J., Chen, H., Wintermark, M., Pauly, J. M., & Zaharchuk, G. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2987026 · Cited by 57 · see citing works
Cross-Modal Distillation to Improve MRI-Based Brain Tumor Segmentation With Missing MRI Sequences
Rahimpour, M., Bertels, J., Radwan, A., Vandermeulen, H., Sunaert, S. G., Vandermeulen, D., Maes, F., Goffin, K. E., & Koole, M. J. B. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3137561 · Cited by 56 · see citing works
Artificial intelligence in glioma imaging: Challenges and advances
Jin, W., Fatehi, M., Abhishek, K., Mallya, M., Toyota, B., & Hamarneh, G. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab8131 · Cited by 55 · see citing works
Self-citationOne model to unite them all: Personalized federated learning of multi-contrast MRI synthesis
Dalmaz, O., Mirza, M. U., Elmas, G., Özbey, M., Dar, S. U. H., Ceyani, E., Oğuz, K. K., Avestimehr, S., & Çukur, T. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103121 · Cited by 54 · see citing works
Scalable Swin Transformer network for brain tumor segmentation from incomplete MRI modalities
Zhang, D., Wang, C., Chen, T., Chen, W., & Shen, Y. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102788 · Cited by 50 · see citing works
Generative adversarial networks for biomedical time series forecasting and imputation
Festag, S., Denzler, J., & Spreckelsen, C. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104058 · Cited by 46 · see citing works
Lyu, Q., Namjoshi, S. V., McTyre, E., Topaloĝlu, Ü., Barcus, R. A., Chan, M. D., Cramer, C. K., Debinski, W., Gürcan, M. N., Lesser, G. J., et al. (2022) · Patterns · doi:10.1016/j.patter.2022.100613 · Cited by 45 · see citing works
Yang, Q., Lin, Y., Wang, J., Bao, J., Wang, X., Ma, L., Zhou, Z., Yang, Q., Cai, S., He, H., et al. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3179981 · Cited by 44 · see citing works
Hamghalam, M., Wang, T., & Lei, B. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.08.014 · Cited by 43 · see citing works
Deep learning‐based convolutional neural network for intramodality brain MRI synthesis
Osman, A. F. I., & Tamam, N. M. (2022) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.13530 · Cited by 43 · see citing works
Yang, H., Sun, J. Z., & Xu, Z. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3301934 · Cited by 40 · see citing works
Huang, P., Li, D., Jiao, Z., Wei, D., Cao, B., Mo, Z., Wang, Q., Zhang, H., & Shen, D. G. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102472 · Cited by 40 · see citing works
MC 2 ‐Net: motion correction network for multi‐contrast brain MRI
Lee, J., Kim, B., & Park, H. (2021) · Magnetic Resonance in Medicine · doi:10.1002/mrm.28719 · Cited by 40 · see citing works
Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation
Maheshwari, H., Liu, Y.-C., & Kira, Z. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00106 · Cited by 37 · see citing works
Yurt, M., Özbey, M., Dar, S., Tınaz, B., Oğuz, K., & Cukur, T. (2020) · Medical Image Anal. · doi:10.1016/j.media.2022.102429 · Cited by 40 · see citing works
Semi-Supervised Learning of MRI Synthesis Without Fully-Sampled Ground Truths
Yurt, M., Dalmaz, O., Dar, S. U. H., Özbey, M., Tınaz, B., Oğuz, K. K., & Çukur, T. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3199155 · Cited by 35 · see citing works
Deep learning‐based multi‐modal computing with feature disentanglement for MRI image synthesis
Fei, Y., Zhan, B., Hong, M., Wu, X., Zhou, J., & Wang, Y. (2021) · Medical Physics · doi:10.1002/mp.14929 · Cited by 34 · see citing works
Style transfer in conditional GANs for cross-modality synthesis of brain magnetic resonance images
Qin, Z., Liu, Z., Zhu, P., & Ling, W. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105928 · Cited by 34 · see citing works
Synthesis-based imaging-differentiation representation learning for multi-sequence 3D/4D MRI
Han, L., Tan, T., Zhang, T., Huang, Y., Wang, X., Gao, Y., Teuwen, J., & Mann, R. M. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103044 · Cited by 33 · see citing works
Enhancing Modality-Agnostic Representations via Meta-learning for Brain Tumor Segmentation
Konwer, A., Hu, X., Bae, J., Xu, X., Chen, C., & Prasanna, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01958 · Cited by 33 · see citing works
Ahmad, S., Ullah, Z., & Gwak, J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111854 · Cited by 33 · see citing works
A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI
Tavse, S., Vijayakumar, V., Bachute, M., Gite, S., & Kotecha, K. (2022) · Future Internet · doi:10.3390/fi14120351 · Cited by 28 · see citing works
Deep learning for brain disorders: from data processing to disease treatment
Burgos, N., Bottani, S., Faouzi, J., Thibeau–Sutre, E., & Colliot, O. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa310 · Cited by 31 · see citing works
A novel one-to-multiple unsupervised domain adaptation framework for abdominal organ segmentation
Xu, X., Chen, Y., Wu, J., Lu, J., Ye, Y., Huang, Y., Dou, X., Li, K., Wang, G., Zhang, S., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102873 · Cited by 28 · see citing works
Hamghalam, M., Frangi, A. F., Lei, B., & Simpson, A. L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87234-2_42 · Cited by 28 · see citing works
Brain Tumor Segmentation for Multi-Modal MRI with Missing Information
Feng, X., Ghimire, K., Kim, D. D., Chandra, R. S., Zhang, H., Peng, J., Han, B., Huang, G., Chen, Q., Patel, S. H., et al. (2023) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-023-00860-7 · Cited by 26 · see citing works
Brain MRI Tumor Segmentation with Adversarial Networks
Giacomello, E., Loiacono, D., & Mainardi, L. T. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207220 · Cited by 25 · see citing works
Bonada, M., Rossi, L. F., Carone, G., Panico, F., Cofano, F., Fiaschi, P., Garbossa, D., Meco, F. D., & Bianconi, A. (2024) · Biomedicines · doi:10.3390/biomedicines12081878 · Cited by 24 · see citing works
Uhm, K.-H., Jung, S., Choi, M. H., Hong, S., & Ko, S.-J. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3219123 · Cited by 24 · see citing works
MIL normalization —— prerequisites for accurate MRI radiomics analysis
Hu, Z., Zhuang, Q., Xiao, Y., Wu, G., Shi, Z., Chen, L., Wang, Y., & Yu, J. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104403 · Cited by 23 · see citing works
Al-Fakih, A., Shazly, A., Mohammed, A., Elbushnaq, M., Ryu, K., Gu, Y. H., Al‐masni, M. A., & Makary, M. M. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.05.008 · Cited by 22 · see citing works
Yang, J., Li, X., Liu, F., Nie, D., Lió, P., Qi, H., & Shen, D. G. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3227262 · Cited by 22 · see citing works
Osman, A. F. I., & Tamam, N. M. (2023) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.14120 · Cited by 22 · see citing works
Xanthis, C. G., Filos, D., Haris, K., & Aletras, A. H. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105817 · Cited by 22 · see citing works
Cross-modality Neuroimage Synthesis: A Survey
Xie, G., Huang, Y., Wang, J., Lyu, J., Zheng, F., Zheng, Y., & Jin, Y. (2023) · ACM Computing Surveys · doi:10.1145/3625227 · Cited by 21 · see citing works
Sabnam, S., & Rajagopal, S. (2024) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2024.2330524 · Cited by 21 · see citing works
Shi, J., Shang, C., Sun, Z., Yu, L. C., Yang, X. Y., & Yan, Z. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681543 · Cited by 21 · see citing works
Bahador, N., Jokelainen, J., Mustola, S. T., & Kortelainen, J. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac23e2 · Cited by 20 · see citing works
A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis
Bian, W., Zhang, Q., Ye, X., & Chen, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16446-0_34 · Cited by 20 · see citing works
Haase, R., Pinetz, T., Kobler, E., Paech, D., Effland, A., Radbruch, A., & Deike‐Hofmann, K. (2023) · Investigative Radiology · doi:10.1097/rli.0000000000000963 · Cited by 20 · see citing works
Hameed, M., Zameer, A., Khan, S. H., & Raja, M. A. Z. (2024) · The European Physical Journal Plus · doi:10.1140/epjp/s13360-024-05220-0 · Cited by 20 · see citing works
Unified Brain MR-Ultrasound Synthesis Using Multi-modal Hierarchical Representations
Dorent, R., Haouchine, N., Kögl, F., Joutard, S., Juvekar, P., Torio, E., Golby, A. J., Sebastien Ourselin, A., Frisken, S. F., Vercauteren, T. K., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43999-5_43 · Cited by 19 · see citing works
Improved generative adversarial imputation networks for missing data
Qin, X., Shi, H., Dong, X., Zhang, S., & Yuan, L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05814-2 · Cited by 19 · see citing works
High Tissue Contrast MRI Synthesis Using Multi-Stage Attention-GAN for Segmentation
Hamghalam, M., Lei, B., & Wang, T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5825 · Cited by 18 · see citing works
An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesis
Han, L., Zhang, T., Huang, Y., Dou, H., Wang, X., Gao, Y., Lu, C., Tan, T., & Mann, R. M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43999-5_5 · Cited by 17 · see citing works
Raut, P., Baldini, G., Schöneck, M., & Caldeira, L. (2024) · Frontiers in Radiology · doi:10.3389/fradi.2023.1336902 · Cited by 17 · see citing works
CT synthesis from MR images using frequency attention conditional generative adversarial network
Wei, K., Kong, W., Liu, L., Wang, J., Li, B., Zhao, B., Li, Z., Zhu, J., & Yu, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107983 · Cited by 17 · see citing works
Feng, Y., Deng, S., Lyu, J., Cai, J., Wei, M.-Q., & Qin, J. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3445969 · Cited by 20 · see citing works
Park, Y. W., Shin, S. J., Eom, J., Lee, H., You, S. C., Ahn, S. S., Lim, S. M., Park, R. W., & Lee, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-10956-9 · Cited by 17 · see citing works
Toward general text-guided multimodal brain MRI synthesis for diagnosis and medical image analysis
Wang, Y., Xiong, H., Sun, K., Bai, S., Dai, L., Ding, Z., Liu, J., Wang, Q., Liu, Q., & Shen, D. G. (2025) · Cell Reports Medicine · doi:10.1016/j.xcrm.2025.102182 · Cited by 16 · see citing works
Liu, B., Zhang, Y., Wang, S., Li, S., & Hong, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108118 · Cited by 16 · see citing works
Learning a robust RGB-Thermal detector for extreme modality imbalance
Tian, C., Yang, C., Zhu, G., Wang, Q., & He, Z. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.05.005 · Cited by 15 · see citing works
A unified hybrid transformer for joint MRI sequences super-resolution and missing data imputation
Wang, Y., Hu, H., Yu, S., Yang, Y., Guo, Y., Song, X., Chen, F., & Liu, Q. (2023) · Physics in Medicine and Biology · doi:10.1088/1361-6560/acdc80 · Cited by 15 · see citing works
Addressing Missing Modality Challenges in MRI Images: A Comprehensive Review
Azad, R., Dehghanmanshadi, M., Khosravi, N., Cohen‐Adad, J., & Merhof, D. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450399 · Cited by 14 · see citing works
Xia, Y., Ravikumar, N., Lassila, T., & Frangi, A. F. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102814 · Cited by 14 · see citing works
A general variation-driven network for medical image synthesis
Chen, Y., Yang, X., Yue, X., Lin, X., Zhang, Q., & Fujita, H. (2024) · Applied Intelligence · doi:10.1007/s10489-023-05017-1 · Cited by 13 · see citing works
Moya‐Sáez, E., de Luis-Garcı́a, R., & Alberola‐López, C. (2023) · Frontiers in Neuroimaging · doi:10.3389/fnimg.2023.1055463 · Cited by 13 · see citing works
Multimodal Foundation Models for Medical Imaging - A Systematic Review and Implementation GuidelinesPreprint
Huang, S.-C., Jensen, M., Yeung, S. Y., Lungren, M. P., Poon, H., & Chaudhari, A. S. (2024) · medRxiv · doi:10.1101/2024.10.23.24316003 · Cited by 15 · see citing works
Zhu, S., Chen, Y., Jiang, S., Chen, W., Liu, C., Wang, Y., Chen, X., Ke, Y., Qin, F., Wang, C., et al. (2025) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi60581.2025.10981275 · Cited by 12 · see citing works
Gurney‐Champion, O. J., Kieselmann, J. P., Wong, K. H., Ng-Cheng-Hin, B., Harrington, K. J., & Oelfke, U. (2020) · Physics and Imaging in Radiation Oncology · doi:10.1016/j.phro.2020.06.002 · Cited by 12 · see citing works
Deep learning for biomedical image reconstruction: a survey
Yedder, H. B., Cardoen, B., & Hamarneh, G. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09861-2 · Cited by 11 · see citing works
Self-citationProbabilistic Brain Extraction in MR Images via Conditional Generative Adversarial Networks
Moazami, S., Ray, D., Pelletier, D., & Oberai, A. A. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3327942 · Cited by 11 · see citing works
Singh, G., Singh, A., Bae, J., Manjila, S., Spektor, V., Prasanna, P., & Lignelli, A. (2024) · Cancer Imaging · doi:10.1186/s40644-024-00769-6 · Cited by 12 · see citing works
Towards Better Cephalometric Landmark Detection With Diffusion Data Generation
Guo, D., Han, W., Lyu, P., Zhou, Y., & Shen, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3557430 · Cited by 12 · see citing works
Wang, T., Wang, H., Deng, J., Zhang, D., Feng, J. X., & Chen, B. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.104536 · Cited by 11 · see citing works
QGFormer: Queries-guided transformer for flexible medical image synthesis with domain missing
Hao, H., Xue, J., Huang, P., Ren, L., & Li, D.-W. (2024) · Expert systems with applications · doi:10.1016/j.eswa.2024.123318 · Cited by 10 · see citing works
Qiao, L., Wang, R., Shu, Y., Xiao, B., Xu, X., Li, B., Yang, L., Li, W., Gao, X., & Lei, B. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112412 · Cited by 10 · see citing works
Wagner, F., Xu, W.-T., Saha, P., Liang, Z.-Y., Whitehouse, D., Menon, D., Newcombe, V. F. J., Voets, N., Noble, J., & Kamnitsas, K. (2024) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00045 · Cited by 16 · see citing works
Multi-Modal Brain Tumor Data Completion Based on Reconstruction Consistency Loss
Jiang, Y., Zhang, S., & Chi, J. (2023) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-022-00697-6 · Cited by 10 · see citing works
- Authors
- Sharma, A., & Hamarneh, G.
- Affiliations
- Simon Fraser University (CA)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2020-04
- Volume
- 39
- Issue
- 4
- Pages
- 1170-1183
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2941736865
- Cited by
- 254
- References
- 51
- Citations by year
- 2026: 37 · 2025: 66 · 2024: 50 · 2023: 32 · 2022: 31 · 2021: 20 · 2020: 17 · 2019: 1
- FWCI
- 11.39
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Summit (Simon Fraser University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Nvidia
- Compute Canada
- Western Canada Research Grid
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- brain MRI, scan time reduction, multi-contrast MRI
- MeSH terms
- Deep Learning, Brain, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
