The DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index.
Missing MRI Pulse Sequence Synthesis Using Multi-Modal Generative Adversarial Network
Open accessAbstract
Magnetic resonance imaging (MRI) is being increasingly utilized to assess, diagnose, and plan treatment for a variety of diseases. The ability to visualize tissue in varied contrasts in the form of MR pulse sequences in a single scan provides valuable insights to physicians, as well as enabling automated systems performing downstream analysis. However, many issues like prohibitive scan time, image corruption, different acquisition protocols, or allergies to certain contrast materials may hinder the process of acquiring multiple sequences for a patient. This poses challenges to both physicians and automated systems since complementary information provided by the missing sequences is lost. In this paper, we propose a variant of generative adversarial network (GAN) capable of leveraging redundant information contained within multiple available sequences in order to generate one or more missing sequences for a patient scan. The proposed network is designed as a multi-input, multi-output network which combines information from all the available pulse sequences and synthesizes the missing ones in a single forward pass. We demonstrate and validate our method on two brain MRI datasets each with four sequences, and show the applicability of the proposed method in simultaneously synthesizing all missing sequences in any possible scenario where either one, two, or three of the four sequences may be missing. We compare our approach with competing unimodal and multi-modal methods, and show that we outperform both quantitatively and qualitatively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Performance of these methods depend on the generation model that imputes the missing modalities.”
From Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation (2025) · cites this paper as [25]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
The multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“4) Curriculum learning: In order to train our proposed method we use a curriculum learning (CL) [45]”
Instance Normalization: The Missing Ingredient for Fast StylizationPreprint
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08022 · Cited by 4,325
Elastix: a toolbox for intensity-based medical image registration
Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2035616 · Cited by 4,311
Research works printed but not linked: 3
- gauthier, 2014, Conditional generative adversarial nets for convolutional face generation, Class Project Stanford CS231N Convolutional Neural Netw Vis Recognit Winter Semester, 2014, 2
- goodfellow, 2014, Generative adversarial nets, Proc Adv Neural Inf Process Syst, 2672
- 2017, 2017 International MICCAI BraTS Challenge
Cited by
Sorted by citations
Diffusion models in medical imaging: A comprehensive survey
Kazerouni, A., Aghdam, E. K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102846 · Cited by 1,006
ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis
Dalmaz, O., Yurt, M., & Çukur, T. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3167808 · Cited by 642
Unsupervised Medical Image Translation With Adversarial Diffusion Models
Özbey, M., Dalmaz, O., Dar, S. U. H., Bedel, H. A., Öztürk, Ş., Güngör, A., & Çukur, T. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3290149 · Cited by 590
Data augmentation for medical imaging: A systematic literature review
Garcea, F., Serra, A., Lamberti, F., & Morra, L. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106391 · Cited by 451
Deep learning based synthesis of MRI, CT and PET: Review and analysis
Dayarathna, S., Islam, K. T., Uribe, S. A., Yang, G., Hayat, M., & Chen, Z. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103046 · Cited by 316
Metadata
- Authors
- Sharma, A., & Hamarneh, G.
- Affiliations
- Simon Fraser University (CA)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2020-04
- Volume
- 39
- Issue
- 4
- Pages
- 1170-1183
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2941736865
- Cited by
- 254
- References
- 51
- Citations by year
- 2026: 37 · 2025: 66 · 2024: 50 · 2023: 32 · 2022: 31 · 2021: 20 · 2020: 17
- FWCI
- 11.39
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Summit (Simon Fraser University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Nvidia
- Compute Canada
- Western Canada Research Grid
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- brain MRI, scan time reduction, multi-contrast MRI
- MeSH terms
- Deep Learning, Brain, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
