The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks
Open accessAbstract
Acquiring images of the same anatomy with multiple different contrasts increases the diversity of diagnostic information available in an MR exam. Yet, the scan time limitations may prohibit the acquisition of certain contrasts, and some contrasts may be corrupted by noise and artifacts. In such cases, the ability to synthesize unacquired or corrupted contrasts can improve diagnostic utility. For multi-contrast synthesis, the current methods learn a nonlinear intensity transformation between the source and target images, either via nonlinear regression or deterministic neural networks. These methods can, in turn, suffer from the loss of structural details in synthesized images. Here, in this paper, we propose a new approach for multi-contrast MRI synthesis based on conditional generative adversarial networks. The proposed approach preserves intermediate-to-high frequency details via an adversarial loss, and it offers enhanced synthesis performance via pixel-wise and perceptual losses for registered multi-contrast images and a cycle-consistency loss for unregistered images. Information from neighboring cross-sections are utilized to further improve synthesis quality. Demonstrations on T1- and T2- weighted images from healthy subjects and patients clearly indicate the superior performance of the proposed approach compared to the previous state-of-the-art methods. Our synthesis approach can help improve the quality and versatility of the multi-contrast MRI exams without the need for prolonged or repeated examinations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002) · NeuroImage · doi:10.1006/nimg.2002.1132 · Cited by 11,888
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
A global optimisation method for robust affine registration of brain images
Jenkinson, M., & Smith, S. (2001) · Medical Image Analysis · doi:10.1016/s1361-8415(01)00036-6 · Cited by 6,872
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437
Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., & Greenspan, H. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.013 · Cited by 1,938
Yang, G., Yu, S., Dong, H., Slabaugh, G., Dragotti, P. L., Ye, X., Liu, F., Arridge, S., Keegan, J., Guo, Y., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2785879 · Cited by 1,081
Quicksilver: Fast predictive image registration – A deep learning approach
Yang, X., Kwitt, R., Styner, M., & Niethammer, M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.008 · Cited by 630 · cited 1×
“first be estimated via multi-modal registration [75].”
Medical Image Synthesis with Deep Convolutional Adversarial Networks
Nie, D., Trullo, R., Lian, J., Wang, L., Petitjean, C., Ruan, S., Wang, Q., & Shen, D. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2814538 · Cited by 625
Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss
Quan, T. M., Nguyen-Duc, T., & Jeong, W.-K. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2820120 · Cited by 609
Sketch2Photo: internet image montage
Chen, T., Cheng, M.-M., Tan, P., Shamir, A., & Hu, S.-M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618470 · Cited by 574
Deep Generative Adversarial Neural Networks for Compressive Sensing MRI
Mardani, M., Gong, E., Cheng, J. Y., Vasanawala, S. S., Zaharchuk, G., Xing, L., & Pauly, J. M. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2858752 · Cited by 552
Mathematical textbook of deformable neuroanatomies.
Miller, M. I., Christensen, G. E., Amit, Y., & Grenander, U. (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.24.11944 · Cited by 462
3D conditional generative adversarial networks for high-quality PET image estimation at low dose
Wang, Y., Yu, B., Wang, L., Zu, C., Lalush, D. S., Lin, W., Wu, X., Zhou, J., Shen, D., & Zhou, L. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.03.045 · Cited by 449
End-to-End Adversarial Retinal Image Synthesis
Costa, P., Galdran, A., Meyer, M. I., Niemeijer, M., Abramoff, M., Mendonca, A. M., & Campilho, A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2759102 · Cited by 432
Attenuation Correction Synthesis for Hybrid PET-MR Scanners: Application to Brain Studies
Burgos, N., Cardoso, M. J., Thielemans, K., Modat, M., Pedemonte, S., Dickson, J., Barnes, A., Ahmed, R., Mahoney, C. J., Schott, J. M., et al. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2340135 · Cited by 364
Artifacts in Magnetic Resonance Imaging
Bekiesińska‐Figatowska, M. (2015) · Polish Journal of Radiology · doi:10.12659/pjr.892628 · Cited by 349
GAN-based synthetic brain MR image generation
Han, C., Hayashi, H., Rundo, L., Araki, R., Shimoda, W., Muramatsu, S., Furukawa, Y., Mauri, G., & Nakayama, H. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363678 · Cited by 344
Multimodal MR Synthesis via Modality-Invariant Latent Representation
Chartsias, A., Joyce, T., Giuffrida, M. V., & Tsaftaris, S. A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2764326 · Cited by 341
Vessel Tortuosity and Brain Tumor Malignancy
Bullitt, E., Zeng, D., Gerig, G., Aylward, S., Joshi, S., Smith, J. K., Lin, W., & Ewend, M. G. (2005) · Academic Radiology · doi:10.1016/j.acra.2005.05.027 · Cited by 285
Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training
Mahmood, F., Chen, R., & Durr, N. J. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2842767 · Cited by 257 · cited 1×
“Thus, it is quite common in deep learning research to perform one-fold cross-validation [30], [35] or even directly adopt hyperparameter selection from published work [24], [28], [29], [38], [48], [62].”
BackgroundHiasa, Y., Otake, Y., Takao, M., Matsuoka, T., Takashima, K., Carass, A., Prince, J. L., Sugano, N., & Sato, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00536-8_4 · Cited by 221
Jiang, D., Dou, W., Vosters, L., Xu, X., Sun, Y., & Tan, T. (2018) · Japanese Journal of Radiology · doi:10.1007/s11604-018-0758-8 · Cited by 219
Huang, Y., Shao, L., & Frangi, A. F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.613 · Cited by 216
Interpreting scan data acquired from multiple scanners: A study with Alzheimer's disease
Stonnington, C. M., Tan, G., Klöppel, S., Chu, C., Draganski, B., Jack, C. R., Chen, K., Ashburner, J., & Frackowiak, R. S. J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.09.066 · Cited by 213
Random forest regression for magnetic resonance image synthesis
Jog, A., Carass, A., Roy, S., Pham, D. L., & Prince, J. L. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.08.009 · Cited by 212 · cited 7×
“Nonlinear mappings are learned on training data via techniques such as nonlinear regression [8], [9], [23] or location-sensitive neural networks [19].”
BackgroundMethodInfluentialSynthesizing retinal and neuronal images with generative adversarial nets
Zhao, H., Li, H., Maurer-Stroh, S., & Cheng, L. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.001 · Cited by 210
Salehinejad, H., Valaee, S., Dowdell, T., Colak, E., & Barfett, J. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461430 · Cited by 207
Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image
Xiang, L., Wang, Q., Nie, D., Zhang, L., Jin, X., Qiao, Y., & Shen, D. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.03.011 · Cited by 184
Magnetic Resonance Image Example-Based Contrast Synthesis
Roy, S., Carass, A., & Prince, J. L. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2282126 · Cited by 162
Adversarial synthesis learning enables segmentation without target modality ground truth
Huo, Y., Xu, Z., Bao, S., Assad, A., Abramson, R. G., & Landman, B. A. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363790 · Cited by 157
Learning-Based Compressive MRI
Gözcü, B., Mahabadi, R. K., Li, Y.-H., Ilıcak, E., Çukur, T., Scarlett, J., & Cevher, V. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2832540 · Cited by 153
Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks
Hinz, T., Navarro-Guerrero, N., Magg, S., & Wermter, S. (2018) · International Journal of Computational Intelligence and Applications · doi:10.1142/s1469026818500086 · Cited by 149
Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network
Nguyen, H. V., Zhou, K., & Vemulapalli, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24553-9_83 · Cited by 140
Problems and preferences in pediatric imaging
Thukral, B. B. (2015) · Indian journal of radiology and imaging - new series/Indian journal of radiology and imaging/Indian Journal of Radiology & Imaging · doi:10.4103/0971-3026.169466 · Cited by 131
3D cGAN based cross-modality MR image synthesis for brain tumor segmentation
Yu, B., Zhou, L., Wang, L., Fripp, J., & Bourgeat, P. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363653 · Cited by 109 · cited 2×
“Perhaps, the closest to our approach are [62] and [73] where conditional”
BackgroundVirtual PET Images from CT Data Using Deep Convolutional Networks: Initial Results
Ben-Cohen, A., Klang, E., Raskin, S. P., Amitai, M. M., & Greenspan, H. K. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68127-6_6 · Cited by 102
Qi, S., Calhoun, V. D., van Erp, T. G. M., Bustillo, J., Damaraju, E., Turner, J. A., Du, Y., Yang, J., Chen, J., Yu, Q., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2725306 · Cited by 99
Magnetic resonance image synthesis through patch regression
Jog, A., Roy, S., Carass, A., & Prince, J. L. (2013) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2013.6556484 · Cited by 84
Huang, Y., Shao, L., & Frangi, A. F. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2781192 · Cited by 84
Whole Brain Segmentation and Labeling from CT Using Synthetic MR Images
Zhao, C., Carass, A., Lee, J., He, Y., & Prince, J. L. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67389-9_34 · Cited by 79
MR image synthesis by contrast learning on neighborhood ensembles
Jog, A., Carass, A., Roy, S., Pham, D. L., & Prince, J. L. (2015) · Medical Image Analysis · doi:10.1016/j.media.2015.05.002 · Cited by 76
Robust Multi-modal MR Image Synthesis
Joyce, T., Chartsias, A., & Tsaftaris, S. A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_40 · Cited by 72
Unsupervised Cross-Modal Synthesis of Subject-Specific Scans
Vemulapalli, R., Nguyen, H. V., & Zhou, S. K. (2015) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.79 · Cited by 69
Extended Modality Propagation: Image Synthesis of Pathological Cases
Cordier, N., Delingette, H., Lê, M., & Ayache, N. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2589760 · Cited by 62
MR contrast synthesis for lesion segmentation
Roy, S., Carass, A., Shiee, N., Pham, D. L., & Prince, J. L. (2010) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2010.5490140 · Cited by 61
Accelerated Magnetic Resonance Imaging by Adversarial Neural Network
Shitrit, O., & Raviv, T. R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67558-9_4 · Cited by 44
Atlas Based Intensity Transformation of Brain MR Images
Roy, S., Jog, A., Carass, A., & Prince, J. L. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-02126-3_6 · Cited by 28
Kim, T. H., & Haldar, J. P. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363523 · Cited by 21
Research works printed but not linked: 26
- yang, 2018, MRI image-to-image translation for cross-modality image registration and segmentation
- nishimura, 1996, Principles of magnetic resonance imaging
- bernstein, 2004, Handbook of MRI Pulse Sequences
- wolterink, 2017, MR-to-CT synthesis using cycle-consistent generative adversarial networks, Proc Neural Inf Process Syst (NIPS)
- hu, 2017, Freehand ultrasound image simulation with spatially-conditioned generative adversarial Networks, Proc Int Workshop Reconstruction Anal Moving Body Organs, 105
- huang, 2018, An introduction to image synthesis with generative adversarial nets
- calimeri, 2017, Biomedical data augmentation using generative adversarial neural Networks, Proc Int Conf Artif Neural Netw, 626
- chartsias, 2017, Adversarial image synthesis for unpaired multi-modal cardiac data, Proc Int Workshop Simul Synth Med Imaging, 3
- guibas, 2017, Synthetic medical images from dual generative adversarial networks
- murugan, 2017, Hyperparameters optimization in deep convolutional neural Network / Bayesian approach with Gaussian process prior
- kingma, 2015, Adam: A method for stochastic optimization, Proc Int Conf Learn Represent, 12
- wolterink, 2017, Deep MR to CT synthesis using unpaired data, Proc Int Workshop Simul Synth Med Imaging, 14
- ulyanov, 2016, Instance normalization The missing ingredient for fast stylization
- dosovitskiy, 2016, Generating images with perceptual similarity metrics based on deep networks, Proc Adv Neural Inf Process Syst, 658
- paszke, 2017, Automatic differentiation in PyTorch, Proc Neural Inf Process Syst (NIPS)
- chollet, 2015, Keras
- team, 2016, Theano A Python framework for fast computation of mathematical expressions
- johnson, 2016, Perceptual losses for real-time style transfer and super-resolution, Vision Computer, 694
- goodfellow, 2014, Generative adversarial networks, Proc Adv Neural Inf Process Syst, 2672
- roy, 2011, A compressed sensing approach for MR tissue contrast synthesis, Proc Biennial Int Conf Inf Process Med Imaging, 371
- bi, 2017, Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs), Proc Int Workshop Reconstruction Anal Moving Body Organs, 43
- sevetlidis, 2016, Whole image synthesis using a deep encoder-decoder Network, Proc Int Workshop Simul Synth Med Imaging, 127
- ye, 2013, Modality propagation: Coherent synthesis of subject-specific scans with data-driven regularization, Proc Int Conf Med Image Comput Comput -Assist Intervent, 606
- iglesias, 2013, Is synthesizing MRI contrast useful for inter-modality analysis?, Proc Int Conf Med Image Comput Comput -Assist Intervent, 631
- lee, 2017, Multi-atlas-based CT synthesis from conventional MRI with patch-based refinement for MRI-based radiotherapy planning, Proc SPIE, 10133
- nie, 2017, Medical image synthesis with deep convolutional adversarial Networks, Proc Med Image Comput Comput -Assist Intervent, 417
Cited by
Generative adversarial network in medical imaging: A review
Yi, X., Walia, E., & Babyn, P. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101552 · Cited by 1,863
A Review of Deep-Learning-Based Medical Image Segmentation Methods
Liu, X., Song, L., Liu, S., & Zhang, Y. (2021) · Sustainability · doi:10.3390/su13031224 · Cited by 868
Shin, H.-C., Tenenholtz, N. A., Rogers, J. K., Schwarz, C. G., Senjem, M. L., Gunter, J. L., Andriole, K. P., & Michalski, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00536-8_1 · Cited by 638
ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis
Dalmaz, O., Yurt, M., & Cukur, T. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3167808 · Cited by 597
Self-citationDeep learning for tomographic image reconstruction
Wang, G., Ye, J. C., & De Man, B. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00273-z · Cited by 531
Metadata
- Authors
- Dar, S. U., Yurt, M., Karacan, L., Erdem, A., Erdem, E., & Cukur, T.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2019-10
- Volume
- 38
- Issue
- 10
- Pages
- 2375-2388
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2786051876
- Cited by
- 608
- References
- 54
- FWCI
- 40.05
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- European Molecular Biology Organization (IG3028)
- Türkiye Bilimsel ve Teknolojik Araştirma Kurumu (118E256)
- BAGEP fellowship awarded
- Nvidia
- separate TUBA GEBIP fellowship
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.