CoChat
Sign Up
Cited by 257Domain Adaptation and Few-Shot LearningAdvanced Vision and ImagingGenerative Adversarial Networks and Image Synthesis

Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training

Open access
Mahmood, Faisal, Chen, Richard, Durr, Nicholas J.·2018·IEEE Transactions on Medical Imaging
DOI 10.1109/tmi.2018.2842767
Open this paper in CoChat — free accountOpen PDF

Abstract

To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tmi.2018.2842767

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Adversarial training uses carefully crafted adversarial examples to attack the training model to check for the integrity and robustness of outputs.

    “The method uses carefully crafted adversarial examples to attack the training model to check for the integrity and robustness of outputs [57,79].”

    From Generative AI in medical practice: in-depth exploration of privacy and security challenges (2024) · cites this paper as [79]

    The Contributions section (p. 2) describes adversarial training as transforming real endoscopy images into synthetic-like images for domain adaptation. Section 4 (p. 4) says the discriminator classifies images as real or synthetic and gives feedback to the transformer. That is a different method from using crafted adversarial examples to attack a model and test the robustness of its outputs. (LLM, conf=0.87)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 51 · Cited by 257

Research works printed but not linked: 31

  1. [2][2] K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan. Unsupervised pixel-level domain adaptation with generative adversarial networks. arXiv preprint arXiv:1612.05424 , 2016. 2 - · arXiv
  2. [3][3] P. Costa, A. Galdran, M. I. Meyer, M. Niemeijer, M. Abr` amoff, A. M. Mendonc ¸a, and A. Campilho. End-toend adversarial retinal image synthesis. IEEE Transactions on Medical Imaging , 2017. 3 -
  3. [5][5] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems , pages 2672-2680, 2014. 2, 3, 4 -
  4. [6][6] H. Greenspan, B. van Ginneken, and R. M. Summers. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Transactions on Medical Imaging , 35(5):1153-1159, 2016. 1 -
  5. [7][7] J. T. Guibas, T. S. Virdi, and P. S. Li. Synthetic medical images from dual generative adversarial networks. arXiv preprint arXiv:1709.01872 , 2017. 2, 3 - · arXiv
  6. [8][8] A. Gupta, A. Vedaldi, and A. Zisserman. Synthetic data for text localisation in natural images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2315-2324, 2016. 3 -
  7. [10][10] D. P. Huttenlocher, G. A. Klanderman, and W. J. Rucklidge. Comparing images using the hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence , 15(9):850-863, 1993. 7 -
  8. [11][11] S. Kohl, D. Bonekamp, H.-P. Schlemmer, K. Yaqubi, M. Hohenfellner, B. Hadaschik, J.-P. Radtke, and K. Maier-Hein. Adversarial networks for the detection of aggressive prostate cancer. arXiv preprint arXiv:1702.08014 , 2017. 3 - · arXiv
  9. [12][12] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pages 1097-1105, 2012. 2 -
  10. [13][13] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740-755. Springer, 2014. 2 -
  11. [14][14] F. Liu, C. Shen, G. Lin, and I. Reid. Learning depth from single monocular images using deep convolutional neural fields. IEEE transactions on pattern analysis and machine intelligence , 38(10):2024-2039, 2016. 5, 6 -
  12. [15][15] M. Mardani, E. Gong, J. Y. Cheng, S. Vasanawala, G. Zaharchuk, M. Alley, N. Thakur, S. Han, W. Dally, J. M. Pauly, et al. Deep generative adversarial networks for compressed sensing automates mri. arXiv preprint arXiv:1706.00051 , 2017. 3 - · arXiv
  13. [16][16] P. Moeskops, M. Veta, M. W. Lafarge, K. A. Eppenhof, and J. P. Pluim. Adversarial training and dilated convolutions for brain mri segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support , pages 56-64. Springer, 2017. 3 -
  14. [17][17] S. Nadeem and A. Kaufman. Computer-aided detection of polyps in optical colonoscopy images. In SPIE Medical Imaging , pages 978525-978525. International Society for Optics and Photonics, 2016. 7, 8 -
  15. [19][19] C. Payer, D. ˇ Stern, H. Bischof, and M. Urschler. Regressing heatmaps for multiple landmark localization using cnns. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 230-238. Springer, 2016. 3 -
  16. [20][20] B. Planche, Z. Wu, K. Ma, S. Sun, S. Kluckner, T. Chen, A. Hutter, S. Zakharov, H. Kosch, and J. Ernst. Depthsynth: Real-time realistic synthetic data generation from cad models for 2.5 d recognition. arXiv preprint arXiv:1702.08558 , 2017. 3 - · arXiv
  17. [21][21] T. Qin, T.-Y. Liu, X.-D. Zhang, D.-S. Wang, and H. Li. Global ranking using continuous conditional random fields. In Advances in neural information processing systems , pages 1281-1288, 2009. 5 -
  18. [22][22] W. Qiu and A. Yuille. Unrealcv: Connecting computer vision to unreal engine. In Computer Vision-ECCV 2016 Workshops , pages 909-916. Springer, 2016. 2 -
  19. [23][23] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 234-241. Springer, 2015. 2 -
  20. [25][25] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems , pages 2234-2242, 2016. 3, 4 -
  21. [26][26] A. Shafaei, J. J. Little, and M. Schmidt. Play and learn: Using video games to train computer vision models. arXiv preprint arXiv:1608.01745 , 2016. 2 - · arXiv
  22. [27][27] D. Shen, G. Wu, and H.-I. Suk. Deep learning in medical image analysis. Annual Review of Biomedical Engineering , (0), 2017. 1 -
  23. [28][28] A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. Wang, and R. Webb. Learning from simulated and unsupervised images through adversarial training. arXiv preprint arXiv:1612.07828 , 2016. 2, 3, 4, 6 - · arXiv
  24. [29][29] H. Su, C. R. Qi, Y. Li, and L. J. Guibas. Render for cnn: Viewpoint estimation in images using cnns trained with ren- dered 3d model views. In Proceedings of the IEEE International Conference on Computer Vision , pages 2686-2694, 2015. 3 -
  25. [30][30] N. Tajbakhsh, S. R. Gurudu, and J. Liang. Automated polyp detection in colonoscopy videos using shape and context information. IEEE transactions on medical imaging , 35(2):630-644, 2016. 6 -
  26. [31][31] G. Varol, J. Romero, X. Martin, N. Mahmood, M. Black, I. Laptev, and C. Schmid. Learning from synthetic humans. arXiv preprint arXiv:1701.01370 , 2017. 3 - · arXiv
  27. [32][32] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600-612, 2004. 8 -
  28. [33][33] J. M. Wolterink, T. Leiner, M. A. Viergever, and I. Isgum. Generative adversarial networks for noise reduction in lowdose ct. IEEE Transactions on Medical Imaging , 2017. 3 -
  29. [34][34] L. Zhang, A. Gooya, and A. F. Frangi. Semi-supervised assessment of incomplete lv coverage in cardiac mri using generative adversarial nets. In International Workshop on Simulation and Synthesis in Medical Imaging , pages 61-68. Springer, 2017. 3 -
  30. [35][35] Y. Zhang, L. Yang, J. Chen, M. Fredericksen, D. P. Hughes, and D. Z. Chen. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 408-416. Springer, 2017. 3 -
  31. [36][36] J.-Y. Zhu, P. Kr¨ ahenb¨ uhl, E. Shechtman, and A. A. Efros. Generative visual manipulation on the natural image manifold. In European Conference on Computer Vision , pages 597-613. Springer, 2016. 3

Authors
Mahmood, F., Chen, R., & Durr, N. J.
Affiliations
Johns Hopkins University (US)
ORCID
Venue
IEEE Transactions on Medical Imaging
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2018-12
Volume
37
Issue
12
Pages
2572-2581
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
DOI
10.1109/tmi.2018.2842767
OpenAlex
W2770363598
Cited by
257
References
51
Citations by year
2026: 17 · 2025: 24 · 2024: 24 · 2023: 27 · 2022: 29 · 2021: 47 · 2020: 46 · 2019: 35 · 2018: 8
FWCI
19.14
Type
journal article
Language
en
Open access
Licence
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Johns Hopkins University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
synthetic medical images, adversarial training, self-regularization, depth estimation, colonoscopy, domain adaptation
MeSH terms
Unsupervised Machine Learning, Endoscopy, Humans, Image Interpretation, Computer-Assisted, Image Processing, Computer-Assisted, Databases, Factual, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review