Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks
Open accessAbstract
Labeled medical imaging data is scarce and expensive to generate. To achieve generalizable deep learning models large amounts of data are needed. Standard data augmentation is a method to increase generalizability and is routinely performed. Generative adversarial networks offer a novel method for data augmentation. We evaluate the use of CycleGAN for data augmentation in CT segmentation tasks. Using a large image database we trained a CycleGAN to transform contrast CT images into non-contrast images. We then used the trained CycleGAN to augment our training using these synthetic non-contrast images. We compared the segmentation performance of a U-Net trained on the original dataset compared to a U-Net trained on the combined dataset of original data and synthetic non-contrast images. We further evaluated the U-Net segmentation performance on two separate datasets: The original contrast CT dataset on which segmentations were created and a second dataset from a different hospital containing only non-contrast CTs. We refer to these 2 separate datasets as the in-distribution and out-of-distribution datasets, respectively. We show that in several CT segmentation tasks performance is improved significantly, especially in out-of-distribution (noncontrast CT) data. For example, when training the model with standard augmentation techniques, performance of segmentation of the kidneys on out-of-distribution non-contrast images was dramatically lower than for in-distribution data (Dice score of 0.09 vs. 0.94 for out-of-distribution vs. in-distribution data, respectively, p < 0.001). When the kidney model was trained with CycleGAN augmentation techniques, the out-of-distribution (non-contrast) performance increased dramatically (from a Dice score of 0.09 to 0.66, p < 0.001). Improvements for the liver and spleen were smaller, from 0.86 to 0.89 and 0.65 to 0.69, respectively. We believe this method will be valuable to medical imaging researchers to reduce manual segmentation effort and cost in CT imaging.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CycleGANs, trained on unpaired contrast and non-contrast CT scans, have been used to generate synthetic non-contrast or contrast CT scans, showing improvements in COVID-19 diagnosis.
“In medicine, a model architecture called 'CycleGANs' , trained on unpaired contrast and non-contrast CT scans, has been used to generate synthetic non-contrast or contrast CT scans 121 , with this approach showing improvements, for instance, in COVID-19 diagnosis 122”
From Multimodal biomedical AI (2022) · cites this paper as [121]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945 · see citing works
Generative Adversarial Networks: An Overview
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765202 · Cited by 4,540 · see citing works
Learning from Simulated and Unsupervised Images through Adversarial Training
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., & Webb, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.241 · Cited by 1,831 · see citing works
A unifying view on dataset shift in classification
Moreno-Torres, J. G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N. V., & Herrera, F. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.06.019 · Cited by 1,029 · see citing works
Data Augmentation Generative Adversarial NetworksPreprint
Antoniou, A., Storkey, A., & Edwards, H. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04340 · Cited by 797 · see citing works
Deep learning in medical imaging and radiation therapy
Sahiner, B., Pezeshk, A. X., Hadjiiski, L. M., Wang, X., Drukker, K., Cha, K. H., Summers, R. M., & Giger, M. L. (2018) · Medical Physics · doi:10.1002/mp.13264 · Cited by 739 · see citing works
Self-citationYan, K., Wang, X., Lu, L., & Summers, R. M. (2018) · Journal of medical imaging · doi:10.1117/1.jmi.5.3.036501 · Cited by 573 · see citing works
Self-citationBrain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge
Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., & Maier-Hein, K. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75238-9_25 · Cited by 528 · see citing works
CNN-based Segmentation of Medical Imaging DataPreprint
Kayalibay, B., Jensen, G. W., & van der Smagt, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03056 · Cited by 505 · see citing works
3D U-Net: Learning Dense Volumetric Segmentation from Sparse AnnotationPreprint
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.06650 · Cited by 488 · see citing works
Zhang, Z., Yang, L., & Zheng, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00963 · Cited by 455 · see citing works
Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired DataPreprint
Almahairi, A., Rajeswar, S., Sordoni, A., Bachman, P., & Courville, A. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10151 · Cited by 356 · see citing works
GAN Augmentation: Augmenting Training Data using Generative Adversarial NetworksPreprint
Bowles, C. T., Chen, L., Guerrero, R., Bentley, P., Gunn, R. N., Hammers, A., Dickie, D. A., Hernández, M. D. C. V., Wardlaw, J. M., & Rueckert, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10863 · Cited by 347 · see citing works
Augmenting Image Classifiers Using Data Augmentation Generative Adversarial Networks
Antoniou, A., Storkey, A., & Edwards, H. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01424-7_58 · Cited by 282 · see citing works
SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth
Huo, Y., Xu, Z., Moon, H., Bao, S., Assad, A., Moyo, T. K., Savona, M. R., Abramson, R. G., & Landman, B. A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2876633 · Cited by 278 · see citing works
Pickhardt, P. J., Lee, S., Liu, J., et al. (2018) · British Journal of Radiology · doi:10.1259/bjr.20180726 · Cited by 95 · see citing works
Self-citationSeeböck, P., Romo‐Bucheli, D., Waldstein, S. M., et al. (2019) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2019.8759158 · Cited by 29 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [3]Goodfellow, I. et al. Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q. (eds) Advances in Neural Information Processing Systems 27, 2672–2680 (Curran Associates, Inc., 2014).
- [19]Wu, Y. & He, K. Group Normalization. ArXiv e-prints, 1803.08494 (2018). · arXiv
- [20]Gonzalez, R. C. & Woods, R. E. Digital Image Processing -, 72–89, 004. aufl. edn (Pearson, München, 2018).
Cited by
Sorted by citations
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365 · see citing works
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793 · see citing works
Guo, P., Wang, P., Zhou, J., Jiang, S., & Patel, V. M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00245 · Cited by 193 · see citing works
Generating synthetic data for medical imaging
Koetzier, L. R., Wu, J., Mastrodicasa, D., Lutz, A., Chung, M., Koszek, W. A., Pratap, J., Chaudhari, A. S., Rajpurkar, P., Lungren, M. P., et al. (2024) · Radiology · doi:10.1148/radiol.232471 · Cited by 111 · see citing works
Vision-Language Navigation with Random Environmental Mixup
Liu, C., Zhu, F., Chang, X., Liang, X., Ge, Z., & Shen, Y.-D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00167 · Cited by 80 · see citing works
Advancing COVID-19 diagnosis with privacy-preserving collaboration in artificial intelligence
Bai, X., Wang, H., Ma, L., Xu, Y., Gan, J., Fan, Z., Yang, F., Ma, K., Yang, J., Bai, S., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00421-z · Cited by 75 · see citing works
Shobayo, O., & Saatchi, R. (2025) · Diagnostics · doi:10.3390/diagnostics15091072 · Cited by 53 · see citing works
Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey
Xu, L., Zhu, S., & Wen, N. (2022) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ac9cb3 · Cited by 35 · see citing works
Barrientos-Espillco, F., Gascó, E., López-González, C. I., Gómez-Silva, M. J., & Pajares, G. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110315 · Cited by 26 · see citing works
- Authors
- Sandfort, V., Yan, K., Pickhardt, P. J., & Summers, R. M.
- Affiliations
- National Institutes of Health (US)
- University of Wisconsin–Madison (US)
- Corresponding author
- Summers, Ronald M.
- Venue
- Scientific Reports
- Published
- 2019-11-15
- Volume
- 9
- Issue
- 1
- Pages
- 16884-16884
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC6858365
- OpenAlex
- W2984306354
- Cited by
- 684
- References
- 19
- Citations by year
- 2026: 54 · 2025: 103 · 2024: 118 · 2023: 128 · 2022: 103 · 2021: 127 · 2020: 50 · 2019: 1
- FWCI
- 31.44
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- U.S. Department of Health & Human Services | NIH | NIH Clinical Center (Intramural)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- CycleGAN, generative adversarial networks, data augmentation, CT image segmentation, synthetic images, out-of-distribution generalization, kidney segmentation, liver segmentation, spleen segmentation, U-Net
- MeSH terms
- Deep Learning, Algorithms, Humans, Image Enhancement, Image Processing, Computer-Assisted, Kidney, Liver, Organ Size, Research Design, Spleen, Tomography, X-Ray Computed, Neural Networks, Computer, Anatomy, Cross-Sectional, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
