The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Generalizing Deep Learning for Medical Image Segmentation to Unseen Domains via Deep Stacked Transformation
Open accessAbstract
Recent advances in deep learning for medical image segmentation demonstrate expert-level accuracy. However, application of these models in clinically realistic environments can result in poor generalization and decreased accuracy, mainly due to the domain shift across different hospitals, scanner vendors, imaging protocols, and patient populations etc. Common transfer learning and domain adaptation techniques are proposed to address this bottleneck. However, these solutions require data (and annotations) from the target domain to retrain the model, and is therefore restrictive in practice for widespread model deployment. Ideally, we wish to have a trained (locked) model that can work uniformly well across unseen domains without further training. In this paper, we propose a deep stacked transformation approach for domain generalization. Specifically, a series of n stacked transformations are applied to each image during network training. The underlying assumption is that the "expected" domain shift for a specific medical imaging modality could be simulated by applying extensive data augmentation on a single source domain, and consequently, a deep model trained on the augmented "big" data (BigAug) could generalize well on unseen domains. We exploit four surprisingly effective, but previously understudied, image-based characteristics for data augmentation to overcome the domain generalization problem. We train and evaluate the BigAug model (with n=9 transformations) on three different 3D segmentation tasks (prostate gland, left atrial, left ventricle) covering two medical imaging modalities (MRI and ultrasound) involving eight publicly available challenge datasets. The results show that when training on relatively small dataset (n = 10~32 volumes, depending on the size of the available datasets) from a single source domain: (i) BigAug models degrade an average of 11%(Dice score change) from source to unseen domain, substantially better than conventional augmentation (degrading 39%) and CycleGAN-based domain adaptation method (degrading 25%), (ii) BigAug is better than "shallower" stacked transforms (i.e. those with fewer transforms) on unseen domains and demonstrates modest improvement to conventional augmentation on the source domain, (iii) after training with BigAug on one source domain, performance on an unseen domain is similar to training a model from scratch on that domain when using the same number of training samples. When training on large datasets (n = 465 volumes) with BigAug, (iv) application to unseen domains reaches the performance of state-of-the-art fully supervised models that are trained and tested on their source domains. These findings establish a strong benchmark for the study of domain generalization in medical imaging, and can be generalized to the design of highly robust deep segmentation models for clinical deployment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 1×
“We use Dice loss [34] as the loss function which naturally balances the positive and negative voxel distribution.”
MethodAdversarial Discriminative Domain Adaptation
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.316 · Cited by 5,154
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 2×
“important regularization techniques related to deep learning’s generalization performance [23].”
BackgroundAutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00020 · Cited by 2,733
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638
3D MRI Brain Tumor Segmentation Using Autoencoder Regularization
Myronenko, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11726-9_28 · Cited by 1,283
Self-citationEvaluation of prostate segmentation algorithms for mri: The promise12 challenge
Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., et al. (2014) · Medical Image Analysis · doi:10.1016/j.media.2013.12.002 · Cited by 857
Simpson, A. L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., van Ginneken, B., Kopp‐Schneider, A., Landman, B. A., Litjens, G. J. S., Menze, B. H., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09063 · Cited by 643
Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge
Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., & Maier-Hein, K. H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-75238-9_25 · Cited by 527
Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI
Zhuang, X., & Shen, J. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.02.006 · Cited by 502
Unsupervised Domain Adaptation in Brain Lesion Segmentation with Adversarial Networks
Kamnitsas, K., Baumgartner, C., Ledig, C., Newcombe, V., Simpson, J., Kane, A., Menon, D., Nori, A., Criminisi, A., Rueckert, D., et al. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-59050-9_47 · Cited by 468
Kickingereder, P., Isensee, F., Tursunova, I., Petersen, J., Neuberger, U., Bonekamp, D., Brugnara, G., Schell, M., Kessler, T., Foltyn, M., et al. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30098-1 · Cited by 454
Computer-Aided Detection of Prostate Cancer in MRI
Litjens, G., Debats, O., Barentsz, J., Karssemeijer, N., & Huisman, H. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2303821 · Cited by 447
Generalizing to Unseen Domains via Adversarial Data Augmentation
Volpi, R., Namkoong, H., Şener, O., Duchi, J. C., Murino, V., & Savarese, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12018 · Cited by 343
GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
Bowles, C. T., Chen, L., Guerrero, R., Bentley, P., Gunn, R. N., Hammers, A., Dickie, D. A., Hernández, M. D. C. V., Wardlaw, J. M., & Rueckert, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10863 · Cited by 327
Fast and sequence-adaptive whole-brain segmentation using parametric Bayesian modeling
Puonti, O., Iglesias, J. E., & Van Leemput, K. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.09.011 · Cited by 258
Deep learning and artificial intelligence in radiology: current applications and future directions
Yasaka, K., & Abe, O. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002707 · Cited by 238 · cited 1×
“generalizability has become one of the major roadblocks for deploying deep learning models into clinical practice [4].”
BackgroundZhang, Y., Miao, S., Mansi, T., & Liao, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00934-2_67 · Cited by 204
Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation
Ghafoorian, M., Mehrtash, A., Kapur, T., Karssemeijer, N., Marchiori, E., Pesteie, M., Guttmann, C. R. G., Leeuw, F.-E. D., Tempany, C. M., van Ginneken, B., et al. (2017) · Radboud Repository (Radboud University) · Cited by 192
Boundary-Weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation
Zhu, Q., Du, B., & Yan, P. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2935018 · Cited by 191
Xiong, Z., Fedorov, V. V., Fu, X., Cheng, E. C., MacLeod, R., & Zhao, J. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2866845 · Cited by 160
Chen, C., Dou, Q., Chen, H., & Heng, P.-A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00919-9_17 · Cited by 159
A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols
Karani, N., Chaitanya, K., Baumgartner, C. F., & Konukoğlu, E. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_54 · Cited by 151
Liu, S., Xu, D., Zhou, S. K., Pauly, O., Grbić, S., Mertelmeier, T., Wicklein, J., Jerebko, A. K., Cai, W., & Comaniciu, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_94 · Cited by 137
Self-citationJia, H., Xia, Y., Song, Y., Zhang, D., Huang, H., Zhang, Y., & Cai, W. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2928056 · Cited by 119
Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images
Ren, J., Hacihaliloglu, I., Singer, E. A., Foran, D. J., & Qi, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_23 · Cited by 118
Computer-aided detection of prostate cancer
Llobet, R., Pérez-Cortés, J.-C., Toselli, A. H., et al. (2006) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2006.03.001 · Cited by 66
Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation
Yang, D., Roth, H. R., Xu, Z., Milletarì, F., Zhang, L., & Xu, D.-G. (2019) · International Conference on Medical Image Computing and Computer-Assisted Intervention · doi:10.1007/978-3-030-32245-8_1 · Cited by 52
Self-citationDomain and Geometry Agnostic CNNs for Left Atrium Segmentation in 3D Ultrasound
Degel, M., Navab, N., & Albarqouni, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_72 · Cited by 47
Generalizing Deep Models for Ultrasound Image Segmentation
Yang, X., Dou, H., Li, R., Wang, X., Bian, C., Li, S., Ni, D., & Heng, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_57 · Cited by 37
Self-citationTrain Here, Deploy There: Robust Segmentation in Unseen Domains
Romera, E., Bergasa, L. M., Álvarez, J. M., & Trivedi, M. M. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500561 · Cited by 34
Pulse Sequence Resilient Fast Brain Segmentation
Jog, A., & Fischl, B. R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00931-1_75 · Cited by 16
A Cross-Center Smoothness Prior for Variational Bayesian Brain Tissue Segmentation
Kouw, W. M., Ørting, S. N., Petersen, J., Pedersen, K. S., & de Bruijne, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20351-1_27 · Cited by 14
Nonlinear Markov Random Fields Learned via Backpropagation
Brudfors, M., Balbastre, Y., & Ashburner, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20351-1_63 · Cited by 8
Cited by
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Domain adaptation for medical image analysis: A survey
Guan, H., & Liu, M. (2022) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3117407 · Cited by 1,057
A review of medical image data augmentation techniques for deep learning applications
Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., & Haworth, A. (2021) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13261 · Cited by 956
SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining
Billot, B., Greve, D. N., Puonti, O., Thielscher, A., Van Leemput, K., Fischl, B., Dalca, A. V., & Iglesias, J. E. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102789 · Cited by 641
Liu, Q., Chen, C., Qin, J., Dou, Q., & Heng, P.-A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00107 · Cited by 541
Metadata
- Authors
- Zhang, L., Wang, X., Yang, D., Sanford, T., Harmon, S., Turkbey, B., Wood, B. J., Roth, H., Myronenko, A., Xu, D., & Xu, Z.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2020-07
- Volume
- 39
- Issue
- 7
- Pages
- 2531-2540
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W3006326483
- Cited by
- 542
- References
- 40
- FWCI
- 22.83
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.