CoChat
Sign Up
PreprintCited by 2

3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation

Lee, Ho Hin, Bao, Shunxing, Huo, Yuankai, Landman, Bennett A.·2022·arXiv
DOI 10.48550/arxiv.2209.15076arXiv 2209.15076
Open this paper in CoChat — free account

Abstract

The recent 3D medical ViTs (e.g., SwinUNETR) achieve the state-of-the-art performances on several 3D volumetric data benchmarks, including 3D medical image segmentation. Hierarchical transformers (e.g., Swin Transformers) reintroduced several ConvNet priors and further enhanced the practical viability of adapting volumetric segmentation in 3D medical datasets. The effectiveness of hybrid approaches is largely credited to the large receptive field for non-local self-attention and the large number of model parameters. In this work, we propose a lightweight volumetric ConvNet, termed 3D UX-Net, which adapts the hierarchical transformer using ConvNet modules for robust volumetric segmentation. Specifically, we revisit volumetric depth-wise convolutions with large kernel size (e.g. starting from $7\times7\times7$) to enable the larger global receptive fields, inspired by Swin Transformer. We further substitute the multi-layer perceptron (MLP) in Swin Transformer blocks with pointwise depth convolutions and enhance model performances with fewer normalization and activation layers, thus reducing the number of model parameters. 3D UX-Net competes favorably with current SOTA transformers (e.g. SwinUNETR) using three challenging public datasets on volumetric brain and abdominal imaging: 1) MICCAI Challenge 2021 FLARE, 2) MICCAI Challenge 2021 FeTA, and 3) MICCAI Challenge 2022 AMOS. 3D UX-Net consistently outperforms SwinUNETR with improvement from 0.929 to 0.938 Dice (FLARE2021) and 0.867 to 0.874 Dice (Feta2021). We further evaluate the transfer learning capability of 3D UX-Net with AMOS2022 and demonstrates another improvement of $2.27\%$ Dice (from 0.880 to 0.900). The source code with our proposed model are available at https://github.com/MASILab/3DUX-Net.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2209.15076

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • 3D UX-Net and Swin-UNETR represent recent high-capacity 3D architectures based on convolutional and transformer-style volumetric modeling.

    “3D UX-Net and Swin-UNETR represent recent high-capacity 3D architectures based on convolutional and transformer-style volumetric modeling.”

    From Correction-aware interactive 3D tumor segmentation with sparse and revisable prompts (2026) · cites this paper as [13]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 41

  1. Atito Sara, Awais Muhammad, and Kittler Josef. Sit: Self-supervised vision transformer. arXiv preprint arXiv:2104.03602, 2021. · arXiv
  2. Lei Ba Jimmy, Ryan Kiros Jamie, and Hinton Geoffrey E Layer normalization. arXiv preprint arXiv:1607.06450, 2016. · arXiv
  3. Bao Hangbo, Dong Li, and Wei Furu. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. · arXiv
  4. Cao Hu, Wang Yueyue, Chen Joy, Jiang Dongsheng, Zhang Xiaopeng, Tian Qi, and Wang Manning. Swin-unet: Unet-like pure transformer for medical image segmentation. arXiv preprint arXiv:2105.05537, 2021. · arXiv
  5. Chen Jieneng, Lu Yongyi, Yu Qihang, Luo Xiangde, Adeli Ehsan, Wang Yan, Lu Le, Yuille Alan L, and Zhou Yuyin Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021. · arXiv
  6. Çiçek Özgün, Abdulkadir Ahmed, Lienkamp Soeren S, Brox Thomas, and Ronneberger Olaf. 3d u-net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisted intervention, pp. 424-432. Springer, 2016.
  7. Ding Xiaohan, Zhang Xiangyu, Ma Ningning, Han Jungong, Ding Guiguang, and Sun Jian. Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13733-13742, 2021.
  8. Ding Xiaohan, Chen Honghao, Zhang Xiangyu, Huang Kaiqi, Han Jungong, and Ding Guiguang. Re- parameterizing your optimizers rather than architectures. arXiv preprint arXiv:2205.15242, 2022a. · arXiv
  9. Ding Xiaohan, Zhang Xiangyu, Han Jungong, and Ding Guiguang. Scaling up your kernels to 31×31: Revisiting large kernel design in cnns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11963-11975, 2022b.
  10. Dosovitskiy Alexey, Beyer Lucas, Kolesnikov Alexander, Weissenborn Dirk, Zhai Xiaohua, Unterthiner Thomas, Dehghani Mostafa, Minderer Matthias, Heigold Georg, Gelly Sylvain, et al. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. · arXiv
  11. Guo Yunhui, Li Yandong, Wang Liqiang, and Rosing Tajana. Depthwise convolution is all you need for learning multiple visual domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 8368-8375, 2019.
  12. Han Zhimeng, Jian Muwei, and Wang Gai-Ge. Convunext: An efficient convolution neural network for medical image segmentation. Knowledge-Based Systems, pp. 109512, 2022.
  13. Hatamizadeh Ali, Nath Vishwesh, Tang Yucheng, Yang Dong, Roth Holger R, and Xu Daguang Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI Brainlesion Workshop, pp. 272-284. Springer, 2022a.
  14. Hatamizadeh Ali, Tang Yucheng, Nath Vishwesh, Yang Dong, Myronenko Andriy, Landman Bennett, Roth Holger R, and Xu Daguang Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 574- 584, 2022b.
  15. He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
  16. He Kaiming, Chen Xinlei, Xie Saining, Li Yanghao, Dollár Piotr, and Girshick Ross Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000-16009, 2022.
  17. Hendrycks Dan and Gimpel Kevin. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. · arXiv
  18. Hu Mu, Feng Junyi, Hua Jiashen, Lai Baisheng, Huang Jianqiang, Gong Xiaojin, and Hua Xian- Sheng. Online convolutional re-parameterization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 568-577, 2022.
  19. Huang Chao, Han Hu, Yao Qingsong, Zhu Shankuan, and Zhou S Kevin 3d u2-net: A 3d universal u-net for multi-domain medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 291-299. Springer, 2019.
  20. Huo Yuankai, Xu Zhoubing, Bao Shunxing, Bermudez Camilo, Moon Hyeonsoo, Parvathaneni Prasanna, Moyo Tamara K, Savona Michael R, Assad Albert, Abramson Richard G, et al. Splenomegaly segmentation on multi-modal mri using deep convolutional networks. IEEE transactions on medical imaging, 38(5):1185-1196, 2018.
  21. Isensee Fabian, Jaeger Paul F, Kohl Simon AA, Petersen Jens, and Maier-Hein Klaus H nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203-211, 2021.
  22. Ji Yuanfeng, Bai Haotian, Yang Jie, Ge Chongjian, Zhu Ye, Zhang Ruimao, Li Zhen, Zhang Lingyan, Ma Wanling, Wan Xiang, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. arXiv preprint arXiv:2206.08023, 2022. · arXiv
  23. Jiang Yun, Zhang Yuan, Lin Xin, Dong Jinkun, Cheng Tongtong, and Liang Jing. Swinbts: a method for 3d multimodal brain tumor segmentation using swin transformer. Brain Sciences, 12(6):797, 2022.
  24. Lee Ho Hin, Tang Yucheng, Bao Shunxing, Abramson Richard G, Huo Yuankai, and Landman Bennett A Rap-net: Coarse-to-fine multi-organ segmentation with single random anatomical prior. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1491-1494. IEEE, 2021.
  25. Li Hao, Nan Yang, and Yang Guang. Lkau-net: 3d large-kernel attention-based u-net for automatic mri brain tumor segmentation. In Annual Conference on Medical Image Understanding and Analysis, pp. 313-327. Springer, 2022.
  26. Liu Ze, Lin Yutong, Cao Yue, Hu Han, Wei Yixuan, Zhang Zheng, Lin Stephen, and Guo Baining. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012-10022, 2021.
  27. Liu Zhuang, Mao Hanzi, Wu Chao-Yuan, Feichtenhofer Christoph, Darrell Trevor, and Xie Saining. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976-11986, 2022.
  28. Ma Jun, Zhang Yao, Gu Song, Zhu Cheng, Ge Cheng, Zhang Yichi, An Xingle, Wang Congcong, Wang Qiyuan, Liu Xin, et al. Abdomenct-1k: Is abdominal organ segmentation a solved problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  29. Myronenko Andriy. 3d mri brain tumor segmentation using autoencoder regularization. In International MICCAI Brainlesion Workshop, pp. 311-320. Springer, 2018.
  30. Payette Kelly, de Dumast Priscille, Kebiri Hamza, Ezhov Ivan, Paetzold Johannes C, Shit Suprosanna, Iqbal Asim, Khan Romesa, Kottke Raimund, Grehten Patrice, et al. An automatic multi-tissue human fetal brain segmentation benchmark using the fetal tissue annotation dataset. Scientific Data, 8(1):1-14, 2021.
  31. Salimans Tim and Kingma Durk P Weight normalization: A simple reparameterization to accelerate training of deep neural networks. Advances in neural information processing systems, 29, 2016.
  32. Shamshad Fahad, Khan Salman, Zamir Syed Waqas, Khan Muhammad Haris, Hayat Munawar, Khan Fahad Shahbaz, and Fu Huazhu Transformers in medical imaging: A survey. arXiv preprint arXiv:2201.09873, 2022. · arXiv
  33. Tang Yucheng, Yang Dong, Li Wenqi, Holger R Roth Bennett Landman, Xu Daguang, Nath Vishwesh, and Hatamizadeh Ali. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20730-20740, 2022.
  34. Ulyanov Dmitry, Vedaldi Andrea, and Lempitsky Victor. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016. · arXiv
  35. Wang Wenxuan, Chen Chen, Ding Meng, Yu Hong, Zha Sen, and Li Jiangyun. Transbts: Multimodal brain tumor segmentation using transformer. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 109-119. Springer, 2021.
  36. Wu Yuxin and He Kaiming. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pp. 3-19, 2018.
  37. Wu Yuxin and Johnson Justin. Rethinking" batch" in batchnorm. arXiv preprint arXiv:2105.07576, 2021. · arXiv
  38. Xie Yutong, Zhang Jianpeng, Shen Chunhua, and Xia Yong. Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation. In International conference on medical image computing and computer-assisted intervention, pp. 171-180. Springer, 2021.
  39. Zhang Zizhao, Zhang Han, Zhao Long, Chen Ting, Arik Sercan Ö, and Pfister Tomas Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 3417-3425, 2022.
  40. Zhou Hong-Yu, Guo Jiansen, Zhang Yinghao, Yu Lequan, Wang Liansheng, and Yu Yizhou. nn- former: Interleaved transformer for volumetric segmentation. arXiv preprint arXiv:2109.03201, 2021. · arXiv
  41. Zhou Zongwei, Siddiquee Md Mahfuzur Rahman, Tajbakhsh Nima, and Liang Jianming Unet+ +: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE transactions on medical imaging, 39(6):1856-1867, 2019.

Cited by

Sorted by citations

Authors
Lee, H. H., Bao, S., Huo, Y., & Landman, B. A.
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2209.15076
arXiv
2209.15076
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review