The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
MISSFormer: An Effective Transformer for 2D Medical Image Segmentation
Abstract
Transformer-based methods are recently popular in vision tasks because of their capability to model global dependencies alone. However, it limits the performance of networks due to the lack of modeling local context and global-local correlations of multi-scale features. In this paper, we present MISSFormer, a Medical Image Segmentation tranSFormer. MISSFormer is a hierarchical encoder-decoder network with two appealing designs: 1) a feed-forward network in transformer block of U-shaped encoder-decoder structure is redesigned, ReMix-FFN, which explore global dependencies and local context for better feature discrimination by re-integrating the local context and global dependencies; 2) a ReMixed Transformer Context Bridge is proposed to extract the correlations of global dependencies and local context in multi-scale features generated by our hierarchical transformer encoder. The MISSFormer shows a solid capacity to capture more discriminative dependencies and context in medical image segmentation. The experiments on multi-organ, cardiac segmentation and retinal vessel segmentation tasks demonstrate the superiority, effectiveness and robustness of our MISSFormer. Specifically, the experimental results of MISSFormer trained from scratch even outperform state-of-the-art methods pre-trained on ImageNet, and the core designs can be generalized to other visual segmentation tasks. The code has been released on Github: https://github.com/ZhifangDeng/MISSFormer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477
UNet++: A Nested U-Net Architecture for Medical Image Segmentation
Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., & Liang, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00889-5_1 · Cited by 9,610
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945
Swin-Unet: Unet-like pure transformer for medical image segmentation
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., & Wang, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25066-8_9 · Cited by 5,640
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 4×
“To reduce the computational complexity, Wang et al. [17] and Liu et al. proposed an efficient and effective hierarchical vision transformer with SRA and window-based mechanism, respectively.”
BackgroundInfluentialSegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., & Zhou, Y. (2021) · arXiv · doi:10.48550/arxiv.2102.04306 · Cited by 4,206 · cited 5×
“TransUNet [22] and CoTr [8] complement global dependencies by adding transformer layers to CNN. Swin-Unet [7] is a pure trans-former to capture the global dependencies by referring to Swin Transformer [15].”
BackgroundMethodInfluentialRethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 4×
“Especially, the research on visual transformer [15], [16], [17], [18], [19], [20], [21] are in full swing and have got a promising performance in vision tasks, encouraged”
BackgroundMethodInfluentialSelf-citationUNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H. R., & Xu, D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00181 · Cited by 3,586 · cited 1×
“Hatamizadeh et al. [35] applied transformer to make a powerful encoder for 3d medical image segmentation with CNN-based decoder.”
BackgroundUNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H., & Xu, D. (2021) · arXiv · doi:10.48550/arxiv.2103.10504 · Cited by 3,580
UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation
Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.-W., & Wu, J. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053405 · Cited by 2,774
Pvt v2: Improved baselines with pyramid vision transformer
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0274-8 · Cited by 2,439
Uformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
CE-Net: Context Encoder Network for 2D Medical Image Segmentation
Gu, Z., Cheng, J., Fu, H., Zhou, K., Hao, H., Zhao, Y., Zhang, T., Gao, S., & Liu, J. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2903562 · Cited by 2,267
PraNet: Parallel Reverse Attention Network for Polyp Segmentation
Fan, D.-P., Ji, G.-P., Zhou, T., Chen, G., Fu, H., Shen, J., & Shao, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59725-2_26 · Cited by 1,499
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer
Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., & Li, J. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87193-2_11 · Cited by 1,045
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935 · cited 1×
“Especially, the research on visual transformer [15], [16], [17], [18], [19], [20], [21] are in full swing and have got a promising performance in vision tasks, encouraged by the great success of transformer in natural language processing (NLP).”
BackgroundTwins: Revisiting the Design of Spatial Attention in Vision Transformers
Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., & Shen, C. (2021) · arXiv · doi:10.48550/arxiv.2104.13840 · Cited by 817
Trainable COSFIRE filters for vessel delineation with application to retinal images
Azzopardi, G., Strisciuglio, N., Vento, M., & Petkov, N. (2015) · Medical Image Analysis · doi:10.1016/j.media.2014.08.002 · Cited by 713
CPFNet: Context Pyramid Fusion Network for Medical Image Segmentation
Feng, S., Zhao, H., Shi, F., Cheng, X., Wang, M., Ma, Y., Xiang, D., Zhu, W., & Chen, X. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2983721 · Cited by 687
Reverse Attention for Salient Object Detection
Chen, S., Tan, X., Wang, B., & Hu, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_15 · Cited by 684
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation
Xie, Y., Zhang, J., Shen, C., & Xia, Y. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87199-4_16 · Cited by 653
Multi-Scale Self-Guided Attention for Medical Image Segmentation
Sinha, A., & Dolz, J. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2986926 · Cited by 570
Conditional Positional Encodings for Vision Transformers
Chu, X., Tian, Z., Zhang, B., Wang, X., & Shen, C. (2021) · arXiv · doi:10.48550/arxiv.2102.10882 · Cited by 489 · cited 2×
“Second, the transformer lacks the ability to extract the local context [30], [31], [32], although Uformer, SegFormer and PVTv2 tried to overcome the limitation by embedding a convolutional layer in feed-forward network directly, we argue that this approach limits the discrimination of features,…”
BackgroundMethodDeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field
Fu, H., Xu, Y., Lin, S., Kee Wong, D. W., & Liu, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_16 · Cited by 453
LocalViT: Analyzing Locality in Vision Transformers
Li, Y., Zhang, K., Cao, J., Timofte, R., Magno, M., Benini, L., & Van Goo, L. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342025 · Cited by 372
CS-Net: Channel and Spatial Attention Network for Curvilinear Structure Segmentation
Mou, L., Zhao, Y., Chen, L., Cheng, J., Gu, Z., Hao, H., Qi, H., Zheng, Y., Frangi, A., & Liu, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32239-7_80 · Cited by 274
How Much Position Information Do Convolutional Neural Networks Encode?
Islam, M. A., Jia, S., & Bruce, N. D. B. (2020) · arXiv · doi:10.48550/arxiv.2001.08248 · Cited by 253
ET-Net: A Generic Edge-aTtention Guidance Network for Medical Image Segmentation
Zhang, Z., Fu, H., Dai, H., Shen, J., Pang, Y., & Shao, L. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32239-7_49 · Cited by 246
Rethinking Skip Connection with Layer Normalization
Liu, F., Ren, X., Zhang, Z., Sun, X., & Zou, Y. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.320 · Cited by 67
Cited by
Chen, J., Mei, J., Li, X., Lu, Y., Yu, Q., Wei, Q., Luo, X., Xie, Y., Adeli, E., Wang, Y., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103280 · Cited by 1,365
UNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation
Shaker, A., Maaz, M., Rasheed, H., Khan, S., Yang, M.-H., & Shahbaz Khan, F. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3398728 · Cited by 500
Advances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 485
Li, J., Chen, J., Tang, Y., Wang, C., Landman, B. A., & Zhou, S. K. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102762 · Cited by 389
SwinBTS: A Method for 3D Multimodal Brain Tumor Segmentation Using Swin Transformer
Jiang, Y., Zhang, Y., Lin, X., Dong, J., Cheng, T., & Liang, J. (2022) · Brain Sciences · doi:10.3390/brainsci12060797 · Cited by 264
Metadata
- Authors
- Huang, X., Deng, Z., Li, D., Yuan, X., & Fu, Y.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2023-05
- Volume
- 42
- Issue
- 5
- Pages
- 1484-1494
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W4312568229
- Cited by
- 594
- References
- 43
- FWCI
- 34.16
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.