The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentation
Open accessAbstract
Medical image segmentation has seen significant improvements with transformer models, which excel in grasping far-reaching contexts and global contextual information. However, the increasing computational demands of these models, proportional to the squared token count, limit their depth and resolution capabilities. Most current methods process D volumetric image data slice-by-slice (called pseudo 3D), missing crucial inter-slice information and thus reducing the model’s overall performance. To address these challenges, we introduce the concept of Deformable Large Kernel Attention (D-LKA Attention), a streamlined attention mechanism employing large convolution kernels to fully appreciate volumetric context. This mechanism operates within a receptive field akin to self-attention while sidestepping the computational overhead. Additionally, our proposed attention mechanism benefits from deformable convolutions to flexibly warp the sampling grid, enabling the model to adapt appropriately to diverse data patterns. We designed both 2D and 3D adaptations of the D-LKA Attention, with the latter excelling in cross-depth data understanding. Together, these components shape our novel hierarchical Vision Transformer architecture, the D-LKA Net. Evaluations of our model against leading methods on popular medical segmentation datasets (Synapse, NIH Pancreas, and Skin lesion) demonstrate its superior performance. Our code is publicly available at GitHub.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 1×
“Consequently, various strategies, including pyramid pooling techniques [27] and dilated convolutions [58] at varying scales, This WACV paper is the Open Access version, provided by the Computer Vision Foundation.”
MethodDeformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · cited 1×
“The concept of utilizing Large Kernels for medical image segmentation is extended by incorporating Deformable Convolutions [20].”
MethodSwin-Unet: Unet-like pure transformer for medical image segmentation
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., & Wang, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25066-8_9 · Cited by 5,640
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., & Zhou, Y. (2021) · arXiv · doi:10.48550/arxiv.2102.04306 · Cited by 4,206 · cited 4×
“Our evaluation follows the setting presented in [15, 46] for 2D and 3D versions.”
MethodInfluentialUNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H. R., & Xu, D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00181 · Cited by 3,586 · cited 1×
“TransUNet [15] is an early example of this approach, incorporating Transformer layers within the CNN bottleneck to model both local and global dependencies.”
MethodUNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H., & Xu, D. (2021) · arXiv · doi:10.48550/arxiv.2103.10504 · Cited by 3,580
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., et al. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363547 · Cited by 2,620
Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images
Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. R., & Xu, D. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-08999-2_22 · Cited by 2,273
Attention gated networks: learning to leverage salient regions in medical images
Schlemper, J., Oktay, O., Schaap, M., Heinrich, M., Kainz, B., Glocker, B., & Rueckert, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.01.012 · Cited by 2,014 · cited 1×
“Also, our approach segments the liver and stomach more accurately than UNETR++, which tends to over-segment these organs.”
BackgroundTransformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,296
Medical Transformer: Gated Axial-Attention for Medical Image Segmentation
Valanarasu, J. M. J., Oza, P., Hacihaliloglu, I., & Patel, V. M. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87193-2_4 · Cited by 1,248
Guo, M.-H., Lu, C.-Z., Liu, Z.-N., Cheng, M.-M., & Hu, S.-M. (2023) · Computational Visual Media · doi:10.1007/s41095-023-0364-2 · Cited by 1,073
(2022) · doi:10.1007/978-3-031-20053-3_27 · Cited by 1,060
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer
Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., & Li, J. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87193-2_11 · Cited by 1,045
Medical Image Segmentation Review: The Success of U-Net
Azad, R., Aghdam, E. K., Rauland, A., Jia, Y., Avval, A. H., Bozorgpour, A., Karimijafarbigloo, S., Cohen, J. P., Adeli, E., & Merhof, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435571 · Cited by 1,022
Self-citationPH2 - A dermoscopic image database for research and benchmarking
Mendonca, T., Ferreira, P. M., Marques, J. S., Marcal, A. R. S., & Rozeira, J. (2013) · 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6610779 · Cited by 1,009
Codella, N., Rotemberg, V., Tschandl, P., Celebi, M. E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.03368 · Cited by 984
DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
Roth, H. R., Lu, L., Farag, A., Shin, H.-C., Liu, J., Turkbey, E. B., & Summers, R. M. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24553-9_68 · Cited by 784
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation
Xie, Y., Zhang, J., Shen, C., & Xia, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.03024 · Cited by 654
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation
Xie, Y., Zhang, J., Shen, C., & Xia, Y. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87199-4_16 · Cited by 653
Review the state-of-the-art technologies of semantic segmentation based on deep learning
Mo, Y., Wu, Y., Yang, X., Liu, F., & Liao, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.005 · Cited by 632
MISSFormer: An Effective Transformer for 2D Medical Image Segmentation
Huang, X., Deng, Z., Li, D., Yuan, X., & Fu, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3230943 · Cited by 594
FAT-Net: Feature adaptive transformers for automated skin lesion segmentation
Wu, H., Chen, S., Chen, G., Wang, W., Lei, B., & Wen, Z. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102327 · Cited by 590
A Survey of Visual Transformers
Liu, Y., Zhang, Y., Wang, Y., Hou, F., Yuan, J., Tian, J., Zhang, Y., Shi, Z., Fan, J., & He, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3227717 · Cited by 549
HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation
Heidari, M., Kazerouni, A., Soltany, M., Azad, R., Aghdam, E. K., Cohen-Adad, J., & Merhof, D. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00614 · Cited by 530
Self-citationUNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation
Shaker, A., Maaz, M., Rasheed, H., Khan, S., Yang, M.-H., & Shahbaz Khan, F. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3398728 · Cited by 500
Advances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 485
Self-citationBi-Directional ConvLSTM U-Net with Densley Connected Convolutions
Azad, R., Asadi-Aghbolaghi, M., Fathy, M., & Escalera, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00052 · Cited by 474
Self-citationMedical Image Segmentation via Cascaded Attention Decoding
Rahman, M. M., & Marculescu, R. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00616 · Cited by 382
Dynamic Multi-Scale Filters for Semantic Segmentation
He, J., Deng, Z., & Qiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00366 · Cited by 320
Skin lesion segmentation via generative adversarial networks with dual discriminators
Lei, B., Xia, Z., Jiang, F., Jiang, X., Ge, Z., Xu, Y., Qin, J., Chen, S., Wang, T., & Wang, S. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101716 · Cited by 279
nnFormer: Interleaved Transformer for Volumetric Segmentation
Zhou, H.-Y., Guo, J., Zhang, Y., Yu, L., Wang, L., & Yu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03201 · Cited by 256
LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation
Xu, G., Zhang, X., He, X., & Wu, X. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-981-99-8543-2_4 · Cited by 228
Multi-scale Context Intertwining for Semantic Segmentation
Lin, D., Ji, Y., Lischinski, D., Cohen-Or, D., & Huang, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_37 · Cited by 173
D-former: a U-shaped Dilated Transformer for 3D medical image segmentation
Wu, Y., Liao, K., Chen, J., Wang, J., Chen, D. Z., Gao, H., & Wu, J. (2023) · Neural Computing and Applications · doi:10.1007/s00521-022-07859-1 · Cited by 172
TransDeepLab: Convolution-Free Transformer-Based DeepLab v3+ for Medical Image Segmentation
Azad, R., Heidari, M., Shariatnia, M., Aghdam, E. K., Karimijafarbigloo, S., Adeli, E., & Merhof, D. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-16919-9_9 · Cited by 169
Self-citationDAE-Former: Dual Attention-Guided Efficient Transformer for Medical Image Segmentation
Azad, R., Arimond, R., Aghdam, E. K., Kazerouni, A., & Merhof, D. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-46005-0_8 · Cited by 165
Self-citationContextual Attention Network: Transformer Meets U-Net
Azad, R., Heidari, M., Wu, Y., & Merhof, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21014-3_39 · Cited by 162
Self-citationHuang, H., Xie, S., Lin, L., Iwamoto, Y., Han, X., Chen, Y., & Tong, R. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/135 · Cited by 158
Capsules for biomedical image segmentation
LaLonde, R., Xu, Z., Irmakci, I., Jain, S., & Bağcı, U. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101889 · Cited by 130
Attention Swin U-Net: Cross-Contextual Attention Mechanism for Skin Lesion Segmentation
Aghdam, E. K., Azad, R., Zarvani, M., & Merhof, D. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230337 · Cited by 120
Self-citation
Cited by
Advances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 485
Self-citation
Metadata
- Authors
- Azad, R., Niggemeier, L., Hüttemann, M., Kazerouni, A., Aghdam, E. K., Velichko, Y., Bagci, U., & Merhof, D.
- Venue
- 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2024-01-03
- Pages
- 1276-1286
- OpenAlex
- W4394596865
- Cited by
- 254
- References
- 53
- FWCI
- 55.02
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.