The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
MedSegDiff-V2: Diffusion-Based Medical Image Segmentation with Transformer
Open accessAbstract
The Diffusion Probabilistic Model (DPM) has recently gained popularity in the field of computer vision, thanks to its image generation applications, such as Imagen, Latent Diffusion Models, and Stable Diffusion, which have demonstrated impressive capabilities and sparked much discussion within the community. Recent investigations have further unveiled the utility of DPM in the domain of medical image analysis, as underscored by the commendable performance exhibited by the medical image segmentation model across various tasks. Although these models were originally underpinned by a UNet architecture, there exists a potential avenue for enhancing their performance through the integration of vision transformer mechanisms. However, we discovered that simply combining these two models resulted in subpar performance. To effectively integrate these two cutting-edge techniques for the Medical image segmentation, we propose a novel Transformer-based Diffusion framework, called MedSegDiff-V2. We verify its effectiveness on 20 medical image segmentation tasks with different image modalities. Through comprehensive evaluation, our approach demonstrates superiority over prior state-of-the-art (SOTA) methodologies. Code is released at https://github.com/KidsWithTokens/MedSegDiff.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Swin-Unet: Unet-like pure transformer for medical image segmentation
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., & Wang, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25066-8_9 · Cited by 5,640
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., & Zhou, Y. (2021) · arXiv · doi:10.48550/arxiv.2102.04306 · Cited by 4,206
UNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H. R., & Xu, D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00181 · Cited by 3,586
UNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H., & Xu, D. (2021) · arXiv · doi:10.48550/arxiv.2103.10504 · Cited by 3,580
Armato, S. G., McLennan, G., Bidaut, L., McNitt‐Gray, M. F., Meyer, C. R., Reeves, A. P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al. (2011) · Medical Physics · doi:10.1118/1.3528204 · Cited by 2,887
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.11487 · Cited by 2,528
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511
Warfield, S. K., Zou, K. H., & Wells, W. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.828354 · Cited by 2,072
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer
Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., & Li, J. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87193-2_11 · Cited by 1,045
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891
DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation
Lin, A., Chen, B., Xu, J., Zhang, Z., Lu, G., & Zhang, D. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3178991 · Cited by 882
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis
Tang, Y., Yang, D., Li, W., Roth, H. R., Landman, B., Xu, D., Nath, V., & Hatamizadeh, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02007 · Cited by 849
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Medical SAM adapter: Adapting segment anything model for medical image segmentation
Wu, J., Wang, Z., Hong, M., Ji, W., Fu, H., Xu, Y., Xu, M., & Jin, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103547 · Cited by 677
Self-citationFAT-Net: Feature adaptive transformers for automated skin lesion segmentation
Wu, H., Chen, S., Chen, G., Wang, W., Lei, B., & Wen, Z. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102327 · Cited by 590
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08934 · Cited by 524
A probabilistic u-net for segmentation of ambiguous images
Kohl, S. A. A., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J. R., Maier-Hein, K. H., Eslami, S. M. A., Rezende, D. J., & Ronneberger, O. (2018) · arXiv · doi:10.48550/arxiv.1806.05034 · Cited by 439
Intriguing Properties of Vision Transformers
Naseer, M., Ranasinghe, K., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2021) · arXiv · doi:10.48550/arxiv.2105.10497 · Cited by 431
AI-derived and Manually corrected segmentations for various IDC Collections
Van Oss, J., Murugesan, G. K., McCrumb, D., & Soni, R. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8345959 · Cited by 360
Med3D: Transfer Learning for 3D Medical Image Analysis
Chen, S., Ma, K., & Zheng, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.00625 · Cited by 317
An open access thyroid ultrasound image database
Pedraza, L., Vargas, C., Narváez, F., Durán, O., Muñoz, E., & Romero, E. (2015) · SPIE Proceedings · doi:10.1117/12.2073532 · Cited by 275
Multi-Task Learning For Thyroid Nodule Segmentation With Thyroid Region Prior
Gong, H., Chen, G., Wang, R., Xie, X., Mao, M., Yu, Y., Chen, F., & Li, G. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9434087 · Cited by 274
SwinBTS: A Method for 3D Multimodal Brain Tumor Segmentation Using Swin Transformer
Jiang, Y., Zhang, Y., Lin, X., Dong, J., Cheng, T., & Liang, J. (2022) · Brain Sciences · doi:10.3390/brainsci12060797 · Cited by 264
AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
Ji, Y., Bai, H., Ge, C., Yang, J., Zhu, Y., Zhang, R., Li, Z., Zhanng, L., Ma, W., Wan, X., et al. (2022) · neural information processing systems · doi:10.52202/068431-2661 · Cited by 264
Implicit Neural Representations with Periodic Activation Functions
Sitzmann, V., Martel, J., Bergman, A. W., Lindell, D. B., & Wetzstein, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09661 · Cited by 261
MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic ModelPreprint
Wu, J., Fu, R., Fang, H., Zhang, Y., Yang, Y., Xiong, H., Liu, H., & Xu, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.00611 · Cited by 231
Self-citationRobust optic disc and cup segmentation with deep learning for glaucoma detection
Yu, S., Xiao, D., Frost, S., & Kanagasingam, Y. (2019) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2019.02.005 · Cited by 223
Multi-Organ Segmentation Over Partially Labeled Datasets With Multi-Scale Feature Abstraction
Fang, X., & Yan, P. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3001036 · Cited by 216
Ambiguous Medical Image Segmentation Using Diffusion Models
Rahman, A., Valanarasu, J. M. J., Hacihaliloglu, I., & Patel, V. M. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01110 · Cited by 214
Boundary and Entropy-Driven Adversarial Learning for Fundus Image Segmentation
Wang, S., Yu, L., Kang Li, B., Yang, X. N., Fu, C., & Heng, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32239-7_12 · Cited by 209
SegDiff: Image Segmentation with Diffusion Probabilistic Models
Amit, T., Shaharbany, T., Nachmani, E., & Wolf, L. (2021) · arXiv · doi:10.48550/arxiv.2112.00390 · Cited by 179
Learning Calibrated Medical Image Segmentation via Multi-rater Agreement Modeling
Ji, W., Yu, S., Wu, J., Ma, K., Bian, C., Bi, Q., Li, J., Liu, H., Cheng, L., & Zheng, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01216 · Cited by 166
Self-citationDiffusion Adversarial Representation Learning for Self-supervised Vessel SegmentationPreprint
Kim, B., Oh, Y., & Ye, J. C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.14566 · Cited by 102
Ultrasonic thyroid nodule detection method based on U-Net network
Chu, C., Zheng, J., & Yong, Z. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105906 · Cited by 78
Accelerating Diffusion Models Via Pre-Segmentation Diffusion Sampling for Medical Image Segmentation
Guo, X., Yang, Y., Ye, C., Lu, S., Peng, B., Huang, H., Xiang, Y., & Ma, T. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230524 · Cited by 61
SeATrans: Learning Segmentation-Assisted Diagnosis Model via Transformer
Wu, J., Fang, H., Shang, F., Yang, D., Wang, Z., Gao, J., Yang, Y., & Xu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16434-7_65 · Cited by 26
Self-citationLeveraging Undiagnosed Data for Glaucoma Classification with Teacher-Student Learning
Wu, J., Yu, S., Shen, L., Ma, K., Fu, R., Liu, H., Di, X., & Zheng, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_71 · Cited by 25
Self-citationValidating automated eye disease screening AI algorithm in community and in-hospital scenarios
Han, R., Cheng, G., Zhang, B., Yang, J., Yuan, M., Yang, D., Wu, J., Liu, J., Zhao, C., Chen, Y., et al. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.944967 · Cited by 25
Self-citationA Multicenter Clinical Study of the Automated Fundus Screening Algorithm
Li, F., Pan, J., Yang, D., Wu, J., Ou, Y., Li, H., Huang, J. M., Xie, H., Ou, D., Wu, X., et al. (2022) · Translational Vision Science & Technology · doi:10.1167/tvst.11.7.22 · Cited by 25
Self-citationUniversal, transferable and targeted adversarial attacksPreprint
Wu, J., & Fu, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11332 · Cited by 17
Self-citationLearning Self-calibrated Optic Disc and Cup Segmentation from Multi-rater Annotations
Wu, J., Fang, H., Wang, Z., Yang, D., Yang, Y., Shang, F., Zhou, W., & Xu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16434-7_59 · Cited by 14
Self-citationOpinions Vary? Diagnosis First!
Wu, J., Fang, H., Yang, D., Wang, Z., Zhou, W., Shang, F., Yang, Y., & Xu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16434-7_58 · Cited by 10
Self-citationDiffusion Models for Implicit Image Segmentation Ensembles
Wolleb, J., Sandkühler, R., Bieder, F., Valmaggia, P., & Cattin, P. C. (2021) · arXiv · doi:10.48550/arxiv.2112.03145
Cited by
A survey of the vision transformers and their CNN-transformer based variants
Khan, A., Rauf, Z., Sohail, A., Khan, A. R., Asif, H., Asif, A., & Farooq, U. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10595-0 · Cited by 451
How to protect copyright data in optimization of large language models?
Chu, T., Song, Z., & Yang, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29741 · Cited by 16
Metadata
- Authors
- Wu, J., Ji, W., Fu, H., Xu, M., Jin, Y., & Xu, Y.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2024-03-24
- Volume
- 38
- Issue
- 6
- Pages
- 6030-6038
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4393154023
- Cited by
- 275
- References
- 49
- FWCI
- 55.89
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.