UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
Open accessThe DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Abstract
The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations: (1) their optimal depth is apriori unknown, requiring extensive architecture search or inefficient ensemble of models of varying depths; and (2) their skip connections impose an unnecessarily restrictive fusion scheme, forcing aggregation only at the same-scale feature maps of the encoder and decoder sub-networks. To overcome these two limitations, we propose UNet++, a new neural architecture for semantic and instance segmentation, by (1) alleviating the unknown network depth with an efficient ensemble of U-Nets of varying depths, which partially share an encoder and co-learn simultaneously using deep supervision; (2) redesigning skip connections to aggregate features of varying semantic scales at the decoder sub-networks, leading to a highly flexible feature fusion scheme; and (3) devising a pruning scheme to accelerate the inference speed of UNet++. We have evaluated UNet++ using six different medical image segmentation datasets, covering multiple imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and electron microscopy (EM), and demonstrating that (1) UNet++ consistently outperforms the baseline models for the task of semantic segmentation across different datasets and backbone architectures; (2) UNet++ enhances segmentation quality of varying-size objects-an improvement over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design) outperforms the original Mask R-CNN for the task of instance segmentation; and (4) pruned UNet++ models achieve significant speedup while showing only modest performance degradation. Our implementation and pre-trained models are available at https://github.com/MrGiovanni/UNetPlusPlus.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 4×
“We choose U-Net (or V-Net for 3D) because it is a common performance baseline for image segmentation.”
BackgroundMethodInfluential
Cited by
Chen, J., Mei, J., Li, X., Lu, Y., Yu, Q., Wei, Q., Luo, X., Xie, Y., Adeli, E., Wang, Y., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103280 · Cited by 1,365
TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation
Zhang, Y., Liu, H., & Hu, Q. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87193-2_2 · Cited by 1,312
Transformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,296
SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images
Fang, S., Li, K., Shao, J., & Li, Z. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2021.3056416 · Cited by 1,182
Recent advances and clinical applications of deep learning in medical image analysis
Chen, X., Wang, X., Zhang, K., Fung, K.-M., Thai, T. C., Moore, K., Mannel, R. S., Liu, H., Zheng, B., & Qiu, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102444 · Cited by 1,080
Dense Nested Attention Network for Infrared Small Target Detection
Li, B., Xiao, C., Wang, L., Wang, Y., Lin, Z., Li, M., An, W., & Guo, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3199107 · Cited by 1,054
Medical Image Segmentation Review: The Success of U-Net
Azad, R., Aghdam, E. K., Rauland, A., Jia, Y., Avval, A. H., Bozorgpour, A., Karimijafarbigloo, S., Cohen, J. P., Adeli, E., & Merhof, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435571 · Cited by 1,022
U-Shape Transformer for Underwater Image Enhancement
Peng, L., Zhu, C., & Bian, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3276332 · Cited by 912
Medical image segmentation using deep learning: A survey
Wang, R., Lei, T., Cui, R., Zhang, B., Meng, H., & Nandi, A. K. (2022) · IET Image Processing · doi:10.1049/ipr2.12419 · Cited by 892
DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation
Jha, D., Riegler, M. A., Johansen, D., Halvorsen, P., & Johansen, H. D. (2020) · 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) · doi:10.1109/cbms49503.2020.00111 · Cited by 819
TransMorph: Transformer for unsupervised medical image registration
Chen, J., Frey, E. C., He, Y., Segars, W. P., Li, Y., & Du, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102615 · Cited by 804
Deep learning in computer vision: A critical review of emerging techniques and application scenarios
Chai, J., Zeng, H., Li, A., & Ngai, E. W. T. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100134 · Cited by 754
Artificial intelligence for the metaverse: A survey
Huynh-The, T., Pham, Q.-V., Pham, X.-Q., Nguyen, T. T., Han, Z., & Kim, D.-S. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105581 · Cited by 710
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation
Xie, Y., Zhang, J., Shen, C., & Xia, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.03024 · Cited by 654
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation
Xie, Y., Zhang, J., Shen, C., & Xia, Y. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87199-4_16 · Cited by 653
Ranjbarzadeh, R., Bagherian Kasgari, A., Jafarzadeh Ghoushchi, S., Anari, S., Naseri, M., & Bendechache, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-90428-8 · Cited by 634
Review the state-of-the-art technologies of semantic segmentation based on deep learning
Mo, Y., Wu, Y., Yang, X., Liu, F., & Liao, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.005 · Cited by 632
Zhu, Z., He, X., Qi, G., Li, Y., Cong, B., & Liu, Y. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.10.022 · Cited by 592
An effective CNN and Transformer complementary network for medical image segmentation
Yuan, F., Zhang, Z., & Fang, Z. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109228 · Cited by 539
MA-Net: A Multi-Scale Attention Network for Liver and Tumor Segmentation
Fan, T., Wang, G., Li, Y., & Wang, H. (2020) · IEEE Access · doi:10.1109/access.2020.3025372 · Cited by 490
Deep learning-based structural health monitoring
Cha, Y.-J., Ali, R., Lewis, J., & Büyükӧztürk, O. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105328 · Cited by 418
Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency
Luo, X., Wang, G., Liao, W., Chen, J., Song, T., Chen, Y., Zhang, S., Metaxas, D. N., & Zhang, S. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102517 · Cited by 409
JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation
Wu, Y.-H., Gao, S.-H., Mei, J., Xu, J., Fan, D.-P., Zhang, R.-G., & Cheng, M.-M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3058783 · Cited by 403
Deep learning based brain tumor segmentation: a survey
Liu, Z., Tong, L., Chen, L., Jiang, Z., Zhou, F., Zhang, Q., Zhang, X., Jin, Y., & Zhou, H. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00815-5 · Cited by 402
H2Former: An Efficient Hierarchical Hybrid Transformer for Medical Image Segmentation
He, A., Wang, K., Li, T., Du, C., Xia, S., & Fu, H. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3264513 · Cited by 399
AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images
Chen, G., Li, L., Dai, Y., Zhang, J., & Yap, M. H. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3226268 · Cited by 395
Sharp U-Net: Depthwise convolutional network for biomedical image segmentation
Zunair, H., & Ben Hamza, A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104699 · Cited by 394
VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images
Sekuboyina, A., Husseini, M. E., Bayat, A., Löffler, M., Liebl, H., Li, H., Tetteh, G., Kukačka, J., Payer, C., Štern, D., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102166 · Cited by 372
Transfer learning techniques for medical image analysis: A review
Kora, P., Ooi, C. P., Faust, O., Raghavendra, U., Gudigar, A., Chan, W. Y., Meenakshi, K., Swaraja, K., Plawiak, P., & Rajendra Acharya, U. (2022) · Biocybernetics and Biomedical Engineering · doi:10.1016/j.bbe.2021.11.004 · Cited by 367
Cross-level Feature Aggregation Network for Polyp Segmentation
Zhou, T., Zhou, Y., He, K., Gong, C., Yang, J., Fu, H., & Shen, D. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109555 · Cited by 347
Jiao, R., Zhang, Y., Ding, L., Xue, B., Zhang, J., Cai, R., & Jin, C. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107840 · Cited by 345
Channel prior convolutional attention for medical image segmentation
Huang, H., Chen, Z., Zou, Y., Lu, M., Chen, C., Song, Y., Zhang, H., & Yan, F. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108784 · Cited by 342
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
Liu, J., Zhang, Y., Chen, J.-N., Xiao, J., Lu, Y., Landman, B. A., Yuan, Y., Yuille, A., Tang, Y., & Zhou, Z. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01934 · Cited by 333
Self-citationDeep learning for medical image segmentation: State-of-the-art advancements and challenges
Rayed, M. E., Islam, S. M. S., Niha, S. I., Jim, J. R., Kabir, M. M., & Mridha, M. F. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101504 · Cited by 332
Maxwell, A. E., Warner, T. A., & Guillén, L. A. (2021) · Remote Sensing · doi:10.3390/rs13132450 · Cited by 327
MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation
Srivastava, A., Jha, D., Chanda, S., Pal, U., Johansen, H., Johansen, D., Riegler, M., Ali, S., & Halvorsen, P. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3138024 · Cited by 318
Efficient attention-based deep encoder and decoder for automatic crack segmentation
Kang, D. H., & Cha, Y.-J. (2022) · Structural Health Monitoring · doi:10.1177/14759217211053776 · Cited by 305
Deep Gradient Learning for Efficient Camouflaged Object Detection
Ji, G.-P., Fan, D.-P., Chou, Y.-C., Dai, D., Liniger, A., & Van Gool, L. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1365-9 · Cited by 284
Zhou, C., & Ling, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3336053 · Cited by 284
Deep learning based synthesis of MRI, CT and PET: Review and analysis
Dayarathna, S., Islam, K. T., Uribe, S., Yang, G., Hayat, M., & Chen, Z. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103046 · Cited by 265
Camouflaged Object Detection via Context-Aware Cross-Level Fusion
Chen, G., Liu, S.-J., Sun, Y.-J., Ji, G.-P., Wu, Y.-F., & Zhou, T. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3178173 · Cited by 264
KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation
Valanarasu, J. M. J., Sindagi, V. A., Hacihaliloglu, I., & Patel, V. M. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3130469 · Cited by 255
Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopy
Yeung, M., Sala, E., Schönlieb, C.-B., & Rundo, L. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104815 · Cited by 186
Deep learning for automated size and shape analysis of nanoparticles in scanning electron microscopy
Bals, J., & Epple, M. (2023) · RSC Advances · doi:10.1039/d2ra07812k · Cited by 93
Liu, G., Ding, Q., Luo, H., Sha, M., Li, X., & Ju, M. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106194 · Cited by 81
Temporomandibular joint segmentation in MRI images using deep learning
Li, M., Punithakumar, K., Major, P. W., Le, L. H., Nguyen, K.-C. T., Pacheco-Pereira, C., Kaipatur, N. R., Nebbe, B., Jaremko, J. L., & Almeida, F. T. (2022) · Journal of Dentistry · doi:10.1016/j.jdent.2022.104345 · Cited by 55
Wang, Y., Tang, P., Zhou, Y., Shen, W., Fishman, E. K., & Yuille, A. L. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3060066 · Cited by 34
DIPPAS: a deep image prior PRNU anonymization scheme
Picetti, F., Mandelli, S., Bestagini, P., Lipari, V., & Tubaro, S. (2022) · EURASIP Journal on Information Security · doi:10.1186/s13635-022-00128-7 · Cited by 26
“trained according to the standard deep learning paradigm, it proves very effective for multidimensional signal processing tasks such as denoising [30], [31], interpolation [32], [33], segmentation [34], [35], inpainting [36], and domain-specific post-processing operators [37], [38].”
MethodGong, E. J., Bang, C. S., Lee, J. J., Jeong, H. M., Baik, G. H., Jeong, J. H., Dick, S., & Lee, G. H. (2023) · Journal of Medical Internet Research · doi:10.2196/50448 · Cited by 14
WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs
Yang, D., Hu, L., Tian, Y., Li, Z., Kelly, C., Yang, B., Yang, C., & Zou, Y. (2024) · arXiv · doi:10.48550/arxiv.2403.07944 · Cited by 5
Papagiannakis, A., Yu, Q., Govers, S. K., Lin, W.-H., Wingreen, N. S., & Jacobs-Wagner, C. (2025) · doi:10.7554/elife.104276.1 · Cited by 4
Wali, S., Khan, M. I., Samina, S., & Raza Sialvi, M. A. (2026) · 2026 11th International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc70586.2026.11686729
Metadata
- Authors
- Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2020-06
- Volume
- 39
- Issue
- 6
- Pages
- 1856-1867
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2996290406
- Cited by
- 4,402
- References
- 70
- FWCI
- 106.81
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.