The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Modality specific U-Net variants for biomedical image segmentation: a survey
Open accessAbstract
With the advent of advancements in deep learning approaches, such as deep convolution neural network, residual neural network, adversarial network; U-Net architectures are most widely utilized in biomedical image segmentation to address the automation in identification and detection of the target regions or sub-regions. In recent studies, U-Net based approaches have illustrated state-of-the-art performance in different applications for the development of computer-aided diagnosis systems for early diagnosis and treatment of diseases such as brain tumor, lung cancer, alzheimer, breast cancer, etc., using various modalities. This article contributes in presenting the success of these approaches by describing the U-Net framework, followed by the comprehensive analysis of the U-Net variants by performing (1) inter-modality, and (2) intra-modality categorization to establish better insights into the associated challenges and solutions. Besides, this article also highlights the contribution of U-Net based frameworks in the ongoing pandemic, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) also known as COVID-19. Finally, the strengths and similarities of these U-Net variants are analysed along with the challenges involved in biomedical image segmentation to uncover promising future research directions in this area.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“[39] 2018 Probabilistic U-Net CT - X - - [40], [41] ED U-Net with conditional variational autoencoder Tong et al.”
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 3×
“A feature or intermediate level fusion approach is adopted, where two independent sub-segmentation networks are constructed to extract dedicated feature maps from each modality and are later fused with the cascaded convolution blocks that follow the V-Net model scheme to finally compute the tumor segmentation mask.”
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665 · cited 2×
“The authors achieved significant improvements over several U-Net variants such as residual U-Net [123], BCDU-Net [90], R2U-Net and attention R2U-Net [56], etc.”
BackgroundMethod3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 2×
“The advantage of using dilated convolutions is that it tends to increase the receptive field without changing the number of training parameters (Yu and Koltun 2016).”
BackgroundDeformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · cited 6×
“[103] integrated 3D U-Net model with deformable convolutions [131] for cardiac MRI segmentation.”
BackgroundMethodInfluentialAttention U-Net: Learning Where to Look for the Pancreas
Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1804.03999 · Cited by 5,458
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · cited 4×
“DeepLab is another family of segmentation models that have improved over the years, where each phase of enhancement is named as DeepLabv1 (Chen et al.”
BackgroundInfluentialImage segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277 · cited 1×
“[62] 2018 UNet++ Multimodality - X X - [40], [63], [64], [65] ND Nested U-Net model”
A Survey on Deep Transfer Learning
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., & Liu, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01424-7_27 · Cited by 3,165
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“To better interpret these models various visualization based approaches are proposed such as local interpretable model-agnostic explanations (LIME) [179], shapley additive explanation (SHAP) [180], partial dependence plots (PDP) [181], anchor [182], etc.”
MethodSelf-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036
Graph Cuts and Efficient N-D Image Segmentation
Boykov, Y., & Funka-Lea, G. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-7934-5 · Cited by 1,910 · cited 1×
“(2019c) proposed a liver CT image segmentation framework named GIU-Net, inspired by the supervised interactive segmentation approach named, graph cut (Boykov and Funka-Lea 2006).”
BackgroundShi, F., Wang, J., Shi, J., Wu, Z., Wang, Q., Tang, Z., He, K., Shi, Y., & Shen, D. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.2987975 · Cited by 1,404
Ultrasound image segmentation: a survey
Noble, J. A., & Boukerroui, D. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2006.877092 · Cited by 1,157 · cited 2×
“There are various clinical applications of ultrasound imaging including cardiology, breast cancer, prostate, and other diseases (Noble and Boukerroui 2006) which can be assisted with automated segmentation.”
BackgroundDeep learning for brain MRI segmentation: state of the art and future directions
Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D. L., & Erickson, B. J. (2017) · Journal of Digital Imaging · doi:10.1007/s10278-017-9983-4 · Cited by 1,129 · cited 1×
“For instance, the performance of brain tumor segmentation models is affected by large variations in brain tumors location, size, shape and heterogeneity (image uniformity, contrast uptake and texture) (Akkus et al. 2017; Wadhwa et al. 2019).”
BackgroundSmith-Bindman, R., Miglioretti, D. L., Johnson, E., Lee, C., Feigelson, H. S., Flynn, M., Greenlee, R. T., Kruger, R. L., Hornbrook, M. C., Roblin, D., et al. (2012) · JAMA · doi:10.1001/jama.2012.5960 · Cited by 898 · cited 1×
“[27] observed the dramatic increase in the utilization of diagnostic imaging in the”
BackgroundA Survey of Model Compression and Acceleration for Deep Neural Networks
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2017) · arXiv · doi:10.48550/arxiv.1710.09282 · Cited by 857
Kendall, A., Badrinarayanan, V., & Cipolla, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.57 · Cited by 784
DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
Roth, H. R., Lu, L., Farag, A., Shin, H.-C., Liu, J., Turkbey, E. B., & Summers, R. M. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24553-9_68 · Cited by 784
Christ, P. F., Elshaer, M. E. A., Ettlinger, F., Tatavarty, S., Bickel, M., Bilic, P., Rempfler, M., Armbruster, M., Hofmann, F., D’Anastasi, M., et al. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_48 · Cited by 625
A probabilistic u-net for segmentation of ambiguous images
Kohl, S. A. A., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J. R., Maier-Hein, K. H., Eslami, S. M. A., Rezende, D. J., & Ronneberger, O. (2018) · arXiv · doi:10.48550/arxiv.1806.05034 · Cited by 439 · cited 2×
“[39] 2018 Probabilistic U-Net CT - X - - [40], [41] ED U-Net with conditional variational autoencoder Tong et al.”
BackgroundKendall, A., Badrinarayanan, V., & Cipolla, R. (2015) · arXiv · doi:10.48550/arxiv.1511.02680 · Cited by 292
Automatic multiorgan segmentation in thorax CT images using U‐net‐ GAN
Dong, X., Lei, Y., Wang, T., Thomas, M., Tang, L., Curran, W. J., Liu, T., & Yang, X. (2019) · Medical Physics · doi:10.1002/mp.13458 · Cited by 275
Hiasa, Y., Otake, Y., Takao, M., Ogawa, T., Sugano, N., & Sato, Y. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2940555 · Cited by 157 · cited 4×
“(2018) Improved U-Net CT - X X - BU Mini-residual connections within encoder-decoder phases Janssens et al. (2018) Two stage U-Net model CT - X X - EU 3D FCN LocalizationNet followed by SegmentationNet Kumar et al. (2018) U-SegNet MRI X - X - BU Integration of skip connections with SegNet Kermi et al. (2018) Residual…”
MethodInfluentialCOVID-19 Chest CT Image Segmentation -- A Deep Convolutional Neural Network Solution
Yan, Q., Wang, B., Gong, D., Luo, C., Zhao, W., Shen, J., Shi, Q., Jin, S., Zhang, L., & You, Z. (2020) · arXiv · doi:10.48550/arxiv.2004.10987 · Cited by 109 · cited 1×
“In another U-Net based implementation, a feature variation block is introduced in the COVID-SegNet model (Yan et al., 2020) to better segment the COVID-19 infected regions by highlighting the boundaries and diverse infected regions.”
MethodPHiSeg: Capturing Uncertainty in Medical Image Segmentation
Baumgartner, C. F., Tezcan, K. C., Chaitanya, K., Hötker, A. M., Muehlematter, U. J., Schawkat, K., Becker, A. S., Donati, O., & Konukoglu, E. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32245-8_14 · Cited by 34
Cited by
Deep Learning for Medical Image-Based Cancer Diagnosis
Jiang, X., Hu, Z., Wang, S., & Zhang, Y. (2023) · Cancers · doi:10.3390/cancers15143608 · Cited by 345
Metadata
- Authors
- Punn, N. S., & Agarwal, S.
- Venue
- Artificial Intelligence Review
- Published
- 2022-03-01
- Volume
- 55
- Issue
- 7
- Pages
- 5845-5889
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W3180604690
- Cited by
- 246
- References
- 55
- FWCI
- 16.19
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.