Image segmentation using deep learning: A survey
Open accessAbstract
Image segmentation is a key task in computer vision and image processing with important applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among others, and numerous segmentation algorithms are found in the literature. Against this backdrop, the broad success of deep learning (DL) has prompted the development of new image segmentation approaches leveraging DL models. We provide a comprehensive review of this recent literature, covering the spectrum of pioneering efforts in semantic and instance segmentation, including convolutional pixel-labeling networks, encoder-decoder architectures, multiscale and pyramid-based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the relationships, strengths, and challenges of these DL-based segmentation models, examine the widely used datasets, compare performances, and discuss promising research directions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior methods for object pose and segmentation typically operate on static inputs and do not allow the agent to interact with the scene.
“However, these methods typically operate on static inputs and do not allow the agent to interact with the scene.”
From Object-Centric Scene Representations Using Active Inference (2024) · cites this paper as Minaee et al., 2021
The Introduction describes segmentation as partitioning images or video frames, supporting the characterization of the surveyed methods as operating on visual inputs. It does not establish that prior object-pose methods typically use static inputs, or say that these methods prevent an agent from interacting with the scene. Section 3.11 also describes an interactive segmentation method, though it does not address agent–scene interaction. (LLM, conf=0.78)
Experimental — accuracy not yet measuredZhou et al. presented U-Net++, a new, broader neural net topology for image processing, to take categorization further.
“Zhou et al. presented U-Net ++ , a new, broader neural net topology for image processing, to take categorization further and farther [68], [70], [71].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [70]
Section 3.3.2 says Zhou et al. developed a nested U-Net architecture, and the References identify their work as U-Net++ for medical image segmentation. The survey does not say that it is a broader topology or that it advances categorization beyond existing models. (LLM, conf=0.82)
Experimental — accuracy not yet measuredSemantic segmentation is essential in visual understanding, and vision-based rover autonomy relies on semantic understanding.
“Semantic segmentation is essential in visual understanding [25], and vision-based rover autonomy relies on semantic understanding [13].”
From Semantic Terrain Segmentation in the Navigation Vision of Planetary Rovers—A Systematic Literature Review (2022) · cites this paper as [25]
Many current medical image segmentation models are task-specific, and their performance can degrade significantly when applied to new tasks or different types of imaging data.
“However, existing methods, often tailored to specific modalities or disease types, lack generalizability across the diverse spectrum of medical image segmentation tasks.”
From Segment anything in medical images (2024) · cites this paper as [6]
Section 3.3.2 describes models initially developed for medical or biomedical segmentation, while Section 6.3 discusses fine-tuning a generic model for a specific target application. Neither passage says that existing medical methods are often tailored to particular modalities or diseases, or that their performance degrades across tasks. Searches for generalizability, domain shift, and performance on different modalities found no passage establishing that claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 207 · Cited by 3,796
Research works printed but not linked: 5
- [16]16. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," in Advances in neural information processing systems, 2014, pp. 2672-2680.
- [63]63. K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
- [110]110. D. Marcos, D. Tuia, B. Kellenberger, L. Zhang, M. Bai, R. Liao, and R. Urtasun, "Learning deep structured active contours end- to-end," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8877-8885.
- [143]143. R. Mohan and A. Valada, "Efficientps: Efficient panoptic segmen- tation," arXiv preprint arXiv:2004.02307, 2020. · arXiv
- [167]167. M. De Deuge, A. Quadros, C. Hung, and B. Douillard, "Unsuper- vised feature learning for classification of outdoor 3d scans," in Australasian Conference on Robitics and Automation, vol. 2, 2013, p. 1.
- Authors
- Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D.
- Affiliations
- Snap (United States) (US)
- University of Waterloo (CA)
- Australian National University (AU)
- Universidad de Extremadura (ES)
- The University of Texas at Dallas (US)
- University of California, Los Angeles (US)
- ORCID
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021
- Volume
- 44
- Issue
- 7
- Pages
- 1-1
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2999607073
- Cited by
- 3,796
- References
- 207
- Citations by year
- 2026: 234 · 2025: 799 · 2024: 885 · 2023: 800 · 2022: 636 · 2021: 349 · 2020: 91 · 2019: 1
- FWCI
- 41.59
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- ANU Open Research (Australian National University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Microsoft Research
- Microsoft Research Asia
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- image segmentation, semantic segmentation, instance segmentation, deep learning, encoder-decoder architecture, multi-scale segmentation, visual attention, generative adversarial networks, recurrent neural networks, pixel labeling, segmentation datasets
- MeSH terms
- Deep Learning, Algorithms, Image Processing, Computer-Assisted, Robotics, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
