The DOI matches its Crossref record; 140 of the 140 references it cites were found in a registry or an index.
A survey on deep learning techniques for image and video semantic segmentation
Abstract
Abstract Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we formulate the semantic segmentation problem and define the terminology of this field as well as interesting background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and goals. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. We also devote a part of the paper to review common loss functions and error metrics for this problem. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Cited by
Deep semantic segmentation of natural and medical images: a review
Asgari Taghanaki, S., Abhishek, K., Cohen, J. P., Cohen-Adad, J., & Hamarneh, G. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09854-1 · Cited by 870
A review of deep learning methods for semantic segmentation of remote sensing imagery
Yuan, X., Shi, J., & Gu, L. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114417 · Cited by 766
An Analysis Of Convolutional Neural Networks For Image Classification
Sharma, N., Jain, V., & Mishra, A. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.05.198 · Cited by 701
(2020) · doi:10.1109/icra40945.2020.9196885 · Cited by 633
Autonomous Cars: Research Results, Issues, and Future Challenges
Hussain, R., & Zeadally, S. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2869360 · Cited by 631
“As a result of recent advances in the popular deep learning architectures such as AlexNet, VGG-16, GoogLeNet, and ResNet, the accuracy of scene understanding and semantic segmentation in autonomous cars [68] have improved.”
BackgroundSurvey on semantic segmentation using deep learning techniques
Lateef, F., & Ruichek, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.02.003 · Cited by 559
Application of deep learning algorithms in geotechnical engineering: a short critical review
Zhang, W., Li, H., Li, Y., Liu, H., Chen, Y., & Ding, X. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09967-1 · Cited by 538
A survey on instance segmentation: state of the art
Hafiz, A. M., & Bhat, G. M. (2020) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-020-00195-x · Cited by 498
Deep learning based image recognition for crack and leakage defects of metro shield tunnel
Huang, H.-W., Li, Q.-T., & Zhang, D.-M. (2018) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2018.04.002 · Cited by 492
Li, S., Zhao, X., & Zhou, G. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12433 · Cited by 490
Understanding Deep Learning Techniques for Image Segmentation
Ghosh, S., Das, N., Das, I., & Maulik, U. (2020) · ACM Computing Surveys · doi:10.1145/3329784 · Cited by 476
Deep learning approaches to biomedical image segmentation
Rizwan I Haque, I., & Neubert, J. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100297 · Cited by 456
Deep learning in ECG diagnosis: A review
Liu, X., Wang, H., Li, Z., & Qin, L. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107187 · Cited by 436
Tien Bui, D., Hoang, N.-D., Martínez-Álvarez, F., Ngo, P.-T. T., Hoa, P. V., Pham, T. D., Samui, P., & Costache, R. (2020) · Science of The Total Environment · doi:10.1016/j.scitotenv.2019.134413 · Cited by 398
Bouwmans, T., Javed, S., Sultana, M., & Jung, S. K. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.04.024 · Cited by 396
Pathology image analysis using segmentation deep learning algorithms
Wang, S., Yang, D. M., Rong, R., Zhan, X., & Xiao, G. (2019) · The American Journal of Pathology · doi:10.1016/j.ajpath.2019.05.007 · Cited by 382
Convolutional Neural Networks for Image-Based High-Throughput Plant Phenotyping: A Review
Jiang, Y., & Li, C. (2020) · Plant Phenomics · doi:10.34133/2020/4152816 · Cited by 380
Huang, M. Q., Ninić, J., & Zhang, Q. B. (2021) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2020.103677 · Cited by 370
Techniques and Challenges of Image Segmentation: A Review
Yu, Y., Wang, C., Fu, Q., Kou, R., Huang, F., Yang, B., Yang, T., & Gao, M. (2023) · Electronics · doi:10.3390/electronics12051199 · Cited by 370
Hoeser, T., & Kuenzer, C. (2020) · Remote Sensing · doi:10.3390/rs12101667 · Cited by 366
A review of deep learning used in the hyperspectral image analysis for agriculture
Wang, C., Liu, B., Liu, L., Zhu, Y., Hou, J., Liu, P., & Li, X. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10018-y · Cited by 362
Lung CT Image Segmentation Using Deep Neural Networks
Ait Skourt, B., El Hassani, A., & Majda, A. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.01.104 · Cited by 346
A graph-based CNN-LSTM stock price prediction algorithm with leading indicators
Wu, J. M.-T., Li, Z., Herencsar, N., Vo, B., & Lin, J. C.-W. (2023) · Multimedia Systems · doi:10.1007/s00530-021-00758-w · Cited by 329
A Review of Deep Learning-Based Semantic Segmentation for Point Cloud
Zhang, J., Zhao, X., Chen, Z., & Lu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2958671 · Cited by 306
Sa, I., Popović, M., Khanna, R., Chen, Z., Lottes, P., Liebisch, F., Nieto, J., Stachniss, C., Walter, A., & Siegwart, R. (2018) · Remote Sensing · doi:10.3390/rs10091423 · Cited by 305
Deep belief network based k-means cluster approach for short-term wind power forecasting
Wang, K., Qi, X., Liu, H., & Song, J. (2018) · Energy · doi:10.1016/j.energy.2018.09.118 · Cited by 304
Skin Lesion Segmentation in Dermoscopic Images with Combination of YOLO and GrabCut Algorithm
Ünver, H. M., & Ayan, E. (2019) · Diagnostics · doi:10.3390/diagnostics9030072 · Cited by 302
A Review on Deep Learning Techniques for Video Prediction
Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., Garcia-Rodriguez, J., & Argyros, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045007 · Cited by 301
Self-citationSemantic segmentation using Vision Transformers: A survey
Thisanke, H., Deshan, C., Chamith, K., Seneviratne, S., Vidanaarachchi, R., & Herath, D. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106669 · Cited by 288
Fully automatic wound segmentation with deep convolutional neural networks
Wang, C., Anisuzzaman, D. M., Williamson, V., Dhar, M. K., Rostami, B., Niezgoda, J., Gopalakrishnan, S., & Yu, Z. (2020) · Scientific Reports · doi:10.1038/s41598-020-78799-w · Cited by 279
Weakly Supervised Deep Learning for Segmentation of Remote Sensing Imagery
Wang, S., Chen, W., Xie, S. M., Azzari, G., & Lobell, D. B. (2020) · Remote Sensing · doi:10.3390/rs12020207 · Cited by 265
Artificial intelligence (AI) in augmented reality (AR)-assisted manufacturing applications: a review
Sahu, C. K., Young, C., & Rai, R. (2021) · International Journal of Production Research · doi:10.1080/00207543.2020.1859636 · Cited by 256
A Survey on Deep Learning Technique for Video Segmentation
Zhou, T., Porikli, F., Crandall, D. J., Van Gool, L., & Wang, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3225573 · Cited by 255
Human action recognition using attention based LSTM network with dilated CNN features
Muhammad, K., Mustaqeem, Ullah, A., Imran, A. S., Sajjad, M., Kiran, M. S., Sannino, G., & de Albuquerque, V. H. C. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.06.045 · Cited by 254
Che, E., Jung, J., & Olsen, M. J. (2019) · Sensors · doi:10.3390/s19040810 · Cited by 253
Wang, Y., Luo, Q., Xie, H., Li, Q., & Sun, G. (2022) · International Journal of Mechanical Sciences · doi:10.1016/j.ijmecsci.2022.107529 · Cited by 149
NuClick: A deep learning framework for interactive segmentation of microscopic images
Alemi Koohbanani, N., Jahanifar, M., Zamani Tajadin, N., & Rajpoot, N. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101771 · Cited by 147
Interpretation of deep learning in genomics and epigenomics
Talukder, A., Barham, C., Li, X., & Hu, H. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa177 · Cited by 142
A bird's-eye view of deep learning in bioimage analysis
Meijering, E. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.08.003 · Cited by 126
Zhou, Y., Wang, B., He, X., Cui, S., & Shao, L. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3045475 · Cited by 155
Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey of Datasets and Methods
Gao, B., Pan, Y., Li, C., Geng, S., & Zhao, H. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3076844 · Cited by 105
Technical and clinical overview of deep learning in radiology
Ueda, D., Shimazaki, A., & Miki, Y. (2019) · Japanese Journal of Radiology · doi:10.1007/s11604-018-0795-3 · Cited by 100
A comprehensive review of recent advances on deep vision systems
Abbas, Q., Ibrahim, M. E. A., & Jaffar, M. A. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-018-9633-3 · Cited by 99
BIM-Integrated Collaborative Robotics for Application in Building Construction and Maintenance
Follini, C., Magnago, V., Freitag, K., Terzer, M., Marcher, C., Riedl, M., Giusti, A., & Matt, D. T. (2020) · Robotics · doi:10.3390/robotics10010002 · Cited by 85
Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles
Gamper, J., & Rajpoot, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01628 · Cited by 75
Semantic and structural image segmentation for prosthetic vision
Sanchez-Garcia, M., Martinez-Cantin, R., & Guerrero, J. J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227677 · Cited by 62
Image-Based Feature Representation for Insider Threat Classification
Gayathri, R. G., Sajjanhar, A., & Xiang, Y. (2020) · Applied Sciences · doi:10.3390/app10144945 · Cited by 63
“Deep Convolutional Neural Networks (DCNNs), one of the most popular deep learning models, proved to be highly efficient in handling visual data, such as images and videos [4].”
BackgroundOn Artificial Intelligence and Deep Learning Within Medical Education
Carin, L. (2020) · Academic Medicine · doi:10.1097/acm.0000000000003630 · Cited by 56
Real-Time Traffic Analysis using Deep Learning Techniques and UAV based Video
Zhang, H., Liptrott, M., Bessis, N., & Cheng, J. (2019) · 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2019.8909879 · Cited by 52
[Retracted] An Automated Deep Learning Model for the Cerebellum Segmentation from Fetal Brain Images
Sreelakshmy, R., Titus, A., Sasirekha, N., Logashanmugam, E., Begam, R. B., Ramkumar, G., & Raju, R. (2022) · BioMed Research International · doi:10.1155/2022/8342767 · Cited by 51
“Transfer learning has been generally used with networks learned on ImageNet information and is superior to randomized weight initialization [15].”
RetractedModel-Agnostic Method for Thoracic Wall Segmentation in Fetal Ultrasound Videos
Shozu, K., Komatsu, M., Sakai, A., Komatsu, R., Dozen, A., Machino, H., Yasutomi, S., Arakaki, T., Asada, K., Kaneko, S., et al. (2020) · Biomolecules · doi:10.3390/biom10121691 · Cited by 38
Dai, Y., Zheng, T., Xue, C., & Zhou, L. (2022) · Remote Sensing · doi:10.3390/rs14246297 · Cited by 35
Karamitrou, A., Sturt, F., Bogiatzis, P., & Beresford-Jones, D. (2022) · Surface Topography: Metrology and Properties · doi:10.1088/2051-672x/ac9492 · Cited by 29
Randive, K., Mohan, R., & Sivakrishna, A. M. (2023) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2023.103434 · Cited by 27
Barrientos-Espillco, F., Gascó, E., López-González, C. I., Gómez-Silva, M. J., & Pajares, G. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110315 · Cited by 26
Analyzing Performance of YOLOx for Detecting Vehicles in Bad Weather Conditions
Ashraf, I., Hur, S., Kim, G., & Park, Y. (2024) · Sensors · doi:10.3390/s24020522 · Cited by 26
Bouget, D., Eijgelaar, R. S., Pedersen, A., Kommers, I., Ardon, H., Barkhof, F., Bello, L., Berger, M. S., Nibali, M. C., Furtner, J., et al. (2021) · Cancers · doi:10.3390/cancers13184674 · Cited by 19
Metadata
- Authors
- Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., & Garcia-Rodriguez, J.
- Venue
- Applied Soft Computing
- Publisher
- Elsevier BV
- Published
- 2018-09
- Volume
- 70
- Pages
- 41-65
- ISSN
- 1568-4946 (ISSN-L)
- OpenAlex
- W2804860796
- Cited by
- 1,490
- References
- 140
- FWCI
- 69.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- Funders
- Universidad de Alicante
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.