Efficient ConvNet for real-time semantic segmentation
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Semantic segmentation is a task that covers most of the perception needs of intelligent vehicles in an unified way. ConvNets excel at this task, as they can be trained end-to-end to accurately classify multiple object categories in an image at the pixel level. However, current approaches normally involve complex architectures that are expensive in terms of computational resources and are not feasible for ITS applications. In this paper, we propose a deep architecture that is able to run in real-time while providing accurate semantic segmentation. The core of our ConvNet is a novel layer that uses residual connections and factorized convolutions in order to remain highly efficient while still retaining remarkable performance. Our network is able to run at 83 FPS in a single Titan X, and at more than 7 FPS in a Jetson TX1 (embedded GPU). A comprehensive set of experiments demonstrates that our system, trained from scratch on the challenging Cityscapes dataset, achieves a classification performance that is among the state of the art, while being orders of magnitude faster to compute than other architectures that achieve top precision. This makes our model an ideal approach for scene understanding in intelligent vehicles applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“We use the Cityscapes dataset [20], a recent dataset of urban scenes that has been widely adopted in”
MethodMulti-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 2×
“Very recent works have achieved top performance by adapting Residual Networks (ResNets) [12] into the segmentation task, and combining them with dilated convolutions [13], which allow exponential expansion of the receptive field without loss of resolution or coverage.”
BackgroundMethodZagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 3×
“In the training phase, we also include Batch-Normalization [18] to accelerate convergence, and Dropout [19] as a regularization measure, although we triplicate its probability (0.3 in contrast to 0.1 used in ENet), as this yielded better results in our architecture.”
MethodInfluentialXNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 1×
“Other works like [4] (Deeplab) have proposed to refine the coarse output by using CRFs, and works like [11] (CRFasRNN) proposed to integrate them inside the convolutional architecture.”
MethodENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826 · cited 5×
“On the other side, some works have focused on efficiency by proposing architectures that can reach real-time segmentation [9][10], but this is normally at the expense of accuracy (e.”
BackgroundMethodResultInfluentialEfficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
Semantic Image Segmentation via Deep Parsing Network
Liu, Z., Li, X., Luo, P., Loy, C.-C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.162 · Cited by 675
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
Ghiasi, G., & Fowlkes, C. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_32 · Cited by 502
Research works printed but not linked: 14
- treml, 2016, Speeding up semantic segmentation for autonomous driving, MLITS NIPS Workshop
- alvarez, 2016, Decomposeme Simplifying convnets for end-to-end learning
- ioffe, 2015, Batch Normalization Accelerating Deep Network Training by Reducing Internal Covariate Shift
- 2014, Semantic image segmentation with deep convolutional nets and fully connected crfs
- badrinarayanan, 2015, Segnet A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling
- lin, 2016, Refinenet Multi-path refinement networks with identity mappings for high-resolution semantic segmentation
- 2016, Deeplab Semantic image segmentation with deep convolutional nets atrous convolution and fully connected crfs
- pohlen, 2016, Full-resolution residual networks for semantic segmentation in street scenes
- 10.1109/CVPR.2015.7298965 · DOI · Paper page
- romera, 2016, Can we unify monocular detectors for autonomous driving by using the pixel-wise semantic segmentation of cnns?
- collobert, 2011, Torch 7: A matlab-like environment for machine learning, Bigl eam NIPS Workshop
- papandreou, 2015, Weakly-and semi-supervised learning of a dcnn for semantic image segmentation
- uhrig, 2016, Pixel-level encoding and depth layering for instance-level semantic labeling
- 10.1007/978-3-319-45886-1_6 · DOI
Cited by
ICNet for Real-Time Semantic Segmentation on High-Resolution Images
Zhao, H., Qi, X., Shen, X., Shi, J., & Jia, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_25 · Cited by 1,709
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation
Romera, E., Alvarez, J. M., Bergasa, L. M., & Arroyo, R. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2750080 · Cited by 1,484
Self-citationCGNet: A Light-Weight Context Guided Network for Semantic Segmentation
Wu, T., Tang, S., Zhang, R., Cao, J., & Zhang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3042065 · Cited by 859
Structured Knowledge Distillation for Semantic Segmentation
Liu, Y., Chen, K., Liu, C., Qin, Z., Luo, Z., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00271 · Cited by 681
Deep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261
Metadata
- Authors
- Romera, E., Alvarez, J. M., Bergasa, L. M., & Arroyo, R.
- Venue
- 2017 IEEE Intelligent Vehicles Symposium (IV)
- Publisher
- IEEE
- Published
- 2017-06
- Pages
- 1789-1794
- ISSN
- 1931-0587 (ISSN-L)
- OpenAlex
- W2740036778
- Cited by
- 125
- References
- 12
- FWCI
- 5.10
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.