The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Locality-Sensitive Deconvolution Networks with Gated Fusion for RGB-D Indoor Semantic Segmentation
Abstract
This paper focuses on indoor semantic segmentation using RGB-D data. Although the commonly used deconvolution networks (DeconvNet) have achieved impressive results on this task, we find there is still room for improvements in two aspects. One is about the boundary segmentation. DeconvNet aggregates large context to predict the label of each pixel, inherently limiting the segmentation precision of object boundaries. The other is about RGB-D fusion. Recent state-of-the-art methods generally fuse RGB and depth networks with equal-weight score fusion, regardless of the varying contributions of the two modalities on delineating different categories in different scenes. To address the two problems, we first propose a locality-sensitive DeconvNet (LS-DeconvNet) to refine the boundary segmentation over each modality. LS-DeconvNet incorporates locally visual and geometric cues from the raw RGB-D data into each DeconvNet, which is able to learn to upsample the coarse convolutional maps with large context whilst recovering sharp object boundaries. Towards RGB-D fusion, we introduce a gated fusion layer to effectively combine the two LS-DeconvNets. This layer can learn to adjust the contributions of RGB and depth over each pixel for high-performance object recognition. Experiments on the large-scale SUN RGB-D dataset and the popular NYU-Depth v2 dataset show that our approach achieves new state-of-the-art results for RGB-D indoor semantic segmentation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 3×
“Inspired by recent CRF-RNN model [29], which leverages pixel-level cues such as intensity and location via conditional random fields (CRF) to refine label agreements of the large-context FCN maps, we try to benefit DeconvNet from pixel-level cues similarly but get rid of the complex training and inference of the CRF m…”
BackgroundMethodSUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
Cited by
Image segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
SO-Net: Self-Organizing Network for Point Cloud Analysis
Li, J., Chen, B. M., & Lee, G. H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00979 · Cited by 985
Ahn, J., & Kwak, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00523 · Cited by 872
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations
Ahn, J., Cho, S., & Kwak, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00231 · Cited by 613
A Brief Survey on Semantic Segmentation with Deep Learning
Hao, S., Zhou, Y., & Guo, Y. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.118 · Cited by 596
Illumination-aware faster R-CNN for robust multispectral pedestrian detection
Li, C., Song, D., Tong, R., & Tang, M. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2018.08.005 · Cited by 533
Li, Z., Zhao, Y., Han, J., Su, Y., Jiao, R., Wen, X., & Pei, D. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467075 · Cited by 464
Self-citationACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803025 · Cited by 449
CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion
Han, J., Chen, H., Liu, N., Yan, C., & Li, X. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2761775 · Cited by 423
Chen, X., Lin, K.-Y., Wang, J., Wu, W., Qian, C., Li, H., & Zeng, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_33 · Cited by 488
Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection
Chen, H., & Li, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00322 · Cited by 336
Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation
Zhang, Z., Cui, Z., Xu, C., Yan, Y., Sebe, N., & Yang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00423 · Cited by 331
Three-Stream Attention-Aware Network for RGB-D Salient Object Detection
Chen, H., & Li, Y. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2891104 · Cited by 324
Depth-Aware CNN for RGB-D Segmentation
Wang, W., & Neumann, U. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_9 · Cited by 284
Fast End-to-End Trainable Guided Filter
Wu, H., Zheng, S., Zhang, J., & Huang, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00197 · Cited by 278
Self-citationDeep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261
Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion
Wang, Y., Sun, F., Lu, M., & Yao, A. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413621 · Cited by 68
Hard pixel mining for depth privileged semantic segmentation
Gu, Z., Niu, L., Zhao, H., & Zhang, L. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3035231 · Cited by 41
Multimodal Attention Network for Continuous-Time Emotion Recognition Using Video and EEG Signals
Choi, D. Y., Kim, D.-H., & Song, B. C. (2020) · IEEE Access · doi:10.1109/access.2020.3036877 · Cited by 39
Metadata
- Authors
- Cheng, Y., Cai, R., Li, Z., Zhao, X., & Huang, K.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 1475-1483
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2609822318
- Cited by
- 229
- References
- 12
- FWCI
- 5.34
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.