The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation
Abstract
In multi-class indoor semantic segmentation using RGB-D data, it has been shown that incorporating depth feature into RGB feature is helpful to improve segmentation accuracy. However, previous studies have not fully exploited the potentials of multi-modal feature fusion, e.g., simply concatenating RGB and depth features or averaging RGB and depth score maps. To learn the optimal fusion of multimodal features, this paper presents a novel network that extends the core idea of residual learning to RGB-D semantic segmentation. Our network effectively captures multilevel RGB-D CNN features by including multi-modal feature fusion blocks and multi-level feature refinement blocks. Feature fusion blocks learn residual RGB and depth features and their combinations to fully exploit the complementary characteristics of RGB and depth data. Feature refinement blocks learn the combination of fused features from multiple levels to enable high-resolution prediction. Our network can efficiently train discriminative multi-level features from each modality end-to-end by taking full advantage of skip-connections. Our comprehensive experiments demonstrate that the proposed architecture achieves the state-of-the-art accuracy on two challenging RGB-D indoor datasets, NYUDv2 and SUN RGB-D.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Microsoft Kinect Sensor and Its Effect
Zhang, Z. (2012) · IEEE Multimedia · doi:10.1109/mmul.2012.24 · Cited by 2,487
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 2×
“Several followup studies [45, 29, 1, 3, 28] proposed sophisticated methods to combine CRF framework into DCNNs.”
MethodSUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
Jegou, S., Drozdzal, M., Vazquez, D., Romero, A., & Bengio, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.156 · Cited by 1,736
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_23 · Cited by 1,528
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511
Colorization using optimization
Levin, A., Lischinski, D., & Weiss, Y. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015780 · Cited by 1,411
FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture
Hazirbas, C., Ma, L., Domokos, C., & Cremers, D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54181-5_14 · Cited by 902
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels
Xiao, J., Owens, A., & Torralba, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.458 · Cited by 803
Kendall, A., Badrinarayanan, V., & Cipolla, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.57 · Cited by 784
Semantic Image Segmentation via Deep Parsing Network
Liu, Z., Li, X., Luo, P., Loy, C.-C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.162 · Cited by 675
Associative hierarchical CRFs for object class image segmentation
Ladicky, L., Russell, C., Kohli, P., & Torr, P. H. S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459248 · Cited by 596
Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
Gupta, S., Arbelaez, P., & Malik, J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.79 · Cited by 595
RGB-(D) scene labeling: Features and algorithms
Ren, X., Bo, L., & Fox, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247999 · Cited by 392
Indoor Semantic Segmentation using depth information
Couprie, C., Farabet, C., Najman, L. A., & LeCun, Y. (2013) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.48550/arxiv.1301.3572 · Cited by 372
Yao, J., Fidler, S., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247739 · Cited by 365
What, Where and How Many? Combining Object Detectors and CRFs
Ladický, Ľ., Sturgess, P., Alahari, K., Russell, C., & Torr, P. H. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_31 · Cited by 310
Kendall, A., Badrinarayanan, V., & Cipolla, R. (2015) · arXiv · doi:10.48550/arxiv.1511.02680 · Cited by 292
A Category-Level 3D Object Dataset: Putting the Kinect to Work
Janoch, A., Karayev, S., Jia, Y., Barron, J. T., Fritz, M., Saenko, K., & Darrell, T. (2013) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-1-4471-4640-7_8 · Cited by 254
ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
Visin, F., Kastner, K., Cho, K., Matteucci, M., Courville, A. C., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00393 · Cited by 218
Higher Order Conditional Random Fields in Deep Neural Networks
Arnab, A., Jayasumana, S., Zheng, S., & Torr, P. H. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_33 · Cited by 201
Fast, Exact and Multi-scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs
Chandra, S., & Kokkinos, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_25 · Cited by 175
LSTM-CF: Unifying Context Modeling and Fusion with LSTMs for RGB-D Scene Labeling
Li, Z., Gan, Y., Liang, X., Yu, Y., Cheng, H., & Lin, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_34 · Cited by 172
Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks
Wang, J., Wang, Z., Tao, D., See, S., & Wang, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_40 · Cited by 161
Exploring Context with Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2708714 · Cited by 141
Semantic Segmentation of RGBD Images with Mutex Constraints
Deng, Z., Todorović, S., & Latecki, L. J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.202 · Cited by 80
Geometry Driven Semantic Labeling of Indoor Scenes
Khan, S. H., Bennamoun, M., Sohel, F. A., & Togneri, R. B. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_44 · Cited by 67
Cited by
Image segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks
Audebert, N., Le Saux, B., & Lefèvre, S. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.11.011 · Cited by 655
A Brief Survey on Semantic Segmentation with Deep Learning
Hao, S., Zhou, Y., & Guo, Y. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.118 · Cited by 596
Survey on semantic segmentation using deep learning techniques
Lateef, F., & Ruichek, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.02.003 · Cited by 559
Metadata
- Authors
- Lee, S., Park, S.-J., & Hong, K.-S.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 4990-4999
- OpenAlex
- W2775906317
- Cited by
- 285
- References
- 48
- FWCI
- 5.99
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.