The DOI matches its Crossref record; 47 of the 49 references it cites were found in a registry or an index.
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis
Open accessAbstract
Analyzing scenes thoroughly is crucial for mobile robots acting in different environments. Semantic segmentation can enhance various subsequent tasks, such as (semantically assisted) person perception, (semantic) free space detection, (semantic) mapping, and (semantic) navigation. In this paper, we propose an efficient and robust RGB-D segmentation approach that can be optimized to a high degree using NVIDIA TensorRT and, thus, is well suited as a common initial processing step in a complex system for scene analysis on mobile robots. We show that RGB-D segmentation is superior to processing RGB images solely and that it can still be performed in real time if the network architecture is carefully designed. We evaluate our proposed Efficient Scene Analysis Network (ESANet) on the common indoor datasets NYUv2 and SUNRGB-D and show that we reach state-of-the-art performance while enabling faster inference. Furthermore, our evaluation on the outdoor dataset Cityscapes shows that our approach is suitable for other areas of application as well. Finally, instead of presenting benchmark results only, we also show qualitative results in one of our indoor application scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“| ESANet [102] | ResNet34 [16] | 80.09 |”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [102]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_20 · Cited by 2,968
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation
Romera, E., Álvarez, J. M., Bergasa, L. M., & Arroyo, R. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2750080 · Cited by 1,514
FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture
Hazirbas, C., Ma, L., Domokos, C., & Cremers, D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54181-5_14 · Cited by 902
ESPNetv2: A Light-Weight, Power Efficient, and General Purpose Convolutional Neural Network
Mehta, S., Rastegari, M., Shapiro, L., & Hajishirzi, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00941 · Cited by 551
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803025 · Cited by 500
Chen, X., Lin, K.-Y., Wang, J., Wu, W., Qian, C., Li, H., & Zeng, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_33 · Cited by 488
Lednet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation
Wang, Y., Zhou, Q., Liu, J., Xiong, J., Gao, G., Wu, X., & Latecki, L. J. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803154 · Cited by 455
Oršić, M., Krešo, I., Bevandić, P., & Šegvić, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01289 · Cited by 440
Depth-Aware CNN for RGB-D Segmentation
Wang, W., & Neumann, U. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_9 · Cited by 314
RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation
Lee, S., Park, S., & Hong, K.-S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.533 · Cited by 294
Self-supervised model adaptation for multimodal semantic segmentation
Valada, A., Mohan, R., & Burgard, W. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01188-y · Cited by 293
SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation?
McCormac, J., Handa, A., Leutenegger, S., & Davison, A. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.292 · Cited by 291
DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic SegmentationPreprint
Li, G., Yun, I., Kim, J., & Kim, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.11357 · Cited by 264
RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic SegmentationPreprint
Jiang, J., Zheng, L., Luo, F., & Zhang, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01054 · Cited by 259
Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation
Lo, S.-Y., Hang, H., Chan, S.-W., & Lin, J.-J. (2019) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3338533.3366558 · Cited by 259
Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation
Chen, L.-Z., Lin, Z., Wang, Z., Yang, Y., & Cheng, M.-M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3049332 · Cited by 177
Gross, H.-M., Boehme, H.-J., Schröter, C., Mueller, S., Koenig, A., Einhorn, E., Martin, C., Merten, M., & Bley, A. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354497 · Cited by 163
Self-citationGroß, H., Mueller, S., Schröter, C., Volkhardt, M., Scheidig, A., Debes, K., Richter, K., & Doering, N. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354230 · Cited by 121
Hung, S.-W., Lo, S.-Y., & Hang, H. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803360 · Cited by 60
Groß, H., Scheidig, A., Müller, S., Schütz, B., Fricke, C., & Meyer, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793693 · Cited by 48
Malleable 2.5D Convolution: Learning Receptive Fields Along the Depth-Axis for RGB-D Scene Parsing
Xing, Y., Wang, J., & Zeng, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58529-7_33 · Cited by 45
Xiang, L., Echtler, F., Kerl, C., Wiedemeyer, T., L., Hanyazou, Gordon, R., Facioni, F., Laborer2008, Wareham, R., et al. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.594510 · Cited by 37
2.5D Convolution for RGB-D Semantic Segmentation
Xing, Y., Wang, J., Chen, X., & Zeng, G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803757 · Cited by 35
Coupling Two-Stream RGB-D Semantic Segmentation Network by Idempotent Mappings
Xing, Y., Wang, J. B., Chen, X., & Zeng, G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803146 · Cited by 31
Generic 2D/3D SLAM with NDT maps for lifelong application
Einhorn, E., & Groß, H. (2013) · European Conference on Mobile Robots · doi:10.1109/ecmr.2013.6698849 · Cited by 30
Xiang, L., Echtler, F., Kerl, C., et al. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.50641 · Cited by 29
3D Neighborhood Convolution: Learning Depth-Aware Features for RGB-D and RGB Semantic Segmentation
Chen, Y., Mensink, T., & Gavves, E. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00028 · Cited by 29
Socially Compliant Human-Robot Interaction for Autonomous Scanning Tasks in Supermarket Environments
Lewandowski, B., Wengefeld, T., Müller, S., Jenny, M., Glende, S., Schröter, C., Bley, A., & Groß, H. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223568 · Cited by 29
Self-citationSmall Data, Big Decisions: Model Selection in the Small-Data RegimePreprint
Bornschein, J., Visin, F., & Osindero, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.12583 · Cited by 26
Efficient Multi-frequency Phase Unwrapping Using Kernel Density Estimation
Lawin, F. J., Forssén, P.-E., & Ovrén, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_11 · Cited by 25
Groß, H., Meyer, S., Scheidig, A., Eisenbach, M., Mueller, S., Trinh, T. Q., Wengefeld, T., Bley, A., Martin, C., & Fricke, C. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989124 · Cited by 25
Self-citationMulti-Modal Attention-based Fusion Model for Semantic Segmentation of RGB-Depth ImagesPreprint
Fooladgar, F., & Shohreh Kasaei, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11691 · Cited by 25
3D Geometry-Aware Semantic Labeling of Outdoor Street Scenes
Zhong, Y., Dai, Y., & Li, H. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545378 · Cited by 13
Multi-Task Deep Learning for Depth-based Person Perception in Mobile Robotics
Seichter, D., Lewandowski, B., Hochemer, D., Wengefeld, T., & Groß, H. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340870 · Cited by 10
Self-citationAutonomous Mobile Gait Training Robot for Orthopedic Rehabilitation in a Clinical Environment
Trinh, T. Q., Vorndran, A., Schuetz, B., Jaeschke, B., Mayfarth, A., Scheidig, A., & Groß, H. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223482 · Cited by 9
Cited by
Sorted by citations
Image segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
A Multilevel Multimodal Fusion Transformer for Remote Sensing Semantic Segmentation
Ma, X., Zhang, X., Pun, M.-O., & Liu, M. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3373033 · Cited by 312
Delivering Arbitrary-Modal Semantic Segmentation
Zhang, J., Liu, R., Shi, H., Yang, K., Reiß, S., Peng, K., Fu, H., Wang, K., & Stiefelhagen, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00116 · Cited by 215
A Crossmodal Multiscale Fusion Network for Semantic Segmentation of Remote Sensing Data
Ma, X., Zhang, X., & Pun, M.-O. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3165005 · Cited by 196
Metadata
- Authors
- Seichter, D., Kohler, M., Lewandowski, B., Wengefeld, T., & Gross, H.-M.
- Affiliations
- Technische Universität Ilmenau (DE)
- Technische Universität Ilmenau,Neuroinformatics and Cognitive Robotics Lab,Ilmenau,Germany,98693
- Venue
- 2021 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2021-05-30
- Pages
- 13525-13531
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W3100969133
- Cited by
- 330
- References
- 49
- Citations by year
- 2026: 36 · 2025: 76 · 2024: 92 · 2023: 70 · 2022: 40 · 2021: 16
- FWCI
- 17.83
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Ministry of Education
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- RGB-D semantic segmentation, mobile robots, real-time inference, semantic mapping, semantic navigation, person perception, NYUv2, cityscape
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
