Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
Open accessIntegrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A very common architectural choice is to adopt a multi-stream architecture for the encoder with a network branch processing each modality and additional network modules connecting the different branches that combine modality-specific features into fused ones and/or carry information across the branches.
“A very common architectural choice is to adopt a multi-stream architecture for the encoder with a network branch processing each modality (e.g., a two-stream architecture for RGB and depth) and additional network modules connecting the different branches that combine modality-specific features into fused ones and/or carry information across the branches [98,99,101].”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [101]
SA-Gate, using a ResNet101 backbone, achieves an mIoU of 81.7 on the Cityscapes dataset for 2.5D semantic segmentation.
“| SA-Gate [101] | ResNet101 [16] | 81.7 |”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [101]
Cites
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853 · see citing works
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Dual Attention Network for Scene Segmentation
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00326 · Cited by 7,351 · see citing works
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
Residual Attention Network for Image Classification
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.683 · Cited by 4,073 · see citing works
Self-citationLi, X., Wang, W., Hu, X., & Yang, J. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00060 · Cited by 3,416 · see citing works
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045 · see citing works
DenseASPP for Semantic Segmentation in Street Scenes
Yang, M., Yu, K., Zhang, C., Li, Z., & Yang, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00388 · Cited by 1,674 · see citing works
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_23 · Cited by 1,528 · see citing works
Learning a Discriminative Feature Network for Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00199 · Cited by 923 · see citing works
Self-citationGated-SCNN: Gated Shape CNNs for Semantic Segmentation
Takikawa, T., Acuna, D., Jampani, V., & Fidler, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00533 · Cited by 870 · see citing works
Xu, D., Ouyang, W., Wang, X., & Sebe, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00077 · Cited by 541 · see citing works
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803025 · Cited by 500 · see citing works
3D Graph Neural Networks for RGBD Semantic Segmentation
Qi, X., Liao, R., Jia, J., Fidler, S., & Urtasun, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.556 · Cited by 482 · see citing works
Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation
Zhang, Z., Cui, Z., Xu, C., Yan, Y., Sebe, N., & Yang, J. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00423 · Cited by 339 · see citing works
ACFNet: Attentional Class Feature Network for Semantic Segmentation
Zhang, F., Chen, Y., Li, Z., Hong, Z., Liu, J., Ma, F., Han, J., & Ding, E. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00690 · Cited by 324 · see citing works
Dynamic Multi-Scale Filters for Semantic Segmentation
He, J., Deng, Z., & Qiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00366 · Cited by 320 · see citing works
Depth-Aware CNN for RGB-D Segmentation
Wang, W., & Neumann, U. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_9 · Cited by 314 · see citing works
RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation
Lee, S., Park, S., & Hong, K.-S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.533 · Cited by 294 · see citing works
Boundary-Aware Feature Propagation for Scene Segmentation
Ding, H., Jiang, X., Liu, A. Q., Magnenat‐Thalmann, N., & Wang, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00692 · Cited by 290 · see citing works
Locality-Sensitive Deconvolution Networks with Gated Fusion for RGB-D Indoor Semantic Segmentation
Cheng, Y., Cai, R., Li, Z., Zhao, X., & Huang, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.161 · Cited by 242 · see citing works
FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic SegmentationPreprint
Wu, H., Zhang, J., Huang, K., Liang, K., & Yu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.11816 · Cited by 242 · see citing works
Choi, S., Kim, J. T., & Choo, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00939 · Cited by 197 · see citing works
LSTM-CF: Unifying Context Modeling and Fusion with LSTMs for RGB-D Scene Labeling
Li, Z., Gan, Y., Liang, X., Yu, Y., Cheng, H., & Lin, L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_34 · Cited by 177 · see citing works
Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks
Wang, J., Wang, Z., Tao, D., See, S. C. W., & Wang, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_40 · Cited by 169 · see citing works
Self-citationCascaded Feature Network for Semantic Segmentation of RGB-D Images
Lin, D., Chen, G., Cohen‐Or, D., Heng, P., & Huang, H. F. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.147 · Cited by 155 · see citing works
Adaptive Context Network for Scene Parsing
Fu, J., Liu, J., Wang, Y., Li, Y., Bao, Y., Tang, J., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00685 · Cited by 155 · see citing works
3D Sketch-Aware Semantic Scene Completion via Semi-Supervised Structure Prior
Chen, X., Lin, K.-Y., Qian, C., Zeng, G., & Li, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00425 · Cited by 133 · see citing works
Self-citationSPGNet: Semantic Prediction Guidance for Scene Parsing
Cheng, B., Chen, L.-C., Wei, Y., Zhu, Y., Huang, Z., Xiong, J., Huang, T. S., Hwu, W., Shi, H., & Uiuc, U. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00532 · Cited by 130 · see citing works
STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling
He, Y., Chiu, W.-C., Keuper, M., & Fritz, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.757 · Cited by 127 · see citing works
Recurrent Scene Parsing with Perspective Understanding in the Loop
Kong, S., & Fowlkes, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00106 · Cited by 114 · see citing works
Hung, S.-W., Lo, S.-Y., & Hang, H. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803360 · Cited by 60 · see citing works
RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic SegmentationPreprint
Deng, L., Yang, M. Z., Li, T., He, Y., & Wang, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00135 · Cited by 57 · see citing works
SPGNet: Semantic Prediction Guidance for Scene ParsingPreprint
Cheng, B., Chen, L.-C., Wei, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09798 · Cited by 35 · see citing works
2.5D Convolution for RGB-D Semantic Segmentation
Xing, Y., Wang, J., Chen, X., & Zeng, G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803757 · Cited by 35 · see citing works
Self-citationCoupling Two-Stream RGB-D Semantic Segmentation Network by Idempotent Mappings
Xing, Y., Wang, J. B., Chen, X., & Zeng, G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803146 · Cited by 31 · see citing works
Self-citation3D Neighborhood Convolution: Learning Depth-Aware Features for RGB-D and RGB Semantic Segmentation
Chen, Y., Mensink, T., & Gavves, E. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00028 · Cited by 29 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731 · see citing works
GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3109518 · Cited by 344 · see citing works
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis
Seichter, D., Kohler, M., Lewandowski, B., Wengefeld, T., & Gross, H.-M. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561675 · Cited by 330 · see citing works
Zhang, Q., Zhao, S., Luo, Y., Zhang, D., Huang, N., & Han, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00266 · Cited by 225 · see citing works
MobileSal: Extremely Efficient RGB-D Salient Object Detection
Wu, Y.-H., Liu, Y., Xu, J., Bian, J.-W., Gu, Y.-C., & Cheng, M.-M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3134684 · Cited by 176 · see citing works
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation
Reza, M. K., Prater-Bennette, A., & Asif, M. S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3476487 · Cited by 61 · see citing works
“In recent years, MML has achieved remarkable success in a wide range of applications, including image segmentation [6, 55, 70], captioning [71, 65], classification [15, 46], sentiment analysis [50, 28], and autonomous driving [61, 45].”
BackgroundMethodRizzoli, G., Barbato, F., & Zanuttigh, P. (2022) · Technologies · doi:10.3390/technologies10040090 · Cited by 58 · see citing works
Decoupling Common and Unique Representations for Multimodal Self-supervised Learning
Wang, Y., Albrecht, C. M., Braham, N. A. A., Liu, C., Xiong, Z., & Zhu, X. X. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73397-0_17 · Cited by 49 · see citing works
Yi (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.056 · Cited by 49 · see citing works
A Unified Multimodal De- and Re-Coupling Framework for RGB-D Motion Recognition
Zhou, B., Wang, P., Wan, J., Liang, Y., & Wang, F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3274783 · Cited by 38 · see citing works
- Authors
- Chen, X., Lin, K.-Y., Wang, J., Wu, W., Qian, C., Li, H., & Zeng, G.
- Affiliations
- Peking University (CN)
- Chinese University of Hong Kong (HK)
- Corresponding author
- Chen, Xiaokang
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2020
- Pages
- 561-577
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W3108601100
- Cited by
- 488
- References
- 43
- Citations by year
- 2026: 56 · 2025: 123 · 2024: 127 · 2023: 90 · 2022: 60 · 2021: 32
- FWCI
- 14.03
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61375022, 61403005, 61632003)
- SenseTime Group
- National Key Research and Development Program of China (2016QY02D0304, 2017YFB1002601)
- Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology (2018IRS11)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- RGB-D semantic segmentation, cross-modal fusion, depth information, depth noise, feature recalibration, gating mechanism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
