The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Abstract
Semantic segmentation is a fundamental task in computer vision, and it has various applications in fields such as robotic sensing, video surveillance, and autonomous driving. A major research topic in urban road semantic segmentation is the proper integration and use of cross-modal information for fusion. Here, we attempt to leverage inherent multimodal information and acquire graded features to develop a novel multilabel-learning network for RGB-thermal urban scene semantic segmentation. Specifically, we propose a strategy for graded-feature extraction to split multilevel features into junior, intermediate, and senior levels. Then, we integrate RGB and thermal modalities with two distinct fusion modules, namely a shallow feature fusion module and deep feature fusion module for junior and senior features. Finally, we use multilabel supervision to optimize the network in terms of semantic, binary, and boundary characteristics. Experimental results confirm that the proposed architecture, the graded-feature multilabel-learning network, outperforms state-of-the-art methods for urban scene semantic segmentation, and it can be generalized to depth data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“| GMNet [112] | ResNet50 [16] | 57.3 |”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [112]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_20 · Cited by 2,968
Understanding Convolution for Semantic Segmentation
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., & Cottrell, G. W. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00163 · Cited by 2,184
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826
Berman, M., Triki, A. R., & Blaschko, M. B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00464 · Cited by 1,163
High-Resolution Representations for Labeling Pixels and RegionsPreprint
Sun, K., Zhao, Y. S., Jiang, B., Cheng, T., Xiao, B., Liu, D., Mu, Y., Wang, X., Liu, W., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04514 · Cited by 941
Learning a Discriminative Feature Network for Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00199 · Cited by 909
FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture
Hazirbas, C., Ma, L., Domokos, C., & Cremers, D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54181-5_14 · Cited by 902
CGNet: A Light-Weight Context Guided Network for Semantic Segmentation
Wu, T., Tang, S., Zhang, R., Cao, J., & Zhang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3042065 · Cited by 859
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes
Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., & Harada, T. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206396 · Cited by 693
Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
Gupta, S., Arbeláez, P., & Malik, J. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.79 · Cited by 603
Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
Pohlen, T., Hermans, A., Mathias, M., & Leibe, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.353 · Cited by 600
Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes
Sun, Y., Zuo, W., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2904733 · Cited by 546
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection
Piao, Y., Ji, W., Li, J., Zhang, M., & Lu, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00735 · Cited by 526
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803025 · Cited by 500
Chen, X., Lin, K.-Y., Wang, J., Wu, W., Qian, C., Li, H., & Zeng, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_33 · Cited by 488
Detect Globally, Refine Locally: A Novel Approach to Saliency Detection
Wang, T., Zhang, L., Wang, S., Lu, H., Yang, G., Ruan, X., & Borji, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00330 · Cited by 479
ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3077058 · Cited by 340
Depth-Aware CNN for RGB-D Segmentation
Wang, W., & Neumann, U. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_9 · Cited by 314
RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation
Lee, S., Park, S., & Hong, K.-S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.533 · Cited by 294
PST900: RGB-Thermal Calibration, Dataset and Segmentation Network
Shivakumar, S. S., Rodrigues, N., Zhou, A., Miller, I. D., Kumar, V., & Taylor, C. J. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196831 · Cited by 273
FuseSeg: Semantic segmentation of urban scenes based on RGB and thermal data fusion
Sun, Y., Zuo, W., Yun, P., Wang, H., & Liu, M. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.2993143 · Cited by 270
RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic SegmentationPreprint
Jiang, J., Zheng, L., Luo, F., & Zhang, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01054 · Cited by 259
MFFENet: Multiscale Feature Fusion and Enhancement Network For RGB–Thermal Urban Road Scene Parsing
Zhou, W., Lin, X., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3086618 · Cited by 253
Zhou, W., Zhu, Y., Lei, J., Wan, J., & Yu, L. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3077767 · Cited by 229
Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
Wang, Q., Gao, J., & Li, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2910667 · Cited by 224
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105484 · Cited by 156
Semantic Segmentation With Context Encoding and Multi-Path Decoding
Ding, H., Jiang, X., Shuai, B., Liu, A. Q., & Wang, G. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2962685 · Cited by 154
TSNet: Three-Stream Self-Attention Network for RGB-D Indoor Semantic Segmentation
Zhou, W., Yuan, J., Lei, J., & Luo, T. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2999462 · Cited by 146
Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction
Zhou, W., Lv, Y., Lei, J., & Yu, L. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2957386 · Cited by 144
Non-Local Aggregation for RGB-D Semantic Segmentation
Zhang, G., Xue, J., Xie, P., Yang, S., & Wang, G. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3066071 · Cited by 103
Reinforcement Cutting-Agent Learning for Video Object Segmentation
Han, J., Yang, L., Zhang, D., Chang, X., & Liang, X. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00946 · Cited by 95
RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic SegmentationPreprint
Deng, L., Yang, M. Z., Li, T., He, Y., & Wang, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00135 · Cited by 92
SCN: Switchable Context Network for Semantic Segmentation of RGB-D Images
Lin, D., Zhang, R., Ji, Y., Li, P., & Huang, H. F. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2885062 · Cited by 86
Salient Object Detection in Stereoscopic 3D Images Using a Deep Convolutional Residual Autoencoder
Zhou, W., Wu, J., Lei, J., Hwang, J., & Yu, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025166 · Cited by 78
EfficientFCN: Holistically-Guided Decoding for Semantic Segmentation
Liu, J., He, J., Zhang, J., Ren, J., & Li, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_1 · Cited by 69
Multiscale multilevel context and multimodal fusion for RGB-D salient object detection
Wu, J., Zhou, W., Luo, T., Yu, L., & Lei, J. (2020) · Signal Processing · doi:10.1016/j.sigpro.2020.107766 · Cited by 68
Variational Context-Deformable ConvNets for Indoor Scene Parsing
Xiong, Z., Yuan, Y., Guo, N., & Wang, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00405 · Cited by 67
Pixel-Wise Attentional Gating for Scene Parsing
Kong, S., & Fowlkes, C. C. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00114 · Cited by 52
Chen, X., Lin, K.-Y., Wang, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09183 · Cited by 48
CEGFNet: Common Extraction and Gate Fusion Network for Scene Parsing of Remote Sensing Images
Zhou, W., Jianhui, J., Lei, J., & Hwang, J. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3109626 · Cited by 48
TMFNet: Three-Input Multilevel Fusion Network for Detecting Salient Objects in RGB-D Images
Zhou, W., Pan, S., Lei, J., & Yu, L. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3097393 · Cited by 37
Deep Binocular Fixation Prediction Using a Hierarchical Multimodal Fusion Network
Zhou, W., Liu, W., Lei, J., Luo, T., & Yu, L. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3051010 · Cited by 34
FRFB: Integrate Receptive Field Block Into Feature Fusion Net for Single Shot Multibox Detector
Zhu, Y. M., Mu, J., Pu, H., & Baiyi, S. (2018) · International Conference on Semantics, Knowledge and Grid (SKG) · doi:10.1109/skg.2018.00032 · Cited by 8
RGBD Semantic Segmentation Based on Global Convolutional Network
Gao, X., Yu, J., & Li, J. (2019) · International Conference on Robotics, Control and Automation (ICRCA) · doi:10.1145/3351180.3351182 · Cited by 4
Salient object detection via a linear feedback control system
Huo, S., Zhou, Y., & Kung, S. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8297085 · Cited by 1
Cited by
Sorted by citations
RIME: A physics-based optimization
Su, H., Zhao, D., Heidari, A. A., Liu, L., Zhang, X., Mafarja, M. M., & Chen, H. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.02.010 · Cited by 1,135
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
Tang, L., Zhang, H., Xu, H., & Ma, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101870 · Cited by 418
Liu, J., Liu, Z., Wu, G., Ma, L., Liu, R., Zhong, W., Luo, Z., & Fan, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00745 · Cited by 346
LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images
Zhou, W., Zhu, Y., Lei, J., Yang, R., & Yu, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3242775 · Cited by 304
Metadata
- Authors
- Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N.
- Affiliations
- Zhejiang University of Science and Technology (CN)
- Zhejiang University (CN)
- University of Washington (US)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2021
- Volume
- 30
- Pages
- 7790-7802
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3198062544
- Cited by
- 344
- References
- 56
- Citations by year
- 2026: 43 · 2025: 90 · 2024: 57 · 2023: 76 · 2022: 70 · 2021: 8
- FWCI
- 18.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (61502429, 61972357)
- Zhejiang Provincial Natural Science Foundation of China (LY18F020012)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Keywords
- RGB-thermal fusion, thermal infrared imagery, multilabel learning, cross-modal fusion, shallow and deep feature fusion, boundary supervision, road scene understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
