GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Abstract
Semantic segmentation is a fundamental task in computer vision, and it has various applications in fields such as robotic sensing, video surveillance, and autonomous driving. A major research topic in urban road semantic segmentation is the proper integration and use of cross-modal information for fusion. Here, we attempt to leverage inherent multimodal information and acquire graded features to develop a novel multilabel-learning network for RGB-thermal urban scene semantic segmentation. Specifically, we propose a strategy for graded-feature extraction to split multilevel features into junior, intermediate, and senior levels. Then, we integrate RGB and thermal modalities with two distinct fusion modules, namely a shallow feature fusion module and deep feature fusion module for junior and senior features. Finally, we use multilabel supervision to optimize the network in terms of semantic, binary, and boundary characteristics. Experimental results confirm that the proposed architecture, the graded-feature multilabel-learning network, outperforms state-of-the-art methods for urban scene semantic segmentation, and it can be generalized to depth data.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GMNet, using a ResNet50 backbone, achieves an mIoU of 57.3 on the MSSSD/MF dataset for RGB + Thermal semantic segmentation.
“| GMNet [112] | ResNet50 [16] | 57.3 |”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [112]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898 · see citing works
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984 · see citing works
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422 · see citing works
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045 · see citing works
BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_20 · Cited by 2,968 · see citing works
Understanding Convolution for Semantic Segmentation
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., & Cottrell, G. W. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00163 · Cited by 2,127 · see citing works
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826 · see citing works
Berman, M., Triki, A. R., & Blaschko, M. B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00464 · Cited by 1,114 · see citing works
Learning a Discriminative Feature Network for Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00199 · Cited by 923 · see citing works
FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture
Hazirbas, C., Ma, L., Domokos, C., & Cremers, D. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54181-5_14 · Cited by 902 · see citing works
CGNet: A Light-Weight Context Guided Network for Semantic Segmentation
Wu, T., Tang, S., Zhang, R., Cao, J., & Zhang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3042065 · Cited by 859 · see citing works
High-Resolution Representations for Labeling Pixels and RegionsPreprint
Sun, K., Zhao, Y. S., Jiang, B., Cheng, T., Xiao, B., Liu, D., Mu, Y., Wang, X., Liu, W., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04514 · Cited by 841 · see citing works
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes
Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., & Harada, T. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206396 · Cited by 693 · see citing works
Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
Gupta, S., Arbeláez, P., & Malik, J. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.79 · Cited by 601 · see citing works
Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
Pohlen, T., Hermans, A., Mathias, M., & Leibe, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.353 · Cited by 591 · see citing works
Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes
Sun, Y., Zuo, W., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2904733 · Cited by 546 · see citing works
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection
Piao, Y., Ji, W., Li, J., Zhang, M., & Lu, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00735 · Cited by 528 · see citing works
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803025 · Cited by 500 · see citing works
Chen, X., Lin, K.-Y., Wang, J., Wu, W., Qian, C., Li, H., & Zeng, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_33 · Cited by 488 · see citing works
Detect Globally, Refine Locally: A Novel Approach to Saliency Detection
Wang, T., Zhang, L., Wang, S., Lu, H., Yang, G., Ruan, X., & Borji, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00330 · Cited by 479 · see citing works
ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3077058 · Cited by 340 · see citing works
Depth-Aware CNN for RGB-D Segmentation
Wang, W., & Neumann, U. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_9 · Cited by 314 · see citing works
RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation
Lee, S., Park, S., & Hong, K.-S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.533 · Cited by 294 · see citing works
FuseSeg: Semantic segmentation of urban scenes based on RGB and thermal data fusion
Sun, Y., Zuo, W., Yun, P., Wang, H., & Liu, M. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.2993143 · Cited by 270 · see citing works
PST900: RGB-Thermal Calibration, Dataset and Segmentation Network
Shivakumar, S. S., Rodrigues, N., Zhou, A., Miller, I. D., Kumar, V., & Taylor, C. J. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196831 · Cited by 269 · see citing works
MFFENet: Multiscale Feature Fusion and Enhancement Network For RGB–Thermal Urban Road Scene Parsing
Zhou, W., Lin, X., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3086618 · Cited by 251 · see citing works
RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic SegmentationPreprint
Jiang, J., Zheng, L., Luo, F., & Zhang, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01054 · Cited by 233 · see citing works
Zhou, W., Zhu, Y., Lei, J., Wan, J., & Yu, L. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3077767 · Cited by 229 · see citing works
Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
Wang, Q., Gao, J., & Li, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2910667 · Cited by 224 · see citing works
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105484 · Cited by 156 · see citing works
Semantic Segmentation With Context Encoding and Multi-Path Decoding
Ding, H., Jiang, X., Shuai, B., Liu, A. Q., & Wang, G. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2962685 · Cited by 154 · see citing works
TSNet: Three-Stream Self-Attention Network for RGB-D Indoor Semantic Segmentation
Zhou, W., Yuan, J., Lei, J., & Luo, T. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2999462 · Cited by 146 · see citing works
Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction
Zhou, W., Lv, Y., Lei, J., & Yu, L. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2957386 · Cited by 143 · see citing works
Non-Local Aggregation for RGB-D Semantic Segmentation
Zhang, G., Xue, J., Xie, P., Yang, S., & Wang, G. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3066071 · Cited by 101 · see citing works
Reinforcement Cutting-Agent Learning for Video Object Segmentation
Han, J., Yang, L., Zhang, D., Chang, X., & Liang, X. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00946 · Cited by 95 · see citing works
SCN: Switchable Context Network for Semantic Segmentation of RGB-D Images
Lin, D., Zhang, R., Ji, Y., Li, P., & Huang, H. F. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2885062 · Cited by 87 · see citing works
Salient Object Detection in Stereoscopic 3D Images Using a Deep Convolutional Residual Autoencoder
Zhou, W., Wu, J., Lei, J., Hwang, J., & Yu, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025166 · Cited by 78 · see citing works
EfficientFCN: Holistically-Guided Decoding for Semantic Segmentation
Liu, J., He, J., Zhang, J., Ren, J., & Li, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_1 · Cited by 69 · see citing works
Variational Context-Deformable ConvNets for Indoor Scene Parsing
Xiong, Z., Yuan, Y., Guo, N., & Wang, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00405 · Cited by 67 · see citing works
Multiscale multilevel context and multimodal fusion for RGB-D salient object detection
Wu, J., Zhou, W., Luo, T., Yu, L., & Lei, J. (2020) · Signal Processing · doi:10.1016/j.sigpro.2020.107766 · Cited by 67 · see citing works
RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic SegmentationPreprint
Deng, L., Yang, M. Z., Li, T., He, Y., & Wang, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00135 · Cited by 57 · see citing works
Pixel-Wise Attentional Gating for Scene Parsing
Kong, S., & Fowlkes, C. C. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00114 · Cited by 54 · see citing works
Chen, X., Lin, K.-Y., Wang, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09183 · Cited by 48 · see citing works
CEGFNet: Common Extraction and Gate Fusion Network for Scene Parsing of Remote Sensing Images
Zhou, W., Jianhui, J., Lei, J., & Hwang, J. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3109626 · Cited by 48 · see citing works
TMFNet: Three-Input Multilevel Fusion Network for Detecting Salient Objects in RGB-D Images
Zhou, W., Pan, S., Lei, J., & Yu, L. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3097393 · Cited by 37 · see citing works
Deep Binocular Fixation Prediction Using a Hierarchical Multimodal Fusion Network
Zhou, W., Liu, W., Lei, J., Luo, T., & Yu, L. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3051010 · Cited by 34 · see citing works
FRFB: Integrate Receptive Field Block Into Feature Fusion Net for Single Shot Multibox Detector
Zhu, Y. M., Mu, J., Pu, H., & Baiyi, S. (2018) · International Conference on Semantics, Knowledge and Grid (SKG) · doi:10.1109/skg.2018.00032 · Cited by 8 · see citing works
RGBD Semantic Segmentation Based on Global Convolutional Network
Gao, X., Yu, J., & Li, J. (2019) · International Conference on Robotics, Control and Automation (ICRCA) · doi:10.1145/3351180.3351182 · Cited by 4 · see citing works
Salient object detection via a linear feedback control system
Huo, S., Zhou, Y., & Kung, S. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8297085 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
RIME: A physics-based optimization
Su, H., Zhao, D., Heidari, A. A., Liu, L., Zhang, X., Mafarja, M. M., & Chen, H. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.02.010 · Cited by 1,097 · see citing works
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731 · see citing works
Tang, L., Zhang, H., Xu, H., & Ma, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101870 · Cited by 396 · see citing works
Liu, J., Liu, Z., Wu, G., Ma, L., Liu, R., Zhong, W., Luo, Z., & Fan, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00745 · Cited by 346 · see citing works
LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images
Zhou, W., Zhu, Y., Lei, J., Yang, R., & Yu, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3242775 · Cited by 304 · see citing works
Dif-Fusion: Toward High Color Fidelity in Infrared and Visible Image Fusion With Diffusion Models
Yue, J., Fang, L., Xia, S., Deng, Y., & Ma, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3322046 · Cited by 304 · see citing works
WaveNet: Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection
Zhou, W., Sun, F., Jiang, Q., Cong, R., & Hwang, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3275538 · Cited by 231 · see citing works
Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption
Liu, J., Wu, G., Liu, Z., Wang, D., Jiang, Z., Ma, L., Zhong, W., Fan, X., & Liu, R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3521416 · Cited by 238 · see citing works
A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends
Younesi, A., Ansari, M., Fazli, M., Ejlali, A., Shafique, M., & Henkel, J. (2024) · IEEE Access · doi:10.1109/access.2024.3376441 · Cited by 228 · see citing works
Delivering Arbitrary-Modal Semantic Segmentation
Zhang, J., Liu, R., Shi, H., Yang, K., Reiß, S., Peng, K., Fu, H., Wang, K., & Stiefelhagen, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00116 · Cited by 202 · see citing works
RGB-T Semantic Segmentation With Location, Activation, and Sharpening
Li, G., Wang, Y., Liu, Z., Zhang, X., & Zeng, D. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3208833 · Cited by 196 · see citing works
Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks
Liang, M., Hu, J., Bao, C., Hua, F., Deng, F., & Lam, T. L. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3272269 · Cited by 179 · see citing works
Zhou, W., Dong, S., Lei, J., & Yu, L. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3164899 · Cited by 179 · see citing works
PGDENet: Progressive Guided Fusion and Depth Enhancement Network for RGB-D Indoor Scene Parsing
Zhou, W., Yang, E., Lei, J., Wan, J., & Yu, L. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3161852 · Cited by 168 · see citing works
Zhou, W., Zhang, H., Yan, W., & Lin, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3275314 · Cited by 151 · see citing works
Edge-Aware Guidance Fusion Network for RGB–Thermal Scene Parsing
Zhou, W., Dong, S., Xu, C., & Qian, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20269 · Cited by 146 · see citing works
Remote-Sensing Scene Classification via Multistage Self-Guided Separation Network
Wang, J., Li, W., Zhang, M., Tao, R., & Chanussot, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3295797 · Cited by 139 · see citing works
Context-Aware Interaction Network for RGB-T Semantic Segmentation
Lv, Y., Liu, Z., & Li, G. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3349072 · Cited by 138 · see citing works
Thermal Infrared Image Colorization for Nighttime Driving Scenes With Top-Down Guided Attention
Luo, F., Li, Y., Zeng, G., Peng, P., Wang, G., & Li, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3145476 · Cited by 115 · see citing works
FRNet: Feature Reconstruction Network for RGB-D Indoor Scene Parsing
Zhou, W., Yang, E., Lei, J., & Yu, L. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3174338 · Cited by 114 · see citing works
A Task-Guided, Implicitly-Searched and Meta-Initialized Deep Model for Image Fusion
Liu, R., Liu, Z., Liu, J., Fan, X., & Luo, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3382308 · Cited by 114 · see citing works
Zhou, W., Zhu, Y., Lei, J., Wan, J., & Yu, L. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3118043 · Cited by 110 · see citing works
SGFNet: Semantic-Guided Fusion Network for RGB-Thermal Semantic Segmentation
Wang, Y., Li, G., & Liu, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3281419 · Cited by 104 · see citing works
Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
Wan, Z., Zhang, P., Wang, Y., Yong, S., Stepputtis, S., Sycara, K. P., & Xie, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00176 · Cited by 114 · see citing works
Cross-Collaborative Fusion-Encoder Network for Robust RGB-Thermal Salient Object Detection
Liao, G., Gao, W., Li, G., Wang, J., & Kwong, S. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3184840 · Cited by 103 · see citing works
Mitigating Modality Discrepancies for RGB-T Semantic Segmentation
Zhao, S., Liu, Y., Jiao, Q., Zhang, Q., & Han, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3233089 · Cited by 102 · see citing works
EGFNet: Edge-Aware Guidance Fusion Network for RGB–Thermal Urban Scene Parsing
Dong, S., Zhou, W., Xu, C., & Yan, W. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3306368 · Cited by 107 · see citing works
DEFNet: Dual-Branch Enhanced Feature Fusion Network for RGB-T Crowd Counting
Zhou, W., Pan, Y. H., Lei, J., Ye, L., & Yu, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3203385 · Cited by 93 · see citing works
Zhou, W., Jianhui, J., Lei, J., & Yu, L. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3159032 · Cited by 94 · see citing works
RGB-T image analysis technology and application: A survey
Song, K., Zhao, Y., Huang, L., Yan, Y., & Meng, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105919 · Cited by 94 · see citing works
Jaafari, A., Panahi, M., Mafi-Gholami, D., Rahmati, O., Shahabi, H., Shirzadi, A., Lee, S., Bui, D. T., & Pradhan, B. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108254 · Cited by 86 · see citing works
An extended MARCOS method for MCGDM under 2-tuple linguistic q-rung picture fuzzy environment
Akram, M., Khan, A., Luqman, A., Senapati, T., & Pamucar, D. (2023) · Engineering applications of artificial intelligence · doi:10.1016/j.engappai.2023.105892 · Cited by 93 · see citing works
Greedy opposition-based learning for chimp optimization algorithm
Khishe, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10343-w · Cited by 76 · see citing works
Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview
Ren, W., Tang, Y. S., Sun, Q., Zhao, C., & Han, Q. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123207 · Cited by 78 · see citing works
Glass Segmentation With RGB-Thermal Image Pairs
Huo, D., Wang, J., Qian, Y., & Yang, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3256762 · Cited by 73 · see citing works
Embedded Control Gate Fusion and Attention Residual Learning for RGB–Thermal Urban Scene Parsing
Zhou, W., Lv, Y. H., Lei, J., & Yu, L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3242651 · Cited by 71 · see citing works
Vision-based food nutrition estimation via RGB-D fusion network
Shao, W., Min, W., Hou, S., Luo, M., Li, T., Zheng, Y., & Jiang, S. (2023) · Food Chemistry · doi:10.1016/j.foodchem.2023.136309 · Cited by 67 · see citing works
Zhou, W., Liu, C., Lei, J., Yu, L., & Luo, T. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.100 · Cited by 64 · see citing works
Pixel Difference Convolutional Network for RGB-D Semantic Segmentation
Yang, J., Bai, L., Sun, Y., Tian, C., Mao, M., & Wang, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3296162 · Cited by 65 · see citing works
CGINet: Cross-modality grade interaction network for RGB-T crowd counting
Pan, Y. F., Zhou, W., Qian, X., Mao, S., Yang, R., & Yu, L. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106885 · Cited by 60 · see citing works
Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding
Liu, Y., Li, C., Xu, S., & Han, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02393-8 · Cited by 59 · see citing works
Rizzoli, G., Barbato, F., & Zanuttigh, P. (2022) · Technologies · doi:10.3390/technologies10040090 · Cited by 58 · see citing works
Vinayaki, V. D., & Kalaiselvi, R. (2022) · Neural Processing Letters · doi:10.1007/s11063-021-10734-0 · Cited by 58 · see citing works
Xu, C. X., Li, Q., Jiang, X., Yu, D., & Zhou, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3216313 · Cited by 56 · see citing works
On the Role of Thermal Imaging in Automotive Applications: A Critical Review
Farooq, M. A., Shariff, W., O'Callaghan, D., Merla, A., & Corcoran, P. (2023) · IEEE Access · doi:10.1109/access.2023.3255110 · Cited by 58 · see citing works
CACFNet: Cross-Modal Attention Cascaded Fusion Network for RGB-T Urban Scene Parsing
Zhou, W., Dong, S., Fang, M., & Yu, L. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3314527 · Cited by 57 · see citing works
PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation
Liu, Z., Liu, J., Zhang, B., Ma, L., Fan, X., & Liu, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611928 · Cited by 56 · see citing works
Instance segmentation ship detection based on improved Yolov7 using complex background SAR images
Yasir, M., Zhan, L., Liu, S., Wan, J., Hossain, M. S., Colak, A. T. I., Liu, M., Islam, Q. U., Mehdi, S. R., & Yang, Q. (2023) · Frontiers in Marine Science · doi:10.3389/fmars.2023.1113669 · Cited by 54 · see citing works
DHFNet: dual-decoding hierarchical fusion network for RGB-thermal semantic segmentation
Cai, Y., Zhou, W., Zhang, L., Yu, L., & Luo, T. (2023) · The Visual Computer · doi:10.1007/s00371-023-02773-6 · Cited by 53 · see citing works
GCNet: Grid-like context-aware network for RGB-thermal semantic segmentation
Liu, J., Zhou, W., Cui, Y., Yu, L., & Luo, T. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.07.041 · Cited by 53 · see citing works
Semantic Segmentation for Thermal Images: A Comparative Survey
Kutuk, Z., & Algan, G. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00043 · Cited by 54 · see citing works
Yang, E., Zhou, W., Qian, X., Lei, J., & Yu, L. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106729 · Cited by 52 · see citing works
He, Q., Sun, X., Diao, W., Yan, Z., Yin, D., & Fu, K. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.08.010 · Cited by 51 · see citing works
RetractedCEGFNet: Common Extraction and Gate Fusion Network for Scene Parsing of Remote Sensing Images
Zhou, W., Jianhui, J., Lei, J., & Hwang, J. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3109626 · Cited by 48 · see citing works
Saravana Balaji, B., Balaji, P., Munshi, A. M., ALMUKADI, W. S., Prabhu, T. N., Venkatachalam, K., & Abouhawwash, M. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1259 · Cited by 54 · see citing works
Infrared and Visible Image Fusion via Multiscale Receptive Field Amplification Fusion Network
Ji, C., Zhou, W., Lei, J., & Ye, L. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3270759 · Cited by 46 · see citing works
MGCNet: Multilevel Gated Collaborative Network for RGB-D Semantic Segmentation of Indoor Scene
Yang, E., Zhou, W., Qian, X., & Yu, L. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3229594 · Cited by 45 · see citing works
CSANet: Contour and Semantic Feature Alignment Fusion Network for Rail Surface Defect Detection
Yang, J., Zhou, W., Wu, R., & Fang, M. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3299218 · Cited by 44 · see citing works
DCANet: Differential convolution attention network for RGB-D semantic segmentation
Bai, L., Yang, J., Tian, C., Sun, Y., Mao, M., Xu, Y., & Wei-rong, X. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111379 · Cited by 44 · see citing works
Swin Transformer Embedding Dual-Stream for Semantic Segmentation of Remote Sensing Imagery
Zhou, X., Zhou, L., Gong, S., Zhong, S., Yan, W. Q., & Huang, Y. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3326967 · Cited by 43 · see citing works
Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning
Dong, S., Feng, Y., Yang, Q., Huang, Y., Liu, D., & Fan, H. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801872 · Cited by 46 · see citing works
A Novel Highway Routing Protocol in Vehicular Ad Hoc Networks Using VMaSC‐LTE and DBA‐MAC Protocols
Khezri, E., Zeinali, E., & Sargolzaey, H. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/1680507 · Cited by 42 · see citing works
An Improved CycleGAN-Based Model for Low-Light Image Enhancement
Tang, G., Ni, J., Chen, Y., Cao, W., & Yang, S. X. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3296167 · Cited by 42 · see citing works
MMSFormer: Multimodal Transformer for Material and Semantic Segmentation
Reza, M. K., Prater-Bennette, A., & Asif, M. S. (2024) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2024.3389812 · Cited by 42 · see citing works
Global contextually guided lightweight network for RGB-thermal urban scene understanding
Gong, T., Zhou, W., Qian, X., Lei, J., & Yu, L. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105510 · Cited by 43 · see citing works
Multi-source collaborative enhanced for remote sensing images semantic segmentation
Zhao, J., Zhang, D., Shi, B., Zhou, Y. Z., Chen, J., Yao, R., & Xue, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.045 · Cited by 41 · see citing works
AGFNet: Adaptive Gated Fusion Network for RGB-T Semantic Segmentation
Zhou, X., Wu, X., Bao, L., Yin, H., Jiang, Q., & Zhang, J. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3528064 · Cited by 41 · see citing works
Depth Repeated-Enhancement RGB Network for Rail Surface Defect Inspection
Wu, J., Zhou, W., Qiu, W., & Yu, L. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3211199 · Cited by 40 · see citing works
Complementary Random Masking for RGB-Thermal Semantic Segmentation
Shin, U., Lee, K., Kweon, I. S., & Oh, J. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611200 · Cited by 49 · see citing works
“Therefore, it naturally led to recent active studies on semantic segmentation of RGB-thermal images [11], [36], [35], [37], [48], [17], [21], [47].”
BackgroundMethodInfluentialRCNet: Dual-Network Resonance Collaboration via Mutual Learning for RGB-D Road Defect Detection
Zhou, W., Ju, Z., Cong, R., & Yan, W. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3617769 · Cited by 39 · see citing works
Hu, H., Shan, W., Tang, Y., Heidari, A. A., Chen, H., Liu, H., Wang, M., Escorcia‐Gutierrez, J., Mansour, R. F., & Chen, J. (2022) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwac119 · Cited by 35 · see citing works
Domain-invariant information aggregation for domain generalization semantic segmentation
Liao, M., Tian, S., Zhang, Y., Hua, G., Zou, W., & Li, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126273 · Cited by 35 · see citing works
Xu, G., Zhou, W., Qian, X., Zhang, Y., Lei, J., & Yu, L. (2023) · Digital Signal Processing · doi:10.1016/j.dsp.2023.104011 · Cited by 34 · see citing works
Wu, J., Zhou, W., Qian, X., Lei, J., Yu, L., & Luo, T. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.01.024 · Cited by 33 · see citing works
Feng, Z., Guo, Y., & Sun, Y. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3376534 · Cited by 34 · see citing works
Learning multi-level structural information for small organ segmentation
Liu, Y., Duan, Y., & Zeng, T. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108418 · Cited by 32 · see citing works
Kumari, N., Anwar, S., & Bhattacharjee, V. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-06942-x · Cited by 32 · see citing works
MFENet: Multitype fusion and enhancement network for detecting salient objects in RGB-T images
Wu, J., Zhou, W., Qian, X., Lei, J., Yu, L., & Luo, T. (2022) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103827 · Cited by 31 · see citing works
A RGB-Thermal based adaptive modality learning network for day–night wildfire identification
Xue, R., Li, Z., Zhang, X., Li, Z., & Song, W. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103554 · Cited by 31 · see citing works
Han, W., Jiang, W. R., Geng, J., & Wang, M. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3526064 · Cited by 30 · see citing works
Integrating satellite and street-level images for local climate zone mapping
Cao, R., Liao, C., Li, Q., Tu, W., Zhu, R., Luo, N., Qiu, G., & Shi, W. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103323 · Cited by 29 · see citing works
Multi-Branch Differential Bidirectional Fusion Network for RGB-T Semantic Segmentation
Liang, W., Shan, C., Yang, Y., & Han, J. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3374793 · Cited by 30 · see citing works
Song, W., Li, Q., Gao, M., Chehri, A., & Jeon, G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125472 · Cited by 28 · see citing works
CGFNet: cross-guided fusion network for RGB-thermal semantic segmentation
Fu, Y., Chen, Q., & Zhao, H. (2022) · The Visual Computer · doi:10.1007/s00371-022-02559-2 · Cited by 29 · see citing works
Remote Sensing Image Road Segmentation Method Integrating CNN-Transformer and UNet
Wang, R., Cai, M., Xia, Z., & Zhou, Z. (2023) · IEEE Access · doi:10.1109/access.2023.3344797 · Cited by 27 · see citing works
EISNet: A Multi-Modal Fusion Network for Semantic Segmentation With Events and Images
Xie, B., Deng, Y., Shao, Z., & Li, Y. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3380255 · Cited by 28 · see citing works
Nagasree, Y., Rupa, C., Akshitha, P., Srivastava, G., Gadekallu, T. R., & Lakshmanna, K. (2023) · Drones · doi:10.3390/drones7010053 · Cited by 27 · see citing works
Li, G., Lin, Y., Ouyang, D., Li, S., Luo, X., Qu, X., Pi, D., & Li, S. E. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3332350 · Cited by 27 · see citing works
Cui, Y., Yu, M., Jiang, G., Peng, Z., & Chen, F. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3205928 · Cited by 26 · see citing works
Multispectral Video Semantic Segmentation: A Benchmark Dataset and Baseline
Ji, W., Li, J., Bian, C., Zhou, Z., Zhao, J., Yuille, A., & Cheng, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00112 · Cited by 26 · see citing works
Liu, Q., Kampffmeyer, M., Jenssen, R., & Salberg, A.-B. (2022) · International Journal of Remote Sensing · doi:10.1080/01431161.2022.2098078 · Cited by 25 · see citing works
DHFNet: Decoupled Hierarchical Fusion Network for RGB-T dense prediction tasks
Chen, H., Wang, Z., Qin, H., & Mu, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127594 · Cited by 24 · see citing works
ECFNet: Efficient cross-layer fusion network for real time RGB-Thermal urban scene parsing
Shen, Z., Wang, J., Weng, Y., Pan, Z., Li, Y., & Wang, J. (2024) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104579 · Cited by 24 · see citing works
Cui, Y., Jiang, G., Yu, M., & Song, Y. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3165935 · Cited by 24 · see citing works
Sun, F., Zhou, W., Ye, L., & Yu, L. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3194843 · Cited by 24 · see citing works
Hu, Y., Xu, F., & Gao, Z. J. (2022) · Buildings · doi:10.3390/buildings12091487 · Cited by 24 · see citing works
A deep thermal-guided approach for effective low-light visible image enhancement
Cao, Y., Tong, X., Wang, F., Yang, J., Cao, Y., Strat, S. T., & Tisse, C.-L. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.12.007 · Cited by 24 · see citing works
DPNet: Dual-Pyramid Semantic Segmentation Network Based on Improved Deeplabv3 Plus
Wang, J., Zhang, X., Yan, T., & Tan, A. (2023) · Electronics · doi:10.3390/electronics12143161 · Cited by 24 · see citing works
Resolving semantic conflicts in RGB-T semantic segmentation
Zhao, S., Jin, Z., Jiao, Q., Zhang, Q., & Han, J. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111398 · Cited by 22 · see citing works
Song, J., Chen, C., Heidari, A. A., Liu, J., Yu, H., & Chen, H. (2022) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwac021 · Cited by 23 · see citing works
- Authors
- Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N.
- Affiliations
- Zhejiang University of Science and Technology (CN)
- Zhejiang University (CN)
- University of Washington (US)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2021
- Volume
- 30
- Pages
- 7790-7802
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3198062544
- Cited by
- 344
- References
- 56
- Citations by year
- 2026: 43 · 2025: 90 · 2024: 57 · 2023: 76 · 2022: 70 · 2021: 8
- FWCI
- 18.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (61502429, 61972357)
- Zhejiang Provincial Natural Science Foundation of China (LY18F020012)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Keywords
- RGB-thermal fusion, thermal infrared imagery, multilabel learning, cross-modal fusion, shallow and deep feature fusion, boundary supervision, road scene understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
