Cited by 550Advanced Neural Network ApplicationsAdvanced Image Fusion TechniquesAutonomous Vehicle Technology and Safety

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes

Sun, Yuxiang, Zuo, Weixun, Liu, Ming·2019·IEEE Robotics and Automation Letters

DOI 10.1109/lra.2019.2904733

Open this paper in CoChat — free account

Abstract

Semantic segmentation is a fundamental capability for autonomous vehicles. With the advancements of deep learning technologies, many effective semantic segmentation networks have been proposed in recent years. However, most of them are designed using RGB images from visible cameras. The quality of RGB images is prone to be degraded under unsatisfied lighting conditions, such as darkness and glares of oncoming headlights, which imposes critical challenges for the networks that use only RGB images. Different from visible cameras, thermal imaging cameras generate images using thermal radiations. They are able to see under various lighting conditions. In order to enable robust and accurate semantic segmentation for autonomous vehicles, we take the advantage of thermal images and fuse both the RGB and thermal information in a novel deep neural network. The main innovation of this letter is the architecture of the proposed network. We adopt the encoder–decoder design concept. ResNet is employed for feature extraction and a new decoder is developed to restore the feature map resolution. The experimental results prove that our network outperforms the state of the arts.

Integrity signals

Registry

Crossref · 10.1109/lra.2019.2904733

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Pyramid Scene Parsing Network
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Untitled work
(2010) · Cited by 14,134

Show all 30 references

Research works printed but not linked: 1

vollmer, 2017, et�al

Sorted by citations

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
Review the state-of-the-art technologies of semantic segmentation based on deep learning
Mo, Y., Wu, Y. X., Yang, X., Liu, F., & Liao, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.005 · Cited by 669
Deep multimodal data fusion
Zhao, F., Zhang, C., & Geng, B. (2024) · ACM Computing Surveys · doi:10.1145/3649447 · Cited by 480
Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity
Tang, L., Zhang, H., Xu, H., & Ma, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101870 · Cited by 418
Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation
Liu, J., Liu, Z., Wu, G., Ma, L., Liu, R., Zhong, W., Luo, Z., & Fan, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00745 · Cited by 346
GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3109518 · Cited by 344
FuseSeg: Semantic segmentation of urban scenes based on RGB and thermal data fusion
Sun, Y., Zuo, W., Yun, P., Wang, H., & Liu, M. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.2993143 · Cited by 270
Self-citation
PST900: RGB-Thermal Calibration, Dataset and Segmentation Network
Shivakumar, S. S., Rodrigues, N., Zhou, A., Miller, I. D., Kumar, V., & Taylor, C. J. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196831 · Cited by 273
Deep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261
Divide-and-Conquer: Confluent Triple-Flow Network for RGB-T Salient Object Detection
Tang, H., Li, Z., Zhang, D., He, S., & Tang, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3511621 · Cited by 277
MFFENet: Multiscale Feature Fusion and Enhancement Network For RGB–Thermal Urban Road Scene Parsing
Zhou, W., Lin, X., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3086618 · Cited by 253
Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption
Liu, J., Wu, G., Liu, Z., Wang, D., Jiang, Z., Ma, L., Zhong, W., Fan, X., & Liu, R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3521416 · Cited by 238
ABMDRNet: Adaptive-weighted Bi-directional Modality Difference Reduction Network for RGB-T Semantic Segmentation
Zhang, Q., Zhao, S., Luo, Y., Zhang, D., Huang, N., & Han, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00266 · Cited by 225
Blitz-SLAM: A semantic SLAM in dynamic environments
Fan, Y., Zhang, Q., Tang, Y., Liu, S., & Han, H. W. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108225 · Cited by 213
SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection
Fan, R., Wang, H., Cai, P., & Liu, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_21 · Cited by 201
Self-citation
RGB-T Semantic Segmentation With Location, Activation, and Sharpening
Li, G., Wang, Y., Liu, Z., Zhang, X., & Zeng, D. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3208833 · Cited by 197
Recent Advances in Thermal Imaging and its Applications Using Machine Learning: A Review
Wilson, A. N., Gupta, K. A., Koduru, B. H., Kumar, A., Jha, A., & Cenkeramaddi, L. R. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3234335 · Cited by 183
MTANet: Multitask-Aware Network With Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding
Zhou, W., Dong, S., Lei, J., & Yu, L. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3164899 · Cited by 183
Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks
Liang, M., Hu, J., Bao, C., Hua, F., Deng, F., & Lam, T. L. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3272269 · Cited by 179
FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation
Deng, F., Feng, H., Liang, M., Wang, H., Yang, Y., Gao, Y., Chen, J., Hu, J., Guo, X., & Lam, T. L. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636084 · Cited by 170
MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation
Zhou, W., Zhang, H., Yan, W., & Lin, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3275314 · Cited by 157
Edge-Aware Guidance Fusion Network for RGB–Thermal Scene Parsing
Zhou, W., Dong, S., Xu, C., & Qian, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20269 · Cited by 146
Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentation
Zhang, S., Zhang, J., Tian, B., Lukasiewicz, T., & Xu, Z. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102656 · Cited by 145
Context-Aware Interaction Network for RGB-T Semantic Segmentation
Lv, Y., Liu, Z., & Li, G. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3349072 · Cited by 145
FRNet: Feature Reconstruction Network for RGB-D Indoor Scene Parsing
Zhou, W., Yang, E., Lei, J., & Yu, L. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3174338 · Cited by 114
Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
Wan, Z., Zhang, P., Wang, Y., Yong, S., Stepputtis, S., Sycara, K. P., & Xie, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00176 · Cited by 114
EGFNet: Edge-Aware Guidance Fusion Network for RGB–Thermal Urban Scene Parsing
Dong, S., Zhou, W., Xu, C., & Yan, W. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3306368 · Cited by 111
Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms
Wang, H., Fan, R., Sun, Y., & Liu, M. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3064089 · Cited by 104
Self-citation
SGFNet: Semantic-Guided Fusion Network for RGB-Thermal Semantic Segmentation
Wang, Y., Li, G., & Liu, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3281419 · Cited by 104
Mitigating Modality Discrepancies for RGB-T Semantic Segmentation
Zhao, S., Liu, Y., Jiao, Q., Zhang, Q., & Han, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3233089 · Cited by 102
Infrared and Visible Image Fusion Techniques Based on Deep Learning: A Review
Sun, C., Zhang, C., & Xiong, N. N. (2020) · Electronics · doi:10.3390/electronics9122162 · Cited by 101
Attention fusion network for multi-spectral semantic segmentation
Xu, J., Lu, K., & Wang, H. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.03.015 · Cited by 99
Attention-Based Multimodal Image Feature Fusion Module for Transmission Line Detection
Choi, H., Yun, J. P., Kim, B. J., Jang, H., & Kim, S. W. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3147833 · Cited by 98
CIMFNet: Cross-Layer Interaction and Multiscale Fusion Network for Semantic Segmentation of High-Resolution Remote Sensing Images
Zhou, W., Jianhui, J., Lei, J., & Yu, L. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3159032 · Cited by 96
RGB-T image analysis technology and application: A survey
Song, K., Zhao, Y., Huang, L., Yan, Y., & Meng, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105919 · Cited by 94
Multimodal Cross-Layer Bilinear Pooling for RGBT Tracking
Xu, Q., Mei, Y., Liu, J., & Li, C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3055362 · Cited by 94
A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation
Zhao, S., & Zhang, Q. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3229359 · Cited by 91
HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images
Vertens, J., Zürn, J., & Burgard, W. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341192 · Cited by 91
CEKD: Cross-Modal Edge-Privileged Knowledge Distillation for Semantic Scene Understanding Using Only Thermal Images
Feng, Z., Guo, Y., & Sun, Y. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3247175 · Cited by 85
Self-citation
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation
Reza, M. K., Prater-Bennette, A., & Asif, M. S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3476487 · Cited by 78
From SLAM to Situational Awareness: Challenges and Survey
Bavle, H., Sánchez-López, J. L., Cimarelli, C., Tourani, A., & Voos, H. (2023) · Sensors · doi:10.3390/s23104849 · Cited by 88
Semantic Image Segmentation: Two Decades of Research
Csurka, G., Volpi, R., & Chidlovskii, B. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000095 · Cited by 84
MMNet: Multi-modal multi-stage network for RGB-T image semantic segmentation
Lan, X., Gu, X., & Gu, X. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02687-7 · Cited by 74
Complementarity-aware cross-modal feature fusion network for RGB-T semantic segmentation
Wu, W., Chu, T., & Liu, Q. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108881 · Cited by 74
Deep Depth Estimation from Thermal Image
Shin, U., Park, J., & Kweon, I. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00107 · Cited by 76
Applying Surface Normal Information in Drivable Area and Road Anomaly Detection for Ground Mobile Robots
Wang, H., Fan, R., Sun, Y., & Liu, M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341340 · Cited by 73
Self-citation
Glass Segmentation With RGB-Thermal Image Pairs
Huo, D., Wang, J., Qian, Y., & Yang, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3256762 · Cited by 73
Self-Supervised Drivable Area and Road Anomaly Segmentation Using RGB-D Data For Robotic Wheelchairs
Wang, H., Sun, Y., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2932874 · Cited by 74
Self-citation
We Learn Better Road Pothole Detection: From Attention Aggregation to Adversarial Domain Adaptation
Fan, R., Wang, H., Bocus, M. J., & Liu, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66823-5_17 · Cited by 72
Self-citation
Embedded Control Gate Fusion and Attention Residual Learning for RGB–Thermal Urban Scene Parsing
Zhou, W., Lv, Y. H., Lei, J., & Yu, L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3242651 · Cited by 71
Object Detection from Thermal Infrared and Visible Light Cameras in Search and Rescue Scenes
Bañuls-Arias, A., Mandow, A., Vázquez-Martín, R., Morales, J., & Garcia-Cerezo, A. J. (2020) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr50563.2020.9292593 · Cited by 74
Self-Training for Class-Incremental Semantic Segmentation
Yu, L., Liu, X., & van de Weijer, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3155746 · Cited by 70
MaeFuse: Transferring Omni Features With Pretrained Masked Autoencoders for Infrared and Visible Image Fusion via Guided Training
Li, J., Jiang, J., Liang, P., Ma, J., & Nie, L. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3541562 · Cited by 80
DBCNet: Dynamic Bilateral Cross-Fusion Network for RGB-T Urban Scene Understanding in Intelligent Vehicles
Zhou, W., Gong, T., Lei, J., & Yu, L. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3298921 · Cited by 68
Meta-Transfer Learning Driven Tensor-Shot Detector for the Autonomous Localization and Recognition of Concealed Baggage Threats
Hassan, T., Shafay, M., Akçay, S., Khan, S. H., Bennamoun, M., Damiani, E., & Werghi, N. (2020) · Sensors · doi:10.3390/s20226450 · Cited by 67
PSPNet-SLAM: A Semantic SLAM Detect Dynamic Object by Pyramid Scene Parsing Network
Long, X., Zhang, W., & Zhao, B. (2020) · IEEE Access · doi:10.1109/access.2020.3041038 · Cited by 67
ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data
Zhang, J., Yang, K., & Stiefelhagen, R. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636109 · Cited by 69
RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing
Li, J., Zhang, Y., Peng, Y., Zhou, G., Chen, Q., & Fan, R. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3388726 · Cited by 69
Oral cancer detection using feature-level fusion and novel self-attention mechanisms
Khan, S. U. R., & Asif, S. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106437 · Cited by 62
Real-time infrared and visible image fusion network using adaptive pixel weighting strategy
Zhang, X., Zhai, H., Liu, J., Wang, Z., & Sun, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101863 · Cited by 61
MDNet: Mamba-Effective Diffusion-Distillation Network for RGB-Thermal Urban Dense Prediction
Zhou, W., Wu, H., & Jiang, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3508058 · Cited by 67
MCNet: Multi-level Correction Network for thermal image semantic segmentation of nighttime driving scene
Xiong, H., Cai, W., & Liu, Q. (2021) · Infrared Physics & Technology · doi:10.1016/j.infrared.2020.103628 · Cited by 62
MS-UDA: Multi-Spectral Unsupervised Domain Adaptation for Thermal Image Semantic Segmentation
Kim, Y.-H., Shin, U., Park, J., & Kweon, I. S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3093652 · Cited by 63
SFAF-MA: Spatial Feature Aggregation and Fusion With Modality Adaptation for RGB-Thermal Semantic Segmentation
He, X., Wang, M., Liu, T., Zhao, L., & Yue, Y. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3267529 · Cited by 60
SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detection
Wang, H., Fan, R., Cai, P., & Liu, M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636723 · Cited by 59
Self-citation
Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives
Rizzoli, G., Barbato, F., & Zanuttigh, P. (2022) · Technologies · doi:10.3390/technologies10040090 · Cited by 58
On the Role of Thermal Imaging in Automotive Applications: A Critical Review
Farooq, M. A., Shariff, W., O'Callaghan, D., Merla, A., & Corcoran, P. (2023) · IEEE Access · doi:10.1109/access.2023.3255110 · Cited by 58
CACFNet: Cross-Modal Attention Cascaded Fusion Network for RGB-T Urban Scene Parsing
Zhou, W., Dong, S., Fang, M., & Yu, L. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3314527 · Cited by 57
Dual-Space Graph-Based Interaction Network for RGB-Thermal Semantic Segmentation in Electric Power Scene
Xu, C. X., Li, Q., Jiang, X., Yu, D., & Zhou, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3216313 · Cited by 56
PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation
Liu, Z., Liu, J., Zhang, B., Ma, L., Fan, X., & Liu, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611928 · Cited by 56
A Total Variation With Joint Norms For Infrared and Visible Image Fusion
Nie, R., Ma, C., Cao, J., Ding, H., & Zhou, D. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3065496 · Cited by 55
Semantic Segmentation for Thermal Images: A Comparative Survey
Kutuk, Z., & Algan, G. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00043 · Cited by 54
Multi-Modal Sensor Fusion-Based Semantic Segmentation for Snow Driving Scenarios
Vachmanus, S., Ravankar, A. A., Emaru, T., & Kobayashi, Y. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3077029 · Cited by 53
Multispectral Fusion Transformer Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, H., Tian, C., Zhang, Z., Huo, Q., Xie, Y., & Li, Z. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3179721 · Cited by 53
GCNet: Grid-like context-aware network for RGB-thermal semantic segmentation
Liu, J., Zhou, W., Cui, Y., Yu, L., & Luo, T. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.07.041 · Cited by 53
DHFNet: dual-decoding hierarchical fusion network for RGB-thermal semantic segmentation
Cai, Y., Zhou, W., Zhang, L., Yu, L., & Luo, T. (2023) · The Visual Computer · doi:10.1007/s00371-023-02773-6 · Cited by 53
iCurb: Imitation Learning-Based Detection of Road Curbs Using Aerial Images for Autonomous Driving
Xu, Z., Sun, Y., & Liu, M. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3056344 · Cited by 53
Self-citation
UTFNet: Uncertainty-Guided Trustworthy Fusion Network for RGB-Thermal Semantic Segmentation
Wang, Q., Yin, C., Song, H., Shen, T., & Gu, Y. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3322452 · Cited by 52
Contrastive learning-based knowledge distillation for RGB-thermal urban scene semantic segmentation
Guo, X., Zhou, W., & Liu, T. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111588 · Cited by 52
An interactive fusion attention-guided network for ground surface hot spring fluids segmentation in dual-spectrum UAV images
Yi, S., Chen, M., Yuan, X., Si, G., & Wang, J. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.01.022 · Cited by 52
Vision-Based Trajectory Planning via Imitation Learning for Autonomous Vehicles
Cai, P., Sun, Y., Chen, Y., & Liu, M. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917149 · Cited by 52
Self-citation
Fusion of thermal and RGB images for automated deep learning based crack detection in civil infrastructure
Alexander, Q. G., Hoskere, V., Narazaki, Y., Maxwell, A., & Spencer, B. F. (2022) · AI in Civil Engineering · doi:10.1007/s43503-022-00002-y · Cited by 56
IGT: Illumination-guided RGB-T object detection with transformers
Chen, K., Liu, J., & Zhang, H. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110423 · Cited by 50
Complementary Random Masking for RGB-Thermal Semantic Segmentation
Shin, U., Lee, K., Kweon, I. S., & Oh, J. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611200 · Cited by 49
“Specifically, they proposed various types of feature fusion modules, such as the na ¨ ıve feature-level fusion [11], [36], [35], multi-scale feature fusion [36], [46], [37], [21], and attention-weighted fusion [44], [46], [8], [21].”
BackgroundMethodInfluential
CCAFFMNet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module
Yi (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.056 · Cited by 49
Residual spatial fusion network for RGB-thermal semantic segmentation
Li, P., Chen, J., Lin, B., & Xu, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127913 · Cited by 49
Re-Synchronization Using the Hand Preceding Model for Multi-Modal Fusion in Automatic Continuous Cued Speech Recognition
Liu, L., Feng, G., Beautemps, D., & Zhang, X.-P. S. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2976493 · Cited by 49
Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning
Dong, S., Feng, Y., Yang, Q., Huang, Y., Liu, D., & Fan, H. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801872 · Cited by 46
Mask-guided modality difference reduction network for RGB-T semantic segmentation
Liang, W., Yang, Y., Li, F., Long, X., & Shan, C. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.12.036 · Cited by 45
Semantic Segmentation to Develop an Indoor Navigation System for an Autonomous Mobile Robot
Teso-Fz-Betoño, D., Zulueta, E., Sánchez-Chica, A., Fernández‐Gámiz, U., & Saenz‐Aguirre, A. (2020) · Mathematics · doi:10.3390/math8050855 · Cited by 47
Robust semantic segmentation based on RGB-thermal in variable lighting scenes
Guo, Z., Xu, L., Xu, Q., & Sun, Z. (2021) · Measurement · doi:10.1016/j.measurement.2021.110176 · Cited by 44
Global contextually guided lightweight network for RGB-thermal urban scene understanding
Gong, T., Zhou, W., Qian, X., Lei, J., & Yu, L. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105510 · Cited by 43
MMSFormer: Multimodal Transformer for Material and Semantic Segmentation
Reza, M. K., Prater-Bennette, A., & Asif, M. S. (2024) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2024.3389812 · Cited by 44
RoadFormer+: Delivering RGB-X Scene Parsing Through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusion
Huang, J., Li, J., Ning, J., Sun, Y., Liu, C., Chen, Q., & Fan, R. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3448251 · Cited by 46
Self-citation
Improving Visual Localization Accuracy in Dynamic Environments Based on Dynamic Region Removal
Cheng, J., Zhang, H., & Meng, M. Q. ‐. (2020) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.2964938 · Cited by 42
MSNet: multispectral semantic segmentation network for remote sensing images
Tao, C., Meng, Y., Li, J., Yang, B., Hu, F., Li, Y., Cui, C., & Zhang, W. (2022) · GIScience & Remote Sensing · doi:10.1080/15481603.2022.2101728 · Cited by 42
AGFNet: Adaptive Gated Fusion Network for RGB-T Semantic Segmentation
Zhou, X., Wu, X., Bao, L., Yin, H., Jiang, Q., & Zhang, J. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3528064 · Cited by 42
Assessing thermal imagery integration into object detection methods on air-based collection platforms
Gallagher, J., & Oughton, E. J. (2023) · Scientific Reports · doi:10.1038/s41598-023-34791-8 · Cited by 39
URTSegNet: A real-time segmentation network of unstructured road at night based on thermal infrared images for autonomous robot system
Liu, X., Wang, J., & Li, J. (2023) · Control Engineering Practice · doi:10.1016/j.conengprac.2023.105560 · Cited by 39

Authors: Sun, Y., Zuo, W., & Liu, M.
Affiliations: Hong Kong University of Science and Technology (HK)
ORCID: Sun, Yuxiang
Zuo, Weixun
Liu, Ming
Venue: IEEE Robotics and Automation Letters
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019-07
Volume: 4
Issue: 3
Pages: 2576-2583
ISSN: 2377-3766 (ISSN-L)
2377-3774
DOI: 10.1109/lra.2019.2904733
OpenAlex: W2921749009
Cited by: 546
References: 30
Citations by year: 2026: 87 · 2025: 127 · 2024: 102 · 2023: 94 · 2022: 51 · 2021: 45 · 2020: 31 · 2019: 9
FWCI: 15.22
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Robotics and Automation Letters (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: Shenzhen Science, Technology, and Innovation Commission (JCYJ20160401100022706)
National Natural Science Foundation of China (U1713211)
Hong Kong University of Science and Technology Project (IGN16EG12)
Hong Kong Research Grant Council (11210017, 21202816)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords: RGB-thermal fusion, semantic segmentation, urban scenes, autonomous vehicles, infrared thermography, encoder-decoder architecture, ResNet feature extraction, low-light conditions, headlight glare

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks