The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes
Abstract
Semantic segmentation is a fundamental capability for autonomous vehicles. With the advancements of deep learning technologies, many effective semantic segmentation networks have been proposed in recent years. However, most of them are designed using RGB images from visible cameras. The quality of RGB images is prone to be degraded under unsatisfied lighting conditions, such as darkness and glares of oncoming headlights, which imposes critical challenges for the networks that use only RGB images. Different from visible cameras, thermal imaging cameras generate images using thermal radiations. They are able to see under various lighting conditions. In order to enable robust and accurate semantic segmentation for autonomous vehicles, we take the advantage of thermal images and fuse both the RGB and thermal information in a novel deep neural network. The main innovation of this letter is the architecture of the proposed network. We adopt the encoder–decoder design concept. ResNet is employed for feature extraction and a new decoder is developed to restore the feature map resolution. The experimental results prove that our network outperforms the state of the arts.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Compared to the object detection problem summarized in Sec. IV-A, there are fewer works on multi-modal semantic segmentation: [92], [119], [124] employ RGB and thermal images, [61] fuses RGB images and depth images from a stereo camera, [125]-[127] combine RGB, thermal, and depth images for semantic segmentation in diverse environments such as forests, [123] fuses RGB images and LiDAR point clouds”
From Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2021) · cites this paper as [124]
In the citing paper
“| RTFNet [106] | ResNet152 [16] | 53.2 |”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [106]
In the citing paper
“| RFNet [106] | ResNet18 [16] | 69.37 |”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [106]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
(2010) · Cited by 14,134
Research works printed but not linked: 1
- vollmer, 2017, et�al
Cited by
Sorted by citations
Items 501–550 of 550
基于可形变VMamba与特征对齐的无人机多模态图像语义分割 (特邀)
Tao, 张. Z., Ruitao, 卢. L., Yaochen, 李. L., Xueli, 谢. X., Jiwei, 范. F., & Xiaogang, 杨. Y. (2026) · Infrared and Laser Engineering · doi:10.3788/irla20260160
A shared and discrepant cue-guided network for RGB-T semantic segmentation
Song, X., Liu, Y., Xue, W., & Zhuang, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134240
Zhang, Z., Zhang, M., Hou, H., Shi, Q., Meng, F., Li, X., Yang, T., Ren, W., & Jing, P. (2026) · Digital Signal Processing · doi:10.1016/j.dsp.2026.106377
An unaligned multimodal cross-domain fusion network for salient object detectionPreprint
Ding, Y., Song, M., Zhao, Y., & Yu, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7071770
Lan, X., & Cui, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3495-0_18
SWDNet: Knowledge-Driven Cross-Modal Network with Sliding Window Difference Modeling
Huang, J., & Xie, R. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2759-4_31
Zhu, Q., Yang, F., Wang, L., Wan, Z., Mo, X., & Shao, F. (2026) · Displays · doi:10.1016/j.displa.2026.103624
Multi-scale rectified mixture-of-experts fusion for RGB–Thermal semantic segmentation
Gozler, D., & Topal, C. (2026) · Image and Vision Computing · doi:10.1016/j.imavis.2026.106109
Intra-Modal Cross-Attention Fusion for Domain Generalized Semantic Segmentation
Bätje, M., Schwalbe, G., Mütze, A., Schwonberg, M., & Leucker, M. (2026) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv66570.2026.11625210
SAR: semantic ambiguity relief network for RGB-T semantic segmentation
Zhao, S., Li, J., Jiao, Q., Zhang, Q., & Han, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104671
Nikkhah, A. F., Kim, K. Y., Zheng, Z., Lakshmi, V., & Heydarian, A. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.107168
RM-MLLM: Robust scene understanding in adverse weather with RGB-multispectral fusion
Ju, H., Yeo, S. D., Park, I., Jin, Y., Nalçakan, Y., & Kim, S. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.112317
Li, C., Wei, X., He, W., & Yin, Y. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3720870
Jin, Q., Zhang, Y., Peng, C., Hu, J., & Li, L. (2026) · Geo-spatial Information Science · doi:10.1080/10095020.2026.2707066
He, X., Guo, B., Zhao, S., & Han, Y. (2026) · Infrared Physics & Technology · doi:10.1016/j.infrared.2026.106793
Ni, Z., Hu, L., Wang, H., Li, F., Tang, M., Ma, G., & Sui, H. (2026) · Remote Sensing · doi:10.3390/rs18162692
Ling, 沈. S., Bo, 倪. N., Qixuan, Y., Haibin, 倪. N., Xiaoyan, 周. Z., & Jianhua, 常. C. (2026) · Laser & Optoelectronics Progress · doi:10.3788/lop260604
Sigma-Based RGB-Hyperspectral Fusion for Semantic Segmentation in Autonomous Driving
Aditya, S., Sai Kiran, K., Sankararao, A. U. G., Reddy, D., & Rajalakshmi, P. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630398
Wang, F., Liu, Q., & Wu, G. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08745-6
HFFNet: Hierarchical feature fusion network for lightweight RGB-T semantic segmentation
Wu, X., Zhou, X., Bao, L., Shi, R., Liu, D., Liu, Z., & Zhang, J. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109510
Liu, Y., Zhang, W., Chen, X., Ma, Z., Ye, J., Zheng, G. Y., Wang, L., & Hou, W. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3724831
Knowledge-guided and reliability-aware fusion network for robust RGB-thermal road scene segmentation
Guo, Y., Chen, Z., Liu, C., Zhao, Y., Jian, M., & Meng, Q. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133975
Multimodal Instance Segmentation of Building Façade Damage with Frequency-Decoupled Feature Fusion
Xu, S., Chen, Z., Wang, Y., Shi, Y., Lu, H., Gu, H., & Sun, Z. (2026) · Buildings · doi:10.3390/buildings16173396
Thermal-gradient-guided RGB–thermal fusion for intelligent facade hollow diagnosis
Huang, J., Diao, Y., Du, Y., & Pang, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134137
Zhen, F., & Sun, Y. (2026) · IEEE Canadian Atlantic Ocean Symposium (CAOS) · doi:10.1109/caos69069.2026.11666192
Self-citationLiu, J., Li, C., Wang, X., & Wang, B. (2026) · Sensors · doi:10.3390/s26175498
Lv, Y. (2026) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/cis-ram71614.2026.11668507
Sultonov, F., Gong, M.-G., Park, S.-J., Kim, I., Kim, J., Yun, S., & Kang, J. (2026) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2026.3729955
Liu, B., Zou, Q., Li, G., & Yu, W. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3730457
Multi-modal interaction enhanced segment anything model (MIE-SAM) for RGB-T salient object detection
Li, Z., Zhang, Y. Y., Zhang, S., & Wang, Z. P. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109586
Cheng, P., Zhang, L., Wang, B., Liu, J., & Sui, H. (2026) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2026.105572
DSDFNet: Dual-Stream Dense-Feedback Network for RGB-T Semantic Segmentation
Zhang, H., Dong, X., & Du, N. (2026) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci69731.2026.11677891
Xing, W., Chen, D., Islam, M. A., & Zhou, J. W. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7446598
Chen, Y., Zhang, J., Gao, Y., & Huang, B. (2026) · Research Square · doi:10.21203/rs.3.rs-10142899/v1
TSWNet: Thermal stereo matching via wavelet reconstruction and edge feature enhancement
梅佳莹, Jiang, W., Zhu, X., Cheng, Y., & Fang, Y. (2026) · Infrared Physics & Technology · doi:10.1016/j.infrared.2026.106916
Jie, 刘. L., Yuhan, 李. L., Juncheng, 李. L., Duan, Q., Linhan, 李. L., Hao, S., Sili, 高. G., Juan, 岳. Y., & Kaijun, 马. M. (2026) · Laser & Optoelectronics Progress · doi:10.3788/lop260697
Depth Adaption SegNet for RGB-T Segmentation
Zhao, S., Zhang, C., Zhu, H., Liu, B., & Zhou, Y. (2026) · Applied Sciences · doi:10.3390/app16199702
Fan, Z., Li, Y., Shen, Z., Wang, J., Yang, X., & Wang, J. (2026) · Digital Signal Processing · doi:10.1016/j.dsp.2026.106525
Focus on the Hard: SAM-Based Prompt-Guided Refinement for RGB-T Semantic Segmentation
Wang, Y., Tang, H., Shi, Y., Yang, K., Han, J., & Zhang, D. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3728642
Backbone-Preserving Association Refinement for Visible-Thermal Multiple Object Tracking
Huang, X., Lin, M., Sang, R., Yue, Y., García‐Fernández, Á. F., Maskell, S., Zhou, Y., Ma, J., Zhu, X., & Lim, E. G. (2026) · International Conference on Automation and Computing (ICAC) · doi:10.1109/icac70325.2026.11712492
Wu, S., Wang, Z., Yu, Z., Zhang, L., Sun, X., Liu, Y., & Chen, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7553541
Peng, Z., Chi, H.-L., Wu, H., & Zhang, Z. (2026) · Engineering Construction & Architectural Management · doi:10.1108/ecam-01-2026-0162
Wang, X., & Yan, G. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.116603
Adjusting modal contributions via differential-driven SoftMax fusion for RGB-IR object detection
Yu, H., Zhang, J., Ding, R., Wei, B., Dang, W., Nie, J., Chen, L., Xichuan, Z., & Liu, H. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109707
A Survey on Degraded Image Segmentation
Chen, L., Fu, Y., Shangguan, J., Xu, J., Peng, Y., & Lyu, F. (2026) · Chinese Journal of Electronics · doi:10.23919/cje.2025.00.245 · Cited by 1
Exploring Dual-to-Single-Encoder Transitions for Real-Time RGB-T Semantic Segmentation
Kim, J., Jung, M., & Moon, B. (2026) · Sensors · doi:10.3390/s26206602
CFEFNet: Cross-Modal Feature Enhancement and Fusion Network for RGB-T Semantic Segmentation
Zhang, Z., & Sun, L. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729509
Elfahimi, H., Aferhane, A., Harba, R., Douzi, H., Canals, R., Vilcahuamán, L., & Arbañil, H. (2026) · Smart Health · doi:10.1016/j.smhl.2026.100730
BSIF-Net: Boundary-aware semantic inconsistency fusion network for RGB-T semantic segmentation
Qiu, H., Yang, R., Jiang, J., Zhan, L., Liu, J., Hu, Y., & Zhou, G. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.135564
Chen, Y., Zhang, J., Fang, Z., Gao, Y., & Huang, B. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117249
Metadata
- Authors
- Sun, Y., Zuo, W., & Liu, M.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2019-07
- Volume
- 4
- Issue
- 3
- Pages
- 2576-2583
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2921749009
- Cited by
- 546
- References
- 30
- Citations by year
- 2026: 87 · 2025: 127 · 2024: 102 · 2023: 94 · 2022: 51 · 2021: 45 · 2020: 31 · 2019: 9
- FWCI
- 15.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Shenzhen Science, Technology, and Innovation Commission (JCYJ20160401100022706)
- National Natural Science Foundation of China (U1713211)
- Hong Kong University of Science and Technology Project (IGN16EG12)
- Hong Kong Research Grant Council (11210017, 21202816)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- RGB-thermal fusion, semantic segmentation, urban scenes, autonomous vehicles, infrared thermography, encoder-decoder architecture, ResNet feature extraction, low-light conditions, headlight glare
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review