Cited by 270Advanced Neural Network ApplicationsAdvanced Image Fusion TechniquesAutonomous Vehicle Technology and Safety

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.

FuseSeg: Semantic Segmentation of Urban Scenes Based on RGB and Thermal Data Fusion

Sun, Yuxiang, Zuo, Weixun, Yun, Peng, Wang, Hengli, Liu, Ming·2021·IEEE Transactions on Automation Science and Engineering

DOI 10.1109/tase.2020.2993143

Open this paper in CoChat — free account

Abstract

Semantic segmentation of urban scenes is an essential component in various applications of autonomous driving. It makes great progress with the rise of deep learning technologies. Most of the current semantic segmentation networks use single-modal sensory data, which are usually the RGB images produced by visible cameras. However, the segmentation performance of these networks is prone to be degraded when lighting conditions are not satisfied, such as dim light or darkness. We find that thermal images produced by thermal imaging cameras are robust to challenging lighting conditions. Therefore, in this article, we propose a novel RGB and thermal data fusion network named FuseSeg to achieve superior performance of semantic segmentation in urban scenes. The experimental results demonstrate that our network outperforms the state-of-the-art networks.Note to Practitioners—This article investigates the problem of semantic segmentation of urban scenes when lighting conditions are not satisfied. We provide a solution to this problem via information fusion with RGB and thermal data. We build an end-to-end deep neural network, which takes as input a pair of RGB and thermal images and outputs pixel-wise semantic labels. Our network could be used for urban scene understanding, which serves as a fundamental component of many autonomous driving tasks, such as environment modeling, obstacle avoidance, motion prediction, and planning. Moreover, the simple design of our network allows it to be easily implemented using various deep learning frameworks, which facilitates the applications on different hardware or software platforms.

Integrity signals

Registry

Crossref · 10.1109/tase.2020.2993143

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Pyramid Scene Parsing Network
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Untitled work
(2010) · Cited by 14,134
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Wide Residual Networks
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244

Show all 60 references

Sorted by citations

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
Review the state-of-the-art technologies of semantic segmentation based on deep learning
Mo, Y., Wu, Y. X., Yang, X., Liu, F., & Liao, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.005 · Cited by 669
GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3109518 · Cited by 344
MFFENet: Multiscale Feature Fusion and Enhancement Network For RGB–Thermal Urban Road Scene Parsing
Zhou, W., Lin, X., Lei, J., Yu, L., & Hwang, J. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3086618 · Cited by 253
Delivering Arbitrary-Modal Semantic Segmentation
Zhang, J., Liu, R., Shi, H., Yang, K., Reiß, S., Peng, K., Fu, H., Wang, K., & Stiefelhagen, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00116 · Cited by 215
RGB-T Semantic Segmentation With Location, Activation, and Sharpening
Li, G., Wang, Y., Liu, Z., Zhang, X., & Zeng, D. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3208833 · Cited by 197
Multi-Modal 3D Object Detection in Autonomous Driving: A Survey
Wang, Y., Mao, Q., Zhu, H., Deng, J., Zhang, Y., Ji, J., Li, H., & Zhang, Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01784-z · Cited by 232
Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks
Liang, M., Hu, J., Bao, C., Hua, F., Deng, F., & Lam, T. L. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3272269 · Cited by 179
MTANet: Multitask-Aware Network With Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding
Zhou, W., Dong, S., Lei, J., & Yu, L. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3164899 · Cited by 183
Cross-Modal Fusion and Progressive Decoding Network for RGB-D Salient Object Detection
Hu, X., Sun, F., Sun, J., Wang, F., & Li, H. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02020-y · Cited by 162
MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation
Zhou, W., Zhang, H., Yan, W., & Lin, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3275314 · Cited by 157
FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation
Deng, F., Feng, H., Liang, M., Wang, H., Yang, Y., Gao, Y., Chen, J., Hu, J., Guo, X., & Lam, T. L. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636084 · Cited by 170
Self-citation
Edge-Aware Guidance Fusion Network for RGB–Thermal Scene Parsing
Zhou, W., Dong, S., Xu, C., & Qian, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20269 · Cited by 146
Context-Aware Interaction Network for RGB-T Semantic Segmentation
Lv, Y., Liu, Z., & Li, G. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3349072 · Cited by 145
MODS—A USV-Oriented Object Detection and Obstacle Segmentation Benchmark
Bovcon, B., Muhovič, J., Vranac, D., Mozetič, D., Perš, J., & Kristan, M. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3124192 · Cited by 131
META-Unet: Multi-Scale Efficient Transformer Attention Unet for Fast and High-Accuracy Polyp Segmentation
Wu, H., Zhao, Z., & Wang, Z. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3292373 · Cited by 112
SGFNet: Semantic-Guided Fusion Network for RGB-Thermal Semantic Segmentation
Wang, Y., Li, G., & Liu, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3281419 · Cited by 104
Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
Wan, Z., Zhang, P., Wang, Y., Yong, S., Stepputtis, S., Sycara, K. P., & Xie, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00176 · Cited by 114
Mitigating Modality Discrepancies for RGB-T Semantic Segmentation
Zhao, S., Liu, Y., Jiao, Q., Zhang, Q., & Han, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3233089 · Cited by 102
EGFNet: Edge-Aware Guidance Fusion Network for RGB–Thermal Urban Scene Parsing
Dong, S., Zhou, W., Xu, C., & Yan, W. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3306368 · Cited by 111
A Comprehensive Review on Deep Learning-Based Data Fusion
Hussain, M., O’Nils, M., Lundgren, J., & Mousavirad, S. J. (2024) · IEEE Access · doi:10.1109/access.2024.3508271 · Cited by 116
CIMFNet: Cross-Layer Interaction and Multiscale Fusion Network for Semantic Segmentation of High-Resolution Remote Sensing Images
Zhou, W., Jianhui, J., Lei, J., & Yu, L. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3159032 · Cited by 96
RGB-T image analysis technology and application: A survey
Song, K., Zhao, Y., Huang, L., Yan, Y., & Meng, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105919 · Cited by 94
A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation
Zhao, S., & Zhang, Q. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3229359 · Cited by 91
Deep Learning in Multimodal Fusion for Sustainable Plant Care: A Comprehensive Review
Yang, Z., Li, Y., Wang, R.-F., Hu, P., & Su, W. (2025) · Sustainability · doi:10.3390/su17125255 · Cited by 88
Glass Segmentation With RGB-Thermal Image Pairs
Huo, D., Wang, J., Qian, Y., & Yang, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3256762 · Cited by 73
From SLAM to Situational Awareness: Challenges and Survey
Bavle, H., Sánchez-López, J. L., Cimarelli, C., Tourani, A., & Voos, H. (2023) · Sensors · doi:10.3390/s23104849 · Cited by 88
Complementarity-aware cross-modal feature fusion network for RGB-T semantic segmentation
Wu, W., Chu, T., & Liu, Q. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108881 · Cited by 74
Deep Depth Estimation from Thermal Image
Shin, U., Park, J., & Kweon, I. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00107 · Cited by 76
Embedded Control Gate Fusion and Attention Residual Learning for RGB–Thermal Urban Scene Parsing
Zhou, W., Lv, Y. H., Lei, J., & Yu, L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3242651 · Cited by 71
DBCNet: Dynamic Bilateral Cross-Fusion Network for RGB-T Urban Scene Understanding in Intelligent Vehicles
Zhou, W., Gong, T., Lei, J., & Yu, L. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3298921 · Cited by 68
Multispectral Semantic Segmentation for Land Cover Classification: An Overview
Ramos, L., & Sappa, A. D. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3438620 · Cited by 70
MS-UDA: Multi-Spectral Unsupervised Domain Adaptation for Thermal Image Semantic Segmentation
Kim, Y.-H., Shin, U., Park, J., & Kweon, I. S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3093652 · Cited by 63
MDNet: Mamba-Effective Diffusion-Distillation Network for RGB-Thermal Urban Dense Prediction
Zhou, W., Wu, H., & Jiang, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3508058 · Cited by 67
Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives
Rizzoli, G., Barbato, F., & Zanuttigh, P. (2022) · Technologies · doi:10.3390/technologies10040090 · Cited by 58
A Total Variation With Joint Norms For Infrared and Visible Image Fusion
Nie, R., Ma, C., Cao, J., Ding, H., & Zhou, D. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3065496 · Cited by 55
Dual-Space Graph-Based Interaction Network for RGB-Thermal Semantic Segmentation in Electric Power Scene
Xu, C. X., Li, Q., Jiang, X., Yu, D., & Zhou, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3216313 · Cited by 56
On the Role of Thermal Imaging in Automotive Applications: A Critical Review
Farooq, M. A., Shariff, W., O'Callaghan, D., Merla, A., & Corcoran, P. (2023) · IEEE Access · doi:10.1109/access.2023.3255110 · Cited by 58
CACFNet: Cross-Modal Attention Cascaded Fusion Network for RGB-T Urban Scene Parsing
Zhou, W., Dong, S., Fang, M., & Yu, L. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3314527 · Cited by 57
Thermal image-based hand gesture recognition for worker-robot collaboration in the construction industry: A feasible study
Wu, H., Li, H., Chi, H.-L., Peng, Z., Chang, S., & Wu, Y. C. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2023.101939 · Cited by 55
Transforming the maintenance of underground infrastructure through Digital Twins: State of the art and outlook
Zhu, H., Huang, M., Ji, P.-Q., Xiao, F., & Zhang, Q. (2025) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2025.106508 · Cited by 62
DHFNet: dual-decoding hierarchical fusion network for RGB-thermal semantic segmentation
Cai, Y., Zhou, W., Zhang, L., Yu, L., & Luo, T. (2023) · The Visual Computer · doi:10.1007/s00371-023-02773-6 · Cited by 53
Multispectral Fusion Transformer Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, H., Tian, C., Zhang, Z., Huo, Q., Xie, Y., & Li, Z. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3179721 · Cited by 53
GCNet: Grid-like context-aware network for RGB-thermal semantic segmentation
Liu, J., Zhou, W., Cui, Y., Yu, L., & Luo, T. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.07.041 · Cited by 53
Semantic Segmentation for Thermal Images: A Comparative Survey
Kutuk, Z., & Algan, G. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00043 · Cited by 54
An interactive fusion attention-guided network for ground surface hot spring fluids segmentation in dual-spectrum UAV images
Yi, S., Chen, M., Yuan, X., Si, G., & Wang, J. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.01.022 · Cited by 52
CCAFFMNet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module
Yi (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.056 · Cited by 49
Contrastive learning-based knowledge distillation for RGB-thermal urban scene semantic segmentation
Guo, X., Zhou, W., & Liu, T. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111588 · Cited by 52
Residual spatial fusion network for RGB-thermal semantic segmentation
Li, P., Chen, J., Lin, B., & Xu, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127913 · Cited by 49
Mask-guided modality difference reduction network for RGB-T semantic segmentation
Liang, W., Yang, Y., Li, F., Long, X., & Shan, C. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.12.036 · Cited by 45
Real-time semantic segmentation for autonomous driving: A review of CNNs, Transformers, and Beyond
Elhassan, M. A. M., Zhou, C., Khan, A. H., Benabid, A., Adam, A. B. M., Mehmood, A., & Wambugu, N. M. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102226 · Cited by 53
Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning
Dong, S., Feng, Y., Yang, Q., Huang, Y., Liu, D., & Fan, H. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801872 · Cited by 46
Global contextually guided lightweight network for RGB-thermal urban scene understanding
Gong, T., Zhou, W., Qian, X., Lei, J., & Yu, L. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105510 · Cited by 43
AGFNet: Adaptive Gated Fusion Network for RGB-T Semantic Segmentation
Zhou, X., Wu, X., Bao, L., Yin, H., Jiang, Q., & Zhang, J. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3528064 · Cited by 42
Complementary Random Masking for RGB-Thermal Semantic Segmentation
Shin, U., Lee, K., Kweon, I. S., & Oh, J. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611200 · Cited by 49
“Specifically, they proposed various types of feature fusion modules, such as the na ¨ ıve feature-level fusion [11], [36], [35], multi-scale feature fusion [36], [46], [37], [21], and attention-weighted fusion [44], [46], [8], [21].”
BackgroundMethod
Ultrasound-Guided Assistive Robots for Scoliosis Assessment With Optimization-Based Control and Variable Impedance
Duan, A., Victorova, M., Zhao, J., Sun, Y., Zheng, Y., & Navarro-Alarcón, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3186504 · Cited by 38
EFASPP U-Net for semantic segmentation of night traffic scenes using fusion of visible and thermal images
Shojaiee, F., & Baleghi, Y. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105627 · Cited by 34
Generalized 3-D Point Set Registration With Hybrid Mixture Models for Computer-Assisted Orthopedic Surgery: From Isotropic to Anisotropic Positional Error
Min, Z., Wang, J., Pan, J., & Meng, M. Q. ‐. (2020) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.3014420 · Cited by 35
CMPFFNet: Cross-Modal and Progressive Feature Fusion Network for RGB-D Indoor Scene Semantic Segmentation
Zhou, W., Xiao, Y., Yan, W., & Yu, L. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3313122 · Cited by 31
Segmentation of Road Negative Obstacles Based on Dual Semantic-Feature Complementary Fusion for Autonomous Driving
Feng, Z., Guo, Y., & Sun, Y. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3376534 · Cited by 34
RFIENet: RGB-thermal feature interactive enhancement network for semantic segmentation of insulator in backlight scenes
Li, B., Wang, T., Zhai, Y., & Yuan, J. (2022) · Measurement · doi:10.1016/j.measurement.2022.112177 · Cited by 33
DooDLeNet: Double DeepLab Enhanced Feature Fusion for Thermal-color Semantic Segmentation
Frigo, O., Martin-Gaffe, L., & Wacongne, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00341 · Cited by 32
Assessing thermal imagery integration into object detection methods on air-based collection platforms
Gallagher, J., & Oughton, E. J. (2023) · Scientific Reports · doi:10.1038/s41598-023-34791-8 · Cited by 39
Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss
Shin, U., Lee, K., Lee, S., & Kweon, I. S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3137895 · Cited by 32
Multi-Branch Differential Bidirectional Fusion Network for RGB-T Semantic Segmentation
Liang, W., Shan, C., Yang, Y., & Han, J. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3374793 · Cited by 30
NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation across RGB-Depth, Polarization, and Thermal Images
Yan, R., Yang, K., & Wang, K. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739390 · Cited by 29
Light Transport Induced Domain Adaptation for Semantic Segmentation in Thermal Infrared Urban Scenes
Chen, J., Liu, Z., Jin, D., Wang, Y., Yang, F., & Bai, X. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3194931 · Cited by 28
A RGB-Thermal Image Segmentation Method Based on Parameter Sharing and Attention Fusion for Safe Autonomous Driving
Li, G., Lin, Y., Ouyang, D., Li, S., Luo, X., Qu, X., Pi, D., & Li, S. E. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3332350 · Cited by 28
Multispectral Video Semantic Segmentation: A Benchmark Dataset and Baseline
Ji, W., Li, J., Bian, C., Zhou, Z., Zhao, J., Yuille, A., & Cheng, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00112 · Cited by 29
DHFNet: Decoupled Hierarchical Fusion Network for RGB-T dense prediction tasks
Chen, H., Wang, Z., Qin, H., & Mu, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127594 · Cited by 24
ECFNet: Efficient cross-layer fusion network for real time RGB-Thermal urban scene parsing
Shen, Z., Wang, J., Weng, Y., Pan, Z., Li, Y., & Wang, J. (2024) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104579 · Cited by 24
Multi-Modal Contrastive Learning for LiDAR Point Cloud Rail-Obstacle Detection in Complex Weather
Wen, L., Peng, Y., Lin, M., Gan, N., & Tan, R. (2024) · Electronics · doi:10.3390/electronics13010220 · Cited by 25
Resolving semantic conflicts in RGB-T semantic segmentation
Zhao, S., Jin, Z., Jiao, Q., Zhang, Q., & Han, J. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111398 · Cited by 22
Self-supervised Monocular Depth Estimation from Thermal Images via Adversarial Multi-spectral Adaptation
Shin, U., Park, K., Lee, B.-U., Lee, K., & Kweon, I. S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00575 · Cited by 25
MiLNet: Multiplex Interactive Learning Network for RGB-T Semantic Segmentation
Liu, J., Liu, H., Li, X., Ren, J., & Xu, X. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3544484 · Cited by 23
Visible and thermal images fusion architecture for few-shot semantic segmentation
Bao, Y., Song, K., Wang, J., Huang, L., Dong, H., & Yan, Y. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103306 · Cited by 23
Region-adaptive and context-complementary cross modulation for RGB-T semantic segmentation
Peng, F., Ding, Z., Chen, Z., Wang, G., Hui, T., Liu, S., & Shi, H. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110092 · Cited by 23
SemanticRT: A Large-Scale Dataset and Method for Robust Semantic Segmentation in Multispectral Images
Ji, W., Li, J., Bian, C., Zhang, Z., & Cheng, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611738 · Cited by 22
StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentation
Li, B., Zhang, D., Zhao, Z., Gao, J., & Li, X. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755110 · Cited by 24
Maximizing Self-Supervision From Thermal Image for Effective Self-Supervised Learning of Depth and Ego-Motion
Shin, U., Lee, K., Lee, B.-U., & Kweon, I. S. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3185382 · Cited by 24
MEFNET: Multi-expert fusion network for RGB-Thermal semantic segmentation
Lai, W., Zeng, F., Hu, X., Li, W., He, S., Liu, Z., & Jiang, Y. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106638 · Cited by 23
Rgb-t semantic segmentation based on cross-operational fusion attention in autonomous driving scenario
Zhang, J., Zhang, R., Yuan, W., & Liu, Y. (2024) · Evolving Systems · doi:10.1007/s12530-024-09567-8 · Cited by 21
Caltech Aerial RGB-Thermal Dataset in the Wild
Lee, C., Anderson, M., Ranganathan, N., Zuo, X., Do, K., Gkioxari, G., & Chung, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73036-8_14 · Cited by 23
Learning Discriminative Feature Representation for Estimating Smoke Density of Smoky Vehicle Rear
Tao, H., & Duan, Q. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3198047 · Cited by 20
Knowledge Distillation SegFormer-Based Network for RGB-T Semantic Segmentation
Zhou, W., Gong, T., & Yan, W. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3517732 · Cited by 20
Towards efficient RGB-T semantic segmentation via feature generative distillation strategy
Zhao, S., Wang, J., Zhang, Q., & Han, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103282 · Cited by 18
TiCoSS: Tightening the Coupling Between Semantic Segmentation and Stereo Matching Within a Joint Learning Framework
Tang, G., Wu, Z., Li, J., Zhong, P., Ye, W., Chen, X., Lu, H., & Fan, R. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3586286 · Cited by 21
Deformation-Resilient Multigranularity Learning for Unaligned RGB–T Semantic Segmentation
Zhou, H., Zhang, Z., Li, C., Tian, C., Xie, Y., Li, Z., & Wu, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3585105 · Cited by 19
GEBNet: Graph-Enhancement Branch Network for RGB-T Scene Parsing
Dong, S., Zhou, W., Qian, X., & Yu, L. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3219350 · Cited by 20
MS-IRTNet: Multistage information interaction network for RGB-T semantic segmentation
Zhang, Z., Liu, Y., & Xue, W. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119442 · Cited by 17
Multi-Modal 3D Object Detection in Autonomous Driving: a SurveyPreprint
Wang, Y., Mao, Q., Zhu, H., Deng, J., Zhang, Y., Ji, J., Li, H., & Zhang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12735 · Cited by 18
Central Attention and a Dual Path Convolutional Neural Network in Real-World Tree Species Recognition
Chung, Y., Chou, C.-A., & Li, C.-Y. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18030961 · Cited by 17
Online Monitoring of Iron Ore Pellet Size Distribution Using Lightweight Convolutional Neural Network
Deo, A. J., Behera, S. K., & Das, D. P. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3253574 · Cited by 17
IGFNet: Illumination-Guided Fusion Network for Semantic Scene Understanding using RGB-Thermal Images
Li, H., & Sun, Y. (2023) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio58561.2023.10354613 · Cited by 15
C⁴Net: Excavating Cross-Modal Context- and Content-Complementarity for RGB-T Semantic Segmentation
Zhao, S., Li, J., & Zhang, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3485655 · Cited by 15
Graph Representation Learning for Infrared and Visible Image Fusion
Li, J., Bai, L., Yang, B., Li, C., & Ma, L. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3557234 · Cited by 14
Multispectral state-space feature fusion: Bridging shared and cross-parametric interactions for object detection
Shen, J., Zhan, H., Dong, S., Zuo, X., Yang, W., & Ling, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103895 · Cited by 15
FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving
Liu, Y., Xu, Z., Huang, H., Wang, L., & Liu, M. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3290348 · Cited by 16
On Exploring Shape and Semantic Enhancements for RGB-X Semantic Segmentation
Yang, Y., Shan, C., Zhao, F., Liang, W., & Han, J. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3296219 · Cited by 13
A Correlation Analysis-Based Multivariate Alarm Method With Maximum Likelihood Evidential Reasoning
Weng, X., Xu, X., Feng, J., Shen, X., Meng, J., & Steyskal, F. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3305524 · Cited by 13

Authors: Sun, Y., Zuo, W., Yun, P., Wang, H., & Liu, M.
Affiliations: Hong Kong University of Science and Technology (HK)
ORCID: Sun, Yuxiang
Zuo, Weixun
Yun, Peng
Wang, Hengli
Liu, Ming
Venue: IEEE Transactions on Automation Science and Engineering
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2021-07
Volume: 18
Issue: 3
Pages: 1000-1011
ISSN: 1545-5955 (ISSN-L)
1558-3783
DOI: 10.1109/tase.2020.2993143
OpenAlex: W3033020497
Cited by: 270
References: 60
Citations by year: 2026: 39 · 2025: 74 · 2024: 50 · 2023: 56 · 2022: 31 · 2021: 19
FWCI: 10.32
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Transactions on Automation Science and Engineering (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: National Natural Science Foundation of China (U1713211)
Research Grant Council of Hong Kong (11210017)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
Keywords: semantic segmentation, urban scenes, infrared thermography, autonomous driving, multimodal sensor fusion, low-light conditions, end-to-end neural network, obstacle detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

FuseSeg: Semantic Segmentation of Urban Scenes Based on RGB and Thermal Data Fusion

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks