Cited by 731Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMultimodal Machine Learning Applications

The DOI matches its Crossref record; 106 of the 107 references it cites were found in a registry or an index.

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

Open access

Zhang, Jiaming, Liu, Huayao, Yang, Kailun, Hu, Xinxin, Liu, Ruiping, Stiefelhagen, Rainer·2023·IEEE Transactions on Intelligent Transportation Systems

DOI 10.1109/tits.2023.3300537

Open this paper in CoChat — free account Open PDF

Abstract

Scene understanding based on image segmentation is a crucial component of autonomous vehicles. Pixel-wise semantic segmentation of RGB images can be advanced by exploiting complementary features from the supplementary modality (${X}$-modality). However, covering a wide variety of sensors with a modality-agnostic model remains an unresolved problem due to variations in sensor characteristics among different modalities. Unlike previous modality-specific methods, in this work, we propose a unified fusion framework, CMX, for RGB-X semantic segmentation. To generalize well across different modalities, that often include supplements as well as uncertainties, a unified cross-modal interaction is crucial for modality fusion. Specifically, we design a Cross-Modal Feature Rectification Module (CM-FRM) to calibrate bi-modal features by leveraging the features from one modality to rectify the features of the other modality. With rectified feature pairs, we deploy a Feature Fusion Module (FFM) to perform sufficient exchange of long-range contexts before mixing. To verify CMX, for the first time, we unify five modalities complementary to RGB, i.e., depth, thermal, polarization, event, and LiDAR. Extensive experiments show that CMX generalizes well to diverse multi-modal fusion, achieving state-of-the-art performances on five RGB-Depth benchmarks, as well as RGB-Thermal, RGB-Polarization, and RGB-LiDAR datasets. Besides, to investigate the generalizability to dense-sparse data fusion, we establish an RGB-Event semantic segmentation benchmark based on the EventScape dataset, on which CMX sets the new state-of-the-art. The source code of CMX is publicly available athttps://github.com/huaaaliu/RGBX_Semantic_Segmentation.

Integrity signals

Registry

Crossref · 10.1109/tits.2023.3300537

Cites

106 of 106 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054

Show all 107 references

Research works printed but not linked: 1

Dosovitskiy, An image is worth 16✗16 words: Transformers for image recognition at scale, 1

Sorted by citations

Items 201–300 of 731

OCCO: LVM-Guided Infrared and Visible Image Fusion Framework Based on Object-Aware and Contextual Contrastive Learning
Li, H., Bian, C., Zhang, Z., Song, X., Li, X., & Wu, X. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02507-2 · Cited by 9
AgriFusion: Multiscale RGB–NIR Fusion for Semantic Segmentation in Airborne Agricultural Imagery
Li, X., Qiao, L., & Yang, C. (2025) · AgriEngineering · doi:10.3390/agriengineering7110388 · Cited by 8
LF-DETR: A Laplacian Frequency Enhanced DETR for Aerial RGB-Infrared Pedestrian Detection
Qi, H., Qin, H., Xiang, X., Yang, C., & Tan, Y. (2026) · Remote Sensing · doi:10.3390/rs18030531 · Cited by 8
FTransMamba: A multi-stage fusion transformer and mamba modeling for multimodal remote sensing scene understanding
Li, Z., Hu, J., Guan, W., Miao, J., Wu, K., Xia, Z., Bin, Y. A. N. G., & Wu, J. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113625 · Cited by 8
Stronger, Steadier & Superior: Geometric Consistency in Depth VFM Forges Domain Generalized Semantic Segmentation
Chen, S., Han, T., Zhang, C., Luo, X. R., Wu, M., Cai, G., & Su, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00776 · Cited by 9
CAFseg: A Semantic segmentation network with cross aggregation fusion strategy for RGB-thermal semantic segmentation
Yi, S., Wu, L., Liu, X., Li, J., & Jiang, G. (2023) · Infrared Physics & Technology · doi:10.1016/j.infrared.2023.105077 · Cited by 8
Deep-Learning-Based Automated Building Information Modeling Reconstruction Using Orthophotos with Digital Surface Models
Wang, D., Jiang, Q., & Liu, J. (2024) · Buildings · doi:10.3390/buildings14030808 · Cited by 8
Temporal Consistency for RGB-Thermal Data-Based Semantic Scene Understanding
Li, H., Chu, H. K., & Sun, Y. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3458594 · Cited by 8
Learning Modality-Agnostic Representation for Semantic Segmentation from Any Modalities
Xu, Z., Lyu, Y., & Wang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72754-2_9 · Cited by 14
Towards Robust Semantic Segmentation of Land Covers in Foggy Conditions
Shi, W., Qin, W., & Chen, A. (2022) · Remote Sensing · doi:10.3390/rs14184551 · Cited by 8
Interactive Efficient Multi-Task Network for RGB-D Semantic Segmentation
Xu, X., Liu, J., & Liu, H. (2023) · Electronics · doi:10.3390/electronics12183943 · Cited by 8
Self-citation
Realtime Multispectral Pedestrian Detection With Visible and Far-Infrared Under Ambient Temperature Changing
Okuda, M., Yoshida, K., & Fujino, T. (2024) · IEEE Open Journal of Intelligent Transportation Systems · doi:10.1109/ojits.2024.3507917 · Cited by 7
HMFENet: Hierarchical Matching Guided Feature Enhancement Network for Few-Shot RGB-Thermal Urban Scene Segmentation
Zhou, X., Ma, Y., Huang, J., Cai, Z., Zhu, Y., & Fan, F. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3549516 · Cited by 7
DepthMatch: Semi-Supervised RGB-D Scene Parsing Through Depth-Guided Regularization
Huang, J., Li, J., Vityazev, S. V., Dvorkovich, A., & Fan, R. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3575640 · Cited by 7
A novel coal-rock recognition method in coal mining face based on fusing laser point cloud and images
Liu, Y., Si, L., Wang, Z., Chen, M., Li, X., Wei, D., & Gu, J. (2025) · International Journal of Mining Science and Technology · doi:10.1016/j.ijmst.2025.05.009 · Cited by 7
MM5: Multimodal image capture and dataset generation for RGB, depth, thermal, UV, and NIR
Brenner, M., Reyes, N. H., Sušnjak, T., & Barczak, A. L. C. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103516 · Cited by 7
DCIFNet: Cross-Modal Fusion With Correction and Interaction for Optical–SAR Land Cover Classification
Ren, B., Liu, B., Wang, Q., Hou, B., Yang, C., & Jiao, L. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3609898 · Cited by 7
Guiding Robotic Cloth Grasping in Darkness: Infrared Semantic Segmentation and Grasping Position Selection
Zhu, X., Zhong, H.-F., Wu, Y. J., Luo, S., & Gao, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3632718 · Cited by 7
Deep learning and remote sensing for scalable building age prediction in urban energy modeling
Li, Q., Xu, G., & Gu, Z. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.116303 · Cited by 7
PAD: Phase–Amplitude Decoupling Fusion for Multimodal Land Cover Classification
Zheng, H., Zhong, X., Liu, B., Xiao, Y., Wen, B., & Li, X. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3621902 · Cited by 7
DCAM-DETR: Dual Cross-Attention Mamba Detection Transformer for RGB–Infrared Anti-UAV Detection
Qin, Z., & Li, Y. (2026) · Information · doi:10.3390/info17010103 · Cited by 7
Filtering and regret network for spacecraft component segmentation based on gray images and depth maps
Liu, X., Hongyuan, W. A. N. G., Zijian, W. A. N. G., Chen, X., Chen, W., & Xie, Z. (2024) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2024.03.007 · Cited by 7
Transferring Multi-Modal Domain Knowledge to Uni-Modal Domain for Urban Scene Segmentation
Liu, P., Ge, Y., Duan, L., Li, W., Luo, H., & Lv, F. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3382880 · Cited by 8
Interdimensional Knowledge Transfer for Semantic Segmentation on LiDAR Point Clouds
Ha, S., Kim, Y.-H., & Park, J. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3426372 · Cited by 7
Automated Detection of Hillforts in Remote Sensing Imagery With Deep Multimodal Segmentation
Canedo, D., Fonte, J., Dias, R., do Pereiro, T., Gonçalves‐Seco, L., Vázquez, M., Georgieva, P., & Neves, A. J. R. (2024) · Archaeological Prospection · doi:10.1002/arp.1958 · Cited by 8
LMF-Net: A Learnable Multimodal Fusion Network for Semantic Segmentation of Remote Sensing Data
Li, J., Zhang, W., Zhang, W., Zhou, R., Li, C., Tong, B., Sun, X., & Fu, K. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3527213 · Cited by 9
Transferring Prior Thermal Knowledge for Snowy Urban Scene Semantic Segmentation
Guo, X., Liu, T., Mou, Y., Chai, S., Ren, B., Wang, Y., Shi, W., Liu, S., & Zhou, W. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3555617 · Cited by 7
MorFormer: Morphology-Aware Transformer for Generalized Pavement Crack Segmentation
Guo, X., Tang, W., Wang, H., Wang, J., Wang, S., Qu, X., & Lin, X. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3558782 · Cited by 7
HAPNet: Toward superior RGB-thermal scene parsing via hybrid, asymmetric, and progressive heterogeneous feature fusion
Li, J., Peng, Y., Xu, Y., Zhang, Y., Sun, M., Chen, Q., Alexander, I., & Fan, R. (2026) · Biomimetic Intelligence and Robotics · doi:10.1016/j.birob.2026.100309 · Cited by 12
DepthFormer: Multimodal Positional Encodings and Cross-Input Attention for Transformer-based Segmentation Networks
Barbato, F., Rizzoli, G., & Zanuttigh, P. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096314 · Cited by 7
SMALNet: Segment Anything Model Aided Lightweight Network for Infrared Image Segmentation
Ding, K., Xiang, S., & Pan, C. (2024) · Infrared Physics & Technology · doi:10.1016/j.infrared.2024.105540 · Cited by 6
Multi-view Feature Extraction via Tunable Prompts is Enough for Image Manipulation Localization
Liu, X., Yang, Y., Wang, H., Ying, Q., Qian, Z., Zhang, X., & Li, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681137 · Cited by 8
VSLNet: Multimodal Data Fusion Network for Tree Species Classification in Overhead Transmission Line Corridors
Du, R., Zhang, H., Huang, Z., Zhong, H., Cao, Y., & Wang, Y. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3459014 · Cited by 6
Frequency-Aware Integrity Learning Network for Semantic Segmentation of Remote Sensing Images
Yang, P., Zhou, W., & Liu, Y. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3524753 · Cited by 6
Multi-modal Salient Object Detection via a Unified Diffusion Model
Zhang, S., Huang, J., Tang, W., Tian, L., Wei, Y., & Liu, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887966 · Cited by 7
Self-Supervised Visual Odometry Based on Scene Appearance-Structure Incremental Fusion
Fu, F., Yang, J., Ma, J., & Zhang, J. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3559077 · Cited by 6
Self-citation
A Full-Lifecycle Calibration Method for Camera and LiDAR in Autonomous Driving
Wei, M., Lu, J., Ye, N., Zhu, Z. H., Zhang, Q., & Wang, Y. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3565783 · Cited by 6
Robust RGB-T Object Detection via Quality-Aware Decision Fusion Based on Dempster–Shafer Theory
Yu, H., Luo, R., Wang, X., Wang, D., Li, Q., & Qiao, S. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3566381 · Cited by 6
RT-X Net: RGB-Thermal Cross Attention Network for Low-Light Image Enhancement
Jha, R., Lenka, A., Ramanagopal, M. S., Sankaranarayanan, A. C., & Mitra, K. (2025) · Proceedings - International Conference on Image Processing · doi:10.1109/icip55913.2025.11084466 · Cited by 7
MaskAdapt: Unsupervised Geometry-Aware Domain Adaptation Using Multimodal Contextual Learning and RGB-Depth Masking
Nadeem, N., Asad, M., Anwar, S., & Bais, A. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00539 · Cited by 6
Multi-modal mamba framework for RGB-T crowd counting with linear complexity
Gan, C.-L., Jia, R., Sun, H.-M., & Song, Y.-C. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112522 · Cited by 6
Unleashing the Power of CNN and Transformer for Balanced RGB-event Video Recognition
Wang, X., Rong, Y., Wang, S., Chen, Y., Wu, Z., Jiang, B., Tian, Y., & Tang, J. (2025) · Machine Intelligence Research · doi:10.1007/s11633-025-1555-3 · Cited by 11
HR-MM Segformer: Enhancing land use and land cover semantic segmentation through transformer-based multisource remote sensing feature fusion
Fan, J., Shi, Z., Du, Y., & Zhuang, C. (2025) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2025.106848 · Cited by 6
HCL-Net: Heterogeneous Collaborative Learning for Lightweight Remote Sensing Image Segmentation
Xie, J., Zhou, W., Xu, C., Liu, Y., & Qiang, F. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3652161 · Cited by 6
Enhancing RGB-IR object detection: a frozen backbone approach with multi-receptive field attention
Lu, B., Liu, H., & Watanabe, H. (2026) · The Visual Computer · doi:10.1007/s00371-026-04378-1 · Cited by 6
Self-citation
Adaptive interactive network for RGB-T salient object detection with double mapping transformer
Dong, F. D., Wang, Y., Zhu, J., & Li, Y. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17747-y · Cited by 6
Segment all roads: Domain generalized freespace detection by robust surface normal information embedding and edge-aware learning
Liu, G., Tian, S., Liao, M., Zhang, Y., Zou, W., & Li, X. (2024) · Displays · doi:10.1016/j.displa.2024.102715 · Cited by 6
Learning Virtual View Selection for 3D Scene Semantic Segmentation
Mu, T., Shen, M.-Y., Lai, Y., & Hu, S. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3421952 · Cited by 6
A comprehensive research on light field imaging: Theory and application
Liu, F., Wang, Y., Yang, Q., Zhou, S., & Zhang, K. (2024) · IET Computer Vision · doi:10.1049/cvi2.12321 · Cited by 6
Trans4Map: Revisiting Holistic Bird’s-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers
Chen, C., Zhang, J., Yang, K., Peng, K., & Stiefelhagen, R. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00400 · Cited by 6
Self-citation
Simple Scalable Multimodal Semantic Segmentation Model
Zhu, Y., & Xiao, N. (2024) · Sensors · doi:10.3390/s24020699 · Cited by 5
Cosine similarity fusion network for real-time Red-Green-Blue and X-modality semantic segmentation of driving scenes
Qashqai, D., Mousavian, E., Shokouhi, S. B., & Mirzakuchaki, S. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114362 · Cited by 12
HDBFormer: Efficient RGB-D Semantic Segmentation With a Heterogeneous Dual-Branch Framework
Wei, S., Zhou, Z., Lu, Z., Yuan, Z., & Su, B. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3496588 · Cited by 5
NTFNet: Narrowing-Then-Fusing network for RGB-TIR semantic segmentation
Liu, Y., Ye, J., He, W., Qu, Z., & Xu, R. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06411-7 · Cited by 5
Modality-Incremental Learning with Disjoint Relevance Mapping Networks for Image-Based Semantic Segmentation
Hegde, N., Muralidhara, S., Schuster, R., & Stricker, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00541 · Cited by 5
Phenotypic dynamics and temporal heritability of tomato architectural traits using an unmanned ground vehicle-based plant phenotyping system
Xie, P., Yang, X., Fang, L., Wang, T., Zheng, J., Jiang, Y., & Cen, H. (2025) · Horticulture Research · doi:10.1093/hr/uhaf109 · Cited by 5
MGFNet: a multiscale gated fusion network for multimodal semantic segmentation
Wu, H., Li, Z., & Wen, Z. (2025) · The Visual Computer · doi:10.1007/s00371-025-03912-x · Cited by 6
ComPtr: Toward Diverse Bi-Source Dense Prediction Tasks via a Simple Yet General Complementary Transformer
Pang, Y., Zhao, X., Zhang, L., & Lu, H. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3578494 · Cited by 6
Multimodal Fusion Network for Power Tower Semantic Segmentation and Inclination Detection
Zhang, H., Liu, Y., Zhong, H., Huang, Z., Du, R., Tang, Y., Cao, Y., & Wang, Y. (2025) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2025.3578133 · Cited by 5
ADBNet: Asymmetric dual-branch network for indoor real-time RGB-D semantic segmentation
Xu, C., Ma, G., Gao, F., Wang, B., & Liu, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113885 · Cited by 5
Polarization-driven camouflaged object detection: a multimodal fusion network with iterative polarimetric feature enhancement
Zhang, X., Ru, J., Wang, Y., & Wu, C. (2025) · Applied Optics · doi:10.1364/ao.570214 · Cited by 5
DiFusionSeg: Diffusion-driven semantic segmentation with multi-modal image fusion for enhanced perception
Wang, Z., He, D., Zhao, L., Liu, B., Zheng, Y., & Zhang, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114481 · Cited by 7
USformer: A U-Shaped Structure Transformer for RGB-Thermal Semantic Segmentation and Traffic Scene Understanding
Yang, F., Shao, F., Mu, B., Chai, X., & Jiang, Q. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3624821 · Cited by 5
Unveiling the Potential of Segment Anything Model 2 for RGB-Thermal Semantic Segmentation with Language Guidance
Zhao, J., Teng, F., Luo, K., Zhao, G., Li, Z., Xu, Z., & Yang, K. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247337 · Cited by 5
Self-citation
Mul-VMamba: Multimodal semantic segmentation using selection-fusion-based vision-Mamba
Ni, R., Guo, Y., Yang, B., Liu, Y., Wang, H., & Hu, C. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115119 · Cited by 5
WHFNet: A Wavelet-Driven Heterogeneous Fusion Network for High-Frequency Enhanced Optical–SAR Remote Sensing Segmentation
Ren, B., Wang, Q., Liu, B., Hou, B., Yang, C., & Jiao, L. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3662689 · Cited by 5
Spatially-Guided Temporal Aggregation for Robust Event-RGB Optical Flow Estimation
Zhou, Q., Hou, J., Yang, M., Deng, Y., Li, Y., & Xiong, J. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3664962 · Cited by 5
SDDS-Net: Space and Depth Encoder-Decoder Convolutional Neural Networks for Real-Time Semantic Segmentation
Ibrahem, H., Salem, A., & Kang, H. (2023) · IEEE Access · doi:10.1109/access.2023.3327323 · Cited by 5
Improving Deep Learning-based Saliency Detection Using Channel Attention Module
Farsi, H., Ghermezi, D., Barati, A., & Mohamadzadeh, S. (2024) · International Journal of Engineering · doi:10.5829/ije.2024.37.11b.21 · Cited by 5
AMATrack: A Unified Network With Asymmetric Multimodal Mixed Attention for RGBD Tracking
Ye, P., Xiao, G., & Liu, J. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3421435 · Cited by 5
HEFANet: hierarchical efficient fusion and aggregation segmentation network for enhanced rgb-thermal urban scene parsing
Shen, Z., Pan, Z., Weng, Y., Li, Y., Wang, J., & Wang, J. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05743-0 · Cited by 5
Self-Enhanced Feature Fusion for RGB-D Semantic Segmentation
Xiang, P., Yao, B., Jiang, Z., & Peng, C. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3475352 · Cited by 5
Trajectory Prediction Model of Electric Vehicle Autonomous Driving Based on Hybrid Attention Transformer Network
Wang, B., Liu, Y., Wang, R., & Sun, Q. (2025) · IET Intelligent Transport Systems · doi:10.1049/itr2.70022 · Cited by 5
CFANet: The Cross-Modal Fusion Attention Network for Indoor RGB-D Semantic Segmentation
Wu, L., Wei, D., & Xu, C. (2025) · Journal of Imaging · doi:10.3390/jimaging11060177 · Cited by 5
An Active Task Cognition Method for Home Service Robot Using Multi-Graph Attention Fusion Mechanism
Cui, Y., Tian, G., Jiang, Z., Zhang, M., Gu, Y., & Wang, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3339292 · Cited by 5
A Foreground-Driven Fusion Network for Gully Erosion Extraction Utilizing UAV Orthoimages and Digital Surface Models
Shen, Y., Su, N., Zhao, C., Yan, Y., Feng, S., Liu, Y., & Xiang, W. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3417398 · Cited by 5
Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation
Liu, R., Zhang, J., Peng, K., Chen, Y., Cao, K., Zheng, J., Sarfraz, M. S., Yang, K., & Stiefelhagen, R. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588722 · Cited by 5
Self-citation
Open-Vocabulary RGB-Thermal Semantic Segmentation
Zhao, G., Huang, J., Yan, X., Wang, Z., Tang, J., Ou, Y., Hu, X., & Peng, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72904-1_18 · Cited by 6
Self-citation
Improving RGB-Thermal Semantic Scene Understanding With Synthetic Data Augmentation for Autonomous Driving
Li, H., Chu, H. K., & Sun, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3548399 · Cited by 6
GSGNet-S*: Graph Semantic Guidance Network via Knowledge Distillation for Optical Remote Sensing Image Scene Analysis
Zhou, W., Li, Y., Huang, J., Yan, W., Fang, M., & Jiang, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3332336 · Cited by 5
Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation
Liu, Y., Zhang, W., Zhao, G., Zhu, J., Vasilakos, A. V., & Wang, L. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3336611 · Cited by 5
Csfnet: A Cosine Similarity Fusion Network for Real-Time Rgb-X Semantic Segmentation of Driving ScenesPreprint
Qashqai, D., Mousavian, E., Shokouhi, S. B., & Mirzakuchaki, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4885740 · Cited by 5
MUSeg: A multimodal semantic segmentation dataset for complex underground mine scenes
Li, S., Kong, Q., Gao, X., Shi, F., Li, L., Zhang, Q., Wang, P., & Yang, K. (2025) · Scientific Data · doi:10.1038/s41597-025-05493-9 · Cited by 5
Self-citation
PDDM: Pseudo Depth Diffusion Model for RGB-PD Semantic Segmentation Based in Complex Indoor Scenes
Xu, X., Liu, H., Wu, J., & Liu, J. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i9.32970 · Cited by 4
Self-citation
A novel knowledge-based multi-modal semi-supervised framework for 3D change detection and mining volume estimation in open-pit mines using GF7 satellite images
Dong, D., Ming, D., Li, M., Xu, H., Wei, Y., & Huang, M. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.07.034 · Cited by 5
A Fine-Scale Segmentation Method for Individual Rubber Trees Based on UAV LiDAR Point Cloud
Li, L., Ye, Z., Li, H., Yan, M., Zhou, G., Wang, X., Wang, H., & Lv, M. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3593292 · Cited by 4
Hybrid long-range feature fusion network for multi-modal waste semantic segmentation
Li, Y., & Zhang, X. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103608 · Cited by 4
Bimodal defect segmentation with Geometric Prior-supported Anti-imbalance Learning for pavement defect evaluation and repair
Wang, Y., Song, K., Liu, Y., Li, T., Yan, Y., & Carneiro, G. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106497 · Cited by 4
Prior knowledge-informed semantic segmentation framework for precise glacial lake mapping from multimodal imagery
Tan, H., Jiang, L., Liu, H., Zhang, T., & Cheng, I. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.09.022 · Cited by 4
Self-citation
CFIFNet: A Road Extraction Network Through Cross-Modal Frequency-Domain Interaction and Fusion From High-Resolution Remote Sensing Image and GPS Trajectory/LiDAR
Zhou, Y., Cai, W., Gao, L., & Hao, X. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3625550 · Cited by 4
Multiple cross-modal complementation network for lightweight RGB-D salient object detection
Zhang, C., Chen, F., Huang, L. F., Peng, Z., & Hu, X. (2025) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2025.104622 · Cited by 4
Self-citation
Bridging RGB-T image fusion and semantic segmentation via multi-task collaborative learning
Lu, T., Wu, H., Fu, W., Fang, L., & Shutao Li, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103988 · Cited by 4
Cross-modal edge-enhanced detector for UAV-based multispectral object detection
Li, G., Ren, G., Wang, J., Zhi, M., Yu, Z., Jiang, B., Guan, H., & Guo, Q. (2025) · Scientific Reports · doi:10.1038/s41598-025-30786-9 · Cited by 4
Multimodal Collaborative Interactive Soft Fusion Network for RGB-Infrared Aerial Image Object Detection
Cheng, B., Xu, B., Gan, W., & Wang, Q. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3648023 · Cited by 4
Adapt, Generate, and Supervise: Geometry-Aware Diffusion-Guided SAM Framework for Remote Sensing Semantic Segmentation
Zhou, W., Xie, J., Xu, C., Liu, Y., & Wang, Y. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3653952 · Cited by 4
Dynamic background-guided asymmetric knowledge distillation network for 3D defect detection
Zhang, F., Hu, W., Wang, Y., & Duan, J. (2026) · Journal of Manufacturing Processes · doi:10.1016/j.jmapro.2026.01.062 · Cited by 4
Reliability aware multimodal detection of active landslides via evidence theoretic fusion of InSAR, optical, and DEM data
Fan, Y., Zhang, H., Xu, Z., Zhang, Y., Guo, Z., Guan, S., Duan, H., & Tang, Y. (2026) · Computers & Geosciences · doi:10.1016/j.cageo.2026.106139 · Cited by 4
Explainable multi-granularity attribution reasoning framework for fake news detection
Ji, W., Lv, H., Zhao, H., & Zimmermann, R. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00093-3 · Cited by 5
Multimodal Image Representation Learning With Limited Visual-Tactile Data
Qiu, L., Da, H., Liu, W., Niu, Y., Wang, H., & Zhao, T. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3678361 · Cited by 4
Domain Adaptation Transformer for Unsupervised Driving-Scene Segmentation in Adverse Conditions
Liu, W., Wang, S., Zhu, J., Xie, X., & Zhang, L. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3461468 · Cited by 4

Authors: Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R.
Affiliations: Karlsruhe Institute of Technology (DE)
Nio (China) (CN)
Hunan University (CN)
ByteDance (China) (CN)
ORCID: Zhang, Jiaming
Yang, Kailun
Hu, Xinxin
Liu, Ruiping
Stiefelhagen, Rainer
Venue: IEEE Transactions on Intelligent Transportation Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2023-12
Volume: 24
Issue: 12
Pages: 14679-14694
ISSN: 1524-9050 (ISSN-L)
1558-0016
DOI: 10.1109/tits.2023.3300537
OpenAlex: W4386179772
Cited by: 731
References: 107
Citations by year: 2026: 225 · 2025: 268 · 2024: 178 · 2023: 53
FWCI: 65.71
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Transactions on Intelligent Transportation Systems (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Federal Ministry of Labor and Social Affairs (BMAS) through the AccessibleMaps Project (01KM151112)
“KIT Future Fields” Project
Ministry of Science, Research and the Arts of Baden-Wurttemberg (MWK) through the Cooperative Graduate School Accessibility through AIBased Assistive Technology (BW6-03)
Federal Ministry of Education and Research (BMBF) through the Fellowship within the IFI Program of the German Academic Exchange Service
HoreKA@KIT Supercomputer Partition
Hangzhou SurImage Technology Company Ltd
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: cross-modal fusion, multimodal fusion, depth images, thermal images, polarization images, event cameras, LiDAR, transformer models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks