Cited by 731Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMultimodal Machine Learning Applications

The DOI matches its Crossref record; 106 of the 107 references it cites were found in a registry or an index.

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

Open access

Zhang, Jiaming, Liu, Huayao, Yang, Kailun, Hu, Xinxin, Liu, Ruiping, Stiefelhagen, Rainer·2023·IEEE Transactions on Intelligent Transportation Systems

DOI 10.1109/tits.2023.3300537

Open this paper in CoChat — free account Open PDF

Abstract

Scene understanding based on image segmentation is a crucial component of autonomous vehicles. Pixel-wise semantic segmentation of RGB images can be advanced by exploiting complementary features from the supplementary modality (${X}$-modality). However, covering a wide variety of sensors with a modality-agnostic model remains an unresolved problem due to variations in sensor characteristics among different modalities. Unlike previous modality-specific methods, in this work, we propose a unified fusion framework, CMX, for RGB-X semantic segmentation. To generalize well across different modalities, that often include supplements as well as uncertainties, a unified cross-modal interaction is crucial for modality fusion. Specifically, we design a Cross-Modal Feature Rectification Module (CM-FRM) to calibrate bi-modal features by leveraging the features from one modality to rectify the features of the other modality. With rectified feature pairs, we deploy a Feature Fusion Module (FFM) to perform sufficient exchange of long-range contexts before mixing. To verify CMX, for the first time, we unify five modalities complementary to RGB, i.e., depth, thermal, polarization, event, and LiDAR. Extensive experiments show that CMX generalizes well to diverse multi-modal fusion, achieving state-of-the-art performances on five RGB-Depth benchmarks, as well as RGB-Thermal, RGB-Polarization, and RGB-LiDAR datasets. Besides, to investigate the generalizability to dense-sparse data fusion, we establish an RGB-Event semantic segmentation benchmark based on the EventScape dataset, on which CMX sets the new state-of-the-art. The source code of CMX is publicly available athttps://github.com/huaaaliu/RGBX_Semantic_Segmentation.

Integrity signals

Registry

Crossref · 10.1109/tits.2023.3300537

Cites

106 of 106 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054

Flagged first, then by citations

Show all 107 references

1 of the 107 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

Dosovitskiy, An image is worth 16✗16 words: Transformers for image recognition at scale, 1

Sorted by citations

Items 101–200 of 731

TiCoSS: Tightening the Coupling Between Semantic Segmentation and Stereo Matching Within a Joint Learning Framework
Tang, G., Wu, Z., Li, J., Zhong, P., Ye, W., Chen, X., Lu, H., & Fan, R. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3586286 · Cited by 21
Deformation-Resilient Multigranularity Learning for Unaligned RGB–T Semantic Segmentation
Zhou, H., Zhang, Z., Li, C., Tian, C., Xie, Y., Li, Z., & Wu, X. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3585105 · Cited by 19
Harmonized Domain Enabled Alternate Search for Infrared and Visible Image Alignment
Jiang, Z., Zhang, Z., & Liu, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3607585 · Cited by 18
Rethinking Multi-Modal Object Detection From the Perspective of Mono-Modality Feature Learning
Zhao, T., Liu, B., Gao, Y., Sun, Y., Yuan, M., & Wei, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00600 · Cited by 18
Early or Late Fusion Matters: Efficient RGB-D Fusion in Vision Transformers for 3D Object Recognition
Tziafas, G., & Kasaei, H. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341422 · Cited by 25
Source-Free Domain Adaptation for RGB-D Semantic Segmentation with Vision Transformers
Rizzoli, G., Shenaj, D., & Zanuttigh, P. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00070 · Cited by 20
SDPT: Semantic-Aware Dimension-Pooling Transformer for Image Segmentation
Cao, H., Chen, G., Zhao, H., Jiang, D., Zhang, X., Tian, Q. C., & Knoll, A. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3417813 · Cited by 19
Satellite remote sensing and bathymetry co-driven deep neural network for coral reef shallow water benthic habitat classification
Chen, H., Cheng, J., Ruan, X., Li, J., Li, Y., Chu, S., Cheng, L., & Ka, Z. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.104054 · Cited by 18
PEAFusion: Parameter-efficient Adaptation for RGB-Thermal fusion-based semantic segmentation
Wang, Y., Chu, H. K., & Sun, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103030 · Cited by 19
Joint dual-stream interaction and multi-scale feature extraction network for multi-spectral pedestrian detection
Hu, W., Fu, C., Cao, R., Zang, Y., Wu, X., Shen, S., & Gao, X. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110768 · Cited by 17
MSTNet-KD: Multilevel Transfer Networks Using Knowledge Distillation for the Dense Prediction of Remote-Sensing Images
Zhou, W., Li, Y., Huan, J., Liu, Y., & Jiang, Q. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3384669 · Cited by 18
Single Frame Semantic Segmentation Using Multi-Modal Spherical Images
Guttikonda, S., & Rambach, J. R. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00319 · Cited by 17
M2VAD : Multiview multimodality transformer-based weakly supervised video anomaly detection
Paulraj, S., & Subramaniyaswamy, V. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105139 · Cited by 17
Polarization-Aware Low-Light Image Enhancement for Nighttime Intelligent Vehicles
Lu, Y., Tian, J., Su, Y., Liu, J., Hao, C., Yue, C., & Yang, X. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3440308 · Cited by 17
Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation
Zhuge, Y., Gu, H., Zhang, L., Qi, J., & Lu, H. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3418980 · Cited by 16
Polarization-based Camouflaged Object Detection with high-resolution adaptive fusion Network
Wang, X., Xu, J., & Ding, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110245 · Cited by 16
LIX: Implicitly Infusing Spatial Geometric Prior Knowledge Into Visual Semantic Segmentation for Autonomous Driving
Guo, S., Long, Z., Wu, Z., Chen, Q., Pitas, I., & Fan, R. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3618378 · Cited by 16
HDC-Net:A multimodal remote sensing semantic segmentation network with hierarchical dual-stream fusion and cross-token interaction
Li, Z., Zhang, Y. J., Hu, J., Wu, K., Miao, J., Wu, J., Li, Z., Yang, B., & Xia, Z. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115416 · Cited by 18
An Efficient RGB-D Indoor Scene-Parsing Solution via Lightweight Multiflow Intersection and Knowledge Distillation
Zhou, W., Zhang, Y., Yan, W., & Ye, L. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3400030 · Cited by 16
Attention-based fusion network for RGB-D semantic segmentation
Li, Z., Guo, C., Zhan, J., & Deng, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128371 · Cited by 16
Local-to-Global Cross-Modal Attention-Aware Fusion for HSI-X Semantic Segmentation
Zhang, X., Yokoya, N., Gu, X., Tian, Q., & Bruzzone, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3465030 · Cited by 18
Cross-Modal Feature Fusion for Field Weed Mapping Using RGB and Near-Infrared Imagery
Fan, X., Ge, C., Yang, X., & Wang, W. (2024) · Agriculture · doi:10.3390/agriculture14122331 · Cited by 16
Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Images
Mei, J., Lin, C., Qiu, Y., Wang, Y., Zhang, H., Wang, Z., & Dai, D. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01459 · Cited by 15
Real-Time Passable Area Segmentation With Consumer RGB-D Cameras for the Visually Impaired
Zou, W., Hua, G., Zhuang, Y., & Tian, S. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3272369 · Cited by 15
Effective Dual-Feature Fusion Network for Transmission Line Detection
Zhou, W., Ji, C., & Fang, M. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3333322 · Cited by 15
DDViT: Advancing lithology identification on FMI image logs through a dual modal transformer model with less information drop
Hou, L. X., Ma, C., Tang, W., Zhou, Y., Ye, S., Chen, X., Zhang, X., Yu, C., Chen, A., Zheng, D., et al. (2024) · Geoenergy Science and Engineering · doi:10.1016/j.geoen.2024.212662 · Cited by 16
UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All
Lyu, Y., Xu, Z., Zhou, J., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02526 · Cited by 20
Memory-efficient cross-modal attention for RGB-X segmentation and crowd counting
Zhang, Y., Choi, S., & Hong, S. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111376 · Cited by 15
Transformer-Based RGB-T Tracking With Channel and Spatial Feature Fusion
Li, Y., Wang, B., & Li, Y. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3579339 · Cited by 15
Reflectance-Guided Progressive Feature Alignment Network for All-Day UAV Object Detection
Zhao, Z., Zhang, W., Xiao, Y., Li, C., & Tang, J. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3574963 · Cited by 14
Multiattentive Perception and Multilayer Transfer Network Using Knowledge Distillation for RGB-D Indoor Scene Parsing
Zhou, W., Jian, B., Liu, Y., & Jiang, Q. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3575088 · Cited by 14
Semantic Prompt and Graph-Convolution-Structure Distillation Framework for Semantic Segmentation of Remote Sensing Images
Zhou, W., Xie, J., & Xu, C. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3675381 · Cited by 14
SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events
Pan, T., Cao, Z., & Wang, L. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610921 · Cited by 17
C⁴Net: Excavating Cross-Modal Context- and Content-Complementarity for RGB-T Semantic Segmentation
Zhao, S., Li, J., & Zhang, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3485655 · Cited by 15
Vision Foundation Model Guided Multimodal Fusion Network for Remote Sensing Semantic Segmentation
Chen, P., Fan, X., Tjahjadi, T., Guan, H., Fu, L., Ye, Q., & Wang, R. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3547880 · Cited by 14
Multispectral state-space feature fusion: Bridging shared and cross-parametric interactions for object detection
Shen, J., Zhan, H., Dong, S., Zuo, X., Yang, W., & Ling, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103895 · Cited by 15
Adaptive dual cross-attention network for multispectral object detection in autonomous driving
Zhang, J., Song, X., Li, Y., Liang, D., Zhang, Z., & Cai, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132012 · Cited by 13
Self-citation
On Exploring Shape and Semantic Enhancements for RGB-X Semantic Segmentation
Yang, Y., Shan, C., Zhao, F., Liang, W., & Han, J. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3296219 · Cited by 13
Indoor semantic segmentation based on Swin-Transformer
Zheng, Y., Xu, Y., Shu, S., & Sarem, M. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103991 · Cited by 13
Bimodal SegNet: Fused instance segmentation using events and RGB frames
Kachole, S., Huang, X., Naeini, F. B., Muthusamy, R., Makris, D., & Zweiri, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110215 · Cited by 13
Morphology-Guided Network via Knowledge Distillation for RGB-D Mirror Segmentation
Zhou, W., Cai, Y., & Qiang, F. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3404654 · Cited by 13
MGSGNet-S*: Multilayer Guided Semantic Graph Network via Knowledge Distillation for RGB-Thermal Urban Scene Parsing
Zhou, W., Wu, H., & Jiang, Q. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3456437 · Cited by 13
Multimodal Data Fusion for Precise Lettuce Phenotype Estimation Using Deep Learning Algorithms
Hou, L., Zhu, Y., Wang, M., Wei, N., Dong, J., Tao, Y., Zhou, J., & Zhang, J. (2024) · Plants · doi:10.3390/plants13223217 · Cited by 13
Self-citation
SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation
Casao, S., Peña, F., Sabater, A., Castillón, R., Gracia, D. S., Montijano, E., & Murillo, A. C. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801797 · Cited by 13
Tuple Perturbation-Based Contrastive Learning Framework for Multimodal Remote Sensing Image Semantic Segmentation
Ye, Y., Dai, J., Zhou, L., Duan, K., Tao, R., Li, W., & Hong, D. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3542868 · Cited by 14
SICFNet: Shared Information Interaction and Complementary Feature Fusion Network for RGB-T traffic scene parsing
Zhang, B., Li, Z., Sun, F., Li, Z., Dong, X., Zhao, X., & Zhang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127171 · Cited by 13
ShareCMP: Polarization-Aware RGB-P Semantic Segmentation
Liu, Z., Wang, B., Wang, L., Chenyu, M., & Li, Y. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3570764 · Cited by 17
Keep the Balance: A Parameter-Efficient Symmetrical Framework for RGB+X Semantic Segmentation
Cai, J., Su, J. K., Li, Q., Yang, W., Wang, S., Zhao, T., He, S., & Liu, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00990 · Cited by 12
A Review of Computer Vision for Industrial-Grade Waste Classification
Thielmann, P., Zhou, Y., Mirbach, B., Stricker, D., & Rambach, J. R. (2025) · IEEE Access · doi:10.1109/access.2025.3603903 · Cited by 12
DDFN: Deblurring Dictionary Encoding Fusion Network for Infrared and Visible Image Object Detection
Lai, J., Geng, J., Deng, X., & Jiang, W. R. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3311176 · Cited by 12
DFNet-Trans: An end-to-end multibranching network for depth estimation for transparent objects
Meng, X., Wen, J., Li, Y., Wang, C., & Zhang, J. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103914 · Cited by 12
Self-citation
Keypoint Fusion for RGB-D Based 3D Hand Pose Estimation
Liu, X., Ren, P., Gao, Y., Wang, J., Sun, H., Qi, Q., Zhuang, Z., & Liao, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28166 · Cited by 12
Lightweight Cross-Modal Information Measure and Propagation for Road Extraction From Remote Sensing Image and Trajectory/LiDAR
Qiu, Y., Lin, C., Mei, J., Sun, Y., Lu, H., & Xu, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3385667 · Cited by 12
SAM-Event-Adapter: Adapting Segment Anything Model for Event-RGB Semantic Segmentation
Yao, B., Deng, Y., Liu, Y., Chen, H., Li, Y., & Yang, Z. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611127 · Cited by 12
Holistic-Based Cross-Attention Modal Fusion Network for Video Sign Language Recognition
Gao, Q. H., Hu, J., Mai, H., & Ju, Z. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3435693 · Cited by 12
MultiPanoWise: holistic deep architecture for multi-task dense prediction from a single panoramic image
Shah, U., Tukur, M., Alzubaidi, M. S., Pintore, G., Gobbetti, E., Househ, M. S., Schneider, J., & Agus, M. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00138 · Cited by 12
PrimKD: Primary Modality Guided Multimodal Fusion for RGB-D Semantic Segmentation
Hao, Z., Xiao, Z., Luo, Y., Guo, J., Wang, J., Shen, L., & Hu, H. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681253 · Cited by 12
Asymmetric multimodal guidance fusion network for realtime visible and thermal semantic segmentation
Yang, B., Guo, Y., Ni, R., Liu, Y., Li, G., & Hu, C. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109881 · Cited by 12
Multi-modal feature integration network for Visible-Depth-Thermal salient object detection
Cui, F., Zhou, X., Bao, L., Wan, B., Shi, R., Chen, Q., & Zhang, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110734 · Cited by 11
Self-citation
MFFNet: a wavelet transform-based multimodal frequency fusion network for remote sensing semantic segmentation
Li, C., Lyu, H., Jing, W., Yuan, Y., & Cheng, G. (2025) · GIScience & Remote Sensing · doi:10.1080/15481603.2025.2534740 · Cited by 11
CATFace: Cross-Attribute-Guided Transformer With Self-Attention Distillation for Low-Quality Face Recognition
Talemi, N. A., Kashiani, H., & Nasrabadi, N. M. (2024) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2023.3349218 · Cited by 11
The Devil’s Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models
Dolatabadi, H. M., Erfani, S., & Leckie, C. A. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00025 · Cited by 19
Dual-branch deep cross-modal interaction network for semantic segmentation with thermal images
Dai, K., & Chen, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108820 · Cited by 11
Deep learning fusion framework for automated coronary artery disease detection using raw heart sound signals
Dai, Y., Liu, P., Hou, W., Kadier, K., Mu, Z., Lu, Z., Chen, P., Ma, X., & Dai, J. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e35631 · Cited by 11
FDGSNet: A Multimodal Gated Segmentation Network for Remote Sensing Image Based on Frequency Decomposition
Cui, J. H., Liu, J., Ni, Y., Wang, J., & Li, M. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3471638 · Cited by 11
MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation
Yang, L., Hoyer, L., Weber, M., Fischer, T., Dai, D., Leal-Taixé, L., Pollefeys, M., Cremers, D., & Gool, L. V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72933-1_19 · Cited by 17
Salient Object Detection From Arbitrary Modalities
Huang, N., Yang, Y., Xi, R., Zhang, Q., Han, J., & Huang, J. H. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3486225 · Cited by 11
Fooling Decision-Based Black-Box Automotive Vision Perception Systems in Physical World
Jia, W., Lu, Z., Yu, R., Li, L., Zhang, H., Liu, Z., & Qu, G. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3347860 · Cited by 11
A neuromorphic dataset for tabletop object segmentation in indoor cluttered environment
Huang, X., Kachole, S., Ayyad, A., Naeini, F. B., Makris, D., & Zweiri, Y. (2024) · Scientific Data · doi:10.1038/s41597-024-02920-1 · Cited by 20
RGBX-DiffusionDet: a framework for multi-modal RGB-X object detection using DiffusionDet
Orfaig, E., Stainvas, I., & Bilik, I. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112460 · Cited by 10
Learning Frequency-Domain Fusion for Multimodal Remote Sensing Semantic Segmentation
Chen, G., Sun, F., Jing, W., Zou, W., Di, D., Song, Y., & Fan, L. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3622749 · Cited by 10
Cross-modality feature fusion for night pedestrian detection
Feng, Y., Luo, E., Lu, H., & Zhai, S. (2024) · Frontiers in Physics · doi:10.3389/fphy.2024.1356248 · Cited by 11
Semantic Guidance Fusion Network for Cross-Modal Semantic Segmentation
Zhang, P., Chen, M., & Gao, M. (2024) · Sensors · doi:10.3390/s24082473 · Cited by 10
A robot grasping detection network based on flexible selection of multi-modal feature fusion structure
Wang, Y., Guo, Z., Chen, Y., Guo, C., Xia, M., & Qi, T. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05427-9 · Cited by 10
BCLNet: Boundary contrastive learning with gated attention feature fusion and multi-branch spatial-channel reconstruction for land use classification
Yue, C., Zhang⋆, Y., Yan, J., Luo, Z., Liu, Y., & Guo, P. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112387 · Cited by 13
RGB-D Cube R-CNN: 3D Object Detection with Selective Modality Dropout
Piekenbrinck, J., Hermans, A., Vaškevičius, N., Linder, T., & Leibe, B. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00205 · Cited by 10
Depth-Guided Texture Diffusion for Image Semantic Segmentation
Sun, W., Li, Y., Ye, Q., Jiao, J., & Zhou, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3484490 · Cited by 12
MFDA: Unified Multi-Task Architecture for Cross-Scene Sea Ice Classification
Chen, Y., Yan, Q., Bashmachnikov, I., Huang, K., Mu, F., Zhang, Z., Xu, M., & Zhao, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3491190 · Cited by 10
A Multimodal Unified Representation Learning Framework With Masked Image Modeling for Remote Sensing Images
Du, D., Liu, T., & Gu, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3494244 · Cited by 10
Expression Prompt Collaboration Transformer for universal referring video object segmentation
Chen, J., Lin, J., Zhong, G., Fu, H., Nai, K., Yang, K., & Li, Z. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113006 · Cited by 12
Self-citation
Remote sensing image interpretation of geological lithology via a sensitive feature self-aggregation deep fusion network
He, K., Dong, J., Ma, H., Cai, Y., Feng, R., Dong, Y., & Wang, L. (2025) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2025.104384 · Cited by 10
HAFFseg: RGB-Thermal semantic segmentation network with hybrid adaptive feature fusion strategy
Yi, S., Chen, M., Liu, X., Li, J., & Chen, L. (2023) · Signal Processing Image Communication · doi:10.1016/j.image.2023.117027 · Cited by 10
PanoGlassNet: Glass Detection With Panoramic RGB and Intensity Images
Chang, Q., Liao, H., Meng, X., Xu, S., & Cui, Y. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3390163 · Cited by 10
Triplet Contrastive Learning Framework With Adversarial Hard-Negative Sample Generation for Multimodal Remote Sensing Images
Chen, Z., Zhang, C., Zhang, B., & He, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3354304 · Cited by 9
A Fourier-Transform-Based Framework With Asymptotic Attention for Mobile Thermal InfraRed Object Detection
Wang, Z., Shen, H., Jiang, W., & Huang, K. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3399193 · Cited by 10
LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories
Weder, S., Blum, H., Engelmann, F., & Pollefeys, M. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00075 · Cited by 9
TAG-fusion: Two-stage attention guided multi-modal fusion network for semantic segmentation
Zhang, Z., Wang, W., Zhu, L., & Tang, Z. (2024) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104807 · Cited by 9
HSPFormer: Hierarchical Spatial Perception Transformer for Semantic Segmentation
Chen, S., Han, T., Zhang, C., Su, J., Wang, R., Chen, Y., Wang, Z., & Cai, G. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3525542 · Cited by 11
Low-Light Enhancement and Global-Local Feature Interaction for RGB-T Semantic Segmentation
Guo, X., Liu, Y., Xue, W., Zhiwei, Z., & Zhuang, Y. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3545511 · Cited by 9
MCKTNet: Multiscale Cross-Modal Knowledge Transfer Network for Semantic Segmentation of Remote Sensing Images
Cui, J. H., Liu, J., Ni, Y., Sun, Y., & Mao, G. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3547442 · Cited by 9
RDCRNet: RGB-T Object Detection Network Based on Cross-Modal Representation Model
Li, Y., Zhan, W., Jiang, Y., & Guo, J. (2025) · Entropy · doi:10.3390/e27040442 · Cited by 14
Pseudo Visible Feature Fine-Grained Fusion for Thermal Object Detection
Li, T., Ye, M., Wu, T., Li, N., Li, S., Tang, S. Y., & Ji, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00629 · Cited by 9
Depth Matters: Exploring Deep Interactions of RGB-D for Semantic Segmentation in Traffic Scenes
Chen, S., Han, T., Zhang, C., Liu, W., Su, J., Wang, Z., & Cai, G. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246921 · Cited by 10
Cross-modal State Space Modeling for Real-time RGB-thermal Wild Scene Semantic Segmentation
Guo, X., Lin, Z., Hu, L., Deng, Z., Liu, T., & Zhou, W. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247008 · Cited by 9
OAFuser: Toward Omni-Aperture Fusion for Light Field Semantic Segmentation
Teng, F., Zhang, J., Peng, K., Wang, Y., Stiefelhagen, R., & Yang, K. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3457931 · Cited by 10
Self-citation
DPSNet: A Detail Perception Synergistic Network for Camouflaged Object Detection
Li, X., Long, S., Yang, J., Lei, J., Li, S., Zhang, J., & Cohen, L. D. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3497181 · Cited by 9
Self-citation
Centering the Value of Every Modality: Towards Efficient and Resilient Modality-Agnostic Semantic Segmentation
Xu, Z., Lyu, Y., Zhou, J., & Wang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72890-7_12 · Cited by 8
CPAL: Cross-Prompting Adapter With LoRAs for RGB+X Semantic Segmentation
Liu, Y., Wu, P., Wang, M., & Liu, J. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3536086 · Cited by 11
RGB-D Domain adaptive semantic segmentation with cross-modality feature recalibration
Fan, Q., Shen, X., Ying, S., Wang, J., & Du, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103117 · Cited by 9
Pseudo 5D hyperspectral light field for image semantic segmentation
Cong, R., Sheng, H., Yang, D., Chen, R., & Cui, Z. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103042 · Cited by 8

Authors: Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R.
Affiliations: Karlsruhe Institute of Technology (DE)
Nio (China) (CN)
Hunan University (CN)
ByteDance (China) (CN)
ORCID: Zhang, Jiaming
Yang, Kailun
Hu, Xinxin
Liu, Ruiping
Stiefelhagen, Rainer
Venue: IEEE Transactions on Intelligent Transportation Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2023-12
Volume: 24
Issue: 12
Pages: 14679-14694
ISSN: 1524-9050 (ISSN-L)
1558-0016
DOI: 10.1109/tits.2023.3300537
OpenAlex: W4386179772
Cited by: 731
References: 107
Citations by year: 2026: 225 · 2025: 268 · 2024: 178 · 2023: 53
FWCI: 65.71
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Transactions on Intelligent Transportation Systems (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Federal Ministry of Labor and Social Affairs (BMAS) through the AccessibleMaps Project (01KM151112)
“KIT Future Fields” Project
Ministry of Science, Research and the Arts of Baden-Wurttemberg (MWK) through the Cooperative Graduate School Accessibility through AIBased Assistive Technology (BW6-03)
Federal Ministry of Education and Research (BMBF) through the Fellowship within the IFI Program of the German Academic Exchange Service
HoreKA@KIT Supercomputer Partition
Hangzhou SurImage Technology Company Ltd
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: cross-modal fusion, multimodal fusion, depth images, thermal images, polarization images, event cameras, LiDAR, transformer models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks