Cited by 731Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMultimodal Machine Learning Applications

The DOI matches its Crossref record; 106 of the 107 references it cites were found in a registry or an index.

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

Open access

Zhang, Jiaming, Liu, Huayao, Yang, Kailun, Hu, Xinxin, Liu, Ruiping, Stiefelhagen, Rainer·2023·IEEE Transactions on Intelligent Transportation Systems

DOI 10.1109/tits.2023.3300537

Open this paper in CoChat — free account Open PDF

Abstract

Scene understanding based on image segmentation is a crucial component of autonomous vehicles. Pixel-wise semantic segmentation of RGB images can be advanced by exploiting complementary features from the supplementary modality (${X}$-modality). However, covering a wide variety of sensors with a modality-agnostic model remains an unresolved problem due to variations in sensor characteristics among different modalities. Unlike previous modality-specific methods, in this work, we propose a unified fusion framework, CMX, for RGB-X semantic segmentation. To generalize well across different modalities, that often include supplements as well as uncertainties, a unified cross-modal interaction is crucial for modality fusion. Specifically, we design a Cross-Modal Feature Rectification Module (CM-FRM) to calibrate bi-modal features by leveraging the features from one modality to rectify the features of the other modality. With rectified feature pairs, we deploy a Feature Fusion Module (FFM) to perform sufficient exchange of long-range contexts before mixing. To verify CMX, for the first time, we unify five modalities complementary to RGB, i.e., depth, thermal, polarization, event, and LiDAR. Extensive experiments show that CMX generalizes well to diverse multi-modal fusion, achieving state-of-the-art performances on five RGB-Depth benchmarks, as well as RGB-Thermal, RGB-Polarization, and RGB-LiDAR datasets. Besides, to investigate the generalizability to dense-sparse data fusion, we establish an RGB-Event semantic segmentation benchmark based on the EventScape dataset, on which CMX sets the new state-of-the-art. The source code of CMX is publicly available athttps://github.com/huaaaliu/RGBX_Semantic_Segmentation.

Integrity signals

Registry

Crossref · 10.1109/tits.2023.3300537

Cites

106 of 106 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
MizAR 60 for Mizar 50
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054

Show all 107 references

Research works printed but not linked: 1

Dosovitskiy, An image is worth 16✗16 words: Transformers for image recognition at scale, 1

Sorted by citations

Items 401–500 of 731

RGB-D Semantic Segmentation for Indoor Modeling Using Deep Learning: A Review
Rached, I., Hajji, R., & Landes, T. (2024) · Lecture notes in geoinformation and cartography · doi:10.1007/978-3-031-43699-4_36 · Cited by 2
Shape-Aware Convolution with Convolutional Kernel Attention for RGB-D Image Semantic Segmentation
Kun, Z. H. O. U., ZHANG, Z., Tang, X. D., Wen, X. U., Xie, J., & Tang, C. (2024) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.2024eap1023 · Cited by 2
Bimodal Segnet: Fused Instance Segmentation Using Events and Rgb Frames for Robotic GraspingPreprint
Kachole, S., Huang, X., Naeini, F. B., Muthuswamy, R., Makris, D., & Zweiri, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4567049 · Cited by 2
Modality adaptation via feature difference learning for depth human parsing
Huang, S., Hui, T., Gong, Y., Peng, F., Fang, Y., Wang, J., Ma, B., Wei, X., & Han, J. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104070 · Cited by 1
Feature Decomposition Transformers for Infrared and Visible Image Fusion
Kim, G., Vien, A. G., Nguyen, D. H., & Lee, C. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10647365 · Cited by 1
Adaptive Multimodal Fusion with Cross-Attention for Robust Scene Segmentation and Urban Economic Analysis
Zhong, C., Zeng, S., & Zhu, H. (2025) · Applied Sciences · doi:10.3390/app15010438 · Cited by 1
Research on Multi-Attention-Based Multimodal Fusion Model
Ma, Y. H., Qin, Y. J., Zhang, H., & Jiang, K. (2024) · International Conference on Electronic Information Technology and Computer Engineering (EITCE) · doi:10.1145/3711129.3711215 · Cited by 1
ProgressiveGlassNet:Glass Detection with Progressive Decoder
Chang, Q., Meng, X., Hong, Z., & Cui, Y. (2024) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa63168.2024.00122 · Cited by 1
Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression
Schmitt, J., Liu, R., Zheng, J., Zhang, J., & Stiefelhagen, R. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919728 · Cited by 3
Self-citation
Ray-Aided Quadruple Affiliation Network for Calculating Tumor-Stroma Ratios in Breast Cancers
Yang, K., Chen, L., Zheng, X., Li, X., Lan, J., Wu, Y., Tsang, J. Y., & Tse, G. M.-K. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3561679 · Cited by 1
Self-citation
Asymmetric Dual-Encoder Network With Clustering and Mutual Contrast Loss for the Semantic Segmentation of Remote-Sensing Images
Zhou, W., Li, Y., & Liu, Y. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3575945 · Cited by 1
Synergistic Fusion Network of Microscopic Hyperspectral and RGB Images for Multi-Perspective Segmentation
Zhang, L., & Wang, Q. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3578726 · Cited by 1
A Spatiotemporal Decoupled Framework Incorporating Contrastive Learning for RGB-D Gesture Recognition
Hu, Y., Wang, S., Hu, Q., & Li, J. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3576794 · Cited by 1
Exploiting Event Temporal Dynamics and Sparsity Characteristics for RGB-Event Fusion Semantic Segmentation
Zhang, Y., Wei, Y., Guo, Y., Chen, J., & Zhong, Y. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733331 · Cited by 1
3CNet: Cross-modal cooperative correction network for RGB-T semantic segmentation
Guo, Y., Chen, Z., Rong, X., Liu, C., Song, L., & Li, Y. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105638 · Cited by 1
LFSSMam: Efficient Aggregation of Multi-Spatial-Angular-Modal Information Using Selective SSM for Light Field Semantic Segmentation
Yan, W., Chen, H., Wu, Q., Zhang, X., Fang, Q., Hu, S., & Wang, Y. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3588269 · Cited by 3
EACNet: Efficient Asymmetric Cross-Modal RGB-D Semantic Segmentation Network for Indoor Robotic Perception
Mao, Y., Chen, Y., Zhu, Q., & Zhang, J. (2025) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls66240.2025.11065774 · Cited by 1
Self-citation
Evidence-Based Fusion for Low-Quality RGB-T Semantic Segmentation
Liang, W., Zhang, X., Yang, Y., & Shan, C. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3593962 · Cited by 1
LiDAR and Camera Fusion for Joint Depth Completion and Panoptic Segmentation Tasks in a Unified Network for 3D Semantic Segmentation
Choi, Y.-H., Bong, E.-J., & Kee, S.-C. (2025) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv64158.2025.11097722 · Cited by 1
RGB-D indoor scene parsing via wavelet sub-band guided transformer
Xie, W., Liu, H., & Li, J. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113675 · Cited by 1
Self-citation
SegCR: A Multimodal and Multitask Complementary Fusion Network for Remote Sensing Semantic Segmentation and Cloud Removal
Wu, S., Zhu, J., Gu, Y., Han, W., Jiang, W. R., & Geng, J. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3603066 · Cited by 1
Pruning-Friendly RGB-T Semantic Segmentation for Real-Time Processing on Edge Devices
Hwang, J. Y., Lee, Y. J., Jung, H. G., & Suhr, J. K. (2025) · Electronics · doi:10.3390/electronics14173408 · Cited by 1
Fuse It or Lose It? Analyzing the Effects of Sensor Diversity on Multimodal Ensembles for Autonomous Vehicle Perception
Mortlock, T., Chen, L., Smereka, J. M., Khargonekar, P. P., & Faruque, M. A. A. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3612873 · Cited by 2
MsResNet: Multi-scale Edge-Enhanced ResNet for RGB-T Image Segmentation
Adhikari, B., Lei, S., Đurić, Z., & Wijesekera, D. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-01169-5_4 · Cited by 1
Spatial-Aware Multi-Modal Information Fusion for Food Nutrition Estimation
Yu, D., Min, W., Jin, X., Jiang, Q., & Jiang, S. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755750 · Cited by 1
Ingredients-Guided and Nutrients-Prompted Network for Food Nutrition Estimation
Zhang, D., Ma, B., Wu, X., & Kittler, J. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754588 · Cited by 1
CMPFNet: semantic segmentation network for cross-modal phased fusion in extreme light scenes
Han, J., Tian, Z., Gao, K., & Lyu, X. (2025) · Pattern Analysis and Applications · doi:10.1007/s10044-025-01569-9 · Cited by 1
Fusion of multimodal information for visual perception of obstacles in underground electric locomotives
Xu, Z., Guo, Y., Wang, W., Wang, S., Yang, T., & Ma, X. (2025) · Measurement · doi:10.1016/j.measurement.2025.119511 · Cited by 1
Do not overestimate RGB: Improving image manipulation detection and localization via multi-noise-view fusion
Shim, J., & Yoon, H. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131915 · Cited by 1
Interactive feature fusion for camera-radar-based vehicle segmentation in bird’s-eye view
Liu, Y., Lu, C., Li, L., Meng, X., Jiang, Q., & Shao, F. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112698 · Cited by 1
SFFR: Spatial–Frequency Feature Reconstruction for Multispectral Aerial Object Detection
Zuo, X., Qu, C., Zhan, H., Shen, J., & Yang, W. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3631708 · Cited by 1
GatedFusion-Net: Per-pixel modality weighting in a five-cue transformer for RGB-D-I-T-UV fusion
Brenner, M., Reyes, N. H., Sušnjak, T., & Barczak, A. L. C. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103986 · Cited by 1
MIFNet: Multi-modal Interactive Fusion Network For Remote Sensing Semantic Segmentation
Cheng, B., Xu, B., Deng, Q., & Shen, T. (2025) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss55030.2025.11243325 · Cited by 1
Efficient Prediction of Dense Visual Embeddings via Distillation and RGB-D Transformers
Fischedick, S. B., Seichter, D., Stephan, B., Schmidt, R., & Groß, H. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11245809 · Cited by 1
MSFFNet: A Multispectral Semantic Segmentation Network With Multi-Scale Feature Fusion for Remote Sensing Images
Zhao, H., Mohamed, R., & Ng, S.-B. (2025) · IEEE Access · doi:10.1109/access.2025.3638879 · Cited by 1
IEMFormer: Internal and External Multi-Fusion Transformer for Indoor RGB-D Semantic Segmentation
Hu, K., Li, W., Gao, G., & Yang, R. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3631433 · Cited by 1
SERF: Spatiotemporal-Aware Event-RGB Fusion for Steering Angle Prediction
Xia, C., Peng, P., Xu, H., Tan, G., Li, L., & Tian, Y. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3641162 · Cited by 1
RoCeDiRNet-3DoF: Robust center direction network for cloth grasping point localization in complex scenes
Luo, J., & Hu, Y. (2026) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2026.105328 · Cited by 1
KeyGeoFusion: A multi-modal keypoint and geometry-aware framework for small and distant 3D object detection in sparse point clouds
Ju, C., Xie, Y., Duan, J., Chang, Y., & Wu, M. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.132842 · Cited by 1
A deep learning model for photovoltaic soiling loss prediction and estimation based on Large Kernel Cross-Attention Fusion
Zheng, S., Yan, P., Ni, S., & Wang, D. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114097 · Cited by 1
Beyond Appearance: Geometric Cues for Robust Video Instance Segmentation
Niu, Q., Zhou, Y., Chen, S., Zhang, T., & Ji, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00406 · Cited by 1
Evaluation of Polarimetric Fusion for Semantic Segmentation in Aquatic Environments
Batista, L. F. W., Bourbon, T., & Pradalier, C. (2025) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip67698.2025.11396891 · Cited by 1
Fine-scale mapping of rice distribution in cloud-prone regions: a multi-scale asymmetric fusion network framework based on Sentinel-1/2 imagery
Ling, X., Cui, T., Sheng, Q., Wang, B. X., Li, J., Liu, X., & (112147), X. X. (2026) · International Journal of Digital Earth · doi:10.1080/17538947.2026.2625542 · Cited by 1
DiffPixelFormer: Differential Pixel-Aware Transformer for RGB-D Indoor Scene Segmentation
Gong, Y., Lu, J., Gao, Y., Zhao, J., Zhang, X., & Rahardja, S. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3677060 · Cited by 1
STENet: Superpixel Token Enhancing Network for RGB-D Salient Object Detection
Chen, J., Li, G., Zhang, Z., Chang, L., & Zeng, D. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3680601 · Cited by 1
Uncertainty-guided and reliable collaborative perception for open heterogeneous systems
Tian, Y., Zhang, X., Zhuo, S., Zhang, D., Tian, Z., Liu, Y., & Du, S. (2026) · Pattern Recognition Letters · doi:10.1016/j.patrec.2026.04.011 · Cited by 1
EG-CMFNet: An Edge-Guided Cross-Modal Fusion Network for Remote Sensing Semantic Segmentation
Zhao, J., Zhou, Z., Zhang, L., Linxin, W., & Lu, Z. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3680719 · Cited by 1
Structure-aware state space modeling with multi-scale feature fusion for railway scene segmentation
Fu, H., Ma, Z., Yang, X., Zhang, W., Hu, L., & Jiang, K. (2026) · Scientific Reports · doi:10.1038/s41598-026-47010-x · Cited by 1
Modality-Decoupled RGB-Thermal Object Detector Via Query Fusion
Tian, C., Zhou, Z., Yang, C., Zhu, G., Zhong, F., & He, Z. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461564 · Cited by 1
MmSAM: multimodal meets SAM2 for efficient remote sensing semantic segmentation
Qingpeng, W., Huang, Z., Cheng, Y., & Bao, Y. (2026) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2026.105301 · Cited by 1
LS-DSPerceiver: A Dual-Domain Feature Encoding and Scale-Aware Perceiver for Robust Multiscenario Landslide Extraction
Chen, M., Zhang, J. Q., Li, C., Guo, W., Fang, T., Zhao, Y., Ge, X., Xu, B. C., Hu, H., & Zhu, Q. Q. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3687990 · Cited by 1
Self-citation
Semantic SLAM with Multi-Modal Perception: Survey on Robust Long-Term Localization for Autonomous Vehicles
Navarro-Pérez, A., Bacca, E., & Bravo, E. F. C. (2026) · Robotics · doi:10.3390/robotics15050088 · Cited by 1
EVOLVE: Event-Guided Deformable Feature Transfer and Dual-Memory Refinement for Low-Light Video Object Segmentation
Baek, J.-H., Oh, J., & Koh, Y. J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01049 · Cited by 1
C2F2T-Net: Cross-modal difference interaction collaborative frequency domain fusion transformer network for collaborative classification of hyperspectral and LiDAR data
Zhao, Y., Song, N., & Bao, W. (2026) · Optics & Laser Technology · doi:10.1016/j.optlastec.2026.115452 · Cited by 1
IG-SAM: Implicitly guided SAM for RGB-thermal semantic segmentation
Zhang, Q., Xing, J., Zhang, H., Yan, R., Hao, M., Yan, L., & Li, Z. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132804 · Cited by 1
A Review of Multimodal Image Feature Fusion Technology and Application
Cao, P., Zhao, Y., Duan, T., Li, L., Xian, C., & Li, S. (2026) · Applied Sciences · doi:10.3390/app16115290 · Cited by 1
Cross-Domain Few-Shot In-Context Learning for Fine-Grained Ship Recognition from Thermal Infrared Maritime Images
Wu, W., Li, R., Yin, Y., Wang, Z., & Cheng, L. (2026) · Ocean Engineering · doi:10.1016/j.oceaneng.2026.126068 · Cited by 1
Comprehensive survey on advances and challenges in RGB-D semantic segmentation
Choi, S., Cho, E., Jung, A., Min, B.-C., Min, J., & Hong, S. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114146 · Cited by 1
SVFNet: Underwater RGB-Sonar fusion classification via cross-modal contrastive learning and Global Query Spatial Attention
Zhang, X., Zhao, Y., Li, J., Xu, H., Liu, L., & Gu, H. (2026) · Ocean Engineering · doi:10.1016/j.oceaneng.2026.126328 · Cited by 1
NeuroSeg-MF: robust neuron segmentation in two-photon Ca 2+ imaging using multi-feature fusion and detection-guided SAM
Xu, Z., Liu, W., Liang, S., Jia, H., Chen, X., Qin, H., & Liao, X. (2026) · Biomedical Optics Express · doi:10.1364/boe.600665
AFDINet: Adaptive frequency disentanglement and interaction in hybrid Mamba–CNN for optical-SAR land cover classification
Ren, B., Gu, J., Yang, C., Hou, B., Wang, Q., Wu, Z., & Jiao, L. (2026) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2026.105406 · Cited by 1
FireRGBTNet: A Lightweight Forest Fire Detection Model Based on Efficient RGB–Thermal Fusion
Ma, Y., Shan, W., Wang, M., Sui, Y., & Wang, M. (2026) · Forests · doi:10.3390/f17080955 · Cited by 1
FCPENet: Feature Complementary Property Enhancement Network for RGB-Thermal Semantic Segmentation
Wang, Q., Song, H., Yin, C., Wang, Q., Wu, H. R., & Shen, T. (2023) · International Conference on Information Systems and Computing Technology (ISCTech) · doi:10.1109/isctech60480.2023.00050 · Cited by 1
CLGFormer: Cross-Level-Guided transformer for RGB-D semantic segmentation
Li, T., Zhou, Q., Wu, D., Sun, M., & Hu, T. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19051-9 · Cited by 1
CRLNet: A Multimodal Peach Detection Network Based on Cooperative Asymptotic Enhancement and the Fusion of Granularity Refinement
Liu, J., He, C., Wang, M., Jiang, Y., Sun, M., Yan, M., & He, M. (2024) · Plants · doi:10.3390/plants13141980 · Cited by 1
A Color and Depth Image Defect Segmentation Framework for EEE Reuse and Recycling
Wu, Y., Zhou, C., Sterkens, W., Piessens, M., Marelle, D. D., Dewulf, W., & Peeters, J. R. (2024) · Electronics Goes Green · doi:10.23919/egg62010.2024.10631245 · Cited by 1
DBCAN: DFormer-Based Cross-Attention Network for RGB Depth Semantic Segmentation
Wu, A., & Fu, L. (2024) · Applied Sciences · doi:10.3390/app14188329 · Cited by 1
Reliable Deep Learning-Based Analysis of Production Areas Using RGB-D Data and Model Confidence Calibration
Bauer, J. C., Yilmaz, K., Wächter, S., & Daub, R. (2024) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa61755.2024.10711012 · Cited by 1
RGB-D Vision Navigation via Motion Saliency Measurement and Twin Reprojection Optimization in Complex Dynamic Scenes
Sun, C., Wu, X. L., & Sun, C. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3485442 · Cited by 1
MaskMentor: Unlocking the Potential of Masked Self-Teaching for Missing Modality RGB-D Semantic Segmentation
Zhao, Z., Li, J., Wang, L., Wang, Y., & Lu, H. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681698 · Cited by 1
Cross-Modality Comprehensive Feature Aggregation Network for Road Scene Segmentation
Yang, Y., Zhang, Z., Hou, H., Zhang, W., & Zhuang, C. (2024) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic64297.2024.00050 · Cited by 1
Advancing Textile Damage Segmentation: A Novel RGBT Dataset and Thermal Frequency Normalization
Rayhan, F., Joshi, J., Ren, G., Hernandez, L., Petreca, B. B., Baurley, S. L., Bianchi‐Berthouze, N., & Cho, Y. (2025) · Sensors · doi:10.3390/s25072306 · Cited by 1
CSANet: Context–Spatial Awareness Network for RGB-T Urban Scene Understanding
Li, R., Wang, Z., Guo, J., & Zhang, C. (2025) · Journal of Imaging · doi:10.3390/jimaging11060188 · Cited by 1
FCEGNet: Feature calibration and edge-guided MLP decoder Network for RGB-D semantic segmentation
Lu, Y., Ge, B., Xia, C., Zhu, X., Zhang, M., Gao, M., Chen, N., Hu, J., & Zhi, J. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104448 · Cited by 2
Spectral-aware Global Fusion for RGB-Thermal Semantic Segmentation
Zhang, C., Wan, Z., Stepputtis, S., Sycara, K. P., & Xie, Y. (2025) · Proceedings - International Conference on Image Processing · doi:10.1109/icip55913.2025.11084451 · Cited by 1
Multimodal Heterogeneous Hypergraph Learning for Incomplete Multimodal Semantic Segmentation of Remote Sensing Images
Han, W., Geng, J., Xu, Z., & Jiang, W. R. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3604075 · Cited by 1
MDFformer: A Multiscale Dual-Modal Fusion Transformer for Semantic Segmentation
Shi, L., Dong, Y., Wang, T., Cao, R., & Wen, X. (2025) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2025.3611465 · Cited by 1
SEANet: RGB and Thermal Maritime Panoptic Dataset and Intermodal Alignment Procedure
Kafka, O., Kaufmann, J., & Rankl, C. (2025) · IEEE International Forum on Research and Technology for Society and Industry (RTSI) · doi:10.1109/rtsi64020.2025.11212215 · Cited by 1
Effective Fusion in Rgb-P Semantic Segmentation: A Global Feature-Based NetworkPreprint
Gao, S., Yang, X., Jiang, L. M., Fu, Z., & Du, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4533307
CFI-Former: Cross-Feature Interaction Transformer for Group Activity RecognitionPreprint
Zhu, X., Wang, D., & Zhou, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4595240
Scale-equivariant convolution for semantic segmentation of depth image
Marumo, H., & Matsubara, T. (2023) · Nonlinear Theory and Its Applications IEICE · doi:10.1587/nolta.15.36
Inputmix: A Strategy to Regularize and Balance Multi-Modality and Multi-View Model Learning
Wang, J., & Nordström, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446664
Crack Detection of Masonry Structure Based on Infrared and Visible Image Fusion and Deep Learning
Lu, Y., Huang, H., & Zhang, C. (2024) · Lecture notes in civil engineering · doi:10.1007/978-981-99-7965-3_25
Rgb-D Fusion Network for Glass SegmentationPreprint
Tao, T., Yang, J., Xiao, J., Zheng, H., & Wang, H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4826321
Cross Modal Calibration for RGB-D Instance Segmentation
Jung, A., & Hong, S. (2024) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2024.22.5.13
Infrared and Visible Image Fusion via Multi-modal Feature Decomposition Transformer
Kim, G., Nguyen, D. H., & Lee, C. (2024) · Journal of Broadcast Engineering · doi:10.5909/jbe.2024.29.3.252
RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for Real-world Applications
Liu, X., Zhang, C., Wang, G., Zhang, R., & Ji, X. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611208 · Cited by 2
Real-World Adaptation for Enhanced Photo-Realistic and Semantic Style Transfer in Indoor PanoramasPreprint
Tukur, M., Shah, U., Jashari, S., Boraey, Y., Pintore, G., Gobbetti, E., Fetais, N., Schneider, J., & Agus, M. (2024) · Research Square · doi:10.21203/rs.3.rs-5024690/v1
GS 2 -GNeSF: Geometry-Semantics Synergy for Generalizable Neural Semantic Fields
Wang, C., & Zhao, N. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681156
How to Choose a Suitable Method in Image Semantic Segmentation-An Experimental Study
Li, Y., Jiang, J., & Xu, C. (2024) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c63300.2024.00152
Multi-branch Underwater Scene Semantic Segmentation by Fusing Depth Information and Enhanced Visual Feature
Zhang, J., Sun, Y., Huang, X., Zhang, H., & Yang, H. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-8658-9_46
Self-citation
RGB-D Image Modeling Method Based on Transformer: RDT
Teng, Q., & He, X. (2024) · International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC) · doi:10.1109/aiotc63215.2024.10748282
Lightweight and Efficient Multimodal Prompt Injection Network for Scene Parsing of Remote Sensing Scene Images
Li, Y., Zhou, W., Meng, J., & Yan, W. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3507784
Research on Multimodal Data Fusion Based on Optimal Adaptive Coding
Ma, Y., Qin, Y., Zhang, H., & Jiang, K. (2024) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice63571.2024.10864105
DSMT: RGB-D Dual-Stream Multi-Task Model for Multi-Granularity Semantic Understanding
Wu, S., Chen, S., Wang, Y., Liu, H., Tao, Z., Zhu, Y., Shen, T., & Wang, K. (2024) · China Automation Congress (CAC) · doi:10.1109/cac63892.2024.10864568
Self-citation
CFH-Net: Transformer-Based Unstructured Road-Free Space Detection Network
Yang, J., Fan, L., & Liu, H. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.062963
Self-citation
Underwater multi-sensor information fusion for salient object detection
Mou, Y., Yuan, G., Song, J., & Li, J. (2026) · PLoS ONE · doi:10.1371/journal.pone.0354804
ECMRN: Efficient Cross-Modal Reparameterization Network for RGB-D Tasks via Prompt LearningPreprint
Jia, D., Song, H., Zhao, C., & Zhang, H. (2025) · doi:10.36227/techrxiv.174495621.18762297/v1
D-SSSS: an Information Perception Algorithm for Apple Fruit Trees via Semi-supervised Semantic Segmentation and Depth Information Blending
He, M., Zheng, Y., & Long, Y. (2025) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit63637.2025.10965313
FuseForm: Multimodal Transformer for Semantic Segmentation
McMillen, J., & Yılmaz, Y. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00070

Authors: Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R.
Affiliations: Karlsruhe Institute of Technology (DE)
Nio (China) (CN)
Hunan University (CN)
ByteDance (China) (CN)
ORCID: Zhang, Jiaming
Yang, Kailun
Hu, Xinxin
Liu, Ruiping
Stiefelhagen, Rainer
Venue: IEEE Transactions on Intelligent Transportation Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2023-12
Volume: 24
Issue: 12
Pages: 14679-14694
ISSN: 1524-9050 (ISSN-L)
1558-0016
DOI: 10.1109/tits.2023.3300537
OpenAlex: W4386179772
Cited by: 731
References: 107
Citations by year: 2026: 225 · 2025: 268 · 2024: 178 · 2023: 53
FWCI: 65.71
Type: journal article
Language: en
Open access: green
arxiv.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Transactions on Intelligent Transportation Systems (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Federal Ministry of Labor and Social Affairs (BMAS) through the AccessibleMaps Project (01KM151112)
“KIT Future Fields” Project
Ministry of Science, Research and the Arts of Baden-Wurttemberg (MWK) through the Cooperative Graduate School Accessibility through AIBased Assistive Technology (BW6-03)
Federal Ministry of Education and Research (BMBF) through the Fellowship within the IFI Program of the German Academic Exchange Service
HoreKA@KIT Supercomputer Partition
Hangzhou SurImage Technology Company Ltd
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: cross-modal fusion, multimodal fusion, depth images, thermal images, polarization images, event cameras, LiDAR, transformer models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks