Deep multimodal fusion for semantic image segmentation: A survey

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148

U-Net: Convolutional Networks for Biomedical Image Segmentation

Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404

Deep learning

LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126

ImageNet: A large-scale hierarchical image database

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333

Gradient-based learning applied to document recognition

Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805

Microsoft COCO: Common Objects in Context

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460

Fully convolutional networks for semantic segmentation

Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818

Xception: Deep Learning with Depthwise Separable Convolutions

Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380

DBCAN: DFormer-Based Cross-Attention Network for RGB Depth Semantic Segmentation

Wu, A., & Fu, L. (2024) · Applied Sciences · doi:10.3390/app14188329 · Cited by 1

Exploring Semantic Understanding and Generative Modeling in Speech-Text Multimodal Data Fusion

Yu, H., Wang, X., Sun, Y., Yang, Y., & Sun, Y. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-3156 · Cited by 1

Estimation of soil properties for sustainable crop production using multisource data fusion

Tsakiridis, N. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-24139-0.00010-2 · Cited by 1

Fast detection of rice striped stem borer (Chilo suppressalis) stress based on UAV sensor and multimodal segmentation method

Chu, B., Guo, Z., Liu, B., Jian, B., & Zhou, Y. (2025) · Plant Growth Regulation · doi:10.1007/s10725-025-01320-8 · Cited by 1

Self-distillation guided Semantic Knowledge Feedback network for infrared–visible image fusion

Zhou, W., Wang, Y., Zuo, L., Ma, D., & Yi, Y. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105566 · Cited by 1

MRNet: A Deep Learning Framework for Drivable Area Detection in Multi-Scenario Unstructured Roads

Yang, J. K., Chen, J., Wang, Y., Sun, S., Xie, H., Wu, J., & Wang, W. (2025) · Electronics · doi:10.3390/electronics14112242 · Cited by 2

Multimodal Breast Ultrasound Segmentation: Combining Visual and Clinical Data

Mendez, A., Xu, M., & Huang, K. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94962-3_16 · Cited by 1

Vector Representations of Multi-modal Data

Taipalus, T., & Lu, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05281-0_18 · Cited by 1

Early Fusion Based CNN Architecture for Visual Servoing Systems

Botezatu, A.-P., Ferariu, L. E., Burlacu, A., & Sauciuc, T.-A. (2022) · International Conference on Methods and Models in Automation and Robotics · doi:10.1109/mmar55195.2022.9874328 · Cited by 1

A hybrid multi-modal visual data cross fusion network for indoor and outdoor scene segmentation

Hu, S., Bonardi, F., Bouchafa, S., & Désiré Sidibé, A. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956575 · Cited by 1

Less Labels, More Modalities: A Self-Training Framework to Reuse Pretrained Networks

Burnel, J.-C., Courtrai, L., & Lefèvre, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37731-0_22 · Cited by 1

Orthogonal Multi-frequency Fusion Based Image Reconstruction and Diagnosis in Diffuse Optical TomographyPreprint

Yedder, H. B., Hamarneh, G., Cardoen, B., Shokoufi, M., & Golnaraghi, F. (2022) · doi:10.36227/techrxiv.21574533.v2 · Cited by 1

Deep Orthogonal Multi-Frequency Fusion for Tomogram-Free Diagnosis in Diffuse Optical ImagingPreprint

Yedder, H. B., Cardoen, B., Shokoufi, M., Golnaraghi, F., & Hamarneh, G. (2022) · doi:10.36227/techrxiv.21574533 · Cited by 1

Polarization based urban scenes understanding

Blanchon, M. (2021) · theses.fr (ABES) · doi:10.70675/9eb0b15bzd1bcz473bza318zfc8c58cc92f1 · Cited by 1

Monocular Stereo Vision of Image Feature-aware Interactive Generation

Tian, T., Song, G., Chen, X., & Li, Y. (2021) · International Conference on Computer Technology and Media Convergence Design (CTMCD) · doi:10.1109/ctmcd53128.2021.00067

Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme

Gaşi, A., Ensarı, T., & Dağtekin, M. (2021) · Acta Infologica · doi:10.26650/acin.835241

Improved Bilinear Block-Diagonal Multi-Modal Fusion Model for Rumor DetectionPreprint

Li, J., & Xu, S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4124465

Orthogonal Multi-frequency Fusion Based Image Reconstruction and Diagnosis in Diffuse Optical TomographyPreprint

Yedder, H. B., Hamarneh, G., Cardoen, B., Shokoufi, M., & Golnaraghi, F. (2022) · doi:10.36227/techrxiv.21574533.v1

Quantity Estimation Method of Warehousing Cargo Based on CSwin-Unet and Pixel Position Weight

Wu, J., Zhang, G., Li, K., Wang, P., & Li, Z. (2022) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml57342.2022.10009779

Deep Orthogonal Multi-Frequency Fusion for Tomogram-Free Diagnosis in Diffuse Optical ImagingPreprint

Yedder, H. B., Hamarneh, G., Cardoen, B., Shokoufi, M., & Golnaraghi, F. (2023) · doi:10.36227/techrxiv.21574533.v3

Improving information fusion in deep learning

Don, D. P. M. K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.241291

Deep Learning Architectures for Pain Recognition Based on Physiological Signals

Thiam, P., Kestler, H. A., & Schwenker, F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37660-3_24

Predicting Lung Cancer Development using a Multimodal Longitudinal Feature Analysis FrameworkPreprint

Xiao, N., Han, J., Qiang, Y., Zhao, J., & Geng, Y. (2024) · Research Square · doi:10.21203/rs.3.rs-3872105/v1

A Multi-Scale Vision Transformer-Based Multimodal Geoai Model for Mapping Arctic Permafrost ThawPreprint

Gu, Z., Li, W., Hsu, C.-Y., Wang, S., Yang, Y., Rogers, B. M., & Liljedahl, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4762408

Parallel Personal Protective Equipment Recognition Network Based On Multimodal Fusion

Yuan, N., Li, N., Qian, H., Zhang, Z., & Zhou, H. (2023) · International Conference on Electronic Information Technology and Computer Engineering (EITCE) · doi:10.1145/3650400.3650680

Semantic Segmentation of Point Clouds and Images for Autonomous Vehicles: Trends, Challenges, and OpportunitiesPreprint

Munia, A. A., Kunjumon, R., Khosravi, A., Hossain, I., & Rahman, A. (2024) · doi:10.36227/techrxiv.171441584.45776959/v1

HaWANet: Road Scene Understanding with Multi-modal Sensor Data Using Height-Width-Driven Attention Network

Chatterjee, S., Xu, J., Kuzhipathalil, A., & Nürnberger, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-80607-0_8

Improving Segmentation by Combining Preoperative CT and Intraoperative CBCT using Synthetic Data

Tschuchnig, M., Steininger, P. A., & Gadermayr, M. (2025) · Informatik aktuell · doi:10.1007/978-3-658-47422-5_49

Deep learning for liver evaluation: A comprehensive review and implications for ulcerative colitis detection

Verma, S., Bala, M., & Angurala, M. (2025) · Measurement Sensors · doi:10.1016/j.measen.2025.101867

DBMamba-UNet: dual-branch mamba in vision for 3D hippocampus image segmentation

Yu, L., & Wang, Z. (2025) · Journal of Electronic Imaging · doi:10.1117/1.jei.34.2.023043

Robustly Optimized Multimedia Local and Global Context Analysis at Multi-modal Balance

Yoon, J., Lim, K., & Choi, C. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707989

Prompt-Driven Approach for Cattle Image Segmentation Based on SAM

Bello, R.-W., Owolawi, P. A., Tu, C.-C., & Wyk, E. V. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-92605-1_5

Multimodal Deep Learning Framework for Skin Lesion Classification

Banothu, B., Tulasiram, J., Savarimuthu, N., & Patil, G. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-93697-5_15

Fine-Grained Feature Fusion for Self-supervised Multimodal Sentiment Analysis

Deng, T., Mei, S., Peng, H., Du, X., Chen, Z., Huang, M., Tang, X., Cao, R., & Liu, W. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9986-5_4

Improved bimodal segmentation of multi-light images based on feature fusion

Zeng, W., Wang, Y., Chen, C., Yao, C., Zheng, D., & Wang, S. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111947

Unsupervised Mineral Segmentation with Graph Neural Networks and Multi-modal SEM Data

Repka, S., Eerola, T., Motl, D., Výravský, J., & Zemčík, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05060-1_3

Evaluating Dense Model-based Approaches for Multimodal Medical Case Retrieval

Pires, C. H. D. P., Nunes, S., & Teixeira, L. F. (2025) · Information Retrieval Research · doi:10.54195/irrj.19769

Revolutionizing Vision Perception with Multimodal Fusion Technologies

Batta, P., Chauhan, R., & Kaur, G. (2025) · doi:10.1002/9781394272624.ch14

Cross-backbone pixel consistency and layer-wise attention fusion for weakly supervised semantic segmentation

Liu, M., Zhu, L., & Cheng, J. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105832

Deep learning architectures for multimodal fusion

Savithri, M., & Gnanamonickam, A. A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-27528-9.00004-8

Design of Feature-Level Multimodal Image Fusion Network and Its Application in Head and Neck Tumor Segmentation

He, S., Han, Q., & Liu, H. (2025) · International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) · doi:10.1109/icicml67980.2025.11333539

Machine learning empowered surface growth of 2D materials: Synthesis, characterization and mechanism

Gao, W., Hua, C., Niu, T., & Zhou, M. (2026) · Surface Science Reports · doi:10.1016/j.surfrep.2026.100677

Key Feature-aware Monocular Stereo Vision Generation Algorithm Towards Smart Media Autostereoscopy

Li, Y., Sun, G., Li, Z. M., & Zhu, Z. (2025) · International Conference on Artificial Intelligence, Virtual Reality and Visualization (AIVRV) · doi:10.1109/aivrv67401.2025.11350427

CDDFuse: A Controllable Denoising Diffusion-based Generation Solution for Infrared and Visible Image FusionPreprint

Liu, Y., Han, M., Wang, Y., Fan, J., & Xu, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6235682

Head and Neck Organ-at-Risk Multi-Modal Semantic Segmentation using 3D-UNet Head and Neck Organ-at-Risk Multi-Modal Semantic Segmentation using 3D-UNet

Gader, T. B. A., Echi, A. K., Amor, A. B., Baccar, H., & Najjar, M. N. (2026) · ELCVIA Electronic Letters on Computer Vision and Image Analysis · doi:10.5565/rev/elcvia.2147

Real-time multimodal semantic scene understanding for autonomous UGV navigation

Zhang, Y. (2021) · theses.fr (ABES) · doi:10.70675/a471d7bez2ff1z44feza0b4z4beabc22b5ab

Mobile robot inspection of building plenum spaces via vision-based multi-sensor fault detection

Lou, G., Liu, S., Shen, Y., Wong, W. Y., Lu, Y., & Duan, M. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.106889

Deep multimodal visual data fusion for outdoor scenes analysis in challenging weather conditions

Hu, S. (2023) · theses.fr (ABES) · doi:10.70675/4b9095acz3d26z4982zb375z6890598d175f

Clinical Knowledge-Guided PET/CT Lesion Segmentation With Interpretable Fusion of Metabolic and Structural Cues

Zhang, S., Zhang, J., Qiu, L., Liu, W., Jin, D., Jiao, W., Lü, L., Yen, T., Yang, S. M., & Yan, K. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3686884

Deep multimodal fusion for aquaculture: a comprehensive review

Li, D., Ji, S., Xu, W., Du, Z., Liu, S., Li, X., & Wang, G. (2026) · Aquaculture International · doi:10.1007/s10499-026-02539-6

DAMM: Dynamic modality Aware weighted embeddings fusion for Multimodal Meme detection

Imam, M., Aich, U., & Sarkar, R. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116126

HD²FI-Net: Hierarchical Dual-Domain Fusion–Interaction Network for RGB-T Semantic Segmentation

Guo, Z., Fei, B., Liu, D., & Zhang, J. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810707

MMGC-AttnNet: A Multilevel Multimodal Cross-Attention Network for High-Resolution Remote Sensing Image Segmentation

Wang, R., Chen, Y., Li, S. L., Geng, Y., Li, S. L., Xu, Z., & Zhang, Y. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3707812

Research on Pavement Crack Monitoring Technology in Urban Road Based on Environment Perception‐Driven Visible Light‐Infrared Fusion

Wu, H., Hu, K., Li, S., Duan, J., Lu, Y., Zheng, W., & Xie, J. (2026) · Structural Control and Health Monitoring · doi:10.1155/stc/3339613

Residual-Guided Expert Specialization for Incomplete Multimodal Learning

Baek, S., Park, J., Sim, J., Jeong, M., Lee, H. C., & Kim, W. H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37023-5_5

Performance Comparison of Multi-Modal Fusion Techniques in Tissue Perfusion Analysis Using Homography Calibration

Çevi̇k, K. K., Morris, B., Claman, B., & Menezes, J. M. (2026) · Sensors · doi:10.3390/s26144585

DIS2:Dis entanglement Meets Dis tillation with Classwise Attention for Robust Remote Sensing Segmentation Under Missing Modalities

Kieu, N., Nguyen, K., Wiliem, A., Fookes, C., & Sridharan, S. (2026) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw68408.2026.00149

MFF-Diff: A multi-level feature fusion conditional diffusion model for PET/CT tumor segmentation

Diao, Z., Cui, M., Yuan, Y., Tong, G., & Chen, Y. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.116260

UGV-based multimodal RGBD–multispectral fusion framework enables high-quality 3D phenotyping of greenhouse lettuce seedlings

Yang, S., Qiu, G., Xia, J., Zhao, Y., Wen, W., Wang, C., Gou, W., Guo, X., & Zhao, C. (2026) · Artificial Intelligence in Agriculture · doi:10.1016/j.aiia.2026.08.015

Exploring Dual-to-Single-Encoder Transitions for Real-Time RGB-T Semantic Segmentation

Kim, J., Jung, M., & Moon, B. (2026) · Sensors · doi:10.3390/s26206602

A review: Deep learning for medical image segmentation using multi-modality fusion

Zhou, T., Ruan, S., & Canu, S. (2019) · Array · doi:10.1016/j.array.2019.100004 · Cited by 696

Emerging Advances in Multimodal Imaging and Fusion Techniques

Vyavahare, A. J., & Sawant, D. A. (2025) · International Journal of Electronics and Computer Applications · doi:10.70968/ijeaca.v2i1.d1019

Application on the Loop of Multimodal Image Fusion: Trends on Deep-Learning Based Approaches

Guacho, G. K. B., Vallejo, P. X. M., Vintimilla, B. X., & Sappa, A. D. (2023) · International Conference on Pattern Recognition Systems (ICPRS) · doi:10.1109/icprs58416.2023.10179005 · Cited by 3

Multi-semantic depth collaborative multi-modality image fusion network

Zhu, H., Li, X., Chen, R., Liu, Q., Chang, X., & Yuan, D. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115756 · Cited by 3

A central multimodal fusion framework for outdoor scene image segmentation

Zhang, Y., Morel, O., Seulin, R., Mériaudeau, F., & Sidibé, D. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10357-y · Cited by 5