Deep Multimodal Learning: A Survey on Recent Advances and Trends
The DOI matches its Crossref record; 109 of the 113 references it cites were found in a registry or an index.
Abstract
The success of deep learning has been a catalyst to solving increasingly complex machine-learning problems, which often involve multiple data modalities. We review recent advances in deep multimodal learning and highlight the state-of the art, as well as gaps and challenges in this active research field. We first classify deep multimodal learning architectures and then discuss methods to fuse learned multimodal representations in deep-learning architectures. We highlight two areas of research-regularization strategies and methods that learn or optimize multimodal fusion structures-as exciting areas for future work.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Additionally, early fusion strategies are sensitive to different sampling rates of modalities [2].”
From Multimodal deep learning for biomedical data fusion: a review (2022) · cites this paper as [2]
In the citing paper
“However, naively concatenating features from different modalities might result in worse performances(Ramachandram &Taylor, 2017).”
From A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction (2022) · cites this paper as Ramachandram &Taylor, 2017
In the citing paper
“Largely, fusion strategies can be categorized according to the state of the input to the fusion layers into early, intermediate and late fusion [2] (blue layer in Figure 2).”
From Multimodal deep learning for biomedical data fusion: a review (2022) · cites this paper as [2]
In the citing paper
“Additionally, the sub-models’ errors can be uncorrelated and thus have complementary effects [2].”
From Multimodal deep learning for biomedical data fusion: a review (2022) · cites this paper as [2]
In the citing paper
“Thus, fusing modalities gradually rather than all in one layer can be beneficial [2].”
From Multimodal deep learning for biomedical data fusion: a review (2022) · cites this paper as [2]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
(2014) · Cited by 33,419
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Cited by
Items 901–1,000 of 1,002
PREDICCIÓN DE RENDIMIENTO DE COSECHAS UTILIZANDO APRENDIZAJE PROFUNDO MULTIMODAL
Jácome-Galarza, L.-R. (2024) · doi:10.22533/at.ed.7242428056
EMERGENCIA DEL PENSAMIENTO MATEMÁTICO EN UNA COMUNIDAD DE ADULTOS MAYORES
Orellana, G. E. R., & Concha, M. I. B. (2024) · doi:10.22533/at.ed.7242428059
SISTEMA DE ENCRIPTACIÓN EN FASE PARA IMÁGENES BASADO EN LA TRANSFORMADA DE HARTLEY FRACCIONARIA
Ortiz, J. M. V., Ruiz, C. J. J., & Perez, R. A. (2024) · doi:10.22533/at.ed.7242428053
Sasano, T. E., de Oliveira, P. D. T. C., da Silva, V. B., Maciel, D. L., Neto, L. S. D. L., & de Almeida, F. M. (2024) · doi:10.22533/at.ed.7242428051
CONSUMO DE ENERGIA ELÉTRICA NO BRASIL: UMA ABORDAGEM COMBINANDO AS MODELAGENS LOESS E SARIMA
Bisognin, C., Gonçalves, C. L., & Jacobi, L. F. (2024) · doi:10.22533/at.ed.7242428052
Cruz, D. M. D. L., Ramos, P. D. C. G., Montejo, J. O., & León, C. E. Y. (2024) · doi:10.22533/at.ed.7242428058
Niimi, J. (2024) · Ouyou toukeigaku · doi:10.5023/jappstat.53.15 · Cited by 1
Perceptual Enhancement Via Model Compression: Introducing the Eloss MetricPreprint
Li, C., Liu, Z., Li, F., Liu, L., Yang, Y., & Che, G. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4865153
Feature Extraction and Fusion of Multi-sensor Data Method for Electromagnetic Pulse Valve Diagnosis
Wang, M., Wang, T., Wang, B., & Ye, Z. (2023) · International Conference on Fluid Power and Mechatronics · doi:10.1109/fpm57590.2023.10565572
Goenka, A. (2022) · International Journal of Research in Science and Technology · doi:10.37648/ijrst.v12i03.009
Zhong, J., Ye, J., Jiang, H., & Barros, P. V. A. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-22270-2.00016-2
Sonny, A. V. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/bswd5
Artificial intelligence caters to banks and customer needs
Bansal, P. (2024) · South Florida Journal of Development · doi:10.46932/sfjdv5n9-027
ERL-MR: Harnessing the Power of Euler Feature Representations for Balanced Multi-modal Learning
Han, W., Cai, C., Guo, Y., & Peng, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681215 · Cited by 5
“Their research underscores that in some cases, the optimal uni-modal network outperforms the multi-modal network trained jointly, a finding which challenges conventional wisdom [35, 43, 50].”
Singh, D. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-6681-9_14
Tang, Z., Xu, W., Yu, B., Huang, J., Chen, X., & He, H. (2024) · International Conference on Mechatronics and Machine Vision in Practice (M2VIP) · doi:10.1109/m2vip62491.2024.10746073
Code Search Oriented Node-Enhanced Control Flow Graph Embedding
Xu, Y., & Peng, W. (2024) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam63643.2024.00016
Generic multimodal spatially graph network for spatially embedded network representation learning
Fan, X., & Hackl, J. (2025) · Applied Network Science · doi:10.1007/s41109-025-00751-6
“Specifically, when coevolving the features from a neighbor node v to a target node u , the node’s regional and position features of node v are abstracted by that of the node u , as shown in the Figure 2.”
Multi-modal Machine Learning in Gastrointestinal Endoscopy: A Review
Chan, I. N., Wong, P. K., Yan, T., Hu, Y., & Chan, C. I. (2024) · International Conference on Biomedical and Bioinformatics Engineering (ICBBE) · doi:10.1145/3707127.3707129 · Cited by 2
“A modality is usually defined as the output of a sensor and provides an associated dataset [28, 42], and some typical examples of common modalities include images, videos, text, audio, and data from other kinds of sensors.”
InfluentialCategory-Guided Fashion Compatibility Modeling
Guan, W., Song, X., Zhou, D., & Nie, L. (2025) · Synthesis lectures on information concepts, retrieval, and services · doi:10.1007/978-3-031-81048-0_2
Research on API Sequence Recommendation Based on Code Property Graph
Luo, X. (2024) · International Conference on Computer, Big Data and Artificial Intelligence (ICCBD+AI) · doi:10.1109/iccbd-ai65562.2024.00058
“Consequently, we utilize early fusion techniques [5] to integrate the abstract syntax tree (AST), control flow graph (CFG), and program dependency graph (PDG).”
HPCNet: Hybrid Pixel and Contour Network for Audio-Visual Speech Enhancement With Low-Quality Video
Chen, H., Zhang, C.-Y., Wang, Q., Du, J., Siniscalchi, S. M., Xiong, S.-F., & Wan, G. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3559763 · Cited by 2
A Multi-Modal Few-Shot Learning Approach for Diagnosing Cantú SyndromePreprint
Sonny, A. V. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/bswd5_v1
Cross-Modal Model-Based Approach for Fast Text-Image Retrieval with Applications
Jing, G., Dong, Z., Zheng, R., Yang, Q., Zhao, H., & Li, M. (2023) · International Conference on Information Processing and Network Provisioning (ICIPNP) · doi:10.1109/icipnp62754.2023.00062
Reinforced Label Denoising for Weakly-Supervised Audio-Visual Video Parsing
Gao, Y., Sun, X., Lv, G., Deng, Y., & Niu, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-5815-2_7 · Cited by 1
Using Information Fusion to Predict Customer Sentiment
Martins, M., Santos, F., de Almeida, A. N., Vieira, S. M., & Sousa, J. M. C. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-74000-8_20
A Systematic Literature Review on Multimodal Aspect-Based Sentiment Analysis
Divakarla, U., Krishna, K. V. H., Sharma, H., & Chandrasekarana, K. (2024) · International Conference on Parallel, Distributed and Grid Computing (PDGC) · doi:10.1109/pdgc64653.2024.10984221
A Review of Modern Channel Estimation Methods: Deep Learning Approaches
Yahya, Z. Z., Ali, D. M., & Abdalla, M. Y. (2025) · International Multi-Conference on Systems, Signals & Devices (SSD) · doi:10.1109/ssd64182.2025.10989820
Federated Cooperative Generalized Linear Model for Distributed Multimodal Data Analysis
Amini, M., Konyar, E., & Gahrooei, M. R. (2025) · IISE Transactions · doi:10.1080/24725854.2025.2511666 · Cited by 1
“Some common late fusion approaches include ensemble learning and deep late fusion (Yokoya et al ., 2017; Ramachandram and Taylor, 2017; Sagi and Rokach, 2018; Samareh et al ., 2018).”
Omnidirectional Linear Attention Module for Penalty Shootout Video Understanding
You, B., Zhang, S., & Yang, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6596-9_4
Zhang, Y., & Zhang, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3578006
“A comprehensive overview of deep learning for high-order vehicular patterns, from early fusion techniques to advanced models like Generative Adversarial Networks (GANs), was presented [12].”
Natarajan, G. (2025) · Technix International Journal for Engineering Research · doi:10.56975/tijer.v12i6.158590
Defect Prediction in CWDM Optical Modules Using Multimodal Learning
Choi, K.-J., Yang, J., Collins, B., Kim, S.-G., Lim, D.-J., & Seong, J.-T. (2025) · IEEE Access · doi:10.1109/access.2025.3582242
Single-Source Dual-Stream Representation Learning for DNA Sequence Classification
Zhou, J., Zhang, Z., Wang, M., Zhou, W., & Li, H. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733433 · Cited by 1
“For now, multi-modal approaches have emerged as a potent solu-tion for enriching information representation, demonstrating robust capabilities across a wide range of tasks [8, 17, 44].”
Generative Adversarial Networks for Longitudinal Data Imputation and Classification
Pingi, S. T. (2025) · Queensland University of Technology · doi:10.5204/thesis.eprints.258292 · Cited by 1
Collision Scenario Analysis for Autonomous Vehicles Using Multimodal Deep Learning Models
Qiu, J., Zhu, L., Zhang, M., Pan, Y., Xu, P., Gao, J. Z., & Liu, J. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00116
“These approaches are usually built by integrating one or more basic methods tailored to each data source [14], [15].”
Mahaseni, B., & Khan, N. (2025) · Empathic Computing · doi:10.70401/ec.2025.0010 · Cited by 4
Deep Learning-Enhanced Systems for Comprehensive Multimodal Health Data
Yugesh, B., & Anuradha, T. (2025) · OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · doi:10.1109/otcon65728.2025.11070888
“Genes, proteo Deep learning algorithms combine genes, proteomes, and EHR data to create patient-specific treatment recommendations [4].”
Guo, Q., Peng, S., Wang, H., Wu, P., & Yao, Y. (2025) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs65393.2025.11069715
Understanding Computer Vision for Intelligent Autonomous Systems
Parveen, S., & Tomar, A. (2025) · Wiley eBooks · doi:10.1002/9781394302734.ch10 · Cited by 1
GTransformer: Multi-view functional granulation and self-attention for tabular data modeling
Liao, L., Chen, Y., Chen, Y., & Lin, Y. (2025) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2025.109547
Ma, Y., He, H., Zhang, G., & Niu, Z. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103612
Intelligent Generative Design: A New Mechanical Design ConceptPreprint
Ning, F., Wang, Y., Li, H., Shi, Y., & Lu, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5393870
DeepCombine: Multimodal Person Recognition
Trojer, S., Bhateja, V., Peer, P., & Emeršič, Ž. (2025) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro65660.2025.11131792
“Ramachandram et al. [3] offer an in-depth analysis of these techniques, including those based on neural networks.”
Multimodal Deep Learning: Bridging Text and Image Classification with Advanced Architectures
Remiel, A. G. J., Sagayam, K. M., Sai, P. M., Nair, R., & Balaji, P. (2025) · Information systems engineering and management · doi:10.1007/978-3-031-96871-6_5
Lu, Z., Kumar, A., Chang, N. H., He, H., Lin, L., Xia, W., Yang, J., & Qu, G. (2025) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad65511.2025.11189201
“Multimodal learning [22], [23] integrates and analyzes data from multiple modalities (such as text, images, audio, and video) under a unified framework.”
Multimodal Emotion Recognition Improvement Based on Stacking Ensemble Learning
Wang, M., & Zhu, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3459-3_30
ImmFinder: A Multiomics-Based Neural Network Approach for Predicting the Immune Genes in Livestock
Thambiraja, M., Karthikeyan, P. P., Sezen, M., Senthilkumar, S., Singh, D., Onteru, S. K., & Yennamalli, R. M. (2025) · OMICS A Journal of Integrative Biology · doi:10.1177/15578100251389910 · Cited by 1
“In MDL, earlier layers capture simpler data abstractions, which are progressively re fi ned in deeper layers to form more informative and complex representations (Ramachandram and Taylor, 2017; Bengio et al., 2013).”
A Study on the Guilin Tourism Visual Corpus Based on Multi modal Geospatial Information Fusion
Sun, Y., & Wu, J. (2025) · International Journal of Natural Resources and Environmental Studies · doi:10.62051/ijnres.v7n3.13 · Cited by 1
“Within the context of information systems and human-computer interaction, Ramachandram and Taylor (2017) and Baltrušaitis et al. (2017) further refined the theory, defining multimodal information fusion as an interdisciplinary system that integrates information from visual, textual, and geospatial…”
TEFusion: A Multimodal-feature Fusion Framework for Image and Radar
Zhu, F., Lv, T., Li, D., Du, X., & Wang, C. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-3739-6_31
A Robust Multimodal Model with Context-Rich Prompts for Traffic Light State Detection
Zheng, J., Murphey, Y. L., Giordani, B., Persad, C. C., & Maher, A. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228651
“Multimodal learning is becoming increasingly popular because it can utilize information from different data sources, such as audio, text, image, etc, to improve model performance [11].”
Application of multisource heterogeneous data fusion algorithm in social management decision-making
Han, W. (2025) · International Conference on Machine Learning, Neural Networks, and Computer Software (MLNNCS) · doi:10.1117/12.3092279
“In recent years, deep-learning algorithms were well applied on feature-level fusion by discovering nonlinear correlations of multi-source data by using neural networks, for instance, using convolutional neural networks for integration of multi-sensor information [11].”
A Simulation Modeling of Temporal Multimodality in Online Streams
Alshareef, A. (2025) · Information · doi:10.3390/info16110999
“Recent artificial intelligence (AI) models [1–3] process multimodality in unprecedented ways, often quite natively using deep neural architectures [4,5].”
It's All Connected: A Survey for Multimodal Arabic AIPreprint
Aldabbas, F., Elsafty, H., & Sifa, R. (2025) · Research Square · doi:10.21203/rs.3.rs-8007923/v1
Wang, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-26405-2 · Cited by 1
“Multi-modal fusion approaches 46,47 combine skeletal and RGB data through feature-level 48 or decision-level 50 integration strategies.”
Extending Light Direction Reconstruction to Outdoor and Surface Datasets
Miller, M., Arzt, J., Nischwitz, A., & Westermann, R. (2025) · Applied Sciences · doi:10.3390/app152312779
Moser, T., Moser, M. J., & Mahnert, A. (2025) · mSystems · doi:10.1128/msystems.00039-24
“Principal component analysis was then used to condense these ratios into two components further.”
InfluentialZhang, L., Zhang, J., Tan, M., & Liang, S. (2025) · Electronics · doi:10.3390/electronics14244858
Deng, L. (2025) · International Conference on Digital Society, Information Science and Risk Management (ICDIR) · doi:10.1109/icdir66749.2025.11280665
“In recent years, deep learning technologies have achieved important breakthroughs in social media analysis [8], particularly the development of attention mechanisms [2,7] and graph neural networks [1,5], providing new technical pathways for complex cross-cultural communication modeling.”
Multimodal AI for Business Insights: Integrating Structured Data with Text, Images, and Voice
Sura, R. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.3760
“…developments in deep learning, natural language processing (NLP), computer vision, and speech recognition, to allow concurrent learning of representation and cross-modal inference, a capability which is gaining momentum as a key requirement of next-generation enterprise intelligence platforms [5].”
The Evolution of Multimodal Fusion: From Rule-Based to Deep Learning
饶, 嘉. (2025) · Computer Science and Application · doi:10.12677/csa.2025.1510244
geng, J., wang, M., Ma, P., yan, Y., Korhonen, J., lin, H., & cui, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5734548
Li, Q., & XIONG, F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5732736
Deep Feature Fusion-Based Human Activity Recognition from Multimodal Sensor Data
Jumana, F., & Deepak, S. (2025) · International Conference on Power, Instrumentation, Control and Computing (PICC) · doi:10.1109/picc67314.2025.11291292
All you need is two domains: Unified RGB-Wavelet transformer for visual representation learning
Fu, Y., Yi, W., Dong, L., Liu, M., & Kong, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115239
Truth or Lie: An Audio-Visual Approach to Deception Detection
Galli, A., Gravina, M., Pascarella, A. E., Serio, F. D., Cipollaro, D., Moscato, V., & Sansone, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-10192-1_16
Sparse Coding for Multimodal Hand Biometrics
Zhang, B., Zhao, S., Fei, L., & Li, S. (2026) · Springer eBooks · doi:10.1007/978-981-95-0970-6_16
FedMAB: adaptive multimodal federated learning with multi-armed bandits
Li, C., Gu, R., & Dong, Z. (2026) · Multimedia Systems · doi:10.1007/s00530-025-02115-7
“Multimodal Federated Learning (MFL) [1] synergistically combines the advantages of multimodal learning [2] and federated learning [3], enabling collaborative training on distributed multimodal data including images, text, and audio across different clients while preserving data privacy.”
GTNet: A Graph-Based Transformer with Cross-Modal Attention Mechanism
Yao, Q., Yang, H., & Ma, H. (2025) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin64977.2025.11279055
“However, due to the complex interaction relationships between multimodal data, these simple combination methods often struggle to fully model the information transfer between modalities [5] [6] [7] .”
A substation collaborative inspection system based on Ad-Hoc network communication nodes
Mingkai, X., Huichao, P., Siyuan, W., Liu, K., & Shiying, S. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.12.541
Dynamic object detection for smart farming using drone technology
Uma Nandhini, D., Rakesh, G., & Vignesh, G. (2026) · AIP conference proceedings · doi:10.1063/5.0298501
Qiu, L. (2026) · New perspectives in organizational communication · doi:10.1007/978-3-032-14713-4_2
Yun, S., Masukawa, R., Hassan, R., Na, M., & Imani, M. (2026) · IEEE Access · doi:10.1109/access.2026.3653660 · Cited by 2
Vu, D.-T., Horng, S., & Nguyen, T.-V. (2025) · International Conference on Artificial Intelligence for Medicine, Health and Care (AIxMHC) · doi:10.1109/aixmhc65380.2025.00026
“Multimodal fusion can be applied at different stages, including early, intermediate, and late fusion [7].”
Capturing Cross-Modal Semantics by Generating Comments for Image-Text Contents
Qian, S., Liu, B., Sun, C., Xu, Z. X., & Wang, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5679-3_10
Yang, F., & Gong, F. (2026) · Scientific Reports · doi:10.1038/s41598-026-36023-1 · Cited by 2
Non-Disassembled Classification of Malware via Attention-Driven Cross Modal CNN
SivaRamaKrishna, K., Marry, P., Sreelekha, K., Shivani, K., Gopal, V., & SatyaHemanthKumar, K. (2025) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic64647.2025.11330865
Interpretable Multimodal Fusion via ROI Graphs and Graph Information Bottleneck
Zhu, Z., Hu, Y., & Wang, Y. (2025) · International Conference on Machine Learning and Computer Application (ICMLCA) · doi:10.1109/icmlca66850.2025.11336725
“Multimodal Fusion Technology (MFT) [1], is responsible for combining information from multiple modalities to make predictions and process different forms of data.”
Son, S., Jeong, H., Yeom, J., Nam, S., An, Y., & Cho, K. H. (2026) · Environmental Research · doi:10.1016/j.envres.2026.123770 · Cited by 2
Cross-Attention Transformer with Textual Prompt Learning for Survival Prediction
Wang, P. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5631-1_21
GSToxi: Gated Cross-Modal Modeling With Graph-Sequence Encoders for Peptide Toxicity Prediction
Wang, L., Fu, X., Shi, T., Ye, X., Sakurai, T., & Liu, Y. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356648
“Fusion can be performed at different stages of a model pipeline—namely at the input level (early fusion), decision level (late fusion), or feature level (intermediate fusion) [19].”
Leal, D. A., de Souza, R. M. M., Souza, L. V. B., Biczyk, M., Amorim, M. M., & de Castro, L. N. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-04160-9_29
Dynamic Quality Assessment-Based Multi-Feature Fusion
Li, Q., Gong, Y., You, J., Hu, H., Peng, C., Peng, D., & Wang, X. (2026) · Electronics · doi:10.3390/electronics15030632 · Cited by 1
Multimodal Augmented Advantage Policy Optimization Method and Its Application
Qian, T., Li, C., & Zuo, G. (2025) · International Conference on Intelligent Medicine and Health (ICIMH) · doi:10.1145/3784929.3784931
“The integration of multimodal data like radiological images and health records is a cornerstone of clinical practice [1].”
Kumari, N., Sharma, R. P., Mehrotra, S., & Ninoria, S. Z. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21257-y
“Glaucoma is difficult to detect in its early stages, resulting in delayed diagnosis and treatment, which can lead to permanent visual loss [8].”
Liu, J., Tao, W., Wang, Y. W., Wu, Q., Xue, L., & Luo, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6194790
Wang, Z., & Li, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-6957-1_5
DAGMP: A Multimodal Learning Approach Jointly Driven by Feature Fusion and Gradient Modulation
Zhou, X., Li, L., & Lin, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-6957-1_32
Abboush, M., & Rausch, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6226241
Classifying natural and non-natural earthquakes based on multimodal data and neural networks
Dai, H., 郝永梅, Miao, F., Luan, S., Zhang, Y., Ma, Q., Li, S., Zhu, J., & Song, J. D. (2026) · Journal of Applied Geophysics · doi:10.1016/j.jappgeo.2026.106164
Flow field reconstruction in a fluidized bed with umbrella-like baffles using limited sensors
Yang, Y., He, X., Chen, X., & Zhou, Q. (2026) · Powder Technology · doi:10.1016/j.powtec.2026.122302
Local and global mortality experience: A novel hierarchical model for regional mortality risk
Nalmpatian, A., Heumann, C., Alkaya, L., & Jackson, W. (2026) · PLoS ONE · doi:10.1371/journal.pone.0312928
TCPFMC: Trustworthy Cyclic Progressive Fusion for Multimodal Classification
Li, A., Miao, D., Gao, T., Mei, S., Gu, F., Chen, C., Wang, A., Fan, H., & Liu, X. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3661974 · Cited by 3
Multipath Attention-UNet for Multimodal Prostate Image Segmentation
Qiu, Z., He, L., Yuan, D., Pan, J., Huang, H., & Hong, Y. (2025) · IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00124
“Most deep multimodal fusion works adopt this intermediate fusion approach [33].”
Liu, C., Bao, J., Chen, T., Huang, B., Gan, Y., Huang, F., & Tao, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6294407
EVLOD: Ensemble Vision-Language Open-vocabulary Detection for Construction Site Object Recognition
Wang, Y., Xiao, R., Kasahara, J. Y. L., Chikushi, S., Nagatani, K., Yamashita, A., & Asama, H. (2026) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii64115.2026.11404560
“Moreover, the advancement of multimodal fusion techniques [35] and attention mechanisms [36], [37] provides a robust technical foundation for developing more powerful ensemble detection frameworks.”
Metadata
- Authors
- Ramachandram, D., & Taylor, G. W.
- Affiliations
- Universiti Sains Malaysia (MY)
- University of Guelph (CA)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 96-108
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W2767290858
- Cited by
- 1,139
- References
- 113
- Citations by year
- 2026: 194 · 2025: 182 · 2024: 192 · 2023: 183 · 2022: 141 · 2021: 111 · 2020: 67 · 2019: 47 · 2018: 19 · 2017: 1
- FWCI
- 15.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal fusion, multimodal representation learning, multimodal architectures, cross-modal learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.