A Survey on Deep Learning for Multimodal Data Fusion
The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
With the wide deployments of heterogeneous networks, huge amounts of data with characteristics of high volume, high variety, high velocity, and high veracity are generated. These data, referred to multimodal big data, contain abundant intermodality and cross-modality information and pose vast challenges on traditional data fusion methods. In this review, we present some pioneering deep learning models to fuse these multimodal big data. With the increasing exploration of the multimodal big data, there are still some challenges to be addressed. Thus, this review presents a survey on deep learning for multimodal data fusion to provide readers, regardless of their original community, with the fundamentals of multimodal deep learning fusion method and to motivate new multimodal data fusion techniques of deep learning. Specifically, representative architectures that are widely used are summarized as fundamental to the understanding of multimodal deep learning. Then the current pioneering multimodal data fusion deep learning models are summarized. Finally, some challenges and future topics of multimodal data fusion deep learning models are described.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For more generally applicable models, several deep learning frameworks have been extended and enable prediction using multimodal data (Gao et al. 2020).”
From Similarity-based multimodal regression (2024) · cites this paper as Gao et al. 2020
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“ResNet (He et al., 2016) A shortcut connection is used to cross several layers to back propagate the network loss to previous layers.”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 6×
“The image subnetwork is a representative deep convolutional neural network, such as Alexnet and Inception, which effectively encodes the image input into a concise representation.”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“LSTM (Hochreiter & Schmidhuber, 1997) The memory block is introduced to model the long-time dependency well.”
BackgroundInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 1×
“Then the gradients of the weight, the visible bias, and the hidden bias are computed in the following forms: Unfortunately, in those gradient-computing equations, the probability x P ( x ) P ( h i = 1 | x ) is difficult to compute (Hinton, Osindero, & Teh, 2006).”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“Similar to the fully connected architecture, the CNN is also trained to fit the training data, using the same algorithm (LeCun et al., 1989; LeCun, Bot-tou, Bengio, & Haffner, 1998; Zeiler & Fergus, 2014).”
Method
Research works printed but not linked: 1
- Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … Bengio, Y. (2014). Generative adversarial nets. In Z.Ghahramani, M.Welling, C.Cortes, N. D.Lawrence, & K. Q.Weinberger (Eds.), Advances in neural information processing systems, 27 (pp. 2672–2680). Red Hook, NY: Curran.
Cited by
Items 701–800 of 837
Deng, Z. (2025) · IEEE International Conference on Civil Aviation Safety and Information Technology (ICCASIT) · doi:10.1109/iccasit66611.2025.11348661
“The survey by [8] highlighted how graph-based approaches can effectively capture heterogeneous interactions between users and items.”
Wu, W., Hou, Y., & Li, Q. (2025) · IEEE International Conference on Intelligent Reality (ICIR) · doi:10.1109/icir68135.2025.11361603
“Multi-modal fusion approaches have shown superior performance but required complex integration frameworks[13].”
Application of Multi-source Data Fusion in Environmental Adaptability Assessment of Power Equipment
Wu, S., zhang, X., Liu, J., Wang, Q., & Yu, H. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-7991-1_52
Acquiring Knowledge from Data Analytics and Performance-Boosting on Multimedia Content
Sama, H., Wan, J., & Firmansyah, M. D. (2026) · Komputasi Jurnal Ilmiah Ilmu Komputer dan Matematika · doi:10.33751/komputasi.v23i1.81
Abboush, M., & Rausch, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6226241
Human–Machine Teaming Agents: A Future Perspective
Teichmann, M., Ragni, M., Vitay, J., Gaedke, M., & Hamker, F. H. (2026) · Advances in Science, Technology & Innovation/Advances in science, technology & innovation · doi:10.1007/978-3-032-03488-5_30 · Cited by 1
Joint Decoding-Based Dynamic Motion Analysis for Track and Field Athletes
Meng, H. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-6946-5_20
Robust Multimodal Sensor Fusion Using Variational Autoencoder
Kodati, P., Chiplunkar, R., & Huang, B. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3667342
“Hence, deep learning-based architectures such as deep Boltzmann machines (DBMs), stacked autoencoders (SAEs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs) for multimodal sensor fusion have been explored to properly capture the intermodal-ity representations [10].”
Energy-Guided Gabor Texture Descriptors for Fusion-Based Multimodal Colon Cancer Detection
Dubey, B., Dwibedi, A., Ghosh, S., & Jana, N. D. (2025) · IEEE Conference on Information and Communication Technology (CICT) · doi:10.1109/cict67193.2025.11399196
“Furthermore, the combination of different modalities presents the issue of how to effectively fuse the disparate descriptors that are obtained from the various imaging sources, with simple averaging being one method that could potentially lead to the loss of some important information [7], [8].”
Ren, J., Chen, X., Cai, X., Guo, K., Chen, L., & Wei, J. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3668129
“Approaches [10], [11] like feature concatenation or late fusion treat modalities equally, leading to redundant or noisy representations that dilute emotionally salient patterns.”
Tan, S., Li, R., Lu, Z., & Dai, Y. (2025) · International Conference on Power System and Renewable Energy (PSRE) · doi:10.1109/psre67836.2025.11397980
“…emphasizing the strengthening of safety management and emergency firefighting capabilities for electrochemical energy storage power plants, and the development of powerful, reliable, and rapid firefighting technologies targeting the unique characteristics of lithium-ion battery fires[8-9].”
Optimized Feature-Level Fusion Model For Multimodal Biometric System
Bharathi, R. K., & Anandaraju, M. B. (2026) · International Journal on Smart Sensing and Intelligent Systems · doi:10.2478/ijssis-2026-0002
MIT-CA: Multi-modal interaction transformer with cross-attention for malware classification
Zhao, M., Yin, P., Wang, H., Hou, F., Wang, X., Song, Y., & Yao, C. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07178-1
“Early multimodal fusion research typically used differ - ent neural networks [ 33] such as LSTM, GRU, etc., as encoders to extract and fuse modal features.”
Chauhan, P., & Ranjan, R. (2025) · International Conference on IoT, Communication and Automation Technology (ICICAT) · doi:10.1109/icicat68430.2025.11414496
Fusion-VTT: Visual-Tactile-Text Fusion Learning for Robotic Object RecognitionPreprint
Yang, L., Zhang, L., & Li, M. (2026) · Research Square · doi:10.21203/rs.3.rs-9067694/v1
Multidimensional Supervised Learning over Big Data: Models, Definitions, and Solutions
Cuzzocrea, A. (2025) · International Conference on Information Communication and Software Engineering (ICICSE) · doi:10.1109/icicse66971.2025.11429755
Chauhan, P., & Ranjan, R. (2026) · Biomedical Materials & Devices · doi:10.1007/s44174-026-00658-5
When Multimodal Interactions Impair Prediction: A Novel Regularized Deep Learning Strategy
Chen, G., Xiao, S., Zhang, C., & Zhao, H. (2026) · INFORMS journal on computing · doi:10.1287/ijoc.2024.0794 · Cited by 1
Ruiqi, D., Boai, Y., Fengbo, Z., Qu, W., & TAO, L. (2026) · Journal of Image and Graphics · doi:10.11834/jig.250511
Liu, Z., Song, Y., Liu, A., Meng, X., Liu, C., Li, S., Wang, X. L., & Teng, G. (2026) · Foods · doi:10.3390/foods15061095 · Cited by 1
“Multimodal learning has therefore emerged as a promising paradigm for agricultural sensing applications [24].”
Sinha, A., Saksena, G., & Chandel, Y. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-20907-8_9
Ramakrishnan, S., Pradeepa, G., Samarasam, B., Maheswari, S. U., Brindha, P., & Parimala Devi, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0778-5.ch009
Liao, C., Liang, W., Liu, S., Huang, T., & Wu, L. (2026) · Journal of Pipeline Systems Engineering and Practice · doi:10.1061/jpsea2.pseng-1971
Survey of malware classification Models for Multi-source data
Hou, L., Li, C., Lin, R., Lin, X., Xia, M., & Shi, Y. (2026) · International Conference on Communication Networks and Machine Learning (CNML) · doi:10.1109/cnml68938.2026.11452551
“[12] Wang Shuo et al. used the deep neural network as the framework, integrated multi-scale feature representation and channel attention mechanism, and combined data augmentation to improve the generalization ability of the model to variant samples.”
Jiang, Z., & Peng, W. (2026) · Scientific Reports · doi:10.1038/s41598-026-46558-y
Visual and Textual Common Semantic Spaces for the Analysis of Multimodal Content
Beltrán, L. V. B. (2021) · theses.fr (ABES) · doi:10.70675/a19cbe0dz6c94z4ff6zb70ez024fd8546d17 · Cited by 1
Characterization and detection of semantic textual outliers
Pantin, J. (2023) · theses.fr (ABES) · doi:10.70675/3abc7c1bz47b8z44b9z9a09z23c263b7a2af
Moreau, H. (2021) · theses.fr (ABES) · doi:10.70675/1b594f01z9f35z47d2zb83fz69b74ff72897
Multimodal Document Understanding with Unified Vision and Language Cross-Modal Learning
Bakkali, S. (2022) · theses.fr (ABES) · doi:10.70675/aeb1fb70za11fz410fzaf33z6a28e0f0f154
A cognitive-inspired multimodal framework for robust deepfake detection in asynchronous media
Khalil, A. M., Benlamine, M. S., Wafa, A. A., Mosa, D. T., Abohany, A. A., & Sayedelahl, M. A. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132393
Artificial Intelligence for Multimodal ESG Report Analysis: Detecting Corporate GreenwashingPreprint
Feng, W., Siau, K. L., & Chen, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6561557
Retracted: Multimodal Learning Predicting Customer Engagement Across Social Platforms
Singh, D., Chaturvedi, A., Bhat, J. B., Lavanya, S. V., & Verma, D. (2026) · International Conference on Innovative Practices in Technology and Management (ICIPTM) · doi:10.1109/iciptm69057.2026.11465864
“Text messaging, visual stimuli and aesthetics, aural feedback and video dynamics, post-appearance context, and interaction possibilities might affect users [3].”
RetractedSaadatseresht, M., Arefi, H., & Torkamandi, F. (2026) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan15020033
A causality-guided lightweight model for 3D multi-modal brain tumor segmentation
Fan, Y., Wang, C., Zhang, X., Yue, Z., Chen, J., & Zhou, Q. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110392
Deep multimodal fusion for aquaculture: a comprehensive review
Li, D., Ji, S., Xu, W., Du, Z., Liu, S., Li, X., & Wang, G. (2026) · Aquaculture International · doi:10.1007/s10499-026-02539-6
“…data processing tasks such as VQA (Liang et al. 2021; Yin et al. 2021; Wang et al. 2022; Jiang and Meng 2023), image captioning (Wang et al. 2020a; Gao et al. 2020a), cross-modal retrieval (Deng et al. 2018; Yu et al. 2020), RGB-depth scene classification (Yuan et al. 2019), multimodal…”
A two-stage riveting holes positioning method based on point cloud mapping for aircraft skin
Jin, M., Zheng, J., Mao, J., Yi, J., Wang, Y., He, X., & Ge, E. (2025) · China Automation Congress (CAC) · doi:10.1109/cac67268.2025.11487200
“Multimodal data fusion methods [17], [18] focus on integrating vision, point clouds and digital model priors to overcome the perceptual limitations of single-modal approaches.”
Data fusion in structural health monitoring
Xu, X., & Liu, C. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45022-8.00013-9
Li, B., Yang, X., Shi, T., Meng, L., & Zhao, C. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02333-7
Federated high order tensor fusion for privacy preserving multimodal social media analysis
Li, W., & Zhang, B. (2026) · PLoS ONE · doi:10.1371/journal.pone.0344980
Li, L., & Li, J. (2026) · Scientific Reports · doi:10.1038/s41598-026-51183-w
“Hybrid fusion interpolates between these extremes, allowing intermediate representations to interact at selected network depths 22 .”
GeoMatch: Natural language-guided cross-view object geo-localization and matching
Wu, Y., Yang, H., Wu, Y., & Zhang, P. (2026) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2026.05.004 · Cited by 2
A Multimodal Data Fusion Model for Solar Radiation Prediction and Its Application in Smart Grids
Yuan, J., & Wang, Y. (2025) · IEEE International Conference on Recent Trends in Computing and Smart Mobility (RCSM) · doi:10.1109/rcsm67767.2025.11507318
“The LSTM unit selectively remembers and forgets information through input, forget, and output gates [12].”
Xu, L., Wang, X., Li, W., Jiang, Y., Cheng, C., Li, Z., Wang, Y., & Yu, L. (2026) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-026-01991-y · Cited by 1
“…to fuse indicators into a single efficiency-oriented score with explicit thresholds; the Colour Modulation Engine to map this state to colour actions under explicit safety constraints; and the Visual Actuation Module to apply these changes to the interface with complete metadata logging [70–73].”
Reddy, K. R., & Chandre, S. (2026) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict68280.2026.11511154
Recognizing anxiety and depression in cancer patients based on speech and facial expressions
Xu, N., Wu, T., Sun, L., Xin, P., Tan, X., Luo, N., & Liu, Y. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1813727
Exploring the Landscape of Data Fusion
Akter, Y. A., Abelló, A., Jovanović, P., Sagi, T., & Hose, K. (2026) · Springer eBooks · doi:10.1007/978-3-032-18765-9_2
Trustworthy Multimodal Federated Learning: A Survey of Feasibility, Adversariality, and SecurityPreprint
Yuan, Z., Luo, Y., Deng, G., Chen, S., Jiang, W., Yang, X., Wu, S., Wang, K., Wang, M., Rao, C., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6849573
Quantitative relationship mining of steel CPSP through multimodal data fusion
Zhang, C., Cao, G., Wu, S., Yan, X., Xiong, W., Wang, H., Liu, N., & Liu, Z. (2026) · Journal of Materials Science · doi:10.1007/s10853-026-13055-6
“This approach not only aids in revealing microstructural evolution patterns under complex working conditions but also significantly improves the accuracy and reliability of steel property prediction [26, 27].”
Wu, X. (2026) · Scientific Reports · doi:10.1038/s41598-026-55756-7
“Gao et al. 8 further reviewed deep learning architectures for multimodal data fusion, categorizing approaches by network structure and fusion strategy.”
Lee, S., & Park, G. (2026) · Sensors · doi:10.3390/s26113520
“Prior studies on multimodal learning have shown that simple feature concatenation may fail when modalities have different noise levels, temporal resolutions, or predictive strengths [8,9].”
InfluentialLA-YOLO: Multi-Modal Recognition Method for Low-Altitude Small Objects in Complex Scenarios
Dong, Y., & Huang, H. (2026) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp69961.2026.11540904
“However, it requires precise spatial alignment and is prone to noise amplification (e.g., from blurry IR images), which further ob-scures fragile features of low-altitude small objects [12].”
Abiew, N. A. K., Gao, L., MENSAH, G., Dong, W., & Mao, Q. (2026) · Image and Vision Computing · doi:10.1016/j.imavis.2026.106072
Huang, Y., Shi, P., He, H., & Hao, X. (2026) · Journal of Field Robotics · doi:10.1002/rob.70252
Multimodal data fusion in deep learning for diabetic retinopathy detection: A systematic review
Wardhani, K. D. K., Kasim, S. B., hassan, R. B., Hidayat, R., Sujon, K. M., & Samad, M. A. (2026) · ICT Express · doi:10.1016/j.icte.2026.06.001
Multimodal graph neural network with large language models for node and link prediction
Peng, B., Xu, H. G., & Che, X. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1758852
Alshanik, F., Husban, S., Mohammad, Y., & Ananzeh, N. (2026) · International Conference on Information and Communication Systems (ICICS) · doi:10.1145/3812734.3813706
“The experiments conducted reveal that feature fusion methods, which incorporate textual embeddings with other structured data features, lead to better event understanding in tasks such as event classification, topic modeling, and situational awareness [3], [10].”
Artificial Intelligence for Multimodal ESG Report Analysis: Detecting Corporate GreenwashingPreprint
Feng, W., Siau, K. L., & Chen, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6996371
Mao, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6995701
Deep learning for cardiovascular disease: a comprehensive review of detection and risk forecasting
Ganeshan, N., & Magesh, G. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1840804
“…and structured health records, but also include imaging, physiological sensor responses, and molecular profiles to develop a dynamic perspective of disease stratification that is beyond the reach of traditional single-modality models (Stahlschmidt et al., 2022; Gao et al., 2020) (Figure 6).”
InfluentialGandhi, V., V, & Deepa, J. (2026) · Scientific Reports · doi:10.1038/s41598-026-52281-5
BF-HFD++: Hidden Follower Detection Based on Spatio-Temporal Behavioral FusionPreprint
Hu, P., Yang, H., Cao, J., Li, J., Wu, J., & Hu, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7008300
Liu, Y., Fan, X., Zhou, D., Fu, Y., Zhao, Y., & Li, D. (2026) · Comprehensive Reviews in Food Science and Food Safety · doi:10.1111/1541-4337.70554
Kostadinov, O. (2026) · Научно списание „Сигурност и отбрана“ · doi:10.70265/iveb3293
Self-Evolving AI Core Framework for Multimodal Financial Reasoning Using a Transformer-Graph
Shah, A. K., Akkenapally, G. C., Kodela, S., & Nair, R. (2025) · IEEE International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA) · doi:10.1109/icccmla66092.2025.11580498
Phu, D. N., Pham‐Quoc, C., & Quoc, N. K. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-24399-7_30
Bridging Modalities: A Survey and Taxonomy of Multimodal AutoML (MMAutoML)
Škrlj, B., & Osojnik, A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11624-4
“This extension is relevant, as modern data sets are characterized by increasing diversity and multimodal nature (Gao et al. 2020).”
Chukkala, C. C., Shibu, D., Thushara, M. G., & Chukkala, N. K. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.05.140
Maeda, H., & Lu, Y. (2026) · Educational and Psychological Measurement · doi:10.1177/00131644261460779 · Cited by 1
“Multimodal prediction approaches depend on how information from different modalities is represented and combined in a deep fusion model (Gao et al., 2020; W. Li, Peng, et al., 2023).”
VrT: A Framework to Model Embodied VR Interaction as a Language-Like Sequence for Emotion Inference
Alaei, S., Marini, M. R., Mora, B., Rodríguez, F. C., Cinque, L., & Foresti, G. L. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2697349
Holden, W. O., Li, G., Agyemang-Duah, E., Poku, R. A., Adenyo, C., Osei-Amponsah, R., & Aggrey, S. E. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.112159
Wang, X., Liu, L., Gan, L., Dai, Y., Zhu, M., Qiu, X., & Xu, L. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111011
Jia, Q., Xue, Z., Liu, Q., Ou, S., Liu, X., & Zhao, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3394-6_49
Yang, S., & Deng, L. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c01209 · Cited by 1
Min, Y., Zhang, H., Chang, S., Guo, X. S., & Song, X. (2026) · Chemical Engineering Science · doi:10.1016/j.ces.2026.124761
Lin, S., He, D., Yang, Y., Xiong, J., & Chen, J. (2026) · International Conference on Energy Systems and Electrical Power (ICESEP) · doi:10.1109/icesep70386.2026.11607867
“…to the Mason equivalent circuit model, the input electrical impedance Z in of the piezoelectric vibrator in thickness-mode vibration is given by [14-16]: Were, 0 Near the resonant frequency, this impedance characteristic can be approximated by a series RLC resonant branch connected in parallel…”
Wang, Y. (2026) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-026-10700-z
“…high emotional sensitivity respond more intensely to emotional stimuli, individuals with analytical cognitive styles rely more on systematic information processing, while strong metacognitive abilities enable monitoring and regulation of emotion-cognition interaction processes (Gao et al., 2020).”
Multimodal fusion for fish behavior analysis in aquaculture: a survey
Wu, A., & Zhou, C. (2026) · Aquaculture International · doi:10.1007/s10499-026-02625-9 · Cited by 2
“Although multimodal systems have achieved notable performance, they still face challenges such as spatiotemporal alignment difficulties and weak generalization ability (Gao et al. 2020; Hu et al. 2023; Zhu et al. 2025a).”
CrossFuse: A Cross-Attentive Intermediate Fusion Framework for Multimodal Entity Matching
Song, J., Cao, Y., Wang, Y., Mu, C., & An, Y. (2026) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs69761.2026.11613502 · Cited by 1
“Effective multimodal fusion is essential for EM, and commonly used strategies include early fusion, late fusion, hybrid fusion, and intermediate fusion [11]–[13].”
Intra-Modal Cross-Attention Fusion for Domain Generalized Semantic Segmentation
Bätje, M., Schwalbe, G., Mütze, A., Schwonberg, M., & Leucker, M. (2026) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv66570.2026.11625210
“Another promising research direction, especially for semantic segmentation, is multimodal fusion, where multiple sensor modalities are combined often through attention-based fusion to exploit complementary information [19], [20].”
Developing Deep Learning Methods for Cognitive Impairment Detection Based on Non-invasive Data
Alsuhaibani, M. (2025) · Digital Commons - DU (University of Denver) · doi:10.56902/etdcrp.2025.98
Diao, Y., & Zhu, L. (2026) · Scientific Reports · doi:10.1038/s41598-026-64259-4
“Deep networks learn hierarchical features through stacked nonlinear maps 17 , which we write compactly as ) and train by minimising The equations that follow are textbook building blocks; we include them to fix notation, and Section III indicates which of them correspond to modules we implement.”
Temporal Modeling with Feature Fusion for Autism Spectrum Disorder Detection from Skeletal Motion
Quatra, M. L., Cammarata, V., Trovato, G., Conti, V., Salerno, V. M., Sorce, S., & Cilia, N. D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31927-2_33
Machine Learning for Multiple Sclerosis: A Systematic Review of Data and Methodologies
Elgawly, M., Elaslan, B. N., Cascio, M., Cinque, L., Foresti, G. L., Marini, M. R., Mecca, A., Collura, A., Ferrazzano, G., Leodori, G., et al. (2026) · IEEE Access · doi:10.1109/access.2026.3719868
“Multimodal approaches that combine these signals with MRI are nascent (12% of studies), currently constrained by (i) cross-modality heterogeneity, (ii) technical challenges in data fusion, and (iii) the absence of curated multimodal benchmarks [33].”
Kim, M.-S., Rhee, J.-W., Lee, S.-H., Park, S.-H., & Sung, T.-E. (2026) · Scientometrics · doi:10.1007/s11192-026-05734-7
“By combining the representations extracted from each data type through joint learning, the model comprehensively learns complementary information, enabling a more thorough observation of the quantitative and qualitative characteristics of each valuation case (Abbas et al., 2023; Gao et al., 2020).”
Elhadi, A. O., Darwish, S. M., & Mahdi, M. A. (2026) · Computers · doi:10.3390/computers15080523
Machine learning of digital traces to detect risk for behavioural addictions online
Burleigh, T. L., Hao, Y., Schivinski, B., Prokofieva, M., Zarate, D., Brown, T., & Stavropoulos, V. (2026) · Nature Reviews Psychology · doi:10.1038/s44159-026-00604-8 · Cited by 1
“Inter-mediate or hybrid fusion combines modality-specific representations within the model, enabling shared latent representations to be learned while retaining some modality-specific structure 83,84,88,89 .”
Lepcha, D. C., Ali, A., Goyal, B., Syed-Abdul, S., Gaur, L., Alon, I., & Zhang, Z. (2026) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.418738
Zhang, X., Cheng, H., Xu, T., Li, X., Cao, D., Dong, X., Li, Y.-A., Liu, X., & Wu, Y. (2026) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2026.104934
Xie, F., Zhang, Y., Shen, W., & Zheng, Y. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01105-7
“Li and Zhang’s partial domain adaptation method demonstrates how multiple classification modules can improve cross-condition transfer in intelligent machinery diagnostics, but it remains an open-loop optimization strategy rather than a controller-coupled adaptation mechanism (Li and Zhang 2020).”
Xu, X., & Zhang, P. (2026) · Scientific Reports · doi:10.1038/s41598-026-68323-x
Boulealam, C., Filali, H., Mahraz, M. A., Tairi, H., & Riffi, J. (2026) · Array · doi:10.1016/j.array.2026.101165
Digital-Twin-Enabled Remote Operation of Autonomous Systems: An Architecture-Driven Survey
Hasan, S., Hafiz, M. S., & Chowdhury, A. R. (2026) · IEEE Sensors Reviews · doi:10.1109/sr.2026.3729195 · Cited by 1
“This is a well-established research area, and dedicated surveys provide a comprehensive treatment of fusion architectures, fusion levels, and their applications in computer vision, sensor fusion, and multimodal learning [67], [68], [66].”
Vetter, D., Ahsan, M., Delicado, D., Neubauer, T. A., Wilke, T., & Roig, G. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014733
“This joint representation is then used for downstream tasks, such as classification [45,46].”
Research on MPC Optimization of Multimodal Train Autonomous Operation System Based on Deep Learning
Chang, X., & Xu, B. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-92-2254-4_27
Tan, C., & Wen, S. (2026) · International Conference on Machine Intelligence and Digital Applications (MIDA) · doi:10.1145/3801438.3801707
Brand Influence Evaluation Model of Enterprises Based on PSO-BP Neural Network
Meng, Z. (2026) · International Conference on Machine Intelligence and Digital Applications (MIDA) · doi:10.1145/3801438.3801467
Wang, J., & Zhang, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-69282-z
FUSION NET: Multi-Modal Deep Fusion Learning For Heterogeneous Big Data Analytics
Kamma, V., S.Kannan, Jyothilakshmi, P., Satyanarayana, K., S.Sridevi, & Priyadharshini, C. (2026) · International Journal of Artificial Intelligence and Machine Learning · doi:10.51483/ijaiml.6.8s.2026.181-191
Metadata
- Authors
- Gao, J., Li, P., Chen, Z., & Zhang, J.
- Affiliations
- Dalian University of Technology (CN)
- Corresponding author
- Gao, Jing
- Li, Peng
- Chen, Zhikui
- Zhang, Jianing
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2020-05-01
- Volume
- 32
- Issue
- 5
- Pages
- 829-864
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W3011727199
- Cited by
- 821
- References
- 99
- Citations by year
- 2026: 152 · 2025: 213 · 2024: 161 · 2023: 129 · 2022: 90 · 2021: 57 · 2020: 19
- FWCI
- 5.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal fusion, multimodal deep learning, heterogeneous networks, deep learning architectures, fusion model
- MeSH terms
- Deep Learning, Algorithms, Artificial Intelligence, Surveys and Questionnaires, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.