A Survey on Deep Learning for Multimodal Data Fusion
The DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
With the wide deployments of heterogeneous networks, huge amounts of data with characteristics of high volume, high variety, high velocity, and high veracity are generated. These data, referred to multimodal big data, contain abundant intermodality and cross-modality information and pose vast challenges on traditional data fusion methods. In this review, we present some pioneering deep learning models to fuse these multimodal big data. With the increasing exploration of the multimodal big data, there are still some challenges to be addressed. Thus, this review presents a survey on deep learning for multimodal data fusion to provide readers, regardless of their original community, with the fundamentals of multimodal deep learning fusion method and to motivate new multimodal data fusion techniques of deep learning. Specifically, representative architectures that are widely used are summarized as fundamental to the understanding of multimodal deep learning. Then the current pioneering multimodal data fusion deep learning models are summarized. Finally, some challenges and future topics of multimodal data fusion deep learning models are described.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For more generally applicable models, several deep learning frameworks have been extended and enable prediction using multimodal data (Gao et al. 2020).”
From Similarity-based multimodal regression (2024) · cites this paper as Gao et al. 2020
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“ResNet (He et al., 2016) A shortcut connection is used to cross several layers to back propagate the network loss to previous layers.”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 6×
“The image subnetwork is a representative deep convolutional neural network, such as Alexnet and Inception, which effectively encodes the image input into a concise representation.”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“LSTM (Hochreiter & Schmidhuber, 1997) The memory block is introduced to model the long-time dependency well.”
BackgroundInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 1×
“Then the gradients of the weight, the visible bias, and the hidden bias are computed in the following forms: Unfortunately, in those gradient-computing equations, the probability x P ( x ) P ( h i = 1 | x ) is difficult to compute (Hinton, Osindero, & Teh, 2006).”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“Similar to the fully connected architecture, the CNN is also trained to fit the training data, using the same algorithm (LeCun et al., 1989; LeCun, Bot-tou, Bengio, & Haffner, 1998; Zeiler & Fergus, 2014).”
Method
Research works printed but not linked: 1
- Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … Bengio, Y. (2014). Generative adversarial nets. In Z.Ghahramani, M.Welling, C.Cortes, N. D.Lawrence, & K. Q.Weinberger (Eds.), Advances in neural information processing systems, 27 (pp. 2672–2680). Red Hook, NY: Curran.
Cited by
Items 601–700 of 837
Text Representation through Multimodal Variational Autoencoder for One-Class Learning
Gôlo, M. P. S. (2022) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.55.2022.tde-23052022-150550
“To deal with these challenges, this study uses early fusion operators based on neural networks [4].”
MethodMental State Assessment in College English Teaching Courses Based on Deep Learning
Bei-bei, J. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/8795842
Intelligent injury prediction for traumatic airway obstruction
Han, Y., Pan, F., Song, H., Luo, R., Li, C., Pi, H., Wang, J., & Li, T. (2022) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-022-02706-w
(2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.002.0014
Artificial Autonomy or Relational Intelligence: How Relationality Matters in Health Monitoring
Ho, A. (2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.003.0002
Artificial Doctoring: The Case of Direct-to-Consumer Health Monitoring
Ho, A. (2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.003.0004
(2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.002.0003
(2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.002.0005
(2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.002.0007
Ho, A. (2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.003.0006
Independent Living With(out) Privacy: Artificial Intelligence Home Health Monitoring
Ho, A. (2023) · Oxford University Press eBooks · doi:10.1093/med/9780197556269.003.0003
Multichannel cross-fusional convolutional neural networks
Shan, C., Ou, J., & Chen, X. (2023) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-224076
“Depth map fusion algorithm refers to the information fusion of data images between multiple sensors by deep learning method [19, 20].”
Li, R. (2023) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict57646.2023.10134143
“Data fusion [1-3] technology can quickly process data and also condense data to support decision-making according to the decision-making goals set by managers.”
BackgroundREVIEW OF VECTOR EMBEDDINGS FUSION METHODS
Shaptala, R. (2022) · Telecommunication and information technologies · doi:10.31673/2412-4338.2022.048489
“Більш глибоко методи злиття на основі нейронних мереж описані у статті [6], у підсумку якої наведені недоліки саме даного типу методів.”
Text Representation through Multimodal Variational Autoencoder for One-Class Learning
Gôlo, M. P. S., & Marcacini, R. M. (2023) · doi:10.5753/ctd.2023.229471
Xu, H. (2023) · eDiss (Georg-August-Universität Göttingen) · doi:10.53846/goediss-9980
Improving information fusion in deep learning
Don, D. P. M. K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.241291
Mutli-Modality Federate Learning with Multi-Source Data for Forest Fire Prediction
Mutakabbir, A. (2023) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2023-15634 · Cited by 1
Di, L., & Yu, G. (2023) · Springer remote sensing/photogrammetry · doi:10.1007/978-3-031-33932-5_14
Cui, W., Ting-feng, L., Congli, C., & Na, D. E. (2023) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces57224.2023.10192606
“With the rapid development of information technology, multimodal data [1, 2, 3] has been widely used on the Internet, and efficient and accurate information retrieval has become very important.”
BackgroundNeural network-based descriptors as input for QSAR
Mizuno, T. (2023) · Elsevier eBooks · doi:10.1016/b978-0-443-15339-6.00007-2 · Cited by 2
Yan, W. P., Chen, C., Liu, Y., & Zhang, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4566637
Text Representation through Multimodal Variational Autoencoder for One-Class Learning
Gôlo, M. P. S., & Marcacini, R. M. (2023) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia_estendido.2023.233158
Representing Health Data and Medical Knowledge for Deep Learning
Hansen, E. R. (2023) · VBN Forskningsportal (Aalborg Universitet) · doi:10.54337/aau621651364
Minyilu, Y., Yimer, M. A., & Meshesha, M. (2024) · Research Square · doi:10.21203/rs.3.rs-3870993/v1 · Cited by 3
Feature Fusion Gate: Improving Transformer Classifier Performance with Controlled Noise
Xiang, Y., & Bai, L. (2024) · Lecture notes in computer science · doi:10.1007/978-981-99-8850-1_43 · Cited by 1
Deep Fair Partition for Hyperspectral Image Classification
He, M., Xia, F., & Stevens, R. L. (2023) · Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing · doi:10.1109/whispers61460.2023.10431031
“The following are some of the latest areas of focus in this field: Multimodal data fusion: The process of multimodal data fusion involves combining data from various sources or modalities to enhance the accuracy and robustness of data analysis [35], [36].”
BackgroundUsing Time Series Clustering to Inform Multimodal CNN Architectures
Sylvester, J. H., Lee, M., Ellis, D. C., Thornton, M. A., & Larson, E. C. (2023) · IEEE Applied Imagery Pattern Recognition Workshop · doi:10.1109/aipr60534.2023.10440715
“Other work by Srivastava and Salakhutdinov utilizes a multimodal generative model, based on deep Boltzmann machines, where multimodal representations are learned via fitting the joint distributions of multimodal data over various generative models [8].”
Detection and metrology of micro-nano particles through the integration of image and laser modes
Tao, Y., & He, H. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad3a8c
“…the algorithm performs particle recognition by leveraging a deep learning network on small batch datasets, thoroughly exploiting particle shape information to train a BP neural network, resulting in a classification model for agglomerated and non-agglomerated micro-nano particles [19, 20].”
Kumar, A., Prasad, S., & Kumar, K. N. P. (2023) · IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) · doi:10.1109/csde59766.2023.10487734
“PR = TP TP+FP [32] The recall is determined as the proportion of Positive samples that were correctly identified as Positive to all Positive samples.”
BackgroundZhang, Z., Xie, J., Zhong, W., Liang, F., Yang, R., & Zhen, X. (2024) · PubMed Central · doi:10.12122/j.issn.1673-4254.2024.01.16 · Cited by 1
Exploration on multimodal data recognition method for Internet of Things based on deep learning
Zheng, X., & Sun, Z. (2024) · Intelligent Decision Technologies · doi:10.3233/idt-230267
“Jing Gao proposed some groundbreaking DL models to fuse these multimodal big data, and summarized the in-depth learning overview of multimodal data fusion [1].”
Construction of Rehabilitation Training Platform Based on Data Fusion and Neural Network Algorithms
Ma, Z., & Ramachandra, A. C. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-97-1975-4_11
Multimodal Data Fusion in Digital Twin System of Marine RanchingPreprint
Deng, X., Zhu, Y., Chen, M., & Chen, L. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4860801
A complete pipeline for the fusion of multiple heterogeneous redundant sources
Reis, M. S., Strelet, E., Castillo, I., Peng, Y., & Chin, S.-T. (2024) · Computer-aided chemical engineering/Computer aided chemical engineering · doi:10.1016/b978-0-443-28824-1.50313-6
Despotović, M., Stumpe, E., Brunauer, W., Lehner, M., & Zeppelzauer, M. (2024) · SN Computer Science · doi:10.1007/s42979-024-03028-4
“While visual information has so far proven to be a rich unimodal source of information for many different tasks, it can also be incorporated into the modeling process simultaneously with other data such as textual data and structured data [18].”
Multimodal Outputs for the Workplace From Generative AI: Effective or Not?
Hai-Jew, S. (2024) · Advances in multimedia and interactive technologies book series · doi:10.4018/979-8-3693-2927-6.ch008
Scoping Review on Image-Text Multimodal Machine Learning Models
Rashid, M. B., & Rivas, P. (2023) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00035
“This proliferation of data encompasses multiple modalities, including visual cues in images, textual semantics, and auditory signals, which collectively provide a more comprehensive representation of the world [3], [4].”
BackgroundXu, M., & Wiese, L. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3559-4_22
End-to-End Semantic Information Transmission
Tao, X., Duan, Y., Qin, Z., Huang, D., & Wang, L. (2024) · Wireless networks · doi:10.1007/978-3-031-64155-8_5
Deformable Multi-Sequence Guided Model for Cholangiocarcinoma Staging
Song, X., Qin, Z., Huang, D., & Qin, N. (2024) · International Conference on Electronic Engineering and Informatics (EEI) · doi:10.1109/eei63073.2024.10695970
“To simulate clinical diagnosis and ensure comprehensive information extraction, model design must consider multimodal fusion [7].”
MethodCustomizable and Programmable Deep Learning
Pal, R., Kar, S., & Sekh, A. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78107-0_7
Research on Heterogeneous Network Data Fusion Based on Deep Learning
Liu, M., Liu, L., Fu, L., Hu, Z., Yu, M., & Tang, X. (2024) · International Conference on Intelligent Communications and Computing (ICICC) · doi:10.1109/icicc63565.2024.10780474
“R ELATED W ORK Gao et al. [8] undertook a review of the research progress of multimodal data fusion, discussed the limitations of traditional data fusion methods, and proposed the application advantages of deep learning models in multimodal data.”
Multi-Frame Sparse Convolutional Learning for Point Cloud Color Denoising
Yang, T., Wu, W., Zhu Li, B., & Zhou, R. (2024) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3696409.3700167
“The main types of data fusion include sensor data fusion, time-series data fusion, spatial data fusion, and modal data fusion [23].”
Quantitative Analysis of Nitrides in Water by Raman Spectroscopy Based on Deep Learning and RpmPreprint
Lai, C., Chen, X., Jiang, X., Xiang, J., & Tang, H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5088675
Review of Multimodal Data Fusion in Machine Learning
Arya, L., Sharma, Y. K., Smitha, & Doma, S. (2025) · Wiley eBooks · doi:10.1002/9781394269969.ch10
Lu, J. (2025) · IEEE Access · doi:10.1109/access.2025.3532774
“Zhang et al. [37] and Gao et al. [41] emphasize the role of CNNs and RNNs in modeling complex interdependencies across modalities such as text, images, and audio.”
Application Research of Cross-Attention Mechanism for Traffic Prediction Based on Heterogeneous Data
Feng, Z. (2025) · ITM Web of Conferences · doi:10.1051/itmconf/20257001004
Task-Oriented Single- and Multi-Source Semantic Source Coding
Chen, L., Li, Y., & Dong, Y. (2025) · IEEE Communications Letters · doi:10.1109/lcomm.2025.3554652
“Additionally, emerging autonomous scenarios necessitate multimodal semantic communication, where multiple users collaborate on a single intelligent task, with correlations among sensors involved [9], [10].”
Fitting item response theory models using deep learning computational frameworksPreprint
Luo, N., Han, Y., He, J., Zhang, X., & Ji, F. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/tjxab_v2
Ma, J., Fu, T., Wang, W., Zhou, Z., & Liang, L. (2024) · Intelligent Decision Technologies · doi:10.1177/18724981241291435
“Therefore, many data processing methods have been proposed, such as data cleaning, 19 data fusion, 20 model training, etc.”
Federated Cooperative Generalized Linear Model for Distributed Multimodal Data Analysis
Amini, M., Konyar, E., & Gahrooei, M. R. (2025) · IISE Transactions · doi:10.1080/24725854.2025.2511666 · Cited by 1
“Partial least square, tensor regression, and hybrid deep neural networks are examples of hybrid fusion (Zhao et al ., 2012; Zhou et al ., 2013; Fang et al ., 2019; Gao et al ., 2020; Gahrooei et al ., 2021).”
Zhang, Y., & Zhang, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3578006
Wang, W., Tai, X., Zhou, C. J., Liu, Z., & Zhu, P. (2025) · Accident Analysis & Prevention · doi:10.1016/j.aap.2025.108140 · Cited by 1
MADM: Microservice Anomaly Detection Based on Multi-source Data
Li, Y., & Ding, Y. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-6966-0_27
Multimodal Methods for Chest Diseases: A Review and Analysis of Recent Technological Trends
Kumar, D., & Gupta, Y. K. (2025) · OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · doi:10.1109/otcon65728.2025.11070741
“…the fusion of images created by CT and PET will be the one that lets clinicians see at once both the anatomical context within which tumors are placed and metabolic activity happening in the tumor, hence enabling much more accurate characterization along with better-informed treatment choices [4].”
A Novel Framework for Solving Pdes: Continuous Physics-Informed Neural NetworksPreprint
pan, Y., Dai, D., & Fang, Z. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5351974
Ensemble Partitioning: A Defense Mechanism Against Membership Inference Attacks in ML Models
Sun, Z., Li, Y., Shi, D., & Chen, C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9872-1_36 · Cited by 1
Optimizing flow model in trade zones based on multimodal graph neural network
Xie, R. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251366555
Deep Learning for Diabetic Retinopathy Detection: A Review of Multimodal Data Fusion ApproachesPreprint
Wardhani, K. D. K., Kasim, S. B., hassan, R. B., Hidayat, R. S., & Sujon, K. M. (2025) · Research Square · doi:10.21203/rs.3.rs-7196434/v1
Personalized Fashion Prediction Using Multimodal Fusion and Collaborative Filtering Algorithms
Liu, R., Kulevome, D. K. B., Wang, W., Cao, F., Li, N., & Tian, F. (2025) · International Symposium on Big Data and Applied Statistics (ISBDAS) · doi:10.1109/isbdas64762.2025.11116843
“With the rise of personalized services, especially private customization and personalized design, consumer demands for uniqueness and a sense of belonging have been met [7] .”
Collaborative Optimization of Multi Modal Sensing Fusion and Visual Navigation
He, P., Li, J., & Mao, Y. (2025) · Transactions on Computer Science and Intelligent Systems Research · doi:10.62051/1gzs4354
Intelligent Generative Design: A New Mechanical Design ConceptPreprint
Ning, F., Wang, Y., Li, H., Shi, Y., & Lu, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5393870
Wang, H., Song, J., Wang, C., & Ma, F. (2025) · International Conference on Computer and Multimedia Technology (ICCMT) · doi:10.1145/3757749.3757849
“The convolutional component extracts local patterns from digital behavior data, while bidirectional Long Short-Term Memory (LSTM) layers process the extracted features to capture sequential relationships, similar to approaches [8].”
Zhang, J., Lai, Y., Feng, H., Luo, X., Li, K., Wang, T., Chang, C., Yang, G., Lin, C., Li, T., et al. (2025) · Medical Physics · doi:10.1002/mp.70021
“34 An ablation experiment of the hybrid-fusion stages was conducted to explore the impact of different kinds of fusion.”
Self-citationChai, A., Fang, Z., Huang, P. H., Guo, C., Wang, L., & Yin, W. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101790
Lai, C., Chen, X., Jiang, X., Xiang, J., & Tang, H. (2025) · Vibrational Spectroscopy · doi:10.1016/j.vibspec.2025.103859
Image Content Retrieval System with Multimodal Feature Fusion
Dai, Y. M. (2025) · International Conference on Computer Vision, Application, and Algorithm (CVAA) · doi:10.1109/cvaa66438.2025.11193415
“While early fusion methods concatenate features at the input level and late fusion combines decisions from separate models, emerging approaches employ attention mechanisms and graph neural networks to model complex inter-modal relationships[7, 8].”
Machine Learning Methods for Identifying and Classifying Images in Archaeological Artifacts
Alexandrov, A. A., Vdovchenkov, E., Nebaba, A. N., Derzhanskaia, A., & Butakova, M. A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-04725-0_10
Piazza, T. D., Lazarus, C., Nempont, O., & Boussel, L. (2025) · Frontiers in Radiology · doi:10.3389/fradi.2025.1672364 · Cited by 2
“The three vector representations associated with different modalities are concatenated [54, 94] into a single vector e [ R 1026 , such that e ¼ [ e V , e [ CLS ] , e A , S ].”
Durães, D. A., Teixeira, R., Bezerra, R., & Nováis, P. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-97119-8_2
Statistical Learning for Functional Data
Cao, J., Wu, S., Nanshan, M., Shi, H., & Wang, L. (2025) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042424-052503
An Integrated Multimodal Deep Learning Framework for Skin Disease Detection
Pradhan, S., Behera, S., Reddy, N. T. K., Mishra, B. K., & Patra, M. R. (2025) · International Conference on Next Generation of Green Information and Emerging Technologies (GIET) · doi:10.1109/giet65294.2025.11234853
“These models, when integrated with transformer-based text processing modules like BERT [7], demonstrated improved performance in multimodal learning environments.”
Yang, Z., Yang, J., Huang, X., Kong, H., Yan, J., Wang, B., Li, R., & Cao, H. (2025) · Advances in Operation Research and Production Management · doi:10.54254/3029-0880/2025.29836
Application of Multimodal Data Fusion Algorithm in Early Screening of Fundus Diseases
Yao, J. (2025) · International Conference on Intelligent Communication Networks and Computational Techniques (ICICNCT) · doi:10.1109/icicnct66124.2025.11232661
“Finally, the hemodynamic features of different abstract levels are aggregated through a multi-scale feature pyramid (FPN) [12].”
Wang, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-26405-2 · Cited by 1
“Feature-level fusion mechanisms integrate information from both modalities at intermediate representation stages, enabling the network to learn joint feature representations that capture cross-modal correlations and dependencies 48 .”
Percepción de Estudiantes Universitarios sobre el Uso de Inteligencias Artificiales en su Educación
Montiel, C. H. M., Pérez, M. E. M., Moreno-Montiel, N., & Moreno-Montiel, B. (2025) · Revista Docentes 2 0 · doi:10.37843/rted.v18i2.723 · Cited by 2
Liu, Y., Min, Z., Li, T., Li, Y., Kang, H., Guo, S., Wu, W., & Zhao, T. (2025) · Journal of Circuits Systems and Computers · doi:10.1142/s0218126626420144
Lu, Y., Wang, B., Du, W. L., Li, X., & Jiang, B. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02185-0
“We can examine this issue within the broader framework of multi-modal fusion [28].”
Lee, J.-S., Lim, G.-M., Jeong, H.-R., Jang, H.-C., Lee, S., & Kim, J. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3625540
“Experimental results in exterior wall defect detection demonstrated that intermediate fusion outperformed the others by effectively preserving modality-specific features [189], [190].”
Shaikh, M., & Patil, S. (2025) · International Conference on Industry 4.0 Technology (I4Tech) · doi:10.1109/i4tech64670.2025.11277500
“In particular, multimodal learning, which integrates multiple data types such as images and text, has shown promising results in improving classification performance by leveraging complementary information [2] , [3].”
Wang, C., Teng, S. W., Florentine, S. K., Sheffield, K., & Zhang, X. (2025) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz67716.2025.11281840
“Multi-branch designs extract spectral, spatial, and joint information at multiple scales and improve unmixing on hyperspectral data [16].”
Zhang, P., Zhang, W., Sun, W., Xu, J., Hu, H., Wang, L., & Wong, L. (2024) · Figshare · doi:10.6084/m9.figshare.25216805
Zhang, Q., Hu, Y., Zhou, C., Zhao, Y., Zhang, N., Zhou, Y., Yang, Y., Zheng, H., Fan, W., Liang, D., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26657893
Shen, W.-F., Tang, H.-W., Li, J.-B., Li, X., & Chen, S. (2024) · Figshare · doi:10.6084/m9.figshare.26560802
Contrasting Traditional and LLMs-Based Approaches For Holistic Analysis of Disparate Data Types
Santra, A., & Chakravarthy, S. (2025) · International Conference on Artificial Intelligence, Big Data, Computing and Data Communication Systems (icABCD) · doi:10.1145/3759023.3759115
“Some of this work has been categorized under data fusion, multi-modal data analysis using deep learning, and others that are limited in scope and context [11, 25].”
Efficient Fireworks Algorithm Equipped with an Explosion Mechanism based on Student's t-distributionPreprint
Cen, S., & Tan, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5940068
DMF-ED: a dual-modal fusion model for emergency department service demand forecasting
Lin, R., Wu, J., & Wu, D. C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130907
Wu, M., Zhao, J., Li, H., & Zhang, Y. (2025) · Remote Sensing · doi:10.3390/rs18010054
A Deep Learning-Based Multimodal Feature Fusion Approach for Sentiment Classification
高, 泽. (2025) · Service Science and Management · doi:10.12677/ssem.2026.151002
Sharing Transport Planning Information in Traffic Simulation Software
Chekina, E. V., & Golovnin, O. K. (2025) · International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA) · doi:10.1109/summa68668.2025.11302212
“Another common method is the use of machine learning to extract and combine features from multi-modal data [14, 15].”
A Review of Multi-source and Multi-modal Anomaly Data Detection for New Power Systems
Xie, Q., Deng, S., & Zhang, X. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-4821-7_1
Reis, M. S. (2026) · Quality Engineering · doi:10.1080/08982112.2024.2428235
“…such as SO-PLS (Næs et al. 2011), NI-SL (Reis 2013a, 2013b), ROSA (Liland, Næs, and Indahl 2016) and data fusion platforms (Becerra et al. 2021; Gao et al. 2020; Sansana et al. 2020; Strelet et al. 2021) provide possible avenues for integrating multiple and heterogeneous data sources; see also…”
Chauhan, P., & Ranjan, R. (2025) · International Conference on Computing, Communication and Automation · doi:10.1109/iccca66364.2025.11325596
“Such datasets span structured, semi-structured, and unstructured forms and are often collected via diverse acquisition methods, creating both opportunities and modeling challenges beyond single-dataset analysis [1], [2].”
Ochieng, J. K., Anino, E., & Olal, F. O. (2026) · Journal of Science Innovation and Creativity · doi:10.58721/jsic.v5i1.1542
Hu, J., Ma, A., Xiao, J., Wang, Y., & Wang, Y. (2026) · ACS ES&T Water · doi:10.1021/acsestwater.5c00925
Metadata
- Authors
- Gao, J., Li, P., Chen, Z., & Zhang, J.
- Affiliations
- Dalian University of Technology (CN)
- Corresponding author
- Gao, Jing
- Li, Peng
- Chen, Zhikui
- Zhang, Jianing
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2020-05-01
- Volume
- 32
- Issue
- 5
- Pages
- 829-864
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W3011727199
- Cited by
- 821
- References
- 99
- Citations by year
- 2026: 152 · 2025: 213 · 2024: 161 · 2023: 129 · 2022: 90 · 2021: 57 · 2020: 19
- FWCI
- 5.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- multimodal fusion, multimodal deep learning, heterogeneous networks, deep learning architectures, fusion model
- MeSH terms
- Deep Learning, Algorithms, Artificial Intelligence, Surveys and Questionnaires, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.