Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Open accessThe DOI matches its Crossref record; 214 of the 218 references it cites were found in a registry or an index.
Abstract
In various disciplines, information about the same phenomenon can be acquired from different types of detectors, at different conditions, in multiple experiments or subjects, among others. We use the term “modality” for each such acquisition framework. Due to the rich characteristics of natural phenomena, it is rare that a single modality provides complete knowledge of the phenomenon of interest. The increasing availability of several modalities reporting on the same system introduces new degrees of freedom, which raise questions beyond those related to exploiting each modality separately. As we argue, many of these questions, or “challenges,” are common to multiple domains. This paper deals with two key issues: “why we need data fusion” and “how we perform it.” The first issue is motivated by numerous examples in science and technology, followed by a mathematical framework that showcases some of the benefits that data fusion provides. In order to address the second issue, “diversity” is introduced as a key concept, and a number of data-driven solutions based on matrix and tensor decompositions are discussed, emphasizing how they account for diversity across the data sets. The aim of this paper is to provide the reader, regardless of his or her community of origin, with a taste of the vastness of the field, the prospects, and the opportunities that it holds.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There are three main types of data fusion that are used in machine learning; early (data-level), intermediate (joint), and late (decision-level)30.”
From Multimodal machine learning in precision health: A scoping review (2022) · cites this paper as [30]
In the citing paper
“These methods vary considerably in their models and applications but generally involve extensions of traditional multivariate analysis techniques such as independent component analysis, canonical correlation analysis, and singular value decomposition (Lahat et al. 2015).”
From Similarity-based multimodal regression (2024) · cites this paper as Lahat et al. 2015
In the citing paper
“While late fusion requires less data and allows for easier integration of multiple sensing strategies and algorithms, it may not fully leverage the complementary strengths of different modalities115.”
From AI-assisted multi-modal information for the screening of depression: a systematic review and meta-analysis (2025) · cites this paper as [115]
In the citing paper
“but to the best of our knowledge, there is no recent work that directly addresses the latest advances in deep multimodal learning particularly for the computer vision community.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [36]
In the citing paper
“Furthermore, some cutting-edge works [34, 36] have been proposed in recent years that address the mechanism of integrating and fusing multimodal representations inside deep learning architectures by showing the reader the possibilities this opens up for the artificial intelligence community.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [36]
Citations
Cites
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Complex brain networks: graph theoretical analysis of structural and functional systems
Bullmore, E., & Sporns, O. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2575 · Cited by 11,820 · cited 2×
“Bullmore and Sporns [165] study brain networks by first constructing separate models of structural and functional networks based on several brain imaging modalities and fuse them using a graph-theoretical framework.”
BackgroundTensor Decompositions and Applications
Kolda, T. G., & Bader, B. W. (2009) · SIAM Review · doi:10.1137/07070111x · Cited by 10,301 · cited 2×
“More properties of tensor decompositions and their uses in various engineering applications can be found for example in [64], [65], [106] and references therein.”
BackgroundHorn, R. A., & Johnson, C. R. (1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810817 · Cited by 10,280
An Introduction to Multivariate Statistical Analysis
Muirhead, R. J., & Anderson, T. W. (1986) · Journal of Business & Economic Statistics · doi:10.2307/1391399 · Cited by 9,082
Independent component analysis, A new concept?
Comon, P. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90029-9 · Cited by 8,205 · cited 2×
“Under these assumptions, separation can be achieved if the statistically independent sources are non-stationary, non-white, or nonGaussian [71], [80]–[82].”
Partial least-squares regression: a tutorial
Geladi, P., & Kowalski, B. R. (1986) · Analytica Chimica Acta · doi:10.1016/0003-2670(86)80028-9 · Cited by 6,992 · cited 1×
“We mention PLS [62], [203] and its multilinear extensions N -way PLS [62], [150] and higher-order PLS (HOPLS) [141].”
MethodPlanck 2013 results. XVI. Cosmological parameters
Planck Collaboration, Ade, P. A. R., Aghanim, N., Armitage-Caplan, C., Arnaud, M., Ashdown, M., Atrio-Barandela, F., Aumont, J., Baccigalupi, C., Banday, A. J., et al. (2014) · Astronomy & Astrophysics · doi:10.1051/0004-6361/201321591 · Cited by 6,521 · cited 1×
“surveys, galaxy clustering [54], [55], and stacked gravitational lensing, also serve as important cosmological probes [51], [52].”
BackgroundHearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · cited 4×
“Most human verbal communication involves seeing the speaker [18].”
BackgroundInfluentialCombining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Research works printed but not linked: 11
- lederman, 2015, Common manifold learning using alternating-diffusion
- stompor, 0, Data analysis of massive data sets a Planck example, LOFAR Workshop
- seyyedi, 2010, Comparing satellite derived rainfall with ground based radar for North-Western Europe
- harshman, 1972, PARAFAC2: Mathematical and technical notes, UCLA Working Papers Phonet, 22, 30
- cardoso, 0, The three easy routes to independent component analysis; contrasts and geometry, Proc ICA, 1
- sorber, 2014, Tensorlab v2 0
- 0, Joint data analysis for enhanced knowledge discovery in metabolomics
- sørensen, 2013, Multidimensional harmonic retrieval via coupled canonical polyadic decomposition
- comon, 0, Supervised classification, a probabilistic approach, Proc Eur Symp Artif Neural Netw, 111
- harshman, 0, Models for analysis of asymmetrical relationships among $N$ objects or stimuli, Proc Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, 1
- adal?, 2013, IEEE Trans Multimedia, 15
Cited by
Items 801–900 of 1,001
Yan, J., Li, D., & Yang, Q. (2025) · IEEE China Conference on System Simulation Technology and its Applications (CCSSTA) · doi:10.1109/ieeeconf65522.2025.11137082 · Cited by 2
Cao, S., Zhao, X., Du, S., & Yin, S. (2025) · Ecological Indicators · doi:10.1016/j.ecolind.2025.114203 · Cited by 5
Narigina, M., Tihak, A., Romānovs, A., Bošković, D., & Merkuryev, Y. (2025) · International Scientific Conference on Information Technology and Management Science of Riga Technical University (ITMS) · doi:10.1109/itms67030.2025.11236709 · Cited by 3
“This can be achieved through quantum simulation of biological effects and quantum optimization algorithms for complex parameter estimation problems [75].”
An adversarial scheme for integrating multi-modal data on protein function
(2025) · doi:10.1016/j.cels.2025.101444 · Cited by 2
Murray, B. A., Coops, N. C., White, J. C., Dick, A., Barbeito, I., & Ragab, A. (2025) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2025.104877 · Cited by 7
Transformer-Based Bimodal Emotion Recognition: Signal Measurement and Transformer Fusion Approach
Soleimani, S., & Osman, H. A. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3615268 · Cited by 5
“The fusion of modalities can be performed in typically three ways: feature-level [71], decision-level [72], and model-level fusion [73].”
Zhao, Y., Qi, K., Qing, Y., Tu, K., Tao, J., Li, H., Yang, C., & Zhang, H. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3649036 · Cited by 2
“Direct fusion of such heterogeneous data often fails to accurately capture complementary information and may introduce additional noise, leading to performance degradation [12].”
Buchaiah, S., & Shakya, P. (2026) · Measurement · doi:10.1016/j.measurement.2025.120157 · Cited by 4
Wu, W. D., Zhao, J., Ren, Y., Wang, Y., Tong, F., Zhao, D., & Peng, H. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.131046 · Cited by 3
Chen, L., Tang, N., Wu, X., Wang, Y., He, C., He, Z., Xie, L., Zhang, X., Chen, X., & Zhou, T. (2026) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1728353 · Cited by 3
Lo, C., Win, T. Y., Rezaeifar, Z., Khan, Z., & Legg, P. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108382 · Cited by 5
Soybean phenological stage identification based on multimodal data and a dynamic gating fusion model
Liu, Q., Jing, H., Wen, X., Wu, X., Yan, L., Yang, Q., Jia, S., Guo, S., fan, F., & Jin, X. (2026) · Smart Agricultural Technology · doi:10.1016/j.atech.2026.101827 · Cited by 2
Integrating Blended Learning Behaviors via Multimodal Fusion for Student Performance Prediction
Wan, H., Yue, S., Li, M., Luo, X., Hu, Y., Che, B., & Wang, J. (2026) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2026.3660059 · Cited by 3
“Multimodal fusion in LA is the technique of combining data from various modalities in order to gain a deeper understanding of the teaching-learning process [29].”
When Vision-Language Model (VLM) Meets Beam Prediction: A Multimodal Contrastive Learning Framework
Wang, J., Tang, B., Xiao, J. M., Cui, Q., Li, X., Liu, Y., & Quek, T. Q. S. (2026) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2026.3663461 · Cited by 2
“The performances of these single-sensor approaches are unstable and highly susceptible to environmentalchanges;consequently,researchersturnedtomultimodal fusion[12].”
Xu, S., Qi, Y., Qi, F., Guo, S., & Zhao, Z. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3667621 · Cited by 7
“Consequently, performing inference over multimodal data [2] has become a core enabler of advanced IoT applications, where multimodal data refers to heterogeneous information acquired from diverse sensing modalities—such as visual, acoustic, and textual sources—through distributed sensor networks.”
Shi, J., Zhang, Y., Li, C., Zhang, J. L., Kang, Z., Wei, S., Fu, Y., Wu, X.-B., Kong, X., Luo, A., et al. (2026) · The Astrophysical Journal Supplement Series · doi:10.3847/1538-4365/ae4003 · Cited by 2
Anumandla, S. K., Krishnakumar, A., Erramchetty, S. K., Kabilan, K., & Sandosh, S. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110006 · Cited by 2
Liu, J., Ding, X., & Gan, Y. (2026) · Clinical Psychology Review · doi:10.1016/j.cpr.2026.102728 · Cited by 5
Bi, Z., Zhang, B., Zhang, W., & Wang, C. (2026) · Computers & Geosciences · doi:10.1016/j.cageo.2026.106169 · Cited by 2
Ning, Z., Zou, Z., Yang, Y., & Jiang, Y. (2026) · Journal of Economic Surveys · doi:10.1111/joes.70099 · Cited by 2
Cao, Y., Sweetser, P., Chen, Z., & Zhu, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791197 · Cited by 2
“Inference was performed using only the held-out participant’s low-complexity session signals to predict their performance outcome in the subsequent high-complexity session, providing a strict test of prospective prediction under escalating task demands.”
Schulze, K., Biavati, F., Föllmer, B., Tsogias, S., Chimed, S., Balogh, H., Nagy, N., Yavuz, F., Stantien, A.-M., Abdelrahman, R.-H., et al. (2026) · European Radiology · doi:10.1007/s00330-026-12501-y · Cited by 2
“This can be achieved by integrating uncertainties from each modality ’ s model into a weighted decision-making process [31].”
Small-data machine learning for resolving degradation challenges in energy devices
Cheng, S., Cui, X., Ramesh, H. N., Chueh, W. C., & Sun, S. (2026) · APL Machine Learning · doi:10.1063/5.0330655 · Cited by 1
A survey on multimodal explainable Artificial Intelligence
Giovannoni, C., Metta, C., Monreale, A., & Rinzivillo, S. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200671 · Cited by 4
Multimodal machine learning in real estate appraisal: models, data, and trends
Huang, C., Liang, B., Li, Z., Wang, J., & Chen, F. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01204-8 · Cited by 3
Jiang, G.-F., Zhou, N., Wang, S.-M., & Ni, Y. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.111903 · Cited by 2
Belief Revision in Multimodal Sensing and Detection Environments
Wickramarathne, T. L. (2019) · IEEE Sensors Letters · doi:10.1109/lsens.2019.2893169 · Cited by 2
“Furthermore, in multimodal data fusion environments, underlying relationships among different sensing modalities are often unknown [1], thus making one reluctant to revise existing beliefs with complete certainty wrt a single conditioning event.”
BackgroundInfluentialAutomation of Processing Multichannel Remote Sensing Images Based on Performance Prediction
Лукин, В. В., Абрамов, С. К., Uss, M., Rubel, O. S., Kussul, N., Vozel, B., & Egiazarian, K. (2020) · IEEE Ukrainian Microwave Week (UkrMW) · doi:10.1109/ukrmw49653.2020.9252654 · Cited by 2
“Then it becomes problematic to find the corresponding control points or similar patches in order to determine parameters of geometric transformations and to carry out image representation in a common coordinate grid [21, 25].”
BackgroundEnhancing Immersiveness in Paramedicine Education XR Simulation Design
Aguayo, C., Cochrane, T., Aiello, S., & Wilkinson, N. (2021) · Pacific Journal of Technology Enhanced Learning · doi:10.24135/pjtel.v3i1.103 · Cited by 2
Xu, Q., Li, C., Wang, X., & Li, H. (2021) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2021.104281 · Cited by 2
Multimodal Remote Sensing Science and Technology
Chen, W., Li, X., & Wang, L. (2022) · Springer eBooks · doi:10.1007/978-981-19-3739-2_2 · Cited by 2
EEG–fMRI Information Fusion: Biophysics and Data Analysis
Trujillo‐Barreto, N. J., Daunizeau, J., Laufs, H., & Friston, K. J. (2022) · Springer eBooks · doi:10.1007/978-3-031-07121-8_28 · Cited by 2
Star-Net: a Multi-Branch Convolutional Network for Multiple Source Image Segmentation
Placidi, G., Cinque, L., Nappi, M., Polsinelli, M., Sciarra, A., & Tortora, G. (2022) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis57111.2022.00034 · Cited by 3
“The goal of multiple source imaging is to extract and combine relevant details from different sources, thus gathering more information than from every single source and from the simple sum of its components [4].”
BackgroundTopFusion: Using Topological Feature Space for Fusion and Imputation in Multi-Modal Data
Myers, A., Kvinge, H., & Emerson, T. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00067 · Cited by 5
“There are three types of data fusion: Early, Late, and Intermediate [19].”
BackgroundA Bottleneck Network with Light Attention for Multimodal Clustering
Liu, J., Mao, Y., Huang, Z., & Ye, Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111037 · Cited by 2
Machine Learning for Data Fusion: A Fuzzy AHP Approach for Open Issues
Kukreja, V., Abraham, A., Kalaiselvi, K., Thilak, K. D., Hariharan, S., & Chen, S.-Y. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.045136 · Cited by 3
Watanuki, S., Nomura, Y., Kiyota, Y., Kubo, M., Fujimoto, K., Okada, J., & Edo, K. (2023) · Applied Sciences · doi:10.3390/app14010378 · Cited by 3
WellFactor: Patient Profiling using Integrative Embedding of Healthcare Data
Choi, D., Xiang, A., Öztürk, Ö., Shrestha, D., Drake, B. L., Haidarian, H., Javed, F., & Park, H. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386138 · Cited by 3
“Some of the commonly utilized methods for information fusion include early fusion which merges raw data at the data representation level [38] and late fusion which solves a given problem applying solution methods separately to each data set and then merges the results [39].”
MethodZhang, Y., Zeng, Z., Wang, T., Chen, Z., & Chen, P. (2023) · International Symposium of Chinese CHI (Chinese CHI) · doi:10.1145/3629606.3629616 · Cited by 3
“Audio-visual multimodality is probably the most intuitive since it uses two of our most informative senses [7] .”
BackgroundMSV: Contribution of Modalities based on the Shapley Value
Jeon, J., Kim, J., Park, J.-W., & Kim, J. (2024) · International Conference on Consumer Electronics · doi:10.1109/icce59016.2024.10444313 · Cited by 5
“[4] However, prediction accuracy does not inform the model’s working, the reason for predictions, and the proper usage of multi-modal information.”
BackgroundTensor Decomposition-Based Data Fusion for Biomarker Extraction from Multiple EEG Experiments
Stunnenberg, K. R., Hendriks, R. C., Vroegop, J. L., Adank, M. L., & Hunyadi, B. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448073 · Cited by 2
“Data fusion techniques combine different data sources and can exploit their complementary advantages [4].”
BackgroundMulti-modal contrastive learning of subcellular organization using DICE
Nasser, R., Schaffer, L. V., Ideker, T., & Sharan, R. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae387 · Cited by 2
“While there are many potential fusion functions (Lahat et al. 2015), we simply use concatenation: Z ¼ ½ g v ð V Þj g x ð X Þ� .”
Multimodal manifold learning using kernel interpolation along geodesic paths
Katz, O., Lederman, R. R., & Talmon, R. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102637 · Cited by 2
Towards Interpretable Visuo-Tactile Predictive Models for Soft Robot Interactions
Donato, E., Thuruthel, T. G., & Falotico, E. (2024) · IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) · doi:10.1109/biorob60516.2024.10719859 · Cited by 5
“Addressing the complexities of multi-modal distributed perception presents several challenges, encompassing issues like encoding, sensory coherence, confidence across modalities, and the intricate process of fusion [7].”
BackgroundAdvances in computer AI-assisted multimodal data fusion techniques
Fan, P., & Wu, Q. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-3232 · Cited by 2
Hermosillo-Reynoso, F., & Torres-Román, D. (2024) · Sensors · doi:10.3390/s24237463 · Cited by 3
Personalized decision-making for agents in face-to-face interaction in virtual reality
Dongye, X., Weng, D., Jiang, H., Tian, Z., Bao, Y., & Chen, P. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01591-7 · Cited by 2
“Nevertheless, as highlighted by Lahat et al. [20], engaging in multimodal interactions within VR environments remains a substantial challenge, primarily due to two key factors.”
Dual-Parameter Bionic Tactile Sensing System
Lyu, C., Dai, J., & Du, Y. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3532650 · Cited by 2
“Moreover, data fusion aims to enable full interaction and information sharing among various data patterns [23].”
Rabiei, Z., & Kordy, H. M. (2025) · Neuroinformatics · doi:10.1007/s12021-025-09716-7 · Cited by 2
“The main advantage of multimodal fusion is that it benefits from the combined capabilities of individual modalities in a joint analysis, rather than analyzing each modality independently (Jiang et al., 2023; Lahat et al., 2015).”
A Meta-analysis of Multimodal Learning Analytics in Special Education
Wang, M., Xiao, W., & Sheng, B. (2024) · International Conference on Educational Technology (ICET) · doi:10.1109/icet62460.2024.10869179 · Cited by 2
“A key characteristic of multimodality is its complementarity, which enables the combination of data to provide multifaceted information about an object, facilitating a more comprehensive and systematic analysis and understanding of the object [1].”
Imaging Chemical Compositions in Three Dimensions
Wen, X., Zhou, W., Huang, Z., Zhang, H., & Han, Y. (2025) · Chemical Research in Chinese Universities · doi:10.1007/s40242-025-4235-5 · Cited by 2
Iqbal, T., Wu, G., Iqbal, Z., Mahmood, M. B., Shafique, A., & Guo, W. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2025.104032 · Cited by 2
Data Fusion in LIBS Food Analysis
Ferreira, D. D. S., Pereira‐Filho, E. R., Pereira, F. M. V., Andrade, D. F., & Gamela, R. R. (2025) · Springer eBooks · doi:10.1007/978-3-031-85975-5_17 · Cited by 2
MILS: Modality Interaction Driven Learning for Logic Synthesis
Zhao, M., Liu, J., Zhai, J., & Shi, C. (2025) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3716368.3735203 · Cited by 2
Mukai, M., Hontani, H., & Yokota, T. (2025) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2025.1594873 · Cited by 2
“There are many TD models such as canonical polyadic decomposition (CPD) [41–43, decomposition [59–61], hierarchical Tucker decomposition [62], tensor wheel decomposition [63], fully connected tensor network [64], t-SVD [65], Hankel tensor decomposition [66–69], and convolutional tensor…”
YOLO-FMEN: Pixel-Level Image Fusion-Based Nighttime Object Detection Network
Chen, Y.-T., & Da, F. (2025) · Chinese Control Conference (CCC) · doi:10.23919/ccc64809.2025.11179509 · Cited by 2
“Lahat [17]. provided a comprehensive overview of multimodal data fusion, highlighting the necessity to understand complex phenomena.”
Bikmaev, R. R., Zolotov, M. D., Popov, A. N., & Наилевич, С. Р. (2019) · Saint Petersburg International Conference on Integrated Navigation Systems · doi:10.23919/icins.2019.8769360 · Cited by 2
“However, the use of different kinds of measuring instruments requires solution of two main tasks: space-time matching and elimination of the uncertainty of sensor readings (such as noise, calibration errors, missing data, and performance restrictions) [5].”
BackgroundMethodMulti-modal Conditional Feature Enhancement for Facial Action Unit Recognition
Lakshminarayana, N. N., Mohan, D. D., Sankaran, N., Setlur, S., & Govindaraju, V. (2020) · Springer eBooks · doi:10.1007/978-3-030-30671-7_7 · Cited by 2
Improved SAR Imaging via Cross-Learning From Camera Images
Gishkori, S., Wright, D., Daniel, L., Gashinova, M. S., & Mulgrew, B. (2021) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2021.3054686 · Cited by 2
“, multiple modalities complementing each other in achieving a common goal in a way that cannot be achieved with a single modality [12].”
BackgroundMultimodal metric learning with local CCA
Yair, O., & Talmon, R. (2016) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2016.7551773 · Cited by 2
“This extensive collection and storage of multimodal data call for the development of data fusion methods and techniques [1].”
BackgroundDetection with multimodal dependent data using low-dimensional random projections
Wimalajeewa, T., & Varshney, P. K. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953032 · Cited by 2
“ntermodal dependence, likelihood ratio based detection, copula theory 1. INTRODUCTION Fusion of high dimensional heterogenous data for different inference problems is challenging in many applications [1]. While likelihood ratio (LR) based detection (with no unknown parameters) is optimal in the Bayesian setting, its o…”
BackgroundJiao, J., Li, W., Deng, Z., & Arain, Q. A. (2017) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881417717059 · Cited by 2
Fusion of Clinical Data: A Case Study to Predict the Type of Treatment of Bone Fractures
Haq, A., & Wilk, S. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-67162-8_29 · Cited by 3
A decentralised multimodal integration of social signals: a bio-inspired approach
Mansouri-Benssassi, E. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3137032 · Cited by 3
“The most commonly used fusion techniques are feature level fusion, decision level fusion and hybrid approaches [21].”
MethodSchur's Lemma for Coupled Reducibility and Coupled Normality
Lahat, D., Christian Jutten, B., & Shapiro, H. M. (2019) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/18m1232462 · Cited by 2
Self-citationA Novel Resource-aware Tensor Decomposition Design Based on Reinforcement Learning
Ghoraani, B. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683659 · Cited by 2
“Tensor decompositions have been extremely helpful in the classification cases that preknown, hand-crafted-features are unavailable [10].”
BackgroundSuperresolution via bilinear fusion of multimodal imaging data
Sarangi, P., & Pal, P. (2019) · Big Data: Learning, Analytics, and Applications · doi:10.1117/12.2520203 · Cited by 2
Combination of Multiple Classification Results Based on K-Class Alpha Integration
Safont, G., Salazar, A., & Vergara, L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20518-8_36 · Cited by 2
A Collaborative Multi-modality Selection Method Based on Data Utility Assessment
Xiao, Y., Gu, Y., Wang, J., & Wu, T. (2019) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/smartworld-uic-atc-scalcom-iop-sci.2019.00120 · Cited by 3
“Conflicts can occur when the data quality is low, since the information provided by some low-quality modalities to the model conflicts with other modalities [2].”
BackgroundApplication of the Reed-Solomon Algorithm as a Remote Sensing Data Fusion Tool for Land Use Studies
Werner, P. (2020) · Algorithms · doi:10.3390/a13080188 · Cited by 2
“[4,11–15] Earth observation (EO) initiatives play a key role in long-term remote-sensing application domains, such as LU / LC mapping, environmental monitoring, and natural resources, as well as in on-demand situations (e.g., crises), when EO data need to be streamed through time-critical workflows…”
BackgroundInfluentialBayesian Transfer Learning Between Uniformly Modelled Bayesian Filters
Jirsa, L., Pavelková, L., & Quinn, A. (2020) · Lecture notes in electrical engineering · doi:10.1007/978-3-030-63193-2_9 · Cited by 3
Yu, M., Li, C., Zeng, X.-W., & Xu, B. (2020) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct50939.2020.9295810 · Cited by 2
“The traditional process of BSS is to find a linear transformation matrix (called unmixing matrix) to make the output components independent of each other as much as possible [6], so as to maximize the recovery of the source signal.”
BackgroundDiscovering Critical Factors Affecting RDF Stores Success
Modoni, G. E., & Sacco, M. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-64619-6_8 · Cited by 2
Immersive Multimodal Environments for Psychomotor Skills Training
Sanusi, K. A. M., İren, D., & Klemke, R. (2021) · Online Publication Service of Würzburg University (Würzburg University) · doi:10.13140/rg.2.2.11172.19842 · Cited by 2
Zhao, Y., & Hu, W. S. (2021) · IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC) · doi:10.1109/icspcc52875.2021.9564487 · Cited by 2
An Intention Understanding Algorithm Based on Multimodal Information Fusion
Dou, S., Feng, Z., Tian, J., Fan, X., Hou, Y., & Zhang, X. (2021) · Scientific Programming · doi:10.1155/2021/8354015 · Cited by 2
“In recent years, many researches have been conducted on multimodal information fusion technology, and multimodal information fusion is a technology to integrate information from different sources [2].”
BackgroundOptimization of Multimodal Japanese Teaching Model Using Virtual Reality
Ye, X. (2022) · Mobile Information Systems · doi:10.1155/2022/7364279 · Cited by 3
Identification of Molecular Subtypes of Breast Cancer Based on Multimodal Deep Learning
Tang, H. (2022) · Frontiers in Medical Science Research · doi:10.25236/fmsr.2022.040410 · Cited by 1
“However, due to factors such as low sensitivity, low positive predictive value, high false positive rate and limited dimension, it is difficult to accurately identify molecular subtypes with single-mode features ([5]).”
BackgroundАлександрович, М. П. (2022) · Vestnik of the Plekhanov Russian University of Economics · doi:10.21686/2413-2829-2022-6-126-136 · Cited by 2
Fusion of Multi-Modal Underwater Ship Inspection Data with Knowledge Graphs
Hirsch, J., Elvesæter, B., Cardaillac, A., Bauer, B., & Waszak, M. (2022) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans47191.2022.9977371 · Cited by 2
“As multi-modal data carries information about the same reality, it helps to use as many different modalities as possible to fully comprehend real-world scenarios [4].”
BackgroundGood, C. D., & Megherbi, D. B. (2022) · IEEE Conference on Technologies for Homeland Security (HST) · doi:10.1109/hst56032.2022.10025446 · Cited by 2
“Following the trends outlined in several data fusion surveys [17], [18], [19], our results showed that late fusion performed better than early fusion, and our hybrid fusion model combining early and late fusion performed the best of all.”
BackgroundResultJiang, J., Guoyu, Z. H. A. O., Yingbo, M. A., Ding, G., & Lanfang, L. I. U. (2023) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2023.01002 · Cited by 3
Fine-Grained Egocentric Action Recognition with Multi-Modal Unsupervised Domain Adaptation
Liu, X., Leí, T., & Jiang, P. (2023) · IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) · doi:10.1109/itnec56291.2023.10082267 · Cited by 3
“We continue to integrate multi-modal data but at the decision-making level via late fusion [10] to avoid inaccurate temporal information fusion.”
BackgroundDigital Health Data, a Way to Take Under Control the Quality During the Elaboration Processes
Bellandi, V., D’Andrea, G., Maghool, S., & Siccardi, S. (2022) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis57111.2022.00015 · Cited by 2
“The review paper [15] describes challenges and techniques related to systems observed using multiple instruments, measurement devices, or acquisition techniques.”
BackgroundA Proximal Approach to IVA-G with Convergence Guarantees
Cosserat, C., Gabrielson, B., Chouzenoux, É., Pesquet, J.-C., & Adalı, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096421 · Cited by 2
Miller, G. F., Vásquez-Correa, J. C., Orozco‐Arroyave, J. R., & Nöth, E. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101584 · Cited by 2
Despotović, M., Koch, D., Thaler, S., Stumpe, E., Brunauer, W., & Zeppelzauer, M. (2024) · MethodsX · doi:10.1016/j.mex.2024.102556 · Cited by 2
“The integration of alternative data extraction methods, as well as the ability to handle alternative data using Machine Learning, can significantly mitigate the difficulties in modeling and provide information from new sources and modalities [ 56 ].”
MethodResultAdvancements in robotic-enabled sensing: A European perspective
Mineo, C. (2024) · Open Research Europe · doi:10.12688/openreseurope.16918.1 · Cited by 2
An Adaptive Algorithm for Tracking Third-Order Coupled Canonical Polyadic Decomposition
Liu, X., Gong, X., Zhao, D. Y., & Lin, Q. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447395 · Cited by 2
“Recently, the general family of coupled canonical polyadic decomposition (C-CPD), as a natural extension of single-dataset based CPD to the multiset scenario, has been playing an important role in the joint analysis of multiple coupled tensor datasets [1]–[15].”
BackgroundDolaptsis, K., Morellos, A., Tziotzios, G., Anagnostis, A., Katerıs, D., & Bochtis, D. (2023) · Experiment@International Conference · doi:10.1109/exp.at2358782.2023.10546102 · Cited by 2
“Multi-modal fusion based on machine learning is capable of learning representations of different modalities at various levels of abstraction, resulting in significantly improved performance [12].”
BackgroundJoint Underdetermined Blind Separation Using Cross Third-Order Cumulant and Tensor Decomposition
Luo, W., Li, X., Li, H., Jin, H., & Yang, R. (2024) · Circuits Systems and Signal Processing · doi:10.1007/s00034-024-02757-4 · Cited by 2
Two models for fusion of medical imaging data: Comparison and connections
Levin‐Schwartz, Y., Calhoun, V. D., & Adalı, T. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953341 · Cited by 2
“INTRODUCTION In many fields, the collection of data from multiple sensors has become common, since each sensor is expected to provide a different, yet complementary view of the system under study [5, 13, 14].”
BackgroundEl-Din, D. M., Hassanein, A. E., & Hassanien, E. E. (2020) · Studies in systems, decision and control · doi:10.1007/978-3-030-47411-9_28 · Cited by 2
Application of Multiblock Analysis on Small Metabolomic Multi-Tissue Dataset
Torell, F., Skotare, T., & Trygg, J. (2020) · Metabolites · doi:10.3390/metabo10070295 · Cited by 2
Joint Channel Estimation Techniques for Muti-User Massive MIMO Relay Networks
Du, J., Ye, S., Chen, Y., & Guan, Y. (2020) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct50939.2020.9295900 · Cited by 2
“In addition, tensor modeling [6-10] benefits from multiple diversity.”
BackgroundKatz, O., Talmon, R., Lo, Y., & Wu, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03619 · Cited by 2
“For a recent comprehensive reviews, see [1, 2, 3].”
Graph-based fusion for change detection in multi-spectral imagesPreprint
Jimenez-Sierra, D. A., Benítez-Restrepo, H. D., Cardonay, H. D. V., & Chanussot, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00786 · Cited by 2
“This analysis provides fundamental data for environmental protection, sustainable development, and maintenance of ecological balance [2].”
BackgroundTask-Oriented Semantic Communications for Multimodal Data.Preprint
Xie, H., Qin, Z., & Li, G. Y. (2021) · arXiv (Cornell University) · Cited by 4
The Tensor Track VII: From Quantum Gravity to Artificial IntelligencePreprint
Ouerfelli, M., Rivasseau, V., & Tamaazousti, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.10326 · Cited by 9
“…addressing Tensor PCA could easily be generalized to Tensor decomposition, which have applications in Tensor faces [36], Hyperspectral imagery [37], DNN compression [38, 39], multimodal data fusion [40], wireless communication [41], computer vision [42], natural language processing [43], etc.”
Fatori, D., Shephard, E., Naspolini, N. F., Guzmán, G. E. C., Wang, J. Y. T., Tótolo, P., Mafra, A. L., Isaias, C., Pereira-Santos, D., Russo, F. B., et al. (2024) · doi:10.31219/osf.io/ed4fb · Cited by 2
Metadata
- Authors
- Lahat, D., Adali, T., & Jutten, C.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- ORCID
- Lahat, Dana
- Venue
- Proceedings of the IEEE
- Published
- 2015-09
- Volume
- 103
- Issue
- 9
- Pages
- 1449-1477
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W1539811621
- Cited by
- 1,296
- References
- 218
- Citations by year
- 2026: 155 · 2025: 239 · 2024: 177 · 2023: 135 · 2022: 123 · 2021: 102 · 2020: 105 · 2019: 93 · 2018: 69 · 2017: 63 · 2016: 31 · 2015: 4
- FWCI
- 18.08
- Type
- journal article
- Language
- en
- Open access
- green
- hal.science
- Locations
- Proceedings of the IEEE (publishedVersion)
- HAL (submittedVersion, other-oa)
- Maryland Shared Open Access Repository (USMAI Consortium) (submittedVersion)
- Digital Repository at the University of Maryland (University of Maryland College Park)
- Funders
- Project CHESS (2012-ERC-AdG-320684))
- National Science Foundation (NSF) (IIS 1017718, CCF 1117056)
- Topics
- Physical Sciences › Mathematics › Computational Mathematics › Tensor decomposition and applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multimodal fusion, modality, diversity, matrix factorization, tensor decomposition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.