Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Open accessThe DOI matches its Crossref record; 214 of the 218 references it cites were found in a registry or an index.
Abstract
In various disciplines, information about the same phenomenon can be acquired from different types of detectors, at different conditions, in multiple experiments or subjects, among others. We use the term “modality” for each such acquisition framework. Due to the rich characteristics of natural phenomena, it is rare that a single modality provides complete knowledge of the phenomenon of interest. The increasing availability of several modalities reporting on the same system introduces new degrees of freedom, which raise questions beyond those related to exploiting each modality separately. As we argue, many of these questions, or “challenges,” are common to multiple domains. This paper deals with two key issues: “why we need data fusion” and “how we perform it.” The first issue is motivated by numerous examples in science and technology, followed by a mathematical framework that showcases some of the benefits that data fusion provides. In order to address the second issue, “diversity” is introduced as a key concept, and a number of data-driven solutions based on matrix and tensor decompositions are discussed, emphasizing how they account for diversity across the data sets. The aim of this paper is to provide the reader, regardless of his or her community of origin, with a taste of the vastness of the field, the prospects, and the opportunities that it holds.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There are three main types of data fusion that are used in machine learning; early (data-level), intermediate (joint), and late (decision-level)30.”
From Multimodal machine learning in precision health: A scoping review (2022) · cites this paper as [30]
In the citing paper
“These methods vary considerably in their models and applications but generally involve extensions of traditional multivariate analysis techniques such as independent component analysis, canonical correlation analysis, and singular value decomposition (Lahat et al. 2015).”
From Similarity-based multimodal regression (2024) · cites this paper as Lahat et al. 2015
In the citing paper
“While late fusion requires less data and allows for easier integration of multiple sensing strategies and algorithms, it may not fully leverage the complementary strengths of different modalities115.”
From AI-assisted multi-modal information for the screening of depression: a systematic review and meta-analysis (2025) · cites this paper as [115]
In the citing paper
“but to the best of our knowledge, there is no recent work that directly addresses the latest advances in deep multimodal learning particularly for the computer vision community.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [36]
In the citing paper
“Furthermore, some cutting-edge works [34, 36] have been proposed in recent years that address the mechanism of integrating and fusing multimodal representations inside deep learning architectures by showing the reader the possibilities this opens up for the artificial intelligence community.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [36]
Citations
Cites
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Complex brain networks: graph theoretical analysis of structural and functional systems
Bullmore, E., & Sporns, O. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2575 · Cited by 11,820 · cited 2×
“Bullmore and Sporns [165] study brain networks by first constructing separate models of structural and functional networks based on several brain imaging modalities and fuse them using a graph-theoretical framework.”
BackgroundTensor Decompositions and Applications
Kolda, T. G., & Bader, B. W. (2009) · SIAM Review · doi:10.1137/07070111x · Cited by 10,301 · cited 2×
“More properties of tensor decompositions and their uses in various engineering applications can be found for example in [64], [65], [106] and references therein.”
BackgroundHorn, R. A., & Johnson, C. R. (1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810817 · Cited by 10,280
An Introduction to Multivariate Statistical Analysis
Muirhead, R. J., & Anderson, T. W. (1986) · Journal of Business & Economic Statistics · doi:10.2307/1391399 · Cited by 9,082
Independent component analysis, A new concept?
Comon, P. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90029-9 · Cited by 8,205 · cited 2×
“Under these assumptions, separation can be achieved if the statistically independent sources are non-stationary, non-white, or nonGaussian [71], [80]–[82].”
Partial least-squares regression: a tutorial
Geladi, P., & Kowalski, B. R. (1986) · Analytica Chimica Acta · doi:10.1016/0003-2670(86)80028-9 · Cited by 6,992 · cited 1×
“We mention PLS [62], [203] and its multilinear extensions N -way PLS [62], [150] and higher-order PLS (HOPLS) [141].”
MethodPlanck 2013 results. XVI. Cosmological parameters
Planck Collaboration, Ade, P. A. R., Aghanim, N., Armitage-Caplan, C., Arnaud, M., Ashdown, M., Atrio-Barandela, F., Aumont, J., Baccigalupi, C., Banday, A. J., et al. (2014) · Astronomy & Astrophysics · doi:10.1051/0004-6361/201321591 · Cited by 6,521 · cited 1×
“surveys, galaxy clustering [54], [55], and stacked gravitational lensing, also serve as important cosmological probes [51], [52].”
BackgroundHearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · cited 4×
“Most human verbal communication involves seeing the speaker [18].”
BackgroundInfluentialCombining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Research works printed but not linked: 11
- lederman, 2015, Common manifold learning using alternating-diffusion
- stompor, 0, Data analysis of massive data sets a Planck example, LOFAR Workshop
- seyyedi, 2010, Comparing satellite derived rainfall with ground based radar for North-Western Europe
- harshman, 1972, PARAFAC2: Mathematical and technical notes, UCLA Working Papers Phonet, 22, 30
- cardoso, 0, The three easy routes to independent component analysis; contrasts and geometry, Proc ICA, 1
- sorber, 2014, Tensorlab v2 0
- 0, Joint data analysis for enhanced knowledge discovery in metabolomics
- sørensen, 2013, Multidimensional harmonic retrieval via coupled canonical polyadic decomposition
- comon, 0, Supervised classification, a probabilistic approach, Proc Eur Symp Artif Neural Netw, 111
- harshman, 0, Models for analysis of asymmetrical relationships among $N$ objects or stimuli, Proc Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, 1
- adal?, 2013, IEEE Trans Multimedia, 15
Cited by
Items 701–800 of 1,001
Comparison of levels and fusion approaches for multimodal biometrics
Sujana, S., & Reddy, V. S. K. (2021) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v23.i2.pp791-801 · Cited by 4
AutoFuse: A Semi-supervised Autoencoder based Multi-Sensor Fusion Framework
Kumar, K., Sahu, S., Majumdar, A., & Chandra, M. G. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533389 · Cited by 4
“It addresses various challenges related to data imperfection, data dimensionality, conflicting data etc., for more robust and accurate inferencing [2].”
BackgroundZhou, X., Liu, H., Wang, B., Huang, J., & Wang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3128929 · Cited by 4
“In addition, tensor modeling [28]–[30] benefits from multiple diversity.”
BackgroundSpeech and Language in Humanoid Robots
Cangelosi, A., & Ogata, T. (2018) · Springer eBooks · doi:10.1007/978-94-007-6046-2_135 · Cited by 4
Efficient Calibration Tool for Thermistor
(2015) · Sensors and Materials · doi:10.18494/sam.2015.1094 · Cited by 4
Recognition of Hand Gesture Using Electromyography Signal: Human-Robot Interaction
Aarthy, S. L., Malathi, V., Hamdi, M., Hilali‐Jaghdam, I., Abdel‐Khalek, S., & Mansour, R. F. (2022) · Journal of Sensors · doi:10.1155/2022/4718684 · Cited by 5
“The very much acknowledged idea that specific regular cycles and peculiarities are communicated under profoundly different actual pretenses prompts multisensor information combination [9].”
BackgroundRetractedAffective Feedback Synthesis Towards Multimodal Text and Image Data
Kumar, P., Bhatt, G., Ingle, O., Goyal, D., & Raman, B. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3589186 · Cited by 5
“Image summarization approaches consider only visual information to produce the summary of the contents, whereas the multimedia contents often contain a combination of visual and textual information [29].”
Tursini, K., Rémy, I., Cam, S. L., Louis-Dorr, V., Malka-Mahieu, H., Schwan, R., Gross, G., Laprévote, V., & Schwitzer, T. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1167654 · Cited by 5
From Nano to Macro: An overview of the IEEE Bio Image and Signal Processing Technical Committee
Aviyente, S., Frangi, A. F., Meijering, E. H. W., Muñoz‐Barrutia, A., Liebling, M., Ville, D. V. D., Olivo‐Marín, J., Kovačević, J., & Unser, M. A. (2023) · IEEE Signal Processing Magazine · doi:10.1109/msp.2023.3242833 · Cited by 8
“One of the main problems for the image and signal processing community in the years ahead will be to develop effective methods for data fusion and integration [42] to maximize the potential of multimodal and correlative imaging, as well as combining imaging and non-imaging (e.”
MethodXiao, S., Wang, J.-X., Wang, J.-W., Chen, R., & Chen, G. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2023.103534 · Cited by 3
Zeng, T., Zhang, W., Zhan, X., Xu, X., Liu, Z., Wang, B., & Zhang, X. (2024) · Remote Sensing · doi:10.3390/rs16060952 · Cited by 5
“Lahat et al. [33] identified the challenges in multimodal data fusion and divided them into two parts: the challenges caused by data collection and the challenges caused by the data source.”
BackgroundMethod of Wireless Sensor Network Data Fusion
Ma, L., Liu, J., & Luo, J. (2017) · International Journal of Online and Biomedical Engineering (iJOE) · doi:10.3991/ijoe.v13i09.7589 · Cited by 4
Fusion of clinical data: A case study to predict the type of treatment of bone fractures
Haq, A., Wilk, S., & Abelló, A. (2019) · International Journal of Applied Mathematics and Computer Science · doi:10.2478/amcs-2019-0004 · Cited by 4
“Applications of the COD method are discussed in detail by Lanckriet et al. (2004) and Kourou et al.”
BackgroundMethodRissolo, D., Hess, M. R., Herrera, J. H., Lo, E., Petrovic, V., Amador, F. E., & Kuester, F. (2019) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xlii-2-w15-989-2019 · Cited by 4
“The fusion of data produced by different imaging modalities (including those used in this study, but also potentially GPR, thermography, or hyperspectral imaging) allows for the visualization and analysis of different phenomena related to the target site (see Lahat et al., 2015).”
BackgroundCombining Chains of Bayesian Models with Markov Melding
Manderson, A. A., & Goudie, R. J. B. (2022) · Bayesian Analysis · doi:10.1214/22-ba1327 · Cited by 5
Active balance of humanoid movement based on dynamic task-prior system
Liu, C., Ning, J., An, K., & Chen, Q. (2017) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881417710793 · Cited by 4
“If N > M in equation (1), the robot system is redundant.”
BackgroundEslahi, N., & Foi, A. (2019) · IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing · doi:10.1109/camsap45676.2019.9022478 · Cited by 4
“Accurate registration is crucial, as joint recovery or fusion under imprecise registration can be detrimental [6,7].”
BackgroundHetMAML: Task-Heterogeneous Model-Agnostic Meta-Learning for Few-Shot Learning Across ModalitiesPreprint
Chen, J., & Zhang, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.07889 · Cited by 4
“However, the problem is that many tasks may not have a complete set of all available modalities due to data corruptness or data missingness [Lahat et al., 2015].”
BackgroundInfluentialStudent Engagement Recognition Using Multimodal Fusion Analytical Technology
Yan, L., Xiao, J., Wu, X., & Li, X. (2022) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste55932.2022.00049 · Cited by 4
“A key property of multimodality is complementarity, in the sense that each modality brings to the whole some type of added value that cannot be deduced or obtained from any of the other modalities in the setup[11].”
BackgroundScalable multimodal assessment of the micro-neighborhood using orthogonal visual inputs
Despotović, M., & Brunauer, W. (2024) · Journal of Housing and the Built Environment · doi:10.1007/s10901-024-10153-2 · Cited by 5
“Different modalities have different statistical distributions, internal structures, and noise components, making them difficult to compare (Lahat et al., 2015).”
Maze of conveniences: re-examining hotel reviews and social media data
Xun, J., Wang, Y., Chong, W. K., Wong, E., & Fan, X. (2025) · Industrial Management & Data Systems · doi:10.1108/imds-09-2024-0881 · Cited by 5
Data Collection in Multimodal Language and Communication Research: A Flexible Decision Framework
Bauer, A., Trettenbrein, P. C., Amici, F., Ćwiek, A., Krause, L.-M., Kuder, A., Ladewig, S. H., Schulder, M., Schumacher, P. B., Spruijt, D., et al. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459261442338 · Cited by 3
Parallel Feature Fusion for Multimodal Scene Recognition on Dual-Core MCUs
Woodward, K., & Kanjo, E. (2025) · IEEE Pervasive Computing · doi:10.1109/mprv.2025.3555334 · Cited by 5
Attend-Fusion: Efficient Audio-Visual Fusion for Video Classification
Awan, M., Nadeem, A., Awan, M. J., Mustafa, A., & Husain, S. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93806-1_15 · Cited by 5
Antonini, A., Coşar, S., Naja, I., Haleem, M. S., Macdonald, J. H., Innominato, P., & Barresi, G. (2025) · Sensors · doi:10.3390/s25123655 · Cited by 4
“The concept of multimodal data is associated with different data types (like image, text, and time series), which need to be linked together for modelling purposes via appropriate data fusion techniques [45].”
A multifactor model using large language models and multimodal investor sentiment
Zhang, J., Zhang, Z., & Wen, J. (2025) · International Review of Economics & Finance · doi:10.1016/j.iref.2025.104281 · Cited by 5
Belamri, P., Proudhon, H., Texier, D., & Ryckelynck, D. (2025) · Materials & Design · doi:10.1016/j.matdes.2025.114599 · Cited by 4
Fusing Cross-Domain Knowledge from Multimodal Data to Solve Problems in the Physical World
Zheng, Y. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768625 · Cited by 6
“To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.”
Xie, H., Huang, Q., Yu, H., Qu, Y., Yuan, M., Wu, H., Wang, J., Zhu, Z., & Li, Z. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103957 · Cited by 6
Advances in the machine learning construction process for citrus quality and disease detection
Mai, R., He, X., Li, Y., Ren, J., Gong, X., & Fan, G. (2025) · Food Control · doi:10.1016/j.foodcont.2025.111849 · Cited by 5
Wang, G., Wang, J., Huang, M., Zhang, J., Huang, X., Zhang, X., & Zhang, W. (2025) · Agriculture · doi:10.3390/agriculture15232402 · Cited by 3
A robust weighted late fusion approach for IoT
Nirala, V., & Ratneshwer (2025) · Internet of Things · doi:10.1016/j.iot.2025.101857 · Cited by 3
FlyAwareV2: A multimodal cross-domain UAV dataset for urban scene understanding
Barbato, F., Caligiuri, M., & Zanuttigh, P. (2026) · Signal Processing Image Communication · doi:10.1016/j.image.2026.117483 · Cited by 3
Visual Knowledge-Enhanced LLaVA for Fine-Grained Multimodal Named Entity Recognition and Grounding
Yuan, L., Cai, Y., Zhu, B., Liu, Z., Deng, Z., Li, Q., & Wang, T. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3651977 · Cited by 3
“W ITHtheincreaseinmultimodaldataontheinternet(suchasTwitterorFacebook)overrecentyears[1],[2],multimodal named entity recognition (MNER) tasks have received widespread attention [3], [4], [5].”
Progressive multi-fidelity learning with neural networks for physical system predictions
Conti, P., Guo, M., Frangi, A., & Manzoni, A. (2026) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2026.118881 · Cited by 5
“…types, a problem often referred to in machine learning as multi-modal learning, impacting a wealth of application fields ranging from cancer biomarker discovery [32], precision medicine [33] and neuroimaging [34] to wearable sensors [35] and meteorological monitoring [36], just to mention a few.”
How do kernel-based sensor fusion algorithms behave under high-dimensional noise?
Ding, X., & Wu, H. (2024) · Information and Inference A Journal of the IMA · doi:10.1093/imaiai/iaad051 · Cited by 3
Rocha, D. S., Fernandes, C. A. R., & Favier, G. (2023) · Digital Signal Processing · doi:10.1016/j.dsp.2023.104143 · Cited by 3
“…matrix-tensor and tensor-tensor factorizations, denoted respectively CMMF, CMTF and CTTF, have been initially used in the context of data fusion [18], with applications in various areas, such as recommendation systems, data mining, bioinformatics, neuroimaging, and signal processing, among many…”
MethodA Literature Review of Data-Driven Multimodal Learning Analytics in Education Based on Citespace
Deng, J., & Zhao, Y. (2022) · International Conference on Educational Technology Management (ICETM) · doi:10.1145/3582580.3582646 · Cited by 3
“…to the diversity of environments and natural processes, it is rarely possible to achieve a complete understanding of a phenomenon, process or environment through a single access method, each of which we call "modal", and the form of data obtained by one access method is called "unimodal data " [1].”
BackgroundDiscriminative fusion of moments-aligned latent representation of multimodality medical data
Xie, J., Zhong, W., Yang, R., Wang, L., & Zhen, X. (2023) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad1271 · Cited by 3
Multi-sensor data fusion perspective for smart grid analytics
Daş, R., & Tuna, G. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-14066-2.00006-2 · Cited by 5
Nakach, F.-Z., Idri, A., & Tchokponhoue, G. A. D. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20504-4 · Cited by 5
“Recently, the technology of multimodal data fusion based on DL has witnessed rapid development [14].”
Hongying, L. (2024) · IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS) · doi:10.1109/icpics62053.2024.10796958 · Cited by 3
“The strategy of cross-domain data consolidation [5-7] seeks to unify data from disparate sources to achieve a understanding of information.”
Wu, P., Lin, J., Ma, Z., & Li, H. (2025) · International Journal of Intelligent Systems · doi:10.1155/int/7184626 · Cited by 4
“Modality refers to a fne-grained concept when compared to multimedia data as each modality within multi-modal data contributes to a unique value to the whole, ofering insights not deducible from other modalities alone [38].”
Crivelli, D., Pellencin, E., Perin, A., & Balconi, M. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1568430 · Cited by 4
Analysis of Multimodal Social Media Data Utilizing VIT Base 16 and GPT-2 for Disaster Response
Gite, S., Patil, S., Pradhan, B., Yadav, M., Basak, S., Rajendra, A., Alamri, A. M., Raykar, K., & Kotecha, K. V. (2025) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-025-10314-7 · Cited by 6
“The growing availability of multiple datasets containing information about the same system, gathered through various acquisition methods, introduces new degrees of freedom [1].”
Dong, X., Zhao, H., Liu, Y., Qu, B., Liu, S., & Xiao, B. (2025) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2025.139761 · Cited by 4
Bourrier, A., Amblard, P., Michel, O., & Christian Jutten, B. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472511 · Cited by 3
“The term “multimodality” generally refers to the observation of a latent phenomenon through different acquisition media [1].”
BackgroundMulti-Sensor Fusion Based Robot Self-Activity Recognition
Luo, D., Ma, Y., Zhang, X., & Wu, X. H. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8624918 · Cited by 3
“Sensor fusion is usually classified into three categories, early fusion or data-level fusion, intermediate fusion or feature-level fusion and late fusion or decision-level fusion [16].”
BackgroundKhacef, L., Rodriguez, L., & Miramond, B. (2020) · Electronics · doi:10.3390/electronics9101605 · Cited by 27
Fog Server Placement for Multimodality Data Fusion in Neuroimaging
Yan, X., Xu, X., Zheng, Y., & Dai, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62223-7_20 · Cited by 3
Photo- and Radioluminescence of ZnO-Precipitated Glass-Ceramics
Masai, H., Yanagida, T., & Fujiwara, T. (2015) · Sensors and Materials · doi:10.18494/sam.2015.1061 · Cited by 3
Assisted Learning: Cooperative AI with Autonomy
Zhou, J., Xian, X., Li, N., & Ding, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414618 · Cited by 3
Determining Exception Context in Assembly Operations from Multimodal Data
Simonič, M., Hrovat, M. M., Džeroski, S., Ude, A., & Nemec, B. (2022) · Sensors · doi:10.3390/s22207962 · Cited by 3
“However, due to the unknown inter-dependencies in raw data, fusion at a higher level of abstraction may be a more helpful approach in practice [35].”
BackgroundIntegration of lncRNAs, Protein-Coding Genes and Pathology Images for Detecting Metastatic Melanoma
Liu, S., Fan, Y., Li, K., Zhang, H., Wang, X., Ju, R., Huang, L., Duan, M., & Zhou, F. (2022) · Genes · doi:10.3390/genes13101916 · Cited by 4
“Both image and OMIC data provide complementary information about melanoma, and the integrated analysis of multimodal data is important to fully utilize these different data modalities [35,36].”
A Discriminant Information Theoretic Learning Framework for Multi-modal Feature Representation
Gao, L., & Guan, L. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3587253 · Cited by 4
“To achieve the intended objectives, the challenges associated with the analysis process need to be properly addressed [10, 11].”
BackgroundMultiview Spectral Clustering of High-Dimensional Observational Data
Messina, A. R., Arvizu, C. M. C., Castillo-Tapia, A., Murillo-Aguirre, E. R., & Rodríguez, O. (2023) · IEEE Access · doi:10.1109/access.2023.3323604 · Cited by 3
“This challenge arises because multimodal data may exhibit high correlation, possess different units, or have varying structures [8], [9].”
BackgroundA novel speech feature fusion algorithm for text-independent speaker recognition
Ma, B., Xu, C., & Zhang, Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-18077-9 · Cited by 3
“The block diagram of the proposed speaker recognition system is shown in Fig.”
MAAL: Multimodality-Aware Autoencoder-based Affordance Learning for 3D Articulated Objects
Liang, Y., Wang, X., Zhu, L., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00027 · Cited by 5
Cross-Modality Calibration in Multi-Input Network for Axillary Lymph Node Metastasis Evaluation
Gravina, M., Santucci, D., Cordelli, E., Soda, P., & Sansone, C. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3397246 · Cited by 3
“Despite the potential benefits, how to exploit diverse information is still a challenging task [16], [17], [18].”
BackgroundInfluentialAdvancements in integrated robotic sensing: A European perspective
Mineo, C. (2024) · Open Research Europe · doi:10.12688/openreseurope.16918.2 · Cited by 3
“Despite the recent works 34,35 , multisource data fusion is still an open problem 36 .”
BackgroundAbada, W., & Bouramoul, A. (2024) · International Conference on Advanced Technologies for Signal and Image Processing · doi:10.1109/atsip62566.2024.10638960 · Cited by 3
“Restrictions apply. the multimodal nature of the data, and managing the longitudinal analysis [25].”
BackgroundAn effective global structure-aware feature aggregation network for multi-modal medical clustering
Guan, R., Quan, H., Li, D., & Hu, D. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126835 · Cited by 6
Langhammer, T., Unterfeld, C., Blankenburg, F., Erk, S., Fehm, L., Haynes, J., Heinzel, S., Hilbert, K., Jacobi, F., Kathmann, N., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-094110 · Cited by 4
Fusion of Multitask fMRI Data with Constrained Independent Vector Analysis
Kumbasar, E. E., Yang, H., Vu, T., Calhoun, V. D., & Adalı, T. (2025) · Conference on Information Sciences and Systems · doi:10.1109/ciss64860.2025.10944699 · Cited by 3
“Recently, data fusion has emerged as a transformative approach for achieving a more comprehensive understanding of brain function by integrating information across multiple functional tasks or imaging modalities [2].”
Big data fusion with knowledge graph: a comprehensive overview
Liu, J., Lan, R., Du, Y., Yuan, X., Xu, H., Li, T., Huang, W. X., & Zhang, P. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06549-4 · Cited by 6
“Several works in the literatures provide systematic reviews of existing data fusion methods from different perspectives [26, 43, 60, 67, 74, 91, 112, 145, 147, 150, 158], including cross-domain data fusion methods [150], urban data fusion methods [67], machine learning-based data fusion methods…”
Injury Prevention and Rehabilitation Using Machine Learning for Athletes
Yadav, M., Choudhury, T., & Huzooree, G. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-5385-1.ch007 · Cited by 5
Castro, J. P., Machado, R. C., Andrade, D. F., Babos, D. V., Pereira‐Filho, E. R., Garcia, J. A., Sperança, M. A., Gamela, R. R., & Costa, V. C. (2025) · Springer eBooks · doi:10.1007/978-3-031-85975-5_2 · Cited by 5
Detection of relationships among multi-modal brain imaging meta-features via information flow
Miller, R. L., Vergara, V. M., & Calhoun, V. D. (2017) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2017.11.006 · Cited by 3
Modelling of Fluid Flow in a Petroleum Reservoir Using an Engineering Approach
Hashan, M., Jahan, L. N., Zaman, T. U., Elhaj, M. A., Imtiaz, S. A., & Hossain, M. E. (2018) · SPE Trinidad and Tobago Section Energy Resources Conference · doi:10.2118/191153-ms · Cited by 4
A New Way of Handling Missing Data in Multi-source Classification Based on Adaptive Imputation
Abdelkhalek, I., Brahim, A. B., & Essoussi, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00856-7_8 · Cited by 3
Disjoint Subspaces for Common and Distinct Component Analysis: Application to Task FMRI Data
Akhonda, M. A. B. S., Long, Q., Bhinge, S., Calhoun, V. D., & Adalı, T. (2019) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2019.8693045 · Cited by 3
“On the other hand, multimodal data refers to information collected for the same phenomena using different modalities under same experimental conditions, tasks or subjects [4], [5].”
BackgroundThe Use of Possibility Theory in the Design of Information Fusion Systems
Solaiman, B., & Bossé, É. (2019) · Information fusion and data science · doi:10.1007/978-3-030-32853-5_9 · Cited by 5
The iCub Multisensor Datasets for Robot and Computer Vision Applications
Kırtay, M., Albanese, U., Vannucci, L., Schillaci, G., Laschi, C., & Falotico, E. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418847 · Cited by 5
“Multisensory integration is an essential skill that biological agents (e.g., humans) can perform with apparent ease to achieve a considerable number of vision-based tasks (Lahat et al. , 2015; Ramachandram & Taylor, 2017).”
Akhonda, M. A. B. S., Gabrielson, B., Calhoun, V. D., & Adalı, T. (2021) · IEEE Data Science Workshop (DSW) · doi:10.1109/dslw51110.2021.9523414 · Cited by 3
“Methods based on latent variable analysis have proven useful for joint data analysis and data fusion across various disciplines such as medical imaging, remote sensing, metabolomics, and chemomet-rics, among others [1–4].”
MacNab, Y. C. (2022) · Test · doi:10.1007/s11749-022-00832-z · Cited by 3
“…cluster analysis, classification, and anomaly detection, say, in the context of multidimensional spatial and image data analytics in general and multimodal (imaging) data fusion and analysis in particular (Sui et al. 2011, Lahat et al. 2015; Ayhan et al. 2017; Yu et al. 2018; Zhang et al. 2020).”
BackgroundForming robot trust in heterogeneous agents during a multimodal interactive game
Kırtay, M., Öztop, E., Kuhlen, A. K., Asada, M., & Hafner, V. V. (2022) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl53763.2022.9962212 · Cited by 3
“We emphasize that, in this work, the term modality refers to sensory readings that are associated with different aspects (e.g., audio and visual) of the experimental tasks [22], [23].”
BackgroundBehmanesh, M., Adibi, P., Chanussot, J., & Ehsani, M. S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128062 · Cited by 3
Improved Coupled Tensor Factorization with Its Applications in Health Data Analysis
Wu, Q., Wang, J., Fan, J., Xu, G., Wu, J. X., Johnson, B. W., Li, X., Do, Q., & Ge, R. (2019) · Complexity · doi:10.1155/2019/1574240 · Cited by 3
“For these types of real-life data fusion tasks, joint analysis is crucial [26, 27].”
BackgroundLiu, G.-R., Lo, Y.-L., Sheu, Y.-C., & Wu, H. (2020) · Springer optimization and its applications · doi:10.1007/978-3-030-61887-2_11 · Cited by 6
“lizationofthe wellknowncanonical correlation analysis (CCA) [45]. While we focus on the diffusion-based approaches in this work, there is a huge literature about sensor fusion, and we refer readers to [46] for a systematic review. 5.1. Alternating diffusion. For AD, we first form two transition matrices Ax ∈ Rn×n and A y…”
BackgroundInfluentialCentralNet: a Multilayer Approach for Multimodal Fusion
Vielzeuf, V., Lechervy, A., Pateux, S., & Jurie, F. (2018) · HAL · doi:10.48550/arxiv.1808.07275 · Cited by 3
Multi-view learning review: understanding methods and their application
배강일, 이영섭, & 임창원 (2019) · 응용통계연구 = The Korean journal of applied statistics · doi:10.5351/kjas.2019.32.1.041 · Cited by 4
Multi-modal Experts Network for Autonomous Driving
Fang, S., & Choromanska, A. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197459 · Cited by 8
“Alternative non-deep learning techniques for multi-sensor data fusion for autonomous driving were investigated in [5], [6], [7], [8].”
BackgroundFuture of Autonomous High-Mobility Military Systems
Bayrak, A. E., Gorsich, D., & Epureanu, B. I. (2020) · SAE International Journal of Connected and Automated Vehicles · doi:10.4271/12-03-03-0016 · Cited by 5
Mid-fusion of road scene polarization images on pretrained RGB neural networks
Omer, K., & Kupinski, M. K. (2021) · Journal of the Optical Society of America A · doi:10.1364/josaa.413604 · Cited by 3
Approximate Canonical Correlation Analysis for common/specific subspace decompositions
Ranta, R., Cam, S. L., Chaudet, B., Tyvaert, L., Maillard, L. G., Colnat‐Coulbois, S., & Louis-Dorr, V. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102780 · Cited by 4
AVIDA: An alternating method for visualizing and integrating data
Dover, K., Cang, Z., Ma, A., Nie, Q. H., & Vershynin, R. (2023) · Journal of Computational Science · doi:10.1016/j.jocs.2023.101998 · Cited by 4
Spectral Flow on the Manifold of SPD Matrices for Multimodal Data ProcessingPreprint
Katz, O., Lederman, R. R., & Talmon, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08062 · Cited by 7
“This particular challenge as well as many others have been the subject of many recent studies on multimodal data fusion [1, 2, 3].”
BackgroundZhan, P., Man, K., & Malone, J. (2021) · doi:10.31234/osf.io/5bprq · Cited by 3
LMM Spectrometric Determination of an Organic CompoundPreprint
Kawchak, K. (2024) · ChemRxiv · doi:10.26434/chemrxiv-2024-qtnkj · Cited by 3
Multimodal Phenotyping Dataset of Driving Fatigue
Li, J., Fu, C., Tang, J., Zhou, L., Chen, H., Zhou, W., Chen, C., & Luo, J. (2026) · Scientific Data · doi:10.1038/s41597-026-06634-4 · Cited by 3
Deep Embedding Data Fusion Scheme Using Variational Graph Auto-Encoder in IoT Environments
fathy, A. M. (2020) · International Journal of Advanced Trends in Computer Science and Engineering · doi:10.30534/ijatcse/2020/28942020 · Cited by 2
Vriesman, D., Britto, A. D. S., Zimmer, A., & Brandmeier, T. (2023) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion52260.2023.10224140 · Cited by 3
Multi-modal fusion for sensing-aided beam tracking in mmWave communications
Bian, Y., Yang, J., Dai, L., Lin, X., Cheng, X., Que, H., Liang, L., & Jin, S. (2024) · Physical Communication · doi:10.1016/j.phycom.2024.102514 · Cited by 4
Overview of Tensor-Based Cooperative MIMO Communication Systems—Part 2: Semi-Blind Receivers
Favier, G., & Rocha, D. S. (2024) · Entropy · doi:10.3390/e26110937 · Cited by 3
ADMH-ER: Adaptive Denoising Multi-Modal Hybrid for Entity Resolution
Zhou, Q., Chen, W., Zhang, L., Liu, A., Fang, J., & Zhao, L. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3526623 · Cited by 3
“Noise interference, multi-modal heterogeneity, and multi-modal semantic irrelevance significantly limit the effective uti-lizationofmulti-modaldata[23],[24].”
Ig3D: Integrating 3D Face Representations in Facial Expression Inference
Dong, L., Wang, X., Setlur, S., Govindaraju, V., & Nwogu, I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91581-9_29 · Cited by 8
Heravi, M. Y., Jang, Y., & PhD, H. C. (2025) · Journal of Construction Engineering and Management · doi:10.1061/jcemd4.coeng-15888 · Cited by 6
Li, K., Zhang, W., Xiao, D., Hou, P., Yan, S., & Mao, X. (2025) · Journal of Vibration Engineering & Technologies · doi:10.1007/s42417-025-01965-9 · Cited by 2
“Hitchcock’s concept of tensor decomposition was later refined by Cattell [18].”
Liu, C., Sun, T., Zeng, W., Wang, Y., Li, X., & Wang, Z. (2025) · European Food Research and Technology · doi:10.1007/s00217-025-04849-0 · Cited by 4
“Data fusion involves directly combining data from each source to form a fusion matrix [34], which has issues such as a strong dependency on the raw data and complex data computations [35].”
MPRNet: A Temporal-Aware Cross-Modal Encoding Framework for Personality Recognition
Li, J., Zhang, H., Chen, J., Gong, J., Wang, Y., Wang, S., & Zhao, Y. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3601134 · Cited by 3
“However, in practice, multimodal fusion presents significant challenges, particularly in effectively integrating heterogeneous data from diverse modalities [20].”
Metadata
- Authors
- Lahat, D., Adali, T., & Jutten, C.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- ORCID
- Lahat, Dana
- Venue
- Proceedings of the IEEE
- Published
- 2015-09
- Volume
- 103
- Issue
- 9
- Pages
- 1449-1477
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W1539811621
- Cited by
- 1,296
- References
- 218
- Citations by year
- 2026: 155 · 2025: 239 · 2024: 177 · 2023: 135 · 2022: 123 · 2021: 102 · 2020: 105 · 2019: 93 · 2018: 69 · 2017: 63 · 2016: 31 · 2015: 4
- FWCI
- 18.08
- Type
- journal article
- Language
- en
- Open access
- green
- hal.science
- Locations
- Proceedings of the IEEE (publishedVersion)
- HAL (submittedVersion, other-oa)
- Maryland Shared Open Access Repository (USMAI Consortium) (submittedVersion)
- Digital Repository at the University of Maryland (University of Maryland College Park)
- Funders
- Project CHESS (2012-ERC-AdG-320684))
- National Science Foundation (NSF) (IIS 1017718, CCF 1117056)
- Topics
- Physical Sciences › Mathematics › Computational Mathematics › Tensor decomposition and applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multimodal fusion, modality, diversity, matrix factorization, tensor decomposition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.