CoChat

Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects

Open access
Lahat, Dana, Adali, Tulay, Jutten, Christian · 2015 · Proceedings of the IEEE

The DOI matches its Crossref record; 214 of the 218 references it cites were found in a registry or an index.

Abstract

In various disciplines, information about the same phenomenon can be acquired from different types of detectors, at different conditions, in multiple experiments or subjects, among others. We use the term “modality” for each such acquisition framework. Due to the rich characteristics of natural phenomena, it is rare that a single modality provides complete knowledge of the phenomenon of interest. The increasing availability of several modalities reporting on the same system introduces new degrees of freedom, which raise questions beyond those related to exploiting each modality separately. As we argue, many of these questions, or “challenges,” are common to multiple domains. This paper deals with two key issues: “why we need data fusion” and “how we perform it.” The first issue is motivated by numerous examples in science and technology, followed by a mathematical framework that showcases some of the benefits that data fusion provides. In order to address the second issue, “diversity” is introduced as a key concept, and a number of data-driven solutions based on matrix and tensor decompositions are discussed, emphasizing how they account for diversity across the data sets. The aim of this paper is to provide the reader, regardless of his or her community of origin, with a taste of the vastness of the field, the prospects, and the opportunities that it holds.

Overview

Integrity signals

Registry

Crossref · 10.1109/jproc.2015.2460697

Cites

214 of 214 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 214 references

Research works printed but not linked: 11

  1. lederman, 2015, Common manifold learning using alternating-diffusion
  2. stompor, 0, Data analysis of massive data sets a Planck example, LOFAR Workshop
  3. seyyedi, 2010, Comparing satellite derived rainfall with ground based radar for North-Western Europe
  4. harshman, 1972, PARAFAC2: Mathematical and technical notes, UCLA Working Papers Phonet, 22, 30
  5. cardoso, 0, The three easy routes to independent component analysis; contrasts and geometry, Proc ICA, 1
  6. sorber, 2014, Tensorlab v2 0
  7. 0, Joint data analysis for enhanced knowledge discovery in metabolomics
  8. sørensen, 2013, Multidimensional harmonic retrieval via coupled canonical polyadic decomposition
  9. comon, 0, Supervised classification, a probabilistic approach, Proc Eur Symp Artif Neural Netw, 111
  10. harshman, 0, Models for analysis of asymmetrical relationships among $N$ objects or stimuli, Proc Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, 1
  11. adal?, 2013, IEEE Trans Multimedia, 15

Cited by

Metadata

Authors
Lahat, D., Adali, T., & Jutten, C.
Affiliations
Centre National de la Recherche Scientifique (FR)
Venue
Proceedings of the IEEE
Published
2015-09
Volume
103
Issue
9
Pages
1449-1477
ISSN
  • 0018-9219 (ISSN-L)
  • 1558-2256
OpenAlex
W1539811621
Cited by
1,296
References
218
Citations by year
2026: 155 · 2025: 239 · 2024: 177 · 2023: 135 · 2022: 123 · 2021: 102 · 2020: 105 · 2019: 93 · 2018: 69 · 2017: 63 · 2016: 31 · 2015: 4
FWCI
18.08
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the IEEE (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • Maryland Shared Open Access Repository (USMAI Consortium) (submittedVersion)
  • Digital Repository at the University of Maryland (University of Maryland College Park)
Funders
  • Project CHESS (2012-ERC-AdG-320684))
  • National Science Foundation (NSF) (IIS 1017718, CCF 1117056)
Topics
  • Physical Sciences › Mathematics › Computational Mathematics › Tensor decomposition and applications
  • Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
multimodal fusion, modality, diversity, matrix factorization, tensor decomposition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.