A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class Problem
Open accessThe DOI matches its Crossref record; the references it cites are not checked yet.
Abstract
In this article we carry out a comparison between early (feature) and late (score) multimodal fusion, for the two-class problem. The comparison is made first from a general perspective, and then from a specific mathematical analysis. Thus, we deduce the error probability expressions for the uncorrelated and correlated multivariate Gaussian distribution, assuming perfect model knowledge (Bayes error rates). We also deduce the corresponding expressions when the model is to be learned from a finite training set, demonstrating its convergence to the Bayes error rates as the training set size goes to infinite. These expressions also demonstrates that early fusion is the best option with model knowledge, and that both early and late fusion degrades due to a finite training set. This degradation is showed to be grater for early fusion due to the dimensionality increase of the feature space, so, eventually, late fusion could be a better option in a practical setting. The mathematical analysis also suggests the convenience of using a, so called,convergence factor, to quantify if a training set size is appropriate for the error probability to be close enough to the Bayes error rate. Different simulated experiments have been made to verify the validity of the mathematical analysis, as well as its possible extension to non-Gaussian models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Such analyses indicate that early fusion is advantageous when modalities are fully observed and jointly informative, as it enables direct modeling of cross-modal feature interactions.”
From Benchmarking Multimodal Deep Fusion Strategies for Heterogeneous Neuroimaging and Cognitive Data Using a Controlled Sex Classification Task (2026) · cites this paper as [44]
In the citing paper
“Conversely, late fusion may be more appropriate in settings characterized by modality-specific noise, conditional independence, or incomplete data.”
From Benchmarking Multimodal Deep Fusion Strategies for Heterogeneous Neuroimaging and Cognitive Data Using a Controlled Sex Classification Task (2026) · cites this paper as [44]
In the citing paper
“These findings are consistent with recent theoretical work comparing early and late fusion strategies in binary multimodal classification problems [44].”
From Benchmarking Multimodal Deep Fusion Strategies for Heterogeneous Neuroimaging and Cognitive Data Using a Controlled Sex Classification Task (2026) · cites this paper as [44]
Citations
The citation lists are queued
They are fetched later and appear here then.
References 29 · Cited by 69
Research works printed but not linked: 2
- li, 2020, Early versus late modality fusion of deep wearable sensor features for personalized prediction of tomorrow’s mood, health, and stress, Proc 42nd Annu Int Conf IEEE Eng Med Biol Soc (EMBC), 5896
- kay, 1998, Fundamentals of Statistical Signal Processing Detection Theory
Metadata
- Authors
- Pereira, L. M., Salazar, A., & Vergara, L.
- Affiliations
- Universitat Politècnica de València (ES)
- Venue
- IEEE Access
- Published
- 2023
- Volume
- 11
- Pages
- 84283-84300
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4384499195
- Cited by
- 69
- References
- 29
- Citations by year
- 2026: 18 · 2025: 34 · 2024: 13 · 2023: 4
- FWCI
- 6.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- RiuNet (Universitat Politècnica de València) (submittedVersion, cc-by-nc-nd)
- Funders
- Ministerio de Ciencia e Innovaci?n (PRE2018-085092)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Keywords
- multimodal fusion, feature-level fusion, decision fusion, binary classification, Bayes error rate, multivariate normal distribution, error probability analysis, convergence factor, feature space dimensionality, correlated features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class Problem. https://app.cochat.ai/papers/10.1109/access.2023.3296098
- The paper
- Pereira, L. M., Salazar, A., & Vergara, L. (2023). A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class Problem. IEEE Access. https://doi.org/10.1109/access.2023.3296098
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.