The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Optimal multimodal fusion for multimedia data analysis
Abstract
Considerable research has been devoted to utilizing multimodal features for better understanding multimedia data. However, two core research issues have not yet been adequately addressed. First, given a set of features extracted from multiple media sources (e.g., extracted from the visual, audio, and caption track of videos), how do we determine the best modalities? Second, once a set of modal-ities has been identified, how do we best fuse them to map to se-mantics? In this paper, we propose a two-step approach. The first step finds statistically independent modalities from raw features. In the second step, we use super-kernel fusion to determine the optimal combination of individual modalities. We carefully ana-lyze the tradeoffs between three design factors that affect fusion performance: modality independence, curse of dimensionality, and fusion-model complexity. Through analytical and empirical studies, we demonstrate that our two-step approach, which achieves a care-ful balance of the three design factors, can improve class-prediction accuracy over traditional techniques.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Attention-based fusion [49,50,51] aligns key features and dynamically assigns weights between modalities for deeper semantic capture.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [49]
Citations
Cites
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986
Goodall, C. R., & Jolliffe, I. T. (1988) · Technometrics · doi:10.2307/1270093 · Cited by 8,797
Jolliffe, I. T. (1986) · Springer series in statistics · doi:10.1007/978-1-4757-1904-8 · Cited by 8,481
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 4,834
A Fast Fixed-Point Algorithm for Independent Component Analysis
Hyvärinen, A., & Erkki Oja, A. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1483 · Cited by 3,373
Query by image and video content: the QBIC system
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petković, D., et al. (1995) · Computer · doi:10.1109/2.410146 · Cited by 3,097
Adaptive Control Processes: A Guided Tour
Bellman, R. E. (1961) · doi:10.1515/9781400874668 · Cited by 2,802
Solving Multiclass Learning Problems via Error-Correcting Output Codes
Dietterich, T. G., & Bakiri, G. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.105 · Cited by 2,739
When Is ''Nearest Neighbor'' Meaningful?
Beyer, K., Goldstein, J., Ramakrishnan, R., & Shaft, U. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-49257-7_15 · Cited by 2,403
A New Learning Algorithm for Blind Signal Separation
Амари, Ш.-И., Cichocki, A. S., & Yang, H. H. (1995) · Cited by 1,904
Research works printed but not linked: 10
- A. Amir , H. W, G. Iyengar , C.- Y. Lin , M. Naphade , A. Natsev , C. Neti , H. J. Nock , J. R. Smith , B. L. Tseng , Y. Wu , and D. Zhang . IBM research TRECVID-2003 system . NIST Text Retrieval Conf. (TREC) , 2003 .]] A. Amir, H. W, G. Iyengar, C.-Y.Lin, M. Naphade, A. Natsev, C. Neti, H. J. Nock, J. R. Smith, B. L. Tseng, Y. Wu, and D. Zhang. IBM research TRECVID-2003 system. NIST Text Retrieval Conf. (TREC), 2003.]]
- 10.5555/645503.656271 · DOI
- 10.5555/521772.791882 · DOI
- 10.5555/1622826.1622834 · DOI
- 10.5555/645496.658058 · DOI
- D. L. Donoho . High-dimensional data analysis: The curses and blessings of dimensionality . American Math. Society Lecture---Match Challenges of the 21st Century , 2000 .]] D. L. Donoho. High-dimensional data analysis: The curses and blessings of dimensionality. American Math. Society Lecture---Match Challenges of the 21st Century, 2000.]]
- Y. Rui , T. S. Huang , and S. F. Chang . Image retrieval: Past, present, and future . International Symposium on Multimedia Information Processing , 1997 .]] Y. Rui, T. S. Huang, and S. F. Chang. Image retrieval: Past, present, and future. International Symposium on Multimedia Information Processing, 1997.]]
- J. R. Smith and S. F. Chang . Automatic image retrieval using color and texture . IEEE Trans Pattern Anal Mach Intell , 1996 .]] J. R. Smith and S. F. Chang. Automatic image retrieval using color and texture. IEEE Trans Pattern Anal Mach Intell, 1996.]]
- 10.5555/1622859.1622868 · DOI
- 10.5555/1760167.1760197 · DOI
Cited by
Sorted by citations
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,080
Multimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279
Early versus late fusion in semantic video analysis
Snoek, C. G. M., Worring, M., & Smeulders, A. W. M. (2005) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1101149.1101236 · Cited by 899
Cross-domain video concept detection using adaptive svms
Yang, J. J., Yan, R., & Hauptmann, A. G. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291276 · Cited by 673
Unified Video Annotation via Multigraph Learning
Wang, M., Hua, X., Hong, R., Tang, J., Qi, G.-J., & Song, Y. (2009) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2009.2017400 · Cited by 443
Metadata
- Authors
- Wu, Y., Chang, E. Y., Chang, K. C.-C., & Smith, J. R.
- Affiliations
- University of California, Santa Barbara (US)
- University of Illinois Urbana-Champaign (US)
- IBM (United States) (US)
- Venue
- Proceedings of the 12th annual ACM international conference on Multimedia
- Publisher
- ACM
- Published
- 2004-10-10
- Pages
- 572-579
- OpenAlex
- W2056952330
- Cited by
- 229
- References
- 35
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 2 · 2023: 2 · 2022: 3 · 2021: 5 · 2020: 5 · 2019: 8 · 2018: 5 · 2017: 6 · 2016: 7 · 2015: 9 · 2014: 12 · 2013: 15 · 2012: 24
- FWCI
- 14.19
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- National Science Foundation
- National Institute of Standards and Technology
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- multimodal fusion, feature fusion, modality selection, curse of dimensionality, video captioning, multimedia data analysis, class prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
