Optimal multimodal fusion for multimedia data analysis
Abstract
Considerable research has been devoted to utilizing multimodal features for better understanding multimedia data. However, two core research issues have not yet been adequately addressed. First, given a set of features extracted from multiple media sources (e.g., extracted from the visual, audio, and caption track of videos), how do we determine the best modalities? Second, once a set of modal-ities has been identified, how do we best fuse them to map to se-mantics? In this paper, we propose a two-step approach. The first step finds statistically independent modalities from raw features. In the second step, we use super-kernel fusion to determine the optimal combination of individual modalities. We carefully ana-lyze the tradeoffs between three design factors that affect fusion performance: modality independence, curse of dimensionality, and fusion-model complexity. Through analytical and empirical studies, we demonstrate that our two-step approach, which achieves a care-ful balance of the three design factors, can improve class-prediction accuracy over traditional techniques.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Attention-based fusion aligns key features and dynamically assigns weights between modalities for deeper semantic capture.
“Attention-based fusion [49,50,51] aligns key features and dynamically assigns weights between modalities for deeper semantic capture.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [49]
Cites
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986 · see citing works
Goodall, C. R., & Jolliffe, I. T. (1988) · Technometrics · doi:10.2307/1270093 · Cited by 8,677 · see citing works
Jolliffe, I. T. (1986) · Springer series in statistics · doi:10.1007/978-1-4757-1904-8 · Cited by 8,367 · see citing works
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 4,834 · see citing works
A Fast Fixed-Point Algorithm for Independent Component Analysis
Hyvärinen, A., & Erkki Oja, A. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1483 · Cited by 3,373 · see citing works
Query by image and video content: the QBIC system
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petković, D., et al. (1995) · Computer · doi:10.1109/2.410146 · Cited by 3,097 · see citing works
Adaptive Control Processes: A Guided Tour
Bellman, R. E. (1961) · doi:10.1515/9781400874668 · Cited by 2,802 · see citing works
Solving Multiclass Learning Problems via Error-Correcting Output Codes
Dietterich, T. G., & Bakiri, G. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.105 · Cited by 2,698 · see citing works
When Is ''Nearest Neighbor'' Meaningful?
Beyer, K., Goldstein, J., Ramakrishnan, R., & Shaft, U. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-49257-7_15 · Cited by 2,403 · see citing works
A New Learning Algorithm for Blind Signal Separation
Амари, Ш.-И., Cichocki, A. S., & Yang, H. H. (1995) · Cited by 1,904 · see citing works
Query by Image and Video Content: The QBIC System
Flickner, M., Sawhney, H., Niblack, W., Ashley, J. J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petković, D., et al. (2002) · Elsevier eBooks · doi:10.1016/b978-155860651-7/50108-x · Cited by 1,559 · see citing works
Optimal aggregation algorithms for middleware
Fagin, R., Lotem, A., & Naor, M. (2001) · doi:10.1145/375551.375567 · Cited by 1,090 · see citing works
Bellman, R. (1964) · IEEE Transactions on Circuit Theory · doi:10.1109/tct.1964.1082221 · Cited by 1,003 · see citing works
A min-max cut algorithm for graph partitioning and data clustering
Ding, C. H. Q., He, X., Hongyuan Zha, B., Gu, M., & Simon, H. D. (2002) · Proceedings · doi:10.1109/icdm.2001.989507 · Cited by 850 · see citing works
Content-based image retrieval with relevance feedback in MARS
Rui, Y., Huang, T. S., & Mehrotra, S. (2002) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.1997.638621 · Cited by 729 · see citing works
(2000) · Applied Physics Letters · doi:10.1162/153244303768966085 · Cited by 583 · see citing works
Issues in Stacked Generalization
Ting, K. M., & Witten, I. H. (1999) · Cited by 521 · see citing works
Combining multiple classifiers by averaging or by multiplying?
Tax, D. M. J., van Breukelen, M., Duin, R. P. W., & Kittler, J. (2000) · Pattern Recognition · doi:10.1016/s0031-3203(99)00138-7 · Cited by 377 · see citing works
Independent component representations for face recognition
Bartlett, M. S., Lades, M. H., & B, T. J. S. A. (1998) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.320144 · Cited by 316 · see citing works
Combining textual and visual cues for content-based image retrieval on the World Wide Web
Cascia, M. L., Sethi, S., & Sclaroff, S. (2002) · doi:10.1109/ivl.1998.694480 · Cited by 249 · see citing works
Audio Vision: Using Audio-Visual Synchrony to Locate Sounds
Hershey, J. R., & Movellan, J. R. (1999) · Cited by 227 · see citing works
Inferring a Semantic Representation of Text via Cross-Language Correlation Analysis
Vinokourov, A., Cristianini, N., & Shawe-Taylor, J. S. (2002) · Cited by 224 · see citing works
Kittler, J., Hater, M., & Duin, R. P. W. (1996) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1996.547205 · Cited by 200 · see citing works
Learning Joint Statistical Models for Audio-Visual Fusion and Segregation
Fisher, J. S., Darrell, T., Freeman, W. T., & Viola, P. A. (2000) · Cited by 176 · see citing works
Image retrieval: content versus context
Westerveld, T. (2000) · University of Twente Research Information · Cited by 93 · see citing works
SVM binary classifier ensembles for image classification
Goh, K.-S., Chang, E. Y., & Cheng, K.-T. T. (2001) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/502585.502652 · Cited by 86 · see citing works
Self-citationDiscovery of a perceptual distance function for measuring image similarity
Li, B., Chang, E. Y., & Wu, Y. (2003) · Multimedia Systems · doi:10.1007/s00530-002-0069-9 · Cited by 77 · see citing works
Self-citationAUDIO/VISUAL INDEPENDENT COMPONENTS
Smaragdis, P., Casey, M. A., & Square, N. (2003) · Cited by 66 · see citing works
Independent component analysis for understanding multimedia content
Kolenda, T., Hansen, L. K., Larsen, J. O., & Winther, O. (2003) · Machine learning for signal processing ... · doi:10.1109/nnsp.2002.1030096 · Cited by 57 · see citing works
The combination limit in multimedia retrieval
Yan, R., & Hauptmann, A. G. (2003) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/957013.957086 · Cited by 56 · see citing works
On Independent Component Analysis for Multimedia Signals
Kolenda, T., Larsen, J. O., & Hansen, L. K. (2000) · Image processing series · doi:10.1201/9781420037562.ch7 · Cited by 34 · see citing works
Vinokourov, A., Hardoon, D. R., & Shawe-Taylor, J. S. (2003) · ePrints Soton (University of Southampton) · Cited by 32 · see citing works
Detection of Documentary Scene Changes by Audio-Visual Fusion
Velivelli, A., Ngo, C., & Huang, T. S. (2003) · Lecture notes in computer science · doi:10.1007/3-540-45113-7_23 · Cited by 15 · see citing works
An ICA algorithm for analyzing multiple data sets
Lukic, A. S., Wernick, M. N., Hansen, L. K., & Strother, S. C. (2003) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2002.1040077 · Cited by 13 · see citing works
Flagged first, then by citations
1 of the 35 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,043 · see citing works
Multimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279 · see citing works
Early versus late fusion in semantic video analysis
Snoek, C. G. M., Worring, M., & Smeulders, A. W. M. (2005) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1101149.1101236 · Cited by 899 · see citing works
Cross-domain video concept detection using adaptive svms
Yang, J. J., Yan, R., & Hauptmann, A. G. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291276 · Cited by 673 · see citing works
Unified Video Annotation via Multigraph Learning
Wang, M., Hua, X., Hong, R., Tang, J., Qi, G.-J., & Song, Y. (2009) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2009.2017400 · Cited by 443 · see citing works
Multimodal Graph-Based Reranking for Web Image Search
Wang, M., Li, H., Tao, D., Lu, K., & Wu, X. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2207397 · Cited by 344 · see citing works
Snoek, C. G. M., & Worring, M. (2007) · now publishers, Inc. eBooks · doi:10.1561/9781601982353 · Cited by 317 · see citing works
Learning from facial aging patterns for automatic age estimation
Geng, X., Zhou, Z., Zhang, Y., Li, G., & Dai, H. (2006) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1180639.1180711 · Cited by 281 · see citing works
High-Order Distance-Based Multiview Stochastic Learning in Image Classification
Yu, J., Rui, Y., Tang, Y. Y., & Tao, D. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2307862 · Cited by 260 · see citing works
G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition
Tang, P., Wang, H., & Kwong, S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.11.023 · Cited by 261 · see citing works
MFAS: Multimodal Fusion Architecture Search
Perez-Rua, J.-M., Vielzeuf, V., Pateux, S., Baccouche, M., & Jurie, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00713 · Cited by 193 · see citing works
Efficient Large-Scale Multi-Modal Classification
Kiela, D., Grave, E., Joulin, A., & Mikolov, T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11945 · Cited by 177 · see citing works
Multimodal information fusion application to human emotion recognition from face and speech
Mansoorizadeh, M., & Charkari, N. M. (2009) · Multimedia Tools and Applications · doi:10.1007/s11042-009-0344-2 · Cited by 164 · see citing works
Audiovisual Fusion: Challenges and New Approaches
Katsaggelos, A. K., Bahaadini, S., & Molina, R. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2459017 · Cited by 140 · see citing works
Feature Correlation Hypergraph: Exploiting High-order Potentials for Multimodal Recognition
Zhang, L., Gao, Y., Hong, C., Feng, Y., Zhu, J., & Cai, D. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2285219 · Cited by 139 · see citing works
Majority Vote of Diverse Classifiers for Late Fusion
Morvant, E., Habrard, A., & Ayache, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44415-3_16 · Cited by 138 · see citing works
Depeursinge, A., Racoceanu, D., Iavindrasana, J., Cohen, G., Platon, A., Poletti, P., & Müller, H. (2010) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2010.04.006 · Cited by 124 · see citing works
The case for multi--tier camera sensor networks
Kulkarni, P., Ganesan, D., & Shenoy, P. J. (2005) · International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV) · doi:10.1145/1065983.1066016 · Cited by 111 · see citing works
Graph based multi-modality learning
Tong, H., He, J., Li, M., Zhang, C., & Ma, W. (2005) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1101149.1101337 · Cited by 108 · see citing works
How many high-level concepts will fill the semantic gap in news video retrieval?
Hauptmann, A. G., Yan, R., & Lin, W. (2007) · ACM International Conference on Image and Video Retrieval (CIVR) · doi:10.1145/1282280.1282369 · Cited by 103 · see citing works
Optimizing multi-graph learning
Wang, M., Hua, X., Yuan, X., Song, Y., & Dai, L.-R. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291431 · Cited by 99 · see citing works
Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
Zhu, L., Shen, J., Jin, H., Zheng, R., & Xie, L. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2383389 · Cited by 96 · see citing works
Local Ensemble Kernel Learning for Object Category Recognition
Lin, Y., Liu, T.-L., & Fuh, C. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383084 · Cited by 89 · see citing works
Video Retrieval Based on Semantic Concepts
Hauptmann, A. G., Christel, M. G., & Yan, R. (2008) · Proceedings of the IEEE · doi:10.1109/jproc.2008.916355 · Cited by 88 · see citing works
A review of text and image retrieval approaches for broadcast news video
Yan, R., & Hauptmann, A. G. (2007) · Information Retrieval · doi:10.1007/s10791-007-9031-y · Cited by 80 · see citing works
Li, S., Kim, J. G., Han, D. H., & Lee, K. S. (2019) · Sensors · doi:10.3390/s19010199 · Cited by 74 · see citing works
Leveraging high-level and low-level features for multimedia event detection
Jiang, L., Hauptmann, A. G., & Xiang, G. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2393412 · Cited by 73 · see citing works
Information assimilation framework for event detection in multimedia surveillance systems
Atrey, P. K., Kankanhalli, M., & Jain, R. (2006) · Multimedia Systems · doi:10.1007/s00530-006-0063-8 · Cited by 71 · see citing works
A unified dimensionality reduction framework for semi-paired and semi-supervised multi-view data
Chen, X., Chen, S., Xue, H., & Zhou, X. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2011.11.008 · Cited by 69 · see citing works
Multiple instance learning for labeling faces in broadcasting news video
Yang, J. J., Yan, R., & Hauptmann, A. G. (2005) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1101149.1101155 · Cited by 68 · see citing works
Acoustic Event Detection and Classification
Temko, A., Nadeu, C., Macho, D., Malkin, R. E., Zieger, C., & Omologo, M. (2009) · Springer eBooks · doi:10.1007/978-1-84882-054-8_7 · Cited by 66 · see citing works
Aslam, A., & Curry, E. (2020) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104095 · Cited by 66 · see citing works
Deep Multimodal Learning: Merging Sensory Data for Massive MIMO Channel Prediction
Yang, Y., Gao, F., Xing, C., An, J., & Alkhateeb, A. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041383 · Cited by 64 · see citing works
Information Fusion in Multimedia Information Retrieval
Kludas, J., Bruno, É., & Marchand‐Maillet, S. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-79860-6_12 · Cited by 61 · see citing works
Multimodal deep learning based on multiple correspondence analysis for disaster management
Pouyanfar, S., Tao, Y., Tian, H., Chen, S., & Shyu, M. (2018) · World Wide Web · doi:10.1007/s11280-018-0636-4 · Cited by 62 · see citing works
Fusing Visual and Clinical Information for Lung Tissue Classification in HRCT Data
Depeursinge, A., Racoceanu, D., Iavindrasana, J., Cohen, G., Platon, A., Poletti, P., & Müller, H. (2009) · Cited by 59 · see citing works
Bokade, R., Navato, A. P., Ouyang, R., Jin, X., Chou, C.-A., Ostadabbas, S., & Mueller, A. V. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113885 · Cited by 59 · see citing works
Fusion of AV features and external information sources for event detection in team sports video
Xu, H., & Chua, T. (2006) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/1126004.1126007 · Cited by 54 · see citing works
Fuzzy integral based information fusion for classification of highly confusable non-speech sounds
Temko, A., Macho, D., & Nadeu, C. (2007) · Pattern Recognition · doi:10.1016/j.patcog.2007.10.026 · Cited by 54 · see citing works
A Multimodal and Multilevel Ranking Scheme for Large-Scale Video Retrieval
Hoi, S. C. H., & Lyu, M. R.-T. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.921735 · Cited by 53 · see citing works
Bioinspired Scene Classification by Deep Active Learning With Remote Sensing Applications
Zhang, L., Su, G., Yin, J., Li, Y., Lin, Q., Zhang, X., & Shao, L. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2981480 · Cited by 52 · see citing works
Interactive patent classification based on multi-classifier fusion and active learning
Zhang, X. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.08.013 · Cited by 48 · see citing works
Multimodal fusion using learned text concepts for image categorization
Zhu, Q., Yeh, M.-C., & Cheng, K.-T. T. (2006) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1180639.1180698 · Cited by 48 · see citing works
Multi-Group Encoder-Decoder Networks to Fuse Heterogeneous Data for Next-Day Air Quality Prediction
Zhang, Y., Lv, Q., Gao, D., Shen, S., Dick, R. P., Hannigan, M., & Liu, Q. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/603 · Cited by 47 · see citing works
Golugula, A., Lee, G., Master, S. R., Feldman, M. D., Tomaszewski, J. E., Speicher, D. W., & Madabhushi, A. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-483 · Cited by 46 · see citing works
Probabilistic models for combining diverse knowledge sources in multimedia retrieval
Hauptmann, A. G., & Yan, R. (2006) · Cited by 46 · see citing works
Web media semantic concept retrieval via tag removal and model fusion
Chen, C., Zhu, Q., Lin, L., & Shyu, M. (2013) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2508037.2508042 · Cited by 45 · see citing works
Liu, N., Dellandréa, E., Chen, L., Zhu, C., Zhang, Y., Bichot, C., Bres, S., & Tellez, B. (2012) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2012.10.009 · Cited by 45 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“This method has multiple advantages, such as strong modal independence, ease of individual optimization, and scalability of multimodal systems [50] .”
Design of Multimodal Dissimilarity Spaces for Retrieval of Video Documents
Bruno, E., Moënne-Loccoz, N., & Marchand‐Maillet, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70801 · Cited by 41 · see citing works
Multimodal information fusion for selected multimedia applications
Guan, L., Wang, Y., Zhang, R., Tie, Y., Bulzacki, A., & Ibrahim, M. (2010) · International Journal of Multimedia Intelligence and Security · doi:10.1504/ijmis.2010.035969 · Cited by 34 · see citing works
A Multimodal Scheme for Program Segmentation and Representation in Broadcast Video Streams
Wang, J., Duan, L., Liu, Q., Lu, H., & Jin, J. S. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.917362 · Cited by 39 · see citing works
Multi-view multi-label active learning for image classification
Zhang, X., Cheng, J., Xu, C., Lu, H., & Ma, S. (2009) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2009.5202484 · Cited by 38 · see citing works
Applying semantic web technologies to knowledge sharing in aerospace engineering
Dadzie, A., Bhagdev, R., Chakravarthy, A., Chapman, S., Iria, J., Lanfranchi, V., Magalhães, J., Petrelli, D., & Ciravegna, F. (2008) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-008-0141-1 · Cited by 37 · see citing works
Image Search Reranking With Query-Dependent Click-Based Relevance Feedback
Zhang, Y., Yang, X., & Mei, T. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2346991 · Cited by 36 · see citing works
A Unified Relevance Feedback Framework for Web Image Retrieval
Cheng, E., Jing, F., & Zhang, L. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2017128 · Cited by 35 · see citing works
Harmonium Models for Semantic Video Representation and Classification
Yang, J. J., Liu, Y., Xing, E. P., & Hauptmann, A. G. (2007) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972771.34 · Cited by 32 · see citing works
Fusing inherent and external knowledge with nonlinear learning for cross-media retrieval
Zhang, H., Liu, Y., & Ma, Z. (2013) · Neurocomputing · doi:10.1016/j.neucom.2012.03.033 · Cited by 32 · see citing works
Multi-modal sensor registration for vehicle perception via deep neural networks
Giering, M. J., Venugopalan, V., & Reddy, K. (2015) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2015.7322485 · Cited by 31 · see citing works
Multi-sensor Self-Quantification of Presentations
Gan, T., Wong, Y., Mandal, B., Chandrasekhar, V., & Kankanhalli, M. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806252 · Cited by 30 · see citing works
Multimodal photo annotation and retrieval on a mobile phone
Anguera, X., Xu, J., & Oliver, N. (2008) · Multimedia Information Retrieval · doi:10.1145/1460096.1460127 · Cited by 30 · see citing works
Non-Intrusive Multi-Modal Estimation of Building Occupancy
Das, A., Pathak, P. H., Jee, J., Chuah, C., & Mohapatra, P. (2017) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3131672.3131680 · Cited by 30 · see citing works
Evaluating feature selection strategies for high dimensional, small sample size datasets
Golugula, A., Lee, G., & Madabhushi, A. (2011) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2011.6090214 · Cited by 29 · see citing works
Multimodal Event Detection in Twitter Hashtag Networks
Yılmaz, Y., & Hero, A. O. (2016) · Journal of Signal Processing Systems · doi:10.1007/s11265-016-1151-4 · Cited by 28 · see citing works
Arakeri, M. P., & Reddy, G. R. M. (2013) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-013-0037-5 · Cited by 28 · see citing works
Multimodal Biometric System and Information Fusion
(2013) · IGI Global eBooks · doi:10.4018/978-1-4666-3646-0.ch004 · Cited by 25 · see citing works
Matheen, M. A., & Sundar, S. (2022) · International Journal of Wireless Information Networks · doi:10.1007/s10776-022-00567-6 · Cited by 25 · see citing works
Tag relevance fusion for social image retrieval
Li, X. (2014) · Multimedia Systems · doi:10.1007/s00530-014-0430-9 · Cited by 24 · see citing works
A cross-media distance metric learning framework based on multi-view correlation mining and matching
Zhang, H., Gao, X., Wu, P., & Xu, X. (2015) · World Wide Web · doi:10.1007/s11280-015-0342-4 · Cited by 24 · see citing works
Grassmann multimodal implicit feature selection
Zhang, L., Tao, D., Liu, X., Sun, L., Song, M., & Chen, C. (2013) · Multimedia Systems · doi:10.1007/s00530-013-0317-1 · Cited by 24 · see citing works
Gandhi, A., Sharma, A., Biswas, A., & Deshmukh, O. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48881-3_58 · Cited by 24 · see citing works
Sparse Linear Integration of Content and Context Modalities for Semantic Concept Retrieval
Zhu, Q., & Shyu, M. (2014) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2014.2384992 · Cited by 23 · see citing works
Concept-based evidential reasoning for multimodal fusion in human–computer interaction
Reddy, B. S., & Basir, O. (2009) · Applied Soft Computing · doi:10.1016/j.asoc.2009.08.026 · Cited by 22 · see citing works
High order pLSA for indexing tagged images
Nikolopoulos, S., Zafeiriou, S. P., Patras, I., & Kompatsiaris, I. Y. (2012) · Signal Processing · doi:10.1016/j.sigpro.2012.08.004 · Cited by 19 · see citing works
Timeline-based information assimilation in multimedia surveillance and monitoring systems
Atrey, P. K., Kankanhalli, M., & Jain, R. (2005) · ACM International Workshop on Video Surveillance & Sensor Networks (VSSN) · doi:10.1145/1099396.1099416 · Cited by 19 · see citing works
Web Multimedia Object Classification Using Cross-Domain Correlation Knowledge
Lu, W., Li, J., Li, T., Guo, W., Zhang, H., & Guo, J. H. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2280895 · Cited by 17 · see citing works
Social Media Modeling and Computing
Hoi, S. C. H., King, I., Jin, R., Xu, D., Boll, S., & Luo, J. (2011) · Springer eBooks · doi:10.1007/978-0-85729-436-4 · Cited by 16 · see citing works
Video semantic analysis based on structure-sensitive anisotropic manifold ranking
Tang, J., Qi, G.-J., Wang, M., & Hua, X. (2009) · Signal Processing · doi:10.1016/j.sigpro.2009.01.020 · Cited by 16 · see citing works
Combining multimodal preferences for multimedia information retrieval
Bruno, É., Kludas, J., & Marchand‐Maillet, S. (2007) · Multimedia Information Retrieval · doi:10.1145/1290082.1290095 · Cited by 16 · see citing works
Series feature aggregation for content-based image retrieval
Zhang, J., & Ye, L. (2008) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2008.11.001 · Cited by 15 · see citing works
Fusing heterogeneous modalities for video and image re-ranking
Tan, H., & Ngo, C. (2011) · International Conference on Multimedia Retrieval · doi:10.1145/1991996.1992011 · Cited by 15 · see citing works
Answering English Questions using Foreign-Language, Semi-Structured Sources
Katz, B., Borchardt, G. C., Felshin, S., Shen, Y. K., & Zaccak, G. (2007) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2007.59 · Cited by 15 · see citing works
Fusion of Multiple Asynchronous Information Sources for Event Detection in Soccer Video
Xu, H., Fong, T.-H., & Chua, T. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521653 · Cited by 15 · see citing works
Encoder-Decoder Based Route Generation Model for Flexible Travel Recommendation
Zhang, J., Ma, M., Gao, X., & Chen, G. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3376231 · Cited by 13 · see citing works
Metric learning with feature decomposition for image categorization
Wang, M., Liu, B., Tang, J., & Hua, X. (2010) · Neurocomputing · doi:10.1016/j.neucom.2009.08.023 · Cited by 13 · see citing works
Summarizing Complex Events: a Cross-Modal Solution of Storylines Extraction and Reconstruction
Xu, S., Wang, S., & Zhang, Y. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1127 · Cited by 13 · see citing works
Multimodal Ensemble Fusion for Disambiguation and Retrieval
Peng, Y., Zhou, X., Wang, D. Z., Patwa, I., Gong, D., & Fang, C. V. (2016) · IEEE Multimedia · doi:10.1109/mmul.2016.26 · Cited by 13 · see citing works
Joint categorization of queries and clips for web-based video search
Zhang, R., Sarukkai, R. R., Chow, J.-H., Dai, W., & Zhang, Z. M. (2006) · Multimedia Information Retrieval · doi:10.1145/1178677.1178705 · Cited by 13 · see citing works
Can feature information interaction help for information fusion in multimedia problems?
Kludas, J., Bruno, É., & Marchand‐Maillet, S. (2008) · Multimedia Tools and Applications · doi:10.1007/s11042-008-0251-y · Cited by 13 · see citing works
Acoustic event detection and classification
Temko, A. (2007) · doi:10.5821/dissertation-2117-94191 · Cited by 13 · see citing works
A Selective Weighted Late Fusion for Visual Concept Recognition
Liu, N., Dellandréa, E., Zhu, C., Bichot, C., & Chen, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33885-4_43 · Cited by 12 · see citing works
Multimodal Preference Aggregation for Multimedia Information Retrieval
Bruno, É., & Marchand‐Maillet, S. (2009) · Journal of Multimedia · doi:10.4304/jmm.4.5.321-329 · Cited by 12 · see citing works
RELIEF-MM: effective modality weighting for multimedia information retrieval
Yilmaz, T., Yazıcı, A., & Kitsuregawa, M. (2014) · Multimedia Systems · doi:10.1007/s00530-014-0360-6 · Cited by 12 · see citing works
Audio-visual speaker identification with multi-view distance metric learning
Zheng, H., Wang, M., & Zhu Li, B. (2010) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2010.5653016 · Cited by 12 · see citing works
Event-based multimedia chronicling systems
Kim, P., Gargi, U., & Jain, R. (2005) · doi:10.1145/1099083.1099085 · Cited by 12 · see citing works
Learning User Queries in Multimodal Dissimilarity Spaces
Bruno, É., Moënne-Loccoz, N., & Marchand‐Maillet, S. (2006) · Lecture notes in computer science · doi:10.1007/11670834_14 · Cited by 12 · see citing works
Multimodal Integration (Image and Text) Using Ontology Alignment
Abu-Shareha, A. A., Rajeswari, M., & Ramachandram, D. (2009) · American Journal of Applied Sciences · doi:10.3844/ajassp.2009.1217.1224 · Cited by 12 · see citing works
Image Retrieval Using Multimodal Keywords
Agrawal, R. K., Grosky, W. I., & Fotouhi, F. A. (2006) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism.2006.91 · Cited by 12 · see citing works
Scalable relevance feedback using click-through data for web image retrieval
Cheng, E., Jing, F., Zhang, L., & Jin, H. (2006) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1180639.1180688 · Cited by 11 · see citing works
A Multi-sensor Framework for Personal Presentation Analytics
Gan, T., Li, J., Wong, Y., & Kankanhalli, M. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3300941 · Cited by 11 · see citing works
- Authors
- Wu, Y., Chang, E. Y., Chang, K. C.-C., & Smith, J. R.
- Affiliations
- University of California, Santa Barbara (US)
- University of Illinois Urbana-Champaign (US)
- IBM (United States) (US)
- Venue
- Proceedings of the 12th annual ACM international conference on Multimedia
- Publisher
- ACM
- Published
- 2004-10-10
- Pages
- 572-579
- OpenAlex
- W2056952330
- Cited by
- 229
- References
- 35
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 2 · 2023: 2 · 2022: 3 · 2021: 5 · 2020: 5 · 2019: 8 · 2018: 5 · 2017: 6 · 2016: 7 · 2015: 9 · 2014: 12 · 2013: 15 · 2012: 24
- FWCI
- 14.19
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- National Science Foundation
- National Institute of Standards and Technology
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- multimodal fusion, feature fusion, modality selection, curse of dimensionality, video captioning, multimedia data analysis, class prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
