CentralNet: A Multilayer Approach for Multimodal Fusion

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works

Gradient-based learning applied to document recognition

Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works

Distributed Representations of Words and Phrases and their Compositionality

Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works

Two-Stream Convolutional Networks for Action Recognition in Videos

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works

Multimodal Machine Learning: A Survey and Taxonomy

Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018 · see citing works

ESC: Dataset for Environmental Sound Classification

Piczak, K. J. (2015) · ACM Multimedia · doi:10.1145/2733373.2806390 · Cited by 2,195 · see citing works

Deep Canonical Correlation Analysis

Andrew, G., Arora, R., Bilmes, J., & Livescu, K. (2013) · Cited by 1,603 · see citing works

Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects

Lahat, D., Adali, T., & Jutten, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2460697 · Cited by 1,296 · see citing works

Multimodal fusion for multimedia analysis: a survey

Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279 · see citing works

YouTube-8M: A Large-Scale Video Classification BenchmarkPreprint

Abu-El-Haija, S., Kothari, N., Lee, J., Natsev, P., Toderici, G. D., Varadarajan, B., & Vijayanarasimhan, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08675 · Cited by 1,135 · see citing works

Collecting Large, Richly Annotated Facial-Expression Databases from Movies

Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2012) · IEEE Multimedia · doi:10.1109/mmul.2012.26 · Cited by 664 · see citing works

ModDrop: Adaptive Multi-Modal Gesture Recognition

Neverova, N., Wolf, C., Taylor, G., & Nebout, F. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2461544 · Cited by 420 · see citing works

Contextual Region-Based Convolutional Neural Network with Multilayer Fusion for SAR Ship Detection

Kang, M., Ji, K., Leng, X., & Lin, Z. (2017) · Remote Sensing · doi:10.3390/rs9080860 · Cited by 338 · see citing works

Multimodal sentiment analysis with word-level fusion and reinforcement learning

Chen, M., Wang, S., Liang, P. P., Baltrušaitis, T., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136801 · Cited by 334 · see citing works

Gated Multimodal Units for Information FusionPreprint

Arévalo, J., Solorio, T., Montes-y-Gómez, M., & Gonzalez, F. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01992 · Cited by 302 · see citing works

Multi-scale Deep Learning for Gesture Detection and Localization

Neverova, N., Wolf, C., Taylor, G. W., & Nebout, F. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16178-5_33 · Cited by 239 · see citing works

ChaLearn Looking at People Challenge 2014: Dataset and Results

Escalera, S., Baró, X., Gonzàlez, J., Bautista, M. A., Madadi, M., Reyes, M., Ponce-López, V., Escalante, H. J., Shotton, J., & Guyon, I. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16178-5_32 · Cited by 211 · see citing works

Temporal multimodal fusion for video emotion classification in the wild

Vielzeuf, V., Pateux, S., & Jurie, F. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3143011 · Cited by 179 · see citing works

Efficient Large-Scale Multi-Modal Classification

Kiela, D., Grave, E., Joulin, A., & Mikolov, T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11945 · Cited by 177 · see citing works

Learning supervised scoring ensemble for emotion recognition in the wild

Hu, P., Cai, D., Wang, S., Yao, A., & Chen, Y. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143009 · Cited by 162 · see citing works

ESC: Dataset for Environmental Sound Classification

Piczak, K. J. (2015) · Harvard Dataverse · doi:10.7910/dvn/ydeput · Cited by 158 · see citing works

Correlational Neural Networks

Chandar, S., Khapra, M. M., Larochelle, H., & Ravindran, B. (2015) · Neural Computation · doi:10.1162/neco_a_00801 · Cited by 133 · see citing works

Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification

Yang, X., Molchanov, P. A., & Kautz, J. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964297 · Cited by 115 · see citing works

Multi-modal emotion recognition using semi-supervised learning and multiple neural networks in the wild

Kim, D. H., Lee, M. K., Choi, D. Y., & Song, B. C. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3143005 · Cited by 45 · see citing works

Learning Joint Multimodal Representation Based on Multi-fusion Deep Neural Networks

Gu, Z., Lang, B., Yue, T., & Huang, L. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70096-0_29 · Cited by 19 · see citing works

Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and TextPreprint

Wang, Z., Kuan, K., Ravaut, M., Manek, G., Song, S., Fang, Y., Kim, S., Chen, N. F., D’Haro, L. F., Tuan, L. A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05461 · Cited by 17 · see citing works

Summary for AVEC 2017

Ringeval, F., Schuller, B. W., Valstar, M., Gratch, J. S., Cowie, R., & Pantić, M. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3132049 · Cited by 14 · see citing works

Modout: Learning to Fuse Modalities via Stochastic Regularization

Li, F., Neverova, N., Wolf, C., & Taylor, G. W. (2016) · Journal of Computational Vision and Imaging Systems · doi:10.15353/vsnl.v2i1.103 · Cited by 11 · see citing works

XFlow: 1D-2D Cross-modal Deep Neural Networks for Audiovisual Classification.Preprint

Cangea, C., Veličković, P., & Lió, P. (2017) · arXiv (Cornell University) · Cited by 3 · see citing works

XFlow: Cross-Modal Deep Neural Networks for Audiovisual Classification

Cangea, C., Veličković, P., & Lió, P. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2945992 · Cited by 2 · see citing works

Multimodal intelligence: Representation learning, information fusion, and applications

Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511 · see citing works

MMTM: Multimodal Transfer Module for CNN Fusion

Joze, H. R. V., Shaban, A., Iuzzolino, M. L., & Koishida, K. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01330 · Cited by 323 · see citing works

SMIL: Multimodal Learning with Severely Missing Modality

Ma, M., Ren, J., Zhao, L., Tulyakov, S., Wu, C., & Peng, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16330 · Cited by 332 · see citing works

Multi-Modal Learning with Missing Modality via Shared-Specific Feature Modelling

Wang, H., Chen, Y., Ma, C., Avery, J., Hull, M. L., & Carneiro, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01524 · Cited by 229 · see citing works

Multimodal Co-learning: Challenges, applications with datasets, recent advances and future directions

Rahate, A., Walambe, R., Ramanna, S., & Kotecha, K. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.003 · Cited by 214 · see citing works

MFAS: Multimodal Fusion Architecture Search

Perez-Rua, J.-M., Vielzeuf, V., Pateux, S., Baccouche, M., & Jurie, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00713 · Cited by 193 · see citing works

Multimodal Deep Learning

Shaban, A., & Yousefi, S. (2024) · Springer optimization and its applications · doi:10.1007/978-3-031-53092-0_10 · Cited by 206 · see citing works

Are Multimodal Transformers Robust to Missing Modality?

Ma, M., Ren, J., Zhao, L., Testuggine, D., & Peng, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01764 · Cited by 198 · see citing works

Supervised Multimodal Bitransformers for Classifying Images and TextPreprint

Kiela, D., Bhooshan, S., Firooz, H., Perez, E., & Testuggine, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.02950 · Cited by 271 · see citing works

PMR: Prototypical Modal Rebalance for Multimodal Learning

Fan, Y., Xu, W., Wang, H., Wang, J., & Guo, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01918 · Cited by 166 · see citing works

Dynamic Multimodal Fusion

Xue, Z., & Marculescu, R. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00256 · Cited by 134 · see citing works

Co-Attention Fusion Network for Multimodal Skin Cancer Diagnosis

He, X., Wang, Y., Zhao, S., & Chen, X. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108990 · Cited by 97 · see citing works

Adaptive Multimodal Emotion Detection Architecture for Social Robots

Heredia, J., Lopes-Silva, E., Cardinale, Y., Díaz-Amado, J., Dongo, I., Graterol, W., & Aguilera, A. (2022) · IEEE Access · doi:10.1109/access.2022.3149214 · Cited by 95 · see citing works

A Multimodality Fusion Deep Neural Network and Safety Test Strategy for Intelligent Vehicles

Nie, J., Yan, J., Yin, H., Ren, L., & Meng, Q. (2020) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2020.3027319 · Cited by 92 · see citing works

Multi-Modal Machine Learning in Engineering Design: A Review and Future Directions

Song, B., Zhou, R., & Ahmed, F. (2023) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4063954 · Cited by 83 · see citing works

Optical and Synthetic Aperture Radar Image Fusion for Ship Detection and Recognition: Current state, challenges, and future prospects

Zhang, Z., Zhang, L., Wu, J., & Guo, W. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3404506 · Cited by 77 · see citing works

AWSnet: An auto-weighted supervision attention network for myocardial scar and edema segmentation in multi-sequence cardiac magnetic resonance images

Wang, K., Yang, X., Miao, J., Li, L., Yao, J., Zhou, P., Xue, W., Zhou, G., Zhuang, X., & Ni, D. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102362 · Cited by 68 · see citing works

Boosting Multi-modal Model Performance with Adaptive Gradient Modulation

Li, H., Li, X., Hu, P., Lei, Y., Li, C., & Zhou, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02030 · Cited by 64 · see citing works

A Simple Visual-Textual Baseline for Pedestrian Attribute Recognition

Cheng, X., Jia, M., Wang, Q., & Zhang, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3178144 · Cited by 54 · see citing works

TMMF: Temporal Multi-Modal Fusion for Single-Stage Continuous Gesture Recognition

Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3108349 · Cited by 50 · see citing works

Expanding Large Pre-trained Unimodal Models with Multimodal Information Injection for Image-Text Multimodal Classification

Liang, T., Lin, G., Wan, M., Li, T., Ma, G., & Lv, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01505 · Cited by 42 · see citing works

Enhancing Multimodal Cooperation via Sample-Level Modality Valuation

Wei, Y., Feng, R., Wang, Z., & Hu, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02581 · Cited by 45 · see citing works

MSAF: Multimodal Split Attention FusionPreprint

Su, L., Hu, C., Li, G., & Cao, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.07175 · Cited by 65 · see citing works

Multimodal Attention Network for Continuous-Time Emotion Recognition Using Video and EEG Signals

Choi, D. Y., Kim, D.-H., & Song, B. C. (2020) · IEEE Access · doi:10.1109/access.2020.3036877 · Cited by 39 · see citing works

Dense Modality Interaction Network for Audio-Visual Event Localization

Liu, S., Quan, W., Wang, C., Liu, Y., Liu, B., & Dong-ming Yan, A. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3150469 · Cited by 37 · see citing works

Multimodal analysis of personality traits on videos of self-presentation and induced behavior

Giritlioğlu, D., Mandira, B., Yilmaz, S. F., Ertenli, C. U., Akgür, B. F., Kınıklıoğlu, M., Kurt, A. G., Mutlu, E., Gürel, Ş. C., & Dibeklioğlu, H. (2020) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-020-00347-7 · Cited by 40 · see citing works

Attention-Enhanced Multimodal Learning for Conceptual Design Evaluations

Song, B., Miller, S. R., & Ahmed, F. (2023) · Journal of Mechanical Design · doi:10.1115/1.4056669 · Cited by 37 · see citing works

Efficient Driver Anomaly Detection via Conditional Temporal Proposal and Classification Network

Su, L., Sun, C., Cao, D., & Khajepour, A. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3158480 · Cited by 35 · see citing works

Common Practices and Taxonomy in Deep Multiview Fusion for Remote Sensing Applications

Mena, F., Arenas, D., Nuske, M., & Dengel, A. R. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3361556 · Cited by 35 · see citing works

Multilevel Sensor Fusion With Deep Learning

Vielzeuf, V., Lechervy, A., Pateux, S., & Jurie, F. (2018) · IEEE Sensors Letters · doi:10.1109/lsens.2018.2878908 · Cited by 35 · see citing works

Auto-weighting for Breast Cancer Classification in Multimodal Ultrasound

Wang, J., Miao, J., Yang, X., Li, R., Zhou, G., Huang, Y., Lin, Z., Xue, W., Jia, X., Zhou, J., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59725-2_19 · Cited by 33 · see citing works

BM-NAS: Bilevel Multimodal Neural Architecture Search

Yin, Y., Huang, S., & Zhang, X. D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20872 · Cited by 33 · see citing works

On-the-Fly Modulation for Balanced Multimodal Learning

Wei, Y., Hu, D., Du, H., & Wen, J.-R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3468315 · Cited by 30 · see citing works

Deep learning with convex probe endobronchial ultrasound multimodal imaging: A validated tool for automated intrathoracic lymph nodes diagnosis

Dai, W., Sun, J., Li, J., Zhi, X., Chen, J., Wang, L., Xu, M., & Xiong, H. (2021) · Endoscopic Ultrasound · doi:10.4103/eus-d-20-00207 · Cited by 30 · see citing works

Multi-View And Multi-Modal Event Detection Utilizing Transformer-Based Multi-Sensor Fusion

Yasuda, M., Ohishi, Y., Saito, S., & Harado, N. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746006 · Cited by 25 · see citing works

A Framework to Evaluate Fusion Methods for Multimodal Emotion Recognition

Peña, D., Aguilera, A., Dongo, I., Heredia, J., & Cardinale, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3240420 · Cited by 26 · see citing works

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85 · see citing works

Deep multimodal learning for cross-modal retrieval: One model for all tasks

Beltrán, L. V. B., Caicedo, J. C., Journet, N., Coustaty, M., Lecellier, F., & Doucet, A. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.02.021 · Cited by 21 · see citing works

DC-NAS: Divide-and-Conquer Neural Architecture Search for Multi-Modal Classification

Liang, X., Fu, P., Guo, Q. J., Zheng, K., & Qian, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29281 · Cited by 21 · see citing works

HateThaiSent: Sentiment-Aided Hate Speech Detection in Thai Language

Maity, K., Poornash, A. S., Bhattacharya, S., Phosit, S., Kongsamlit, S., Saha, S., & Pasupa, K. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3376958 · Cited by 21 · see citing works

Multilabel Genre Prediction Using Deep-Learning Frameworks

Ünal, F. Z., Güzel, M. S., Bostancı, E., Açıcı, K., & Aşuroğlu, T. (2023) · Applied Sciences · doi:10.3390/app13158665 · Cited by 21 · see citing works

MSMFN: An Ultrasound Based Multi-Step Modality Fusion Network for Identifying the Histologic Subtypes of Metastatic Cervical Lymphadenopathy

Meng, Z., Zhu, Y., Pang, W., Tian, J., Nie, F., & Wang, K. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3222541 · Cited by 18 · see citing works

Dual-branch hybrid network for lesion segmentation in gastric cancer images

He, D., Zhang, Y., Huang, H., Si, Y., Wang, Z., & Li, Y. (2023) · Scientific Reports · doi:10.1038/s41598-023-33462-y · Cited by 18 · see citing works

Revisiting spatio-temporal layouts for compositional action recognition

Radevski, G., Moens, M., & Tuytelaars, T. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.278 · Cited by 17 · see citing works

CoDF-Net: coordinated-representation decision fusion network for emotion recognition with EEG and eye movement signals

Gong, X., Dong, Y., & Zhang, T. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01964-w · Cited by 17 · see citing works

Transfer Learning for Optical and SAR Data Correspondence Identification With Limited Training Labels

Zhang, M., Li, W., Tao, R., & Wang, S. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3044643 · Cited by 16 · see citing works

Multi-label movie genre classification based on multimodal fusion

Cai, Z.-H., Ding, H., Wu, J., Xi, Y., Wu, X., & Cui, X. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16121-2 · Cited by 15 · see citing works

Negative Review or Complaint? Exploring Interpretability in Financial Complaints

Das, S., Singh, A., Saha, S., & Maurya, A. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3338357 · Cited by 16 · see citing works

An information bottleneck approach for feature selection

Zhang, Q., Lu, M., Yu, S., Xin, J., & Chen, B. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111564 · Cited by 15 · see citing works

CrisisKAN: Knowledge-Infused and Explainable Multimodal Attention Network for Crisis Event Classification

Gupta, S., Saini, N., Kundu, S., & Das, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56060-6_2 · Cited by 13 · see citing works

Hierarchical Scale Awareness for object detection in Unmanned Aerial Vehicle Scenes

Wang, S., Wan, C., Yan, J., Li, S., Sun, T., Chi, J., Yang, G., Chen, C., & Yu, T. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112487 · Cited by 13 · see citing works

Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks

Yang, Y., Pan, H., Jiang, Q.-Y., Xu, Y., & Tang, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3547417 · Cited by 15 · see citing works

Multimodal Multitask Emotion Recognition using Images, Texts and Tags

Fortin, M. P., & Chaib-draa, B. (2019) · doi:10.1145/3326459.3329165 · Cited by 13 · see citing works

Copula-Based Data Augmentation on a Deep Learning Architecture for Cardiac Sensor Fusion

Silva, D., Leonhardt, S. K., & Antink, C. H. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3040551 · Cited by 13 · see citing works

N24News: A New Dataset for Multimodal News Classification

Wang, Z., Xu, S., Zhang, X., & Yang, J. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5c2xbka7swxq · Cited by 17 · see citing works

Handling unexpected inputs: incorporating source-wise out-of-distribution detection into SAR-optical data fusion for scene classification

Gawlikowski, J., Saha, S., Niebling, J., & Zhu, X. X. (2023) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-023-01008-z · Cited by 12 · see citing works

ACF: An Adaptive Compression Framework for Multimodal Network in Embedded Devices

Cai, Q., Liu, X., Zhang, K., Xie, X. N., Tong, X., & Li, K. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3303350 · Cited by 12 · see citing works

FedMMR: Multi-Modal Federated Learning via Missing Modality Reconstruction

Wang, S., Qu, Z., Liu, Y., Kan, S., Liang, Y., & Wang, J. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688162 · Cited by 12 · see citing works

Multimodal Semantic Consistency-Based Fusion Architecture Search for Land Cover Classification

Li, X., Lei, L., Zhang, C., & Kuang, G. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3193273 · Cited by 12 · see citing works

Language-Model-Based Paired Variational Autoencoders for Robotic Language Learning

Özdemir, O., Kerzel, M., Weber, C., Lee, J. H., & Wermter, S. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3204452 · Cited by 12 · see citing works

Modality-Specific Learning Rate Control for Multimodal Classification

Fujimori, N., Endo, R., Kawai, Y., & Mochizuki, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-41299-9_32 · Cited by 12 · see citing works

A multiscale neural architecture search framework for multimodal fusion

Lv, J., Sun, Y., Ye, Q., Feng, W., & Lv, J. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121005 · Cited by 12 · see citing works

Depression detection in read and spontaneous speech: A Multimodal approach for lesser-resourced languages

Daly, K., & Olukoya, O. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107959 · Cited by 12 · see citing works

Chameleon: A Multimodal Learning Framework Robust to Missing Modalities

Liaqat, M. I., Nawaz, S., Zaheer, M. Z., Saeed, M. S., Sajjad, H., Schepper, T. D., Nandakumar, K., Khan, M. H., Gallo, I., & Schedl, M. (2025) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-025-00370-y · Cited by 11 · see citing works

MULTI-MODAL DEEP LEARNING WITH SENTINEL-3 OBSERVATIONS FOR THE DETECTION OF OCEANIC INTERNAL WAVES

Drees, L., Kusche, J., & Roscher, R. (2020) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-annals-v-2-2020-813-2020 · Cited by 11 · see citing works

A comprehensive survey on image fusion: Which approach fits which need

Bernardi, G., Brisebarre, G., Roman, S., Ardabilian, M., & Dellandréa, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103594 · Cited by 11 · see citing works

COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut

Zhao, B., Xiong, H., Bian, J., Guo, Z., Xu, C., & Dou, D. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3002614 · Cited by 10 · see citing works

Multimodal Fusion Refiner Networks

Sankaran, S., Yang, D., & Lim, S.-N. (2021) · arXiv · doi:10.48550/arxiv.2104.03435 · Cited by 10 · see citing works

TransRNAm: Identifying Twelve Types of RNA Modifications by an Interpretable Multi-Label Deep Learning Model Based on Transformer

Chen, T., Wu, T., Pan, D., Xie, J., Jin, Z. X., Wang, X., Quan, L., & Lyu, Q. (2023) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2023.3307419 · Cited by 9 · see citing works

Multi-Scale Features Are Effective for Multi-Modal Classification: An Architecture Search Viewpoint

Fu, P., Liang, X., Qian, Y., Guo, Q. J., Zhang, Y., Huang, Q., & Tang, K. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3470996 · Cited by 9 · see citing works

Narrowband Radar Automatic Target Recognition Based on a Hierarchical Fusing Network With Multidomain Features

Gao, Y., Zhou, Y., Wang, Y., & Zhuo, Z. (2020) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2020.2993039 · Cited by 9 · see citing works

Enhancing Multimodal Learning via Hierarchical Fusion Architecture Search With Inconsistency Mitigation

Long, K., Xie, G., Ma, L., Li, Q., Huang, M., Lv, J., & Lu, Z. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3599673 · Cited by 8 · see citing works

Investigate How Market Behaves: Toward an Explanatory Multitasking Based Analytical Model for Financial Investments

Das, S., Chowdhury, U., Lijin, N. S., Deep, A., Saha, S., & Maurya, A. (2024) · IEEE Access · doi:10.1109/access.2024.3369033 · Cited by 8 · see citing works

Using DeepSignalingFlow to mine signaling flows interpreting mechanism of synergy of cocktails

Zhang, H., Chen, Y., Payne, P., & Li, F. (2024) · npj Systems Biology and Applications · doi:10.1038/s41540-024-00421-w · Cited by 8 · see citing works

Enhancing Acute Bilirubin Encephalopathy Diagnosis with Multi-Modal MRI: A Deep Learning Approach

Zhang, H., & Xia, S. (2024) · Applied Sciences · doi:10.3390/app14062464 · Cited by 7 · see citing works

A Multi-Stream Recurrent Neural Network for Social Role Detection in Multiparty Interactions

Zhang, L., & Radke, R. J. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2992394 · Cited by 7 · see citing works

Modeling the nonlinear responses of soft network materials assisted by masked-fusion artificial neural network

Cao, S., Wang, H., Lai, Y., Bo, R., & Feng, X. (2023) · Materials Today Communications · doi:10.1016/j.mtcomm.2023.106013 · Cited by 7 · see citing works

TDSF-Net: Tensor Decomposition-Based Subspace Fusion Network for Multimodal Medical Image Classification

Zhang, Y., Xu, G., Zhao, M. Y., Wang, H., Shi, F., & Chen, S. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3541170 · Cited by 6 · see citing works

Exploring Deep Learning-Based Multi-modality Fusion Approaches in Classification of Lung Diseases: A Review

Shingan, G. G., & Ranjan, P. (2025) · Learning and analytics in intelligent systems · doi:10.1007/978-3-031-82706-8_10 · Cited by 6 · see citing works

Cross-Aligned Fusion For Multimodal Understanding

Rajora, A., Gupta, S., & Kundu, S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00559 · Cited by 6 · see citing works

Fused and Modified Evolutionary Optimization of Multiple Intelligent Systems Using ANN, SVM approaches

Al‐bayati, J. S. H., & Üstündağ, B. B. (2020) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2020.013329 · Cited by 6 · see citing works

Multimodal dynamic graph convolutional network for crowdfunding success prediction

Cai, Z.-H., Ding, H., Xu, M., & Cui, X. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111313 · Cited by 6 · see citing works

Deep Stacked Ensemble Recommender

Otunba, R., Rufai, R. A., & Lin, J. (2019) · International Conference on Scientific and Statistical Database Management · doi:10.1145/3335783.3335809 · Cited by 6 · see citing works

Fusion of LiDAR and Camera Images in End-to-end Deep Learning for Steering an Off-road Unmanned Ground Vehicle

Warakagoda, N. D., Dirdal, J. A., & Faxvaag, E. (2019) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion43075.2019.9011341 · Cited by 6 · see citing works

A central multimodal fusion framework for outdoor scene image segmentation

Zhang, Y., Morel, O., Seulin, R., Mériaudeau, F., & Sidibé, D. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10357-y · Cited by 5 · see citing works

Optimizing Movie Selections: A Multi-Task, Multi-Modal Framework with Strategies for Missing Modality Challenges

Raj, S., Agrawal, P., Saha, S., Singh, B., & Pedanekar, N. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3635947 · Cited by 5 · see citing works

LandGPT: a multimodal large language model for parcel-level land use classification with multi-source data

Zhu, G., Tang, M., Ma, Y., Hu, Z., Yu, C., Zhang, X., Hu, H., Guan, Q., & Yao, Y. (2025) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2025.2506533 · Cited by 6 · see citing works

Feature Stacking Fusion in Multimodal Neural Architecture Search

Long, K., Ma, L., Gao, Y., & Yu, G. (2024) · International Conference on Data-driven Optimization of Complex Systems (DOCS) · doi:10.1109/docs63458.2024.10704481 · Cited by 6 · see citing works

MMRDF: An improved multitask multi‐modal rumor detection framework

Jiang, F., Liang, G., Yang, J., & Chen, L. (2023) · Electronics Letters · doi:10.1049/ell2.12811 · Cited by 5 · see citing works

A Model-Level Fusion-Based Multi-Modal Object Detection and Recognition Method

Chen, G., & Zhang, L. (2023) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait60137.2023.10528389 · Cited by 5 · see citing works

SSPRA: A Robust Approach to Continuous Authentication Amidst Real-World Adversarial Challenges

Chen, F., Xin, J., & Phoha, V. (2024) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2024.3369590 · Cited by 3 · see citing works

Diagnosis of Lung Cancer Using Endobronchial Ultrasonography Image Based on Multi-Scale Image and Multi-Feature Fusion Framework

Wang, H., Nakajima, T., Shikano, K., Nomura, Y., & Nakaguchi, T. (2025) · Tomography · doi:10.3390/tomography11030024 · Cited by 4 · see citing works

Multilingual multimodal cyberbullying detection through adaptive and hierarchical fusion

Ismail, W. S., Ullah, H., Adnan, M., & Ullah, F. (2026) · Array · doi:10.1016/j.array.2026.100689 · Cited by 7 · see citing works

Incorporating Domain Knowledge Graph into Multimodal Movie Genre Classification with Self-Supervised Attention and Contrastive Learning

Li, J., Qi, G., Zhang, C., Chen, Y., Tan, Y., Xia, C., & Tian, Y. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612085 · Cited by 4 · see citing works

On the Robustness and Reliability of Late Multi-Modal Fusion using Probabilistic Circuits

Sidheekh, S., Tenali, P., Mathur, S., Blasch, E., & Natarajan, S. (2024) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion59988.2024.10706372 · Cited by 4 · see citing works

Co-Compressing and Unifying Deep CNN Models for Efficient Human Face and Speaker Recognition

Wan, T. S. T., Lee, J., Chan, Y.-M., & Chen, C. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00060 · Cited by 4 · see citing works

Multimodal Differentiation of Obstacles in Repeated Adaptive Human-Computer Interactions

Putze, F., & Salous, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450641 · Cited by 4 · see citing works

Active learning to measure opinion and violence in French newspapers

Guélorget, P., Gadek, G., Zaharia, T. B., & Grilhères, B. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.08.021 · Cited by 4 · see citing works

Bayesian mixture variational autoencoders for multi-modal learning

Liao, K.-T., Huang, B., Yang, C.-C., & Lin, S.-D. (2022) · Machine Learning · doi:10.1007/s10994-022-06272-y · Cited by 4 · see citing works

Emotion-Recognition Algorithm Based on Weight-Adaptive Thought of Audio and Video

Cheng, Y., Zhou, D., Wang, S., & Wen, L. (2023) · Electronics · doi:10.3390/electronics12112548 · Cited by 5 · see citing works

Early vs Late Fusion in Multimodal Convolutional Neural Networks

Gadzicki, K., Khamsehashari, R., & Zetzsche, C. (2020) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion45008.2020.9190246 · Cited by 256 · see citing works

Learn to combine modalities in multimodal deep learning

Liu, K., Li, Y., Xu, N., & Natarajan, P. (2018) · arXiv · doi:10.48550/arxiv.1805.11730 · Cited by 136 · see citing works

Learning Joint Multimodal Representation Based on Multi-fusion Deep Neural Networks

Gu, Z., Lang, B., Yue, T., & Huang, L. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70096-0_29 · Cited by 19 · see citing works

Method of graph fusion of multimodal data using multitask learning

Merkotan, D., & Trotsko, O. (2025) · Communication informatization and cybersecurity systems and technologies · doi:10.58254/viti.8.2025.11.133

Robust Deep Multi-modal Learning Based on Gated Information Fusion Network

Kim, J., Koh, J., Kim, Y., Choi, J., Hwang, Y., & Choi, J. W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_6 · Cited by 76 · see citing works