Multimodal Co-learning: Challenges, applications with datasets, recent advances and future directions

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148

MizAR 60 for Mizar 50

Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

Gradient-based learning applied to document recognition

Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

You Only Look Once: Unified, Real-Time Object Detection

Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803

ImageNet Large Scale Visual Recognition Challenge

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380

Glove: Global Vectors for Word Representation

Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071

Integrative multimodal hybrid data fusion for mortality prediction

Abuhamad, H., Zainudin, S., & Bakar, A. A. (2026) · Scientific Reports · doi:10.1038/s41598-026-36296-6 · Cited by 8

MCDGait: multimodal co-learning distillation network with spatial-temporal graph reasoning for gait recognition in the wild

Xiong, J., Zou, S., Tang, J., & Tjahjadi, T. (2024) · The Visual Computer · doi:10.1007/s00371-024-03426-y · Cited by 7

EMPC: Efficient multi-view parallel co-learning for semi-supervised action recognition

Tong, A., Tang, C., & Wang, W. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124634 · Cited by 7

Autonomic Computing Based Respiratory Disorders Assessment Using Speech Parameters: A Systematic Review

Shrivastava, P., Tripathi, N., Dewangan, B. K., Singh, B. K., Choudhury, T., Kotecha, K. V., & Dewangan, S. (2023) · International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT) · doi:10.1109/ismsit58785.2023.10304972 · Cited by 6

Audio-Text Multimodal Speech Recognition via Dual-Tower Architecture for Mandarin Air Traffic Control Communications

Ge, S., Ren, J., Shi, Y., Zhang, Y., Yang, S., & Yang, J. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.046746 · Cited by 7

Multi-modal co-learning for silent speech recognition based on ultrasound tongue images

Guo, M., Wei, J., Zhang, R., Zhao, Y., & Fang, Q. (2024) · Speech Communication · doi:10.1016/j.specom.2024.103140 · Cited by 8

Audio–visual self-supervised representation learning: A survey

Alsuwat, M. S., Al-Shareef, S., & Alghamdi, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129750 · Cited by 7

SRIF-RCNN: Sparsely represented inputs fusion of different sensors for 3D object detection

Li, X., & Kong, D. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03594-1 · Cited by 6

Learning Modality Consistency and Difference Information with Multitask Learning for Multimodal Sentiment Analysis

Fang, C., Liang, F., Li, T., & Guan, F. (2024) · Future Internet · doi:10.3390/fi16060213 · Cited by 6

Human in the collaborative loop: a strategy for integrating human activity recognition and non-invasive brain-machine interfaces to control collaborative robots

Pilacinski, A., Christ, L., Boshoff, M., Iossifidis, I., Adler, P., Miro, M., Kuhlenkötter, B., & Klaes, C. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1383089 · Cited by 9

Fusion for Visual-Infrared Person ReID in Real-World Surveillance Using Corrupted Multimodal Data

Josi, A., Alehdaghi, M., Cruz, R. M. O., & Granger, É. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02396-5 · Cited by 6

Brain-Machine Cross-Modal Alignment via Sample Relational Learning for Visual Classification

Liu, D., Dai, W., Liu, H., Yi, H., & Kong, W. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3736416 · Cited by 5

Systematic literature review on hate speech detection in Indian low-resource languages

Pannerselvam, K., & Rajiakodi, S. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00432-5 · Cited by 5

Optimization Strategies in Consumer Choice Behavior for Personalized Recommendation Systems Based on Deep Reinforcement Learning

Wei, Z., Liang, Y., & Zhang, C. (2025) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.368009 · Cited by 5

A scalable multi-modal learning fruit detection algorithm for dynamic environments

Mao, L., Guo, Z., Liu, M., Li, Y., Wang, L., & Li, J. (2025) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1518878 · Cited by 5

Multimodal Ensemble with Conditional Feature Fusion for Dysgraphia Diagnosis in Children from Handwriting Samples

Kunhoth, J., Saleh, M., Al-Maadeed, S. A., & Akbari, Y. (2026) · Cognitive Computation · doi:10.1007/s12559-026-10546-1 · Cited by 5

Interpreting wealth distribution via poverty map inference using multimodal data

Espín-Noboa, L., Kertész, J., & Karsai, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583862 · Cited by 5

Representations in Developing Statistical Thinking in Secondary Education

Selvakumar, P., Sharma, A., Roy, S. S., Mohit, M., & Manjunath, T. C. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-9934-7.ch005 · Cited by 5

Unpaired multi-modal tumor segmentation with structure adaptation

Zhou, P., Chen, H., Li, Y., & Peng, Y. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03610-4 · Cited by 5

Multi-modal co-learning for Earth observation: enhancing single-modality models via modality collaboration

Mena, F., Ienco, D., Dantas, C. F., Interdonato, R., & Dengel, A. R. (2025) · Machine Learning · doi:10.1007/s10994-025-06903-0 · Cited by 5

Host–Microbe Interactions: Prospects of Machine Learning and Deep Learning Technologies in Animal Viral Disease Management

Lu, Y., Li, X., El‐Aty, A. M. A., Ju, X., & Yong, Y. (2025) · Veterinary Sciences · doi:10.3390/vetsci12121129 · Cited by 4

Deep multimodal learning for time series analysis in social computing: a survey

Yang, C., Chen, Y., Li, Z., Wang, X., Shi, K., Yao, L., Xu, G., & Guo, Z. (2025) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-025-00363-x · Cited by 5

Second Language Acquisition Support Through Media Literacy and Digital Media Platforms

Mehta, V., Selvakumar, P., Anute, N., Gandhimathi, S., Seelam, P. K., & Manjunath, T. C. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-3496-7.ch011 · Cited by 4

Dimension-Wise Gated Cross-Attention for Multimodal Sentiment Analysis

Hossain, M. J., Hossain, M. M., Chaki, S., Mridha, M. F., Rahman, M. S., & Moni, M. A. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3718381 · Cited by 4

Multimodal 3D Object Retrieval

Pegia, M. E., Jónsson, B. Þ., Moumtzidou, A., Diplaris, S., Gialampoukidis, I., Vrochidis, S., & Kompatsiaris, I. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53302-0_14 · Cited by 4

Learning Robust Representations of Tonic-Clonic Seizures With Cyclic Transformer

Zhang, J., Swinnen, L., Chatzichristos, C., Paesschen, W. V., & Vos, M. D. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3375123 · Cited by 4

LUSTER: Link Prediction Utilizing Shared-Latent Space Representation in Multi-Layer Networks

Yang, R., Ali, M. A., Wang, H., Chen, J., & Wang, D. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714631 · Cited by 4

TextAI 3.0 (Multimodal): multimodal sentiment analysis using attention-enabled ensemble-based deep learning in hyperbolic space

Singh, J., Sanga, P., Khanna, N. N., Laird, J. R., Singh, I. M., Kalra, M. K., Mantella, L. E., Johri, A. M., Faa, G., Tiwari, E., et al. (2025) · International Journal of Information Technology · doi:10.1007/s41870-025-02541-w · Cited by 4

Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy

Pattnayak, P., Patel, H. L., Kumar, B., Agarwal, A., Banerjee, I., Panda, S., & Kumar, T. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-14041-8_4 · Cited by 30

Deep Multimodal Networks for M-type Star Classification with Paired Spectrum and Photometric Image

Gao, J., Chen, J., Wei, J., Jiang, B., & Luo, A.-L. (2023) · Publications of the Astronomical Society of the Pacific · doi:10.1088/1538-3873/acc7ca · Cited by 4

Integrating affective computing and deep learning for learning path optimization in vocational education

Zhang, H., Leong, W. Y., & Li, Y. (2025) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/17671 · Cited by 3

An Intelligent Human-Machine Interface Based on Eye Tracking for Written Communication of Patients with Locked-in Syndrome

Alagusabai, A., Sandhiya, R., Bhavani, M. D., Kukreti, K., Mohanaprakash, T. A., & Banu, A. H. (2023) · International Conference on Advances in Computing, Communication and Applied Informatics (ACCAI) · doi:10.1109/accai58221.2023.10200158 · Cited by 3

Multi-modal Learning with Missing Modality in Predicting Axillary Lymph Node Metastasis

Zhang, S., Zheng, S., Shui, Z., Li, H., & Yang, L. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385578 · Cited by 4

Emotion Detection using Deep Learning

Chaithra, G., Shrinivas, P. A., Naidu, B. G., Vinay, G. S., Supreeth, S., & Biradar, A. (2024) · International Conference on Networks, Multimedia and Information Technology (NMITCON) · doi:10.1109/nmitcon62075.2024.10699079 · Cited by 3

Synergy-CLIP: Extending CLIP With Multi-Modal Integration for Robust Representation Learning

Cho, S., Jeon, J., Kim, M., & Kim, J. (2025) · IEEE Access · doi:10.1109/access.2025.3559663 · Cited by 4

Using large multimodal models to predict outfit compatibility

Chang, C.-L., Chen, Y., & Jiang, D.-X. (2025) · Decision Support Systems · doi:10.1016/j.dss.2025.114457 · Cited by 2

Cyber Social Threats: A Data-centric AI Perspective

Kulkarni, A. (2025) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3696673.3723071 · Cited by 2

Cross-Modal Federated Learning among Unimodal Devices

Deng, Y., He, N., Li, X., Wu, F., Zhang, Y., & Ren, J. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749467 · Cited by 2

MOSS: Multi-Modal Representation Learning on Sequential Circuits

Wang, M., Sun, B., Mu, J., Gu, F., Han, B., Yang, T., Zhang, X., Liu, S., Wen, Y., Wang, H., et al. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132437 · Cited by 2

A review of recent advances in generative artificial intelligence models for biomolecular sciences

Jiang, J., Li, D., Wang, G., Hayes, N., Shi, Y.-Z., Qiu, H., Zhang, B., Zhou, T., & Wei, G. (2025) · Acta Pharmaceutica Sinica B · doi:10.1016/j.apsb.2025.12.012 · Cited by 2

Multimodal machine learning in real estate appraisal: models, data, and trends

Huang, C., Liang, B., Li, Z., Wang, J., & Chen, F. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01204-8 · Cited by 2

Electronic Equipment Waste Generated by Computers and its Effect on Public Health in India

Dewangan, S., Mundeja, P., Deshpande, B., Dewangan, B. K., Choudhury, T., & Kotecha, K. V. (2023) · International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT) · doi:10.1109/ismsit58785.2023.10304954 · Cited by 2

Multimodal semantic enhanced representation network for micro-video event detection

Li, Y., Liu, X., Zhang, L., Tian, H., & Jing, P. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112255 · Cited by 2

Advancements in Multimodal Social Media Post Summarization: Integrating GPT-4 for Enhanced Understanding

Alam, M. J., Hossain, I., Puppala, S., & Talukder, S. K. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00307 · Cited by 2

Enhancing Road Safety: The Role of Intelligent Driver Drowsiness Detection Systems

Khare, S., & Talwandi, N. S. (2024) · IEEE International Conference on Blockchain and Distributed Systems Security (ICBDS) · doi:10.1109/icbds61829.2024.10837023 · Cited by 2

Select for better learning: identifying high-quality training data for a multimodal cyclic transformer

Zhang, J., Liu, Z., Chatzichristos, C., Michiels, S., Paesschen, W. V., Hughes, D., & Vos, M. D. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/adbec0 · Cited by 2

Book Topic and Emotion Classification by Child Readers in a Library in Taiwan

Wu, K.-C., Chiu, T., Chen, C.-C., Liu, C.-C., & Hsu, F.-M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2453609 · Cited by 1

Multi-modal data augmentation based on masked modeling for image–text retrieval

Wang, M., Chao, G., Wang, X., Chen, Y., Xie, X., & Chu, D. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113821 · Cited by 1

Navigating machine learning for travel behavior analysis: A comprehensive guide to inferring trip purposes using transit survey and automatic fare collection data fusion

Baek, K., & Khani, A. (2026) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2026.105726 · Cited by 1

A hierarchical interaction multimodal model for feature fusion based on RoBERTa-Keyword-ViT

Wang, Y., Zhou, Y., Zhong, M., Mei, Y., Fujita, H., & Aljuaid, H. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06764-z · Cited by 2

Multimodal confusion state detection in Japanese text reading using Vision Transformers, facial video, and eye tracking

Ito, Y., Ishizawa, C., & Kageyama, Y. (2025) · Displays · doi:10.1016/j.displa.2025.103267 · Cited by 2

Filling the Gaps: A Multitask Hybrid Multiscale Generative Framework for Missing Modality in Remote Sensing Semantic Segmentation

Kieu, N., Nguyen, K., Wiliem, A., Fookes, C., Sridharan, S., & Wiliem, A. (2025) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta68720.2025.11302479 · Cited by 1

Pitch-Aware multi-feature fusion for classifying statements, questions, and exclamations in low-resource languages

Abdulrahman, A. O. (2026) · Computer Speech & Language · doi:10.1016/j.csl.2026.101941 · Cited by 1

AN OVERVIEW OF MULTIMODAL LEARNING: CONCEPTS, CHALLENGES, APPLICATIONS AND DATASETS

Ateş, A. H., & Yurtsever, U. (2026) · Konya Journal of Engineering Sciences · doi:10.36306/konjes.1716813 · Cited by 1

Modality Graphs Predict Cross-Modal Reconstructability But Not Fusion Gain: A Preregistered Study in Machine Fault Diagnosis

Nikolenko, S. (2026) · IEEE Access · doi:10.1109/access.2026.3741122 · Cited by 1

Progressive Learning of a Multimodal Classifier Accounting for Different Modality Combinations

John, V., & Kawanishi, Y. (2023) · Sensors · doi:10.3390/s23104666 · Cited by 1

Enhancing Scientific Image Classification through Multimodal Learning: Insights from Chest X-Ray and Atomic Force Microscopy Datasets

Meshnick, D. C., Shahini, N., Ganguly, D., Wu, Y., French, R. H., & Chaudhary, V. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386478 · Cited by 2

BreastRegNet: A Deep Learning Framework for Registration of Breast Faxitron and Histopathology Images

Golestani, N., Wang, A., Bean, G. R., & Rusu, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47425-5_17 · Cited by 2

Research on Image Translation Problems Based on Multimodal Data Set Fusion

Zhou, L. (2024) · International Journal of Computer Science and Information Technology · doi:10.62051/ijcsit.v3n3.03 · Cited by 1

Masked Visual Pre-training for RGB-D and RGB-T Salient Object Detection

Qi, Y., Guo, R., Li, Z., Niu, D., & Qu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8620-6_4 · Cited by 1

Modal-Centric Insights Into Multimodal Federated Learning for Smart Healthcare: A Survey

Wang, D., Liu, W., Gao, L., Qu, Y., Zhang, H., & Shi, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1548-3_10 · Cited by 1

A Global–Local Fusion Model Exploring Temporal–Spatial Dependence for Multimodal Hand Gesture Recognition

Duan, S., Wu, L., Liu, A., & Chen, X. (2025) · IEEE Transactions on Medical Robotics and Bionics · doi:10.1109/tmrb.2025.3550646 · Cited by 1

Innovation of Multimodal Learning Paths Based on Learning Behavior and Sentiment Analysis in AI Digital Intelligence Platform

Wang, L., Peng, N., Liu, L., & Wei, S. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.246 · Cited by 3

CAMEL: Combination of asymmetrically dual representation learning with mutual data filtering and masked language modeling for text-based person retrieval

Nguyen, A. D., & Nguyen, H. N. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132418 · Cited by 1

Enhancing Road Safety: The Role of Intelligent Driver Drowsiness Detection SystemsPreprint

Patra, S., Kumar, B. C., Pal, A., Kumar, R., Kumar, S., & Pal, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4824938 · Cited by 1

Combining Real and Synthetic Data to Overcome Limited Training Datasets in Multimodal LearningPreprint

Marini, N., Liang, Z., Rajaraman, S., Xue, Z., & Antani, S. (2025) · medRxiv · doi:10.1101/2025.07.16.25331662 · Cited by 2

Multimodal feature learning framework for disease biomarker discovery

Ghandikota, S. K., Yella, J., & Jegga, A. G. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995233

Application of Cloud Computing and Big Data in Accounting Software

Jayalakshmi., M., Savaliya, R. R., Atmakuri, P., Prasad, M. V. R., Kafila, & Kanika, K. (2023) · International Conference on Science Technology Engineering and Mathematics (ICONSTEM) · doi:10.1109/iconstem56934.2023.10142699

Mutli-Modality Federate Learning with Multi-Source Data for Forest Fire Prediction

Mutakabbir, A. (2023) · doi:10.22215/etd/2023-15634

Autonomous Systems Functional Safety Overview with Multimodality and Explainability Perspectives

Tiako, P., & Kamsu-Foguem, B. (2023) · Chronicle of computing · doi:10.55432/978-1-6692-0003-1_1 · Cited by 1

Multimodality in Media Retrieval

Pegia, M. E. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3657583

Scoping Review on Image-Text Multimodal Machine Learning Models

Rashid, M. B., & Rivas, P. (2023) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00035

Cascaded Encoder-Decoder Reconstruction Network with Gated Mechanism for Multimodal Emotion Recognition under Missing Modalities

Sun, L., Li, X., Zhang, J., Bai, C., & Pan, J. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650413

Multimodal Coordinated Representation Learning Based on Evidence Theory

Li, W., Han, D., Dezert, J., & Yang, Y. (2024) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion59988.2024.10706295

Latent Challenges of Multimodal Deep Learning Models: Taxonomy and Survey

Kumar, S., Ivanova, O., Vorfolomeyeva, O., & Kumar, R. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-97-8160-7_4

Аналiза і клясифікация русиньской бесіды языковым модельом штучной інтеліґенциі OpenAI Whisper

Małecki, P., & Piotrowska, M. (2024) · Rocznik Ruskiej Bursy · doi:10.12797/rrb.20.2024.20.10

Wheat Breeding Across the Full Lifecycle with Multimodal Deep Learning: Practical Review and Future Strategies

Zhang, L., & Li, X. (2024) · International Conference on Artificial Intelligence and Automation Control (AIAC) · doi:10.1109/aiac63745.2024.10899714

The Enhanced vision-language model developing for fine-grained difficult sample classification tasks

Xia, A., Zhu, B. B., Chen, Y., & Wang, R. (2024) · International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC) · doi:10.1109/icairc64177.2024.10900073

Topologically Consistent Prototype Network for Incomplete Multimodal Learning

Wang, Y., Zhang, X.-Y., & Liu, C. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10448-8

Optimization of MOOC resource recommendation algorithm for art studies based on the google dataflow architecture

Han, T., & Zhang, Q. (2025) · IEEE Conference on Telecommunications, Optics and Computer Science (TOCS) · doi:10.1117/12.3067996

A soft sensing method for mechanical properties of hot-rolled strips based on improved co-training

Shi, B., Xue, J., & Ye, H. (2025) · Chinese Journal of Chemical Engineering · doi:10.1016/j.cjche.2025.04.010

GOMFuNet: A Geometric Orthogonal Multimodal Fusion Network for Enhanced Prediction Reliability

Guo, Y., & Zhong, R. (2025) · Mathematics · doi:10.3390/math13111791

Multimodal Contrastive Learning for Music with Incomplete Modalities

Nakata, K., & Eguchi, K. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733488

Research on Optimization of Hint Words of Large Model in Multimodal Learning

Cheng, C., Lu, F., & Wu, Z. (2025) · Learning and analytics in intelligent systems · doi:10.1007/978-3-031-98607-9_3

Language Learning in the XR Era: Possibilities and Challenges of Immersive Technology

Yasuda, A., & Maruyama, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97769-5_18

The Application and Development of Machine Learning in the Assessment of Cognitive Abilities in Preschool Children

Liu, H. (2025) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste64638.2025.11092167

A Step towards Interpretable Multimodal AI Models with MultiFIX

Malafaia, M., Schlender, T., Alderliesten, T., & Bosman, P. A. N. (2025) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3712255.3734292

A Comparative Study on Multi-Modal Fusion Methods for Object Recognition

Cao, X., Han, D., & Li, W. (2025) · Chinese Control Conference (CCC) · doi:10.23919/ccc64809.2025.11179566

TriModal Enhanced Fusion Network: Advancing Multimodal Representation and Fusion for Enhanced Multimodal Intent Recognition

Wang, Y., Wang, K., Zhang, H., Fang, M., & Liu, S. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11210194

Cross-Modal Meta-Generalization for Silent Speech Recognition via Ultrasound Tongue Images

Guo, M., Fang, Q., Zhang, H., Lu, W., & Wei, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3646808

Lidar-Based 3D Perception for Intelligent Vehicles Using Polar Coordinate Voxels

Luxing, L., & Wei, C. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-7850-1_41

A Novel Responsible AI Model for Fusion Fake News Detection

Xia, H., Wang, Y., Zhang, Z., Liu, D. Y., & Sun, J. (2026) · Journal of Database Management · doi:10.4018/jdm.397913

Multimodal sentiment analysis: integrating text, image, and audio

George, J. P. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-27528-9.00017-6 · Cited by 1

STDC: Sparse Transformer Deep Collaboration Prompt Tuning for Industrial Multimodal Large Models

Bei, Y., Wang, K., & Zhao, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5567-3_11

Align and Adapt: Multimodal Multiview Human Activity Recognition Under Arbitrary View Combinations

Nguyen, D., & Le‐Khac, N. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3718255

Redundancy-Free View Alignment for Multimodal Human Activity Recognition with Arbitrarily Missing ViewsPreprint

Nguyen, D., & Le‐Khac, N. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6216286

Towards Fine-Grained Knowledge Tracing by Hierarchical Fusion of Multiple Question Attributes

Shen, S., Mo, Q., Huang, Z., Su, Y., Zhu, L., Lu, J., & Liu, Q. (2026) · ACM Transactions on Information Systems · doi:10.1145/3797886

Infrared and Visible Image Fusion Using Ternary Cycle-Consistent Adversarial Networks

Ge, K., Wang, X., Han, S., Zhou, G., & Wang, Q. (2025) · International Conference on Virtual Reality and Visualization · doi:10.1109/icvrv67992.2025.00124

Automatic analysis of cardiac function with artificial intelligence : multimodal approach for portable echocardiographic devices

Yang, Y. (2023) · doi:10.70675/e5ad2298z0bf7z4fe8za059z7fba326f7e8d

yakRNA Design: A semantic multimodal RNA composerPreprint

Pinpin, L. N., & Khan, Y. A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.04.22.720245

Multimodal Learning and its Applications

Haddag, A. (2023) · doi:10.13140/rg.2.2.33217.57440

Tutorial on Multimodal Machine Learning: Principles, Challenges, and Open Questions

Liang, P. P., & Morency, L. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3610661.3617602 · Cited by 48

Deep Multimodal Learning with Missing Modality: A SurveyPreprint

Wu, R., Hu, W., Chen, H., & Carneiro, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.07825 · Cited by 182

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 86

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Barua, A., Ahmed, M. U., & Begum, S. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7736385