Multimodal Co-learning: Challenges, applications with datasets, recent advances and future directions

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148

MizAR 60 for Mizar 50

Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

Gradient-based learning applied to document recognition

Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

You Only Look Once: Unified, Real-Time Object Detection

Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803

ImageNet Large Scale Visual Recognition Challenge

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380

Glove: Global Vectors for Word Representation

Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071

Multimodal learning with transformers: A survey

Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072

Multi-fault diagnosis of Industrial Rotating Machines using Data-driven approach : A review of two decades of research

Gawde, S., Patil, S., Kumar, S., Kamat, P. V., Kotecha, K. V., & Abraham, A. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106139 · Cited by 327

Domain Adaptation: Challenges, Methods, Datasets, and Applications

Singhal, P., Walambe, R. A., Ramanna, S., & Kotecha, K. V. (2023) · IEEE Access · doi:10.1109/access.2023.3237025 · Cited by 276

A survey of multimodal hybrid deep learning for computer vision: Architectures, applications, trends, and challenges

Bayoudh, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102217 · Cited by 180

Review of the Li-Ion Battery, Thermal Management, and AI-Based Battery Management System for EV Application

Ghalkhani, M., & Habibi, S. R. (2022) · Energies · doi:10.3390/en16010185 · Cited by 171

A comprehensive investigation of multimodal deep learning fusion strategies for breast cancer classification

Nakach, F.-Z., Idri, A., & Göçeri, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10984-z · Cited by 146

A review on EEG-based multimodal learning for emotion recognition

Pillalamarri, R., & Shanmugam, U. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11126-9 · Cited by 153

A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)

Deldjoo, Y., He, Z., McAuley, J., Korikov, A., Sanner, S., Arnau Ramisa, A., Vidal, R., Sathiamoorthy, M., Kasirzadeh, A., & Milano, S. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671474 · Cited by 177

Dynamic Vision-Based Machinery Fault Diagnosis with Cross-Modality Feature Alignment

Li, X., Yu, S., Lei, Y., Li, N., & Yang, B. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124470 · Cited by 112

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Barua, A., Ahmed, M. U., & Begum, S. (2023) · IEEE Access · doi:10.1109/access.2023.3243854 · Cited by 122

From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape

McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102

Multimodal Alignment and Fusion: A Survey

Li, S., & Tang, H. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02667-1 · Cited by 267

Visuo-haptic object perception for robots: an overview

Navarro-Guerrero, N., Toprak, S., Josifovski, J., & Jamone, L. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10091-y · Cited by 86

GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning

Deng, L., Yang, J., Yuan, S., Zou, H., Lu, C. X., & Xie, L. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3203559 · Cited by 75

Federated Learning on Multimodal Data: A Comprehensive Survey

Lin, Y., Gao, Y., Gong, M., Zhang, S., Zhang, Y., & Li, Z. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1398-0 · Cited by 73

An effective multimodal representation and fusion method for multimodal intent recognition

Huang, X., Ma, T., Jia, L., Zhang, Y., Rong, H., & Al-Nabhan, N. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126373 · Cited by 72

Latent information-guided one-step multi-view fuzzy clustering based on cross-view anchor graph

Zhang, C., Chen, L., Shi, Z., & Ding, W. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102025 · Cited by 72

Reviewing Multimodal Machine Learning and Its Use in Cardiovascular Diseases Detection

Moshawrab, M., Adda, M., Bouzouane, A., Ibrahim, H., & Raad, A. (2023) · Electronics · doi:10.3390/electronics12071558 · Cited by 77

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 86

A novel multi-label pest image classifier using the modified Swin Transformer and soft binary cross entropy loss

Guo, Q., Wang, C., Xiao, D., & Huang, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107060 · Cited by 68

Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions

Bhatt, P. R., Sethi, A., Tasgaonkar, V., Shroff, J., Pendharkar, I., Desai, A., Sinha, P., Deshpande, A., Joshi, G., Rahate, A., et al. (2023) · Brain Informatics · doi:10.1186/s40708-023-00196-6 · Cited by 75

Rice Transformer: A Novel Integrated Management System for Controlling Rice Diseases

Patil, R. R., & Kumar, S. (2022) · IEEE Access · doi:10.1109/access.2022.3200688 · Cited by 59

Excavating multimodal correlation for representation learning

Mai, S., Ya, S., Zeng, Y., & Hu, H. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.003 · Cited by 57

Two Birds With One Stone: Knowledge-Embedded Temporal Convolutional Transformer for Depression Detection and Emotion Recognition

Zheng, W., Yan, L., & Wang, F. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3282704 · Cited by 58

A deep fusion model for stock market prediction with news headlines and time series data

Chen, P., Boukouvalas, Z., & Corizzo, R. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10303-1 · Cited by 60

Multimodal vision-based human action recognition using deep learning: a review

Shafizadegan, F., Naghsh‐Nilchi, A. R., & Shabaninia, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10730-5 · Cited by 57

Learning across Diverse Biomedical Data Modalities and Cohorts: Challenges and Opportunities for Innovation

Rajendran, S., Pan, W., Sabuncu, M. R., Chen, Y., Zhou, J., & Wang, F. (2024) · Patterns · doi:10.1016/j.patter.2023.100913 · Cited by 47

Automated residential layout generation and editing using natural language and images

Zeng, P., Gao, W., Li, J., Yin, J., Chen, J., & Lu, S. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106133 · Cited by 48

Deep learning methods improve genomic prediction of wheat breeding

Montesinos‐López, A., Crespo‐Herrera, L., Dreisigacker, S., Gerard, G., Vitale, P., Pierre, C. S., Govindan, V., Tadesse, Z., Flores, M. C., Pérez‐Rodríguez, P., et al. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1324090 · Cited by 43

Incorporating stock prices and text for stock movement prediction based on information fusion

Zhang, Q., Zhang, Y., Bao, F., Liu, Y., Zhang, C., & Лю, П. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107377 · Cited by 43

Employing multimodal co-learning to evaluate the robustness of sensor fusion for industry 5.0 tasks

Rahate, A., Mandaokar, S., Chandel, P., Walambe, R. A., Ramanna, S., & Kotecha, K. V. (2022) · Soft Computing · doi:10.1007/s00500-022-06802-9 · Cited by 39

Literature survey of multi-track music generation model based on generative confrontation network in intelligent composition

Liu, W. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04914-5 · Cited by 39

Clinical applications of graph neural networks in computational histopathology: A review

Meng, X., & Zou, T. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107201 · Cited by 42

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Kieu, N., Nguyen, K., Nazib, A., Fernando, T., Fookes, C., & Sridharan, S. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3378348 · Cited by 38

Multimodal fusion techniques: Review, data representation, information fusion, and application areas

Hangloo, S., & Arora, B. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130827 · Cited by 46

Adapt and explore: Multimodal mixup for representation learning

Lin, R., & Hu, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102216 · Cited by 34

Survey of Multimodal Federated Learning: Exploring Data Integration, Challenges, and Future Directions

Adam, M., Albaseer, A. M., Baroudi, U. A., & Abdallah, M. M. (2025) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2025.3554537 · Cited by 36

Personalized uncertainty quantification in artificial intelligence

Chakraborti, T., Banerji, C. R. S., Marandon, A., Hellon, V., Mitra, R., Lehmann, B., Bräuninger, L., McGough, S. F., Turkay, Ç., Frangi, A. F., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01024-8 · Cited by 37

A survey on advancements in image–text multimodal models: From general techniques to biomedical implementations

Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 58

Backdooring Multimodal Learning

Han, X., Wu, Y., Zhang, Q., Zhou, Y. H., Xu, Y., Qiu, H., Xu, G., & Zhang, T. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00031 · Cited by 33

Physical knowledge-driven feature fusion and reconstruction network for fault diagnosis with incomplete multisource data

Sun, D., Li, Y., Jia, S., Gao, S., Noman, K., & Eliker, K. (2024) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2024.112222 · Cited by 32

Multimodal Masked Autoencoder Based on Adaptive Masking for Vitiligo Stage Classification

Fan, X., Li, Z., Jiang, S. S., Li, C., Li, S., Gao, T., He, K., Chen, J., Zhang, J., & Zhang, J. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01521-7 · Cited by 32

Missing-modality enabled multi-modal fusion architecture for medical data

Wang, M., Fan, S., Li, Y., Xie, Z., & Chen, H. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104796 · Cited by 31

Operating Condition Recognition of Industrial Flotation Processes Using Visual and Acoustic Bimodal Autoencoder With Manifold Learning

Liu, C., Wang, Y., Fang, Y., Yang, C., & Gui, W. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3359416 · Cited by 30

Energy consumption intelligent modeling and prediction for additive manufacturing via multisource fusion and inter-layer consistency

Wang, K., Yu, L., Xu, J., Zhang, S., & Qin, J. (2022) · Computers & Industrial Engineering · doi:10.1016/j.cie.2022.108720 · Cited by 34

Multimodal Boosting: Addressing Noisy Modalities and Identifying Modality Contribution

Mai, S., Ya, S., Xiong, A., Zeng, Y., & Hu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3306489 · Cited by 34

Energy efficiency design for eco-friendly additive manufacturing based on multimodal attention fusion

Wang, K., Song, Y., Sheng, H., Xu, J., Zhang, S., & Qin, J. (2022) · Journal of Manufacturing Processes · doi:10.1016/j.jmapro.2022.05.010 · Cited by 31

Multi-Modal CNN Features Fusion for Emotion Recognition: A Modified Xception Model

Shahzad, H. M., Bhatti, S. M., Jaffar, A., Rashid, M., & Akram, S. (2023) · IEEE Access · doi:10.1109/access.2023.3310428 · Cited by 28

Multimodal Deep Learning Integration of Image, Weather, and Phenotypic Data Under Temporal Effects for Early Prediction of Maize Yield

Shamsuddin, D., Danilevicz, M. F., Al-Mamun, H. A., Bennamoun, M., & Edwards, D. (2024) · Remote Sensing · doi:10.3390/rs16214043 · Cited by 29

A review and critical analysis of multimodal datasets for emotional AI

Al-Azani, S., & El-Alfy, E.-S. M. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11271-1 · Cited by 37

A co-learning method to utilize optical images and photogrammetric point clouds for building extraction

Xie, Y., Tian, J., & Zhu, X. X. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.103165 · Cited by 27

Multimodal unified generalization and translation network for intelligent fault diagnosis under dynamic environments

Zhao, C., Shen, W., Zio, E., & Ma, H. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112559 · Cited by 26

Additive manufacturing energy consumption measurement and prediction in fabricating lattice structure based on recallable multimodal fusion network

Wang, K., Song, Y., Huang, Z., Sun, Y., Xu, J., & Zhang, S. (2022) · Measurement · doi:10.1016/j.measurement.2022.111215 · Cited by 29

Deep learning on multi-view sequential data: a survey

Xie, Z., Yang, Y. Z., Zhang, Y., Wang, J., & Du, S. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10332-z · Cited by 28

Alignment-Enhanced Interactive Fusion Model for Complete and Incomplete Multimodal Hand Gesture Recognition

Duan, S., Wu, L., Liu, A., & Chen, X. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3335101 · Cited by 25

Intent recognition model based on sequential information and sentence features

Wu, T., Wang, M., Xi, Y., & Zhao, Z. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127054 · Cited by 31

Multi-modal learning-based algae phyla identification using image and particle modalities

Kwon, D. H., Lee, M. J., Jeong, H., Park, S., & Cho, K. H. (2025) · Water Research · doi:10.1016/j.watres.2025.123172 · Cited by 25

Building Multimodal Knowledge Bases With Multimodal Computational Sequences and Generative Adversarial Networks

Chen, D., & Zhang, R. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3291503 · Cited by 23

A Short Survey on Deep Learning for Multimodal Integration: Applications, Future Perspectives and Challenges

Dimitri, G. M. (2022) · Computers · doi:10.3390/computers11110163 · Cited by 22

Research on the perception of the terrain image of the tourism destination based on multimodal user-generated content data

Hu, T., & Geng, J. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1801 · Cited by 21

A Review of the Applications, Benefits, and Challenges of Generative AI for Sustainable Toxicology

Alam, F., Alnazzawi, T. S. M., Mehmood, R., & Almaghthawi, A. (2025) · Current Research in Toxicology · doi:10.1016/j.crtox.2025.100232 · Cited by 25

Navigating the Multimodal Landscape: A Review on Integration of Text and Image Data in Machine Learning Architectures

Rashid, M. B., Rahaman, M. S., & Rivas, P. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6030074 · Cited by 23

TS2ACT: Few-Shot Human Activity Sensing with Cross-Modal Co-Learning

Xia, K., Li, W., Gan, S., & Lu, S. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631445 · Cited by 19

Multimedia analysis of robustly optimized multimodal transformer based on vision and language co-learning

Yoon, J., Choi, G. H., & Choi, C. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101922 · Cited by 19

Deep Learning models for cultural pattern recognition: preserving intangible heritage of Li ethnic subgroups through intelligent documentation systems

Sun, J., Khalid, K. A. T., & Kay, C. S. (2025) · Future Technology · doi:10.55670/fpll.futech.4.3.12 · Cited by 18

Multimodal representation learning over heterogeneous networks for tag-based music retrieval

da Silva, A. C. M., Silva, D. F., & Marcacini, R. M. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117969 · Cited by 16

Employing Co-Learning to Evaluate the Explainability of Multimodal Sentiment Analysis

Jain, D. K., Rahate, A., Joshi, G., Walambe, R. A., & Kotecha, K. V. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3176403 · Cited by 17

Cross-Modal Audio-Visual Co-Learning for Text-Independent Speaker Verification

Liu, M., Lee, K. A., Wang, L., Zhang, H., Zeng, C., & Dang, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095883 · Cited by 17

Attention-Based Multimodal Deep Learning on Vision-Language Data: Models, Datasets, Tasks, Evaluation Metrics and Applications

Bose, P., Rana, P., & Ghosh, P. (2023) · IEEE Access · doi:10.1109/access.2023.3299877 · Cited by 18

Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning

Lai, S., Li, J., Guo, G., Hu, X., Li, Y., Tan, Y., Song, Z., Liu, Y., Ren, Z., Wang, C., et al. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651442 · Cited by 16

TransG-net: transformer and graph neural network based multi-modal data fusion network for molecular properties prediction

Zhang, T., Chen, S., Wulamu, A., Guo, X., Li, Q., & Zheng, H. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04351-0 · Cited by 16

An Empirical Study of Adaptive Feedback to Enhance Cognitive Ability in Programming Learning among College Students: A Perspective Based on Multimodal Data Analysis

Fu, W.-S., Zhang, J., Zhang, D., Li, T., Lan, M., & Liu, N. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331241313126 · Cited by 15

Emotion Recognition by Learning the Manifold of Fused Multiscale Information of EEG Signals

Li, C., Zhang, S., Mu, Y., Yang, L., Peng, Y., Li, F., Zhang, Y., Liang, Z., Cao, Z., Wan, F., et al. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3555226 · Cited by 16

A Sign Language Recognition Framework Based on Cross-Modal Complementary Information Fusion

Zhang, J., Wang, Q., & Wang, Q. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3377095 · Cited by 14

Make it till you fake it: Construction-centric computational framework for simultaneous image synthetization and multimodal labeling

Tohidifar, A., Kim, D., & Lee, S. H. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105696 · Cited by 14

Learning incremental audio–visual representation for continual multimodal understanding

Zhu, B., Wang, C., Xu, K., Feng, D., Zhou, Z., & Zhu, X. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112513 · Cited by 14

Coco-Attention for Tumor Segmentation in Weakly Paired Multimodal MRI Images

Zhou, P., Li, Y., Chen, H., & Peng, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3262548 · Cited by 14

Alignment-assisted Frequency Fusion Network for RGB-infrared vehicle detection

Chen, Z., Xiang, W., Lin, Z., Yang, K., Liu, Y., & Shi, Z. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130505 · Cited by 13

Multimodal large language models for medical image diagnosis: Challenges and opportunities

Zhang, A., Zhao, E., Wang, R., Zhang, X., Wang, J., & Chen, E. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104895 · Cited by 16

Intrinsic explainability of multimodal learning for crop yield simulation

Najjar, H., Pathak, D., Nuske, M., & Dengel, A. R. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.111003 · Cited by 15

Multimodal learning for fetal distress diagnosis using a multimodal medical information fusion framework

Zhang, Y., Deng, Y., Zhou, Z., Zhang, X., Jiao, P., & Zhao, Z. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.1021400 · Cited by 13

A Collaborative Multimodal Learning-Based Framework for COVID-19 Diagnosis

Gao, Y., Gong, M., Ong, Y.-S., Qin, A. K., Wu, Y., & Xie, F. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3290188 · Cited by 13

An fNIRS representation and fNIRS-scales multimodal fusion method for auxiliary diagnosis of amnestic mild cognitive impairment

Cheng, S., Shang, P., Zhang, Y., Guan, J., Chen, Y., Lv, Z., Huang, S., Liu, Y., & Xie, H. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106646 · Cited by 13

SoK: Behind the Accuracy of Complex Human Activity Recognition Using Deep Learning

Nguyen, D., & Le‐Khac, N. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650322 · Cited by 13

Investigation and Empirical Analysis of Transfer Learning for Industrial IoT Networks

Yadav, P., Rishiwal, V., Yadav, M., Alotaibi, A., Maurya, V., Agarwal, U., & Sharma, S. (2024) · IEEE Access · doi:10.1109/access.2024.3499741 · Cited by 13

Relative Norm Alignment for Tackling Domain Shift in Deep Multi-modal Classification

Planamente, M., Plizzari, C., Peirone, S. A., Barbara Caputo, B., & Bottino, A. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-01998-9 · Cited by 14

Multimodal-Based Learning Improves Students' Reading Ability in Perspective of Systemic Functional Linguistics

Purba, R., Herman, H., Fatmawati, E., Saputra, N., & Sabata, Y. N. (2023) · Journal of Education Technology · doi:10.23887/jet.v7i1.54336 · Cited by 12

Multimodal deep learning for allergenic proteins prediction

Yu, L., Luo, Y., Wu, S., Chen, S., Xue, L., Jing, R., & Luo, J. (2025) · BMC Biology · doi:10.1186/s12915-025-02347-z · Cited by 11

Mining and reasoning of data uncertainty-induced imprecision in deep image classification

Zhang, Z., Ning, L., Liu, Z., Yang, Q., & Ding, W. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.014 · Cited by 11

Multimodal Co-Learning for Building Change Detection: A Domain Adaptation Framework Using VHR Images and Digital Surface Models

Xie, Y., Yuan, X., Zhu, X. X., & Tian, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3362680 · Cited by 11

A Social Media Dataset and H-GNN-Based Contrastive Learning Scheme for Multimodal Sentiment Analysis

Peng, J., He, Y., Chang, Y., Lu, Y., Zhang, P., Ou, Z., & Yu, Q. (2025) · Applied Sciences · doi:10.3390/app15020636 · Cited by 11

A Stack Based Multimodal Machine Learning Model for Breast Cancer Diagnosis

Kayıkçı, Ş., & Khoshgoftaar, T. M. (2022) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/hora55278.2022.9800004 · Cited by 11

Styled and characteristic Peking opera facial makeup synthesis with co-training and transfer conditional styleGAN2

Shen, Y., Duan, O., Xin, X., Yan, M. G., & Li, Z. (2024) · Heritage Science · doi:10.1186/s40494-024-01463-3 · Cited by 9

Transfer Learning of Human Activities Based on IMU Sensors: A Review

Ashry, S., Das, S., Rafiei, M., Baumbach, J., & Baumbach, L. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3510097 · Cited by 10

FDFR-Net: A fruit ripeness classification method using multimodal learning technique

Wang, Y., Jin, X., Huang, W., Wang, X., & Yu, W. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130126 · Cited by 9

Evidence combination with multi-granularity belief structure for pattern classification

Zuo, K., Li, X., Yu, L., Shen, T., Dong, Y., & Dezert, J. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121577 · Cited by 8

Cross-language dissemination of Chinese classical literature using multimodal deep learning and artificial intelligence

Bai, Y., & Lei, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-05921-1 · Cited by 8

MDCNN: A multimodal dual-CNN recursive model for fake news detection via audio- and text-based speech emotion recognition

Wu, H., Li, H., Fang, X., Tang, M., Yu, H., Yu, B., Li, M., Jing, Z., Meng, Y., Chen, W., et al. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103313 · Cited by 8

Comparison Analysis of Multimodal Fusion for Dangerous Action Recognition in Railway Construction Sites

Amel, O., Siebert, X., & Mahmoudi, S. A. (2024) · Electronics · doi:10.3390/electronics13122294 · Cited by 8

A survey on semiconductor wafer yield prediction by artificial intelligence

Zhao, D., Chen, Y., Guo, P., Chen, F., Jing, R., Xu, X., Chen, Y., Liu, F., & Wu, B. (2025) · Microelectronics Journal · doi:10.1016/j.mejo.2025.106958 · Cited by 7

Multimodal Learning and its Applications

Haddag, A. (2023) · doi:10.13140/rg.2.2.33217.57440

Tutorial on Multimodal Machine Learning: Principles, Challenges, and Open Questions

Liang, P. P., & Morency, L. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3610661.3617602 · Cited by 48

Deep Multimodal Learning with Missing Modality: A SurveyPreprint

Wu, R., Hu, W., Chen, H., & Carneiro, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.07825 · Cited by 182

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 86

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Barua, A., Ahmed, M. U., & Begum, S. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7736385