Multimodal Co-learning: Challenges, applications with datasets, recent advances and future directions

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148

MizAR 60 for Mizar 50

Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

Gradient-based learning applied to document recognition

Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

You Only Look Once: Unified, Real-Time Object Detection

Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803

ImageNet Large Scale Visual Recognition Challenge

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380

Glove: Global Vectors for Word Representation

Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071

SMOTE synthetic minority over-sampling technique

Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545

Distributed Representations of Words and Phrases and their Compositionality

Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533

Learning Multiple Layers of Features from Tiny Images

Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889

A survey on transfer learning

Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372

Generative Adversarial Nets

Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119

GAN（Generative Adversarial Nets）

淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515

Learning Transferable Visual Models From Natural Language Supervision

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293

Momentum Contrast for Unsupervised Visual Representation Learning

He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347

Deep Contextualized Word Representations

Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983

WordNet: An electronic lexical database . Ed. by Christiane Fellbaum. Cambridge, MA: MIT Press, 1998. Pp. xxii, 423.

Kilgarriff, A. (2000) · Language · doi:10.2307/417141 · Cited by 11,529

A simple framework for contrastive learning of visual representations

Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857

Conditional Generative Adversarial Nets

Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248

Multitask Learning

Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636

A Survey on Bias and Fairness in Machine Learning

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812

Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network

Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680

Combining labeled and unlabeled data with co-training

Blum, A. L., & Mitchell, T. M. (1998) · Conference on Learning Theory · doi:10.1145/279943.279962 · Cited by 5,597

ADASYN: Adaptive synthetic sampling approach for imbalanced learning

He, H., Bai, Y., Garcia, E. A., & Shutao Li, A. (2008) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4633969 · Cited by 5,271

Hierarchical Attention Networks for Document Classification

Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022

Multimodal Machine Learning: A Survey and Taxonomy

Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018

Reading digits in natural images with unsupervised feature learning

Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261

An Introduction to Convolutional Neural NetworksPreprint

O’Shea, K., & Nash, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08458 · Cited by 4,082

IEMOCAP: Interactive emotional dyadic motion capture database

Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869

Unsupervised Feature Learning via Non-parametric Instance Discrimination

Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial NetsPreprint

Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 3,557

Generalizing from a Few Examples: A Survey on Few-shot Learning

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2021) · ACM Computing Surveys · doi:10.1145/3386252 · Cited by 3,165

Conditional Image Synthesis With Auxiliary Classifier GANsPreprint

Odena, A., Olah, C., & Shlens, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09585 · Cited by 3,030

One-shot learning of object categories

Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029

Optimization as a Model for Few-Shot Learning

Ravi, S., & Larochelle, H. (2017) · International Conference on Learning Representations · Cited by 2,766

Untitled work

(2011) · Cited by 2,752

Untitled work

(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524

Explaining Explanations: An Overview of Interpretability of Machine Learning

Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507

Generative Adversarial Text to Image Synthesis

Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494

From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

Young, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00166 · Cited by 2,439

LXMERT: Learning Cross-Modality Encoder Representations from Transformers

Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319

DeViSE: A Deep Visual-Semantic Embedding Model

Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,309

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109

Tensor Fusion Network for Multimodal Sentiment Analysis

Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102

Contrastive Multiview Coding

Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997

Deep Boltzmann machines

Salakhutdinov, R., & Hinton, G. E. (2009) · Cited by 1,890

Classification in the Presence of Label Noise: A Survey

Frenay, B., & Verleysen, M. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2292894 · Cited by 1,777

Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy LabelsPreprint

Han, B., Yao, Q., Yu, X., Niu, G., Xu, M., Hu, W., Tsang, I. W.-H., & Sugiyama, M. (2018) · arXiv (Cornell University) · doi:10.5555/3327757.3327944 · Cited by 1,674

Contrastive Learning for Unpaired Image-to-Image Translation

Park, T., Efros, A. A., Zhang, R., & Zhu, J.-Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_19 · Cited by 1,663

A review of affective computing: From unimodal analysis to multimodal fusion

Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644

Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint

Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,357

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

Hendrycks, D., & Dietterich, T. (2019) · arXiv · doi:10.48550/arxiv.1903.12261 · Cited by 1,335

Domain Generalization: A Survey

Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315

VideoBERT: A Joint Model for Video and Language Representation Learning

Sun, C., Myers, A., Vondrick, C., Murphy, K. P., & Schmid, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00756 · Cited by 1,255

Interpretable Machine Learning: Fundamental Principles and 10 Grand ChallengesPreprint

Rudin, C., Chen, C.-F., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2021) · Statistics Survey · doi:10.48550/arxiv.2103.11251 · Cited by 1,254

VisualBERT: A Simple and Performant Baseline for Vision and Language

Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198

Efficient Low-rank Multimodal Fusion With Modality-Specific Factors

Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170

Translating Videos to Natural Language Using Deep Recurrent Neural Networks

Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R. J., & Saenko, K. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1173 · Cited by 1,043

Multi-view learning overview: Recent progress and new challenges

Zhao, J., Xie, X., Xu, X., & Sun, S. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.007 · Cited by 1,002

Learning from massive noisy labeled data for image classification

Xiao, T., Xia, T., Yang, Y., Huang, C., & Wang, X. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298885 · Cited by 998

Zero-Shot Learning Through Cross-Modal Transfer

Socher, R., Ganjoo, M., Sridhar, H., Bastani, O., Manning, C. D., & Ng, A. Y. (2013) · arXiv · doi:10.48550/arxiv.1301.3666 · Cited by 997

Interpretable machine learning: Fundamental principles and 10 grand challenges

Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958

“Put-that-there”

Bolt, R. A. (1980) · ACM SIGGRAPH Computer Graphics · doi:10.1145/965105.807503 · Cited by 921

Meta-Learning in Neural Networks: A Survey

Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886

A Comprehensive Survey on Domain Adaptation for Visual Applications

Csurka, G. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-58347-1_1 · Cited by 879

A survey of multi-view machine learning

Sun, S. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1362-6 · Cited by 878

MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted LabelsPreprint

Jiang, L., Zhou, Z., Leung, T., Li, L.-J., & Fei-Fei, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05055 · Cited by 865

A survey on deep learning for multimodal data fusion

Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821

Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review

Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806

Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions

Ringeval, F., Sonderegger, A., Sauer, J., & Lalanne, D. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553805 · Cited by 771

Towards Automatic Learning of Procedures From Web Instructional Videos

Zhou, L., Xu, C., & Corso, J. J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12342 · Cited by 722

SoundNet: Learning Sound Representations from Unlabeled VideoPreprint

Aytar, Y., Vondrick, C., & Torralba, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09001 · Cited by 678

Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization

Wei, H., Feng, L., Chen, X., & An, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01374 · Cited by 649

End-to-End Learning of Visual Representations From Uncurated Instructional Videos

Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Šivic, J., & Zisserman, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00990 · Cited by 646

GRAM: Graph-based Attention Model for Healthcare Representation Learning

Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098126 · Cited by 637

Gated Multimodal Units for Information FusionPreprint

Arévalo, J., Solorio, T., Montes-y-Gómez, M., & Gonzalez, F. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01992 · Cited by 610

What Makes Training Multi-modal Classification Networks Hard?

Wang, W., Tran, D., & Feiszli, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01271 · Cited by 588

Jointly Modeling Embedding and Translation to Bridge Video and Language

Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.497 · Cited by 572

Cross-modal Retrieval with Correspondence Autoencoder

Feng, F., Wang, X., & Li, R. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654902 · Cited by 572

Multiple instance classification: Review, taxonomy and comparative study

Amores, J. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2013.06.003 · Cited by 562

Deep Co-Training for Semi-Supervised Image Recognition

Qiao, S., Shen, W., Zhang, Z., Wang, B., & Yuille, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_9 · Cited by 552

How does Disagreement Help Generalization against Label Corruption?Preprint

Yu, X., Han, B., Yao, J., Niu, G., Tsang, I. W.-H., & Sugiyama, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.04215 · Cited by 538

Deep Multimodal Representation Learning: A Survey

Guo, W., Wang, J., & Wang, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916887 · Cited by 535

A Survey of Zero-Shot Learning: Settings, Methods, and Applications

Wang, W., Zheng, V. W., Yu, H., & Miao, C. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3293318 · Cited by 528

Scene Graph Generation from Objects, Phrases and Region Captions

Li, Y., Ouyang, W., Zhou, B., Wang, K., & Wang, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.142 · Cited by 524

Found in Translation: Learning Robust Joint Representations by Cyclic Translations between Modalities

Pham, H., Liang, P. P., Manzini, T., Morency, L.-P., & Póczos, B. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016892 · Cited by 512

Multimodal intelligence: Representation learning, information fusion, and applications

Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511

Billion-scale semi-supervised learning for image classificationPreprint

Yalnız, İ. Z., Jeǵou, H., Chen, K., Paluri, M., & Mahajan, D. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00546 · Cited by 478

An Overview of Deep Semi-Supervised LearningPreprint

Ouali, Y., Hudelot, C., & Tami, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05278 · Cited by 471

VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and TextPreprint

Akbari, H., Yuan, L., Qian, R., Chuang, W.-H., Chang, S., Cui, Y., & Gong, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11178 · Cited by 469

Contrastive Multiview Coding

Tian, Y., Krishnan, D., & Isola, P. (2019) · arXiv · doi:10.48550/arxiv.1906.05849 · Cited by 465

Counterfactual VQA: A Cause-Effect Look at Language Bias

Niu, Y., Tang, K., Zhang, H., Lu, Z., Hua, X., & Wen, J.-R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01251 · Cited by 464

WebVision Database: Visual Learning and Understanding from Web DataPreprint

Li, W., Wang, L., Wei, L., Agustsson, E., & Gool, L. V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02862 · Cited by 462

Multimodal semi-supervised learning for image classification

Guillaumin, M., Verbeek, J. J., & Schmid, C. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540120 · Cited by 437

Untitled work

(2008) · Cited by 436

Domain Adaptation in Computer Vision Applications

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-58347-1 · Cited by 430

Multi-Task Learning with Deep Neural Networks: A Survey

Crawshaw, M. (2020) · arXiv · doi:10.48550/arxiv.2009.09796 · Cited by 428

12-in-1: Multi-Task Vision and Language Representation Learning

Lu, J., Goswami, V., Rohrbach, M., Parikh, D., & Lee, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01045 · Cited by 420

Zero-data learning of new tasks

Larochelle, H., Erhan, D., & Bengio, Y. (2008) · Cited by 406

Multimodal sentiment analysis using hierarchical fusion with context modeling

Majumder, N., Hazarika, D., Gelbukh, A. F., Cambria, E., & Poria, S. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.07.041 · Cited by 392

MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels

Jiang, L., Zhou, Z., Leung, T., Li, L.-J., & Fei-Fei, L. (2018) · International Conference on Machine Learning · Cited by 371

Mental health monitoring with multimodal sensing and machine learning: A survey

Garcia-Ceja, E., Riegler, M., Nordgreen, T., Jakobsen, P., Oedegaard, K. J., & Tørresen, J. (2018) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2018.09.003 · Cited by 356

MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · arXiv · doi:10.48550/arxiv.1606.06259 · Cited by 350

SMIL: Multimodal Learning with Severely Missing Modality

Ma, M., Ren, J., Zhao, L., Tulyakov, S., Wu, C., & Peng, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16330 · Cited by 332 · cited 7×

Learning Factorized Multimodal Representations

Tsai, Y.-H. H., Liang, P. P., Zadeh, A., Morency, L.-P., & Salakhutdinov, R. (2018) · arXiv · doi:10.48550/arxiv.1806.06176 · Cited by 323

Deep Learning Human Mind for Automated Visual Classification

Spampinato, C., Palazzo, S., Kavasidis, I., Giordano, D., Souly, N., & Shah, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.479 · Cited by 319

A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines

Charte, D., Charte, F., García, S., del Jesus, M. J., & Herrera, F. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.12.007 · Cited by 308

Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation

Xia, Y., Yang, D., Yu, Z., Liu, F., Cai, J., Yu, L., Zhu, Z., Xu, D., Yuille, A., & Roth, H. R. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101766 · Cited by 290

Untitled work

(2011) · Cited by 277

RUBi: Reducing Unimodal Biases in Visual Question AnsweringPreprint

Cadène, R., Dancette, C., Ben-younes, H., Cord, M., & Parikh, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10169 · Cited by 262

Missing Modalities Imputation via Cascaded Residual Autoencoder

Tran, L., Liu, X., Zhou, J., & Jin, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.528 · Cited by 256

Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis

Akhtar, M. S., Chauhan, D. S., Ghosal, D., Poria, S., Ekbal, A., & Bhattacharyya, P. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1034 · Cited by 238

Video-based emotion recognition in the wild using deep transfer learning and score fusion

Kaya, H., Gürpınar, F., & Salah, A. A. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.012 · Cited by 235 · cited 4×

A Shared Task on Multimodal Machine Translation and Crosslingual Image Description

Specia, L., Frank, S., Sima’an, K., & Elliott, D. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2346 · Cited by 228

Weakly-Shared Deep Transfer Networks for Heterogeneous-Domain Knowledge Propagation

Shu, X., Qi, G.-J., Tang, J., & Wang, J. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806216 · Cited by 223

Towards Automatic Learning of Procedures from Web Instructional VideosPreprint

Zhou, L., Xu, C., & Corso, J. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09788 · Cited by 216

A Simple Way to Make Neural Networks Robust Against Diverse Image Corruptions

Rusak, E., Schott, L., Zimmermann, R. S., Bitterwolf, J., Bringmann, O., Bethge, M., & Brendel, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_4 · Cited by 207

A Review on Explainability in Multimodal Deep Neural Nets

Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204

Modality Distillation with Multiple Stream Networks for Action Recognition

Garcia, N. C., Morerio, P., & Murino, V. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_7 · Cited by 201

Decoupling "when to update" from "how to update"

Malach, E., & Shalev‐Shwartz, S. (2017) · neural information processing systems · Cited by 201

MFAS: Multimodal Fusion Architecture Search

Perez-Rua, J.-M., Vielzeuf, V., Pateux, S., Baccouche, M., & Jurie, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00713 · Cited by 193

Learning Alignment for Multimodal Emotion Recognition from Speech

Xu, H., Zhang, H., Han, K., Wang, Y., Peng, Y., & Li, X. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3247 · Cited by 191

Review Study of Interpretation Methods for Future Interpretable Machine Learning

Mi, J., Li, A., & Zhou, L. (2020) · IEEE Access · doi:10.1109/access.2020.3032756 · Cited by 168

CentralNet: A Multilayer Approach for Multimodal Fusion

Vielzeuf, V., Lechervy, A., Pateux, S., & Jurie, F. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11024-6_44 · Cited by 166

GRAM: Graph-based Attention Model for Healthcare Representation LearningPreprint

Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07012 · Cited by 163

Self-training Improves Pre-training for Natural Language Understanding

Du, J., Grave, É., Gunel, B., Chaudhary, V., Çelebi, O., Auli, M., Stoyanov, V., & Conneau, A. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.426 · Cited by 159

Probing the Need for Visual Context in Multimodal Machine Translation

Çağlayan, O., Madhyastha, P., Specia, L., & Barrault, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1422 · Cited by 157

A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis

Delbrouck, J.-B., Tits, N., Brousmiche, M., & Dupont, S. (2020) · doi:10.18653/v1/2020.challengehml-1.1 · Cited by 139

Multiple Classifier Systems for the Classification of Audio-Visual Emotional States

Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S. J., Kächele, M., Schmidt, M., Heiko Neumann, B., Palm, G., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_47 · Cited by 135

Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

Narkhede, P., Walambe, R. A., Mandaokar, S., Chandel, P., Kotecha, K. V., & Ghinea, G. (2021) · Applied System Innovation · doi:10.3390/asi4010003 · Cited by 131

FairFace: Face Attribute Dataset for Balanced Race, Gender, and AgePreprint

Kärkkäinen, K., & Joo, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04913 · Cited by 129

Heterogeneous Domain Generalization Via Domain Mixup

Wang, Y., Li, H., & Kot, A. C. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053273 · Cited by 127

Contrastive Learning for Weakly Supervised Phrase Grounding

Gupta, T., Vahdat, A., Chechik, G., Yang, X., Kautz, J., & Hoiem, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_44 · Cited by 125

Affective Computing and Intelligent Interaction

D’Mello, S., Martin, J.-C., Schuller, B., & Graesser, A. C. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8 · Cited by 122

Decoupling "when to update" from "how to update"Preprint

Malach, E., & Shalev‐Shwartz, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02613 · Cited by 118

Multimodal Prototypical Networks for Few-shot Learning

Pahde, F., Puscas, M., Klein, T., & Nabi, M. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00269 · Cited by 116

Towards Unsupervised Image Captioning With Shared Multimodal Embeddings

Laina, I., Rupprecht, C., & Navab, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00751 · Cited by 111

Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces

Gómez, L., Patel, Y., Rusiñol, M., Karatzas, D., & Jawahar, C. V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.218 · Cited by 109

Multimodal Learning with Incomplete Modalities by Knowledge Distillation

Wang, Q., Zhan, L., Thompson, P. M., & Zhou, J. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403234 · Cited by 102

Deep Multimodal Transfer Learning for Cross-Modal Retrieval

Zhen, L., Hu, P., Peng, X., Goh, R. S. M., & Zhou, J. T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3029181 · Cited by 97

CMCGAN: A Uniform Framework for Cross-Modal Visual-Audio Mutual Generation

Hao, W., Zhang, Z., & Guan, H. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12329 · Cited by 81

A Comprehensive Introduction to Label Noise

Frénay‬, B., & Kabán, A. (2014) · Cited by 77

Cross-Subject and Cross-Modal Transfer for Generalized Abnormal Gait Pattern Recognition

Gu, X., Guo, Y., Deligianni, F., Lo, B. P. L., & Yang, G. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3009448 · Cited by 77

Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention

Duan, B., Tang, H., Wang, W., Zong, Z., Yang, G., & Yan, Y. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00406 · Cited by 72

Multimodal Disentangled Domain Adaption for Social Media Event Rumor Detection

Zhang, H., Qian, S., Fang, Q., & Xu, C. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3042055 · Cited by 71

Missing Modality Transfer Learning via Latent Low-Rank Constraint

Ding, Z., Shao, M., & Fu, Y. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2462023 · Cited by 68

Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models

Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.195 · Cited by 68

Label-Noise Robust Generative Adversarial Networks

Kaneko, T., Ushiku, Y., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00257 · Cited by 66

A Unified Framework for Multimodal Domain Adaptation

Qi, F., Yang, X., & Xu, C. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240633 · Cited by 65

Cross-Modal Data Programming Enables Rapid Medical Machine Learning

Dunnmon, J., Ratner, A., Saab, K. K., Khandwala, N., Markert, M. S., Sagreiya, H., Goldman, R. E., Lee‐Messer, C., Lungren, M. P., Rubin, D. L., et al. (2020) · Patterns · doi:10.1016/j.patter.2020.100019 · Cited by 64

Clustering on Multiple Incomplete Datasets via Collective Kernel Learning

Shao, W., Shi, X., & Yu, P. S. (2013) · Proceedings · doi:10.1109/icdm.2013.117 · Cited by 63

Deep multisensor learning for missing-modality all-weather mapping

Zheng, Z., Ma, A., Zhang, L., & Zhong, Y. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.12.009 · Cited by 63

MasakhaNER: Named entity recognition for African languages

(2021) · HAL · doi:10.1162/tacl · Cited by 61

Deep Manifold-to-Manifold Transforming Network for Skeleton-based Action Recognition

Zhang, T., Zheng, W., Cui, Z., Zong, Y., Li, C., Zhou, X., & Yang, J. J. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2966878 · Cited by 59

Multimodal Encoder-Decoder Attention Networks for Visual Question Answering

Chen, C., Han, D., & Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.2975093 · Cited by 59

A cross-disciplinary comparison of multimodal data fusion approaches and applications: Accelerating learning through trans-disciplinary information sharing

Bokade, R., Navato, A. P., Ouyang, R., Jin, X., Chou, C.-A., Ostadabbas, S., & Mueller, A. V. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113885 · Cited by 59

CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation LearningPreprint

Peng, Y., Qi, J., & Yuan, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.05106 · Cited by 58

Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning

Rajendran, J., Khapra, M. M., Chandar, S., & Ravindran, B. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1021 · Cited by 57

Does my multimodal model learn cross-modal interactions? It’s harder to tell than you might think!

Hessel, J., & Lee, L. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.62 · Cited by 57

Multi-Task Learning of Hierarchical Vision-Language Representation

Nguyen, D.-K., & Okatani, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01074 · Cited by 54

Coordinated Joint Multimodal Embeddings for Generalized Audio-Visual Zero-shot Classification and Retrieval of Videos

Parida, K. K., Matiyali, N., Guha, T., & Sharma, G. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093438 · Cited by 49

Multiple Instance Learning

Herrera, F., Ventura, S., Bello, R., Chris Cornelis, A., Zafra, A., Sánchez-Tarragó, D., & Vluymans, S. (2016) · Springer eBooks · doi:10.1007/978-3-319-47759-6_2 · Cited by 42

Cross-Dataset Adaptation for Visual Question Answering

Sha, F., Hu, H., & Chao, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00599 · Cited by 41

Complex Object Classification

Yang, Y., Wu, Y.-F., Zhan, D., Liu, Z., & Jiang, Y. Y. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220012 · Cited by 41

Self-training Improves Pre-training for Natural Language UnderstandingPreprint

Du, J., Grave, É., Gunel, B., Chaudhary, V., Çelebi, O., Auli, M., Stoyanov, V., & Conneau, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02194 · Cited by 41

Large Scale Audiovisual Learning of Sounds with Weakly Labeled Data

Fayek, H. M., & Kumar, A. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/78 · Cited by 40

Foundations of Multimodal Co-learning

Zadeh, A., Liang, P. P., & Morency, L.-P. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.001 · Cited by 40

WordNet: An electronic lexical database. Christiane Fellbaum (Ed.). Cambridge, MA: MIT Press, 1998. Pp. 423.

Oram, P. (2001) · Applied Psycholinguistics · doi:10.1017/s0142716401221079 · Cited by 38

Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between ModalitiesPreprint

Pham, H., Liang, P. P., Manzini, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07809 · Cited by 38

Robust Student Network Learning

Guo, T., Xu, C., He, S., Shi, B., Xu, C., & Tao, D. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2929114 · Cited by 38

HMTL: Heterogeneous Modality Transfer Learning for Audio-Visual Sentiment Analysis

Seo, S., Na, S., & Kim, J. (2020) · IEEE Access · doi:10.1109/access.2020.3006563 · Cited by 38

Audio–visual domain adaptation using conditional semi-supervised Generative Adversarial Networks

Athanasiadis, C., Hortal, E., & Asteriadis, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.09.106 · Cited by 37

MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding

Wang, Q., Tan, H. X., Shen, S., Mahoney, M. W., & Yao, Z. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.159 · Cited by 37

Multiple Instance Learning: Foundations and Algorithms

Herrera, F., Ventura, S., Bello, R., Chris Cornelis, A., Zafra, A., Snchez-Tarrag, D., & Vluymans, S. (2016) · Springer eBooks · doi:10.1007/978-3-319-47759-6 · Cited by 36

A Closer Look at the Robustness of Vision-and-Language Pre-trained ModelsPreprint

Li, L., Gan, Z., & Liu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08673 · Cited by 31

Multimodal Human Computer Interaction and Pervasive Services

Grifoni, P. (2009) · IGI Global eBooks · doi:10.4018/978-1-60566-386-9 · Cited by 30

Multimodal Transfer Deep Learning with Applications in Audio-Visual RecognitionPreprint

Moon, S., Kim, S., & Wang, H. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.3121 · Cited by 29

Deep Co-Attention Network for Multi-View Subspace Learning

Zheng, L., Cheng, Y., Yang, H., Cao, N., & He, J. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449801 · Cited by 29

Multimodal deep generative adversarial models for scalable doubly semi-supervised learning

Du, C., Du, C., & He, H. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.11.003 · Cited by 27

Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection

Mathur, L., & Matarić, M. J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413550 · Cited by 27

Modularized Textual Grounding for Counterfactual Resilience

Fang, Z., Kong, S., Fowlkes, C. C., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00654 · Cited by 26

Associative Multichannel Autoencoder for Multimodal Word Representation

Wang, S., Zhang, J., & Zong, C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1011 · Cited by 25

Contrastive Visual-Linguistic PretrainingPreprint

Shi, L., Shuang, K., Geng, S., Su, P., Jiang, Z., Gao, P., Fu, Z., de Melo, G., & Su, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13135 · Cited by 25

A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks

Lin, A. S., Rao, S., Çelikyılmaz, A., Nouri, E., Brockett, C., Dey, D., & Dolan, B. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.440 · Cited by 24

Counterfactual VQA: A Cause-Effect Look at Language BiasPreprint

Niu, Y., Tang, K., Zhang, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04315 · Cited by 23

Enriched music representations with multiple cross-modal contrastive learning

Ferraro, A., Favory, X., Drossos, K., Kim, Y., & Bogdanov, D. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3071082 · Cited by 23

Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

Liang, P. P., Wu, P., Ziyin, L., Morency, L.-P., & Salakhutdinov, R. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475247 · Cited by 22

Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning

Yan, L., Liu, D., Song, Y., & Yu, C. B. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341398 · Cited by 22

ViCo: Word Embeddings From Visual Co-Occurrences

Gupta, T., Schwing, A. G., & Hoiem, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00752 · Cited by 21

Relating by Contrasting: A Data-efficient Framework for Multimodal Generative ModelsPreprint

Shi, Y., Paige, B., Torr, P. H. S., & Siddharth, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01179 · Cited by 21

Intelligence Science and Big Data Engineering

Sun, C., Fang, F., Zhou, Z.-H., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-42057-3 · Cited by 21

Probing Multimodal Embeddings for Linguistic Properties: the Visual-Semantic Case

Lindström, A. D., Björklund, J., Bensch, S., & Drewes, F. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.64 · Cited by 21

Multimodal learning with transformers: A survey

Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072

Multi-fault diagnosis of Industrial Rotating Machines using Data-driven approach : A review of two decades of research

Gawde, S., Patil, S., Kumar, S., Kamat, P. V., Kotecha, K. V., & Abraham, A. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106139 · Cited by 327

Domain Adaptation: Challenges, Methods, Datasets, and Applications

Singhal, P., Walambe, R. A., Ramanna, S., & Kotecha, K. V. (2023) · IEEE Access · doi:10.1109/access.2023.3237025 · Cited by 276

A survey of multimodal hybrid deep learning for computer vision: Architectures, applications, trends, and challenges

Bayoudh, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102217 · Cited by 180

Review of the Li-Ion Battery, Thermal Management, and AI-Based Battery Management System for EV Application

Ghalkhani, M., & Habibi, S. R. (2022) · Energies · doi:10.3390/en16010185 · Cited by 171

Multimodal Learning and its Applications

Haddag, A. (2023) · doi:10.13140/rg.2.2.33217.57440

Tutorial on Multimodal Machine Learning: Principles, Challenges, and Open Questions

Liang, P. P., & Morency, L. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3610661.3617602 · Cited by 48

Deep Multimodal Learning with Missing Modality: A SurveyPreprint

Wu, R., Hu, W., Chen, H., & Carneiro, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.07825 · Cited by 182

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 86

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

Barua, A., Ahmed, M. U., & Begum, S. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7736385