MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Open accessAbstract
Multimodal Sentiment Analysis is an active area of research that leverages multimodal signals for affective understanding of user-generated videos. The predominant approach, addressing this task, has been to develop sophisticated fusion techniques. However, the heterogeneous nature of the signals creates distributional modality gaps that pose significant challenges. In this paper, we aim to learn effective modality representations to aid the process of fusion. We propose a novel framework, MISA, which projects each modality to two distinct subspaces. The first subspace is modality-invariant, where the representations across modalities learn their commonalities and reduce the modality gap. The second subspace is modality-specific, which is private to each modality and captures their characteristic features. These representations provide a holistic view of the multimodal data, which is used for fusion that leads to task predictions. Our experiments on popular sentiment analysis benchmarks, MOSI and MOSEI, demonstrate significant gains over state-of-the-art models. We also consider the task of Multimodal Humor Detection and experiment on the recently proposed UR_FUNNY dataset. Here too, our model fares better than strong baselines, establishing MISA as a useful multimodal framework.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
On CMU-MOSEI, C-MIB yields 0.7% improvement on Acc2 and 1.2% improvement on F1 score compared to MISA, and improves Acc7 and Corr compared to state-of-the-art methods.
“On CMU-MOSEI dataset, our proposed C-MIB yields 0.7% improvement on Acc2 and 1.2% improvement on F1 score compared to the current state-of-the-art MISA [53]. The performance on Acc7 and Corr is also improved compared to the state-of-the-art methods.”
From Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations (2023) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Multimodal Transformer for Unaligned Multimodal Language Sequences
Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1656 · Cited by 2,259 · see citing works
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102 · see citing works
Self-citationEkman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,975 · see citing works
OpenFace: An open source facial behavior analysis toolkit
Baltrušaitis, T., Robinson, P., & Morency, L. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477553 · Cited by 1,771 · see citing works
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644 · see citing works
Self-citationMultimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582 · see citing works
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433 · see citing works
Self-citationSimultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,203 · see citing works
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · see citing works
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170 · see citing works
From Utterance to Text: The Bias of Language in Speech and Writing
Olson, D. (1977) · Harvard Educational Review · doi:10.17763/haer.47.3.8840364413869005 · Cited by 1,065 · see citing works
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950 · see citing works
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939 · see citing works
Self-citationMemory Fusion Network for Multi-view Sequential Learning
Zadeh, A., Liang, P. P., Mazumder, N., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12021 · Cited by 855 · see citing works
Self-citationMultimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages
Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.94 · Cited by 852 · see citing works
COVAREP — A collaborative voice analysis repository for speech technologies
Degottex, G., Kane, J., Drugman, T., Raitio, T., & Scherer, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853739 · Cited by 695 · see citing works
Adversarial Multi-task Learning for Text Classification
Liu, P., Qiu, X., & Huang, X. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1001 · Cited by 620 · see citing works
Poria, S., Cambria, E., & Gelbukh, A. F. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1303 · Cited by 553 · see citing works
Self-citationDeep Multimodal Representation Learning: A Survey
Guo, W., Wang, J., & Wang, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916887 · Cited by 535 · see citing works
Multi-attention Recurrent Network for Human Communication Comprehension
Zadeh, A., Liang, P. P., Poria, S., Vij, P., Cambria, E., & Morency, L. (2018) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v32i1.12024 · Cited by 535 · see citing works
Self-citationPham, H., Liang, P. P., Manzini, T., Morency, L.-P., & Póczos, B. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016892 · Cited by 512 · see citing works
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Wang, Y., Shen, Y., Liu, Z., Liang, P. P., Zadeh, A., & Morency, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017216 · Cited by 493 · see citing works
Sun, Z., Sarma, P., Sethares, W., & Liang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6431 · Cited by 410 · see citing works
Multimodal sentiment analysis using hierarchical fusion with context modeling
Majumder, N., Hazarika, D., Gelbukh, A. F., Cambria, E., & Poria, S. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.07.041 · Cited by 385 · see citing works
Self-citationBeneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Poria, S., Hazarika, D., Majumder, N., & Mihalcea, R. F. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3038167 · Cited by 344 · see citing works
Self-citationMultimodal sentiment analysis with word-level fusion and reinforcement learning
Chen, M., Wang, S., Liang, P. P., Baltrušaitis, T., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136801 · Cited by 334 · see citing works
Peng, Y., & Qi, J. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3284750 · Cited by 290 · see citing works
Mai, S., Hu, H., & Xing, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5347 · Cited by 237 · see citing works
Detection of Glottal Closure Instants From Speech Signals: A Quantitative Review
Thomas Drugman, A., Thomas, M. R., Guðnason, J., Naylor, P. A., & Dutoit, T. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2170835 · Cited by 236 · see citing works
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis
Akhtar, M. S., Chauhan, D. S., Ghosal, D., Poria, S., Ekbal, A., & Bhattacharyya, P. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1034 · Cited by 233 · see citing works
Self-citationMultimodal Language Analysis with Recurrent Multistage Fusion
Liang, P. P., Liu, Z., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1014 · Cited by 230 · see citing works
(1998) · doi:10.1093/oso/9780195104462.001.0001 · Cited by 223 · see citing works
Multi-level Multiple Attentions for Contextual Multimodal Sentiment Analysis
Poria, S., Cambria, E., Hazarika, D., Mazumder, N., Zadeh, A., & Morency, L. (2017) · Proceedings · doi:10.1109/icdm.2017.134 · Cited by 215 · see citing works
Self-citationContextual Inter-modal Attention for Multi-modal Sentiment Analysis
Ghosal, D., Akhtar, M. S., Chauhan, D. S., Poria, S., Ekbal, A., & Bhattacharyya, P. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1382 · Cited by 196 · see citing works
Self-citationUnsupervised Domain Clusters in Pretrained Language Models
Aharoni, R., & Goldberg, Y. (2020) · arXiv · doi:10.48550/arxiv.2004.02105 · Cited by 187
Unsupervised Domain Clusters in Pretrained Language Models
Aharoni, R., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.692 · Cited by 181 · see citing works
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor
Hasan, M. K., Rahman, W., Bagher Zadeh, A., Zhong, J., Tanveer, M. I., Morency, L.-P., & Hoque, M. (. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1211 · Cited by 174 · see citing works
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift
Ruder, S., & Plank, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1096 · Cited by 169 · see citing works
Mai, S., Hu, H., & Xing, S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1046 · Cited by 153 · see citing works
Extending Long Short-Term Memory for Multi-View Structured Learning
Rajagopalan, S. S., Morency, L., Baltrušaitis, T., & Goecke, R. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_21 · Cited by 138 · see citing works
Context-aware Interactive Attention for Multi-modal Sentiment and Emotion Analysis
Chauhan, D. S., Akhtar, M. S., Ekbal, A., & Bhattacharyya, P. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1566 · Cited by 97 · see citing works
Deep Variational Canonical Correlation Analysis
Wang, W., Yan, X., Lee, H., & Livescu, K. (2016) · arXiv · doi:10.48550/arxiv.1610.03454 · Cited by 89 · see citing works
Mai, S., Xing, S., & Hu, H. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2925966 · Cited by 89 · see citing works
Cross-Domain Sentiment Classification with Target Domain Specific Information
Peng, M., Zhang, Q., Jiang, Y., & Huang, X. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1233 · Cited by 88 · see citing works
Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks
Hu, G., Yang, H., Yuan, Y., Zhang, Z., Lu, Z., Mukherjee, S. S., Hospedales, T. M., Robertson, N. M., & Yang, Y. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.404 · Cited by 87 · see citing works
Multimodal sentiment analysis based on multi-head attention mechanism
Chen, X., Lu, G., & Yan, J. (2020) · International Conference on Machine Learning and Soft Computing (ICMLSC) · doi:10.1145/3380688.3380693 · Cited by 78 · see citing works
Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Pham, H., Manzini, T., Liang, P. P., & Póczos, B. (2018) · doi:10.18653/v1/w18-3308 · Cited by 76 · see citing works
Human Conversation Analysis Using Attentive Multimodal Networks with Hierarchical Encoder-Decoder
Gu, Y., Li, X., Huang, K., Fu, S., Yang, K., Chen, S., Zhou, M., & Marsic, I. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240714 · Cited by 39 · see citing works
Sun, Z., Sarma, P. K., Sethares, W. A., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05544 · Cited by 26 · see citing works
Image-Text Multi-Modal Representation Learning by Adversarial BackpropagationPreprint
Park, G., & Im, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08354 · Cited by 24 · see citing works
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Poria, S., Hazarika, D., Majumder, N., & Mihalcea, R. (2020) · arXiv · doi:10.48550/arxiv.2005.00357
Self-citation
Flagged first, then by citations
Research works printed but not linked: 13
- Galen Andrew, Raman Arora, Jeff A. Bilmes, and Karen Livescu. 2013. Deep Canonical Correlation Analysis. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013 (JMLR Workshop and Conference Proceedings, Vol. 28). JMLR.org, Atlanta, GA, USA, 1247-1255. http://proceedings.mlr.press/ v28/andrew13.html
- Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain Separation Networks. In Advances in Neural Information Processing Systems 29. Curran Associates, Inc., Barcelona, Spain, 343-351. http://papers.nips.cc/paper/6254-domain-separation-networks
- Randall Davis. 2012. Multi-View Latent Variable Discriminative Models for Action Recognition. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (CVPR '12). IEEE Computer Society, USA, 2120-2127.
- Thomas Drugman and Abeer Alwan. 2011. Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics. In INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association. ISCA, Florence, Italy, 1973-1976. http://www.isca-speech.org/archive/interspeech_ 2011/i11_1973.html
- Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, and Davide Testuggine. 2019. Su- pervised Multimodal Bitransformers for Classifying Images and Text. In Visually Grounded Interaction and Language (ViGIL), NeurIPS 2019 Workshop.
- Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. ViL- BERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks. In Advances in Neural Information Process- ing Systems 32.
- Curran Associates, Inc., Vancouver, BC, Canada, 13- 23. http://papers.nips.cc/paper/8297-vilbert-pretraining-task-agnostic- visiolinguistic-representations-for-vision-and-language-tasks
- Rada Mihalcea. 2012. Multimodal Sentiment Analysis. In Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis (Jeju, Republic of Korea) (WASSA '12). Association for Computational Linguistics, USA, 1.
- Mathieu Salzmann, Carl Henrik Ek, Raquel Urtasun, and Trevor Darrell. 2010. Factorized Orthogonal Latent Spaces. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010 (JMLR Proceedings, Vol. 9). JMLR.org, Sardinia, Italy, 701-708. http://proceedings.mlr.press/v9/ salzmann10a.html
- Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Learning Factorized Multimodal Representations. In 7th International Conference on Learning Representations, ICLR 2019. OpenRe- view.net, New Orleans, LA, USA. https://openreview.net/forum?id=rygqqsA9KX
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30.
- Curran Asso- ciates, Inc., Long Beach, CA, USA, 5998-6008. http://papers.nips.cc/paper/7181- attention-is-all-you-need
- Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz. 2017. Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning. In 5th International Conference on Learning Representations, ICLR 2017. OpenReview.net, Toulon, France. https: //openreview.net/forum?id=SkB-_mcel
Cited by
Sorted by citations
Yu, W., Xu, H., Yuan, Z., & Wu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17289 · Cited by 816 · see citing works
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Poria, S., Hazarika, D., Majumder, N., & Mihalcea, R. F. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3038167 · Cited by 344 · see citing works
Self-citationHybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis
Mai, S., Zeng, Y., Zheng, S., & Hu, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3172360 · Cited by 301 · see citing works
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition
Hu, G., Lin, T.-E., Zhao, Y., Lu, G., Wu, Y., & Li, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.534 · Cited by 304 · see citing works
Sun, L., Lian, Z., Liu, B., & Tao, J. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3274829 · Cited by 283 · see citing works
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis
Han, W., Chen, H., Gelbukh, A. F., Zadeh, A., Morency, L., & Poria, S. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479919 · Cited by 261 · see citing works
Self-citationA Unified Multi-Task Semantic Communication System for Multimodal Data
Zhang, G., Hu, Q., Qin, Z., Cai, Y., Yu, G., & Tao, X. (2024) · IEEE Transactions on Communications · doi:10.1109/tcomm.2024.3364990 · Cited by 233 · see citing works
Decoupled Multimodal Distilling for Emotion Recognition
Li, Y., Wang, Y., & Cui, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00641 · Cited by 249 · see citing works
Transformer-based Feature Reconstruction Network for Robust Multimodal Sentiment Analysis
Yuan, Z., Li, W., Xu, H., & Yu, W. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475585 · Cited by 211 · see citing works
Exploiting BERT for Multimodal Target Sentiment Classification through Input Space Translation
Khan, Z., & Fu, Y. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475692 · Cited by 203 · see citing works
Cross-Modal Enhancement Network for Multimodal Sentiment Analysis
Wang, D., Liu, S., Wang, Q., Tian, Y., He, L., & Gao, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3183830 · Cited by 191 · see citing works
State of the art: a review of sentiment analysis based on sequential transfer learning
Chan, J. Y.-L., Bea, K. T., Leow, S. M. H., Phoong, S. W., & Cheng, W. K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10183-8 · Cited by 187 · see citing works
CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation
Sun, H., Wang, H., Liu, J., Chen, Y., & Lin, L. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548025 · Cited by 176 · see citing works
Mai, S., Zeng, Y., & Hu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3171679 · Cited by 182 · see citing works
Latent Structure Mining With Contrastive Modality Fusion for Multimedia Recommendation
Zhang, J., Zhu, Y., Liu, Q., Zhang, M., Wu, S., & Wang, L. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3221949 · Cited by 170 · see citing works
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation
Lian, Z., Chen, L., Sun, L., Liu, B., & Tao, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3234553 · Cited by 165 · see citing works
TMBL: Transformer-based multimodal binding learning model for multimodal sentiment analysis
Huang, J., Zhou, J. W., Tang, Z., Lin, J., & Chen, C. Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111346 · Cited by 162 · see citing works
Li, J., Wang, X., Lv, G., & Zeng, Z. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3260635 · Cited by 146 · see citing works
Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks
Yang, X., Feng, S., Zhang, Y., & Wang, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.28 · Cited by 145 · see citing works
Video sentiment analysis with bimodal information-augmented multi-head attention
Wu, T., Peng, J., Zhang, W., Zhang, H., Tan, S., Yi, F., Ma, C., & Huang, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107676 · Cited by 137 · see citing works
RAFT: Robust Adversarial Fusion Transformer for multimodal sentiment analysis
Wang, R., Xu, D., Cascone, L., Wang, Y.-Y., Chen, H., Zheng, J., & Zhu, X.-X. (2025) · Array · doi:10.1016/j.array.2025.100445 · Cited by 128 · see citing works
Multimodal research in vision and language: A review of current and emerging trends
Uppal, S., Bhagat, S., Hazarika, D., Majumder, N., Poria, S., Zimmermann, R., & Zadeh, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.009 · Cited by 112 · see citing works
Self-citationDynamic interactive multiview memory network for emotion recognition in conversation
Wen, J., Jiang, D., Tu, G., Liu, C., & Cambria, E. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.009 · Cited by 102 · see citing works
AF: An Association-Based Fusion Method for Multi-Modal Classification
Liang, X., Qian, Y., Guo, Q. J., Cheng, H., & Liang, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125995 · Cited by 98 · see citing works
Multimodal sentiment analysis: A survey
Lai, S., Hu, X., Xu, H., Ren, Z., & Liu, Z. (2023) · Displays · doi:10.1016/j.displa.2023.102563 · Cited by 106 · see citing works
Li, B., Fei, H., Liao, L., Zhao, Y., Teng, C., Chua, T., Ji, D., & Li, F. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612053 · Cited by 95 · see citing works
Le, H.-D., Lee, G.-S., Kim, S.-H., Kim, S., & Yang, H.-J. (2023) · IEEE Access · doi:10.1109/access.2023.3244390 · Cited by 97 · see citing works
Multi-modal emotion recognition using tensor decomposition fusion and self-supervised multi-tasking
Wang, R., Zhu, J., Wang, S., Wang, T., Huang, J., & Zhu, X. (2024) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-024-00347-3 · Cited by 95 · see citing works
Liu, R., Zuo, H., Lian, Z., Schuller, B. W., & Li, H. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3378570 · Cited by 96 · see citing works
Gkoumas, D., Li, Q., Lioma, C., Yu, Y., & Song, D. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.09.005 · Cited by 94 · see citing works
Chen, F., Shao, J., Zhu, S., & Shen, H. T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01036 · Cited by 91 · see citing works
Multimodal transformer with adaptive modality weighting for multimodal sentiment analysis
Wang, Y., He, J., Wang, D., Wang, Q., Wan, B., & Luo, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127181 · Cited by 91 · see citing works
SMIN: Semi-Supervised Multi-Modal Interaction Network for Conversational Emotion Recognition
Lian, Z., Liu, B., & Tao, J. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3141237 · Cited by 89 · see citing works
BAFN: Bi-Direction Attention Based Fusion Network for Multimodal Sentiment Analysis
Tang, J., Liu, D., Jin, X., Peng, Y. K., Zhao, Q., Ding, Y., & Kong, W. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3218018 · Cited by 90 · see citing works
TeFNA: Text-centered fusion network with crossmodal attention for multimodal sentiment analysis
Huang, C., Zhang, J., Wu, X., Wang, Y., Li, M., & Huang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110502 · Cited by 89 · see citing works
Wang, J. Z., Zhao, S., Wu, C., Adams, R. B., Newman, M. G., Shafir, T., & Tsachor, R. P. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3273517 · Cited by 98 · see citing works
Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities
Zeng, J., Liu, T., & Zhou, J. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3532064 · Cited by 94 · see citing works
MissModal: Increasing Robustness to Missing Modality in Multimodal Sentiment Analysis
Lin, R., & Hu, H. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00628 · Cited by 91 · see citing works
CFN-ESA: A Cross-Modal Fusion Network With Emotion-Shift Awareness for Dialogue Emotion Recognition
Li, J., Wang, X., Liu, Y., & Zeng, Z. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3389453 · Cited by 83 · see citing works
AoM: Detecting Aspect-oriented Information for Multimodal Aspect-Based Sentiment Analysis
Zhou, R., Guo, W., Liu, X., Yu, S., Zhang, Y., & Yuan, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.519 · Cited by 90 · see citing works
Brain-Machine Coupled Learning Method for Facial Emotion Recognition
Liu, D., Dai, W., Zhang, H., Jin, X., Cao, J., & Kong, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3257846 · Cited by 80 · see citing works
Disentanglement Translation Network for multimodal sentiment analysis
Zeng, Y., Yan, W., Mai, S., & Hu, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102031 · Cited by 84 · see citing works
GPT-4V with emotion: A zero-shot benchmark for Generalized Emotion Recognition
Lian, Z., Sun, L., Sun, H., Chen, K., Wen, Z., Gu, H., Liu, B., & Tao, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102367 · Cited by 82 · see citing works
Contrastive Multimodal Fusion with TupleInfoNCE
Liu, Y., Fan, Q., Zhang, S., Dong, H., Funkhouser, T. A., & Yi, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00079 · Cited by 77 · see citing works
Multimodal Learning using Optimal Transport for Sarcasm and Humor Detection
Pramanick, S., Roy, A., & Johns, V. M. P. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00062 · Cited by 78 · see citing works
Sun, H., Chen, Y., & Lin, L. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3233070 · Cited by 77 · see citing works
Zuo, H., Liu, R., Zhao, J., Gao, G., & Li, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095836 · Cited by 75 · see citing works
Fan, C., Zhu, K., Tao, J., Yi, G., Xue, J., & Lv, Z. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3423671 · Cited by 79 · see citing works
Zeng, Y., Li, Z., Tang, Z., Chen, Z., & Ma, H. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119240 · Cited by 74 · see citing works
An effective multimodal representation and fusion method for multimodal intent recognition
Huang, X., Ma, T., Jia, L., Zhang, Y., Rong, H., & Al-Nabhan, N. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126373 · Cited by 72 · see citing works
Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimoda Emotion Recognition
Li, D., Wang, Y., Funakoshi, K., & Okumura, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.996 · Cited by 84 · see citing works
Multi-Task Momentum Distillation for Multimodal Sentiment Analysis
Lin, R., & Hu, H. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3282410 · Cited by 71 · see citing works
Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis
Sun, T. F., Wang, W., Jing, L., Cui, Y., Song, X., & Nie, L. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548211 · Cited by 70 · see citing works
MIntRec: A New Dataset for Multimodal Intent Recognition
Zhang, H., Xu, H., Wang, X. Y., Zhou, Q., Zhao, S., & Teng, J. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547906 · Cited by 70 · see citing works
Tellamekala, M. K., Amiriparian, S., Schuller, B. W., André, E., Giesbrecht, T., & Valstar, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3325770 · Cited by 75 · see citing works
AtCAF: Attention-based causality-aware fusion network for multimodal sentiment analysis
Huang, C., Chen, J., Huang, Q., Wang, S., Tu, Y., & Huang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102725 · Cited by 70 · see citing works
Lu, Q., Sun, X., Gao, Z., Long, Y., Feng, J., & Zhang, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103538 · Cited by 68 · see citing works
Multimodal sentiment analysis based on multi-layer feature fusion and multi-task learning
Cai, Y., Li, X., Zhang, Y., Li, J., Zhu, F., & Rao, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-85859-6 · Cited by 68 · see citing works
Multimodal Phased Transformer for Sentiment Analysis
Cheng, J., Fostiropoulos, I., BOEHM, B. W., & Soleymani, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.189 · Cited by 66 · see citing works
Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition
Guo, Z., Jin, T., & Zhao, Z. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.94 · Cited by 77 · see citing works
Cheng, H., Yang, Z., Zhang, X., & Yang, Y. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3265653 · Cited by 65 · see citing works
Li, M., Yang, D., Zhao, X. X., Wang, S., Wang, Y., Yang, K., Sun, M., Kou, D., Qian, Z., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01184 · Cited by 64 · see citing works
Dynamically Adjust Word Representations Using Unaligned Multimodal Information
Guo, J., Tang, J., Dai, W., Ding, Y., & Kong, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548137 · Cited by 63 · see citing works
Survey on multimodal approaches to emotion recognition
Aruna Gladys, A., & Vetriselvi, V. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126693 · Cited by 62 · see citing works
Contrastive-Based Removal of Negative Information in Multimodal Emotion Analysis
Wang, R., Wang, Y.-Y., Cambria, E., Fan, X.-H., Yu, X.-H., Huang, Y., Xiaosong, E., & Zhu, X.-X. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10463-9 · Cited by 70 · see citing works
MEDT: Using Multimodal Encoding-Decoding Network as in Transformer for Multimodal Sentiment Analysis
Qi, Q., Lin, L., Zhang, R., & Xue, C. (2022) · IEEE Access · doi:10.1109/access.2022.3157712 · Cited by 63 · see citing works
Learning to Learn Better Unimodal Representations via Adaptive Multimodal Meta-Learning
Ya, S., Mai, S., & Hu, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3178231 · Cited by 59 · see citing works
Chen, Q., Huang, G., & Wang, Y. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3192728 · Cited by 58 · see citing works
Semantic Alignment Network for Multi-Modal Emotion Recognition
Hou, M.-X., Zhang, Z., Liu, C., & Lu, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3247822 · Cited by 58 · see citing works
Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Space
Xie, Z., Yang, Y. Z., Wang, J., Liu, X., & Li, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3376564 · Cited by 63 · see citing works
Excavating multimodal correlation for representation learning
Mai, S., Ya, S., Zeng, Y., & Hu, H. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.003 · Cited by 57 · see citing works
Large language models meet text-centric multimodal sentiment analysis: a survey
Yang, H., Zhao, Y., Wu, Y., Wang, S., Zheng, T., Zhang, H., Ma, Z., Che, W., Wang, S., Wei, S., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4593-8 · Cited by 88 · see citing works
Lin, R., & Hu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3303711 · Cited by 56 · see citing works
Multimodal sentiment analysis with unimodal label generation and modality decomposition
Zhu, L., Zhao, H., Zhu, Z., Zhang, C., & Kong, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102787 · Cited by 55 · see citing works
Modality-invariant temporal representation learning for multimodal sentiment classification
Sun, H., Liu, J., Chen, Y., & Lin, L. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.031 · Cited by 57 · see citing works
TCHFN: Multimodal sentiment analysis based on Text-Centric Hierarchical Fusion Network
Hou, J., Omar, N., Tiun, S. B., Saad, S., & He, Q. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112220 · Cited by 54 · see citing works
Learning fine-grained representation with token-level alignment for multimodal sentiment analysis
Li, X., Zhang, H., Dong, Z., Cheng, X., Liu, Y., & Zhang, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126274 · Cited by 55 · see citing works
Huan, R., Zhong, G., Chen, P., & Liang, R. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3338769 · Cited by 53 · see citing works
A comprehensive survey on deep learning-based approaches for multimodal sentiment analysis
Ghorbanali, A., & Sohrabi, M. K. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10555-8 · Cited by 51 · see citing works
Shared-Private Memory Networks For Multimodal Sentiment Analysis
Zhao, X., Chen, Y., Liu, S., & Tang, B. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3222023 · Cited by 50 · see citing works
Understanding and Constructing Latent Modality Structures in Multi-Modal Representation Learning
Jiang, Q., Chen, C., Zhao, H., Chen, L., Ping, Q., Tran, S. D., Xu, Y., Zeng, B., & Chilimbi, T. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00740 · Cited by 53 · see citing works
Dual-Perspective Fusion Network for Aspect-Based Multimodal Sentiment Analysis
Wang, D., Tian, C., Xiao, L., Zhao, L., He, L., & Wang, Q. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3321435 · Cited by 50 · see citing works
Modeling Hierarchical Uncertainty for Multimodal Emotion Recognition in Conversation
Chen, F., Shao, J., Zhu, A., Ouyang, D., Liu, X., & Shen, H. T. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3185119 · Cited by 49 · see citing works
Li, Z., Guo, Q., Pan, Y., Ding, W., Yu, J., Zhang, Y., Liu, W., Chen, H., Wang, H., & Xie, Y. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101891 · Cited by 49 · see citing works
Yang, D., Liu, Y., Huang, C., Li, M., Zhao, X. X., Wang, Y., Yang, K., Wang, Y., Zhai, P., & Zhang, L. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110370 · Cited by 48 · see citing works
MF-BERT: Multimodal Fusion in Pre-Trained BERT for Sentiment Analysis
He, J., & Hu, H. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3139856 · Cited by 47 · see citing works
MIA-Net: Multi-Modal Interactive Attention Network for Multi-Modal Affective Analysis
Li, S., Zhang, T., Chen, B., & Chen, C. L. P. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3259010 · Cited by 47 · see citing works
MALN: Multimodal Adversarial Learning Network for Conversational Emotion Recognition
Ren, M., Huang, X., Liu, J., Liu, M., Li, X., & Liu, A.-A. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3273577 · Cited by 47 · see citing works
FMSA-SC: A Fine-Grained Multimodal Sentiment Analysis Dataset Based on Stock Comment Videos
Song, L., Chen, S., Meng, Z., Sun, M., & Shang, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3363641 · Cited by 48 · see citing works
Video-Based Cross-Modal Auxiliary Network for Multimodal Sentiment Analysis
Chen, R., Zhou, W., Li, Y., & Zhou, H. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3197420 · Cited by 46 · see citing works
Robust Multimodal Sentiment Analysis via Tag Encoding of Uncertain Missing Modalities
Zeng, J., Zhou, J., & Liu, T. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3207572 · Cited by 49 · see citing works
Unified Multi-modal Pre-training for Few-shot Sentiment Analysis with Prompt-based Learning
Yu, Y., Zhang, D., & Li, S. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548306 · Cited by 46 · see citing works
Hybrid cross-modal interaction learning for multimodal sentiment analysis
Fu, Y., Zhang, Z., Yang, R., & Yao, C. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127201 · Cited by 46 · see citing works
A fine-grained modal label-based multi-stage network for multimodal sentiment analysis
Peng, J., Wu, T., Zhang, W., Cheng, F., Tan, S., Yi, F., & Huang, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119721 · Cited by 45 · see citing works
AIA-Net: Adaptive Interactive Attention Network for Text–Audio Emotion Recognition
Zhang, T., Li, S., Chen, B., Yuan, H., & Chen, C. L. P. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3195739 · Cited by 44 · see citing works
Multimodal Mutual Attention-Based Sentiment Analysis Framework Adapted to Complicated Contexts
He, L., Wang, Z., Wang, L., & Li, F. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3276075 · Cited by 45 · see citing works
Weakening the Dominant Role of Text: CMOSI Dataset and Multimodal Semantic Enhancement Network
Jin, C., Luo, C., Yan, M. G., Zhao, G., Zhang, G., & Zhang, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3282953 · Cited by 44 · see citing works
Fu, Z., Liu, F., Xu, Q., Fu, X., & Qi, J. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2444-y · Cited by 44 · see citing works
Zhang, Q., Shi, L., Liu, P., Zhu, Z., & Xu, L. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03343-4 · Cited by 43 · see citing works
RetractedMSAF: Multimodal Split Attention FusionPreprint
Su, L., Hu, C., Li, G., & Cao, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.07175 · Cited by 65 · see citing works
- Authors
- Hazarika, D., Zimmermann, R., & Poria, S.
- Affiliations
- National University of Singapore (SG)
- Singapore University of Technology and Design (SG)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 1122-1131
- OpenAlex
- W3093051361
- Cited by
- 1,073
- References
- 45
- Citations by year
- 2026: 280 · 2025: 315 · 2024: 238 · 2023: 136 · 2022: 78 · 2021: 23 · 2020: 3
- FWCI
- 22.92
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion, cc-by)
- Funders
- Ministry of Education - Singapore (MOE2018-T2-1-103)
- DSO National Laboratories - Singapore (RTDST190702)
- RIE 2020 Advanced Manufacturing and Engineering (AME) programmatic grant (A19E2b0098)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multimodal sentiment analysis, modality gap, modality-invariant representations, modality-specific representations, subspace projection, multimodal fusion, user-generated video, humor detection, MOSI, MOSEI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
