The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index, and 1 are not checked yet.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Open accessAbstract
AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This dataset encompasses 3837 videos collected from over 1000 distinctive YouTube speakers, with a reasonably balanced gender distribution of 57% male and 43% female speakers, highlighting the dataset’s focus on inclusivity.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The thematic diversity of the CMU-MOSEI dataset is notable, with a spread across 250 different themes. The three predominant themes are comments (16.2%), debates (2.9%), and consultations (1.8%), while the rest of the themes are nearly evenly distributed.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One of CMU-MOSEI’s unique characteristics is its annotations at the utterance level, with 23,259 samples annotated in this manner.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [30] is a crucial resource in multimodal sentiment analysis and emotion recognition, distinguished by its scale, which is unmatched in the field.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
The cited source says what the claim attributes to it.
In the citing paper
“We generate open-ended AQAs about speech using the four commonly used datasets IEMOCAP [23], MOSEI [24], LibriTTS [25], and Vox-Celeb2 [26,27].”
From Joint Audio and Speech Understanding (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Research works printed but not linked: 3
- Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep-a col- laborative voice analysis repository for speech tech- nologies. In Acoustics, Speech and Signal Process- ing (ICASSP), 2014 IEEE International Conference on. IEEE, pages 960-964.
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmerman. 2018. Memn: Multimodal emotional memory network for emotion recognition in dyadic conversational videos. In NAACL.
- Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Re- current Highway Networks. arXiv preprint arXiv:1607.03474 . · arXiv
Cited by
Sorted by citations
Items 301–400 of 1,003
A Multimodal Sentiment Analysis Method Based on Fuzzy Attention Fusion
Zhi, Y., Li, J., Wang, H., Chen, J., & Wei, W. (2024) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2024.3434614 · Cited by 28
“We conduct a series of experiments on two MSA datasets including CMU-MOSI [41] and CMU-MOSEI [42] Our experiments are based on the Pytorch, and an Nvidia 3090 GPU is used to support the training.”
MethodLinking emotions to behaviors through deep transfer learning
Li, H., Baucom, B., & Georgiou, P. (2020) · PeerJ Computer Science · doi:10.7717/peerj-cs.246 · Cited by 27
“In addition, the CMU-MOSEI dataset has a significant data imbalance issue: the true label in each emotion is highly under-represented.”
MethodResultInfluentialThe FaceChannel: A Light-weight Deep Neural Network for Facial Expression Recognition
Barros, P., Churamani, N., & Sciutti, A. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00070 · Cited by 26
“The development of such models was supported by the collection of several “in-the-wild” datasets [14], [15], [16], [17] that provided large amounts of well-labelled data.”
A Optimized BERT for Multimodal Sentiment Analysis
Wu, J., Zhu, T., Zhu, J., Li, T., & Wang, C. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3566126 · Cited by 29
“There are similar data sets, as it happens, CMU-MOSEI[2], YouTube[17], ICT-MMMO[26] and so on.”
BackgroundQAP: A Quantum-Inspired Adaptive-Priority-Learning Model for Multimodal Emotion Recognition
Li, Z., Zhou, Y., Liu, Y., Zhu, F., Yang, C., & Hu, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.772 · Cited by 26
“For CMU-MOSEI, we calculate the weighted accuracy and F1-score of each emotion and the average values.”
BackgroundInfluentialSaha, T., Gupta, D., Saha, S., & Bhattacharyya, P. (2020) · Cognitive Computation · doi:10.1007/s12559-019-09704-5 · Cited by 28
M-BERT: Injecting Multimodal Information in the BERT StructurePreprint
Rahman, W., Hasan, K. M. A., Zadeh, A., Morency, L., & Hoque, M. E. (2019) · arXiv (Cornell University) · Cited by 26
Self-citationAnalyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling FusionPreprint
Mai, S., Xing, S., He, J., Zeng, Y., & Hu, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13572 · Cited by 26
“5) Analysis of Model Complexity: To analyze the model complexity of Multimodal Graph, we use the number of trainable parameters on the CMU-MOSEI dataset as the proxy for its space complexity, and compare it to the state-of-the-art unaligned fusion methods, as reported in Table V.”
BackgroundMethodInfluentialQuan, Z., Sun, T., Su, M., & Wei, J. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/4767437 · Cited by 25
“Our extensive experiments on the publicly available and popular datasets CMU-MOSI [14] and CMU-MOSEI [15] show that MGHF shows strong competitiveness over previous complex interaction and fusion baselines. contributions of this paper are summarized as follows: (i) A gated cyclic hierarchical fusion…”
BackgroundMethodDu, J., Jin, J., Zhuang, J., & Zhang, C. (2024) · Scientific Reports · doi:10.1038/s41598-024-54872-6 · Cited by 25
“MTAG 15 Modal-Temporal Attention Graph (MTAG) can capable of both fusion and alignment. while utilizing substantially lower number of parameters than a transformer-based model such as MulT 33 .”
MethodSentiment analysis of social media comments based on multimodal attention fusion network
Liu, Z., Yang, T., Chen, W., Chen, J., Li, Q., & Zhang, J. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112011 · Cited by 25
Li, Z.-H., Liu, P.-B., Pan, Y.-S., Ding, W.-P., Yu, J., Chen, H.-R., Liu, W.-H., Luo, Y.-M., & Wang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128940 · Cited by 25
Large Language Models in Psychiatry: Current Applications, Limitations, and Future Scope
Liu, Z., Bao, Y., Zeng, S., Qian, R., Deng, M., Gu, A., Li, J., Wang, W., Cai, W., Li, W., et al. (2024) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020046 · Cited by 25
Li, J., Wang, X., & Zeng, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3581236 · Cited by 25
“The datasets utilized in this work include IEMOCAP [47] (both 6-way and 4-way settings), MELD [48], and CMU-MOSEI [49].”
InfluentialDynamic Invariant-Specific Representation Fusion Network for Multimodal Sentiment Analysis
He, J., Yanga, H., Zhang, C., Chen, H.-R., & Xua, Y.-S. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/2105593 · Cited by 26
“Therefore, different LSTMs are used to extract features of different modalities in many methods, such as memory fusion network (MFN) [9], graph-memory fusion network (Graph -MFN) [10].”
MethodCMJRT: Cross-Modal Joint Representation Transformer for Multimodal Sentiment Analysis
Xu, M. Y., Liang, F., Su, X., & Fang, C. (2022) · IEEE Access · doi:10.1109/access.2022.3219200 · Cited by 25
“The CMU-MOSEI dataset [28] is an improvement over CMU-MOSI with greater diversity in samples, speakers, and topics.”
MethodUniSA: Unified Generative Framework for Sentiment Analysis
Li, Z., Lin, T.-E., Wu, Y., Liu, M., Tang, F., Zhao, M., & Li, Y. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612336 · Cited by 32
“It is the current SOTA model for MOSI [77] and MOSEI [78].”
BackgroundMethodInfluentialXiao, L., Zhou, E., Wu, X., Yang, S., Ma, T., & He, L. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9860020 · Cited by 24
Gradient-Guided Modality Decoupling for Missing-Modality Robustness
Wang, H., Luo, S., Hu, G., & Zhang, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i14.29474 · Cited by 24
Securing IP in edge AI: neural network watermarking for multimodal models
Nie, H., & Lu, S. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05746-x · Cited by 24
Feng, G., Weng, F., Lu, W. P., Xu, L., Zhu, W., Tan, M., & Weng, P. (2025) · International Journal of General Medicine · doi:10.2147/ijgm.s516247 · Cited by 31
Multimodal Behavior Modeling for Socially Interactive Agents
Pélachaud, C., Busso, C., & Heylen, D. (2021) · ACM eBooks · doi:10.1145/3477322.3477331 · Cited by 28
A multimodal time-series method for gifting prediction in live streaming platforms
Xi, D., Tang, L., Chen, R., & Xu, W. X. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103254 · Cited by 26
Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss
Franceschini, R., Fini, E., Beyan, C., Conti, A., Arrigoni, F., & Ricci, E. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956589 · Cited by 27
“The CMU-MOSEI [43] is the largest multimodal in-thewild dataset in the MER domain.”
BackgroundMethodInfluentialAn Assessment of In-the-Wild Datasets for Multimodal Emotion Recognition
Aguilera, A., Mellado, D., & Rojas, F. (2023) · Sensors · doi:10.3390/s23115184 · Cited by 24
“CMU-MOSEI [75] CMU Multimodal Opinion Sentiment and Emotion Intensity Wild Language, vision, and audio 23,453 annotated sentences from more than 1000 online speakers and 250 different topics 1438 happy, 502 sad, 384 angry, 55 fear, 150 disgust, 78 surprise”
BackgroundMultimodal Reaction: Information Modulation for Cross-Modal Representation Learning
Zeng, Y., Mai, S., Yan, W., & Hu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3293335 · Cited by 25
“In this section, we compare the proposed model with other methods on two MSA datasets including the CMU-MOSI [32] and CMU-MOSEI [72] datasets.”
MethodSelf-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition
Yang, H., Gao, X., Wu, J., Gan, T., Ding, N., Jiang, F., & Nie, L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.390 · Cited by 24
“With the development of streaming services, many ERC datasets such as IEMOCAP (Busso et al., 2008), MELD (Poria et al., 2019), and MO-SEI (Bagher Zadeh et al., 2018) provide a new platform for ERC researchers.”
MethodCo-Located Human–Human Interaction Analysis Using Nonverbal Cues: A Survey
Beyan, C., Vinciarelli, A., & Bue, A. D. (2023) · ACM Computing Surveys · doi:10.1145/3626516 · Cited by 27
“, CMU-MOSEI [202] having 23500 video clips, AffectNet [127] having millions of annotated images).”
BackgroundMulti-label Emotion Analysis in Conversation via Multimodal Knowledge Distillation
Anand, S., Devulapally, N. K., Bhattacharjee, S. D., & Yuan, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612517 · Cited by 23
“ImportanttonotethatcomparedtobothEmoReactandElderReactdatasetswhichhavepresentedsamplesfrom15discreteemotionstates, theCMU-MOSEIdatasetissignificantlylargerandpresentssamplesonly from 6 emotion categories.”
BackgroundMethodInfluentialMultimodal Sentiment Analysis With Mutual Information-Based Disentangled Representation Learning
Sun, H., Niu, Z., Wang, H., Yu, X., Liu, J., Chen, Y.-W., & Lin, L. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3529732 · Cited by 28
“2) CMU-MOSEI: CMU-MOSEI [39] is an enlarged version of CMU-MOSI [37].”
Huang, C., Lin, Z., Huang, Q., Huang, X., Jiang, F., & Chen, J. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01806-y · Cited by 26
Actual Cause-Guided Adaptive Gradient Scaling for Balanced Multimodal Sentiment Analysis
Chen, J., Huang, Q., Huang, C., & Huang, X. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3736415 · Cited by 24
“Its architecture primarily consists of unimodal feature extraction and cross-modal MoEs.”
Two-Level Multimodal Fusion for Sentiment Analysis in Public Security
Sun, J., Yin, H., Tian, Y., Wu, J., Shen, L., & Chen, L. (2021) · Security and Communication Networks · doi:10.1155/2021/6662337 · Cited by 23
“…datasets: CMU Multimodal Corpus of Sentiment Intensity (CMU-MOSI) [32] and CMU Multi-modal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) [33] datasets for sentiment analysis task and Interactive Emotional Dyadic Motion Capture (IEMOCAP) [34] dataset for emotion classification task.…”
MethodCross-modal dynamic convolution for multi-modal emotion recognition
Wen, H., You, S., & Fu, Y. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103178 · Cited by 23
Multimodal sentiment and emotion recognition in hyperbolic space
Araño, K. A., Orsenigo, C., Soto, M., & Vercellis, C. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115507 · Cited by 23
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398 · Cited by 25
Climate and Weather: Inspecting Depression Detection via Emotion Recognition
Wu, W., Wu, M., & Yu, K. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746634 · Cited by 24
“The emotion datsets involved are IEMOCAP [16] and CMUMOSEI [17].”
MethodGe, S., Jiang, Z., Cheng, Z., Wang, C., Yin, Y., & Gu, Q. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583258 · Cited by 23
“Multi-modal Multi-label Emotion Recognition (MMER) task aims to identify emotions from multi-modal data, so as to meet the requirements of some special applications, such as chatbot and emotional conversation systems [30].”
BackgroundMethodRobust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis
Mao, H., Zhang, B., Xu, H., Yuan, Z., & Liu, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.27078 · Cited by 30
A graph convolution-based heterogeneous fusion network for multimodal sentiment analysis
Zhao, T., Peng, J., Huang, Y., Wang, L., Zhang, H., & Cai, Z. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05151-w · Cited by 23
“CMU-MOSEI The CMU-MOSEI dataset [28] is a multi-modal sentiment analysis dataset extractedfrom 5,000videos on YouTube, which is a further extension of CMU-MOSI.”
Multimodal Human-Agent Dialogue Corpus with Annotations at Utterance and Dialogue Levels
Komatani, K., & Okada, S. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597447 · Cited by 23
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85
“M ULTI B ENCH contains 4 datasets involving fusing language , video , and audio time-series data to predict sentiment (CMU-MOSI [181]), emotions (CMU-MOSEI [183]), humor (UR-FUNNY [64]), and sarcasm (MUS T ARD [24]).”
BackgroundSelf-citationSentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis
Fan, Q., Han, J., He, Y., Zheng, T., & Zheng, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.821 · Cited by 23
Dynamic Causal Disentanglement Model for Dialogue Emotion Detection
Su, Y., Wei, Y., Nie, W., Zhao, S., & Liu, A.-A. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3406710 · Cited by 23
“MOSEI [36] is a multimodal emotion recognition dataset that annotated video clips from 1000 different speakers.”
Addressing data scarcity in speech emotion recognition: A comprehensive review
Kakuba, S., & Han, D. S. (2024) · ICT Express · doi:10.1016/j.icte.2024.11.003 · Cited by 22
Towards robust sentiment analysis with multimodal interaction graph and hybrid contrastive learning
Gong, P., Liu, J., Zhang, X., Li, X., Wei, L., & He, H. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111870 · Cited by 22
AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-Modal Alignment
Li, Y., Xing, Y., Lan, X., Li, X., Chen, H., & Jiang, D. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02307 · Cited by 30
“This capability is fundamental to a wide range of applications such as visual-language understanding [18, 42, 48] and audio-visual analysis [19, 21, 45, 46].”
Li, X., & Chen, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63031-7_26 · Cited by 22
“Based on the successes in video, image, audio and language processing, multimodal sentiment analysis has been studied extensively and produced impressive results in recent years (Liang et al., 2018; Liu et al., 2018; Mai et al., 2019; Wang et al., 2019; Zadeh et al., 2018c).”
BackgroundMethodInfluentialChen, C., Hong, H., Guo, J., & Song, B. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3263801 · Cited by 22
“6) Impact of Loss Factor α , β and λ : For CMU-MOSE dataset and CMU-MOSEI dataset, Fig.”
MethodInfluentialSLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Shon, S., Arora, S., Lin, C.-J., Pasad, A., Wu, F. F., Sharma, R. S., Wu, W.-L., Lee, H.-Y., Livescu, K., & Watanabe, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.496 · Cited by 40
pmBQA: Projection-based Blind Point Cloud Quality Assessment via Multimodal Learning
Xie, W., Wang, K., Ju, Y., & Wang, M. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611998 · Cited by 22
Multi-Modal and Multi-Task Depression Detection with Sentiment Assistance
Teng, S., Chai, S., Liu, J., Tateyama, T., Lin, L., & Chen, Y. (2024) · International Conference on Consumer Electronics · doi:10.1109/icce59016.2024.10444213 · Cited by 25
Multi-Grained Multimodal Interaction Network for Sentiment Analysis
Fang, L., Liu, G., & Zhang, R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446351 · Cited by 21
“We evaluate our proposed model on CMU-MOSI [13] and CMU-MOSEI [14].”
MethodDabbabi, K., & Mars, A. (2024) · Journal of Systems Science and Systems Engineering · doi:10.1007/s11518-024-5607-y · Cited by 24
“…in weighted accuracy (WA) by approximately 2%, outperforming the 24-MFCCs tested for the six-class emotion recognition task on the CMU-MOSEI dataset (Liang et al. 2018, Zadeh et al. 2018, Gideon et al. 2017, Tsai et al. 2019, Vaishnav et al. 2021, Nair and Gomathi 2022, Shukla et al. 2022).”
Yang, X., Wu, W., Feng, S., Wang, M., Wang, D., Li, Y., Sun, Q., Zhang, Y., Fu, X., & Poria, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103204 · Cited by 29
Self-citationYang, L., Zhong, J., Wen, T., & Liao, Y. J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103230 · Cited by 21
Yu, L., Ge, Y., Ansari, S., Imran, M. A., & Ahmad, W. (2025) · Sensors · doi:10.3390/s25154763 · Cited by 26
Yang, Z., He, Q., Yu, M., Du, N., & Lu, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103571 · Cited by 26
Cross-Modal Sentiment Sensing with Visual-Augmented Representation and Diverse Decision Fusion
Zhang, S., Li, B., & Yin, C. (2021) · Sensors · doi:10.3390/s22010074 · Cited by 22
“Two additional datasets, CMU-Multimodal Opinion Sentiment Intensity (CMUMOSI) [53] and CMU-Multimodal Opinion Sentiment and Emotion Intensity (CMUMOSEI) [54] are employed to evaluate and compare the performance of HFN.”
MethodDeep Tensor Evidence Fusion Network for Sentiment Classification
Wang, Z., Xu, G., Zhou, X., Kim, J., Zhu, H., & Deng, L. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3197994 · Cited by 22
“As a result, several approaches, such as memory fusion network (MFN) [30] and graph-MFN [31], used various versions of LSTM to extract features from various perspectives.”
BackgroundMethodWu, Y., Zhang, Z., Peng, P., Zhao, Y., & Qin, B. (2022) · doi:10.1145/3551876.3554813 · Cited by 21
“CMU-MOSEI is another multi-modal emotion recognition dataset.”
MethodInfluentialVisual language integration: A survey and open challenges
Park, S., & Kim, Y. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100548 · Cited by 24
Zheng, Y., Gong, J., Wen, Y., & Zhang, P. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122728 · Cited by 23
Hu, R., Yi, J., Chen, A., & Chen, L. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3388670 · Cited by 22
“The CMU multimodal opinion sentiment and emotion intensity (MOSEI) [28] dataset is an extended version of the MOSI dataset, including more discourses, more speakers, samples, and topics.”
BackgroundMultimodal Embeddings From Language Models for Emotion Recognition in the Wild
Tseng, S.-Y., Narayanan, S. S., & Georgiou, P. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3065598 · Cited by 23
Label distribution for multimodal machine learning
Ren, Y. C., Xu, N., Ling, M., & Geng, X. (2021) · Frontiers of Computer Science · doi:10.1007/s11704-021-0611-6 · Cited by 22
“• Graph-MFN [36]: it is a Dynamic Fusion Graph (DFG) based Memory Fusion Network (MFN).”
MethodLiu, F., Shen, S.-Y., Fu, Z.-W., Wang, H.-Y., Zhou, A.-M., & Qi, J.-Y. (2022) · Entropy · doi:10.3390/e24071010 · Cited by 21
“In this section, we evaluate our model on CMU-MOSEI [19].”
PAMoE-MSA: polarity-aware mixture of experts network for multimodal sentiment analysis
Huang, C., Lin, Z., Han, Z., Huang, Q., Jiang, F., & Huang, X. (2025) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-025-00362-y · Cited by 20
“• We conduct extensive experiments on three datasets, CMU-MOSI [20] and CMU-MOSEI [21] and CH-123 opening” appears more frequently in negative samples within the datasets,existingmethodsmaymisassociate“mouthopening”withneg-ativesentiment SIMS [22].”
Toward Robust Multimodal Sentiment Analysis using multimodal foundational models
Zhao, X., Poria, S., Li, X., Chen, Y., & Tang, B. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126974 · Cited by 28
Self-citationDeep Multimodal Sequence Fusion by Regularized Expressive Representation Distillation
Guo, X., Kong, A. W., & Kot, A. C. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3142448 · Cited by 21
Can We Exploit All Datasets? Multimodal Emotion Recognition Using Cross-Modal Translation
Yoon, Y. C. (2022) · IEEE Access · doi:10.1109/access.2022.3183587 · Cited by 23
“To evaluate the performance of the emotion recognition task, we applied the proposed method to the CMU Multimodal Opinion sentiment and emotion intensity (CMU-MOSEI) dataset [38] and the interactive emotional dyadic motion capture database (IEMOCAP) dataset [39].”
MethodImpact of Annotator Demographics on Sentiment Dataset Labeling
Ding, Y., You, J., Machulla, T., Jacobs, J., Sen, P., & Höllerer, T. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555632 · Cited by 25
“Left two figures in each category show similarity with original ratings from [83].”
BackgroundMethodInfluentialPeng, C., Chen, K., Shou, L., & Chen, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i13.29374 · Cited by 20
Zhang, R., Chi, X.-W., Liu, G., Zhang, W., Du, Y., & Wang, F. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3567535 · Cited by 32
“We selected three multimodal sentiment analysis datasets, encompassing vision, text, and audio, to align with real-world applications such as smart homes.”
Detecting Topic-Oriented Speaker Stance in Conversational Speech
Lai, C., Alex, B., Moore, J. D., Tian, L., Hori, T., & Francesca, G. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2632 · Cited by 19
“Most current emotion recognition work focuses on the ‘Big Six’ emotion categories [28, 7], which only includes one clearly positive category: happiness.”
BackgroundDominant SIngle-Modal SUpplementary Fusion (SIMSUF) for Multimodal Sentiment Analysis
Huang, J., Ji, Y., Qin, Z., Yang, Y., & Shen, H. T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3344358 · Cited by 21
“The CMU-MOSEI dataset [54] is an upgraded version of the CMU-MOSI dataset.”
MethodA dual-channel multimodal sentiment analysis framework based on three-way decision
Wang, X., Wang, M., Cui, H., & Zhang, Y. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109174 · Cited by 19
Injecting Multimodal Information Into Pre-Trained Language Model for Multimodal Sentiment Analysis
Mai, S., Zeng, Y., Xiong, A., & Hu, H. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3553149 · Cited by 21
“…multiple modalities [5], and popular fusion approaches include cross-modal attention [5], [6], [7], tensor fusion [8], [9], graph fusion [10], [11], [12], etc. Recently, with the rise of powerful pre-trained language models (PLMs) [13], [14], [15], a few methods try to fine-tune PLMs with…”
Yang, Y., Dong, X., & Qiang, Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i24.34755 · Cited by 27
M2H2: A Multimodal Multiparty Hindi Dataset For Humor Recognition in Conversations
Chauhan, D. S., Singh, G. V., Majumder, N., Zadeh, A., Ekbal, A., Bhattacharyya, P., Morency, L., & Poria, S. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479959 · Cited by 19
“The model shows the state-of-the-art (SOTA) for MOSI [33] and MOSEI [32] datasets.”
BackgroundSelf-citationQi, Q., Lin, L., & Zhang, R. (2021) · Information · doi:10.3390/info12090342 · Cited by 19
“We evaluate the proposed model on the benchmark dataset of sentiment and sentiment analysis, namely, the CMU Multimodal View Sentiment and Mood Intensity (CMU-MOSEI) dataset [32].”
InfluentialMultimodal Sentiment Analysis Based on Interactive Transformer and Soft Mapping
Li, Z., Guo, Q., Feng, C., Deng, L., Zhang, Q., Zhang, J., Wang, F., & Sun, Q. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/6243347 · Cited by 18
Hateful Memes Detection Based on Multi-Task Learning
Ma, Z., Yao, S., Wu, L., Gao, S., & Zhang, Y. (2022) · Mathematics · doi:10.3390/math10234525 · Cited by 19
MMFN: Emotion recognition by fusing touch gesture and facial expression information
Li, Y., Meng, Q., Wang, Y., & Hou, H.-R. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120469 · Cited by 24
Lee, C.-C., Chaspari, T., Provost, E. M., & Narayanan, S. S. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3276209 · Cited by 23
“, either podcasts [47] or YouTube videos [52]), rather than from natural dialogs, again raising the specter of mismatch with respect to an intended use case.”
BackgroundHan, W., Chen, H., Kan, M., & Poria, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.717 · Cited by 21
Self-citationNORMSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly
Fung, Y. R., Chakrabarty, T., Guo, H., Rambow, O., Muresan, S., & Ji, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.941 · Cited by 49
Multi-View Interactive Representations for Multimodal Sentiment Analysis
Tang, Z., Xiao, Q., Qin, Y., Zhou, X., Zhou, J. T., & Li, K. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3357480 · Cited by 20
“We conduct extensive experiments on multiple public datasets, including [27], [28], and [29].”
MethodChen, R., Zhou, W., Hu, H., Fei, Z., Fei, M., & Zhou, H. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112372 · Cited by 19
Ishikawa, S., & Sugiura, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3108500 · Cited by 19
VAE-Based Adversarial Multimodal Domain Transfer for Video-Level Sentiment Analysis
Wang, Y., Wu, J., Furumai, K., Wada, S., & Kurihara, S. (2022) · IEEE Access · doi:10.1109/access.2022.3174215 · Cited by 19
Wang, H., Yang, M.-C., Li, Z., Liu, Z., Hu, J., Fu, Z.-W., & Liu, F. (2022) · Comput. Animat. Virtual Worlds · doi:10.1002/cav.2090 · Cited by 14
Moving From Narrative to Interactive Multi-Modal Sentiment Analysis: A Survey
Ma, J., Rong, L., Zhang, Y., & Tiwari, P. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3610288 · Cited by 20
“YouTube, CMU-MOSI, and CMU-MOSEI datasets were collected from the YouTube website.”
HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence
Wortman, B., & Wang, J. Z. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3324902 · Cited by 24
“More specifically, Paul Ekman’s research into basic universal emotions serves as the foundation for most annotation schemes currently used [14], [15], [16], [17], [18], [19].”
BackgroundA Multi-Modal ELMo Model for Image Sentiment Recognition of Consumer Data
Rong, L., Ding, Y., Wang, M., Saddik, A. E., & Hossain, M. S. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3357543 · Cited by 18
“We have selected two widely recognized benchmark datasets, namely, CMU-MOSI [28] and CMU-MOSEI [29].”
MethodArunaGladys, A., & Vetriselvi, V. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111553 · Cited by 36
Emotion Fusion-Sense (Emo Fu-Sense) – A novel multimodal emotion classification technique
Umair, M., Rashid, N., Khan, U. S., Hamza, A., & Iqbal, J. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106224 · Cited by 18
Joint training strategy of unimodal and multimodal for multimodal sentiment analysis
Meng, L., Zhu, Z., Li, K., Zhou, L., Zhao, Z., & Pei, H. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105172 · Cited by 18
Jia, K., Meng, F., & Liang, J. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121543 · Cited by 19
Metadata
- Authors
- Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- California Miramar University (US)
- Nanyang Technological University (SG)
- Agency for Science, Technology and Research (SG)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 2236-2246
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2883409523
- Cited by
- 1,433
- References
- 62
- Citations by year
- 2026: 281 · 2025: 300 · 2024: 266 · 2023: 202 · 2022: 153 · 2021: 113 · 2020: 73 · 2019: 39 · 2018: 6
- FWCI
- 17.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- CMU-MOSEI, multimodal fusion, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
