The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index, and 1 are not checked yet.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Open accessAbstract
AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This dataset encompasses 3837 videos collected from over 1000 distinctive YouTube speakers, with a reasonably balanced gender distribution of 57% male and 43% female speakers, highlighting the dataset’s focus on inclusivity.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The thematic diversity of the CMU-MOSEI dataset is notable, with a spread across 250 different themes. The three predominant themes are comments (16.2%), debates (2.9%), and consultations (1.8%), while the rest of the themes are nearly evenly distributed.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One of CMU-MOSEI’s unique characteristics is its annotations at the utterance level, with 23,259 samples annotated in this manner.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [30] is a crucial resource in multimodal sentiment analysis and emotion recognition, distinguished by its scale, which is unmatched in the field.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
The cited source says what the claim attributes to it.
In the citing paper
“We generate open-ended AQAs about speech using the four commonly used datasets IEMOCAP [23], MOSEI [24], LibriTTS [25], and Vox-Celeb2 [26,27].”
From Joint Audio and Speech Understanding (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Research works printed but not linked: 3
- Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep-a col- laborative voice analysis repository for speech tech- nologies. In Acoustics, Speech and Signal Process- ing (ICASSP), 2014 IEEE International Conference on. IEEE, pages 960-964.
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmerman. 2018. Memn: Multimodal emotional memory network for emotion recognition in dyadic conversational videos. In NAACL.
- Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Re- current Highway Networks. arXiv preprint arXiv:1607.03474 . · arXiv
Cited by
Sorted by citations
Items 501–600 of 1,003
Causal Inference for Modality Debiasing in Multimodal Emotion Recognition
Kim, J., Hong, J., & Choi, Y. (2024) · Applied Sciences · doi:10.3390/app142311397 · Cited by 12
Li, X., Dong, Z., Cheng, X., Miao, D., Zhang, H., Wang, T., Zhang, X., & Li, Z. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129341 · Cited by 11
Nguyen, C.-D. T., Nguyen, T. T., Vu, D., & Luu, A. N. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.980 · Cited by 15
Capturing High-Level Semantic Correlations via Graph for Multimodal Sentiment Analysis
Qian, F., Han, J., Guan, Y., Song, W., & He, Y. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3359570 · Cited by 11
“We evaluate our approaches using two well-established datasets: CMU-MOSI [26] and CMU-MOSEI [27], both of which involve the expression of speaker sentiments through three modalities: text, audio, and visual.”
MethodA Fine-Grained Tri-Modal Interaction Model for Multimodal Sentiment Analysis
Zhi, Y., Li, J., Wang, H., Chen, J., & Cao, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447872 · Cited by 11
“CMU-MOSEI [22] is the largest dataset for MSA and it collects 16326, 1871, 4659 segments for training, validation, and testing.”
BackgroundDin, I. U., Almogren, A., Rodrigues, J. J. P. C., & Altameem, A. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3422788 · Cited by 11
Multimodal Consistency-Based Teacher for Semi-Supervised Multimodal Sentiment Analysis
Yuan, Z., Fang, J., Xu, H., & Gao, K. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3430543 · Cited by 14
“To evaluate the effectiveness of the proposed method, experiments are conducted on two benchmark MSA datasets, namely SIMS v2 [45] and CMU-MOSEI [51].”
MethodResultZhu, Z., Cheng, X., Chen, Z., Chen, Y., Zhang, Y., Wu, X., Zheng, Y., & Xing, B. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681623 · Cited by 13
Personalized emotion analysis based on fuzzy multi-modal transformer model
Liu, J., Ang, M. C., Chaw, J. K., Ng, K. W., & Kor, A.-L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05954-5 · Cited by 13
“These generated utterances perfectly align with the preprocessed data from prior work [115].”
A Social Media Dataset and H-GNN-Based Contrastive Learning Scheme for Multimodal Sentiment Analysis
Peng, J., He, Y., Chang, Y., Lu, Y., Zhang, P., Ou, Z., & Yu, Q. (2025) · Applied Sciences · doi:10.3390/app15020636 · Cited by 11
“The CMU-MOSEI [7] dataset is an improvement on it.”
Multimodal sentiment analysis with text-augmented cross-modal feature interaction attention network
Zhang, H., Peng, J., & Cai, Z. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113078 · Cited by 12
A Survey on the Impact of Pre-Trained Language Models in Sentiment Classification Task
Gautam, H., Gaur, A., & Yadav, D. K. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00805-z · Cited by 14
“The CMU-MOSEI [154] dataset are the largest and comprehensive multimodal sentiment analysis dataset consisting of 23,452 video clips across 250 different topics from 1000 speakers.”
Quantum Federated Learning for Multimodal Data: A Modality-Agnostic Approach
Pokharel, A., Rahman, R., Morris, T., & Nguyen, D. C. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00059 · Cited by 16
“For our proposed method, we have used the Carnegie Mel-lon University Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [53].”
Gupta, C., Gill, N. S., Gulia, P., Kumar, A., Karamti, H., Moges, D. M., & Safra, I. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01300-9 · Cited by 11
Dense Fusion Network with Multimodal Residual for Sentiment Classification
Deng, H., Kang, P., Yang, Z., Hao, T., Li, Q., & Liu, W. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428321 · Cited by 11
“The baselines include a number of sentiment analysis ap-poraches: Tensor Fusion Network (TFN) [2], Low rank Multimodal Fusion (LMF) [6], Memory Fusion Network (MFN) [7], Dynamic fusion graph (DFG) [14], Early Fusion LSTM (EF-LSTM) [1], Bi-directional Contextual LSTM (BC-LSTM) [15], Convolutional Multiple Kernel Learni…”
BackgroundWerewolf-XL: A Database for Identifying Spontaneous Affect in Large Competitive Group Interactions
Zhang, K., Wu, X., Xie, X., Zhang, X., Zhang, H., Chen, X., & Sun, L. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3101563 · Cited by 11
“The CMU-MOSEI [31] contains 3,228 videos and 23,453 annotated sentences from more than 1000 online speakers and 250 frequently used topics in online videos with categorical and continuous-valued emotion labels for multimodal sentiment analysis and emotion recognition.”
BackgroundMulti-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos
Ma, L., Yu, Y., Liang, T., & Liu, T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0351-0_21 · Cited by 11
Language-Agnostic Model for Aspect-Based Sentiment Analysis
Akhtar, M. S., Abhishek, K., Ekbal, A., Biemann, C., & Bhattacharyya, P. (2019) · International Conference on Computational Semantics (IWCS) · doi:10.18653/v1/w19-0413 · Cited by 11
“…2016), multi-lingual sentiment analysis (Balamurali et al., 2012; Mishra et al., 2017; Brun et al., 2016; Kumar et al., 2016), multi-modal sentiment analysis (Poria et al., 2017; Zadeh et al., 2018; Ghosal et al., 2018), sentiment analysis in Twitter (Ghosh et al., 2015; Mohammad et al., 2013) etc.”
BackgroundMulti-Modal Language Analysis with Hierarchical Interaction-Level and Selection-Level Attentions
Zhang, D., Wu, L., Li, S., Zhu, Q., & Zhou, G. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00130 · Cited by 11
“As an upcoming research area, the modeling of human language across both verbal and nonverbal behaviors extends natural language processing to multi-modal sentiment analysis [1] and multimodal emotion recognition [2] from multi-modal temporal signals including text (spoken words), vision (facial expressions, gestures)…”
InfluentialDeep Learning for Multimodal Emotion Recognition-Attentive Residual Disconnected RNN
Chandra, E., & Hsu, J. Y.-J. (2019) · Conference on Technologies and Applications of Artificial Intelligence (TAAI) · doi:10.1109/taai48200.2019.8959913 · Cited by 10
“TABLE I CMU MOSEI DATASET COMPARED WITH OTHER DATASETS [2]”
BackgroundMethodInfluentialOn the Use of Modality-Specific Large-Scale Pre-Trained Encoders for Multimodal Sentiment Analysis
Ando, A., Masumura, R., Takashima, A., Suzuki, S., Makishima, N., Suzuki, K., Moriya, T., Ashihara, T., & Satō, H. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022548 · Cited by 12
Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML)
(2020) · doi:10.18653/v1/2020.challengehml-1 · Cited by 11
“Poria et al. (2017) successfully used RNN-based deep networks for multi-modal emotion recognition, which was followed by multiple other works (Chen et al., 2017; Zadeh et al., 2018a; Zadeh et al., 2018c) giving results far better than what was seen before.”
BackgroundMethodResultInfluentialCross-Lingual Bimodal Emotion Recognition with LLM-Based Label Smoothing
Рюмина, Е. В., Axyonov, A., Abdulkadirov, T., Koryakovskaya, D., & Ryumin, D. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9110285 · Cited by 10
Dynamic Weighted Multitask Learning and Contrastive Learning for Multimodal Sentiment Analysis
Wang, X., Zhang, M., Chen, B., Wei, D., & Shao, Y. (2023) · Electronics · doi:10.3390/electronics12132986 · Cited by 13
Towards Arabic Multimodal Dataset for Sentiment Analysis
Haouhat, A., Bellaouar, S., Nehar, A., & Cherroun, H. (2023) · International Conference on Intelligent Data Science Technologies and Applications (IDSTA) · doi:10.1109/idsta58916.2023.10317847 · Cited by 14
“Our methodology is inspired by both MOSEI [21] and CMU-MOSI [22] dataset-building processes while taking into account Arabic specificities.”
MethodTransformer-based adaptive contrastive learning for multimodal sentiment analysis
Hu, Y., Huang, X., Wang, X., Lin, H., & Zhang, R. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19152-5 · Cited by 13
“Wu et al. [29], building upon the CMU-MOSI dataset [30], constructed three real-world datasets.”
Al-Saadawi, H. F. T., Daş, B., & Daş, R. (2024) · Displays · doi:10.1016/j.displa.2024.102731 · Cited by 10
Mobile Virtual Assistant for Multi-Modal Depression-Level Stratification
Wu, E. H., Gao, T., Chung, C., Chen, C., Tsai, C.-F., & Yeh, S. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3451114 · Cited by 10
“Original waveform audio files in CMU-MOSEI [50] contained 3838 data from more than 1000 online YouTube speakers.”
InfluentialAdaptive Multimodal Graph Integration Network for Multimodal Sentiment Analysis
Gong, P., Liu, J., Zhang, X., Li, X., Wei, L., & He, H. (2024) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3507576 · Cited by 11
“CMU-MOSEI [47] is a significant improvement over MOSI, with a higher number of utterances and more variety in samples, speakers, and themes.”
Affective Computing Databases: In-Depth Analysis of Systematic Reviews and Surveys
Vaz, P. J., Rodrigues, J. M. F., & Cardoso, P. J. S. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3507289 · Cited by 11
Tripartite interaction representation learning for multi-modal sentiment analysis
Wang, B., Dong, G., Zhao, Y., Li, R., Yin, W., & Lu, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126279 · Cited by 10
Wen, Y., Li, Q., Zhou, Y., Gao, Y., Wen, Z., Tao, J., & Li, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3578618 · Cited by 12
“Early works [28], [29] utilized auxiliary software such as P2FA for word-level alignment, which is very time-consuming.”
Bimodal speech emotion recognition via contrastive self-alignment learning
Wang, C., Wen, G., Yang, P., & Liu, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128605 · Cited by 11
Redundancy-Adaptive Multimodal Learning for imperfect data
Chen, M., Yao, J., Xing, L., Wang, Y., Zhang, Y., & Wang, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107821 · Cited by 13
Multimodal Sentiment Analysis Using BiGRU and Attention-Based Hybrid Fusion Strategy
Liu, Z., Zhou, B., Meng, L., & Huang, G. (2023) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.038835 · Cited by 10
Curriculum Learning Meets Weakly Supervised Multimodal Correlation Learning
Mai, S., Ya, S., & Hu, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.209 · Cited by 13
“Graph-based fusion methods(Mai et al., 2020; Zadeh et al., 2018c; Behmanesh et al., 2021) learn interactions on the graph across time series and modalities.”
InfluentialJoint Objective and Subjective Fuzziness Denoising for Multimodal Sentiment Analysis
Jiang, X., Xu, X., Lu, H., He, L., & Shen, H. T. (2024) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2024.3405541 · Cited by 10
HighlightMe: Detecting Highlights from Human-Centric Videos
Bhattacharya, U., Wu, G., Petrangeli, S., Swaminathan, V., & Manocha, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00805 · Cited by 10
“A wide body of work has focused on multimodal action recognition [5, 45, 33, 10] and emotion recognition [3, 26, 61, 38, 39].”
BackgroundUnsupervised Multimodal Language Representations using Convolutional AutoencodersPreprint
Koromilas, P., & Γιαννακόπουλος, Θ. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03007 · Cited by 10
“Following the existing literature [8, 2, 5, 15, 3, 16, 10, 17, 9, 13], we report the binary accuracy and weighted averaged f1 metrics on sentiment for MOSEI, and on each of the 4 emotions of IEMOCAP in an one-vsall manner.”
InfluentialA Unified Transformer-based Network for Multimodal Emotion RecognitionPreprint
Ali, K., & Hughes, C. E. (2023) · doi:10.36227/techrxiv.23916123 · Cited by 10
Xie, Y., Niu, J., Zhang, Y., & Ren, F. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3307972 · Cited by 12
“Formultimodallanguagesequence modeling,explicitword-levelalignmentisusedinthemajority ofpastefforts[15],[22],[23].”
BackgroundXu, H. (2023) · Springer eBooks · doi:10.1007/978-981-99-5776-7_6 · Cited by 9
A Novel Multimodal Sentiment Analysis Model Based on Gated Fusion and Multi-Task Learning
Sun, X., Ren, X., & Xie, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446040 · Cited by 9
“In early works, most methods consider the interactions be-tween different modalities, such as Tensor Fusion Networks *Corresponding author (TFN) [1], Memory Fusion Networks (MFN) [2], and Dynamic Fusion Graph (DFG) [3].”
MethodYu, B., & Shi, Z. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06422-0 · Cited by 10
“There are several graph-based methods that are also widely used in multimodal studies, such as the graph memory fusion network (Graph-MFN) [32] and the graph fusion network (GFN) [33].”
Automatic movie genre classification & emotion recognition via a BiProjection Multimodal Transformer
Moreno-Galván, D. A., López, R., González, L. C., Montes-y-Gómez, M., Sánchez-Vega, F., & López-Monroy, A. P. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102641 · Cited by 14
Lee, S., Lai, B., Ryan, F., Boote, B., & Rehg, J. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01382 · Cited by 11
“Other multimodal approaches utilized the interconnection between spoken language and visual expressions to recognize human sentiment and emotion [9, 21, 22, 39, 42, 53].”
Overview of Multimodal Machine Learning
Al‐Zoghby, A. M., Al-Awadly, E. M. K., Ebada, A. I., & Awad, W. A. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3701031 · Cited by 11
Ji, M., Wei, N., Zhou, J., & Wang, X. (2024) · Electronics · doi:10.3390/electronics13214175 · Cited by 9
Zheng, W., Yu, J., & Xia, R. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681638 · Cited by 12
“To validate the effectiveness of our framework UniVA , we conduct experiments on two benchmark datasets: MOSEI [62] and M 3 ED [66].”
A Unified Biosensor–Vision Multi-Modal Transformer network for emotion recognition
Ali, K., & Hughes, C. E. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107232 · Cited by 9
Global distilling framework with cognitive gravitation for multimodal emotion recognition
Li, H., Zhong, H., Xu, C., Liu, X., Wen, G., & Liu, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2024.129306 · Cited by 9
Mohan, J., Reddy, M. S., Premjith, B., Jyothish Lal, G., & Chakravarthi, B. R. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3712260 · Cited by 10
“The paper by Zadeh et al. [56] introduces the CMU-MOSEI (Multimodal Opinion Sentiment and Emotion Intensity) dataset.”
Zhang, Y., Zhong, H., Alhusaini, N., Chen, G., & Wu, C. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113121 · Cited by 9
Sun, Z., Liu, H., Li, H., Li, Y., & Zhang, W. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105081 · Cited by 9
Qin, J., Liu, F., & Zong, L. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107449 · Cited by 9
Multi-modal sentiment recognition with residual gating network and emotion intensity attention
Wang, Y., Guo, X., Hou, X., Miao, Z., Yang, X., & Guo, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107483 · Cited by 9
POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models
He, J., Wang, X., Liu, S., Wu, G., Silva, C. T., & Qu, H. (2025) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis64226.2025.00010 · Cited by 15
“We demonstrate the effectiveness of our system on two different datasets for multimodal content comprehension tasks: CMU-MOSEI [2] for multimodal sentiment analysis, and WTaG [3] for user intent understanding.”
Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis
Luo, M., Jiang, Y., & Mai, S. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755591 · Cited by 19
Multimodal Classification and Out-of-Distribution Detection for Multimodal Intent Understanding
Zhang, H., Zhou, Q., Xu, H., Su, J., Evans, R. H., & Gao, K. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3618541 · Cited by 19
“Feature Extraction: Following [1], [42], we initially extract feature embeddings for text, video, and audio modalities to facilitate efficient multimodal fusion.”
Quan, Z., Sun, T., Su, M., Wei, J., Zhang, X., & Zhong, S. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945514 · Cited by 10
“The CMU-MOSEI dataset [13] has a larger number of vocalizations, a larger sample, speakers, and topics.”
BackgroundMethodMMATR: A Lightweight Approach for Multimodal Sentiment Analysis Based on Tensor Methods
Koromilas, P., Nicolaou, M. A., Γιαννακόπουλος, Θ., & Panagakis, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097030 · Cited by 9
“…models the parameters of the unimodal feature extraction networks (e.g., BERT) are not included in the final number of parameters, since it is a common practice [12, 10, 13, 14, 15, 16, 17, 4, 8, 2, 3, 1] to use pre-extracted features in order to evaluate the performance on multimodal sentiment.”
MethodInfluentialSemi-Supervised Multimodal Emotion Recognition with Class-Balanced Pseudo-labeling
Chen, H., Guo, C., Li, Y., Zhang, P., & Jiang, D. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612864 · Cited by 9
“In addition to transformers [5], some approaches [10, 14, 19, 34] convert multimodal features into graph-structured data and utilize graph neural networks to learn multimodal representations, yielding promising results.”
BackgroundAbdusalomov, A. B., Kutlimuratov, A., Nasimov, R., & Whangbo, T. K. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.044466 · Cited by 9
VAD: A Video Affective Dataset With Danmu
Wang, S., Li, X., Zheng, F., Pan, J., Li, X., Chang, Y., Zhu, Z., Li, Q., Wang, J., & Xiao, Y. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3382503 · Cited by 9
Towards Asynchronous Multimodal Signal Interaction and Fusion via Tailored Transformers
Yang, D., Kuang, H., Yang, K., Li, M., & Zhang, L. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3409211 · Cited by 9
“MOSEI[35]consistsof22,856 annotatedvideosegmentsfrom1,000differentspeakersunder 250socialchannelsonthetopicscovered.”
End-to-End Learning for Multimodal Emotion Recognition in Video With Adaptive Loss
Huynh, V. T., Yang, H.-J., Lee, G.-S., & Kim, S.-H. (2021) · IEEE Multimedia · doi:10.1109/mmul.2021.3080305 · Cited by 9
MMTrans-MT: A Framework for Multimodal Emotion Recognition Using Multitask Learning
Shen, J., Zheng, J., & Wang, X. (2021) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci52617.2021.9435906 · Cited by 9
“In this work, experiments are conducted on two multimodal emotion datasets: CMU-MOSEI [5] and IEMOCAP [6].”
MethodAn interactive VR platform with emotion recognition for self-attachment intervention
Polydorou, N., & Edalat, A. (2021) · EAI Endorsed Transactions on Pervasive Health and Technology · doi:10.4108/eai.14-9-2021.170951 · Cited by 9
“Later, the model was modified and improved by another member of our group, who has adapted the model in order to do multi-label emotion classification using the CMU-MOSEI dataset [50] for training.”
MethodChampagne: Learning Real-world Conversation from Large-Scale Web Videos
Han, S., Hessel, J., Dziri, N., Choi, Y., & Yu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01421 · Cited by 17
“1); 2) CHAMPAGNE outperforms existing SOTA models on two social interaction understanding benchmarks: CMU-MOSEI [62] and Visual Comet [38] (§4.”
MethodGate-Fusion Transformer for Multimodal Sentiment Analysis
Xie, L., & Zhang, X.-Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59830-3_3 · Cited by 9
Integrating Multimodal Information in Large Pretrained TransformersPreprint
Rahman, W., Hasan, M. K., Lee, S., Zadeh, A., Mao, C., Morency, L., & Hoque, M. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05787 · Cited by 39
“MARN (SOTA3) models view-specific interactions using hybrid LSTM memories and cross-modal interactions us-ing a Multi-Attention Block(MAB) (Zadeh et al., 2018c).”
BackgroundMethodInfluentialSelf-citationCorrespondence Learning for Deep Multi-Modal Recognition and Fraud Detection
Park, J., Kim, M.-H., & Choi, D. (2021) · Electronics · doi:10.3390/electronics10070800 · Cited by 9
“In the CMU-MOSI [20] and CMU-MOSEI [29] datasets, there are consistent performance improvements across multiple baselines [22,23,41] and datasets.”
BackgroundMethodInfluentialTEASEL: A Transformer-Based Speech-Prefixed Language ModelPreprint
Arjmand, M., Dousti, M. J., & Moradi, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05522 · Cited by 21
“This area includes multimodal sentiment analysis [2, 3] and multimodal emotion recognition [4, 5].”
BackgroundScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally DescriptorsPreprint
Luo, H., Ji, L., Huang, Y., Wang, B., Ji, S., & Li, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01368 · Cited by 20
“3) We report new records on three benchmark datasets, including IEMOCAP (Busso et al., 2008), CMU-MOSI (Zadeh et al., 2016), and CMU-MOSEI (Zadeh et al., 2018b).”
InfluentialSparse Fusion for Multimodal TransformersPreprint
Ding, Y., Rich, A., Wang, M. L., Stier, N., Turk, M., Sen, P., & Höllerer, T. (2022) · doi:10.31219/osf.io/f7jvn · Cited by 9
Global Local Fusion Neural Network for Multimodal Sentiment Analysis
Hu, X., & Yamamura, M. (2022) · Applied Sciences · doi:10.3390/app12178453 · Cited by 8
Unraveling the Inner Workings of Massive Language Models
Babu, C. V. S., Anniyappa, C. S. A., & Dharma Sastha, B. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-3860-5.ch008 · Cited by 8
TriSAT: Trimodal Representation Learning for Multimodal Sentiment Analysis
Huan, R., Zhong, G., Chen, P., & Liang, R. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3458812 · Cited by 9
“CMU-MOSEI [19]isanextensivesentimentanalysisdataset, encompassing 22777 opinionated speech videos.”
BackgroundMethodActive Learning with Task Adaptation Pre-training for Speech Emotion Recognition
Li, D., Zhang, Y., Wang, Y., Funakoshi, K., & Okumura, M. (2024) · Journal of Natural Language Processing · doi:10.5715/jnlp.31.825 · Cited by 10
MMCert: Provable Defense Against Adversarial Attacks to Multi-Modal Models
Wang, Y., Fu, H., Zou, W. P., & Jia, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02328 · Cited by 12
“Multi-modal models have shown impressive results across a variety of applications, such as scene understanding [24], object detection [16, 39, 47], sentiment analysis [8, 26, 58, 59], visual question answering [3, 18], and semantic segmentation [11, 31].”
BackgroundGRACE: GRadient-based Active Learning with Curriculum Enhancement for Multimodal Sentiment Analysis
Li, X., Ye, W., Zhang, Y., & Sun, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681617 · Cited by 11
“Given the subjectivity and ambiguity inherent in sentiment analysis, each sample in MSA often requires multiple annotators [46, 47] to derive a reliable average sentiment score, significantly increasing the overall cost of dataset annotation.”
Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis
Wang, N., & Wang, Q. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3702996 · Cited by 9
“The CMU-MOSEI dataset [27], an expanded corpus of movie review videos sourced from YouTube, contains 22,856 segments.”
Multi-level language interaction transformer for multimodal sentiment analysis
Li, Y., Liu, A., & Lu, Y. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00923-x · Cited by 11
Quantum neural networks for multimodal sentiment, emotion, and sarcasm analysis
Singh, J., Bhangu, K. S., Alkhanifer, A. A., AlZubi, A. A., & Ali, F. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.03.023 · Cited by 8
Mamba-assisted modality subspace complementary fusion for multimodal sentiment analysis
Chen, Q., Tang, Y., & Liu, H. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.03.021 · Cited by 9
MMER-LMF: Multi-Modal Emotion Recognition in Lightweight Modality Fusion
Kim, E. H., MyungJin, L., & Shin, J. (2025) · Electronics · doi:10.3390/electronics14112139 · Cited by 8
Zhi, Z., Liu, Z., Elbadawi, M., Daneshmend, A., Orlu, M., Basit, A. W., Demosthenous, A., & Rodrigues, M. R. D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130502 · Cited by 8
Verma, B., Meel, P., & Vishwakarma, D. K. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111195 · Cited by 8
He, X., Pan, Y., Guo, X., Xu, Z., & Yang, C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3580779 · Cited by 8
“To evaluate the performance of our SSGDL model, we utilize three well-established benchmark datasets: CMU-MOSI [36], CMU-MOSEI [37], and CH-SIMS [38].”
CAETFN: Context Adaptively Enhanced Text-Guided Fusion Network for Multimodal Sentiment Analysis
Li, J., Liu, R., Miao, Q., Wang, D., & Liu, X. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3590246 · Cited by 9
“We evaluate the proposed model on two benchmark multi-modal sentiment analysis datasets: CMU-MOSI [72] and CMU-MOSEI [73], respectively.”
Chen, J., Zhong, Y., Huang, Q., Huang, C., Jiang, F., Huang, X., & Wang, X. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3606964 · Cited by 9
“We conduct experiments on four publicly available datasets (CMU-MOSI [29], CMU-MOSEI [30], UR-FUNNY [31], CH-SIMS [32]) to validate the effectiveness of our proposed method.”
Li, H., Han, H., Xu, C., Chen, T., Liu, X., & Wen, G. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114127 · Cited by 11
Рюмина, Е. В., Axyonov, A., Koryakovskaya, D., Abdulkadirov, T., Egorova, A., Fedchin, S., Zaburdaev, A., & Ryumin, D. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8030056 · Cited by 8
Context-Dependent Multimodal Sentiment Analysis Based on a Complex Attention Mechanism
Deng, L., Liu, B., Li, Z., Ma, J., & Li, H. (2023) · Electronics · doi:10.3390/electronics12163516 · Cited by 8
“Ekman emotions {happiness, sadness, anger, fear, disgust, and surprise} are also annotated on a Likert scale of [0, 3] for denoting sentiment x: [no evidence for x, weak x, x, strong x] [20].”
BackgroundMultimodal Fusion Interactions: A Study of Human and Automatic Quantification
Liang, P. P., Cheng, Y., Salakhutdinov, R., & Morency, L. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3577190.3614151 · Cited by 8
“07 for MOSEI, − 0 .”
BackgroundSelf-citationFinding the Right Moment: Human-Assisted Trailer Creation via Task Composition
Papalampidi, P., Keller, F., & Lapata, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3323030 · Cited by 13
“Previous efforts have focused on tweets [65], YouTube opinion videos [66], talkshows [67], and recordings of human interactions [68].”
MMAN-M2: Multiple multi-head attentions network based on encoder with missing modalities
Li, J., Li, L., Sun, R., Yuan, G., Wang, S., & Sun, S. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.11.029 · Cited by 8
Su, X., Huang, Z., Su, Y., Trisedya, B. D., Dou, Y., & Zhao, Y. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3390223 · Cited by 10
Emotion Recognition from Facial Expression Using Deep Learning Techniques
Mishra, P., Verma, A. S., Chaudhary, P., & Dutta, A. (2024) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct61223.2024.10543313 · Cited by 13
Multimodal Sentimental Privileged Information Embedding for Improving Facial Expression Recognition
Sun, N., You, C., Zheng, W., Liu, J., Chai, L., & Sun, H. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3415625 · Cited by 8
“Researchers have developed various multimodal sentiment databases, such as the English database IEMOCAP [13], CMU-MOSI [14], CMU-MOSEI [15], and the Chinese database CH-SIMS [16].”
Metadata
- Authors
- Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- California Miramar University (US)
- Nanyang Technological University (SG)
- Agency for Science, Technology and Research (SG)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 2236-2246
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2883409523
- Cited by
- 1,433
- References
- 62
- Citations by year
- 2026: 281 · 2025: 300 · 2024: 266 · 2023: 202 · 2022: 153 · 2021: 113 · 2020: 73 · 2019: 39 · 2018: 6
- FWCI
- 17.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- CMU-MOSEI, multimodal fusion, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
