The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index, and 1 are not checked yet.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Open accessAbstract
AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This dataset encompasses 3837 videos collected from over 1000 distinctive YouTube speakers, with a reasonably balanced gender distribution of 57% male and 43% female speakers, highlighting the dataset’s focus on inclusivity.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The thematic diversity of the CMU-MOSEI dataset is notable, with a spread across 250 different themes. The three predominant themes are comments (16.2%), debates (2.9%), and consultations (1.8%), while the rest of the themes are nearly evenly distributed.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One of CMU-MOSEI’s unique characteristics is its annotations at the utterance level, with 23,259 samples annotated in this manner.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [30] is a crucial resource in multimodal sentiment analysis and emotion recognition, distinguished by its scale, which is unmatched in the field.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
The cited source says what the claim attributes to it.
In the citing paper
“We generate open-ended AQAs about speech using the four commonly used datasets IEMOCAP [23], MOSEI [24], LibriTTS [25], and Vox-Celeb2 [26,27].”
From Joint Audio and Speech Understanding (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Research works printed but not linked: 3
- Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep-a col- laborative voice analysis repository for speech tech- nologies. In Acoustics, Speech and Signal Process- ing (ICASSP), 2014 IEEE International Conference on. IEEE, pages 960-964.
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmerman. 2018. Memn: Multimodal emotional memory network for emotion recognition in dyadic conversational videos. In NAACL.
- Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Re- current Highway Networks. arXiv preprint arXiv:1607.03474 . · arXiv
Cited by
Sorted by citations
Items 601–700 of 1,003
Semantic Feature Scheduling and Rate Control in Multi-Modal Distributed Network
Gao, H., Yu, G., He, Y., & Liu, Y. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3480352 · Cited by 11
“• CMU-MOSEI [41] is an extension of CMU-MOSI with more than 23 , 400 videos from over 1 , 000 YouTube speakers.”
Human-Guided Modality Informativeness for Affective States
Wörtwein, T., Sheeber, L., Allen, N. B., Cohn, J. F., & Morency, L. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3481004 · Cited by 8
“Similarly, language-driven aspects of affective states haven been predicted as part of sentiment analysis [34, 43] and to some degree as part of dialogue acts [5].”
BackgroundSelf-citationTowards Noise Robust Speech Emotion Recognition Using Dynamic Layer Customization
Wilf, A., & Provost, E. M. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597437 · Cited by 10
MT-TCCT: Multi-task Learning for Multimodal Emotion Recognition
Wang, Y., Chen, Z., Chen, S., & Zhu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15934-3_36 · Cited by 8
Analysis of infoware and software for human affective states recognition
Dvoynikova, A., Markitantov, M., Рюмина, Е. В., Uzdiaev, M., Величко, А. Н., Ryumin, D., Lyakso, E., & Karpov, A. A. (2022) · Informatics and Automation · doi:10.15622/ia.21.6.2 · Cited by 10
“Так создатели корпуса CMU-MOSEI [22] извлекли 12 MFCC, F0, функции сегментации (не)вокализованных звуков, параметры наклона пиков и коэффициенты дисперсии максимумов.”
InfluentialTowards Temporal Event Detection: A Dataset, Benchmarks and Challenges
Yang, Z., Yang, Z., Guo, Z., Lin, Z., Zhu, H., Li, Q., & Liu, W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3276523 · Cited by 9
“…methods include MCTN [45], MFM [66], MISA [46] and MIB [47], sequential network-based fusion meth-ods include EF_LSTM [67], LF_DNN [48], Graph-MFN [68] and MFN [49], tensor-based fusion methods include LMF [51] and TFN [50], transformer-based fusion method include Authorized licensed use limited…”
MethodVoloshina, T., & Makhnytkina, O. V. (2023) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct58615.2023.10143065 · Cited by 9
“During preprocessing broken audio and video sources were removed, CMU - MOSEI (CMU Multimodal Opinion Sentiment and Emotion Intensity) [11] is the largest dataset for emotionality evaluation.”
MethodMORSE: MultimOdal sentiment analysis for Real-life SEttings
Yao, Y., Pérez‐Rosas, V., Abouelenien, M., & Burzo, M. G. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418821 · Cited by 7
“Despite the great improvement brought by existing datasets [33, 53, 54], which are mainly collected in controlled manners, fewer studies have been conducted on multimodal sentiment analysis tasks in real-life settings.”
BackgroundMethodInfluentialVariational Auto-DecoderPreprint
Zadeh, A., Lim, Y. C., Liang, P. P., & Morency, L. (2019) · arXiv (Cornell University) · Cited by 8
Self-citationA dataset for multimodal music information retrieval of Sotho-Tswana musical videos
Oguike, O. E., & Primus, M. (2024) · Data in Brief · doi:10.1016/j.dib.2024.110672 · Cited by 8
A Comprehensive Survey on Multi-Modal Conversational Emotion Recognition with Deep LearningPreprint
Shou, Y., Meng, T., Ai, W., Yin, N., & Li, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5017731 · Cited by 8
Hierarchical Knowledge Stripping for Multimodal Sentiment Analysis
Xiong, A., Zeng, Y., & Hu, H. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3456117 · Cited by 8
“…former focuses on the intra-modal and inter-modal interaction by designing ingenious fusion strategies, such as tensor fusion [7], [8], [9], [10], [11], graph-based fusion [12], [13], [14], attention mechanism-based fusion [15], [16], [17], [18], [19], and prior knowledge-based fusion [20], [21].”
Mutual Information of Crossmodal Utterance Representation for Multimodal Sentiment Analysis
Xufei, Yue, D., & Wei, X. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3466968 · Cited by 9
“CMU-MOSEI [38]: The CMU-MOSEI dataset is an enriched and expanded version of the CMU-MOSI dataset.”
CMDAF: Cross-Modality Dual-Attention Fusion Network for Multimodal Sentiment Analysis
Guo, W., Su, K., Jiang, B., Xie, K., & Liu, J. (2024) · Applied Sciences · doi:10.3390/app142412025 · Cited by 10
MulDeF: A Model-Agnostic Debiasing Framework for Robust Multimodal Sentiment Analysis
Huan, R., Zhong, G., Chen, P., & Liang, R. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521836 · Cited by 8
Privacy-Preserving Multimodal Sentiment Analysis
Xu, H., Li, W., Takabi, D., Seo, D., & Cai, Z. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3527864 · Cited by 7
“CMU-MOSEI (MOSEI) dataset [50] contains 23 453 annotated Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
Multimodal Emotion Recognition Method Based on Domain Generalization and Graph Neural Networks
Xie, J., Wang, Y., Meng, T., Tai, J., Zheng, Y., & Varatnitski, Y. I. (2025) · Electronics · doi:10.3390/electronics14050885 · Cited by 7
“It contains annotations for 7 emotions, ranging from highly negative ( − 3) to highly positive (+3), as well as 6 emotional labels, including happiness, sadness, disgust, fear, surprise, and anger [28].”
Zhang, T., Zhu, Y., Wu, B., Zheng, C., Tan, J., & Xiong, Z. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107314 · Cited by 7
Multi-level Conflict-Aware Network for Multi-modal Sentiment Analysis
Gao, Y., Wu, H., & Zhang, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888597 · Cited by 7
“We evaluate MCAN on CMU-MOSI [22] and CMU-MOSEI [23] datasets, which are the most widely used benchmark for MSA.”
HGTFM: Hierarchical Gating-Driven Transformer Fusion Model for Robust Multimodal Sentiment Analysis
Yang, C., Liang, Z., Yan, D., Hu, Z., & Wu, T. (2025) · IEEE Access · doi:10.1109/access.2025.3560641 · Cited by 7
“GRAPH_MFN [42].”
Chen, M., Liu, J., Li, X., Zhang, Y., Liu, H., Jiao, J., & He, H. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3569089 · Cited by 7
“DFG[22]:DFGcanflexiblymodelinter-modaldependencies .”
CIEG-Net: Context Information Enhanced Gated Network for multimodal sentiment analysis
Chen, Z., Lu, C., & Wang, Y. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111785 · Cited by 8
SSMIM: Semi-supervised Multimodal Interaction Model for real-time conversational emotion recognition
Liu, H., & Zhong, K. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07373-w · Cited by 9
“We evaluated our proposed algorithm on three widely used multimodal datasets: IEMOCAP [55], MELD [56], and CMU-MOSEI [57] both of which include textual, audio, and visual signals.”
Qiu, X., Wei, Y., Tan, X., Xu, W., Wang, H., Ma, J., Huang, J., & Fang, Z. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111917 · Cited by 7
Yang, C., Liang, Z., Liu, T., Hu, Z., & Yan, D. (2025) · Electronics · doi:10.3390/electronics14153088 · Cited by 8
Predicting affective engagement and mental strain from prosodic speech features
Yache, V. P., Moradbakhti, L., Neuner, I., & Veselinović, T. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1656292 · Cited by 6
“Multimodal approaches that integrate textual, acoustic, and visual modalities have demonstrated superior accuracy, particularly in capturing complex affective states during natural interactions (34, 35).”
Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark
Hu, J., Shi, H., Dai, C., Li, Z., Song, P., & Wang, M. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755726 · Cited by 10
“CMU-MOSEI[60] consists of numerous YouTube video clips, each containing at least one recognized emotion, further enriching the multimodal emotion analysis.”
Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities
Luo, Y., Wang, S., Xu, Z., Li, Y., Tang, F., & Su, J. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11208961 · Cited by 8
Sun, N., Zhang, W., Zheng, W., Liu, J., Chai, L., & Wu, C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3612369 · Cited by 7
“The CMU-MOSEI dataset [15] is an extended version of CMU-MOSI, with larger training samples, more speakers, and a wider range of video topics.”
He, X., Pan, Y., Xu, Z., Li, Z., Guo, X., & Yang, C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3614159 · Cited by 7
“To assess the performance of our AL-HCL model, we employ three widely recognized benchmark datasets, including CMU-MOSI [41], CMU-MOSEI [42] and CH-SIMS [45], both of which are used for MSA.”
Systematic literature review on sentiment analysis using transformers
Mostafavi, S., Yahyavi, Y., & Ravanmehr, R. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01021-z · Cited by 7
天津师范大学, 张, 敬., 李, 彦., & 天津师范大学 (2023) · 计算机科学 · doi:10.55375/cps.2023.3.2 · Cited by 7
Beyond Additive Fusion: Learning Non-Additive Multimodal Interactions
Wörtwein, T., Sheeber, L., Allen, N. B., Cohn, J. F., & Morency, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.344 · Cited by 7
“Model-specific quantification : Models can indicate how much they rely on potentially non-additive interactions (Zadeh et al., 2018; Tsai et al., 2020).”
BackgroundSelf-citationAudio-Based Emotion Recognition Using Self-Supervised Learning on an Engineered Feature Space
Nimitsurachat, P., & Washington, P. Y. (2024) · AI · doi:10.3390/ai5010011 · Cited by 7
“These annotations are assigned by 3 crowdsourced judges from the Amazon Mechanical Turk platform who received prior training and received a 98% or higher [20] approval rate to assure high quality annotations.”
BackgroundMethodWang, P., Liu, S., & Chen, J. (2024) · Applied Sciences · doi:10.3390/app14051934 · Cited by 8
Circular Decomposition and Cross-Modal Recombination for Multimodal Sentiment Analysis
Liang, H., Xie, W., He, X., Song, S., & Shen, L. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446166 · Cited by 7
“In this paper, we use two of the most commonly used public datasets for the evaluation, i.e. CMU-MOSI [18] and CMU-MOSEI [19].”
MethodAggregation Design for Personalized Federated Multi-Modal Learning Over Wireless Networks
Yin, B., Chen, Z., & Tao, M. (2024) · IEEE Communications Letters · doi:10.1109/lcomm.2024.3414451 · Cited by 7
“The MOSEI dataset [16] comprises data from seven categories, encompassing visual, audio, and textual modalities.”
MethodDevelopment of multimodal sentiment recognition and understanding
Tao, J., Fan, C., Lian, Z., Lyu, Z., Shen, Y., & Liang, S. (2024) · Journal of Image and Graphics · doi:10.11834/jig.240017 · Cited by 8
Yang, S., Cai, J., & Liao, L. (2024) · Journal of Intelligent Information Systems · doi:10.1007/s10844-024-00858-9 · Cited by 7
“Incaseswherethedatasetlacksspecificuni-modalsentiment Incaseswherethedatasetlacksspecificuni-modalsentiment annotations,suchasMOSIandMOSEI,substitutingmultimodallabelsforuni-modallabels inuni-modaltasklearning,thoughchallenginginfullycapturingsentimentdistinctions,can stillleadtoperformanceimprovements.”
InfluentialMultimodal Emotion Recognition using Deep Convolution and Recurrent Network
Sajid, M., Afzal, M., & Shoaib, M. (2021) · doi:10.1109/icai52203.2021.9445262 · Cited by 7
“Method F1 score Mean Accuracy BaseLine [18] 0.756 - Multiplicative Fusion [19] In this paper, we performed the experiments on the proposed multi-modal architecture for the automatic human emotion recognition.”
MethodInfluentialA Survey of Sentiment Analysis Based on Multi-Modal Information
Xiao, J., & Luo, X. (2022) · Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC) · doi:10.1109/ipec54454.2022.9777333 · Cited by 7
“Zadeh et al. studied an improved gated memory network to store modal internal information and interaction information between modalities [17].”
BackgroundLearning modality-fused representation based on transformer for emotion analysis
Shi, P., Hu, M. Q., Ren, F., Shi, X., & Xu, L. (2022) · Journal of Electronic Imaging · doi:10.1117/1.jei.31.6.063032 · Cited by 7
Self-supervised Cross-modal Pretraining for Speech Emotion Recognition and Sentiment Analysis
Chu, I., Chen, Z., Yu, X., Han, M., Xiao, J., & Chang, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.375 · Cited by 7
“Our pretrained cross-modal transformer achieves the SOTA performance on both the IEMOCAP emotion dataset and CMU-MOSEI sentiment dataset.”
BackgroundMethodInfluentialBranch-Fusion-Net for Multi-Modal Continuous Dimensional Emotion Recognition
Li, C., Xie, L., & Pan, H. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3160373 · Cited by 8
Guhan, P., Agarwal, M., Awasthi, N., Reeves, G. M., Manocha, D., & Bera, A. (2020) · arXiv (Cornell University) · Cited by 7
Evaluating Significant Features in Context-Aware Multimodal Emotion Recognition with XAI MethodsPreprint
Shaikh, T., Khalane, A., Makwana, R., & Ullah, A. (2023) · doi:10.22541/au.167407909.97031004/v1 · Cited by 7
Fu, Y., Fu, J., Xue, H., & Xu, Z. (2024) · Electronics · doi:10.3390/electronics13142835 · Cited by 6
Querying as Prompt: Parameter-Efficient Learning for Multimodal Language Model
Liang, T., Huang, J., Kong, M. C., Chen, L., & Zhu, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02536 · Cited by 7
“Our evaluation metrics, consistent with [55], include mean absolute error (MAE), Pearson correlation (Corr), accuracy (Acc-2), and F1 score.”
MethodInfluentialMultimodal Transformer Fusion for Emotion Recognition: A Survey
Belaref, A., & Séguier, R. (2024) · doi:10.1109/icnlp60986.2024.10692953 · Cited by 6
MECG: modality-enhanced convolutional graph for unbalanced multimodal representations
Tang, J., Ni, B., Yang, Y., Ding, Y., & Kong, W. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06729-y · Cited by 6
“• CMU-MOSEI dataset [19] is an upgraded version of CMU-MOSI on sample number.”
A method for multimodal sentiment analysis: adaptive interaction and multi-scale fusion
Wang, H., Cao, J., Liu, J., & Song, J. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00957-1 · Cited by 6
“The first three datasets (CMU-MOSEI, CMU-MOSI, and CH-SIMSv2) primarily comprise social media content (e.g., YouTube vlogs and user-generated short videos), serving as standard benchmarks for multimodal sentiment analysis in online interaction scenarios.”
InfluentialA comprehensive survey on sentiment analysis: Framework, techniques, and applications
Chandan, M. K., & Mandal, S. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100777 · Cited by 11
QD-MSA : A quantum distributed tensor network framework for multimodal sentiment analysis
Li, Y., Li, Y., Qi, Z., Yang, H., Shang, R., & Jiao, L. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103404 · Cited by 7
Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages
Zou, H., Lv, F., Zheng, D., Chng, E. S., & Rajan, D. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209040 · Cited by 6
“M5SER is a large-scale multilingual speech emotion dataset, consisting of emotional audio signals in five languages, covering seven basic emotions: neutral, happy, sad, angry, surprise, disgust, and fear.”
KUŞ, İ., Koçak, C., & Keleş, A. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200615 · Cited by 11
Busso, C., Lotfian, R., Sridhar, K., Salman, A. N., Lin, W.-C., Goncalves, L., Parthasarathy, S., Naini, A. R., Leem, S.-G., Martinez-Lucas, L., et al. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3678489 · Cited by 6
“…from sources such as col-loquial conversations (SEMAINE [33], RECOLA [9], TUM-AVIC [27]), television programs (VAM [12], MELD [13], CHEAVD [41], UrduSER [35]), the Internet (BIIC-Podcast [15], WHiSER [32], CMU-MOSI [36], CMU-MOSEI [24], CMU-MOSEAS [23]), and customer service calls (CEMO [26]).”
Xie, S., & Li, J. (2024) · International Journal of Information Technologies and Systems Approach · doi:10.4018/ijitsa.335940 · Cited by 6
Prompting Audios Using Acoustic Properties for Emotion Representation
Dhamyal, H., Elizalde, B., Deshmukh, S., Wang, H., Raj, B., & Singh, R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448213 · Cited by 6
“CMU-MOSEI [19] 23K 9 ang, exc, fear, sad frus, neu, sur, hap, dis”
Wang, J. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1449527 · Cited by 6
Li, Y., Weng, W., Liu, C., & Lin, L. (2024) · Pattern Analysis and Applications · doi:10.1007/s10044-024-01320-w · Cited by 6
“In order to evaluate the effectiveness of the CSMF-SPC model proposed in this paper, we use two public multimodal sentiment analysis datasets for comparative experiments: CMU-MOSI [28] and CMU-MOSEI [29].”
Zhang, Z., Xu, W., Dong, Z., Wang, K., Wu, Y., Peng, J., Wang, R., & Huang, D. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3506554 · Cited by 8
Manifold knowledge-guided feature fusion network for multimodal sentiment analysis
Wang, X., Wang, M., Cui, H. N., & Zhang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127537 · Cited by 6
Wang, C., Zhang, Q., Dong, J., Fang, H., Schaefer, G., Liu, R., & Yi, P. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113638 · Cited by 6
Adapting general disentanglement-based speaker anonymization for enhanced emotion preservation
Miao, X., Zhang, Y., Wang, X., Tomashenko, N., Soh, D. C. L., & McLoughlin, I. V. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101810 · Cited by 6
CLIP-driven attention network for multimodal sentiment analysis
Lv, J., Yang, Q., Tian, S., Liu, B., & Yu, L. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07411-7 · Cited by 6
“To validate the generalization capability of the CDAN model across diverse modality combinations and data distributions, we conducted a systematic evaluation on the CMU-MOSI [43] dataset, a benchmark for multimodal sentiment analysis.”
MODIFF-8 to better motivate: Live adaptive human-socially interactive agent interaction
Younsi, N., Pélachaud, C., & Chaby, L. (2026) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-026-00477-4 · Cited by 7
“Facial expressions were extracted using OpenFace [48], a widely used tool for the extraction of social signals [40, 49–52], and encoded based on the Facial Action Coding System (FACS) [11].”
Chakravarthula, S. N., Baucom, B., Narayanan, S. S., & Georgiou, P. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101162 · Cited by 9
“, 2012), a sentence (Zadeh et al., 2018) or a speaker turn (Busso et al.”
BackgroundSelf-Supervised Audio-and-Text Pre-training with Extremely Low-Resource Parallel Data
Kang, Y., Liu, T., Li, H., Hao, Y., & Ding, W. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21334 · Cited by 6
Deep Learning Approaches on Multimodal Sentiment Analysis
Cai, Z., Gao, H., Li, J., & Wang, X. (2022) · IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA) · doi:10.1109/eebda53927.2022.9745018 · Cited by 6
“For example, the Gate-Fusion Transformer may perform well on CMU-MOSI and CMU-MOSEI, but it may have a bad performance on other datasets.”
MethodResultInfluentialTopic and Style-aware Transformer for Multimodal Emotion Recognition
Qiu, S., Sekhar, N., & Singhal, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.130 · Cited by 8
“The Memory Fusion Network synchronizes multi-modal sequences using a multi-view gated memory that stores intra-view and cross-view interactions through time (Zadeh et al., 2018).”
BackgroundAnalysis of Emotion Annotation Strength Improves Generalization in Speech Emotion Recognition Models
Palotti, J., Narula, G., Raheem, L., & Bay, H. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00619 · Cited by 6
“• CMU-MOSEI [14, 18]: the CMU Multimodal Opinion Sentiment and Emotion Intensity dataset is a largescale multimodal dataset for sentiment and emotion analysis in videos.”
BackgroundMethodInfluentialModelling conversational agent with empathy mechanism
Dai, X., Liu, Z., Liu, T., Zuo, G., Xu, J., Shi, C., & Wang, Y. (2023) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101206 · Cited by 6
Lu, C., & Fu, X. (2024) · IEEE Access · doi:10.1109/access.2024.3363028 · Cited by 7
“An extension of CMU-MOSI, the CMU-MOSEI dataset [33] maintains identical annotation schema.”
MethodFang, C., Liang, F., Li, T., & Guan, F. (2024) · Future Internet · doi:10.3390/fi16060213 · Cited by 6
From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation
Agarwal, D., Agrawal, T., Ferrari, L. M., & Brémond, F. (2021) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss52988.2021.9663793 · Cited by 6
PT-CoDE: Pre-trained Context-Dependent Encoder for Utterance-level Emotion RecognitionPreprint
Jiao, W., Lyu, M. R.-T., & King, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.08916 · Cited by 9
“Generally, the learning rate of 2 × 10 − works well 1 by (Luo et al., 2018); 2 by (Khosla, 2018). for all the datasets except MOSEI ∗ , on which we find 5 × 10 − 5 works better.”
InfluentialM5Product: A Multi-modal Pretraining Benchmark for E-commercial Product Downstream TasksPreprint
Dong, X. M., Zhan, X., Wu, Y., Wei, Y., & Liang, X. (2021) · arXiv (Cornell University) · doi:10.13140/rg.2.2.35620.81281 · Cited by 6
Deep Emotion Recognition using Facial, Speech and Textual Cues: A SurveyPreprint
Zhang, T., & Tan, Z. (2021) · doi:10.36227/techrxiv.15184302.v2 · Cited by 6
Granformer: A granular transformer net with linear complexity
Wang, K., Sun, X., & Shen, T. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128380 · Cited by 5
Multimodal Sentiment Analysis for the Malay Language: New Corpus using CNN-based Framework
Taylor, S., & Fauzi, F. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3703445 · Cited by 5
“In this section, we introduce our new Malay Multimodal Sentiment corpus, MyMSC, with unimodal (text) and multimodal level annotations in detail.”
Beyond Single Emotion: Multi-label Approach to Conversational Emotion Recognition
Kang, Y., & Cho, Y.-S. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i23.34609 · Cited by 5
“Particularly, multi-label emotion classification (ER) has seen active research progress (Alhuzali and Ananiadou 2021; Lin et al. 2023) with the release of datasets, which provide multi-label emotion annotations for tweets (Mohammad et al. 2018) and multimodal language (Zadeh et al. 2018).”
Prompt-matching synthesis model for missing modalities in sentiment analysis
Liu, J., Wang, Y., Yang, J., Shang, F., & He, F. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113519 · Cited by 7
Emotion recognition in live broadcasting: a multimodal deep learning framework
Abbas, R., Schuller, B. W., Li, X., Lin, C., & Li, X. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01780-y · Cited by 5
“Furthermore, we conducted performance evaluation of this method using the FIFA, League of Legends [57], IEMOCAP [58], and CMU-MOSEI [59] datasets in comparison to various existing methods.”
Cyclic Data Distillation Semi-Supervised Learning for Multi-Modal Emotion Recognition
Li, S., Zhang, T., & Chen, C. L. P. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3581786 · Cited by 6
“8: end for 9: Assign k = k ini 10: while L v decreases within ten consecutive epochs do 11: for i = 1 to r do 12: 13: P U = f MAE ( ¯ P U 1 , . . . , ¯ P UJ ) 14: [67] is a large multi-modal sentiment analysis dataset.”
Adaptive Graph Learning with Multimodal Fusion for Emotion Recognition in Conversation
Liu, J., Li, J., Dong, J., Mo, Z., Liu, N., Li, Q., & Yuan, Y. (2025) · Biomimetics · doi:10.3390/biomimetics10070414 · Cited by 8
“We selected two widely used datasets for emotion recognition: the Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset [7] and the Multimodal Opinion Sentiment and Emotion Intensity (MOSEI) dataset [6].”
Self-attention mechanism prior to modality fusion for multimodal sentiment analysis
Chen, Z., Lu, C., & Wang, Y. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01868-5 · Cited by 5
“CMU-MOSEI [35]: The CMU-MOSEI dataset is an expanded version of CMU-MOSI, offering a larger scale and richer content.”
He, X., Pan, Y., Chen, Y., Li, Z., Xu, Z., Yang, C., & Wang, K. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3597474 · Cited by 5
“To assess the performance of our AL-ARCF model, we employ three widely recognized benchmark datasets: the English multimodal affective computing datasets CMU-MOSI [35] and CMU-MOSEI [36], along with the Chinese MSA dataset CH-SIMS [50].”
Gradient and Structure Consistency in Multimodal Emotion Recognition
Shi, Q., Ye, M., Huang, W., Du, B., & Zong, X. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3608664 · Cited by 5
“In consideration of fair comparison [10], [13] and privacy concerns [54], [57], all modal information utilized in the experiment is extracted features.”
Affection-Guided Bottleneck Diffusion for Missing Modality Issue in Multimodal Affective Computing
Lin, R., He, Q., Zeng, Y., Mai, S., Huang, L., & Hu, H. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3632063 · Cited by 7
“CMU-MOSEI [84] significantly extends the dataset size, including 20,000 video clips segmented from 3,228 videos covering 250 diverse topics.”
Meng, J., Zhu, Z., Lu, Q., Pei, H., Li, K., Ren, Y., Li, M. (., Sun, X., & Zhao, D. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115438 · Cited by 5
TPFN: A Text-Guided Progressive Fusion Network for Multimodal Sentiment Analysis
Ou, L., & Li, Z. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3663280 · Cited by 5
“We evaluate our approach on four widely used Multimodal Sentiment Analysis (MSA) datasets: CMU-MOSI [34], CMU-MOSEI [35], CH-SIMS [36] and IEMOCAP [37].”
Cai, L., Liu, D., Liu, L., & Mou, R. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3678350 · Cited by 7
“CMU-MOSEI [39] is an extension of CMU-MOSI, comprising 22 856 YouTube video clips with audio, text, and video data.”
SGIR: Star Graph-Based Interaction for Efficient and Robust Multimodal Representation
Ding, Z., Lan, G., Song, Y., & Yang, Z. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3321404 · Cited by 5
Xu, G., Deng, L., Li, Y., Wei, Y., Zhou, X., & Zhu, H. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3324462 · Cited by 5
“MOSEI dataset [39] is an upgraded version of MOSI dataset, which has three modes: text, speech and image.”
MethodMultimodal sentiment analysis using Multi-Layer Fusion Convolution Neural Network
Basha, D. K., Sunil, G., Hussein, A. H. A., Mukesh, S., & Adnan, M. M. (2023) · International Conference on Ambient Intelligence, Knowledge Informatics and Industrial Electronics (AIKIIE) · doi:10.1109/aikiie60097.2023.10390433 · Cited by 5
“This research utilized two multimodal sentiment analysis datasets obtained from two benchmarks such as CMU-MOSI [21] and CMU-MOSEI [22] 1) CMU-MOSI: In multimodal sentiment analysis, the CMU-MOSI dataset is frequently utilized.”
MethodMultimodal Sentiment Analysis Based on 3D Stereoscopic Attention
Huang, J., Pu, Y., Zhou, D., Shi, H., Zhao, Z., Xu, D., & Cao, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446572 · Cited by 5
“The CMU-MOSEI [17], which is the extended version of the CMU-MOSI dataset, consists of 22,856 annotated video clips from more than 1000 online speakers and 250 different topics.”
BackgroundFusing Modality-Specific Representations and Decisions for Multimodal Emotion Recognition
Ruan, Y.-P., Han, S., Li, T., & Wu, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447035 · Cited by 5
“…of user-generated multimodal online content, the topic of multimodal emotion recognition (MER) has attracted considerable research attention [6, 7, 8], which can generally achieve better performance than those based on a single modality, due to the complementary information of different…”
BackgroundHwang, Y., & Kim, J.-H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00557 · Cited by 5
“CMU-MOSEI [59] dataset contains 23,453 annotated video segments from 1,000 speakers addressing 250 different topics.”
BackgroundMTFR: An universal multimodal fusion method through Modality Transfer and Fusion Refinement
Guo, X., Tian, S., Yu, L., He, X., & Wang, Z. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108844 · Cited by 5
Metadata
- Authors
- Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- California Miramar University (US)
- Nanyang Technological University (SG)
- Agency for Science, Technology and Research (SG)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 2236-2246
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2883409523
- Cited by
- 1,433
- References
- 62
- Citations by year
- 2026: 281 · 2025: 300 · 2024: 266 · 2023: 202 · 2022: 153 · 2021: 113 · 2020: 73 · 2019: 39 · 2018: 6
- FWCI
- 17.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- CMU-MOSEI, multimodal fusion, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
