The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index, and 1 are not checked yet.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Open accessAbstract
AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This dataset encompasses 3837 videos collected from over 1000 distinctive YouTube speakers, with a reasonably balanced gender distribution of 57% male and 43% female speakers, highlighting the dataset’s focus on inclusivity.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The thematic diversity of the CMU-MOSEI dataset is notable, with a spread across 250 different themes. The three predominant themes are comments (16.2%), debates (2.9%), and consultations (1.8%), while the rest of the themes are nearly evenly distributed.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One of CMU-MOSEI’s unique characteristics is its annotations at the utterance level, with 23,259 samples annotated in this manner.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [30] is a crucial resource in multimodal sentiment analysis and emotion recognition, distinguished by its scale, which is unmatched in the field.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
The cited source says what the claim attributes to it.
In the citing paper
“We generate open-ended AQAs about speech using the four commonly used datasets IEMOCAP [23], MOSEI [24], LibriTTS [25], and Vox-Celeb2 [26,27].”
From Joint Audio and Speech Understanding (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Research works printed but not linked: 3
- Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep-a col- laborative voice analysis repository for speech tech- nologies. In Acoustics, Speech and Signal Process- ing (ICASSP), 2014 IEEE International Conference on. IEEE, pages 960-964.
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmerman. 2018. Memn: Multimodal emotional memory network for emotion recognition in dyadic conversational videos. In NAACL.
- Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Re- current Highway Networks. arXiv preprint arXiv:1607.03474 . · arXiv
Cited by
Sorted by citations
Items 701–800 of 1,003
Evaluation of Data Inconsistency for Multi-modal Sentiment Analysis
Wang, Y., & Wu, M. (2024) · Communications in computer and information science · doi:10.1007/978-981-96-1045-7_25 · Cited by 10
Sentiment Analysis and Rating Video Game Dimensions via NLP
Yuan, L., Ding, M., Meng, F., & Tian, Y. (2025) · Applied and Computational Engineering · doi:10.54254/2755-2721/2024.20540 · Cited by 5
UniC: a dataset for emotion analysis of videos with multimodal and unimodal labels
Du, Q., Labat, S., Demeester, T., & Hoste, V. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09837-0 · Cited by 5
A dynamic weighted fusion model for multimodal sentiment analysis
Yang, L. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04201-9 · Cited by 5
“Finally, 709-dimensional frame-level visual features are extracted by CH-SIMS [26], 35-dimensional by CMU-MOSI [27] and CMU-MOSEI [28].”
Hierarchical Multimodal Fusion Network with Dynamic Multi-task Learning
Wang, T., & Chen, S. (2021) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri51335.2021.00034 · Cited by 5
“Zadeh et al. tried to use a Dynamic Fusion Graph (DFG) to model the n-modal dynamics [23].”
MethodER-MRL: Emotion Recognition based on Multimodal Representation Learning
Guo, X., Wang, Y., Miao, Z., Yang, X., Guo, J., Hou, X., & Zao, F. (2022) · International Conference on Information Science and Technology · doi:10.1109/icist55546.2022.9926848 · Cited by 5
“• Graph-based fusion model: Graph-MFN [29].”
Generic Attention-model Explainability by Weighted Relevance Accumulation
Huang, Y., Jia, A., Zhang, X., & Zhang, J. (2023) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3595916.3626437 · Cited by 7
“GenAtt [6] further considers the equivalent attention across heads to avoid distorted relevance by introducing AttGrad [21] to balance attention gradient and values in relevance computation.”
BackgroundTravel Agency Task Dialogue Corpus: A Multimodal Dataset with Age-Diverse Speakers
Inaba, M., Chiba, Y., Qi, Z., Higashinaka, R., Komatani, K., Miyao, Y., & Nagai, T. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3675166 · Cited by 5
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) [32] is one of the largest, containing 23,500 YouTube videos of 1,000 people, each utterance of which is annotated with an emotion label.”
BackgroundA Review of Multimodal Sentiment Analysis: Modal Fusion and Representation
Fu, H., & Lu, H. (2024) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc61514.2024.10592484 · Cited by 6
“An expansion of MOSI, the MOSEI dataset [50] comprises 23454 YouTube videos that were taken at a 20 Hz sampling rate for audio signals and a 15 Hz sampling rate for visual signals.”
MethodMulti-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition
Halawa, M., Blume, F., Bideau, P., Maier, M., Rahman, R. A., & Hellwich, O. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00463 · Cited by 9
Teng, S., Chai, S., Liu, J., Tateyama, T., Lin, L., & Chen, Y. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782904 · Cited by 5
“For the second challenge: we pre-train our encoder on the large-scale emotional dataset CMU-MOSEI[8].”
Luo, Y., Wang, M., & Ye, Q. (2024) · Electronic Commerce Research · doi:10.1007/s10660-024-09937-z · Cited by 5
“According to Zadeh et al. [65], the MultiComp OpenFace2.”
Nie, X., Yang, L. T., Li, Z., Deng, X., Fan, F., & Yang, Z. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3459929 · Cited by 6
Learning-Based Confidence Estimation for Multi-modal Classifier Fusion
Nadeem, U., Bennamoun, M., Sohel, F. A., & Togneri, R. B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-36711-4_26 · Cited by 5
Interpretable Multimodal Routing for Human Multimodal Language.
Tsai, Y.-H. H., Martín, Q. M., Yang, M., Salakhutdinov, R., & Morency, L. (2020) · Cited by 4
Self-citationMulti-modal Emotion Recognition for User Adaptation in Social Robots
Schiffmann, M., Thoma, A., & Richert, A. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-79997-7_16 · Cited by 5
AtM-DNN: A Multimodal Attention Fusion Network with Auxiliary Function for Sentiment Classification
Ji, H., & Xu, X. (2022) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd54268.2022.9776143 · Cited by 5
Towards Improvement of Grounded Cross-lingual Natural Language Inference with VisioTextual Attention
De, A., Desarkar, M. S., & Ekbal, A. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100023 · Cited by 6
Yüksel, K. A., Ferreira, T., Gündüz, A., Al-Badrashiny, M. A. S. A. A., & Javadi, G. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193003 · Cited by 5
“The referenceless metric was trained and validated on a large corpus that combined unique outputs from all publicly available compression levels of OpenAI’s Whisper ASR model [1] for each audio sample available at CMU MOSEI and MOSEAS datasets [16, 17] containing a total of 134 hours of speech from Youtube videos of 2…”
MethodMulti-Corpus Learning for Audio–Visual Emotions and Sentiment Recognition
Рюмина, Е. В., Markitantov, M., & Karpov, A. A. (2023) · Mathematics · doi:10.3390/math11163519 · Cited by 5
Enhancing Sentence Representation with Visually-supervised Multimodal Pre-training
Li, Z., Yang, L. T., Nie, X., Ren, B., & Deng, X. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612254 · Cited by 5
“Besides we perform multimodal sentiment classification on the MOSEI dataset [48], which includes 23,453 videos of utterances and has two distinct tasks: sentiment analysis and emotion recognition.”
MethodDoes Visual Self-Supervision Improve Learning of Speech Representations?Preprint
Shukla, A., Petridis, S., & Pantić, M. (2020) · arXiv (Cornell University) · Cited by 5
Variational Auto-Decoder: A Method for Neural Generative Modeling from Incomplete DataPreprint
Zadeh, A., Lim, Y.-C., Liang, P. P., & Morency, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00840 · Cited by 22
“Lowest achieved MSE for r = 0 (no missingness) in CMU-MOSEI and CMU-MOSI are respectively 001975 , 0 .”
InfluentialSelf-citationMultimodal End-to-End Group Emotion Recognition using Cross-Modal AttentionPreprint
Evtodienko, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05890 · Cited by 12
“answering [16], image captioning [17], emotion recognition [18], affect recognition [19].”
A Unified Transformer-based Network for Multimodal Emotion RecognitionPreprint
Ali, K., & Hughes, C. E. (2023) · doi:10.36227/techrxiv.23916123.v1 · Cited by 5
Cai, T., Wang, S., Wang, J., Xiong, Y., & Liu, L. (2025) · Frontiers of digital education. · doi:10.1007/s44366-025-0063-x · Cited by 5
“Graph-MFN replaces the original fusion component in the memory fusion network with the new dynamic fusion graph to achieve better interpretability and efficiency (Zadeh et al., 2018).”
Li, Z., Liu, Y., Yang, C., Zhou, Y., & Hu, S. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3590929 · Cited by 5
“The objective of the MER task is to predict the emotion label in a video, which may be a rating label of positive/negative degree [9], [10], [11], [12], [13], [14], or a fine-grained label such as happiness, anger, sadness, etc [15], [16], [17], [18].”
State-of-the-art Multimodal Emotion Recognition: A comprehensive survey and taxonomy
Yazici, A., Kucukyilmaz, T., Dokeroglu, T., Sharipbay, A., Lee, M.-H., & Tyler, B. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200642 · Cited by 11
Dictionary Multi-Modal Temporal Graph Learning
Liu, M., Liang, K., Li, M., Zhu, X., & Liu, X. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3679419 · Cited by 4
“2) Some methods use the concepts of temporal / dynamic graph learning to model other types of data, which is essentially a cross-application of graph learning rather than focusing on graph data itself [49], [50].”
MISSRAG: Addressing the Missing Modality Challenge in Multimodal Large Language Models
Pipoli, V., Saporita, A., Bolelli, F., Cornia, M., Baraldi, L., Grana, C., Cucchiara, R., & Ficarra, E. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00308 · Cited by 8
“…of our findings through extensive experiments on five multimodal datasets, covering a broad spectrum of tasks such as, audio-visual question answering (MUSIC-AVQA [27]), audio-visual captioning (VALOR-32K [11], CharadesEgo [43]), and audio-video-text sentiment analysis (MOSI [50], MOSEI [3]).”
Li, H., Baucom, B., Narayanan, S. S., & Georgiou, P. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101226 · Cited by 4
Frame-level nonverbal feature enhancement based sentiment analysis
Zheng, C., Peng, J., Wang, L., Zhu, L., Guo, J., & Cai, Z. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125148 · Cited by 4
Zhao, X., Miao, X., Xu, X., Liu, Y., & Cao, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650083 · Cited by 4
“For example, [23] proposed Graph-MFN regards each modality as one node and uses a dynamic fusion graph to fuse features.”
MethodOptimizing Interactive Online Education Through Advanced Multimodal Emotion Recognition Techniques
Zhou, K., Li, D., & Alataş, B. (2024) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.355765 · Cited by 4
MEAS: Multimodal Emotion Analysis System for Short Videos on Social Media Platforms
Wei, Q., Zhou, Y., Xiang, S., Xiao, L., & Zhang, Y. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3490846 · Cited by 9
“(cid:1) (cid:1) CMU-MOSI [5] (cid:1) (cid:1) CMU-MOSEI [4] (cid:1) (cid:1) UR-FUNNY [6] (cid:1) (cid:1) CH-SIMS [7] (cid:1) (cid:1) CMU-MOSEAS [3] (cid:1) (cid:1) SV-FEND.”
Mori, E., Qiu, Y., Kataoka, H., & Aoki, Y. (2025) · Sensors · doi:10.3390/s25020477 · Cited by 4
“Similarly, advancements in vision-based AI have enhanced emotion recognition and the analysis of social interactions, employing datasets [15–17] that scrutinize human behavior through images and videos.”
InfluentialAMRE: Adaptive Multilevel Redundancy Elimination for Multimodal Mobile Inference
Cai, Q., Chu, R., Zhang, K., Liu, X., Tong, X., Xie, X. N., Chen, J., & Li, K. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3549422 · Cited by 4
Fu, Y., Liu, Q., Song, Q., Zhang, P., & Liao, G. (2025) · Applied Sciences · doi:10.3390/app15084509 · Cited by 4
Bimodal Sentiment Analysis Based on a Pre-Trained Model and Masked Attention Fusion
Cai, L., Li, H., Zhang, N., & He, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3564826 · Cited by 4
“MFN [34]: It uses the CMU-MOSEI dataset and an inter-pretable dynamic fusion graph to perform sentiment analysis and understanding with multimodal information.”
SetPeER: Set-Based Personalized Emotion Recognition With Weak Supervision
Tran, M., Yin, Y., & Soleymani, M. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3568024 · Cited by 4
“Specifically, we apply the labeling process to two well-established emotion datasets, CMU-MOSEI [27], and MSP-Face [26], and compare the generated weak labels against the ground-truth annotations provided by these datasets.”
InfluentialA Multimodal Sentiment Analysis Approach Based on Multiview Cross-Modal Fusion
Zhi, Y., Li, J., Wang, H., & Chen, J. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3583711 · Cited by 5
“We conduct experiments on two multimodal benchmarks CMU-MOSI [50] and CMU-MOSEI [51].”
Huang, D., Gao, R., Zhang, G., & Yu, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129750 · Cited by 4
Intra-Sample and Intra-Modal Enhancement for Multimodal Sentiment Analysis With Missing Modalities
Yan, Z., Zhang, Y., Deng, J., & Ren, F. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3645559 · Cited by 4
“For regression tasks, which include the MOSI [3] and MOSEI [47] dataset, L1loss is adapted as in prior research [29].”
Audio-Visual Speech Emotion Recognition by Disentangling Emotion and Identity Attributes
Ito, K., Fujioka, T., Sun, Q., & Nagamatsu, K. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-809 · Cited by 4
“In particular, the studies by [11, 12, 13] proposed SER combining tri-modal (audio, visual, and lexical) training on the CMU-MOSEI dataset [14], which is composed of speakers’ videos from YouTube.”
MethodUnimodal and Crossmodal Refinement Network for Multimodal Sequence Fusion
Guo, X., Kong, A. W., Zhou, H., Wang, X., & Wang, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.720 · Cited by 4
“MARN (Zadeh et al., 2018b) To extract complementary information and to perform multimodal sequence fusion, many meth-ods have investigated the fusion strategies.”
BackgroundMethodInfluentialMultiViz: Towards User-Centric Visualizations and Interpretations of Multimodal Models
Liang, P. P., Lyu, Y., Chhablani, G., Jain, N., Deng, Z., Wang, X., Morency, L.-P., & Salakhutdinov, R. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3585604 · Cited by 4
Self-citationPerspective Chapter: Embracing the Complexity of Human Emotion
Thabit, S. A. (2024) · Artificial intelligence · doi:10.5772/intechopen.112701 · Cited by 5
Person re-identification method based on fine-grained feature fusion and self-attention mechanism
Yin, K., Ding, Z., Dong, Z., Ji, X., Wang, Z., Chen, D., Li, Y., Yin, G., & Wang, Z. (2024) · Computing · doi:10.1007/s00607-024-01270-5 · Cited by 4
“Amir et al. [24] proposed a multi-feature fusion method, tensor fusion network (TFN), to cross learn the dynamic features within modes, effectively solving the dynamic modeling within modes, and achieving SOTA on CMU-MOSI dataset [31].”
Ma, L., Li, J., Shao, D., Yan, J., Wang, J., & Yan, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3405586 · Cited by 4
“, MOSI [10], and SIMS [11] training samples, the distributions of the top 5 most common words in MOSEI, MOSI, and SIMS, as well as the predictions of Self-MM [12], the MAG-BERT [13] model, and ours for the test samples. highlight the exceptional generalizability and debiasing capabilities of BCD-MM.”
MethodInfluentialA Cross-Modal Transformer Based Model for Box-office Revenue Prediction
Madongo, C. T., Tang, Z., & Hassan, J. (2024) · Journal of Advances in Information Technology · doi:10.12720/jait.15.7.822-837 · Cited by 5
“The CMU-MOSEI dataset [36], which contains 23,500 sentence expressions, was used to fine-tune our textual model.”
MethodChen, Q., Dong, S., & Wang, P. (2024) · Symmetry · doi:10.3390/sym16070934 · Cited by 4
“The pervasiveness of incomplete datasets often compromises the endeavor to decipher sentiment accurately from multimodal data.”
BackgroundSmartRAN: Smart Routing Attention Network for multimodal sentiment analysis
Guo, X., Tian, S., Yu, L., & He, X. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05839-7 · Cited by 4
“We conducted experiments on two widely used MSA benchmark datasets: CMU-MOSI [41] and CMU-MOSEI [42].”
Meng, J., Zhu, Z., Qi, J., & Zhang, H. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06524-9 · Cited by 4
“However, for the CMU-MOSI [16] and CMU-MOSEI [17] datasets lacking unimodal annotations, it used modality-specific representations to predict multi-modal labels in the unimodal prediction subtask, which leads to insufficient learning of differentiated information among modalities.”
Hu, C., Liu, J., Li, X., Li, M., & He, H. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06588-7 · Cited by 4
“CMU-MOSEI [28] is proposed based on CMU-MOSI, which contains 22,856 video clips from YouTube.”
InfluentialGated Fusion Network With Progressive Modality Enhancement for Sentiment Analysis
Feng, X., Gong, W., Shi, T., Jing, Z., Li, K., & Gonzàlez, J. (2024) · IEEE Access · doi:10.1109/access.2024.3490772 · Cited by 4
Semantic-Driven Crossmodal Fusion for Multimodal Sentiment Analysis
Liu, P., Wang, Z., & Huang, F. J. (2024) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.359985 · Cited by 4
Text-dominant strategy for multistage optimized modality fusion in multimodal sentiment analysis
Wu, J., Wang, J., Jing, S., Liu, J., Zhang, T., Han, M., Zhan, P., & Zuo, G. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01518-2 · Cited by 4
“This paper utilizes the CMU-MOSI [32] and CMU-MOSEI [33] datasets.”
Multi-Task Supervised Alignment Pre-Training for Few-Shot Multimodal Sentiment Analysis
Yang, J., Cao, J., & Duan, C. (2025) · Applied Sciences · doi:10.3390/app15042095 · Cited by 4
SKE-MSA: Enhancing Representation Learning with VAD Lexicon for Multimodal Sentiment Analysis
Yang, Y., Dong, X., Qiang, Y., & Si, W. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890333 · Cited by 4
“Extensive experiments were conducted on three popular datasets: CMU-MOSI [18], CMU-MOSEI [19], and CH-SIMS [20].”
Weng, Y., Wang, H., Gao, T., Li, K., Niu, S., & Du, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889029 · Cited by 4
“We test our model on the CMU-MOSEI [26] dataset, which includes 22,856 videos: 16,326 for training, 1,871 for validation, and 4,659 for testing.”
Chen, G., Qian, Z., Qiu, S., Zhang, D., & Zhou, R. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105322 · Cited by 5
Sentiment analysis survey: datasets, techniques, applications, tools, and challenges
Hankar, M., Mzili, T., Kasri, M., & Beni‐Hssane, A. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02499-y · Cited by 6
Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy
Pattnayak, P., Patel, H. L., Kumar, B., Agarwal, A., Banerjee, I., Panda, S., & Kumar, T. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-14041-8_4 · Cited by 30
Lightweight Models for Multimodal Sequential Data.
Sourav, S., & Ouyang, J. (2021) · Cited by 4
A New View of Multi-modal Language Analysis: Audio and Video Features as Text “Styles”
Sun, Z., Sarma, P. K., Liang, Y., & Sethares, W. A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.167 · Cited by 4
“On CMU-MOSI and CMU-MOSEI, binary accuracy, weighted F1 score, mean absolute error, 7-class accuracy, and correlation with human labels are reported.”
BackgroundMethodInfluentialLeveraging Sentiment Analysis Knowledge to Solve Emotion Detection Tasks
Nguyen-The, M., Lamghari, S., Bilodeau, G.-A., & Rockemann, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37660-3_29 · Cited by 4
Emotion-aware Human–Robot Interaction and Social Robots
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502407 · Cited by 4
An Enroll-to-Verify Approach for Cross-Task Unseen Emotion Class Recognition
Li, J.-L., & Lee, C.-C. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3183166 · Cited by 4
“…[20] uncertainty [21] inter-category relationship [22] task transfer [11] cross-task verification (Ours) Task dimension [16] category [14], [15] dimension [18], [20] category [9], [17], [19] dimension [21] category [22] dimension ! category [11] category ! new category (Ours) Cross…”
BackgroundBayesian mixture variational autoencoders for multi-modal learning
Liao, K.-T., Huang, B., Yang, C.-C., & Lin, S.-D. (2022) · Machine Learning · doi:10.1007/s10994-022-06272-y · Cited by 4
“The selected datasets are CMU-MOSI (Zadeh et al., 2016), UR-FUNNY (Hasan et al., 2019), CMU-MOSEI (Bagher Zadeh et al., 2018), MUSTARD (Castro et al., 2019) and AV-MNIST (Vielzeuf et al., 2018).”
MethodPUMAVE-D: panjab university multilingual audio and video facial expression dataset
Singh, L., Aggarwal, N., & Singh, S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14102-5 · Cited by 5
“However, the majority of researchers have not evaluated the consistency in annotator responses during the subjective assessment of recorded audio-video clips [1, 3, 8–10, 12, 14, 16, 19, 20, 23, 24, 36, 41].”
BackgroundInfluentialSelecting Language Models Features VIA Software-Hardware Co-Design
Pandelea, V., Ragusa, E., Gastaldo, P., & Cambria, E. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097191 · Cited by 4
“We perform the experiments on the two datasets, CMUMOSEI [18] and MELD [19].”
MethodSelf-citationMultimodal Sentiment Analysis Method Based on Multi-task Learning
Li, J., & Zhao, H. (2023) · doi:10.1145/3614008.3614055 · Cited by 4
“The CMU-MOSEI [15] dataset consists of 3,229 videos covering a corpus of 23,000 utterances from over 1,000 online YouTube speakers.”
Xiao, S., Wang, J., Wang, J., Chen, R., & Chen, G. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103534 · Cited by 4
Yang, J., Li, Z., & Du, X. (2023) · Data Technologies and Applications · doi:10.1108/dta-08-2023-0414 · Cited by 4
Nguyen, C.-V. T., Nguyen, H.-S., Le, D.-T., & Ha, Q.-T. (2023) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse59128.2023.10299456 · Cited by 4
“We consider the CMU-MOSI [16] and CMU-MOSEI [1] datasets for Multimodal Sentiment Analysis task, and MELD [9] for Multimodal Emotion Recognition task.”
MethodYang, Z., Li, Z., Zhu, D., & Zhou, Y. (2024) · Information Technology And Control · doi:10.5755/j01.itc.53.1.35060 · Cited by 4
Multi-modal Arousal and Valence Estimation under Noisy Conditions
Dresvyanskiy, D., Markitantov, M., Yu, J., Kaya, H., & Karpov, A. A. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00480 · Cited by 4
“We selected the best-performing audio and visual models, extracted embeddings, and experimented with the fusion of modality-specific features based on a cross-attention mechanism inspired by [2, 60].”
RISC: A Corpus for Shout Type Classification and Shout Intensity Prediction
Fukumori, T., Ishida, T., & Yamashita, Y. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3473302 · Cited by 4
“…shout intensity of each speech sample on a seven-point scale from 1 (not a shout at all) to 7 (very shout-like), following prior works regarding speech evaluation through listening experiments [39], [40], [41] and subjective evaluation experiments based on crowdsourcing services [42], [43], [44].”
BackgroundAn Improved Model of Multi-attention LSTM for Multimodal Sentiment Analysis
Wang, A., Sun, B., & Jin, R. (2019) · doi:10.1145/3374587.3374606 · Cited by 5
“We evaluate our proposed approach on two benchmark datasets, namely CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus [17] and the recently published CMU Multi-modal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [18] .”
MethodLDNN: Linguistic Knowledge Injectable Deep Neural Network for Group Cohesiveness Understanding
Wang, Y., Wu, J., Huang, J., Hattori, G., Takishima, Y., Wada, S., Kimura, R., Chen, J., & Kurihara, S. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418830 · Cited by 4
“[20, 32] proposed an emotion recognition method that attempts to achieve human-like emotion recognition by integrating features extracted from visual, audio and linguistic data.”
BackgroundMethodCoupled Systems for Modeling Rapport Between Interlocutors
Sharma, S., Gangadhara, K. G., Fei, X., Slowe, A. S., Frank, M. G., & Nwogu, I. (2021) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg52635.2021.9667067 · Cited by 4
Towards Multimodal Sentiment Analysis Inspired by the Quantum Theoretical Framework
Li, Q., Stefani, A., Toto, G., Buccio, E. D., & Melucci, M. (2020) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr49039.2020.00044 · Cited by 4
Wilf, A., & Provost, E. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11226 · Cited by 5
“MOSEI contains 23,500 utterances extracted from “in the wild” videos on Youtube, labelled for sentiment in the range -3 to 3 [19].”
BackgroundEnhancing Multimodal Affective Analysis with Learned Live Comment Features
Deng, Z., Ananthram, A., & McKeown, K. R. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i15.33785 · Cited by 5
Multilevel Representation Disentanglement Framework for Multimodal Sentiment Analysis
Jia, N., Bai, Z., Xiong, T., & Guo, M. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3562827 · Cited by 3
“MOSEI [17] is a dataset containing 22,856 movie review video clips from YouTube.”
Guo, H., & Akyildiz, I. F. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3567880 · Cited by 4
“In this paper, we evaluate the trade-offs among the task accuracy, end-to-end latency, and energy consumption.”
Singh, H. P., Prashar, P., Reddy, G. S. B. C., & Mishra, S. K. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113995 · Cited by 3
Prakash, J., V, Vijay, S. A. A., Gopikrishnan, S., Driss, M., & Boulila, W. (2025) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2025.3594186 · Cited by 3
“To validate our approach, we conducted extensive experiments on three widely used benchmark datasets: CMU-MOSI [10], CMU-MOSEI [11], and YouTube-8M [12].”
Hierarchical Text-Guided Refinement Network for Multimodal Sentiment Analysis
Su, Y., & Zhao, X. Y. (2025) · Entropy · doi:10.3390/e27080834 · Cited by 3
“The availability of benchmark datasets like CMU-MOSI [7] and CMU-MOSEI [8] has catalyzed the development of MSA models.”
LDW: Label Divergence Weighting for Multimodal Sentiment Analysis
Du, Q., Langhe, L. D., Lefever, E., & Hoste, V. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758160 · Cited by 3
“Subsequent models, like Memory Fusion Network (MFN) [54] and its graph-based extension (Graph-MFN) [55], integrated temporal dynamics and aimed to capture inter-modal dependencies over time.”
Analysis of the Impact of Foreign Language Learners Emotional State on Learning Outcomes
Lu, Q. (2025) · IEEE Access · doi:10.1109/access.2025.3635632 · Cited by 3
“The CMU-MOSEI [35] dataset is recognized as one of the most extensive and detailed multimodal resources available.”
CMFF: A cross-modal multi-layer feature fusion network for multimodal sentiment analysis
Zheng, S., Zhang, J., Deng, Y., & Chen, L. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113868 · Cited by 3
Affective computing–driven virtual rehabilitation: a systematic survey
Zhao, L., Liu, D., Zhang, H., Yang, G., Zheng, D., Yan, J., & He, L. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00442-3 · Cited by 4
Visual Label Augmentation-Driven Multimodal Emotion Recognition
Wei, Q., Zhou, Y., Zhou, J., Ye, L., & Zhang, Y. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113886 · Cited by 3
Translation-based multimodal learning: a survey
Lu, Z., Liao, Y., & Li, J. (2025) · Intelligence & Robotics · doi:10.20517/ir.2025.40 · Cited by 4
Kim, J., & Kim, J. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892785 · Cited by 4
“All the models use the designated validation set of CMU-MOSI and CMU-MOSEI for finding the best hyperparameters.”
MethodInfluentialCoupling Sentiment and Arousal Analysis Towards an Affective Dialogue Manager
Mallol-Ragolta, A., & Schuller, B. W. (2024) · IEEE Access · doi:10.1109/access.2024.3361750 · Cited by 3
“We opt for the exploitation of the GloVe word embeddings for consistency with previous works in the literature exploiting the CMU-MOSEI dataset [6], [17].”
MethodInfluentialIntegrative Sentiment Analysis: Leveraging Audio, Visual, and Textual Data
Chu, J. S., & Ghanta, S. (2024) · International Conference on AI, Machine Learning and Applications (AIMLA) · doi:10.5121/csit.2024.140211 · Cited by 3
Multi-task disagreement-reducing multimodal sentiment fusion network
Zijun, W., Naicheng, J., Xinyue, C., & Sun, B. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105158 · Cited by 3
Smile: Spiking Multi-Modal Interactive Label-Guided Enhancement Network for Emotion Recognition
Guo, M., Li, W., Wang, C., Ge, Y., & Wang, C. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688152 · Cited by 3
“To evaluate the effectiveness of our proposed SMILE against other methods, we conduct experiments on two datasets CMU-MOSEI [21] and NEMu [6].”
Method
Metadata
- Authors
- Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- California Miramar University (US)
- Nanyang Technological University (SG)
- Agency for Science, Technology and Research (SG)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 2236-2246
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2883409523
- Cited by
- 1,433
- References
- 62
- Citations by year
- 2026: 281 · 2025: 300 · 2024: 266 · 2023: 202 · 2022: 153 · 2021: 113 · 2020: 73 · 2019: 39 · 2018: 6
- FWCI
- 17.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- CMU-MOSEI, multimodal fusion, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
