The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index, and 1 are not checked yet.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Open accessAbstract
AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, Louis-Philippe Morency. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This dataset encompasses 3837 videos collected from over 1000 distinctive YouTube speakers, with a reasonably balanced gender distribution of 57% male and 43% female speakers, highlighting the dataset’s focus on inclusivity.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The thematic diversity of the CMU-MOSEI dataset is notable, with a spread across 250 different themes. The three predominant themes are comments (16.2%), debates (2.9%), and consultations (1.8%), while the rest of the themes are nearly evenly distributed.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One of CMU-MOSEI’s unique characteristics is its annotations at the utterance level, with 23,259 samples annotated in this manner.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset [30] is a crucial resource in multimodal sentiment analysis and emotion recognition, distinguished by its scale, which is unmatched in the field.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [30]
The cited source says what the claim attributes to it.
In the citing paper
“We generate open-ended AQAs about speech using the four commonly used datasets IEMOCAP [23], MOSEI [24], LibriTTS [25], and Vox-Celeb2 [26,27].”
From Joint Audio and Speech Understanding (2023) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Research works printed but not linked: 3
- Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarep-a col- laborative voice analysis repository for speech tech- nologies. In Acoustics, Speech and Signal Process- ing (ICASSP), 2014 IEEE International Conference on. IEEE, pages 960-964.
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmerman. 2018. Memn: Multimodal emotional memory network for emotion recognition in dyadic conversational videos. In NAACL.
- Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Re- current Highway Networks. arXiv preprint arXiv:1607.03474 . · arXiv
Cited by
Sorted by citations
Items 801–900 of 1,003
Global information regulation network for multimodal sentiment analysis
Xie, S., Chen, Q., Fang, X., & Sun, Q. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105297 · Cited by 4
Multimodal intent recognition based on text-guided cross-modal attention
Li, Z., Peng, J., Lin, X., & Cai, Z. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06583-2 · Cited by 3
“Several unimodal language understanding tasks are extended to be multimodal with relative datasets, including sentiment analysis [5–7], humor analysis [8], semantic understanding [9], and so on.”
Multimodal learning rebalanced: Negative correlation ensembles for improved performance
Wang, Z., Zhang, T., & Huang, W. X. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107686 · Cited by 3
Meta-Learning for Incomplete Multimodal Sentiment Analysis
Tu, G., Wu, T., Luo, X., Zeng, X., Li, W., & Xu, R. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730231 · Cited by 4
Multimodal prototypical network for interpretable sentiment classification
Song, C., Chao, K., Jia, B., & Shen, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-19850-6 · Cited by 5
Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition
Zhang, S., Zhang, J., Zhang, Z., & Li, L. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11210197 · Cited by 3
“The CMU-MOSEI [10] dataset provides annotations for both tasks, and models like MTL [1] and UniMSE [2] have introduced unified frameworks to address them collaboratively.”
InfluentialIMCN: Identifying Modal Contribution Network for Multimodal Sentiment Analysis
Zhang, Q., Shi, L., Liu, P., Zhu, Z., & Xu, L. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956114 · Cited by 3
“CMU-MOSI and CMU-MOSEI used COVAREP [24] to obtain acoustic features XA and Facet to obtain facial expression features XV .”
BackgroundInfluentialEmovo: A real-time anger detector on the smartphone using acoustic signal
Tong, Y., Mo, W., & Sun, Y. (2023) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3594806.3594833 · Cited by 3
“Additionally, implementing emotion classification for anger detection requires considerable processing parameters[20, 21], making it resource-intensive in terms of battery and memory.”
BackgroundMultimodal rough set transformer for sentiment analysis and emotion recognition
Sun, X., He, H., Tang, H., Zeng, K., & Shen, T. (2023) · IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS) · doi:10.1109/ccis59572.2023.10263177 · Cited by 3
“…and the results on the CMU - MOSEI dataset are shown in Table II, the results Major fusion model CAE - LR[30] 78.00 CNN MFN[8] 76.00 RNN Graph - MFN[9] 76.90 RNN RAVEN[25] 79.10 RNN Mu - Net[27] 82.10 RNN MMLatch[28] 82.40 RNN MCTN [33] 79.80 Transformer LMF - MulT[31] 80.80 Transformer MulT[13]…”
MethodInfluentialHYBRID RECOMMENDER FOR VIRTUAL ART COMPOSITIONS WITH VIDEO SENTIMENTS ANALYSIS
Kuchuk, H., & Kuliahin, А. (2024) · Advanced Information Systems · doi:10.20998/2522-9052.2024.1.09 · Cited by 3
Strizhkova, V., Ferrari, L. M., Kachmar, H., Dantcheva, A., & Brémond, F. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00472 · Cited by 3
“In the case of MEAD, we use all views as input and target data, and also use 5-fold cross-validation, as we are the first to apply this dataset for FER.”
Multi-Modal Graph-Based Sentiment Analysis via Hybrid Contrastive Learning
Zhang, J. (2024) · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE) · doi:10.1109/icsece61636.2024.10729439 · Cited by 3
Web Semantic-Enhanced Multimodal Sentiment Analysis Using Multilayer Cross-Attention Fusion
Liu, Y., & Shi-qiu, Y. (2024) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.360653 · Cited by 3
PSACF: Parallel-Serial Attention-Based Cross-Fusion for Multimodal Emotion Recognition
Zhang, Y., Liu, Y., & Cheng, C. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822849 · Cited by 4
“We compare the proposed PSACF with the current state-of-the-art MER methods on the CMU-MOSI [10] and CMU-MOSEI [11] datasets.”
A Comprehensive Survey on Multimodal Sentiment Analysis
Dhal, D., Mohanty, J. P., & Rout, S. S. (2024) · International Conference on Intelligent Computing and Sustainable Innovations in Technology (IC-SIT) · doi:10.1109/ic-sit63503.2024.10862066 · Cited by 3
Feature Engineering Trends in Text-Based Affective Computing: Rules to Advance Deep Learning Models
Pattun, G., & Kumar, P. (2025) · International Research Journal of Multidisciplinary Technovation · doi:10.54392/irjmt2528 · Cited by 4
“A largest data set contains 23l utterances [47, 58].”
Modality-Guided Refinement Learning for Multimodal Emotion Recognition
Cho, S. (2025) · IEEE Access · doi:10.1109/access.2025.3554708 · Cited by 4
Emolysis: A Multimodal Open-Source Group Emotion Analysis and Visualization Toolkit
Ghosh, S., Cai, Z., Gupta, P., Sharma, G., Dhall, A., Hayat, M., & Gedeon, T. (2024) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw63320.2024.00023 · Cited by 3
“It can provide real-time feedback to the user in the form of visualizations to enhance user experience.”
Using Speaker-Aligned Graph Memory Block in Multimodally Attentive Emotion Recognition Network
Li, J.-L., & Lee, C.-C. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1688 · Cited by 3
“We evaluate our framework on a large speaker set CMU-MOSEI database [17] and achieve a 74.”
BackgroundMethodInfluentialUsing Multimodal Transformers in Affective Computing
Vazquez-Rodriguez, J. (2021) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw52867.2021.9666396 · Cited by 4
Chu, L., Xu, J., Zhao, J., Chen, Y., & Huang, J. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/2241310 · Cited by 4
“In 2018, Bagher Zadeh’s team introduced CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI), the largest dataset of sentiment analysis and emotion recognition at that time [31].”
BackgroundContextualized Representations for Low-resource Utterance Tagging
Paranjape, B., & Neubig, G. (2019) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w19-5909 · Cited by 3
“…conversations have been collected in different domains to support research in data-driven dialogue systems (Serban et al., 2015), affective computing (Zadeh et al., 2018; Busso et al., 2008; Park et al., 2014), clinical psychology (Althoff et al., 2016) and tutoring systems (Sinha et al., 2015).”
BackgroundVariational Fusion for Multimodal Sentiment AnalysisPreprint
Majumder, N., Poria, S., Krishnamurthy, G., Chhaya, N., Mihalcea, R. F., & Gelbukh, A. F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06008 · Cited by 5
“MARN (Zadeh et al., 2018c) In this model the intra-modal and cross-modal interactions are modeled with hybrid LSTM memory component.”
BackgroundMethodInfluentialSelf-citationMultimodal Embeddings from Language ModelsPreprint
Tseng, S.-Y., Georgiou, P., & Narayanan, S. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04302 · Cited by 3
Towards Improved Model Design for Authorship Identification: A Survey on Writing Style UnderstandingPreprint
Ma, W., Liu, R., Wang, L., & Vosoughi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14445 · Cited by 4
“Benchmark datasets on multi-modal ER include the IEMOCAP (Busso et al., 2008), MOUD (P´erez-Rosas et al., 2013), ICT-MMMO (W¨ollmer et al., 2013), MOSI (Zadeh et al., 2016) and MOSEI (Bagher Zadeh et al., 2018) datasets.”
Emerging Trends of Multimodal Research in Vision and Language.Preprint
Uppal, S., Bhagat, S., Hazarika, D., Majumder, N., Poria, S., Zimmermann, R., & Zadeh, A. (2020) · arXiv (Cornell University) · Cited by 3
Self-citationModality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion RecognitionPreprint
Dai, W., Liu, Z., Yu, T., & Fung, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09629 · Cited by 5
“Additionally, for CMU-MOSEI, the Graph Memory Fusion Network (Graph-MFN) (Zadeh et al., 2018b) and a multi-task learning (MTL)”
BackgroundMethodInfluentialMultimodal End-to-End Sparse Model for Emotion RecognitionPreprint
Dai, W., Cahyawijaya, S., Liu, Z., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.09666 · Cited by 3
“We follow the feature extraction procedure in the previous works (Zadeh et al., 2018b; Tsai et al., 2018, 2019; Rahman et al., 2020).”
BackgroundMethodInfluentialAn Estimation of Online Video User Engagement from Features of Continuous EmotionsPreprint
Stappen, L., Baird, A., Lienhart, M., Bätz, A., & Schuller, B. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01633 · Cited by 3
“C A R , some previously collected datasets harvested YouTube as their primary source [27], [28].”
BackgroundDeep Emotion Recognition using Facial, Speech and Textual Cues: A SurveyPreprint
Zhang, T., & Tan, Z. (2021) · doi:10.36227/techrxiv.15184302 · Cited by 3
Zero-Shot Transfer for Wildlife Bioacoustics DetectionPreprint
Miao, Z., Elizalde, B., Deshmukh, S., Kitzes, J. A., Wang, H., Dodhia, R. M., & Ferres, J. L. (2023) · Research Square · doi:10.21203/rs.3.rs-3180218/v1 · Cited by 3
NRAFN: a non-text reinforcement and adaptive fusion network for multimodal sentiment analysis
Wei, J., & Shao, X.-H. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19433-z · Cited by 2
“As research progresses, an increasing number of multimodal fusion strategies have emerged, including tensor-based fusion [19,20], graph-based fusion [35,36], translation-based fusion [37,38], and attention-based fusion [39–41].”
Zhang, Y., Zhong, H., Chen, G., Alhusaini, N., Zhao, S., & Wu, C. (2024) · International Conference on Networking and Network Applications (NaNA) · doi:10.1109/nana63151.2024.00088 · Cited by 3
“CMU-MOSEI [27] is an improved version of CMU-MOSI, containing 22856 annotated video clips (discourse) from 1000 different speakers and 250 different topics.”
BackgroundSMURF: Statistical Modality Uniqueness and Redundancy Factorization
Wörtwein, T., Allen, N. B., Cohn, J. F., & Morency, L. (2024) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3678957.3685716 · Cited by 3
Self-citation$\mathcal {P}$owMix: A Versatile Regularizer for Multimodal Sentiment Analysis
Georgiou, E., Avrithis, Y., & Potamianos, A. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3496316 · Cited by 3
“With the focus of MSA research on developing advanced architectures and diverse learning strategies, there is a clear demand for versatile multimodal regularization methods.”
InfluentialZhu, C., Chen, M., Li, H., Zhang, S., Liang, H., Sun, C., Liu, Y., & Chen, J. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10678-1 · Cited by 2
“We evaluate our model on four widely used multimodal sentiment analysis datasets, namely SIMS [20], SIMSV2 [10], MOSI [53], and MOSEI [29].”
TACL: A Trusted Action-enhanced Curriculum Learning Approach to Multimodal Affective Computing
Yu, T., Wang, J., Luo, J., Wang, J., & Zhou, G. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129195 · Cited by 2
Liu, X., Chen, Q., Cheung, Y., & Peng, S. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3550645 · Cited by 2
“2) CMU-MOSEI: It is a large multimodal sentiment analysis and gender-balanced emotional dataset [28], consisting of 22 856 videos derived from 5000 videos and 1000 distinct speakers [28].”
Dynamic Tuning and Multi-Task Learning-Based Model for Multimodal Sentiment Analysis
Liang, Y., Tohti, T., Hu, W., Kong, B., Han, D., Yan, T., & Hamdulla, A. (2025) · Applied Sciences · doi:10.3390/app15116342 · Cited by 2
Chen, S., Zhou, D., Wei, C., Li, X., Yao, H.-Y., & Luo, H. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131059 · Cited by 3
Emotion Recognition using multi-modal data and self-supervised learning techniques
Neuwirt, M. (2025) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3733155.3734897 · Cited by 2
“CMU Multimodal Opinion Sentiment and Emotion Inten-sity (CMU-MOSEI) [2]: This dataset includes 3,229 videos from 1000 YouTube speakers.”
Hierarchical Signal Calibration and Refinement for Multimodal Sentiment Analysis
Ren, B., Cao, T., Zhang, Z., Bai, S., & Liu, N. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3603884 · Cited by 2
“Weevaluateourproposedmodelontwostandardbenchmarks, CMU-MOSI [25] and CMU-MOSEI [26], adhering to the standard data splits and evaluation protocols for fair comparison.”
Deploying Emotion-Aware Speech GANs on Edge Devices: A Real-Time Optimization Framework
Meenakshisundaram, N., & Sajiv, G. (2025) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc65114.2025.11212317 · Cited by 2
“Real-time on-device emotion generation addresses this issue by ensuring that sensitive speech data does not leave the device, which is particularly critical in healthcare applications, where privacy and data security are paramount [17].”
D$$^2$$M: Decomposition-diffusion model for enhanced multimodal sentiment analysis
Zong, L., Xu, W., Hu, D., & Wang, J. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06979-0 · Cited by 2
“Since our task requires datasets that provide text, visual, and audio information simultaneously, we employ the following datasets: CMU-MOSI [ 38 ], CMU-MOSEI [ 39], and UR-FUNNY [ 40 ].”
Attention-Based Denoising Cross-Modal Multi-Level Fusion Network for Multimodal Sentiment Analysis
Wang, K., & Pan, S. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227992 · Cited by 2
“We conducted comprehensive experimental evaluations on two widely used datasets, namely CMU-MOSI [5] and CMU-MOSEI [22].”
DCGRM-Net: Dual-Channel Guided Reconstruction Mamba Network for robust multimodal sentiment analysis
Liu, S., Zhao, H., Chen, Y., Kong, F., & Li, K. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104491 · Cited by 2
Hidayati, S. C., Lee, J. R., & Samuel, K. D. (2025) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc65261.2025.11249269 · Cited by 2
“We evaluate the proposed model using three widely adopted benchmark datasets: CMU-MOSEI [21], IEMOCAP [22], and CREMA-D [23].”
From Feature Alignment to Multimodal Fusion: A Two-Stage Primary Modality-Guided Approach for MSA
Ma, G., Ren, X., Jiang, Y., Guan, H., & Xu, B. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3771041 · Cited by 2
“We compare TSPMG with 12 state-of-the-art MSA methods, including EF-LSTM [37], LF-DNN [36], TFN [39], LMF [21], MFN [40], Graph-MFN [41], MulT [33], PMR [22], MISA [10], MAG-BERT [30], DMD [16], and DLF [34].”
Visual Semantic Knowledge Discovery for Multimodal Intent Recognition
Cheng, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3642324 · Cited by 2
“Leveraging the rapid advancements in the closely related field of multimodal sentiment analysis [27], [28], [29], EMOTyDA [25] re-annotates multimodal sentiment datasets [30], [31] with dialogue act labels, which can be regarded as a type of communicative intents in conversations.”
A text guided multimodal scale path fusion network for multimodal sentiment analysis
Liu, S., Zhao, H., Chen, Y., Kong, F., & Li, K. (2025) · Scientific Reports · doi:10.1038/s41598-025-32637-z · Cited by 2
Frequency-aware experts with multi-stage fusion for multimodal sentiment analysis
Zhu, X., & Li, Y. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-01006-7 · Cited by 4
“Extensive experi - ments on CMU-MOSI and CMU-MOSEI demonstrate consistent improvements over strong baselines, while ablation and robustness analyses confirm the contribution of each compo - nent and the overall resilience of the architecture.”
InfluentialLUCID: lexicon-augmented fusion with ordinal calibration for multimodal sentiment analysis
Yuan, F., Qi, K., Liu, J., Lu, C., & Chen, W. (2026) · Multimedia Systems · doi:10.1007/s00530-025-02208-3 · Cited by 2
“We conduct experiments on two widely used English multimodal sentiment analysis benchmarks, CMU-MOSI [35] and CMU-MOSEI [36], as well as one Chinese dataset, CH-SIMS [37].”
TB-MEN: Text-Centric Bidirectional Modality Enhancement Network for Multimodal Sentiment Analysis
Zhang, J., Yin, M., Wei, G., Shi, D., & Guo, Y. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3660469 · Cited by 2
“The heat map presented in the figure clearly depicts the corresponding Acc-2 values when the audio dimension and the visual dimension are combined with different values of 16, 32, 48, 64, 80 and 96 respectively.”
Xie, Y. (2026) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.402041 · Cited by 2
Yu, J., Li, M., Deng, H., Gao, W. G. W., & Zhu, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7081-2_34 · Cited by 2
Beneficial Noise Learning for Robust Multimodal Fusion
Zhu, A., Hu, M. Q., Wang, X., & Ren, F. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668492 · Cited by 3
“Fig.1(a)illustrates that in the MOSEI dataset [16], approximately 65.97% of sentiment values lie within the narrow range [ − 1, 1].”
InfluentialChowdhury, J. H., & Ramanna, S. (2026) · Computational Intelligence · doi:10.1111/coin.70219 · Cited by 2
“Fuzzy and Rough Set techniques have recently been applied to multimodal emotion recognition [20, 21] because they are effective in addressing key challenges such as uncertainty, ambiguity, and heterogeneity across multimodal data sources.”
Liu, Z., Liu, S., Yan, W., Song, P., Song, Y., & Gao, R. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792577 · Cited by 2
“These datasets are respectively named: MOSEI [49], MELD [31], SIMS-V2 [24], and CHERMA [33].”
Kushwaha, N., Cambria, E., & Hussain, A. (2026) · Cognitive Computation · doi:10.1007/s12559-026-10573-y · Cited by 2
“● The model achieves exceptional performance with 96.23% average accuracy across seven bench - mark datasets (IEMOCAP [ 10 ], CMU-MOSEI [ 27], MELD [8], RAVDESS [28 ], CMU-MOSI, CREMA-D [29], SAVEE), representing improvements of 7.97– 10.04% over baseline neural-only approaches, while maintaining…”
InfluentialSelf-citationA survey on multimodal explainable Artificial Intelligence
Giovannoni, C., Metta, C., Monreale, A., & Rinzivillo, S. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200671 · Cited by 2
Multimodal Video Understanding: A Capability-Based Survey of Alignment, Expression, and Reasoning
Zhao, R., Pu, D., Li, C., Zhang, Z. D., Xingzhu, P., & Ma, Y. (2026) · Applied Sciences · doi:10.3390/app16147154 · Cited by 2
Shyness Trait Recognition for Schoolchildren via Multi-View Features of Online Writing
Tian, X., Jing, L., Luo, F., & Zhang, S. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3077410 · Cited by 2
“In practice, multi-view learning has been widely used for personality recognition when multi-modal data is available [42], [44], [45], [46].”
BackgroundMethodNFCMF: Noise Filtering and CrossModal Fusion for Multimodal Sentiment Analysis
Su, G., He, J., Li, X., Lu, M., & Yang, H. (2021) · International Conference on Asian Language Processing · doi:10.1109/ialp54817.2021.9675197 · Cited by 2
Bimodal Emotion Recognition for the Patients with Depression
Wang, X., Zhao, S., & Wang, Y. (2021) · IEEE International Conference on Signal and Image Processing (ICSIP) · doi:10.1109/icsip52628.2021.9688837 · Cited by 2
“During that year, Zadeh et al. also built up their own dataset, the CMU-MOSEI dataset [6].”
BackgroundInformation Bottleneck based Representation Learning for Multimodal Sentiment Analysis
Zhang, T., Zhang, H., Xiang, S., & Wu, T. (2022) · International Conference on Control Engineering and Artificial Intelligence (CCEAI) · doi:10.1145/3522749.3523069 · Cited by 2
“CMU-MOSEI also provides emotions of happiness, sadness, anger, fear, disgust, surprise which annotated on a [0,3] Likert scale.”
MethodInfluentialLarge Pretrained Models on Multimodal Sentiment Analysis
Song, Y., Fan, X., Yang, Y., Ren, G., & Pan, W. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-9423-3_63 · Cited by 2
A Dual-branch Enhanced Multi-task Learning Network for Multimodal Sentiment Analysis
Geng, W., Li, X., & Bian, Y. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592260 · Cited by 2
“Compared with the single branch network (SBEM, variant of DBEM model) and SOTA network models, the experimental results on the two public datasets CH-SIMS [20] and CMU-MOSEI [24] validate the effectiveness of the DBEM model.”
BackgroundMethodResultInfluentialBody-Focused Expression Analysis: A Conceptual Framework
Rodrigues, J. M. F., & Cardoso, P. J. S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35897-5_42 · Cited by 3
USURP: Universal Single-Source Adversarial Perturbations on Multimodal Emotion Recognition
Low, Y.-Y., Phan, R. C. ‐., Pal, A., & Chang, X. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222572 · Cited by 2
“The CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) [19] contains 23,453 annotated video segments from 1,000 distinct speakers and 250 topics acquired from social media channels.”
BackgroundYang, Y.-C. B., & Zhang, L. (2023) · doi:10.1145/3607542.3617355 · Cited by 2
“Many datasets like CMU-MOSEI [29] only record the speaker’s voice and facial expressions, which makes the trained model inaccurate in real-world scenarios, and the speaker’s voice is mainly English, while our RTS dataset is in French.”
BackgroundAn End-to-End Transformer with Progressive Tri-Modal Attention for Multi-modal Emotion Recognition
Wu, Y., Peng, P., Zhang, Z., Zhao, Y., & Qin, B. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8540-1_32 · Cited by 2
Feng, B. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/33/20230224 · Cited by 2
Guhan, P., Awasthi, N., McDonald, K., Bussell, K., Reeves, G. M., Manocha, D., & Bera, A. (2024) · JMIR Formative Research · doi:10.2196/46390 · Cited by 7
“There exist datasets such as Carnegie Mellon University Multimodal Corpus of Sentimental Intensity (CMU-MOSI) [22], Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) [23], and Stanford Emotional Narratives Dataset (SEND) [24] that capture such settings.”
Enhancing Multimodal Sentiment Recognition Based on Cross-Modal Contrastive Learning
Cui, L., & Yang, S. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688113 · Cited by 2
“In this study, we evaluate the proposed framework using three benchmark datasets, namely CMU-MOSI [15], CMU-MOSEI [16], and CH-SIMS [20].”
MethodZhao, H., Ju, Y., & Gao, Y. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688301 · Cited by 2
“For example, RGCN [10] can handle the influence of different edge types on nodes, which is ignored by GCNs. Graph-MFN [11]and MMDFN [6] both use dynamic fusion graph.”
MethodInfluentialYang, J., Wang, L., Qi, Y. F., Chen, H., & Li, J. (2024) · Applied Sciences · doi:10.3390/app14199121 · Cited by 2
“The CMU-MOSEI (Multimodal Opinion Sentiment and Emotion Intensity) dataset contains more than 23,000 video clips covering the six basic emotions, with each clip containing audio, text, and video information [14].”
A Cross-Multi-modal Fusion Approach for Enhanced Engagement Recognition
Dresvyanskiy, D., Karpov, A. A., & Minker, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78014-1_1 · Cited by 2
Hierarchical Fusion Network for Multimodal Sentiment Analysis
Hua, Y., Ji, R., Yang, S., Geng, Y., Gao, W., & Tan, Y. (2024) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml62565.2024.10779845 · Cited by 2
“Two benchmark datasets for multimodal sentiment analysis, CMU-MOSI (CMU Multimodal Opinion Sentiment Intensity) [23] and CMU-MOSEI (CMU Multimodal Opinion Sentiment and Emotion Intensity) [15], are used to evaluate the model, and their data partition structure is shown in Table I.”
Enhancing Multimodal Sentiment Analysis with Dynamic Weight Adjustment and Adversarial Training
Zhu, Y., & Liu, X. (2024) · China Automation Congress (CAC) · doi:10.1109/cac63892.2024.10865061 · Cited by 2
Investigating HuBERT-Based Speech Emotion Recognition Generalisation Capability
Li, L., Glackin, C., Cannings, N., Veneziano, V., Barker, J. W., Oduola, O., Woodruff, C., Laird, T., Laird, J., & Sun, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-84353-2_16 · Cited by 2
Nguyen, D.-K., Lim, E., Kim, S.-H., Yang, H.-J., & Kim, S. (2025) · Applied Sciences · doi:10.3390/app15052862 · Cited by 2
“Regarding the CH-SIMS dataset, this work follows the approaches by Liu et al. [29] and Zadeh et al. [28], recording the experimental results in two formats: multiclass classification and regression.”
BiMSA: Multimodal Sentiment Analysis Based on BiGRU and Bidirectional Interactive Attention
Wang, Q., Yu, H., Wang, Y., & Bian, H. (2025) · The European Journal on Artificial Intelligence · doi:10.1177/30504554251319444 · Cited by 2
“The CMU-MOSEI dataset surpasses its predecessor, the CMU-MOSI dataset (Zadeh et al., 2018).”
Feng, C., Yang, H., Wang, S., & Li, X. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07474-6 · Cited by 3
“We conducted experiments on two public academic datasets: CMU-MOSI [23] and CMU-MOSEI [24].”
Modality emotion semantic correlation analysis for multimodal emotion recognition
Zhang, Y., Xie, D., Luo, D., & Sun, B. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110467 · Cited by 2
Towards Emotion Analysis in Short-form Videos: A Large-Scale Dataset and Baseline
Wu, X., Sun, H., Xue, J., Nie, J., Kong, X., Zhai, R., Huang, D., & He, L. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733453 · Cited by 4
“Besides, different from the human-centered VEA datasets ( e.g. , [4, 37, 62]), eMotions originating from SVs is directed towards in-the-wild scenes, implying that the emotional elicitation requires the whole duration, not short clips of video.”
Multi-Microphone and Multi-Modal Emotion Recognition in Reverberant Environment
Cohen, O., Hazan, G., & Gannot, S. (2025) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco63237.2025.11226096 · Cited by 2
A Cross-Modal Attention and Multi-task Learning Based Approach for Multi-modal Sentiment Analysis
Song, Y., Fan, X., Yang, Y., Ren, G., & Pan, W. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-9423-3_20 · Cited by 2
MSDD: A Multimodal Language Dateset for Stance Detection
Hu, M., Liu, P., Wang, W., Zhang, H., & Lin, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28953-8_10 · Cited by 3
Zhao, T. G., Peng, J., Wang, L., & Zheng, C. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10393942 · Cited by 2
“CMU-MOSEI [19] is an additional supplement to the CMU-MOSI dataset, which consists of 22,856 video clips from YouTube.”
MethodResearch Advanced in Multimodal Emotion Recognition Based on Deep Learning
Kong, W. (2024) · Highlights in Science Engineering and Technology · doi:10.54097/p3yprn36 · Cited by 2
“[8] proposed a dynamic fusion graph (DFG) technique to fuse multiple modalities.”
MethodQuantum-inspired Multimodal Representation.
Li, Q., & Melucci, M. (2019) · Padua Research Archive (University of Padova) · Cited by 2
Improving Emotional Expression Recognition of Robots Using Regions of Interest from Human Data
Bloem, A. C., Barakova, E. I., Hootsmans, I. M., Opheij, L. M., Toebosch, R., Kerzel, M., & Barros, P. V. A. (2020) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3371382.3378359 · Cited by 2
“With the recent ease to training and deploying deep learning models and the availability of "in-the-wild" datasets [12, 20, 26, 27], emotion recognition in real-life interactions with technologies such as robots became possible and many related applications are in rapid development [15, 17, 18, 24].”
BackgroundMulti-Head Modularization to Leverage Generalization Capability in Multi-Modal Networks
Lee, J.-T., Park, H., Yun, S., & Chang, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20698 · Cited by 2
“We evaluate our method for the multi-modal sentiment analysis on CMU-MOSEI (Zadeh et al. 2018) dataset which includes 23,454 movie review video clips.”
MethodLeveraging Pre-trained Language Model for Speech Sentiment AnalysisPreprint
Shon, S., Brusco, P., Pan, J. S., Han, K. J., & Watanabe, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06598 · Cited by 2
“The evaluation was performed on a large sentiment dataset published by [9] that mitigates drawbacks in conventional datasets such as being scripted [11] or single speaker monologues [12].”
A Multimodal Sentiment Dataset for Video RecommendationPreprint
Tang, H., Liu, H., Xiao, X., & Wu, H. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08333 · Cited by 2
“For example, concatenation (Cambria et al., 2017), Tensor Fusion Network (TFN) (Zadeh et al., 2017), Low-rank Multimodal Fusion (LMF) (Liu et al., 2018), Memory Fusion Network (MFN) (Zadeh et al., 2018a), Dynamic Fusion Graph (DFG) (Zadeh et al., 2018b).”
InfluentialRelay Variational Inference: A Method for Accelerated Encoderless VIPreprint
Zadeh, A., Benoit, S., & Morency, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13422 · Cited by 2
“CMU-MOSI Dataset: CMU Multimodal Sentiment Intensity (CMU-MOSI) is an in-the-wild dataset of multimodal sentiment analysis [15].”
Self-citationPrefAce: Face-Centric Pretraining with Self-Structure Aware Distillation
Hu, S., Wang, Z., Hu, P., Peng, X., Wu, J., Zhu, H., & Ong, Y.-S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29147 · Cited by 2
“…(MS-celeb-1M (Guo et al. 2016)), Facial Attribute Recognition(CelebA (Liu et al. 2015), CelebV-HQ (Zhu et al. 2022)), Facial Emotion Recognition (CMU-MOSEI (Bagher Zadeh et al. 2018)), DeepFake Detection (FF++ (Rossler et al. 2019)) and Lip Synchronization (LRS2 (Chung and Zisser-man 2017)).”
BackgroundEnhancing Emotion Detection with Sentiment Analysis InsightsPreprint
Parker, E., Nasir, W., & Benson, O. (2024) · Preprints.org · doi:10.20944/preprints202404.0059.v1 · Cited by 2
Introduction to Applied Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502400 · Cited by 1
Emotion Recognition in the Wild
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502404 · Cited by 2
Metadata
- Authors
- Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- California Miramar University (US)
- Nanyang Technological University (SG)
- Agency for Science, Technology and Research (SG)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 2236-2246
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2883409523
- Cited by
- 1,433
- References
- 62
- Citations by year
- 2026: 281 · 2025: 300 · 2024: 266 · 2023: 202 · 2022: 153 · 2021: 113 · 2020: 73 · 2019: 39 · 2018: 6
- FWCI
- 17.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- CMU-MOSEI, multimodal fusion, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
