The DOI matches its DataCite record; 33 of the 33 references it cites were found in a registry or an index.
MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos
Open accessAbstract
People are sharing their opinions, stories and reviews through online video sharing websites every day. Studying sentiment and subjectivity in these opinion videos is experiencing a growing attention from academia and industry. While sentiment analysis has been successful for text, it is an understudied research question for videos and multimedia content. The biggest setbacks for studies in this direction are lack of a proper dataset, methodology, baselines and statistical analysis of how information from different modality sources relate to each other. This paper introduces to the scientific community the first opinion-level annotated corpus of sentiment and subjectivity analysis in online videos called Multimodal Opinion-level Sentiment Intensity dataset (MOSI). The dataset is rigorously annotated with labels for subjectivity, sentiment intensity, per-frame and per-opinion annotated visual features, and per-milliseconds annotated audio features. Furthermore, we present baselines for future studies in this direction as well as a new multimodal fusion approach that jointly models spoken words and visual gestures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The dataset includes 93 randomly selected videos from YouTube, involving 89 different speakers—41 females and 48 males.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“CMU-MOSI is a collection of 2 , 199 opinion video clips each rigorously annotated with labels for subjectivity, sentiment intensity, per-frame, and per-opinion annotated visual features, and permilliseconds annotated audio features [181].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [181]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Therefore the features used in this paper cannot be used to identify the speaker [181, 183].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [181]
Experimental — accuracy not yet measuredIn the citing paper
“The videos are segmented into 2199 clips, each rated on an emotional polarity scale, from +3 (signifying strongly positive sentiment) to −3 (signifying strongly negative sentiment).”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
In the citing paper
“The Multimodal Opinion-level Sentiment Intensity (CMU-MOSI) dataset, introduced by Zadeh et al. in 2016 [28], marks a significant milestone in multimodal sentiment analysis databases due to its unique characteristic of incorporating subjective sentiment and emotional intensity annotations.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,723
Praat: Doing Phonetics by Computer
(2011) · Ear and Hearing · doi:10.1097/aud.0b013e31821473f7 · Cited by 8,010
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,646
Pang, B., & Lee, L. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218990 · Cited by 3,376
Lexicon-based methods for sentiment analysis
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011) · Computational Linguistics · doi:10.1162/coli_a_00049 · Cited by 3,301
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Computers and the Humanities · doi:10.1007/s10579-005-7880-9 · Cited by 1,794
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/979617.979640 · Cited by 822
COVAREP — A collaborative voice analysis repository for speech technologies
Degottex, G., Kane, J., Drugman, T., Raitio, T., & Scherer, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853739 · Cited by 695
Research works printed but not linked: 1
- [37]37. Vinodhini, G., and R. M. Chandrasekaran. "Sentiment analysis and opinion mining: a survey." International Journal 2, no. 6 (2012).
Cited by
Sorted by citations
Items 101–200 of 351
Weakening the Dominant Role of Text: CMOSI Dataset and Multimodal Semantic Enhancement Network
Jin, C., Luo, C., Yan, M. G., Zhao, G., Zhang, G., & Zhang, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3282953 · Cited by 44
A multimodal fusion emotion recognition method based on multitask learning and attention mechanism
Xie, J., Wang, J., Wang, Q., Yang, D., Gu, J., Tang, Y., & Varatnitski, Y. I. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126649 · Cited by 47
Graph Capsule Aggregation for Unaligned Multimodal Sequences
Wu, J., Mai, S., & Hu, H. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479931 · Cited by 45
MSAF: Multimodal Split Attention FusionPreprint
Su, L., Hu, C., Li, G., & Cao, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.07175 · Cited by 72
A feature-based restoration dynamic interaction network for multimodal sentiment analysis
Zeng, Y., Li, Z., Chen, Z., & Ma, H. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107335 · Cited by 42
Defending Multimodal Fusion Models against Single-Source Adversaries
Yang, K., Lin, W.-Y., Barman, M., Condessa, F., & Kolter, J. Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00335 · Cited by 48
Factorized Multimodal Transformer for Multimodal Sequential LearningPreprint
Zadeh, A., Mao, C., Shi, K., Zhang, Y., Liang, P. P., Poria, S., & Morency, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09826 · Cited by 51
Self-citationTriple disentangled representation learning for multimodal affective analysis
Zhou, Y., Liang, X., Chen, H., Zhao, Y., Chen, X., & Yu, L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102663 · Cited by 41
TF-BERT: Tensor-based fusion BERT for multimodal sentiment analysis
Hou, J., Omar, N., Tiun, S. B., Saad, S., & He, Q. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107222 · Cited by 43
Multi-Attention Multimodal Sentiment Analysis
Kim, T., & Lee, B. (2020) · International Conference on Multimedia Retrieval · doi:10.1145/3372278.3390698 · Cited by 39
Token-disentangling Mutual Transformer for multimodal emotion recognition
Yin, G., Liu, Y., Liu, T., Zhang, H., Fang, F., Tang, C., & Jiang, L. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108348 · Cited by 43
HMTL: Heterogeneous Modality Transfer Learning for Audio-Visual Sentiment Analysis
Seo, S., Na, S., & Kim, J. (2020) · IEEE Access · doi:10.1109/access.2020.3006563 · Cited by 38
Li, Z., Huang, Z., Pan, Y., Yu, J., Liu, W., Chen, H., Luo, Y., Wu, D., & Wang, H. H. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124236 · Cited by 36
A Multi-Stage Hierarchical Relational Graph Neural Network for Multimodal Sentiment Analysis
Gong, P., Liu, J., Zhang, X., & Li, X. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096644 · Cited by 36
Adapt and explore: Multimodal mixup for representation learning
Lin, R., & Hu, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102216 · Cited by 34
Enhancing multimodal depression diagnosis through representation learning and knowledge transfer
Yang, S., Cui, L., Wang, L., Wang, T., & You, J. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e25959 · Cited by 37
Quantum Cognitively Motivated Decision Fusion for Video Sentiment Analysis
Gkoumas, D., Li, Q., Dehdashti, S., Melucci, M., Yu, Y., & Song, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16165 · Cited by 34
Inferring Sentiments from Supervised Classification of Text and Speech cues using Fuzzy Rules
Vashishtha, S., & Susan, S. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.03.348 · Cited by 33
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models
Ko, D., Choi, J., Choi, H. K., On, K.-W., Roh, B., & Kim, H. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01925 · Cited by 38
Wang, X., Lyu, J., Kim, B., Parameshachari, B. D., Li, K., & Li, Q. (2024) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2024.3419140 · Cited by 38
FV2ES: A Fully End2End Multimodal System for Fast Yet Effective Video Emotion Recognition Inference
Wei, Q., Huang, X., & Zhang, Y. (2022) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2022.3215245 · Cited by 32
Multimodal Sentiment Analysis with Preferential Fusion and Distance-aware Contrastive Learning
Ma, F., Zhang, Y., & Sun, X. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00237 · Cited by 31
On-the-Fly Modulation for Balanced Multimodal Learning
Wei, Y., Hu, D., Du, H., & Wen, J.-R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3468315 · Cited by 30
SSLMM: Semi-Supervised Learning with Missing Modalities for Multimodal Sentiment Analysis
Wang, Y., Jian, H., Zhuang, J., Guo, H., & Leng, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103058 · Cited by 34
Yu, H., Gui, L., Madaio, M., Ogan, A., Cassell, J., & Morency, L. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123413 · Cited by 32
Self-citationSentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors
Wu, Y., Zhao, Y., Yang, H., Chen, S., Qin, B., Cao, X., & Zhao, W. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.109 · Cited by 33
Modality-specific Learning Rates for Effective Multimodal Additive Late-fusion
Yao, Y., & Mihalcea, R. F. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.143 · Cited by 36
MAG+: An Extended Multimodal Adaptation Gate for Multimodal Sentiment Analysis
Zhao, X., Chen, Y., Li, W., Gao, L., & Tang, B. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746536 · Cited by 30
Hwang, Y., & Kim, J.-H. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.2 · Cited by 31
Multimodal Sentiment Analysis—A Comprehensive Survey From a Fusion Methods Perspective
Zhao, K., Zheng, M., Li, Q., & Liu, J. (2025) · IEEE Access · doi:10.1109/access.2025.3554665 · Cited by 39
Towards Language-Guided Visual Recognition via Dynamic Convolutions
Luo, G., Zhou, Y., Sun, X., Wu, Y., Gao, Y., & Ji, R. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01871-1 · Cited by 31
Zhang, D., Li, S., Zhu, Q., & Zhou, G. (2020) · IEEE Access · doi:10.1109/access.2020.2969205 · Cited by 28
Decoupled cross-attribute correlation network for multimodal sentiment analysis
Zhao, X., Li, X., Jiang, R., & Tang, B. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102897 · Cited by 31
Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition
Delbrouck, J.-B., Tits, N., & Dupont, S. (2020) · doi:10.18653/v1/2020.nlpbt-1.1 · Cited by 30
Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment
Yu, T., Gao, H., Lin, T.-E., Yang, M., Wu, Y., Ma, W., Wang, C., Huang, F., & Li, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.438 · Cited by 33
He, X., Fang, Y., Xiang, N., Li, Z., Wang, Q., Yang, C., Wang, H. H., & Pan, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129532 · Cited by 31
Zeng, Y., Mai, S., & Hu, H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.109 · Cited by 31
LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations
Dutta, S., & Ganapathy, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889998 · Cited by 38
Analyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling FusionPreprint
Mai, S., Xing, S., He, J., Zeng, Y., & Hu, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13572 · Cited by 26
Multimodal PEAR Chain-of-Thought Reasoning for Multimodal Sentiment Analysis
Li, Y., Lan, X., Chen, H., Lü, K., & Jiang, D. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3672398 · Cited by 29
Li, Z.-H., Liu, P.-B., Pan, Y.-S., Ding, W.-P., Yu, J., Chen, H.-R., Liu, W.-H., Luo, Y.-M., & Wang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128940 · Cited by 25
Multimodal Learning for Multi-omics: A Survey
Tabakhi, S., Suvon, M. N. I., Ahadian, P., & Lu, H. (2022) · World Scientific Annual Review of Artificial Intelligence · doi:10.1142/s2811032322500047 · Cited by 27
Muti-Modal Emotion Recognition via Hierarchical Knowledge Distillation
Sun, T., Wei, Y., Ni, J., Liu, Z., Song, X., Wang, Y., & Nie, L. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3385180 · Cited by 26
Dual Low-Rank Multimodal Fusion
Jin, T., Huang, S., Li, Y., & Zhang, Z. M. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.35 · Cited by 25
Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
Lakomkin, E., Zamani, M. A., Weber, C., Magg, S., & Wermter, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794468 · Cited by 23
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Augenstein, I., Gella, S., Ruder, S., Kann, K., Can, B., Conneau, A., Welbl, J., Ren, X., & Rei, M. (2019) · doi:10.18653/v1/w19-43 · Cited by 24
Two-Level Multimodal Fusion for Sentiment Analysis in Public Security
Sun, J., Yin, H., Tian, Y., Wu, J., Shen, L., & Chen, L. (2021) · Security and Communication Networks · doi:10.1155/2021/6662337 · Cited by 23
Multimodal sentiment and emotion recognition in hyperbolic space
Araño, K. A., Orsenigo, C., Soto, M., & Vercellis, C. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115507 · Cited by 23
Multimodal Human-Agent Dialogue Corpus with Annotations at Utterance and Dialogue Levels
Komatani, K., & Okada, S. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597447 · Cited by 23
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85
“M ULTI B ENCH contains 4 datasets involving fusing language , video , and audio time-series data to predict sentiment (CMU-MOSI [181]), emotions (CMU-MOSEI [183]), humor (UR-FUNNY [64]), and sarcasm (MUS T ARD [24]).”
BackgroundSelf-citationSentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis
Fan, Q., Han, J., He, Y., Zheng, T., & Zheng, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.821 · Cited by 23
Li, X., & Chen, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63031-7_26 · Cited by 22
AMSA: Adaptive Multimodal Learning for Sentiment Analysis
Wang, J., Mou, L., Ма, Л., Huang, T., & Gao, W. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3572915 · Cited by 24
Chen, C., Hong, H., Guo, J., & Song, B. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3263801 · Cited by 22
Yang, X., Wu, W., Feng, S., Wang, M., Wang, D., Li, Y., Sun, Q., Zhang, Y., Fu, X., & Poria, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103204 · Cited by 29
Toward Robust Multimodal Sentiment Analysis using multimodal foundational models
Zhao, X., Poria, S., Li, X., Chen, Y., & Tang, B. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126974 · Cited by 28
Deep Multimodal Sequence Fusion by Regularized Expressive Representation Distillation
Guo, X., Kong, A. W., & Kot, A. C. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3142448 · Cited by 21
Multimodal Sentiment Analysis: Review, Application Domains and Future Directions
Gandhi, A., Adhvaryu, K., & Khanduja, V. (2021) · IEEE Pune Section International Conference (PuneCon) · doi:10.1109/punecon52575.2021.9686504 · Cited by 21
Hybrid Multimodal Fusion for Humor Detection
Xu, H., Liu, W., Liu, J., Li, M., Feng, Y., Peng, Y., Shi, Y., Sun, X. N., & Wang, M. (2022) · doi:10.1145/3551876.3554802 · Cited by 23
Combating Human Trafficking with Deep Multimodal ModelsPreprint
Tong, E., Zadeh, A., Jones, C., & Morency, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02735 · Cited by 20
Self-citationQi, Q., Lin, L., & Zhang, R. (2021) · Information · doi:10.3390/info12090342 · Cited by 19
Ishikawa, S., & Sugiura, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3108500 · Cited by 19
VAE-Based Adversarial Multimodal Domain Transfer for Video-Level Sentiment Analysis
Wang, Y., Wu, J., Furumai, K., Wada, S., & Kurihara, S. (2022) · IEEE Access · doi:10.1109/access.2022.3174215 · Cited by 19
From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining
Garcia, A., Colombo, P., d’Alché–Buc, F., Essid, S., & Clavel, C. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1556 · Cited by 18
ArunaGladys, A., & Vetriselvi, V. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111553 · Cited by 36
Joint training strategy of unimodal and multimodal for multimodal sentiment analysis
Meng, L., Zhu, Z., Li, K., Zhou, L., Zhao, Z., & Pei, H. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105172 · Cited by 18
An Ensemble Approach to Utterance Level Multimodal Sentiment Analysis
Huddar, M. G., Sannakki, S. S., & Rajpurohit, V. S. (2018) · International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS) · doi:10.1109/ctems.2018.8769162 · Cited by 18
Beyond Emotion: A Multi-Modal Dataset for Human Desire Understanding
Jia, A., He, Y. L., Zhang, Y., Uprety, S., Song, D., & Lioma, C. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.108 · Cited by 19
An overview of Multimodal Sentiment Analysis research: Opportunities and Difficulties
Ullah, M. A., Islam, M. M., Azman, N. B., & Zaki, Z. M. (2017) · IEEE International Conference on Imaging, Vision & Pattern Recognition (icIVPR) · doi:10.1109/icivpr.2017.7890858 · Cited by 18
Modeling the Clause-Level Structure to Multimodal Sentiment Analysis via Reinforcement Learning
Zhang, D., Li, S., Zhu, Q., & Zhou, G. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00131 · Cited by 17
Pérez-Toro, P. A., Vásquez-Correa, J. C., Bocklet, T., Nöth, E., & Orozco‐Arroyave, J. R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3112543 · Cited by 17
Employing Co-Learning to Evaluate the Explainability of Multimodal Sentiment Analysis
Jain, D. K., Rahate, A., Joshi, G., Walambe, R. A., & Kotecha, K. V. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3176403 · Cited by 17
Multimodal fusion: A study on speech-text emotion recognition with the integration of deep learning
Shang, Y., & Fu, T. (2024) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2024.200436 · Cited by 17
Cross-modal context-gated convolution for multi-modal sentiment analysis
Wen, H., You, S., & Fu, Y. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.03.025 · Cited by 19
MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task
Yao, Y., Papakostas, M., Burzo, M. G., Abouelenien, M., & Mihalcea, R. F. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.216 · Cited by 25
Lai, S., Li, J., Guo, G., Hu, X., Li, Y., Tan, Y., Song, Z., Liu, Y., Ren, Z., Wang, C., et al. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651442 · Cited by 16
KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos
Lakomkin, E., Magg, S., Weber, C., & Wermter, S. (2018) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d18-2016 · Cited by 16
Multimodal sentiment analysis with asymmetric window multi-attentions
Lai, H., & Yan, X. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11234-y · Cited by 15
Zhao, X., Li, X., Jiang, R., & Tang, B. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102745 · Cited by 18
Zhu, M., He, X., Qiao, B., Luo, Y., Li, Z., & Pan, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113249 · Cited by 15
OSAN: A One-Stage Alignment Network to Unify Multimodal Alignment and Unsupervised Domain Adaptation
Liu, Y., Qiao, L., Lu, C., Yin, D., Chen, L., Peng, H., & Ren, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00346 · Cited by 15
AVES: An Audio-Visual Emotion Stream Dataset for Temporal Emotion Detection
Li, Y., Gan, W., Lü, K., Jiang, D., & Jain, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3440924 · Cited by 16
KEBR: Knowledge Enhanced Self-Supervised Balanced Representation for Multimodal Sentiment Analysis
Zhu, A., Hu, M. Q., Wang, X., Yang, J., Tang, Y., & Ren, F. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681163 · Cited by 18
An Entanglement-driven Fusion Neural Network for Video Sentiment Analysis
Gkoumas, D., Li, Q., Yu, Y., & Song, D. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/239 · Cited by 15
Multimodal Sentiment Analysis on Unaligned Sequences Via Holographic Embedding
Ma, Y., & Ma, B. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747646 · Cited by 14
Zhao, X., Chen, Y., Chen, Y.-T., Liu, S., & Tang, B. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859747 · Cited by 14
Multimodal Emotional Classification Based on Meaningful Learning
Filali, H., Riffi, J., Boulealam, C., Mahraz, M. A., & Tairi, H. (2022) · Big Data and Cognitive Computing · doi:10.3390/bdcc6030095 · Cited by 15
Liu, P., Zheng, X. W., Li, H., Liu, J., Ren, Y., Zhu, H., & Sun, L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096470 · Cited by 13
Predict and Use: Harnessing Predicted Gaze to Improve Multimodal Sarcasm Detection
Tiwari, D., Kanojia, D., Ray, A., Nunna, A., & Bhattacharyya, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.988 · Cited by 16
Multimodal Sentiment Analysis Using Deep Learning
Sharma, R., Ngoc, T. L., & Sadat, F. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00240 · Cited by 13
Dutta, P., & Saha, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.570 · Cited by 15
Weakly-supervised Multi-task Learning for Multimodal Affect RecognitionPreprint
Dai, W., Cahyawijaya, S., Bang, Y., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11560 · Cited by 15
DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances
Ghosh, S., Lepcha, S., Sakshi, S., Shah, R. R., & Umesh, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10752 · Cited by 12
Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion
Wu, S., Dai, D., Qin, Z., Liu, T., Lin, B., Cao, Y., & Sui, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.124 · Cited by 18
ACF: An Adaptive Compression Framework for Multimodal Network in Embedded Devices
Cai, Q., Liu, X., Zhang, K., Xie, X. N., Tong, X., & Li, K. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3303350 · Cited by 15
Zhang, H., Yan, Y., Cai, Z., Zhan, P., Chen, B. Y., Jiang, B., & Xie, B. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112346 · Cited by 14
TEMSA:Text enhanced modal representation learning for multimodal sentiment analysis
Chen, J., Song, S., Tan, Y., & Xia, H. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104391 · Cited by 13
The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling
Barros, P. V. A., Churamani, N., Lim, A., & Wermter, S. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925530 · Cited by 12
VideoAdviser: Video Knowledge Distillation for Multimodal Transfer Learning
Wang, Y., Zeng, D., Wada, S., & Kurihara, S. (2023) · IEEE Access · doi:10.1109/access.2023.3280187 · Cited by 13
Metadata
- Authors
- Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.06259
- OpenAlex
- W2465534249
- Cited by
- 350
- References
- 33
- Citations by year
- 2026: 2 · 2025: 38 · 2024: 74 · 2023: 52 · 2022: 42 · 2021: 58 · 2020: 32 · 2019: 24 · 2018: 19 · 2017: 7 · 2016: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- sentiment intensity, subjectivity, online video sharing, multimodal fusion, video sentiment analysis, audio features, annotated corpus, spoken word
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
