The DOI matches its DataCite record; 33 of the 33 references it cites were found in a registry or an index.
MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos
Open accessAbstract
People are sharing their opinions, stories and reviews through online video sharing websites every day. Studying sentiment and subjectivity in these opinion videos is experiencing a growing attention from academia and industry. While sentiment analysis has been successful for text, it is an understudied research question for videos and multimedia content. The biggest setbacks for studies in this direction are lack of a proper dataset, methodology, baselines and statistical analysis of how information from different modality sources relate to each other. This paper introduces to the scientific community the first opinion-level annotated corpus of sentiment and subjectivity analysis in online videos called Multimodal Opinion-level Sentiment Intensity dataset (MOSI). The dataset is rigorously annotated with labels for subjectivity, sentiment intensity, per-frame and per-opinion annotated visual features, and per-milliseconds annotated audio features. Furthermore, we present baselines for future studies in this direction as well as a new multimodal fusion approach that jointly models spoken words and visual gestures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The dataset includes 93 randomly selected videos from YouTube, involving 89 different speakers—41 females and 48 males.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“The videos are segmented into 2199 clips, each rated on an emotional polarity scale, from +3 (signifying strongly positive sentiment) to −3 (signifying strongly negative sentiment).”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“CMU-MOSI is a collection of 2 , 199 opinion video clips each rigorously annotated with labels for subjectivity, sentiment intensity, per-frame, and per-opinion annotated visual features, and permilliseconds annotated audio features [181].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [181]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Therefore the features used in this paper cannot be used to identify the speaker [181, 183].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [181]
Experimental — accuracy not yet measuredIn the citing paper
“The Multimodal Opinion-level Sentiment Intensity (CMU-MOSI) dataset, introduced by Zadeh et al. in 2016 [28], marks a significant milestone in multimodal sentiment analysis databases due to its unique characteristic of incorporating subjective sentiment and emotional intensity annotations.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,723
Praat: Doing Phonetics by Computer
(2011) · Ear and Hearing · doi:10.1097/aud.0b013e31821473f7 · Cited by 8,010
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,646
Pang, B., & Lee, L. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218990 · Cited by 3,376
Lexicon-based methods for sentiment analysis
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011) · Computational Linguistics · doi:10.1162/coli_a_00049 · Cited by 3,301
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Computers and the Humanities · doi:10.1007/s10579-005-7880-9 · Cited by 1,794
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/979617.979640 · Cited by 822
COVAREP — A collaborative voice analysis repository for speech technologies
Degottex, G., Kane, J., Drugman, T., Raitio, T., & Scherer, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853739 · Cited by 695
Research works printed but not linked: 1
- [37]37. Vinodhini, G., and R. M. Chandrasekaran. "Sentiment analysis and opinion mining: a survey." International Journal 2, no. 6 (2012).
Cited by
Sorted by citations
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Self-citationMultimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433
Self-citationEfficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170
Self-citationSentiment analysis using deep learning architectures: a review
Yadav, A., & Vishwakarma, D. K. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09794-5 · Cited by 841
Yu, W., Xu, H., Yuan, Z., & Wu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17289 · Cited by 816
Integrating Multimodal Information in Large Pretrained Transformers
Rahman, W., Hasan, M. K., Lee, S., Zadeh, A. B., Mao, C., Morency, L., & Hoque, M. E. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.214 · Cited by 778
Self-citationAn introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools
Lauriola, I., Lavelli, A., & Aiolli, F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.103 · Cited by 755
CH-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotation of Modality
Yu, W., Xu, H., Meng, F., Zhu, Y., Ma, Y., Wu, J., Zou, J., & Yang, K. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.343 · Cited by 482
Sun, Z., Sarma, P., Sethares, W., & Liang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6431 · Cited by 410
Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges
Das, R., & Singh, T. D. (2023) · ACM Computing Surveys · doi:10.1145/3586075 · Cited by 428
Multimodal sentiment analysis based on fusion methods: A survey
Zhu, L., Zhu, Z., Zhang, C., Xu, Y., & Kong, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.02.028 · Cited by 375
Recent trends in deep learning based personality detection
Mehta, Y., Majumder, N., Gelbukh, A. F., & Cambria, E. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09770-z · Cited by 352
TETFN: A text enhanced transformer fusion network for multimodal sentiment analysis
Wang, D., Guo, X., Tian, Y., Liu, J., He, L., & Luo, X. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.109259 · Cited by 349
Multimodal sentiment analysis with word-level fusion and reinforcement learning
Chen, M., Wang, S., Liang, P. P., Baltrušaitis, T., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136801 · Cited by 334
Self-citationA Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural NetworksPreprint
Poria, S., Cambria, E., Hazarika, D., & Vij, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.08815 · Cited by 368
Recent advancements and challenges of NLP-based sentiment analysis: A state-of-the-art review
Jim, J. R., Talukder, M. A. R., Malakar, P., Kabir, M. M., Nur, K. M., & Mridha, M. F. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100059 · Cited by 420
SMIL: Multimodal Learning with Severely Missing Modality
Ma, M., Ren, J., Zhao, L., Tulyakov, S., Wu, C., & Peng, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16330 · Cited by 332
“• CMU Multimodal Opinion Sentiment Intensity (CMUMOSI) (Zadeh et al. 2016) consists of 2, 199 opinion video clips from YouTube movie reviews.”
BackgroundHybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis
Mai, S., Zeng, Y., Zheng, S., & Hu, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3172360 · Cited by 307
Domain Adaptation: Challenges, Methods, Datasets, and Applications
Singhal, P., Walambe, R. A., Ramanna, S., & Kotecha, K. V. (2023) · IEEE Access · doi:10.1109/access.2023.3237025 · Cited by 276
Multimodal Emotion Recognition with Deep Learning: Advancements, challenges, and future directions
Vijayaraghavan, G., Mala, T., Dhanasekaran, P., & Uma, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102218 · Cited by 263
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
Rahate, A., Walambe, R., Ramanna, S., & Kotecha, K. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.003 · Cited by 214
Exploiting BERT for Multimodal Target Sentiment Classification through Input Space Translation
Khan, Z., & Fu, Y. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475692 · Cited by 203
Sentiment analysis using deep learning approaches: an overview
Habimana, O., Li, Y., Li, R., Gu, X., & Yu, G. (2019) · Science China Information Sciences · doi:10.1007/s11432-018-9941-6 · Cited by 198
Cross-Modal Enhancement Network for Multimodal Sentiment Analysis
Wang, D., Liu, S., Wang, Q., Tian, Y., He, L., & Gao, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3183830 · Cited by 196
Sentiment Analysis in Social Networks
Gonçalves, J. P. M. (2016) · Elsevier eBooks · doi:10.1016/c2015-0-01864-0 · Cited by 184
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor
Hasan, M. K., Rahman, W., Bagher Zadeh, A., Zhong, J., Tanveer, M. I., Morency, L.-P., & Hoque, M. (. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1211 · Cited by 174
Self-citationMultimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment
Gu, Y., Yang, K., Fu, S., Chen, S., Li, X., & Marsic, I. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1207 · Cited by 160
Multimodal Emotion Recognition With Transformer-Based Self Supervised Feature Fusion
Siriwardhana, S., Kaluarachchi, T. I., Billinghurst, M., & Nanayakkara, S. C. (2020) · IEEE Access · doi:10.1109/access.2020.3026823 · Cited by 171
A Comprehensive Survey on Deep Learning Multi-Modal Fusion: Methods, Technologies and Applications
Jiao, T., Guo, C., Feng, X., Chen, Y., & Song, J. (2024) · Computers, Materials & Continua · doi:10.32604/cmc.2024.053204 · Cited by 158
TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis
Wang, Z., Wan, Z., & Wan, X. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380000 · Cited by 156
Mai, S., Hu, H., & Xing, S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1046 · Cited by 153
AOBERT: All-modalities-in-One BERT for multimodal sentiment analysis
Kim, K.-H., & Park, S. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.022 · Cited by 149
Multimodal Video Sentiment Analysis Using Deep Learning Approaches, a Survey
Abdu, S. A., Yousef, A. H., & Salem, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.003 · Cited by 154
Quantum-inspired multimodal fusion for video sentiment analysis
Li, Q., Gkoumas, D., Lioma, C., & Melucci, M. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.08.006 · Cited by 143
Kalateh, S., Estrada-Jimenez, L. A., Nikghadam-Hojjati, S., & Barata, J. (2024) · IEEE Access · doi:10.1109/access.2024.3430850 · Cited by 170
Video sentiment analysis with bimodal information-augmented multi-head attention
Wu, T., Peng, J., Zhang, W., Zhang, H., Tan, S., Yi, F., Ma, C., & Huang, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107676 · Cited by 138
Recognizing Emotions in Video Using Multimodal DNN Feature Fusion
Williams, J., Kleinegesse, S., Comanescu, R., & Radu, O. (2018) · Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML) · doi:10.18653/v1/w18-3302 · Cited by 132
Shenoy, A., & Sardana, A. (2020) · doi:10.18653/v1/2020.challengehml-1.3 · Cited by 124
Gated Mechanism for Attention Based Multi Modal Sentiment Analysis
Kumar, A., & Vepa, J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053012 · Cited by 123
The OMG-Emotion Behavior Dataset
Barros, P., Churamani, N., Lakomkin, E., Siqueira, H., Sutherland, A., & Wermter, S. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489099 · Cited by 121
Siriwardhana, S., Reis, A., Weerasekera, R., & Nanayakkara, S. C. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1212 · Cited by 137
Interactional Stancetaking in Online Forums
Kiesling, S. F., Pavalanathan, U., Fitzpatrick, J., Han, X., & Eisenstein, J. (2018) · Computational Linguistics · doi:10.1162/coli_a_00334 · Cited by 120
A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning
Wang, L., Peng, J., Zheng, C., Zhao, T., & Zhu, L. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103675 · Cited by 111
Behavioral and Physiological Signals-Based Deep Multimodal Approach for Mobile Emotion Recognition
Yang, K., Wang, C., Gu, Y., Sarsenbayeva, Z., Tag, B., Dingler, T., Wadley, G., & Gonçalves, J. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3100868 · Cited by 115
Dynamic interactive multiview memory network for emotion recognition in conversation
Wen, J., Jiang, D., Tu, G., Liu, C., & Cambria, E. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.009 · Cited by 102
Multimodal sentiment analysis with unidirectional modality translation
Yang, B., Shao, B., Wu, L., & Lin, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.09.041 · Cited by 101
AF: An Association-Based Fusion Method for Multi-Modal Classification
Liang, X., Qian, Y., Guo, Q. J., Cheng, H., & Liang, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125995 · Cited by 98
Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
Liang, P. P., Liu, Z., Tsai, Y.-H. H., Zhao, Q., Salakhutdinov, R., & Morency, L.-P. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1152 · Cited by 95
Self-citationMultimodal sentiment analysis: A survey
Lai, S., Hu, X., Xu, H., Ren, Z., & Liu, Z. (2023) · Displays · doi:10.1016/j.displa.2023.102563 · Cited by 106
A Survey of Cutting-edge Multimodal Sentiment Analysis
Singh, U., Abhishek, K., & Azad, H. K. (2024) · ACM Computing Surveys · doi:10.1145/3652149 · Cited by 103
Multi-modal emotion recognition using tensor decomposition fusion and self-supervised multi-tasking
Wang, R., Zhu, J., Wang, S., Wang, T., Huang, J., & Zhu, X. (2024) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-024-00347-3 · Cited by 95
Multimodal End-to-End Sparse Model for Emotion Recognition
Dai, W., Cahyawijaya, S., Liu, Z., & Fung, P. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.417 · Cited by 91
Gkoumas, D., Li, Q., Lioma, C., Yu, Y., & Song, D. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.09.005 · Cited by 96
Stappen, L., Baird, A., Schumann, L., & Schuller, B. W. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3097002 · Cited by 92
Multimodal transformer with adaptive modality weighting for multimodal sentiment analysis
Wang, Y., He, J., Wang, D., Wang, Q., Wan, B., & Luo, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127181 · Cited by 104
Multimodal Emotion Recognition Fusion Analysis Adapting BERT With Heterogeneous Feature Unification
Lee, S., Han, D. K., & Ko, H. (2021) · IEEE Access · doi:10.1109/access.2021.3092735 · Cited by 99
Emotional Speech-Driven Animation with Content-Emotion Disentanglement
Daněček, R., Chhatre, K., Tripathi, S., Wen, Y., Black, M. J., & Bolkart, T. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618183 · Cited by 103
What You Say and How You Say It Matters: Predicting Stock Volatility Using Verbal and Vocal Cues
Qin, Y. W., & Yang, Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1038 · Cited by 94
MissModal: Increasing Robustness to Missing Modality in Multimodal Sentiment Analysis
Lin, R., & Hu, H. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00628 · Cited by 98
Tang, J., Li, K., Jin, X., Cichocki, A. S., Zhao, Q., & Kong, W. (2021) · doi:10.18653/v1/2021.acl-long.412 · Cited by 87
Efficient Low-rank Multimodal Fusion with Modality-Specific FactorsPreprint
Liu, Z., Shen, Y., Lakshminarasimhan, V., Liang, P. P., Zadeh, A., & Morency, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00064 · Cited by 80
Self-citationMultimodal Sentiment Analysis With Two-Phase Multi-Task Learning
Yang, B., Wu, L., Zhu, J., Shao, B., Lin, X., & Liu, T. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3178204 · Cited by 82
Zeng, Y., Li, Z., Tang, Z., Chen, Z., & Ma, H. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119240 · Cited by 74
Convolutional Experts Constrained Local Model for Facial Landmark Detection
Zadeh, A., Baltrušaitis, T., & Morency, L. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.256 · Cited by 72
Self-citationMulti-Task Momentum Distillation for Multimodal Sentiment Analysis
Lin, R., & Hu, H. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3282410 · Cited by 75
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications
Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 86
Xiao, G., Tu, G., Zheng, L., Zhou, T., Li, X., Ahmed, S. H., & Jiang, D. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3015381 · Cited by 67
Multimodal Sentiment Analysis via RNN variants
Agarwal, A., Yadav, A., & Vishwakarma, D. (2019) · International Conference on Big Data, Cloud Computing, Data Science & Engineering · doi:10.1109/bcd.2019.8885108 · Cited by 61
Multimodal Phased Transformer for Sentiment Analysis
Cheng, J., Fostiropoulos, I., BOEHM, B. W., & Soleymani, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.189 · Cited by 69
Li, M., Yang, D., Lei, Y., Wang, S., Wang, S., Su, L., Yang, K., Wang, Y., Sun, M., & Zhang, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28871 · Cited by 75
Sentiment Analysis: Comprehensive Reviews, Recent Advances, and Open Challenges
Lu, Q., Sun, X., Long, Y., Gao, Z., Feng, J. H., & Sun, T. S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3294810 · Cited by 77
Li, M., Yang, D., Zhao, X., Wang, S., Wang, Y., Yang, K., Sun, M., Kou, D., Qian, Z., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01184 · Cited by 78
Combating Human Trafficking with Multimodal Deep Models
Tong, E., Zadeh, A., Jones, C., & Morency, L. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1142 · Cited by 83
Self-citationA cognitive brain model for multimodal sentiment analysis based on attention neural networks
Li, Y., Zhang, K., Wang, J., & Gao, X. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.021 · Cited by 64
Natural Language Supervision For General-Purpose Audio Representations
Elizalde, B., Deshmukh, S., & Wang, H. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448504 · Cited by 61
MEDT: Using Multimodal Encoding-Decoding Network as in Transformer for Multimodal Sentiment Analysis
Qi, Q., Lin, L., Zhang, R., & Xue, C. (2022) · IEEE Access · doi:10.1109/access.2022.3157712 · Cited by 66
Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment AnalysisPreprint
Chen, F., Luo, Z., Xu, Y., & Ke, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.08138 · Cited by 88
Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Space
Xie, Z., Yang, Y. Z., Wang, J., Liu, X., & Li, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3376564 · Cited by 65
Sentiment analysis based on text information enhancement and multimodal feature fusion
Liu, Z., Cai, L., Yang, W., & Liu, J. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110847 · Cited by 59
Deep Multimodal Emotion Recognition on Human Speech: A Review
Koromilas, P., & Giannakopoulos, T. (2021) · Applied Sciences · doi:10.3390/app11177962 · Cited by 58
Embracing Unimodal Aleatoric Uncertainty for Robust Multimodal Fusion
Gao, Z., Jiang, X., Xu, X., Shen, F., Li, Y., & Shen, H. T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02538 · Cited by 79
DNN Multimodal Fusion Techniques for Predicting Video Sentiment
Williams, J. A., Comanescu, R., Radu, O., & Tian, L. (2018) · doi:10.18653/v1/w18-3309 · Cited by 56
Large language models meet text-centric multimodal sentiment analysis: a survey
Yang, H., Zhao, Y., Wu, Y., Wang, S., Zheng, T., Zhang, H., Ma, Z., Che, W., Wang, S., Wei, S., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4593-8 · Cited by 104
Improving Depression Level Estimation by Concurrently Learning Emotion Intensity
Qureshi, S. A., Dias, G. H., Hasanuzzaman, M., & Saha, S. (2020) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2020.2998234 · Cited by 58
Lin, R., & Hu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3303711 · Cited by 59
Multimodal sentiment analysis with unimodal label generation and modality decomposition
Zhu, L., Zhao, H., Zhu, Z., Zhang, C., & Kong, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102787 · Cited by 55
SWAFN: Sentimental Words Aware Fusion Network for Multimodal Sentiment Analysis
Chen, M., & Li, X. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.93 · Cited by 54
Joint multimodal sentiment analysis based on information relevance
Chen, D., Su, W., Wu, P., & Hua, B. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103193 · Cited by 60
TCHFN: Multimodal sentiment analysis based on Text-Centric Hierarchical Fusion Network
Hou, J., Omar, N., Tiun, S. B., Saad, S., & He, Q. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112220 · Cited by 55
Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognition
Zhang, S., Chen, M., Chen, J., Li, Y.-F., Wu, Y., Li, M., & Zhu, C. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107340 · Cited by 54
Shou, Y., Liu, H., Cao, X., Meng, D., & Dong, B. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3498443 · Cited by 53
ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis
Yu, Y., Zhao, M. M., Qi, S.-A., Sun, F., Wang, B., Guo, W., Wang, X. L., Yang, L., & Niu, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.860 · Cited by 58
Feature Fusion for Multimodal Emotion Recognition Based on Deep Canonical Correlation Analysis
Zhang, K., Li, Y., Wang, J., Wang, Z., & Li, X. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3112314 · Cited by 51
Shared-Private Memory Networks For Multimodal Sentiment Analysis
Zhao, X., Chen, Y., Liu, S., & Tang, B. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3222023 · Cited by 50
Xiao, L., Wu, X., Wu, W., Yang, J., & He, L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747542 · Cited by 49
Li, Z., Guo, Q., Pan, Y., Ding, W., Yu, J., Zhang, Y., Liu, W., Chen, H., Wang, H., & Xie, Y. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101891 · Cited by 49
FMSA-SC: A Fine-Grained Multimodal Sentiment Analysis Dataset Based on Stock Comment Videos
Song, L., Chen, S., Meng, Z., Sun, M., & Shang, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3363641 · Cited by 49
CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French
Zadeh, A. B., Cao, Y., Hessner, S., Liang, P. P., Poria, S., & Morency, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.141 · Cited by 46
Self-citationMultimodal Mutual Attention-Based Sentiment Analysis Framework Adapted to Complicated Contexts
He, L., Wang, Z., Wang, L., & Li, F. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3276075 · Cited by 45
Metadata
- Authors
- Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.06259
- OpenAlex
- W2465534249
- Cited by
- 350
- References
- 33
- Citations by year
- 2026: 2 · 2025: 38 · 2024: 74 · 2023: 52 · 2022: 42 · 2021: 58 · 2020: 32 · 2019: 24 · 2018: 19 · 2017: 7 · 2016: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- sentiment intensity, subjectivity, online video sharing, multimodal fusion, video sentiment analysis, audio features, annotated corpus, spoken word
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
