The DOI matches its DataCite record; 33 of the 33 references it cites were found in a registry or an index.
MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos
Open accessAbstract
People are sharing their opinions, stories and reviews through online video sharing websites every day. Studying sentiment and subjectivity in these opinion videos is experiencing a growing attention from academia and industry. While sentiment analysis has been successful for text, it is an understudied research question for videos and multimedia content. The biggest setbacks for studies in this direction are lack of a proper dataset, methodology, baselines and statistical analysis of how information from different modality sources relate to each other. This paper introduces to the scientific community the first opinion-level annotated corpus of sentiment and subjectivity analysis in online videos called Multimodal Opinion-level Sentiment Intensity dataset (MOSI). The dataset is rigorously annotated with labels for subjectivity, sentiment intensity, per-frame and per-opinion annotated visual features, and per-milliseconds annotated audio features. Furthermore, we present baselines for future studies in this direction as well as a new multimodal fusion approach that jointly models spoken words and visual gestures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The dataset includes 93 randomly selected videos from YouTube, involving 89 different speakers—41 females and 48 males.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“CMU-MOSI is a collection of 2 , 199 opinion video clips each rigorously annotated with labels for subjectivity, sentiment intensity, per-frame, and per-opinion annotated visual features, and permilliseconds annotated audio features [181].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [181]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Therefore the features used in this paper cannot be used to identify the speaker [181, 183].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [181]
Experimental — accuracy not yet measuredIn the citing paper
“The videos are segmented into 2199 clips, each rated on an emotional polarity scale, from +3 (signifying strongly positive sentiment) to −3 (signifying strongly negative sentiment).”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
In the citing paper
“The Multimodal Opinion-level Sentiment Intensity (CMU-MOSI) dataset, introduced by Zadeh et al. in 2016 [28], marks a significant milestone in multimodal sentiment analysis databases due to its unique characteristic of incorporating subjective sentiment and emotional intensity annotations.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,723
Praat: Doing Phonetics by Computer
(2011) · Ear and Hearing · doi:10.1097/aud.0b013e31821473f7 · Cited by 8,010
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,646
Pang, B., & Lee, L. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218990 · Cited by 3,376
Lexicon-based methods for sentiment analysis
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011) · Computational Linguistics · doi:10.1162/coli_a_00049 · Cited by 3,301
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Computers and the Humanities · doi:10.1007/s10579-005-7880-9 · Cited by 1,794
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/979617.979640 · Cited by 822
COVAREP — A collaborative voice analysis repository for speech technologies
Degottex, G., Kane, J., Drugman, T., Raitio, T., & Scherer, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853739 · Cited by 695
Poria, S., Cambria, E., & Gelbukh, A. F. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1303 · Cited by 559
A principal component analysis of facial expressions
Calder, A. J., Burton, A. M., Miller, P. C., Young, A. W., & Akamatsu, S. (2001) · Vision Research · doi:10.1016/s0042-6989(01)00002-5 · Cited by 471
Towards multimodal sentiment analysis: harvesting opinions from the web
Morency, L.-P., Mihalcea, R., & Doshi, P. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070509 · Cited by 470
Self-citationYoutube movie reviews: Sentiment analysis in an audio-visual context
Wollmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.34 · Cited by 403
Self-citationLearning Multilingual Subjective Language via Cross-Lingual Projections
Mihalcea, R. F., Banea, C., & Wiebe, J. (2007) · University of North Texas Digital Library (University of North Texas) · Cited by 395
Human expression recognition from motion using a radial basis function network architecture
Rosenblum, M., Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536309 · Cited by 284
Multimodal Sentiment Analysis of Spanish Online Videos
Rosas, V. P., Mihalcea, R. F., & Morency, L. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.9 · Cited by 228
Self-citationEmotion Recognition In The Wild Challenge 2014
Dhall, A., Goecke, R., Joshi, J., Sikka, K., & Gedeon, T. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2666275 · Cited by 216
Cross-Domain Co-Extraction of Sentiment and Topic Lexicons
Li, F., Pan, S. J., Jin, O., Yang, Q., & Zhu, X. (2012) · Cited by 175
Exploring Fusion Methods for Multimodal Emotion Recognition with Missing Data
Wagner, J., Andre, E., Lingenfelser, F., & Kim, J. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.12 · Cited by 156
Extracting Opinion Expressions with semi-Markov Conditional Random Fields
Yang, B., & Cardie, C. (2012) · Cited by 139
Balog, K., Mishne, G., & de Rijke, M. (2006) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/1608974.1609010 · Cited by 118
Multimodal subjectivity analysis of multiparty conversation
Raaijmakers, S., Truong, K. P., & Wilson, T. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613774 · Cited by 104
Context-Enhanced Citation Sentiment Detection
Athar, A., & Teufel, S. (2012) · Cited by 104
Continuous Conditional Neural Fields for Structured Regression
Baltrušaitis, T., Robinson, P., & Morency, L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_39 · Cited by 101
Self-citationLatent Variable Models for Semantic Orientations of Phrases
Takamura, H., Inui, T., & Okumura, M. (2006) · Cited by 97
Liars and Saviors in a Sentiment Annotated Corpus of Comments to Political Debates
Carvalho, P., Sarmento, L., Teixeira, J., & Silva, M. J. (2011) · Cited by 90
Summarizing Emails with Conversational Cohesion and Subjectivity
Carenini, G., Ng, R. T., & Zhou, X. (2008) · Cited by 66
Rendering of Eyes for Eye-Shape Registration and Gaze EstimationPreprint
Wood, E., Baltrušaitis, T., Zhang, X., Sugano, Y., Robinson, P., & Bulling, A. (2015) · arXiv (Cornell University) · doi:10.17863/cam.32367 · Cited by 59
Why Question Answering using Sentiment Analysis and Word Classes
Oh, J., Torisawa, K., Hashimoto, C., Kawada, T., Saeger, S. D., Kazama, J., & Wang, Y. (2012) · Cited by 52
Can prosody inform sentiment analysis? Experiments on short spoken reviews
Mairesse, F., Polifroni, J. H., & Fabbrizio, G. D. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289066 · Cited by 46
Fusion of Acoustic and Linguistic Features for Emotion Detection
Metze, F., Polzehl, T., & Wagner, M. (2009) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2009.32 · Cited by 20
Manual annotation of opinion categories in meetings
Somasundaran, S., Wiebe, J., Hoffmann, P., & Litman, D. J. (2006) · doi:10.3115/1641991.1641998 · Cited by 14
Research works printed but not linked: 1
- [37]37. Vinodhini, G., and R. M. Chandrasekaran. "Sentiment analysis and opinion mining: a survey." International Journal 2, no. 6 (2012).
Cited by
Sorted by citations
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Self-citationMultimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433
Self-citationEfficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170
Self-citationSentiment analysis using deep learning architectures: a review
Yadav, A., & Vishwakarma, D. K. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09794-5 · Cited by 841
Yu, W., Xu, H., Yuan, Z., & Wu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17289 · Cited by 816
Metadata
- Authors
- Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.06259
- OpenAlex
- W2465534249
- Cited by
- 350
- References
- 33
- Citations by year
- 2026: 2 · 2025: 38 · 2024: 74 · 2023: 52 · 2022: 42 · 2021: 58 · 2020: 32 · 2019: 24 · 2018: 19 · 2017: 7 · 2016: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- sentiment intensity, subjectivity, online video sharing, multimodal fusion, video sentiment analysis, audio features, annotated corpus, spoken word
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
