The DOI matches its Crossref record; 160 of the 160 references it cites were found in a registry or an index.
A survey of multimodal sentiment analysis
Open accessAbstract
Abstract Sentiment analysis aims to automatically uncover the underlying attitude that we hold towards an entity. The aggregation of these sentiment over a population represents opinion polling and has numerous applications. Current text-based sentiment analysis rely on the construction of dictionaries and machine learning models that learn sentiment from large text corpora. Sentiment analysis from text is currently widely used for customer satisfaction assessment and brand perception analysis, among others. With the proliferation of social media, multimodal sentiment analysis is set to bring new opportunities with the arrival of complementary data streams for improving and going beyond text-based sentiment analysis. Since sentiment can be detected through affective traces it leaves, such as facial and vocal displays, multimodal sentiment analysis offers promising avenues for analyzing facial and vocal expressions in addition to the transcript or textual content. These approaches leverage emotion recognition and context inference to determine the underlying polarity and scope of an individual's sentiment. In this survey, we define sentiment and the problem of multimodal sentiment analysis and review recent developments in multimodal sentiment analysis in different domains, including spoken reviews, images, video blogs, human–machine and human–human interactions. Challenges and opportunities of this emerging field are also discussed leading to our thesis that multimodal sentiment analysis holds a significant untapped potential.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 1×
“, WordNet [134], ConceptNet [135], and General Inquirer [136] for inferring the speaker’s sentiment.”
BackgroundAdvances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2017) · doi:10.1109/cvpr35066.2017 · Cited by 7,141
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 6,961
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,720
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
What are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,650
Recognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220619 · Cited by 3,345
Lexicon-based methods for sentiment analysis
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011) · Computational Linguistics · doi:10.1162/coli_a_00049 · Cited by 3,301
International Affective Picture System (IAPS) : Technical Manual and Affective Ratings
Lang, P. J. (1995) · Medical Entomology and Zoology · Cited by 2,870
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining
Baccianella, S., Esuli, A., & Sebastiani, F. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jktnrykui9s · Cited by 2,666
Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642
Self-citationAppraisal processes in emotion: Theory, methods, research.
Scherer, K. R., Schorr, A., & Johnstone, T. (2001) · Cited by 2,362
Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews
Turney, P. D. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073153 · Cited by 2,352
Thumbs up? Sentiment Classification using Machine Learning Techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205070 · Cited by 2,191
Predicting the future with social media
Asur, S., & Huberman, B. A. (2010) · 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology · doi:10.1109/wi-iat.2010.63 · Cited by 2,081
(2007) · Cited by 2,018
IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2001)
Hillman, P., Hannah, J. M., & Renshaw, D. (2001) · Cited by 1,998
Norms of valence, arousal, and dominance for 13,915 English lemmas
Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0314-x · Cited by 1,959
Mining the peanut gallery: opinion extraction and semantic classification of product reviews
Dave, K., Lawrence, S., & Pennock, D. M. (2003) · Proceedings of the twelfth international conference on World Wide Web - WWW '03 · doi:10.1145/775152.775226 · Cited by 1,916
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Language Resources and Evaluation · doi:10.1007/s10579-005-7880-9 · Cited by 1,755
Twitter power: Tweets as electronic word of mouth
Jansen, B. J., Zhang, M., Sobel, K., & Chowdury, A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21149 · Cited by 1,642
Determining the sentiment of opinions
Kim, S.-M., & Hovy, E. (2004) · Proceedings of the 20th international conference on Computational Linguistics - COLING '04 · doi:10.3115/1220355.1220555 · Cited by 1,482
Sentiment strength detection in short informal text
Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., & Kappas, A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21416 · Cited by 1,449
A Survey of Opinion Mining and Sentiment Analysis
Liu, B., & Zhang, L. (2012) · Mining Text Data · doi:10.1007/978-1-4614-3223-4_13 · Cited by 1,323
New Avenues in Opinion Mining and Sentiment Analysis
Cambria, E., Schuller, B., Xia, Y., & Havasi, C. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.30 · Cited by 1,285
Self-citationLearning Sentiment-Specific Word Embedding for Twitter Sentiment Classification
Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., & Qin, B. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1146 · Cited by 1,181
The world of emotions is not two-dimensional
Fontaine, J. R. J., Scherer, K. R., Roesch, E. B., & Ellsworth, P. C. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.02024.x · Cited by 1,179
The general inquirer: A computer approach to content analysis.
Smith, M. S., Stone, P. J., & Dunphy, D. C. (1966) · Medical Entomology and Zoology · Cited by 1,165
Association for Computational Linguistics
Hirst, G. (2006) · Encyclopedia of Language & Linguistics · doi:10.1016/b0-08-044854-2/05234-2 · Cited by 1,140
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012
Müller, A., Sejdinović, D., & Piechocki, R. J. (2012) · Cited by 1,027
SemEval-2016 Task 4: Sentiment Analysis in Twitter
Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., & Stoyanov, V. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1001 · Cited by 966
Large-scale visual sentiment ontology and detectors using adjective noun pairs
Borth, D., Ji, R., Chen, T., Breuel, T., & Chang, S.-F. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502282 · Cited by 851
Self-citationSUN attribute database: Discovering, annotating, and recognizing scene attributes
Patterson, G., & Hays, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247998 · Cited by 836
Li, L.-J., Su, H., Fei-Fei, L., & Xing, E. P. (2018) · KiltHub Repository · doi:10.1184/r1/6475985 · Cited by 827
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics - · doi:10.3115/979617.979640 · Cited by 815
ImageNet classification with deep convolutional neural networks
Krizhevsky (2012) · Medical Entomology and Zoology · Cited by 799
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Self-citationThe General Inquirer: A Computer Approach to Content Analysis.
Hartman, J. J., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvia, D. M. (1967) · American Sociological Review · doi:10.2307/2092070 · Cited by 723
A systems approach to appraisal mechanisms in emotion
Sander, D., Grandjean, D., & Scherer, K. R. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.001 · Cited by 700
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 685
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D'mello, S. K., & Kory, J. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 628
Proceedings of the 26th International Conference on World Wide Web - WWW '17
(2017) · doi:10.1145/3038912 · Cited by 555
Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks
You, Q., Luo, J., Jin, H., & Yang, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9179 · Cited by 548
Fusing audio, visual and textual clues for sentiment analysis from multimodal content
Poria, S., Cambria, E., Howard, N., Huang, G.-B., & Hussain, A. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.01.095 · Cited by 534
Poria, S., Cambria, E., & Gelbukh, A. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1303 · Cited by 533
A model of textual affect sensing using real-world knowledge
Liu, H., Lieberman, H., & Selker, T. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604067 · Cited by 531
The Berlin Affective Word List Reloaded (BAWL-R)
Võ, M. L. H., Conrad, M., Kuchinke, L., Urton, K., Hofmann, M. J., & Jacobs, A. M. (2009) · Behavior Research Methods · doi:10.3758/brm.41.2.534 · Cited by 525
Polanyi, L., & Zaenen, A. (2006) · The Information Retrieval Series · doi:10.1007/1-4020-4102-0_1 · Cited by 521
Sentiment analysis of blogs by combining lexical knowledge with text classification
Melville, P., Gryc, W., & Lawrence, R. D. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557156 · Cited by 503
Development and use of a gold-standard data set for subjectivity classifications
Wiebe, J. M., Bruce, R. F., & O'Hara, T. P. (1999) · Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics - · doi:10.3115/1034678.1034721 · Cited by 503
Towards multimodal sentiment analysis: harvesting opinions from the web
Morency, L.-P., Mihalcea, R., & Doshi, P. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070509 · Cited by 470
SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods
Ribeiro, F. N., Araújo, M., Gonçalves, P., André Gonçalves, M., & Benevenuto, F. (2016) · EPJ Data Science · doi:10.1140/epjds/s13688-016-0085-1 · Cited by 468
Proceedings of The 12th International Workshop on Semantic Evaluation
(2018) · doi:10.18653/v1/2026.semeval-1 · Cited by 452
NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets
Mohammad, S. M., Kiritchenko, S., & Zhu, X. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.6242 · Cited by 451
OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit
Eyben, F., Wollmer, M., & Schuller, B. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349350 · Cited by 418
Self-citationEmpath: Understanding Topic Signals in Large-Scale Text
Fast, E., Chen, B., & Bernstein, M. S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858535 · Cited by 413
Representing General Relational Knowledge in ConceptNet 5
Speer, R., & Havasi, C. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3zycidvg6dgg · Cited by 407
Youtube movie reviews: Sentiment analysis in an audio-visual context
Wollmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.34 · Cited by 403
Self-citationHuman language reveals a universal positivity bias
Dodds, P. S., Clark, E. M., Desu, S., Frank, M. R., Reagan, A. J., Williams, J. R., Mitchell, L., Harris, K. D., Kloumann, I. M., Bagrow, J. P., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1411678112 · Cited by 401
Opinion Mining with Deep Recurrent Neural Networks
Irsoy, O., & Cardie, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1080 · Cited by 384 · cited 1×
“Deep Recurrent Neural Networks have been applied to the task of subjectivity detection [20], and word vector representations can combine supervised and unsupervised learning when applied to sentiment analysis [80].”
MethodAre they different? Affect, feeling, emotion, sentiment, and opinion detection in text
Munezero, M., Montero, C. S., Sutinen, E., & Pajunen, J. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2317187 · Cited by 374
Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word Corpus
Turney, P., & Littman, M. L. (2002) · arXiv (Cornell University) · doi:10.4224/8914027 · Cited by 359
The emotions: a philosophical introduction
(2013) · Choice Reviews Online · doi:10.5860/choice.50-2587 · Cited by 357
The Spanish adaptation of ANEW (Affective Norms for English Words)
Redondo, J., Fraga, I., Padrón, I., & Comesaña, M. (2007) · Behavior Research Methods · doi:10.3758/bf03193031 · Cited by 345
Khosla, A., Das Sarma, A., & Hamid, R. (2014) · Proceedings of the 23rd international conference on World wide web · doi:10.1145/2566486.2567996 · Cited by 343
openSMILE - The Munich Versatile and Fast Open-Source Audio Feature Extractor
Eyben, F. ;. (2013) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 321
Recurrent Human Pose Estimation
Belagiannis, V., & Zisserman, A. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.64 · Cited by 313
(2020) · Routledge eBooks · doi:10.4324/9781003064060 · Cited by 299
2022 IEEE International Conference on Image Processing (ICIP)
(2022) · doi:10.1109/icip46576.2022 · Cited by 294
Affect analysis of text using fuzzy semantic typing
Subasic, P., & Huettner, A. (2001) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.940962 · Cited by 292
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288 · cited 1×
“Linguistic features were used to train a linear SVM, and audiovisual features were used to train a Bidirectional Long-Short-Term-Memory (BLSTM) recurrent neural network [137].”
MethodSelf-citationMeasuring the Happiness of Large-Scale Written Expression: Songs, Blogs, and Presidents
Dodds, P. S., Christopher, D. Æ., & Danforth, M. (2015) · Cited by 280
The 2017 Conference on Empirical Methods on Natural Language Processing (EMNLP)
Narayan, S., Gardent, C., Cohen, S. B., & Shimorina, A. (2015) · Cited by 276
Measuring the Happiness of Large-Scale Written Expression: Songs, Blogs, and Presidents
Dodds, P. S., & Danforth, C. M. (2010) · Journal of Happiness Studies · doi:10.1007/s10902-009-9150-9 · Cited by 260
DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks
Chen, T., Borth, D., Darrell, T. J., & Chang, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.8586 · Cited by 257
Self-citationCoooolll: A Deep Learning System for Twitter Sentiment Classification
Tang, D., Wei, F., Qin, B., Liu, T., & Zhou, M. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2033 · Cited by 243
Multimodal Sentiment Analysis of Spanish Online Videos
Perez Rosas, V., Mihalcea, R., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.9 · Cited by 227
Analyzing and predicting sentiment of images on the social web
Siersdorfer, S., Minack, E., Deng, F., & Hare, J. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874060 · Cited by 218
SentiCap: Generating Image Descriptions with Sentiments
Mathews, A., Xie, L., & He, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10475 · Cited by 213
Multilingual Twitter Sentiment Classification: The Role of Human Annotators
Mozetič, I., Grčar, M., & Smailović, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0155036 · Cited by 203
Sentribute: image sentiment analysis from a mid-level perspective
Yuan, J., Mcdonough, S., You, Q., & Luo, J. (2013) · Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining · doi:10.1145/2502069.2502079 · Cited by 199
From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction
Campos, V., Jou, B., & Giró-i-Nieto, X. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.011 · Cited by 197
Self-citationEmotional valence categorization using holistic image features
Yanulevskaya, V., van Gemert, J. C., Roth, K., Herbold, A. K., Sebe, N., & Geusebroek, J. M. (2008) · 2008 15th IEEE International Conference on Image Processing · doi:10.1109/icip.2008.4711701 · Cited by 190
Borth, D., Chen, T., Ji, R., & Chang, S.-F. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502268 · Cited by 188
Self-citationSentiment Analysis: From Opinion Mining to Human-Agent Interaction
Clavel, C., & Callejas, Z. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2444846 · Cited by 176
Predicting Ad Liking and Purchase Intent: Large-Scale Analysis of Facial Responses to Ads
McDuff, D., Kaliouby, R. E., Cohn, J. F., & Picard, R. W. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2384198 · Cited by 164
Dellaert, F., Polzin, T., & Waibel, A. (1996) · 4th International Conference on Spoken Language Processing (ICSLP 1996) · doi:10.21437/icslp.1996-462 · Cited by 158
Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction
Wei-ning, W., Ying-lin, Y., & Sheng-ming, J. (2006) · 2006 IEEE International Conference on Systems, Man and Cybernetics · doi:10.1109/icsmc.2006.384667 · Cited by 158
Deriu, J., Gonzenbach, M., Uzdilli, F., Lucchi, A., Luca, V. D., & Jaggi, M. (2016) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s16-1173 · Cited by 153
Twitter Sentiment Analysis: A Bootstrap Ensemble Framework
Hassan, A., Abbasi, A., & Zeng, D. D. (2013) · International Conference on Social Computing · doi:10.1109/socialcom.2013.56 · Cited by 150
Visual Sentiment Prediction with Deep Convolutional Neural NetworksPreprint
Xu, C., Cetintas, S., Lee, K.-C., & Li, L.-J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.5731 · Cited by 149
Predicting Emotions in User-Generated Videos
Jiang, Y., Xu, B., & Xue, X. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8724 · Cited by 149
Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification
Deriu, J., Lucchi, A., Luca, V. D., Severyn, A., Müller, S., Cieliebak, M., Hofmann, T., & Jaggi, M. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052611 · Cited by 148
Object Bank: An Object-Level Image Representation for High-Level Visual Recognition
Li, L.-J., Su, H., Lim, Y., & Fei-Fei, L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0660-x · Cited by 146
Development of Multimodal Interfaces: Active Listening and Synchrony
Esposito, A. A., Campbell, N., Vogel, C., Hussain, A., Nijholt, A., Esposito, A. A., Campbell, N., Campbell, N., Vogel, C. M., Hussain, A., et al. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12397-9 · Cited by 138
Object-Based Visual Sentiment Concept Analysis and Application
Chen, T., Yu, F. X., Chen, J., Cui, Y., Chen, Y., & Chang, S. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654935 · Cited by 136
Self-citationDirection-based text interpretation as an information access refinement
Hearst, M. A. (1992) · Cited by 135
Context-Sensitive Dynamic Ordinal Regression for Intensity Estimation of Facial Action Units
Rudovic, O. O., Pavlović, V., & Pantić, M. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2356192 · Cited by 122
Yahoo! For Amazon: Sentiment Parsing from Small Talk on the WebPreprint
Das, S. R., & Chen, M. Y. (2001) · SSRN Electronic Journal · doi:10.2139/ssrn.276189 · Cited by 121
Can we understand van gogh's mood?
Jia, J., Wu, S., Wang, X., Hu, P., Cai, L., & Tang, J. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2396330 · Cited by 119
Sentic Computing: Exploitation of Common Sense for the Development of Emotion-Sensitive Systems
Cambria, E., Hussain, A., Havasi, C., & Eckl, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12397-9_12 · Cited by 111
Deep Cross Residual Learning for Multitask Visual Recognition
Jou, B., & Chang, S. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964309 · Cited by 102
Self-citationHow useful are corpus-based methods for extrapolating psycholinguistic variables?
Mandera, P., Keuleers, E., & Brysbaert, M. (2015) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2014.988735 · Cited by 99
Webis: An Ensemble for Twitter Sentiment Detection
Hagen, M., Potthast, M., Büchner, M., & Stein, B. (2015) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s15-2097 · Cited by 96
Benchmarking Twitter Sentiment Analysis Tools
Abbasi, A., Hassan, A., & Dhar, M. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5dap2x48c977 · Cited by 95
Bousmalis, K., Méhu, M., & Pantić, M. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.07.003 · Cited by 93
Diving Deep into Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment Prediction
Campos, V., Salvador, A., Giró-i-Nieto, X., & Jou, B. (2015) · doi:10.1145/2813524.2813530 · Cited by 82
Self-citationPredicting Viewer Perceived Emotions in Animated GIFs
Jou, B., Bhattacharya, S., & Chang, S. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2656408 · Cited by 81
Self-citationA conceptual dependency parser for natural language
Schank, R. C., & Tesler, L. (1969) · International Conference on Computational Linguistics · doi:10.3115/990403.990405 · Cited by 75
Measuring Voter's Candidate Preference Based on Affective Responses to Election Debates
McDuff, D., el Kaliouby, R., Kodra, E., & Picard, R. W. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.67 · Cited by 72
On the computation of point of view
Sack, W. (1994) · National Conference on Artificial Intelligence · Cited by 66
Towards a comprehensive computational model foraesthetic assessment of videos
Bhattacharya, S., Nojavanasghari, B., Chen, T., Liu, D., Chang, S., & Shah, M. A. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2508119 · Cited by 65
Self-citationThe dynamics of emotions in online interaction
García, D., Kappas, A., Küster, D., & Schweitzer, F. (2016) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000119463 · Cited by 62
Milstead, J. L. (1993) · Information Processing & Management · doi:10.1016/0306-4573(93)90064-k · Cited by 60
Sentiment extraction from natural audio streams
Kaushik, L., Sangwan, A., & Hansen, J. H. L. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639321 · Cited by 57
Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation
Kaltwang, S., Todorović, S., & Pantić, M. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2501824 · Cited by 55
Analysis of adjective-noun word pair extraction methods for online review summarization
Yatani, K., Novati, M., Trusty, A., & Truong, K. N. (2011) · doi:10.5591/978-1-57735-516-8/ijcai11-461 · Cited by 51
Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity
Walecki, R., Rudovic, O., Pavlovic, V., & Pantic, M. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2016.530 · Cited by 50
Self-citationCHI Conference on Human Factors in Computing Systems
Limerick, H. (2015) · Cited by 50
Exploiting Multimodal Affect and Semantics to Identify Politically Persuasive Web Videos
Siddiquie, B., Chisholm, D., & Divakaran, A. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2820732 · Cited by 48
Can prosody inform sentiment analysis? Experiments on short spoken reviews
Mairesse, F., Polifroni, J. H., & Fabbrizio, G. D. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289066 · Cited by 45
Predicting Viewer Affective Comments Based on Image Content in Social Media
Chen, Y., Chen, T., Hsu, W. H., Liao, H.-Y. M., & Chang, S. (2014) · International Conference on Multimedia Retrieval · doi:10.1145/2578726.2578756 · Cited by 43
Self-citationInfinite Hidden Conditional Random Fields for Human Behavior Analysis
Bousmalis, K., Zafeiriou, S. P., Morency, L., & Pantić, M. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2224882 · Cited by 43
Robust Correlated and Individual Component Analysis
Panagakis, Y., Nicolaou, M. A., Zafeiriou, S. P., & Pantić, M. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2497700 · Cited by 43
Emotional states vs. emotional words in social media
Beasley, A., & Mason, W. (2015) · Proceedings of the ACM Web Science Conference · doi:10.1145/2786451.2786473 · Cited by 38
Emotion recognition using imperfect speech recognition
Metze, F., Batliner, A. M., Eyben, F., Polzehl, T., Schuller, B. W., & Steidl, S. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-202 · Cited by 36
Self-citationMicro-opinion Sentiment Intensity Analysis and Summarization in Online Videos
Zadeh, A. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2823317 · Cited by 32
Robust Canonical Correlation Analysis: Audio-visual fusion for learning continuous interest
Nicolaou, M. A., Panagakis, Y., Zafeiriou, S. P., & Pantić, M. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853852 · Cited by 28
Automatic sentiment extraction from YouTube videos
Kaushik, L., Sangwan, A., & Hansen, J. H. L. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707736 · Cited by 28
Automatic Detection of CSA Media by Multi-modal Feature Fusion for Law Enforcement Support
SCHULZE, C. H., Henter, D., Borth, D., et al. (2014) · doi:10.1145/2578726.2578772 · Cited by 26
Why We Watch the News: A Dataset for Exploring Sentiment in Broadcast Video News
Ellis, J. G., Jou, B., & Chang, S. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663237 · Cited by 25
Self-citationThe Community and the Crowd: Multimedia Benchmark Dataset Development
Larson, M., Soleymani, M., Eskevich, M., Serdyukov, P., Ordelman, R., & Jones, G. J. F. (2012) · IEEE Multimedia · doi:10.1109/mmul.2012.27 · Cited by 25
Self-citationSentiment analysis of online spoken reviews
Pérez‐Rosas, V., & Mihalcea, R. F. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-243 · Cited by 22
A Benchmark Comparison of State-of-the-Practice Sentiment Analysis Methods.Preprint
Ribeiro, F. N., Araújo, M., Gonçalves, P., Benevenuto, F., & Gonçalves, M. A. (2015) · arXiv (Cornell University) · Cited by 18
Mapping Images to Sentiment Adjective Noun Pairs with Factorized Neural NetsPreprint
Narihira, T., Borth, D., Yu, S. X., Ni, K. S., & Darrell, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06838 · Cited by 15
Langlet, C., & Clavel, C. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.05.038 · Cited by 14
Automatic recognition of attitudes in video blogs — prosodic and visual feature analysis
Madzlan, N. A., Han, J., Bonin, F., & Campbell, N. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-415 · Cited by 14
Sentiment retrieval on web reviews using spontaneous natural speech
Pereira, J. C., Luque, J., & Anguera, X. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854470 · Cited by 12
Langlet, C., & Clavel, C. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344545 · Cited by 11
Liu, H., Jou, B., Chen, T., Topkara, M., Pappas, N., Реди, М., & Chang, S. (2016) · International Conference on Multimedia Retrieval · doi:10.1145/2911996.2912030 · Cited by 9
Self-citationVariational Infinite Hidden Conditional Random Fields
Bousmalis, K., Zafeiriou, S. P., Morency, L., Pantić, M., & Ghahramani, Z. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2388228 · Cited by 7
Toward Automatic Censorship Detection in Microblogs
Morrison, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13186-3_51 · Cited by 2
Cited by
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
A survey on sentiment analysis methods, applications, and challenges
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10144-1 · Cited by 1,528
Sentiment analysis using deep learning architectures: a review
Yadav, A., & Vishwakarma, D. K. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09794-5 · Cited by 841
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.025 · Cited by 783
Sun, Z., Sarma, P., Sethares, W., & Liang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6431 · Cited by 410
Metadata
- Authors
- Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.-F., & Pantic, M.
- Venue
- Image and Vision Computing
- Publisher
- Elsevier BV
- Published
- 2017-09
- Volume
- 65
- Pages
- 3-14
- ISSN
- 0262-8856 (ISSN-L)
- OpenAlex
- W2753840835
- Cited by
- 601
- References
- 160
- FWCI
- 22.14
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.