The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark
Open accessAbstract
Psychological research results have confirmed that people can have different emotional reactions to different visual stimuli. Several papers have been published on the problem of visual emotion analysis. In particular, attempts have been made to analyze and predict people's emotional reaction towards images. To this end, different kinds of hand-tuned features are proposed. The results reported on several carefully selected and labeled small image data sets have confirmed the promise of such features. While the recent successes of many computer vision related tasks are due to the adoption of Convolutional Neural Networks (CNNs), visual emotion analysis has not achieved the same level of success. This may be primarily due to the unavailability of confidently labeled and relatively large image data sets for visual emotion analysis. In this work, we introduce a new data set, which started from 3+ million weakly labeled images of different emotions and ended up 30 times as large as the current largest publicly available visual emotion data set. We hope that this data set encourages further research on visual emotion analysis. We also perform extensive benchmarking analyses on this large data set using the state of the art methods including CNNs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
(2008) · Journal of Machine Learning Research · Cited by 33,051
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
International Affective Picture System (IAPS) : Technical Manual and Affective Ratings
Lang, P. J. (1995) · Medical Entomology and Zoology · Cited by 2,870
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534
Fully Convolutional Networks for Semantic Segmentation
Long, J., Shelhamer, E., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.4038 · Cited by 2,489
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213
A Bio‐Informational Theory of Emotional Imagery
Lang, P. J. (1979) · Psychophysiology · doi:10.1111/j.1469-8986.1979.tb01511.x · Cited by 1,750 · cited 3×
“2014), including (1) IAPSSubset: This data set is a subset of the International Affective Picture System (IAPS) (Lang, Bradley, and Cuthbert 1999). This data set is categorized into eight emotional categories as shown in Figure 1 in a study conducted in (Mikels et al. 2005). (2) ArtPhoto: Machajdik and Hanbury (2010)…”
BackgroundMethod(2011) · doi:10.5591/978-1-57735-516-8/ijcai11-210 · Cited by 1,229
Valdez, P., & Mehrabian, A. (1994) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.123.4.394 · Cited by 1,104 · cited 2×
“For example, based on art theory (Itten and Van Haagen 1973; Valdez and Mehrabian 1994), Machajdik and Hanbury (Machajdik and Hanbury 2010) defined eight different kinds of pixel level features (e.g. color, texture and composition), which have been empirically proved to be related to emotional…”
MethodEmotion, motivation, and anxiety: brain mechanisms and psychophysiology
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1998) · Biological Psychiatry · doi:10.1016/s0006-3223(98)00275-3 · Cited by 1,043 · cited 1×
“Psychological studies have provided evidence that human emotions can be aroused by visual content, e.g. images (Lang 1979; Lang, Bradley, and Cuthbert 1998; Joshi et al. 2011).”
BackgroundAVA: A large-scale database for aesthetic visual analysis
Murray, N., Marchesotti, L., & Perronnin, F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247954 · Cited by 936
Affective image classification using features inspired by psychology and art theory
Machajdik, J., & Hanbury, A. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873965 · Cited by 852
Large-scale visual sentiment ontology and detectors using adjective noun pairs
Borth, D., Ji, R., Chen, T., Breuel, T., & Chang, S.-F. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502282 · Cited by 851
The Effects of Color on Emotions
Kane, B. A., Campbell, J. E., & Katz, G. S. (2013) · PsycEXTRA Dataset · doi:10.1037/e314722004-001 · Cited by 730
Emotional category data on images from the international affective picture system
Mikels, J. A., Fredrickson, B. L., Larkin, G. R., Lindberg, C. M., Maglio, S. J., & Reuter-Lorenz, P. A. (2005) · Behavior Research Methods · doi:10.3758/bf03192732 · Cited by 677
Learning Convolutional Feature Hierarchies for Visual Recognition
Kavukcuoglu, K., Sermanet, P., Boureau, Y.-L., Gregor, K., Mathieu, M., & LeCun, Y. (2010) · neural information processing systems · Cited by 604
Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks
You, Q., Luo, J., Jin, H., & Yang, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9179 · Cited by 548
Self-citationAesthetics and Emotions in Images
Joshi, D., Datta, R., Fedorovskaya, E., Luong, Q.-T., Wang, J., Li, J., & Luo, J. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.941851 · Cited by 408
Self-citationRAPID: Rating Pictorial Aesthetics using Deep Learning
Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654927 · Cited by 383
Self-citationExploring Principles-of-Art Features For Image Emotion Recognition
Zhao, S., Gao, Y., Jiang, X., Yao, H., Chua, T.-S., & Sun, X. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654930 · Cited by 363
The art of color : the subjective experience and objective rationale of color
Itten, J. (1961) · Cited by 308
Extracting moods from pictures and sounds: towards truly personalized TV
Hanjalic, A. (2006) · IEEE Signal Processing Magazine · doi:10.1109/msp.2006.1621452 · Cited by 227
Emotional valence categorization using holistic image features
Yanulevskaya, V., van Gemert, J. C., Roth, K., Herbold, A. K., Sebe, N., & Geusebroek, J. M. (2008) · 2008 15th IEEE International Conference on Image Processing · doi:10.1109/icip.2008.4711701 · Cited by 190
On shape and the computability of emotions
Lu, X., Suryanarayan, P., Adams, R. B., Li, J., Newman, M. G., & Wang, J. Z. (2012) · Proceedings of the 20th ACM international conference on Multimedia · doi:10.1145/2393347.2393384 · Cited by 163
Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction
Wei-ning, W., Ying-lin, Y., & Sheng-ming, J. (2006) · 2006 IEEE International Conference on Systems, Man and Cybernetics · doi:10.1109/icsmc.2006.384667 · Cited by 158
Can we understand van gogh's mood?
Jia, J., Wu, S., Wang, X., Hu, P., Cai, L., & Tang, J. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2396330 · Cited by 119
A multi-layer hybrid framework for dimensional emotion classification
Nicolaou, M. A., Güneş, H., & Pantić, M. (2011) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2072298.2071906 · Cited by 42
Cited by
DOTA: A Large-Scale Dataset for Object Detection in Aerial Images
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00418 · Cited by 3,414
Self-citationObject Detection in Aerial Images: A Large-Scale Benchmark and Challenges
Ding, J., Xue, N., Xia, G.-S., Bai, X., Yang, W., Yang, M. Y., Belongie, S., Luo, J., Datcu, M., Pelillo, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3117983 · Cited by 678
Self-citationArtEmis: Affective Language for Visual Art
Achlioptas, P., Ovsjanikov, M., Haydarov, K., Elhoseiny, M., & Guibas, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01140 · Cited by 161
RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
Wang, Y., Shen, S., & Lim, B. Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581402 · Cited by 120
Multimodality in tourism and hospitality: a critical and narrative review
Yu, X., & Cheng, M. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105245 · Cited by 24
Metadata
- Authors
- You, Q., Luo, J., Jin, H., & Yang, J.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2016-02-21
- Volume
- 30
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2347880541
- Cited by
- 348
- References
- 35
- FWCI
- 32.90
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.