Learning deep features for image emotion classification
Abstract
Images can both express and affect people's emotions. It is intriguing and important to understand what emotions are conveyed and how they are implied by the visual content of images. Inspired by the recent success of deep convolutional neural networks (CNN) in visual recognition, we explore two simple, yet effective deep learning-based methods for image emotion analysis. The first method uses off-the-shelf CNN features directly for classification. For the second method, we fine-tune a CNN that is pre-trained on a large dataset, i.e. ImageNet, on our target dataset first. Then we extract features using the fine-tuned CNN at different location at multiple levels to capture both the global and local information. The features at different location are aggregated using the Fisher Vector for each level and concatenated to form a compact representation. From our experimental results, both the deep learning-based methods outperforms traditional methods based on generic image descriptors and hand-crafted features.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · see citing works
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588 · see citing works
Multi-scale Orderless Pooling of Deep Convolutional Activation Features
Gong, Y., Wang, L., Guo, R., & Lazebnik, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_26 · Cited by 1,043 · see citing works
Affective image classification using features inspired by psychology and art theory
Machajdik, J., & Hanbury, A. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1873965 · Cited by 873 · see citing works
High level describable attributes for predicting aesthetics and interestingness
Dhar, S., Ordóñez, V., & Berg, T. L. (2011) · doi:10.1109/cvpr.2011.5995467 · Cited by 507 · see citing works
Aesthetics and Emotions in Images
Joshi, D., Datta, R., Fedorovskaya, E. A., Luong, Q.-T., Wang, J. Z., Li, J., & Luo, J. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.941851 · Cited by 407 · see citing works
Assessing the aesthetic quality of photographs using generic image descriptors
Marchesotti, L., Perronnin, F., Larlus, D., & Csurka, G. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126444 · Cited by 406 · see citing works
Exploring Principles-of-Art Features For Image Emotion Recognition
Zhao, S., Gao, Y., Jiang, X., Yao, H., Chua, T., & Sun, X. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654930 · Cited by 376 · see citing works
Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction
Wei-ning, W., Yu, Y., & Jiang, S. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2006.384667 · Cited by 160 · see citing works
Algorithmic inferencing of aesthetics and emotion in natural images: An exposition
Datta, R., Li, J., & Wang, J. Z. (2008) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2008.4711702 · Cited by 151 · see citing works
Can we understand van gogh's mood?
Jia, J., Wu, S., Wang, X., Hu, P., Cai, L., & Tang, J. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2396330 · Cited by 119 · see citing works
Affective Image Retrieval via Multi-Graph Learning
Zhao, S., Yao, H., Yang, Y., & Zhang, Y. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2655035 · Cited by 112 · see citing works
Quantifying aesthetics of visual design applied to automatic design
Jahanian, A. (2016) · Springer theses · doi:10.1007/978-3-319-31486-0 · Cited by 11 · see citing works
Aesthetic quality inference for online fashion shopping
Chen, M., & Allebach, J. P. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2045269 · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
Classification of Hyperspectral Images by Gabor Filtering Based Deep Network
Kang, X., Li, C., Li, S., & Lin, H. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2017.2767185 · Cited by 151 · see citing works
- Authors
- Chen, M., Zhang, L., & Allebach, J. P.
- Affiliations
- Purdue University West Lafayette (US)
- Venue
- 2015 IEEE International Conference on Image Processing (ICIP)
- Publisher
- IEEE
- Published
- 2015-09
- Pages
- 4491-4495
- ISSN
- 1522-4880 (ISSN-L)
- OpenAlex
- W2296756268
- Cited by
- 53
- References
- 16
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 1 · 2023: 7 · 2022: 4 · 2021: 5 · 2020: 7 · 2019: 13 · 2018: 1 · 2017: 5 · 2016: 2
- FWCI
- 3.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Conference on Image Processing (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- deep features, convolutional neural networks, fine-tuning, Fisher vectors, multi-level feature aggregation, global and local features, ImageNet pre-training, visual emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
