Image Aesthetic Assessment: An experimental survey
Open accessThe DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
This article reviews recent computer vision techniques used in the assessment of image aesthetic quality. Image aesthetic assessment aims at computationally distinguishing high-quality from low-quality photos based on photographic rules, typically in the form of binary classification or quality scoring. A variety of approaches has been proposed in the literature to try to solve this challenging problem. In this article, we summarize these approaches based on visual feature types (hand-crafted features and deep features) and evaluation criteria (data set characteristics and evaluation metrics). The main contributions and novelties of the reviewed approaches are highlighted and discussed. In addition, following the emergence of deep-learning techniques, we systematically evaluate recent deep-learning settings that are useful for developing a robust deep model for aesthetic scoring.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In theory, however, LMX can be generically applied to most EAs, e.g.multi-objective EAs [23,29], evolutionary strategies [1,5], or in support of open-ended evolution [121], but simply swapping it in as the genetic variation operator.”
From Language Model Crossover: Variation through Few-Shot Prompting (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“It is worth pointing out that other more recent and deeper models such as ResNet [63] or Inception-ResNet [64] could serve as a pre-trained model.”
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“19, we observe that the convolutional response of the vanilla VGG-16 (ImageNet) for object recognition typically finds a precise focus of the salient object in view, while the 1-column CNN baseline (DAN-1-original for aesthetic quality classification) outputs an “aestheticallyoriented” salient region where both the ob…”
InfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 2×
“ence PSNR / SSIM / VIF High-quality Image 7.69 / -0.13 / 0.04 Low-quality Image 8.50 / 0.12 / 0.03 Fig. 2. Quality measurement by Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) [31] and Visual Information Fidelity (VIF) [32] (higher is better, typically measured against a referencing groundtruth…”
BackgroundMethodInfluentialImage Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105 · cited 1×
“e Quality Metrics Image quality metrics are defined in an attempt to quantitatively measure the objective quality of an image. This is typically used in image restoration applications (superresolution [34], de-blur [35] and de-artifacts [36]) where we have a default high-quality reference image for comparison. However,…”
BackgroundSelf-citationHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“, freezing some earlier layers to prevent overfitting [65].”
BackgroundDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336 · cited 1×
“Pretraining using triplets: The triplet loss is inspired by Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) [68] and Large Margin Nearest Neighbor (LMNN) [69].”
MethodVisual categorization with bags of keypoints
Csurka, G. (2004) · European Conference on Computer Vision · Cited by 4,953
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · cited 2×
“It is worth pointing out that other more recent and deeper models such as ResNet [63] or Inception-ResNet [64] could serve as a pre-trained model.”
Research works printed but not linked: 15
- [33] Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel, 'Handwritten digit recognition with a backpropagation network,' in Advacnes in Neural Information Processing Systems (NIPS) , 1989. -
- [32] H. R. Sheikh and A. C. Bovik, 'A visual information fidelity approach to video quality assessment,' in The First International Workshop on Video Processing and Quality Metrics for Consumer Electronics . Citeseer, 2005, pp. 23-25. -
- [59] Y. Kao, R. He, and K. Huang, 'Visual aesthetic quality assessment with multi-task deep learning,' arXiv preprint arXiv:1604.04970 , 2016. - · arXiv
- wang, 2016, Brain-inspired deep networks for image aesthetics assessment
- [3] J. S. Barbara London, John Upton, Photography . Pearson, 2005. -
- [5] G. T. Fechner, Vorschule der aesthetik . Breitkopf & H¨ artel, 1876, vol. 1. -
- [48] L. Marchesotti, F. Perronnin, and F. Meylan, 'Learning beautiful (and ugly) attributes.' in British Machine Vision Conference (BMVC) , vol. 7, 2013, pp. 1-11. -
- [77] C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu, 'Deeplysupervised nets.' in International Conference on Artificial Intelligence and Statistics , 2015. -
- [81] A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'Imagenet classification with deep convolutional neural networks,' in Advances in Neural Information Processing Systems (NIPS) , 2012, pp. 1097-1105. -
- [2] I. J, Design and Form: The Basic Course at the Bauhaus and Later . Wiley, Chichester, 1975. -
- [1] M. Freeman, 'The complete guide to light and lighting,' 2007. -
- [104] M. Nishiyama, T. Okabe, Y. Sato, and I. Sato, 'Sensation-based photo cropping,' in Proceedings of the ACM International Conference on Multimedia , 2009, pp. 669-672. -
- [16] S. Zeki and J. Nash, Inner vision: An exploration of art and the brain . Oxford university press Oxford, 1999, vol. 415. -
- [84] L. v. d. Maaten and G. Hinton, 'Visualizing data using t-SNE,' Journal of Machine Learning Research , vol. 9, no. Nov, pp. 25792605, 2008. -
- [18] T. Ang, Digital photographer's handbook . Dorling Kindersley Publishing, Incorporated, 2002. -
Cited by
Perceptual image quality assessment: a survey
Zhai, G., & Min, X. (2020) · Science China Information Sciences · doi:10.1007/s11432-019-2757-1 · Cited by 655
A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping
Wang, W., Shen, J., & Ling, H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840724 · Cited by 348
“Many methods have been proposed for this topic, as surveyed in [43].”
MethodDistort-and-Recover: Color Enhancement Using Deep Reinforcement Learning
Park, J., Lee, J.-Y., Yoo, D., & Kweon, I. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00621 · Cited by 211
“There has been a long line of works to assess the aesthetic scores of images as summarized in [2], but there is still no winning method that gives satisfying results.”
BackgroundPersonality-Assisted Multi-Task Learning for Generic and Personalized Image Aesthetics Assessment
Li, L., Zhu, H., Zhao, S., Ding, G., & Lin, W. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2968285 · Cited by 163
“The ACC is the most popular criteria for evaluating the performance of classification [1], and is computed by”
BackgroundA2-RL: Aesthetics Aware Reinforcement Learning for Image Cropping
Li, D., Wu, H., Zhang, J., & Huang, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00855 · Cited by 138
“Recently, thanks to the fast development of deep learning and newly proposed large scale datasets [22], there are many new works [16, 20, 8] which accomplish aesthetic quality assessment with convolutional neural networks.”
BackgroundAesthetic-Driven Image Enhancement by Adversarial Learning
Deng, Y., Loy, C. C., & Tang, X. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240531 · Cited by 149
“1000-class classification in the original ResNet-101 is replaced by a 2-neuron fully-connected layer. We pre-train discriminator D for binary aesthetic classification with the cross-entropy loss as in [5]. After pre-training, the discriminator network D is appended with another 1-neuron fully-connected layer to perform…”
BackgroundMethodInfluentialSelf-citationAttention-based Multi-Patch Aggregation for Image Aesthetic Assessment
Sheng, K., Dong, W., Ma, C., Mei, X., Huang, F., & Hu, B.-G. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240554 · Cited by 142
“Although some early methods only consider global factors [5, 13, 22], most recent approaches propose combining holistic scene information and fine-grained details in a multi-patch (MP) aggregation network [6, 12, 23, 34].”
BackgroundDeep Cropping via Attention Box Prediction and Aesthetics Assessment
Wang, W., & Shen, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.240 · Cited by 131
“Many methods have been proposed for this topic, we refer the reader to [5] for a more detailed survey.”
Hierarchical Layout-Aware Graph Convolutional Network for Unified Aesthetics Assessment
She, D., Lai, Y., Yi, G., & Xu, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00837 · Cited by 130
“Automatic image aesthetics assessment (IAA) has at-tracted increasing attention in recent years due to its potential applications, e.g ., image retrieval, album photo recommendation, image enhancement [7,14,44], etc .”
BackgroundMethodA Unified Probabilistic Formulation of Image Aesthetic Assessment
Zeng, H., Cao, Z., Zhang, L., & Bovik, A. C. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2941778 · Cited by 121
“In the following, we discuss the representative approaches to each of the IAA tasks, and refer interested readers to [9], [19] for more comprehensive reviews of the existing literature.”
BackgroundMethodGood View Hunting: Learning Photo Composition from Dense View Pairs
Wei, Z., Zhang, J., Shen, X., Lin, Z., Měch, R., Hoai, M., & Samaras, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00570 · Cited by 123
“Recently, with the emergence of deep learning, many works attempt to train photo composition models directly from data [42, 18, 10, 31, 16].”
BackgroundA Deep Learning Perspective on Beauty, Sentiment, and Remembrance of Art
Cetinić, E., Lipić, T., & Grgić, S. (2019) · IEEE Access · doi:10.1109/access.2019.2921101 · Cited by 112
“For a more detailed overview of studies in computational aesthetics, we refer the reader to [36], [37].”
BackgroundDistribution-Oriented Aesthetics Assessment With Semantic-Aware Hybrid Network
Cui, C., Liu, H., Lian, T., Nie, L., Zhu, L., & Yin, Y. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2875357 · Cited by 93
“It is typically cast as a classification or regression problem, in which the fundamental challenge is how to extract appropriate image features [4].”
BackgroundA multimodal convolutional neuro-fuzzy network for emotion understanding of movie clips
Nguyen, T.-L., Kavuri, S., & Lee, M. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.06.010 · Cited by 84
Personalized Image Aesthetics Assessment via Meta-Learning With Bilevel Gradient Optimization
Zhu, H., Li, L., Wu, J., Zhao, S., Ding, G., & Shi, G. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2984670 · Cited by 109
“Most studies of IAA hold that there are generic rules for human visual aesthetics [1].”
BackgroundTheme-Aware Visual Attribute Reasoning for Image Aesthetics Assessment
Li, L., Huang, Y., Wu, J., Yang, Y., Li, Y., Guo, Y., & Shi, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3249185 · Cited by 107
“In the literature, a number of IAA models have been reported [3].”
BackgroundComposition-Aware Image Aesthetics Assessment
Liu, D., Puri, R., Kamath, N., & Bhattacharya, S. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093412 · Cited by 95
“A literature survey on image aesthetics assessment can be found in [8], and we will describe the most relevant works here.”
BackgroundLanguage Model Crossover: Variation through Few-Shot Prompting
Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., & Lehman, J. (2024) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3694791 · Cited by 75
“One contribution of this paper is to demonstrate how evolutionary methods can easily leverage this growing eco-system to evolve high-quality artifacts in diverse applications.”
Language model crossover: Variation through few-shot prompting
Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., & Lehman, J. (2023) · arXiv · doi:10.48550/arxiv.2302.12170 · Cited by 75
Reliable and Efficient Image Cropping: A Grid Anchor Based Approach
Zeng, H., Li, L., Cao, Z., & Zhang, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00610 · Cited by 73
“Though a number of image cropping methods have been developed [34, 11, 5, 6, 10, 15, 22, 36], many of them do not release the source code or executable program.”
BackgroundMethodInfluentialTowards Artistic Image Aesthetics Assessment: a Large-scale Dataset and a New Method
Yi, R., Tian, H., Gu, Z., Lai, Y., & Rosin, P. L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02144 · Cited by 135
“With the ever-growing scale of online visual data, image aesthetic assessment (IAA) shows great potential in a variety of applications such as photo recommendation, image ranking and image search [6].”
Composition and Style Attributes Guided Image Aesthetic Assessment
Celona, L., Leonardi, M., Napoletano, P., & Rozza, A. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3191853 · Cited by 78
“However, methods based on hand-crafted features can only achieve limited success [9]: (i) hand-crafted features can not exhaustively model the variations of photographic rules between different categories of images; (ii) hand-crafted features are heuristics, and so it is challenging to mathematically model some photog…”
MethodPredicting Aesthetic Score Distribution Through Cumulative Jensen-Shannon Divergence
Jin, X., Wu, L., Li, X., Chen, S., Peng, S., Chi, J., Ge, S., Song, C., & Zhao, G. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11286 · Cited by 68
“Thus, the loss functions of [10] and [47] are the same.”
BackgroundGrid Anchor Based Image Cropping: A New Benchmark and An Efficient Model
Zeng, H., Li, L., Cao, Z., & Zhang, L. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3024207 · Cited by 60
“Regarding the model complexity, most recent cropping models are based on the AlexNet [10], [21] or VGG16 architecture [11], [20], [24], which are too heavy to be deployed on computational resource limited devices such as mobile phones and drones.”
Influential2D and 3D Image Quality Assessment: A Survey of Metrics and Challenges
Niu, Y., Zhong, Y., Guo, W., Shi, Y., & Chen, P. (2018) · IEEE Access · doi:10.1109/access.2018.2885818 · Cited by 65
“(This figure comes from [130]).”
Leveraging Expert Feature Knowledge for Predicting Image Aesthetics
Kucer, M., Loui, A. C. P., & Messinger, D. W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2845100 · Cited by 55
Aesthetic Attributes Assessment of Images
Jin, X., Wu, L., Zhao, G., Li, X., Zhang, X., Ge, S., Zou, D., Zhou, B., & Zhou, X. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350970 · Cited by 72
“ach aesthetic attribute of an image. 2 RELATED WORK Before deep learning era, many hand-crafted features [7, 16] are designed for aesthetic image classification and scoring as surveyed by Deng et al. [9]. Deep learning methods are proposed recently for aesthetic assessment[11, 14, 15, 17, 18, 20, 21, 23, 24, 31]. They…”
Zhang, J., Miao, Y., & Yu, J. (2021) · IEEE Access · doi:10.1109/access.2021.3083075 · Cited by 72
“Deng et al. [33] systematically reviewed approaches based on visual feature types, dataset characteristics, evaluation metrics, and also conducted experiments to compare the predictive performances of various deep learning settings.”
BackgroundCamera eats first: exploring food aesthetics portrayed on social media using deep learning
Gambetti, A., & Han, Q. (2022) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-09-2021-1206 · Cited by 64
Image Aesthetic Assessment Assisted by Attributes through Adversarial Learning
Pan, B., Wang, S., & Jiang, Q. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301679 · Cited by 60
“A comprehensive survey on image aesthetic assessment can be found in (Joshi et al. 2011; Deng, Loy, and Tang 2017).”
BackgroundFast A3RL: Aesthetics-Aware Adversarial Reinforcement Learning for Image Cropping
Li, D., Wu, H., Zhang, J., & Huang, K. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2914360 · Cited by 58
When Images Backfire: The Effect of Customer-Generated Images on Product Rating Dynamics
Guan, Y., Tan, Y., Wei, Q., & Chen, G. (2023) · Information Systems Research · doi:10.1287/isre.2023.1201 · Cited by 57
Assessing the Visual Esthetics of User Interfaces: A Ten-Year Systematic Mapping
Lima, A. L. D. S., & von Wangenheim, C. G. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1926118 · Cited by 60
Lee, J.-T., & Kim, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00128 · Cited by 64
“This personalization is challenging, as noted in [5].”
BackgroundPhotographic composition classification and dominant geometric element detection for outdoor scenes
Lee, J.-T., Kim, H., Lee, C., & Kim, C. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.05.018 · Cited by 58
EVA: An Explainable Visual Aesthetics Dataset
Chen, K., Valenzise, G., & Dufaux, F. (2020) · doi:10.1145/3423268.3423590 · Cited by 63
“Therefore, we selected images from AVA with associated scores within the range [4,9].”
BackgroundMethodInfluentialProgress and Opportunities in Modelling Just-Noticeable Difference (JND) for Multimedia
Lin, W., & Ghinea, G. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3106503 · Cited by 64
Learning Personalized Image Aesthetics From Subjective and Objective Attributes
Zhu, H., Zhou, Y. Z., Li, L., Li, Y., & Guo, Y. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3123468 · Cited by 41
“The range of aesthetic ratings is [1, 5], and a higher value corresponds to higher image aesthetics.”
BackgroundInfluentialRepresentation learning of image composition for aesthetic prediction
Zhao, L., Shang, M., Gao, F., Li, R., Huang, F., & Yu, J. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2020.103024 · Cited by 40
Making Images Real Again: A Comprehensive Survey on Deep Image CompositionPreprint
Niu, L., Cong, W., Liu, L., Hong, Y., Zhang, B., Liang, J., & Zhang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14490 · Cited by 127
“Although abundant deep learning methods [75, 134, 21, 34, 101, 92, 126] have achieved significant progress on image aesthetic evaluation, only a few of them [101, 92, 126] can be viewed as composition-aware methods which explicitly”
Neural aesthetic image reviewer
Wang, W., Yang, S., Zhang, W., & Zhang, J. (2019) · IET Computer Vision · doi:10.1049/iet-cvi.2019.0361 · Cited by 54
“can not only perform prediction on image aesthetics, but also produce the comments “Fantastic colors and great sharpness” for the topleft image. In contrast to the image aesthetics classification task [3, 14, 15], we extend it by adding a language generator, which can produce aesthetic reviews or insights, as being hel…”
BackgroundAesCLIP: Multi-Attribute Contrastive Learning for Image Aesthetics Assessment
Sheng, X., Li, L., Chen, P., Wu, J., Dong, W., Yang, Y., Xu, L., Li, Y., & Shi, G. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611969 · Cited by 63
“IAA has extensive applications in a variety of fields, e.g. , album curation, image retrieval, image editing [5, 13, 45], etc .”
BackgroundEAT: An Enhancer for Aesthetics-Oriented Transformers
He, S., Ming, A., Zheng, S., Zhong, H., & Ma, H. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611881 · Cited by 59
“As digital photography expands rapidly, image aesthetics assessment (IAA) has become one of the most important criteria to automatically assess whether the image meets users’ aesthetic preferences [7, 33].”
BackgroundAesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception
Huang, Y., Sheng, X., Yang, Z., Yuan, Q., Duan, Z., Chen, P., Li, L., Lin, W., & Shi, G. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680649 · Cited by 109
“Referring to common settings in the field of image quality assessment [9, 26], we recommend that experimental participants establish a uniform measure of five aesthetic grades and guarantee language diversity.”
Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid ApproachPreprint
Burnap, A., Hauser, J. R., & Timoshenko, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3421771 · Cited by 33
“These contributions are a key difference of our work when contrasted with recent deep learning approaches; e.g., (Kang et al. 2017; Pan et al. 2017; Vasileva et al. 2018, Sbai et al. 2019) and the review by (Deng, Loy, and Tang 2017).”
ResultPersonalized image quality assessment with Social-Sensed aesthetic preference
Cui, C., Yang, W., Shi, C., Wang, M., Nie, X., & Yin, Y. (2019) · Information Sciences · doi:10.1016/j.ins.2019.10.011 · Cited by 36
Visualization Assessment: A Machine Learning Approach
Fu, X., Wang, Y., Dong, H., Cui, W., & Zhang, H. (2019) · Visualization · doi:10.1109/visual.2019.8933570 · Cited by 30
“In the last decade, a number of researchers in the field of computer vision and multimedia have tried to assess such images from various perspectives, such as aesthetics [9, 31], memorability [7, 17, 20], and interestingness [12].”
Coarse-to-Fine Image Aesthetics Assessment With Dynamic Attribute Selection
Huang, Y., Li, L., Chen, P., Wu, J., Yang, Y., Li, Y., & Shi, G. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3389452 · Cited by 33
“W ITH the explosive growth of high quality images in the Internet and social networks, image aesthetics assessment (IAA) has become an interesting research topic and is now attracting increasing attention in both academia and industry [1], [2], [3].”
BackgroundLearning with privileged information for photo aesthetic assessment
Shu, Y., Li, Q., Liu, S., & Xu, G. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.04.142 · Cited by 30
“A comprehensive survey related to recent computer vision techniques used in the assessment of photo aesthetic quality can be found in [4, 2].”
BackgroundComposing Semantic Collage for Image Retargeting
Liu, S., Wei, Z., Sun, Y., Ou, X., Lin, J., Liu, B., & Yang, M. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2836313 · Cited by 31
“Other retargeting methods [18]–[20] crop images to improve visual quality of photographs [21], [22].”
BackgroundProduct Design Award Prediction Modeling: Design Visual Aesthetic Quality Assessment via DCNNs
Wu, J., Xing, B., Si, H., Dou, J., Wang, J., Zhu, Y., & Liu, X. (2020) · IEEE Access · doi:10.1109/access.2020.3039715 · Cited by 27
“Apart from the aesthetic perception studies by psychological research method, the image aesthetic can be studied by subjective visual attributes analysis [40]–[44].”
Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color
Thong, W. E., Joniak, P., & Xiang, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00452 · Cited by 49
“Such cropping algorithms are notably useful to select which region of a large image to display on a smaller screen [19,70,76].”
BackgroundMeasuring photography aesthetics with deep CNNs
Reddy, G. V. S., Mukherjee, S., & Thakur, M. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2019.1300 · Cited by 26
“Photo aesthetic quality assessment using computational techniques has been a subject of interest to researchers during the last few decades [24].”
BackgroundQuantifying Visual Image Quality: A Bayesian View
Duanmu, Z., Liu, W., Wang, Z., & Wang, Z. (2021) · Annual Review of Vision Science · doi:10.1146/annurev-vision-100419-120301 · Cited by 38
“The application scope includes, but is not limited to, computer graphics [47], video compression [127], video streaming [21], camera process [23], printing [39], visual displays [75], stereo vision [43], reduced-reference quality assessment [109], degraded-reference quality assessment [2], multi-exposure fusion [53],…”
BackgroundPersonality Driven Multi-task Learning for Image Aesthetic Assessment
Li, L., Zhu, H., Zhao, S., Ding, G., Jiang, H., & Tan, A. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00081 · Cited by 25
“Image aesthetic assessment (IAA) aims at measuring people’s aesthetic perception of images through photographic rules [1].”
BackgroundMeta-Learning Perspective for Personalized Image Aesthetics Assessment
Wang, N., Su, J., Li, L., Xu, X., & Luo, J. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803119 · Cited by 25
“With the success of deep learning on visual content understanding, solving the task of image aesthetics rating has become increasingly accurate[2][3][4][5] and practical[6][7][8].”
BackgroundHyNet: A novel hybrid deep learning approach for efficient interior design texture retrieval
Chen, J., Shao, Z., Cen, C., & Li, J. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16579-0 · Cited by 35
“Hand-craftedfeaturescanbedividedintoglobalfeaturesandlocalfeatures[29,30].”
Transfer learning features for predicting aesthetics through a novel hybrid machine learning method
Carballal, A., Fernandez-Lozano, C., Heras, J., & Romero, J. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04065-4 · Cited by 23
MSENet: Marbling score estimation network for automated assessment of Korean beef
Lee, H.-J., Koh, Y. J., Kim, Y., Lee, S. H., Lee, J. H., & Seo, D. W. (2022) · Meat Science · doi:10.1016/j.meatsci.2022.108784 · Cited by 23
Context-aware Attention Network for Predicting Image Aesthetic Subjectivity
Xu, M., Zhong, J.-X., Ren, Y., Liu, S., & Li, G. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413834 · Cited by 22
“These works can be mainly divided into two tasks: binary classification task [4, 17, 18, 20, 21] and regression task [16, 22].”
BackgroundMethodAdvances and Challenges in Computational Image Aesthetics
Valenzise, G., Chen, K., & Frederic Dufaux, A. (2021) · Springer eBooks · doi:10.1007/978-3-030-81465-6_6 · Cited by 33
“Traditionally, data-driven methods have been concentrating on predicting point estimates such as the average aesthetic score [19, 47].”
Lightness-aware contrast enhancement for images with different illumination conditions
Hao, S., Guo, Y., & Wei, Z. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6257-1 · Cited by 21
“In the future research, we plan to introduce bio-inspired quality assessment models [9, 39] or image aesthetic evaluation [4] into our model.”
MethodA Novel Feature Fusion Method for Computing Image Aesthetic Quality
Li, X., Li, X., Zhang, G., & Zhang, X. (2020) · IEEE Access · doi:10.1109/access.2020.2983725 · Cited by 22
“As seen from a recent review by Deng et al. [8], many ideas for CNN model design are derived from AlexNet [13], which achieved the most advanced results for the first time in the ImageNet competition.”
BackgroundMethodConsolidated Dataset and Metrics for High-Dynamic-Range Image Quality
Mikhailiuk, A., Pérez‐Ortiz, M., Yue, D., Suen, W., & Mantiuk, R. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3076298 · Cited by 29
“Here, the quality may be perceived in terms of creative composition and execution of an image, rather than artifacts [32].”
BackgroundGrand Challenges in Image Processing
Dufaux, F. (2021) · Frontiers in Signal Processing · doi:10.3389/frsip.2021.675547 · Cited by 28
“Another closely related topic is image esthetic assessment (Deng et al., 2017).”
BackgroundMulti-Task Convolutional Neural Network for Image Aesthetic Assessment
Soydaner, D., & Wagemans, J. P. (2024) · IEEE Access · doi:10.1109/access.2024.3349961 · Cited by 24
“Because of the importance and complexity of the problem, the literature on automated image aesthetic assessment is extensive [4], [40].”
BackgroundDeep Learning Based Image Aesthetic Quality Assessment- A Review
Chounchenani, M. D., Shahbahrami, A., Hassanpour, R., & Gaydadjiev, G. N. (2025) · ACM Computing Surveys · doi:10.1145/3716820 · Cited by 27
“While numerous studies have explored Image Aesthetic Quality Assessment (IAQA) in various aspects [2, 23, 139, 163, 172], the primary focus has often been on traditional methods, while recent research and techniques have often been overlooked.”
Fabbro, D. D., Catissi, G., Borba, G. B., Lima, L. M., Hingst‐Zaher, E., Rosa, J. O., Victor, E. D. S., Bernardes, L. V. M. M. C. F. M. R. J. L. F., Souza, T., & Leão, E. R. (2021) · Scientific Reports · doi:10.1038/s41598-021-91013-9 · Cited by 25
“The photographs were grouped into nine categories: Pale Bird (50 images), Colorful Bird (50), Sky (50), Flower (50), Sea (50), Insects (50), Water (50), Landscape (27), and Forest/woodland (26).”
BackgroundAnchor-based knowledge embedding for image aesthetics assessment
Li, L., Zhi, T., Shi, G., Yang, Y., Xu, L., Li, Y., & Guo, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.03.058 · Cited by 20
Toward Transparent Deep Image Aesthetics Assessment With Tag-Based Content Descriptors
Hou, J., Lin, W., Fang, Y., Wu, H., Chen, C., Liao, L., & Liu, W. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3308852 · Cited by 20
“A robust IAA model can be used by social network service providers to rank images in search or recommendation results [1], [2], [3], [4], and by social network users to manage their own albums, select the best photos to share, and even guide them in taking and editing photos [2], [4], [5], [6].”
Deep Neural Network-Based Click-Through Rate Prediction using Multimodal Features of Online Banners
Xia, B., Wang, X., Yamasaki, T., Aizawa, K., & Seshime, H. (2019) · IEEE International Conference on Multimedia Big Data · doi:10.1109/bigmm.2019.00-29 · Cited by 19
“Notably, many studies used multipatch aggregation methods [18, 19, 20, 21].”
BackgroundResultMulti-Level Transitional Contrast Learning for Personalized Image Aesthetics Assessment
Yang, Z., Li, L., Yang, Y., Li, Y., & Lin, W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3290479 · Cited by 29
“[1 , S ] denotes the set of positive integers from 1 to S (Taking FLICKR-AES [9] as an example, it is [1 , 5] ).”
BackgroundImage Cropping with Spatial-aware Feature and Rank Consistency
Wang, C., Niu, L., Zhang, B., & Zhang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00969 · Cited by 37
Hybrid CNN-transformer based meta-learning approach for personalized image aesthetics assessment
Yan, X. H., Shao, F., Chen, H., & Jiang, Q. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.104044 · Cited by 18
Medel-Vera, C., Britton, S., & Gates, W. F. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100501 · Cited by 34
Composition-Guided Neural Network for Image Cropping Aesthetic Assessment
Ni, S., Shao, F., Chai, X., Chen, H., & Ho, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3215003 · Cited by 17
“However, the performance of hand-design features is not very satisfactory [36].”
BackgroundPersonalized Image Aesthetics Assessment with Attribute-guided Fine-grained Feature Representation
Zhu, H., Shao, Z., Zhou, Y. Z., Wang, G., Chen, P., & Li, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611942 · Cited by 17
“In light of this, the exploration of image aesthetics assessment (IAA) for automatic evaluation of users’ visual aesthetics has garnered significant attention from researchers in the fields of image representation and signal processing [2, 4].”
BackgroundBodini, M. (2019) · Inventions · doi:10.3390/inventions4030034 · Cited by 17
“For instance, see [4].”
Image Aesthetics Assessment Using Graph Attention Network
Ghosal, K., & Smolić, A. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956162 · Cited by 25
“4We provide a detailed discussion on this and comparison with the classification-based approaches [12], [9], [54], [55], [11], [10], [22] using accuracy and balanced accuracy in the supplementary material.”
Analysis of Deep Features for Image Aesthetic Assessment
Jang, H., & Lee, J. (2021) · IEEE Access · doi:10.1109/access.2021.3060171 · Cited by 17
“The research of image aesthetic assessment has evolved from hand-crafted feature-based conventional machine learning methods to deep learning-based methods [4].”
MethodTransductive Aesthetic Preference Propagation for Personalized Image Aesthetics Assessment
Li, Y., Yang, Y., Li, H., Chen, H., Xu, L., Li, L., Li, Y., & Guo, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548244 · Cited by 20
“With explosive growth of camera phones and self-media market, the ability of photo management has shown great essentiality in many real-world problems [6].”
BackgroundImage Aesthetics Assessment Using Composite Features from off-the-Shelf Deep Models
Fu, X., Yan, J., & Fan, C. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451133 · Cited by 17
“Consequently, deep learning-based techniques have been adopted in aesthetics assessment tasks over the past few years and have successfully gained better performance than conventional approaches [1, 2, 7].”
BackgroundIAACS: Image aesthetic assessment through color composition and space formation
Yang, B., Zhu, C., Li, F. W. B., Wei, T., Liang, X., & Wang, Q. (2023) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2022.06.006 · Cited by 19
Property-Specific Aesthetic Assessment With Unsupervised Aesthetic Property Discovery
Lee, J.-T., Lee, C., & Kim, C. (2019) · IEEE Access · doi:10.1109/access.2019.2936289 · Cited by 15
“In an extensive survey of aesthetic assessment techniques in [35], it was shown that CNN-based algorithms outperform the others.”
BackgroundInfluentialRegression or classification? New methods to evaluate no-reference picture and video quality models
Tu, Z., Chen, C.-J., Chen, L., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414232 · Cited by 18
Learning image aesthetic subjectivity from attribute-aware relational reasoning network
Zhu, H., Zhou, Y. Z., Yao, R., Wang, G., & Yang, Y. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.02.008 · Cited by 15
Style-adaptive photo aesthetic rating via convolutional neural networks and multi-task learning
Gao, F., Li, Z., Yu, J., Yu, J., Huang, Q., & Tian, Q. C. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.06.099 · Cited by 15
A Comprehensive Survey on Image Aesthetic Quality Assessment
Yang, H., Ping, S., He, S., Pan, D., Ying, Z., & Lei, L. (2019) · IEEE/ACIS International Conference on Computer and Information Science (ICIS) · doi:10.1109/icis46139.2019.8940355 · Cited by 15
“There are some surveys about image aesthetics [2] [4], however, these papers do not relate to different approaches based on different aesthetic tasks, especially aesthetic distribution, aesthetic factors, aesthetic description.”
BackgroundConsolidated Dataset and Metrics for High-Dynamic-Range Image Quality
Mikhailiuk, A., Perez-Ortiz, M., Yue, D., Suen, W., & Mantiuk, R. (2021) · UCL Discovery (University College London) · Cited by 14
Privileged multi-task learning for attribute-aware aesthetic assessment
Shu, Y., Li, Q., Liu, L., & Xu, G. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108921 · Cited by 14
Towards Unified Aesthetics and Emotion Prediction in Images
Yu, J., Cui, C., Geng, L., Ma, Y., & Yin, Y. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803388 · Cited by 18
“Typically, aesthetics assessment [4] and emotion recognition [5] are two fundamental problems in user perception understanding, which aim to predict human aesthetic and emotional reactions evoked by visual stimuli, respectively.”
BackgroundAn Information Theory Approach to Aesthetic Assessment of Visual Patterns
Khalili, A., & Bouchachia, H. (2021) · Entropy · doi:10.3390/e23020153 · Cited by 14
Semi-Supervised Adversarial Learning for Attribute-Aware Photo Aesthetic Assessment
Shu, Y., Li, Q., Liu, L., & Xu, G. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3117709 · Cited by 16
Unveiling the factors of aesthetic preferences with explainable AI
Soydaner, D., & Wagemans, J. P. (2024) · British Journal of Psychology · doi:10.1111/bjop.12707 · Cited by 22
“The interdisciplinary field of computational aesthetics is a blend of these two disciplines, dedicated to developing computational methods for aesthetics research [3–5].”
BackgroundFusion global and local deep representations with neural attention for aesthetic quality assessment
Zhang, X., Gao, X.-B., Lu, W., Yu, Y., & He, L.-H. (2019) · Signal processing. Image communication · doi:10.1016/j.image.2019.05.021 · Cited by 10
Synergetic Assessment of Quality and Aesthetic: Approach and Comprehensive Benchmark Dataset
Zhang, K., Zhu, D., Min, X., Gao, Z., & Zhai, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3303933 · Cited by 18
“The previous studies [1], [2] also provide sufficient evidence that human subjective senses are influenced by comprehensive stimuli in a different way than perceptual quality or aesthetic experience in isolation.”
BackgroundVisual order of Chinese ink paintings
Fan, Z., & Kang Zhang, A. (2020) · Visual Computing for Industry Biomedicine and Art · doi:10.1186/s42492-020-00059-5 · Cited by 15
AutoPhoto: Aesthetic Photo Capture using Reinforcement Learning
Alzayer, H., Lin, H., & Bala, K. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636788 · Cited by 19
“Computer vision models have also been developed to estimate the aesthetics of images [19].”
Personalized Image Aesthetics Assessment via Multi-Attribute Interactive Reasoning
Zhu, H., Zhou, Y. Z., Shao, Z., Du, W.-L., Wang, G., & Li, Q. (2022) · Mathematics · doi:10.3390/math10224181 · Cited by 14
Predicting Subjectivity in Image Aesthetics Assessment
Chen, K., Valenzise, G., & Frederic Dufaux, A. (2019) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2019.8901716 · Cited by 13
“This is due, on one hand, to the potential impact that aesthetic quality prediction has on applications such as image enhancement, recommendation or retrieval [1].”
Background
Metadata
- Authors
- Deng, Y., Loy, C. C., & Tang, X.
- Affiliations
- Chinese University of Hong Kong (HK)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-07
- Volume
- 34
- Issue
- 4
- Pages
- 80-106
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W2529088810
- Cited by
- 306
- References
- 95
- Citations by year
- 2026: 14 · 2025: 29 · 2024: 26 · 2023: 40 · 2022: 39 · 2021: 39 · 2020: 43 · 2019: 43 · 2018: 29 · 2017: 4
- FWCI
- 13.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- image aesthetic assessment, handcrafted features, deep features, deep learning, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.