Image Aesthetic Assessment: An experimental survey
Open accessThe DOI matches its Crossref record; 100 of the 100 references it cites were found in a registry or an index.
Abstract
This article reviews recent computer vision techniques used in the assessment of image aesthetic quality. Image aesthetic assessment aims at computationally distinguishing high-quality from low-quality photos based on photographic rules, typically in the form of binary classification or quality scoring. A variety of approaches has been proposed in the literature to try to solve this challenging problem. In this article, we summarize these approaches based on visual feature types (hand-crafted features and deep features) and evaluation criteria (data set characteristics and evaluation metrics). The main contributions and novelties of the reviewed approaches are highlighted and discussed. In addition, following the emergence of deep-learning techniques, we systematically evaluate recent deep-learning settings that are useful for developing a robust deep model for aesthetic scoring.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In theory, however, LMX can be generically applied to most EAs, e.g.multi-objective EAs [23,29], evolutionary strategies [1,5], or in support of open-ended evolution [121], but simply swapping it in as the genetic variation operator.”
From Language Model Crossover: Variation through Few-Shot Prompting (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“It is worth pointing out that other more recent and deeper models such as ResNet [63] or Inception-ResNet [64] could serve as a pre-trained model.”
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“19, we observe that the convolutional response of the vanilla VGG-16 (ImageNet) for object recognition typically finds a precise focus of the salient object in view, while the 1-column CNN baseline (DAN-1-original for aesthetic quality classification) outputs an “aestheticallyoriented” salient region where both the ob…”
InfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 2×
“ence PSNR / SSIM / VIF High-quality Image 7.69 / -0.13 / 0.04 Low-quality Image 8.50 / 0.12 / 0.03 Fig. 2. Quality measurement by Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) [31] and Visual Information Fidelity (VIF) [32] (higher is better, typically measured against a referencing groundtruth…”
BackgroundMethodInfluentialImage Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105 · cited 1×
“e Quality Metrics Image quality metrics are defined in an attempt to quantitatively measure the objective quality of an image. This is typically used in image restoration applications (superresolution [34], de-blur [35] and de-artifacts [36]) where we have a default high-quality reference image for comparison. However,…”
BackgroundSelf-citationHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“, freezing some earlier layers to prevent overfitting [65].”
BackgroundDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336 · cited 1×
“Pretraining using triplets: The triplet loss is inspired by Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) [68] and Large Margin Nearest Neighbor (LMNN) [69].”
MethodInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · cited 2×
“It is worth pointing out that other more recent and deeper models such as ResNet [63] or Inception-ResNet [64] could serve as a pre-trained model.”
Visual categorization with bags of keypoints
Csurka, G. (2004) · Cited by 4,379
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., & Saul, L. K. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1577078 · Cited by 4,044 · cited 2×
“We discuss two popular approaches below. pre-training using triplets: The triplet loss is inspired by Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) [91] and Large Margin Nearest Neighbor (LMNN) [92].”
MethodCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752 · cited 1×
“All baselines are implemented based on the Caffe package [59].”
Method(1989) · neural information processing systems · Cited by 3,736
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 1×
“DAN-1-regression with cropping data - Alternatively, to include additional image cropping data {x i , Y crop i }i∈[1,N ′], where Y crop i = [x, y, width, height], we follow insights in [90] and add a window regression layer to learn a mapping from the convolutional response (see Fig 17c).”
MethodA model of aesthetic appreciation and aesthetic judgments
Leder, H., Belke, B., Oeberst, A., & Augustin, D. (2004) · British Journal of Psychology · doi:10.1348/0007126042369811 · Cited by 1,968 · cited 2×
“For example, human’s general reward circuitry produces pleasure when people look at beautiful objects [11], and the subsequent aesthetic judgment consists of the appraisal of the valence of such perceived objects [8], [9], [10], [12].”
BackgroundTraining linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920 · cited 20×
“They use SVM [52] to train a ranking model for image pairs of {IhighQuality, IlowQuality}.”
MethodInfluentialThe Balanced Accuracy and Its Posterior Distribution
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.764 · Cited by 1,816 · cited 2×
“uating testing partition with imbalanced distribution [54], [55].”
BackgroundFisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641 · cited 1×
“The more general image features like Bagof-Visual Words (BOV) [44] and Fisher Vector (FV) [45] are explored in [20], [21], [22].”
BackgroundBarrett, L. F., Mesquita, B., Ochsner, K. N., & Gross, J. J. (2007) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085709 · Cited by 1,425 · cited 2×
“…neuroaesthetics, researchers argue that there is a certain connection between human aesthetic experience and the sensation caused by visual stimuli regardless of source, culture, and experience [6], which is supported by activations in specific regions of the visual cortex [7], [8], [9], [10].”
BackgroundHigh-quality motion deblurring from a single image
Shan, Q., Jia, J., & Agarwala, A. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360672 · Cited by 1,386
Person Re-identification by Multi-Channel Parts-Based CNN with Improved Triplet Loss Function
Cheng, D., Gong, Y., Zhou, S., Wang, J., & Zheng, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.149 · Cited by 1,247 · cited 1×
“It is widely used in many recent vision studies [55], [70], [71], [72], aiming to bring data of the same class closer, while data of different classes further away.”
BackgroundStudying Aesthetics in Photographic Images Using a Computational Approach
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744078_23 · Cited by 1,107 · cited 5×
“[8] combine low-level features and high-level features that are typically used for image retrieval and train an SVM classifier for binary classification of images aesthetic quality.”
BackgroundMethodInfluentialLee, C., Xie, S., Gallagher, P., Zhang, Z., & Zhuowen Tu, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 993
Learning Deep Representation for Imbalanced Classification
Huang, C., Li, Y., Loy, C. C., & Tang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.580 · Cited by 988 · cited 2×
“uating testing partition with imbalanced distribution [54], [55].”
BackgroundSelf-citationAVA: A large-scale database for aesthetic visual analysis
Murray, N., Marchesotti, L., & Perronnin, F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247954 · Cited by 936
Compression Artifacts Reduction by a Deep Convolutional Network
Dong, C., Deng, Y., Loy, C. C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.73 · Cited by 883 · cited 1×
“Fine-tuning from trained CNN has been proved in [61], [62] to be an effective initialization approach.”
MethodSelf-citationEfficient mini-batch training for stochastic optimization
Li, M., Zhang, T., Chen, Y., & Smola, A. J. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623612 · Cited by 742 · cited 1×
“Random sampling: By randomly selecting examples for mini-batches [66], [67], we in fact select from a distribution of the training partition.”
MethodThe Design of High-Level Features for Photo Quality Assessment
Ke, Y., Tang, X., & Jing, F. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.303 · Cited by 630 · cited 4×
“[9] are among the first to cast aesthetic understanding of images into a binary classification problem.”
BackgroundMethodInfluentialSelf-citationHigh level describable attributes for predicting aesthetics and interestingness
Dhar, S., Ordonez, V., & Berg, T. L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995467 · Cited by 500 · cited 4×
“(2011) [17] 16000-image private set from DPChallenge PR-curve Nishiyama et al.”
BackgroundMethodInfluentialPhoto Aesthetics Ranking Network with Attributes and Content Adaptation
Kong, S., Shen, X., Lin, Z., Mech, R., & Fowlkes, C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_40 · Cited by 468 · cited 8×
“96 - The authors in [22], [25], [26], [28], [29], [30], [33], [34], [35] have not released detailed results.”
BackgroundMethodInfluentialPhoto and Video Quality Evaluation: Focusing on the Subject
Luo, Y., & Tang, X. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88690-7_29 · Cited by 444 · cited 5×
“Regional image features [13], [14], [15] later prove to be effective in complementing the global features.”
BackgroundMethodInfluentialSelf-citationLearning Deep Representation for Face Alignment with Auxiliary Attributes
Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2469286 · Cited by 424 · cited 1×
“Multi-task learning with image category prediction: Can aesthetic prediction be facilitated provided that a model understands which category the image belongs to? Following the work in [73] where auxiliary information is used to regularized the learning of the main task, we investigate TABLE 7 Triplets pre-training an…”
BackgroundSelf-citationThe general inefficiency of batch training for gradient descent learning
Wilson, D. R., & Martinez, T. R. (2003) · Neural Networks · doi:10.1016/s0893-6080(03)00138-2 · Cited by 418 · cited 1×
“Random sampling: By randomly selecting examples for mini-batches [66], [67], we in fact select from a distribution of the training partition.”
MethodAesthetics and Emotions in Images
Joshi, D., Datta, R., Fedorovskaya, E., Luong, Q.-T., Wang, J., Li, J., & Luo, J. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.941851 · Cited by 408
Assessing the aesthetic quality of photographs using generic image descriptors
Marchesotti, L., Perronnin, F., Larlus, D., & Csurka, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126444 · Cited by 405 · cited 5×
“Conventional approaches [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22] typically adopt handcrafted features to computationally model the photographic rules (lighting, contrast), global image layout (rule-of-thirds) and typical objects (human profiles, animals, plants) in images.”
BackgroundMethodInfluentialRAPID: Rating Pictorial Aesthetics using Deep Learning
Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654927 · Cited by 383 · cited 15×
“As shown in Table 5, fine-tuning from the vanilla AlexNet yields better performance than simply training the base net of RAPID from scratch.”
BackgroundMethodInfluentialNaturalizing aesthetics: Brain areas for aesthetic appraisal across sensory modalities
Brown, S., Gao, X., Tisdelle, L., Eickhoff, S. B., & Liotti, M. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.012 · Cited by 376 · cited 1×
“human aesthetic experience and the sensation caused by visual stimuli regardless of source, culture, and experience [6], which is supported by activations in specific regions of the visual cortex [7], [8], [9], [10]. For example, human’s general reward circuitry produces pleasure when people look at beautiful objects […”
BackgroundThe Structure of Emotion: Evidence From Neuroimaging Studies
Barrett, L. F., & Wager, T. D. (2006) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2006.00411.x · Cited by 366 · cited 1×
“hetic experience and the sensation caused by visual stimuli regardless of source, culture, and experience [6], which is supported by activations in specific regions of the visual cortex [7], [8], [9], [10]. For example, human’s general reward circuitry produces pleasure when people look at beautiful objects [11], and t…”
BackgroundChatterjee, A., & Vartanian, O. (2016) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13035 · Cited by 351
Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation
Lu, X., Lin, Z., Shen, X., Mech, R., & Wang, J. Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.119 · Cited by 321 · cited 10×
“State-of-the-art approaches [28], [29], [30], [31] for image aesthetic classification typically adopt multi-column CNNs (Fig.”
BackgroundMethodInfluentialProspects for a cognitive neuroscience of visual aesthetics: (514602010-003)
Chatterjee, A. (2003) · PsycEXTRA Dataset · doi:10.1037/e514602010-003 · Cited by 309 · cited 2×
“vations in the visual cortex can be attributed to the processing of various early, intermediate and late visual features of the stimuli including orientation, shape, color grouping and categorization [13], [14], [15], [16]. Artists intentionally incorporate such features to facilitate desired perceptual and emotional…”
BackgroundComposition-Preserving Deep Photo Aesthetics Assessment
Mai, L., Jin, H., & Liu, F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.60 · Cited by 277 · cited 4×
“These rule-based approaches are intuitive as they try to explicitly model the criteria that humans use in evaluating the aesthetic quality of an image. However, more recent studies [23], [24], [25], [26] have shown that using a data-driven approach is more effective, as the amount of training data available grows from…”
BackgroundMethodResultInfluentialA framework for photo-quality assessment and enhancement based on visual aesthetics
Bhattacharya, S., Sukthankar, R., & Shah, M. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873990 · Cited by 268 · cited 4×
“[16], [40] propose compositional features using relative foreground position and visual weight ratio to model the relations of foreground objects and the background scene, then an support vector regressor is trained.”
BackgroundMethodInfluentialContent-based photo quality assessment
Luo, W., Wang, X., & Tang, X. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126498 · Cited by 258
Self-citationCavanagh, P. (2005) · Nature · doi:10.1038/434301a · Cited by 257 · cited 1×
“Artists intentionally incorporate such features to facilitate desired perceptual and emotional effects in viewers, forming a set of guidelines as they create artworks to induce desired responses in the nervous systems of perceivers [16], [17].”
BackgroundLiu, L., Chen, R., Wolf, L., & Cohen‐Or, D. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01616.x · Cited by 255
Content-Based Photo Quality Assessment
Tang, X., Luo, W., & Wang, X. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2269899 · Cited by 253 · cited 5×
“twined aesthetic rules. The majority of feature types can be classified into handcrafted features and deep features. Conventional approaches [20], [21], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49] typically adopt handcrafted features to computationally model the photographic rules (ligh…”
BackgroundMethodResultInfluentialSelf-citationPolyNet: A Pursuit of Structural Diversity in Very Deep Networks
Zhang, X., Li, Z., Loy, C. C., & Lin, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.415 · Cited by 249 · cited 1×
“ields the best performance in both the balanced accuracy and overall accuracy. It is worth pointing out that other more recent and deeper models such as ResNet [86], Inception-ResNet [87], or PolyNet [88], could serve as a pre-trained model. Nevertheless, owing to the typically small size of aesthetic datasets, precau…”
MethodSelf-citationRating Image Aesthetics Using Deep Learning
Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477040 · Cited by 239 · cited 8×
“51% standard partition RAPID - improved version (2015) [30] AVA Overall accuracy 75.”
BackgroundMethodInfluentialThe neural correlates of subjective pleasantness
Kühn, S., & Gallinat, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.065 · Cited by 233 · cited 1×
“For example, human’s general reward circuitry produces pleasure when people look at beautiful objects [11], and the subsequent aesthetic judgment consists of the appraisal of the valence of such perceived objects [8], [9], [10], [12].”
BackgroundSubjective and Objective Quality Assessment of Image: A SurveyPreprint
Mohammadi, P., Ebrahimi-Moghadam, A., & Shirani, S. (2014) · arXiv.org · doi:10.48550/arxiv.1406.7799 · Cited by 209 · cited 1×
“Some good surveys on this topic can be found in [6], [7].”
BackgroundAesthetic quality classification of photographs based on color harmony
Nishiyama, M., Okabe, T., Sato, I., & Sato, Y. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995539 · Cited by 208 · cited 7×
“(2011) [14] 12k-image private set from DPChallenge Overall accuracy 77.”
BackgroundMethodInfluentialProbabilistic Graphlet Transfer for Photo Cropping
Zhang, L., Song, M., Zhao, Q., Liu, X., Bu, J., & Chen, C. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2223226 · Cited by 202 · cited 1×
“Aesthetics-based approaches [81], [82], [83] assess the attractiveness of some proposed candidate crop windows with low-level image features and rules of photographic composition.”
BackgroundThe brain's specialized systems for aesthetic and perceptual judgment
Ishizu, T., & Zeki, S. (2013) · European Journal of Neuroscience · doi:10.1111/ejn.12135 · Cited by 182 · cited 1×
“ween human aesthetic experience and the sensation caused by visual stimuli regardless of source, culture, and experience [6], which is supported by activations in specific regions of the visual cortex [7], [8], [9], [10]. For example, human’s general reward circuitry produces pleasure when people look at beautiful obje…”
BackgroundFusion of Multichannel Local and Global Structural Cues for Photo Aesthetics Evaluation
Zhang, L., Gao, Y., Zimmermann, R., Tian, Q., & Li, X. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2303650 · Cited by 168 · cited 1×
“[43] where image descriptors that characterize local and global structural aesthetics from multiple visual channels are designed.”
BackgroundHybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation
Xie, G.-S., Zhang, X.-Y., Yan, S., & Liu, C.-L. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2511543 · Cited by 168 · cited 1×
“Features extracted from the convolutional layers of the model were aggregated into a convolutional Fisher representation, as done in [98].”
MethodCheng, B., Ni, B., Yan, S., & Tian, Q. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1873992 · Cited by 154 · cited 1×
“Aesthetics-based approaches [81], [82], [83] assess the attractiveness of some proposed candidate crop windows with low-level image features and rules of photographic composition.”
BackgroundAlgorithmic inferencing of aesthetics and emotion in natural images: An exposition
Datta, R., Li, J., & Wang, J. Z. (2008) · 2008 15th IEEE International Conference on Image Processing · doi:10.1109/icip.2008.4711702 · Cited by 150 · cited 1×
“Net dataset and The DPChallenge dataset are introduced in [5], [36].”
MethodAutomated Aesthetic Analysis of Photographic Images
Aydın, T. O., Smolić, A., & Groß, M. H. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2325047 · Cited by 139
Classification of Digital Photos Taken by Photographers or Home Users
Tong, H., Li, M., Zhang, H.-J., He, J., & Zhang, C. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30541-5_25 · Cited by 138 · cited 4×
“[10] adopts boosting to combine global low-level simple features (blurriness, contrast, colorfulness and saliency) in order to classify professional photograph and ordinary snapshots.”
BackgroundMethodInfluentialLearning the Change for Automatic Image Cropping
Yan, J., Lin, S. C.-F., Kang, S. B., & Tang, X. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.130 · Cited by 135 · cited 2×
“Following the same evaluation settings in [84], [85], we use average overlap ratio and average boundary displacement error to quantify the performance of automatic image cropping.”
BackgroundMethodSelf-citationSaliency-enhanced image aesthetics class prediction
Wong, L.-K., & Low, K. (2009) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2009.5413825 · Cited by 131 · cited 4×
“While [13], [14], [39] adopt the SVM classifier, Lo et al.”
BackgroundMethodInfluentialAutomatic Image Cropping: A Computational Complexity Study
Chen, J., Bai, G., Liang, S., & Li, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.61 · Cited by 116 · cited 1×
“[87] aim at extracting the most visually important region as the image crop.”
BackgroundQuery-Dependent Aesthetic Model With Deep Learning for Photo Quality Assessment
Tian, X., Dong, Z., Yang, K., & Mei, T. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2479916 · Cited by 100 · cited 5×
“[27] train a CNN with 4 convolution layers and 2 fully-connected layers to learn aesthetic features from data.”
BackgroundMethodInfluentialA multi-scene deep learning model for image aesthetic evaluation
Wang, N., Zhao, M., Wang, L., Huang, J., Cai, C., & Xu, X. (2016) · Signal Processing Image Communication · doi:10.1016/j.image.2016.05.009 · Cited by 100 · cited 5×
“96 - The authors in [22], [25], [26], [28], [29], [30], [33], [34], [35] have not released detailed results.”
BackgroundMethodInfluentialPhoto assessment based on computational visual attention model
Sun, X., Yao, H., Ji, R., & Liu, S. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631351 · Cited by 94 · cited 5×
“(2009) [11] 600-image private set Euclidean distance 3.”
BackgroundMethodInfluentialHierarchical aesthetic quality assessment using deep convolutional neural networks
Kao, Y., Huang, K., & Maybank, S. (2016) · Signal Processing Image Communication · doi:10.1016/j.image.2016.05.004 · Cited by 94 · cited 4×
“96 - The authors in [22], [25], [26], [28], [29], [30], [33], [34], [35] have not released detailed results.”
BackgroundMethodInfluentialAesthetic quality assessment of consumer photos with faces
Li, C., Gallagher, A., Loui, A. C. P., & Chen, T. (2010) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2010.5651833 · Cited by 92 · cited 4×
“[46] propose a regression model that targets only consumer photos with face.”
BackgroundMethodInfluentialVisual Link Retrieval in a Database of Paintings
Seguin, B., Striolo, C., di Lenardo, I., & Kaplan, F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46604-0_52 · Cited by 91 · cited 1×
“It is widely used in many recent vision studies [55], [70], [71], [72], aiming to bring data of the same class closer, while data of different classes further away.”
Visual aesthetic quality assessment with a regression model
Kao, Y., Wang, C., & Huang, K. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351067 · Cited by 90 · cited 1×
“Some other works [23], [27], [74], [75], [76] on AVA datasets uses only a small subset of images for evaluation, which is not directly comparable to canonical state-of-the-arts on the AVA standard partition (see Table 3).”
BackgroundFreeman, M. (2007) · Routledge eBooks · doi:10.4324/9780080556161 · Cited by 81
Photo Quality Assessment with DCNN that Understands Image Well
Dong, Z., Shen, X., Li, H., & Tian, X. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-14442-9_57 · Cited by 75 · cited 5×
“[23] propose to adopt the generic features from penultimate layer output of AlexNet [51] with spatial pyramid pooling.”
BackgroundMethodInfluentialPerceptual quality assessment based on visual attention analysis
You, J., Perkis, A., Hannuksela, M. M., & Gabbouj, M. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631356 · Cited by 73
Scale and Object Aware Image Thumbnailing
Sun, J. G., & Ling, H. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0618-z · Cited by 72
Scenic photo quality assessment with bag of aesthetics-preserving features
Su, H.-H., Chen, T., Kao, C.-C., Hsu, W. H., & Chien, S. (2011) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2072298.2071977 · Cited by 71 · cited 2×
“[48] propose bag-of-aesthetics-preserving features for scenic/landscape photographs.”
MethodBrain-Inspired Deep Networks for Image Aesthetics AssessmentPreprint
Wang, Z., Chang, S., Dolcos, F., Beck, D. M., Liu, D., & Huang, T. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.04155 · Cited by 68 · cited 13×
“Positive examples (high-quality images) that are wrongly classified by BDN and DMA-net but correctly classified by the DAN-1 baseline.”
BackgroundMethodInfluentialClive Bell’s “Significant Form” and the neurobiology of aesthetics
Zeki, S. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00730 · Cited by 56 · cited 1×
“From the beginning of psychological aesthetics studies by Fechner [5] to modern neuroaesthetics, researchers argue that there is a certain connection between human aesthetic experience and the sensation caused by visual stimuli regardless of source, culture, and experience [6], which is supported by activations in spe…”
BackgroundMavridaki, E., & Mezaris, V. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7350927 · Cited by 50 · cited 1×
“Some other works [23], [27], [74], [75], [76] on AVA datasets uses only a small subset of images for evaluation, which is not directly comparable to canonical state-of-the-arts on the AVA standard partition (see Table 3).”
BackgroundAssessment of Photo Aesthetics with Efficiency
Lo, K.-Y., Liu, K.-H., & Chen, C. (2014) · Cited by 49
Modeling photo composition and its application to photo re-arrangement
Park, J., Lee, J., Tai, Y., & Kweon, I. S. (2012) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2012.6467466 · Cited by 48 · cited 1×
“rlap ratio, higher is better. The second number (shown in parenthesis) is average boundary displacement error, lower is better. Previous work *Photographer 1 Photographer 2 Photographer 3 Park et al. [88] 0.6034 (0.1062) 0.5823 (0.1128) 0.6085 (0.1102) Yan et al. [85] 0.7487 (0.0667) 0.7288 (0.0720) 0.7322 (0.0719) Wa…”
InfluentialA Survey on Different Approaches Used In Image Quality Assessment
George, A. G., & Prabavathy, A. K. (2013) · Cited by 45
The Good, the Bad, and the Ugly: Predicting Aesthetic Image Labels
Wu, Y., Bauckhage, C., & Thurau, C. (2010) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2010.392 · Cited by 42 · cited 3×
“[41] propose the use of Gabor filter responses to estimate the position of the main object in images, and extract low-level HSV-color features from global and central image regions.”
BackgroundMethodArdizzone, E., Bruno, A., & Mazzola, G. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41181-6_78 · Cited by 41 · cited 1×
“Attention/Saliencybased approaches [78], [79], [80] typically extract the primary subject region in the scene of interest according to attention scores or saliency maps as the image crops.”
BackgroundA holistic approach to aesthetic enhancement of photographs
Bhattacharya, S., Sukthankar, R., & Shah, M. (2011) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/2037676.2037678 · Cited by 41 · cited 3×
“[16], [40] propose compositional features using relative foreground position and visual weight ratio to model the relations of foreground objects and the background scene, then an support vector regressor is trained.”
BackgroundMethodWandell, B. A., Dumoulin, S. O., & Brewer, A. A. (2009) · Elsevier eBooks · doi:10.1016/b978-008045046-9.00241-2 · Cited by 40 · cited 1×
“These activations in the visual cortex can be attributed to the processing of various early, intermediate and late visual features of the stimuli including orientation, shape, color grouping and categorization [13], [14], [15], [16].”
BackgroundMulti-level photo quality assessment with multi-view features
Dong, Z., & Tian, X. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.05.095 · Cited by 40 · cited 1×
“Some other works [23], [27], [74], [75], [76] on AVA datasets uses only a small subset of images for evaluation, which is not directly comparable to canonical state-of-the-arts on the AVA standard partition (see Table 3).”
BackgroundRelative features for photo quality assessment
Yeh, M.-C., & Cheng, Y.-C. (2012) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2012.6467496 · Cited by 28 · cited 4×
“(2012) [19] 309-image private set Kendalls Tau-b measure 0.”
BackgroundMethodInfluentialToward correlating and solving abstract tasks using convolutional neural networks
Peng, K., & Chen, T. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477616 · Cited by 28 · cited 4×
“[25] propose to train CNNs of AlexNet-like architecture for 8 different abstract tasks (emotion classification, artist classification, artistic style classification, aesthetic classification, fashion style classification, architectural style classification, memorability prediction and interestingness prediction).”
BackgroundMethodInfluentialAesthetic visual quality evaluation of Chinese handwritings
Sun, R., Lian, Z., Tang, Y., & Xiao, J. (2015) · International Conference on Artificial Intelligence · Cited by 28
Change-Based Image Cropping with Exclusion and Compositional Features
Yan, J., Lin, S. C.-F., Kang, S. B., & Tang, X. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0801-5 · Cited by 23 · cited 6×
“2 seconds for generating an output image crop on GPU and ∼2 seconds on CPU (compared to ∼11 seconds on CPU in [84]).”
BackgroundMethodInfluentialSelf-citationAssessing photo quality with geo-context and crowdsourced photos
Yin, W., Mei, T., & Chen, C. W. (2012) · Visual Communications and Image Processing · doi:10.1109/vcip.2012.6410821 · Cited by 19 · cited 3×
“(2012) [49] 13k-image private set from Flickr Overall accuracy 81% Lienhard et al.”
BackgroundMethodAutomatic Thumbnail Generation Based on Visual Representativeness and Foreground Recognizability
Huang, J., Chen, H., Wang, B., & Lin, S. C.-F. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.37 · Cited by 19 · cited 1×
“[86] target the visual representativeness and foreground recognizability when cropping and resizing an image to generate its thumbnail.”
BackgroundFunctional Neuroanatomy of the Human Visual System: A Review of Functional MRI Studies
Greenlee, M. W., & Tse, P. U. (2007) · Springer eBooks · doi:10.1007/978-3-540-33679-2_8 · Cited by 17 · cited 1×
“These activations in the visual cortex can be attributed to the processing of various early, intermediate and late visual features of the stimuli including orientation, shape, color grouping and categorization [13], [14], [15], [16].”
BackgroundLearning Relative Aesthetic Quality with a Pairwise Approach
Lv, H., & Tian, X. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-27671-7_41 · Cited by 14 · cited 3×
“Different metrics for performance evaluation of image aesthetic assessment models are used across the literature: classification accuracy [8], [10], [13], [16], [20], [22], [23], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [39], [40], [41], [47], [49], precision-and-recall curve [9], [13], [14],…”
BackgroundMethodLearning an Aesthetic Photo Cropping Cascade
Wang, P., Lin, Z., & Měch, R. (2015) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2015.66 · Cited by 13 · cited 1×
“vious work *Photographer 1 Photographer 2 Photographer 3 Park et al. [111] 0.6034 (0.1062) 0.5823 (0.1128) 0.6085 (0.1102) Yan et al. [108] 0.7487 (0.0667) 0.7288 (0.0720) 0.7322 (0.0719) Wang et al. [112] 0.7823 (0.0623) 0.7697 (0.0617) 0.7725 (0.0701) Yan et al. [107] 0.7974 (0.0528) 0:7857 (0.0567) 0.7723 (0.0594)…”
InfluentialLow Level Features for Quality Assessment of Facial Images
Lienhard, A., Ladret, P., & Caplier, A. (2015) · International Conference on Computer Vision Theory and Applications · doi:10.5220/0005308805450552 · Cited by 13 · cited 3×
“(2015) [47] Human Face Scores 250-image dataset Overall accuracy 86.”
BackgroundMethodA statistic approach for photo quality assessment
Lo, L.-Y., & Chen, J.-C. (2012) · International Conference on Information Security and Intelligent Control (ISIC) · doi:10.1109/isic.2012.6449719 · Cited by 11 · cited 5×
“Regional image features [13], [14], [15] later prove to be effective in complementing the global features.”
BackgroundMethodInfluentialSelf-restraint object recognition by model based CNN learning
Wang, Y., & Deng, W. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532438 · Cited by 7 · cited 1×
“It is widely used in many recent vision studies [55], [70], [71], [72], aiming to bring data of the same class closer, while data of different classes further away.”
Saliency based automatic image cropping using support vector machine classifier
Jaiswal, N., & Meghrajani, Y. K. (2015) · International Conference on Innovations in Information, Embedded and Communication Systems · doi:10.1109/iciiecs.2015.7193184 · Cited by 6 · cited 1×
“Attention/Saliencybased approaches [78], [79], [80] typically extract the primary subject region in the scene of interest according to attention scores or saliency maps as the image crops.”
BackgroundEngineering Deep Representations for Modeling Aesthetic PerceptionPreprint
Chen, Y., Hu, Y., Zhang, L., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07699 · cited 4×
“(2016) [32] CUHK-PQ Overall accuracy 88.”
BackgroundMethodInfluential
Research works printed but not linked: 12
- sheikh, 0, A visual information fidelity approach to video quality assessment, Proc 1st Int Workshop Video Processing and Quality Metrics for Consumer Electronics, 23
- kao, 2016, Visual aesthetic quality assessment with multi-task deep learning
- zhang, 2016, Describing human aesthetic perception by deeply-learned attributes from Flickr
- london, 2005, Photography
- fechner, 1876, Vorschule der Aesthetik, 1
- marchesotti, 0, Learning beautiful (and ugly) attributes, Proc British Machine Vision Conf (BMVC), 7, 1
- itten, 1975, Design and Form The Basic Course at the Bauhaus and Later
- freeman, 2007, The Complete Guide to Light and Lighting in Digital Photography (A Lark Photography Book)
- nishiyama, 0, Sensation-based photo cropning, Proc ACM Int Conf Multimedia, 669
- zeki, 1999, Inner Vision An Exploration of Art and the Brain
- van der maaten, 2008, Visualizing data using t-SNE, J Machine Learning Res, 9, 2579
- ang, 2002, Digital Photographer's Handbook
Cited by
Perceptual image quality assessment: a survey
Zhai, G., & Min, X. (2020) · Science China Information Sciences · doi:10.1007/s11432-019-2757-1 · Cited by 655
A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping
Wang, W., Shen, J., & Ling, H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840724 · Cited by 348
“Many methods have been proposed for this topic, as surveyed in [43].”
MethodDistort-and-Recover: Color Enhancement Using Deep Reinforcement Learning
Park, J., Lee, J.-Y., Yoo, D., & Kweon, I. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00621 · Cited by 211
“There has been a long line of works to assess the aesthetic scores of images as summarized in [2], but there is still no winning method that gives satisfying results.”
BackgroundPersonality-Assisted Multi-Task Learning for Generic and Personalized Image Aesthetics Assessment
Li, L., Zhu, H., Zhao, S., Ding, G., & Lin, W. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2968285 · Cited by 163
“The ACC is the most popular criteria for evaluating the performance of classification [1], and is computed by”
BackgroundA2-RL: Aesthetics Aware Reinforcement Learning for Image Cropping
Li, D., Wu, H., Zhang, J., & Huang, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00855 · Cited by 138
“Recently, thanks to the fast development of deep learning and newly proposed large scale datasets [22], there are many new works [16, 20, 8] which accomplish aesthetic quality assessment with convolutional neural networks.”
Background
Metadata
- Authors
- Deng, Y., Loy, C. C., & Tang, X.
- Affiliations
- Chinese University of Hong Kong (HK)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-07
- Volume
- 34
- Issue
- 4
- Pages
- 80-106
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W2529088810
- Cited by
- 306
- References
- 95
- Citations by year
- 2026: 14 · 2025: 29 · 2024: 26 · 2023: 40 · 2022: 39 · 2021: 39 · 2020: 43 · 2019: 43 · 2018: 29 · 2017: 4
- FWCI
- 13.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- image aesthetic assessment, handcrafted features, deep features, deep learning, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.