The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping
Abstract
We study the problem of photo cropping, which aims to find a cropping window of an input image to preserve as much as possible its important parts while being aesthetically pleasant. Seeking a deep learning-based solution, we design a neural network that has two branches for attention box prediction (ABP) and aesthetics assessment (AA), respectively. Given the input image, the ABP network predicts an attention bounding box as an initial minimum cropping window, around which a set of cropping candidates are generated with little loss of important information. Then, the AA network is employed to select the final cropping window with the best aesthetic quality among the candidates. The two sub-networks are designed to share the same full-image convolutional feature map, and thus are computationally efficient. By leveraging attention prediction and aesthetics assessment, the cropping model produces high-quality cropping results, even with the limited availability of training data for photo cropping. The experimental results on benchmark datasets clearly validate the effectiveness of the proposed approach. In addition, our approach runs at 5 fps, outperforming most previous solutions. The code and results are available at: https://github.com/shenjianbing/DeepCropping.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2016) · Cited by 31,590
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Frequency-tuned salient region detection
Achanta, R., Hemami, S., Estrada, F., & Susstrunk, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206596 · Cited by 4,167
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Harel, J., Koch, C., & Perona, P. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0073 · Cited by 3,439
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Visual attention: The past 25 years
Carrasco, M. (2011) · Vision Research · doi:10.1016/j.visres.2011.04.012 · Cited by 2,483
Learning to predict where humans look
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459462 · Cited by 2,016
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Context-Aware Saliency Detection
Goferman, S., Zelnik-Manor, L., & Tal, A. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.272 · Cited by 1,567
Salient Object Detection: A Benchmark
Borji, A., Cheng, M.-M., Jiang, H., & Li, J. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487833 · Cited by 1,364
The Secrets of Salient Object Segmentation
Li, Y., Hou, X., Koch, C., Rehg, J. M., & Yuille, A. L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.43 · Cited by 1,338
Saliency Based on Information Maximization
Bruce, N. D. B., & Tsotsos, J. K. (2005) · neural information processing systems · Cited by 1,307
SUN: A Bayesian framework for saliency using natural statistics
Zhang, L., Tong, M. H., Marks, T. K., Shan, H., & Cottrell, G. W. (2008) · Journal of Vision · doi:10.1167/8.7.32 · Cited by 1,227
Studying Aesthetics in Photographic Images Using a Computational Approach
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744078_23 · Cited by 1,107
Learning to Detect A Salient Object
Liu, T., Sun, J., Zheng, N.-N., Tang, X., & Shum, H.-Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383047 · Cited by 946
AVA: A large-scale database for aesthetic visual analysis
Murray, N., Marchesotti, L., & Perronnin, F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247954 · Cited by 936
Image Signature: Highlighting Sparse Salient Regions
Hou, X., Harel, J., & Koch, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.146 · Cited by 834
Jiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729
Video Salient Object Detection via Fully Convolutional Networks
Wang, W., Shen, J., & Shao, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2754941 · Cited by 642
Self-citationThe Design of High-Level Features for Photo Quality Assessment
Ke, Y., Tang, X., & Jing, F. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.303 · Cited by 630
Deep Visual Attention Prediction
Wang, W., & Shen, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2787612 · Cited by 585
Self-citationSaliency-aware geodesic video object segmentation
Wang, W., Shen, J., & Porikli, F. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298961 · Cited by 524
Self-citationHigh level describable attributes for predicting aesthetics and interestingness
Dhar, S., Ordonez, V., & Berg, T. L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995467 · Cited by 500
Saliency-Aware Video Object Segmentation
Wang, W., Shen, J., Yang, R., & Porikli, F. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2662005 · Cited by 472
Self-citationShallow and Deep Convolutional Networks for Saliency Prediction
Pan, J., Sayrol, E., Giro-I-Nieto, X., McGuinness, K., & OConnor, N. E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.71 · Cited by 471
Photo Aesthetics Ranking Network with Attributes and Content Adaptation
Kong, S., Shen, X., Lin, Z., Mech, R., & Fowlkes, C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_40 · Cited by 468
Dynamic visual attention: searching for coding length increments
Hou, X., & Zhang, L. (2008) · CaltechAUTHORS (California Institute of Technology) · Cited by 431
Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images
Vig, E., Dorr, M., & Cox, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.358 · Cited by 428
Automatic thumbnail cropping and its effectiveness
Suh, B., Ling, H., Bederson, B. B., & Jacobs, D. W. (2003) · Proceedings of the 16th annual ACM symposium on User interface software and technology · doi:10.1145/964696.964707 · Cited by 421
Self-citationAssessing the aesthetic quality of photographs using generic image descriptors
Marchesotti, L., Perronnin, F., Larlus, D., & Csurka, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126444 · Cited by 405
Real-Time Superpixel Segmentation by DBSCAN Clustering Algorithm
Shen, J., Hao, X., Liang, Z., Liu, Y., Wang, W., & Shao, L. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2616302 · Cited by 384
Self-citationRAPID: Rating Pictorial Aesthetics using Deep Learning
Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654927 · Cited by 383
A framework for visual saliency detection with applications to image thumbnailing
Marchesotti, L., Cifarelli, C., & Csurka, G. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459467 · Cited by 322
Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation
Lu, X., Lin, Z., Shen, X., Mech, R., & Wang, J. Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.119 · Cited by 321
Image Aesthetic Assessment: An experimental survey
Deng, Y., Loy, C. C., & Tang, X. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2696576 · Cited by 306 · cited 1×
“Many methods have been proposed for this topic, as surveyed in [43].”
MethodComposition-Preserving Deep Photo Aesthetics Assessment
Mai, L., Jin, H., & Liu, F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.60 · Cited by 277
Revisiting Video Saliency: A Large-Scale Benchmark and a New Model
Wang, W., Shen, J., Guo, F., Cheng, M.-M., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00514 · Cited by 268
Self-citationBoosting bottom-up and top-down visual features for saliency estimation
Borji, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247706 · Cited by 266
Content-based photo quality assessment
Luo, W., Wang, X., & Tang, X. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126498 · Cited by 258
Rating Image Aesthetics Using Deep Learning
Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477040 · Cited by 239
Exploiting Surroundedness for Saliency Detection: A Boolean Map Approach
Zhang, J., & Sclaroff, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2473844 · Cited by 235
Discriminant Saliency for Visual Recognition from Cluttered Scenes
Gao, D., & Vasconcelos, N. M. (2004) · neural information processing systems · Cited by 223
Aesthetic quality classification of photographs based on color harmony
Nishiyama, M., Okabe, T., Sato, I., & Sato, Y. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995539 · Cited by 208
Salient Object Detection Driven by Fixation Prediction
Wang, W., Shen, J., Dong, X., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00184 · Cited by 205
Self-citationProbabilistic Graphlet Transfer for Photo Cropping
Zhang, L., Song, M., Zhao, Q., Liu, X., Bu, J., & Chen, C. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2223226 · Cited by 202
Predicting eye fixations using convolutional neural networks
Liu, N., Han, J., Zhang, D., Wen, S., & Liu, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298633 · Cited by 201
The discriminant center-surround hypothesis for bottom-up saliency
Gao, D., Mahadevan, V., & Vasconcelos, N. M. (2007) · neural information processing systems · Cited by 195
Cheng, B., Ni, B., Yan, S., & Tian, Q. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1873992 · Cited by 154
Correspondence Driven Saliency Transfer
Wang, W., Shen, J., Shao, L., et al. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2601784 · Cited by 143
Self-citationCong, R., Lei, J., Fu, H., Huang, Q., Cao, X., & Hou, C. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2763819 · Cited by 137
Learning the Change for Automatic Image Cropping
Yan, J., Lin, S. C.-F., Kang, S. B., & Tang, X. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.130 · Cited by 135
Automatic Image Cropping: A Computational Complexity Study
Chen, J., Bai, G., Liang, S., & Li, Z. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.61 · Cited by 133
Deep Cropping via Attention Box Prediction and Aesthetics Assessment
Wang, W., & Shen, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.240 · Cited by 131
Self-citationChen, F., Lin, Z., Měch, R., & Shen, X. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654979 · Cited by 113
Blind Image Quality Assessment Using Semi-supervised Rectifier Networks
Tang, H., Joshi, N., & Kapoor, A. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.368 · Cited by 108
Mishra, A. K., Aloimonos, Y., Cheong, L. F., & Kassim, A. A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.171 · Cited by 98
Quantitative Analysis of Automatic Image Cropping Algorithms: A Dataset and Comparative Study
Chen, Y., Huang, T.-W., Chang, K.-H., Tsai, Y.-C., Chen, H.-T., & Chen, B. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.32 · Cited by 95
Scale and Object Aware Image Thumbnailing
Sun, J. G., & Ling, H. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0618-z · Cited by 72
Self-citationScenic photo quality assessment with bag of aesthetics-preserving features
Su, H.-H., Chen, T., Kao, C.-C., Hsu, W. H., & Chien, S. (2011) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2072298.2071977 · Cited by 71
Modeling photo composition and its application to photo re-arrangement
Park, J., Lee, J., Tai, Y., & Kweon, I. S. (2012) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2012.6467466 · Cited by 48
Cited by
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization
Jiang, P.-T., Zhang, C.-B., Hou, Q., Cheng, M.-M., & Wei, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3089943 · Cited by 861
Salient Object Detection in the Deep Learning Era: An In-Depth Survey
Wang, W., Lai, Q., Fu, H., Shen, J., Ling, H., & Yang, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3051099 · Cited by 811
Self-citationSalient Object Detection With Pyramid Attention and Salient Edges
Wang, W., Zhao, S., Shen, J., Hoi, S. C. H., & Borji, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00154 · Cited by 535
Self-citationSwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection
Liu, Z., Tan, Y., He, Q., & Xiao, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3127149 · Cited by 384
Learning Selective Self-Mutual Attention for RGB-D Saliency Detection
Liu, N., Zhang, N., & Han, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01377 · Cited by 335
Metadata
- Authors
- Wang, W., Shen, J., & Ling, H.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-07-01
- Volume
- 41
- Issue
- 7
- Pages
- 1531-1544
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2804743778
- Cited by
- 348
- References
- 70
- FWCI
- 26.54
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.