The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Visual Saliency Detection Based on Multiscale Deep CNN Features
Open accessAbstract
Visual saliency is a fundamental problem in both cognitive and computational sciences, including computer vision. In this paper, we discover that a high-quality visual saliency model can be learned from multiscale features extracted using deep convolutional neural networks (CNNs), which have had many successes in visual recognition tasks. For learning such saliency models, we introduce a neural network architecture, which has fully connected layers on top of CNNs responsible for feature extraction at three different scales. The penultimate layer of our neural network has been confirmed to be a discriminative high-level feature vector for saliency detection, which we call deep contrast feature. To generate a more robust feature, we integrate handcrafted low-level features with our deep contrast feature. To promote further research and evaluation of visual saliency models, we also construct a new large database of 4447 challenging images and their pixelwise saliency annotations. Experimental results demonstrate that our proposed method is capable of achieving the state-of-the-art performance on all public benchmarks, improving the F-measure by 6.12% and 10%, respectively, on the DUT-OMRON data set and our new data set (HKU-IS), and lowering the mean absolute error by 9% and 35.3%, respectively, on these two data sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216
Efficient Graph-Based Image Segmentation
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000022288.19776.77 · Cited by 6,176
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759
Contour Detection and Hierarchical Image Segmentation
Arbeláez, P., Maire, M., Fowlkes, C., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.161 · Cited by 5,454
Computational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Frequency-tuned salient region detection
Achanta, R., Hemami, S., Estrada, F., & Susstrunk, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206596 · Cited by 4,167
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Harel, J., Koch, C., & Perona, P. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0073 · Cited by 3,439
Saliency Detection: A Spectral Residual Approach
Hou, X., & Zhang, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383267 · Cited by 3,422
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Global contrast based salient region detection
Cheng, M.-M., Zhang, G.-X., Mitra, N. J., Huang, X., & Hu, S.-M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995344 · Cited by 2,976
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Saliency Detection via Graph-Based Manifold Ranking
Yang, C., Zhang, L., Lu, H., Ruan, X., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.407 · Cited by 2,554
Global Contrast Based Salient Region Detection
Cheng, M.-M., Mitra, N. J., Huang, X., Torr, P. H. S., & Hu, S.-M. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345401 · Cited by 2,463
Saliency filters: Contrast based filtering for salient region detection
Perazzi, F., Krahenbuhl, P., Pritch, Y., & Hornung, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247743 · Cited by 2,110
Learning to predict where humans look
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459462 · Cited by 2,016
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Learning to Detect a Salient Object
Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., & Shum, H.-Y. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.70 · Cited by 1,777
Hierarchical Saliency Detection
Yan, Q., Xu, L., Shi, J., & Jia, J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.153 · Cited by 1,773
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511
Seam carving for content-aware image resizing
Avidan, S., & Shamir, A. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276390 · Cited by 1,362
The Secrets of Salient Object Segmentation
Li, Y., Hou, X., Koch, C., Rehg, J. M., & Yuille, A. L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.43 · Cited by 1,338
Saliency Optimization from Robust Background Detection
Zhu, W., Liang, S., Wei, Y., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.360 · Cited by 1,330
Visual saliency based on multiscale deep features
Li, G., & Yu, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299184 · Cited by 1,163
Self-citationSalient Object Detection: A Discriminative Regional Feature Integration Approach
Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., & Li, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.271 · Cited by 1,147
Seam carving for content-aware image resizing
Avidan, S., & Shamir, A. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276390 · Cited by 1,092
Saliency detection by multi-context deep learning
Zhao, R., Ouyang, W., Li, H., & Wang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298731 · Cited by 995
Contrast-based image attention analysis by using fuzzy growing
Ma, Y.-F., & Zhang, H.-J. (2003) · Proceedings of the eleventh ACM international conference on Multimedia · doi:10.1145/957013.957094 · Cited by 965
Deep Contrast Learning for Salient Object Detection
Li, G., & Yu, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.58 · Cited by 704
Self-citationA unified approach to salient object detection via low rank matrix recovery
Shen, X., & Wu, Y. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247758 · Cited by 691
Deep networks for saliency detection via local estimation and global search
Wang, L., Lu, H., Ruan, X., & Yang, M.-H. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298938 · Cited by 676
Summarizing visual data using bidirectional similarity
Simakov, D., Caspi, Y., Shechtman, E., & Irani, M. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587842 · Cited by 613
Efficient Salient Region Detection with Soft Image Abstraction
Cheng, M.-M., Warrell, J., Lin, W.-Y., Zheng, S., Vineet, V., & Crook, N. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.193 · Cited by 568
Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration
Alpert, S., Galun, M., Basri, R., & Brandt, A. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383017 · Cited by 485
iCoseg: Interactive co-segmentation with intelligent scribble guidance
Batra, D., Kowdle, A., Parikh, D., Luo, J., & Chen, T. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540080 · Cited by 468
Saliency detection via Cellular Automata
Qin, Y., Lu, H., Xu, Y., & Wang, H. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298606 · Cited by 460
Natural scene statistics at the centre of gaze
Reinagel, P., & Zador, A. M. (1999) · Network Computation in Neural Systems · doi:10.1088/0954-898x/10/4/304 · Cited by 434
Salient Object Detection: A Benchmark
Borji, A., Sihite, D. N., & Itti, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_30 · Cited by 416
Fusing generic objectness and visual saliency for salient object detection
Chang, K.-Y., Liu, T.-L., Chen, H.-T., & Lai, S.-H. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126333 · Cited by 345
A framework for visual saliency detection with applications to image thumbnailing
Marchesotti, L., Cifarelli, C., & Csurka, G. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459467 · Cited by 322
Boosting bottom-up and top-down visual features for saliency estimation
Borji, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247706 · Cited by 266
Saliency Aggregation: A Data-Driven Approach
Mai, L., Niu, Y., & Liu, F. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.150 · Cited by 152
Rother, C., Bordeaux, L., Hamadi, Y., & Blake, A. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141965 · Cited by 152
Deep Contrast Learning for Salient Object DetectionPreprint
Li, G., & Yu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.01976 · Cited by 143
Self-citationCategory-Independent Object-Level Saliency Detection
Jia, Y., & Han, M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.221 · Cited by 138
Wang, K., Lin, L., Lu, J., et al. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2432712 · Cited by 77
Weighted attentional blocks for probabilistic object tracking
Wu, H., Li, G., & Luo, X. (2013) · The Visual Computer · doi:10.1007/s00371-013-0823-3 · Cited by 47
Self-citationRuobing, W., Yu, Y., & Wang, W. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.117 · Cited by 34
Self-citation
Cited by
Object Detection With Deep Learning: A Review
Zhao, Z.-Q., Zheng, P., Xu, S.-T., & Wu, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876865 · Cited by 5,545
U2-Net: Going deeper with nested U-structure for salient object detection
Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O. R., & Jagersand, M. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107404 · Cited by 2,407
BASNet: Boundary-Aware Salient Object Detection
Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., & Jagersand, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00766 · Cited by 1,565
Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation
Fu, H., Cheng, J., Xu, Y., Wong, D. W. K., Liu, J., & Cao, X. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2791488 · Cited by 834
Reverse Attention for Salient Object Detection
Chen, S., Tan, X., Wang, B., & Hu, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_15 · Cited by 684
Metadata
- Authors
- Li, G., & Yu, Y.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2016-11
- Volume
- 25
- Issue
- 11
- Pages
- 5012-5024
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2518599539
- Cited by
- 394
- References
- 56
- FWCI
- 30.58
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.