The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Deep Visual Attention Prediction
Open accessAbstract
In this paper, we aim to predict human eye fixation with view-free scenes based on an end-to-end deep learning architecture. Although convolutional neural networks (CNNs) have made substantial improvement on human attention prediction, it is still needed to improve the CNN-based attention models by efficiently leveraging multi-scale features. Our visual attention network is proposed to capture hierarchical saliency information from deep, coarse layers with global saliency information to shallow, fine layers with local saliency response. Our model is based on a skip-layer network structure, which predicts human attention from multiple convolutional layers with various reception fields. Final saliency prediction is achieved via the cooperation of those global and local predictions. Our model is learned in a deep supervision manner, where supervision is directly fed into multi-level layers, instead of previous approaches of providing supervision only at the output layer and propagating this supervision back to earlier layers. Our model thus incorporates multi-level saliency predictions within a single network, which significantly decreases the redundancy of previous approaches of learning multiple network streams with different input scales. Extensive experimental analysis on various challenging benchmark data sets demonstrate our method yields the state-of-the-art performance with competitive inference time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
Holistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,096
Saliency Detection via Graph-Based Manifold Ranking
Yang, C., Zhang, L., Lu, H., Ruan, X., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.407 · Cited by 2,554
Learning to predict where humans look
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459462 · Cited by 2,016
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Context-Aware Saliency Detection
Goferman, S., Zelnik-Manor, L., & Tal, A. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.272 · Cited by 1,567
Salient Object Detection: A Benchmark
Borji, A., Cheng, M.-M., Jiang, H., & Li, J. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487833 · Cited by 1,364
The Secrets of Salient Object Segmentation
Li, Y., Hou, X., Koch, C., Rehg, J. M., & Yuille, A. L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.43 · Cited by 1,338
SUN: A Bayesian framework for saliency using natural statistics
Zhang, L., Tong, M. H., Marks, T. K., Shan, H., & Cottrell, G. W. (2008) · Journal of Vision · doi:10.1167/8.7.32 · Cited by 1,227
Saliency detection by multi-context deep learning
Zhao, R., Ouyang, W., Li, H., & Wang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298731 · Cited by 995
Learning to Detect A Salient Object
Liu, T., Sun, J., Zheng, N.-N., Tang, X., & Shum, H.-Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383047 · Cited by 946
Saliency, attention, and visual search: An information theoretic approach
Bruce, N. D. B., & Tsotsos, J. K. (2009) · Journal of Vision · doi:10.1167/9.3.5 · Cited by 855
Learning to Refine Object Segments
Pinheiro, P. O., Lin, T.-Y., Collobert, R., & Dollár, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_5 · Cited by 731
Jiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729
Holistically-nested edge detection
Xie, S., & Tu, Z. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1004-z · Cited by 682
A coherent computational approach to model bottom-up visual attention
Le Meur, O., Le Callet, P., Barba, D., & Thoreau, D. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.86 · Cited by 642
Video Salient Object Detection via Fully Convolutional Networks
Wang, W., Shen, J., & Shao, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2754941 · Cited by 642
Self-citationSALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks
Huang, X., Shen, C., Boix, X., & Zhao, Q. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.38 · Cited by 597
Saliency-aware geodesic video object segmentation
Wang, W., Shen, J., & Porikli, F. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298961 · Cited by 524
Self-citationSaliency Detection: A Boolean Map Approach
Zhang, J., & Sclaroff, S. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.26 · Cited by 495
Saliency-Aware Video Object Segmentation
Wang, W., Shen, J., Yang, R., & Porikli, F. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2662005 · Cited by 472
Self-citationShallow and Deep Convolutional Networks for Saliency Prediction
Pan, J., Sayrol, E., Giro-I-Nieto, X., McGuinness, K., & OConnor, N. E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.71 · Cited by 471
Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images
Vig, E., Dorr, M., & Cox, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.358 · Cited by 428
A deep multi-level network for saliency prediction
Cornia, M., Baraldi, L., Serra, G., & Cucchiara, R. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7900174 · Cited by 360
Predicting human gaze beyond pixels
Xu, J., Jiang, M., Wang, S., Kankanhalli, M. S., & Zhao, Q. (2014) · Journal of Vision · doi:10.1167/14.1.28 · Cited by 353
Lazy Random Walks for Superpixel Segmentation
Shen, J., Du, Y., Wang, W., & Li, X. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2302892 · Cited by 339
Self-citationGao, D., Han, S., & Vasconcelos, N. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.27 · Cited by 294
SUN: Top-down saliency using natural statistics
Kanan, C., Tong, M. H., Zhang, L., & Cottrell, G. W. (2009) · Visual Cognition · doi:10.1080/13506280902771138 · Cited by 262
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
Kümmerer, M., Theis, L., & Bethge, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.1045 · Cited by 231
Yao, X., Han, J., Zhang, D., & Nie, F. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2694222 · Cited by 230
Saliency-based discriminant tracking
Mahadevan, V., & Vasconcelos, N. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206573 · Cited by 227
Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization
Zhang, L., Zhang, Q., & Xiao, C. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2465159 · Cited by 196
Saliency propagation from simple to difficult
Gong, C., Tao, D., Liu, W., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298868 · Cited by 156
Kruthiventi, S. S. S., Gudisa, V., Dholakiya, J. H., & Babu, R. V. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.623 · Cited by 145
Correspondence Driven Saliency Transfer
Wang, W., Shen, J., Shao, L., et al. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2601784 · Cited by 143
Self-citationEnd-to-End Saliency Mapping via Probability Distribution Prediction
Jetley, S., Murray, N., & Vig, E. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.620 · Cited by 140
Revealing Event Saliency in Unconstrained Video Collection
Zhang, D., Han, J., Jiang, L., Ye, S., & Chang, X. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2658957 · Cited by 128
Probabilistic learning of task-specific visual attention
Borji, A., Sihite, D. N., & Itti, L. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247710 · Cited by 124
Discriminative Multi-instance Multitask Learning for 3D Action Recognition
Yang, Y., Deng, C. D., Gao, S., Liu, W., Tao, D., & Gao, X. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2626959 · Cited by 112
Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks
Liu, N., Han, J., Liu, T., & Li, X. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2628878 · Cited by 112
Salient Object Segmentation via Effective Integration of Saliency and Objectness
Ye, L., Liu, Z., Li, L., Shen, L., Bai, C., & Wang, Y. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2693022 · Cited by 92
Multi-task human action recognition via exploring super-category
Yang, Y., Liu, R., Deng, C. D., & Gao, X. (2015) · Signal Processing · doi:10.1016/j.sigpro.2015.10.035 · Cited by 48
Cited by
Underwater scene prior inspired deep underwater image and video enhancement
Li, C., Anwar, S., & Porikli, F. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2019.107038 · Cited by 1,252
DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer
Tang, W., He, F., Liu, Y., Duan, Y., & Si, T. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3234340 · Cited by 571
See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks
Lu, X., Wang, W., Ma, C., Shen, J., Shao, L., & Porikli, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00374 · Cited by 521
Self-citationA Deep Network Solution for Attention and Aesthetics Aware Photo Cropping
Wang, W., Shen, J., & Ling, H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840724 · Cited by 348
Self-citationMultiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images
Li, R., Zheng, S., Zhang, C., Duan, C., Su, J., Wang, L., & Atkinson, P. M. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3093977 · Cited by 341
Metadata
- Authors
- Wang, W., & Shen, J.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2018-05
- Volume
- 27
- Issue
- 5
- Pages
- 2368-2378
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2612135493
- Cited by
- 585
- References
- 49
- FWCI
- 28.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.