The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations
Open accessAbstract
Understanding and predicting the human visual attention mechanism is an active area of research in the fields of neuroscience and computer vision. In this paper, we propose DeepFix, a fully convolutional neural network, which models the bottom-up mechanism of visual attention via saliency prediction. Unlike classical works, which characterize the saliency map using various hand-crafted features, our model automatically learns features in a hierarchical fashion and predicts the saliency map in an end-to-end manner. DeepFix is designed to capture semantics at multiple scales while taking global context into account, by using network layers with very large receptive fields. Generally, fully convolutional nets are spatially invariant-this prevents them from modeling location-dependent patterns (e.g., centre-bias). Our network handles this by incorporating a novel location-biased convolutional layer. We evaluate our model on multiple challenging saliency data sets and show that it achieves the state-of-the-art results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
Harel, J., Koch, C., & Perona, P. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0073 · Cited by 3,439
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Global contrast based salient region detection
Cheng, M.-M., Zhang, G.-X., Mitra, N. J., Huang, X., & Hu, S.-M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995344 · Cited by 2,976
(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534
Global Contrast Based Salient Region Detection
Cheng, M.-M., Mitra, N. J., Huang, X., Torr, P. H. S., & Hu, S.-M. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345401 · Cited by 2,463
Learning to predict where humans look
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459462 · Cited by 2,016
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
The Secrets of Salient Object Segmentation
Li, Y., Hou, X., Koch, C., Rehg, J. M., & Yuille, A. L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.43 · Cited by 1,338
Visual saliency based on multiscale deep features
Li, G., & Yu, Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299184 · Cited by 1,163
Saliency Based on Information Maximization
Bruce, N. D. B., & Tsotsos, J. K. (2005) · neural information processing systems · Cited by 1,134
Tatler, B. W. (2007) · Journal of Vision · doi:10.1167/7.14.4 · Cited by 1,028
Saliency detection by multi-context deep learning
Zhao, R., Ouyang, W., Li, H., & Wang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298731 · Cited by 995
Interactions between attention and working memory
Awh, E., Vogel, E. K., & Oh, S.-H. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2005.08.023 · Cited by 883 · cited 1×
“It uses information available in the working memory, thereby biasing attention towards areas of the scene important to the current behavioral goals [3].”
Components of bottom-up gaze allocation in natural images
Peters, R. J., Iyer, A., Itti, L., & Koch, C. (2005) · Vision Research · doi:10.1016/j.visres.2005.03.019 · Cited by 739
Jiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729
SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks
Huang, X., Shen, C., Boix, X., & Zhao, Q. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.38 · Cited by 597
Covariance Tracking using Model Update Based on Lie Algebra
Porikli, F., Tuzel, O., & Meer, P. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.94 · Cited by 562
Visual Attention: Bottom-Up Versus Top-Down
Connor, C. E., Egeth, H. E., & Yantis, S. (2004) · Current Biology · doi:10.1016/j.cub.2004.09.041 · Cited by 529
A Benchmark of Computational Models of Saliency to Predict Human Fixations
Judd, T., Durand, F., & Torralba, A. B. (2012) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 511
Saliency Detection: A Boolean Map Approach
Zhang, J., & Sclaroff, S. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.26 · Cited by 495
Shallow and Deep Convolutional Networks for Saliency Prediction
Pan, J., Sayrol, E., Giro-I-Nieto, X., McGuinness, K., & OConnor, N. E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.71 · Cited by 471
Top-down control of visual attention in object detection
Oliva, A., Torralba, A., Castelhano, M. S., & Henderson, J. M. (2004) · Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429) · doi:10.1109/icip.2003.1246946 · Cited by 467
Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images
Vig, E., Dorr, M., & Cox, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.358 · Cited by 428
A visual attention model for adapting images on small displays
Chen, L.-Q., Xie, X., Fan, X., Ma, W.-Y., Zhang, H.-J., & Zhou, H.-Q. (2003) · Multimedia Systems · doi:10.1007/s00530-003-0105-4 · Cited by 419
Visual saliency estimation by nonlinearly integrating features using region covariances
Erdem, E., & Erdem, A. (2013) · Journal of Vision · doi:10.1167/13.4.11 · Cited by 396
Computational visual attention systems and their cognitive foundations: A survey
Frintrop, S., Rome, E., & Christensen, H. I. (2010) · ACM Transactions on Applied Perception · doi:10.1145/1658349.1658355 · Cited by 389
Quantifying center bias of observers in free viewing of dynamic natural scenes
Tseng, P. H., Carmi, R., Cameron, I. G. M., Munoz, D. P., & Itti, L. (2009) · Journal of Vision · doi:10.1167/9.7.4 · Cited by 362
Predicting human gaze beyond pixels
Xu, J., Jiang, M., Wang, S., Kankanhalli, M. S., & Zhao, Q. (2014) · Journal of Vision · doi:10.1167/14.1.28 · Cited by 353
Saliency-Aware Video Compression
Hadizadeh, H., & Bajic, I. V. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2282897 · Cited by 326
Methods for comparing scanpaths and saliency maps: strengths and weaknesses
Le Meur, O., & Baccino, T. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0226-9 · Cited by 314
Intrinsic and extrinsic effects on image memorability
Bylinskii, Z., Isola, P., Bainbridge, C., Torralba, A., & Oliva, A. (2015) · Vision Research · doi:10.1016/j.visres.2015.03.005 · Cited by 266
Depth Matters: Influence of Depth Cues on Visual Saliency
Lang, C., Nguyen, T. V., Katti, H., Yadati, K., Kankanhalli, M., & Yan, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_8 · Cited by 264
Learning a saliency map using fixated locations in natural scenes
Zhao, Q., & Koch, C. (2011) · Journal of Vision · doi:10.1167/11.3.9 · Cited by 256
Analysis of Scores, Datasets, and Models in Visual Saliency Prediction
Borji, A., Tavakoli, H. R., Sihite, D. N., & Itti, L. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.118 · Cited by 221
Attentional Selection for Object Recognition — A Gentle Way
Walther, D., Itti, L., Riesenhuber, M., Poggio, T., & Koch, C. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-36181-2_47 · Cited by 218
CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research
Borji, A., & Itti, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.03581 · Cited by 212
Predicting eye fixations using convolutional neural networks
Liu, N., Han, J., Zhang, D., Wen, S., & Liu, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298633 · Cited by 201
What is a Salient Object? A Dataset and a Baseline Model for Salient Object Detection
Borji, A. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2383320 · Cited by 172
Dense saliency-based spatiotemporal feature points for action recognition
Rapantzikos, K. E., Avrithis, Y., & Kollias, S. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206525 · Cited by 156
Image saliency by isocentric curvedness and color
Valenti, R., Sebe, N., & Gevers, T. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459240 · Cited by 155
Kruthiventi, S. S. S., Gudisa, V., Dholakiya, J. H., & Babu, R. V. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.623 · Cited by 145
Self-citationGarcía-Díaz, A., Leborán, V., Fdez-Vidal, X. R., & Pardo, X. M. (2012) · Journal of Vision · doi:10.1167/12.6.17 · Cited by 138
How many bits does it take for a stimulus to be salient?
Khatoonabadi, S. H., Vasconcelos, N. M., Bajić, I. V., & Shan, Y. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299189 · Cited by 110
Applying computational tools to predict gaze direction in interactive visual environments
Peters, R. J., & Itti, L. (2008) · ACM Transactions on Applied Perception · doi:10.1145/1279920.1279923 · Cited by 106
Itti, L. (2007) · Scholarpedia · doi:10.4249/scholarpedia.3327 · Cited by 104
Modeling fixation locations using spatial point processes
Barthelmé, S., Trukenbrod, H. A., Engbert, R., & Wichmann, F. A. (2013) · Journal of Vision · doi:10.1167/13.12.1 · Cited by 64
Learning to predict eye fixations for semantic contents using multi-layer sparse network
Shen, C., & Zhao, Q. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.09.053 · Cited by 49
Cited by
Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model
Cornia, M., Baraldi, L., Serra, G., & Cucchiara, R. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2851672 · Cited by 474
Understanding Low- and High-Level Contributions to Fixation Prediction
Kummerer, M., Wallis, T. S. A., Gatys, L. A., & Bethge, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.513 · Cited by 282
Revisiting Video Saliency: A Large-Scale Benchmark and a New Model
Wang, W., Shen, J., Guo, F., Cheng, M.-M., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00514 · Cited by 268
Human Gaze Assisted Artificial Intelligence: A Review
Zhang, R., Saran, A., Liu, B., Zhu, Y., Guo, S., Niekum, S., Ballard, D., & Hayhoe, M. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/689 · Cited by 48
Metadata
- Authors
- Kruthiventi, S. S. S., Ayush, K., & Babu, R. V.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2017-09
- Volume
- 26
- Issue
- 9
- Pages
- 4446-4456
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2210809762
- Cited by
- 478
- References
- 62
- FWCI
- 25.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.