SALICON: Saliency in Context
The DOI matches its Crossref record; 38 of the 40 references it cites were found in a registry or an index.
Abstract
Saliency in Context (SALICON) is an ongoing effort that aims at understanding and predicting visual attention. This paper presents a new method to collect large-scale human data during natural explorations on images. While current datasets present a rich set of images and task-specific annotations such as category labels and object segments, this work focuses on recording and logging how humans shift their attention during visual exploration. The goal is to offer new possibilities to (1) complement task-specific annotations to advance the ultimate goal in visual understanding, and (2) understand visual attention and learn saliency models, all with human attentional data at a much larger scale. We designed a mouse-contingent multi-resolutional paradigm based on neurophysiological and psychophysical studies of peripheral vision, to simulate the natural viewing behavior of humans. The new paradigm allowed using a general-purpose mouse instead of an eye tracker to record viewing behaviors, thus enabling large-scale data collection. The paradigm was validated with controlled laboratory as well as large-scale online data. We report in this paper a proof-of-concept SALICON dataset of human “free-viewing” data on 10,000 images from the Microsoft COCO (MS COCO) dataset with rich contextual information. We evaluated the use of the collected data in the context of saliency prediction, and demonstrated them a good source as ground truth for the evaluation of saliency algorithms.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To solve this problem, supervised attention methods tried to use additional loss functions to learn to predict gold attention on a manually labeled corpus [31,49,102].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [49]
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,702
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
The VideoToolbox software for visual psychophysics: transforming numbers into movies
Pelli, D. G. (1997) · Spatial Vision · doi:10.1163/156856897x00366 · Cited by 11,098
Harel, J., Koch, C., & Perona, P. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0073 · Cited by 3,439
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
Learning to predict where humans look
Judd, T., Ehinger, K., Durand, F., & Torralba, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459462 · Cited by 2,016
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Research works printed but not linked: 3
- 10.1097/00006324-199806000-00024 · doi:10.1097/00006324-199806000-00024
- hunziker, 2006, Im Auge des Lesers: foveale und periphere Wahrnehmung - vom Buchstabieren zur Lesefreude, Trans-media Verlag
- shen, 2014, Webpage saliency, ECCV, 33
Cited by
Items 701–736 of 736
Indexation bio-inspirée pour la recherche d'images par similarité
Michaud, D. (2018) · theses.fr (ABES) · doi:10.70675/100cc7d7zae8bz447az8d92z5d9607455e96 · Cited by 1
Salient Object Segmentation in 360° images/videos and light field
Zhang, Y. (2022) · theses.fr (ABES) · doi:10.70675/7b4980a9z109dz4ccez8cf5z0fef8b2c9cb6
Bruckert, A. (2022) · theses.fr (ABES) · doi:10.70675/262b3877zb18ez4253zb3adzae021c3b708e · Cited by 1
Visual quality of rendered 3D meshes with color attributes : Subjective and objective evaluation
Nehmé, Y. (2021) · theses.fr (ABES) · doi:10.70675/d7b16080za911z400dz9b22z099d29a5e415
A Gaze into the Art World : Predicting Visual Attention Using Deep Learning
Kerkouri, M. A. (2024) · theses.fr (ABES) · doi:10.70675/0cb19464zf546z44c8z9f0azbc4762fdca0e · Cited by 2
Revising the use of deep learning for effective point clouds quality assessment
Tliba, M. (2024) · theses.fr (ABES) · doi:10.70675/f90cc53ezc254z4385za778z0a193fee136b
Fournier, J. (2024) · theses.fr (ABES) · doi:10.70675/b66eb7d5z757dz4ffaz9272z37070604de9f
Pelissier-Combescure, M. (2024) · theses.fr (ABES) · doi:10.70675/874519fczcde2z43c1zb881zef150fa5605b
Deep learning for adaptive 360° video streaming in virtual reality
Guimard, Q. (2023) · theses.fr (ABES) · doi:10.70675/d712f5c7zf97fz4b2cz903dze42113dd5a2c
Xu-Darme, R. (2023) · theses.fr (ABES) · doi:10.70675/1c468aadz651fz460czb151za8c972f432a2 · Cited by 1
Transformation-tolerant object recognition in tree shrews despite lacking a foveaPreprint
Meyer, E. E., Ong, W. S., Song, C., Cottaris, N. P., Zhang, L.-Q., Collina, J. S., Brainard, D. H., & Arcaro, M. J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.04.10.717715
Salience prediction methods for video cropping in sidewalk footage
Costa, S. M., Damaceno, R. J. P., Morimitsu, H., & Junior, R. M. C. (2026) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2026.5895
Combining Deep Learning Mechanisms to Predict Interests from Gaze
Izumi, A., Harada, F., & Shimakawa, H. (2026) · Sensors · doi:10.3390/s26102975
Is this it? Benchmarking Scanpath Metrics for Information Display
Wang, Y., Nagasawa, J., Shi, D., Jiao, C., Jiang, Y., & Bulling, A. (2026) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3797246.3805691
How do we look at images through polygonal apertures: The PolyCAT Dataset
Aklilu, B., & Ben‐Shahar, O. (2026) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3797246.3806795
Visitor and Kangaroo as a Large-Scale AOI-Structured Yarbus Replication Dataset
Popelka, S., Fačevicová, K., Vymazal, J., & Beitlová, M. (2026) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3797246.3806792
Nagasawa, J., Isomoto, T., Hiroe, M., & Nagamatsu, T. (2026) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3797246.3803032
Studying Webcam-based Gaze Estimation and Mouse Coordination for Cognitive Inferences
Doubabi, A., Bektaş, K., Aamouche, A., Kabtane, H. E., & Andaloussi, A. A. (2026) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3797246.3805849
MCFW-Gaze: MultiContext Free-Web Gaze Dataset
Nagasawa, J., Isomoto, T., Hiroe, M., & Nagamatsu, T. (2026) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3797246.3806660
Attention Comparison in Task-Free vs. Task-Driven Viewing: AI vs. Real Images
Şimşek, İ., & Ulu, E. (2026) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/ichora69329.2026.11537066
Windarto, Y. E., Lee, G., & Jhang, K. (2026) · IEEE Access · doi:10.1109/access.2026.3704615
Pathologist Attention–Aligned Report Generation for Prostate Histopathology
Xue, R., Singh, S., Chakraborty, S., Marza, P., Yaskiv, O., Friedman, C., Sheuka, N., Friedman, P., Ramlal, B., Knudsen, B., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38224-5_35
ObjectVisA-120: Object-based Visual Attention Prediction in Interactive Street-crossing Environments
Vozniak, I., Müller, P., Lipp, N., Sprenger, J., Poddubnyy, K., Hovhannisyan, D., Müller, C., Bulling, A., & Slusallek, P. (2026) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv66570.2026.11624005
Liu, Y., Shi, W., Zhang, S., Zhang, A., & Peng, L. (2026) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi15080363
XAI or Attention: Improving Performance of Object Detectors with XBL
Zhukov, A., Benois‐Pineau, J., Youssef, A., Zemmari, A., Mosbah, M., & Taillandier, V. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630047
AI-Assisted Layout Generation for E-Book Interfaces Based on Visual Attention Prediction
Wang, B., Yang, H., Yu, M., Hou, J. M., & Cao, M. (2026) · International Conference on Artificial Intelligence and Generative Design · doi:10.1145/3830911.3830922
Adaptive Parallax-Based Contrast Enhancement for Power-Efficient Stereoscopic OLED Displays
Huang, F.-Y., & Ruan, S.-J. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3728614
Machine Learning in Its Social Context
Fiebrink, R., Gillies, M. F. P., & Ramos, G. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.015
The Anatomy of the Saliency Cropping Leviathan
Prabhu, V. U., & Birhane, A. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.018
Spectral Vs Topological Attention in Event-Based Vision
Pham, T.-T.-D., Tran, Q.-T., Mazna, R., & Martinet, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38410-2_4
Where Might You Look? Task-Driven Gaze Prediction for Attack-Surface Estimation in Mixed Reality
Sprintson, M., Williford, J. R., & Xu, J. (2026) · Workshop on Enhancing Security, Privacy, and Trust in Extended Reality · doi:10.1145/3842203.3844588
HuGDAT: Guiding Transformer Attention with Human Vision
Waseem, A., Ruiu, P., Lagorio, A., Nixon, S., & Tistarelli, M. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831146
CS-FEM: Class-Specific Feature Explanation Method and Its Relation to Grad-CAM
Un, A. F., Đình, N. L., Benois‐Pineau, J., & Escudero-Viñolo, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39388-3_30
Metadata
- Authors
- Jiang, M., Huang, S., Duan, J., & Zhao, Q.
- Affiliations
- National University of Singapore (SG)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 1072-1080
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1934890906
- Cited by
- 729
- References
- 40
- Citations by year
- 2026: 30 · 2025: 58 · 2024: 75 · 2023: 76 · 2022: 75 · 2021: 100 · 2020: 93 · 2019: 84 · 2018: 60 · 2017: 44 · 2016: 27 · 2015: 7
- FWCI
- 18.87
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- visual attention, saliency prediction, peripheral vision
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.