The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Objects predict fixations better than early saliency
Open accessAbstract
Humans move their eyes while looking at scenes and pictures. Eye movements correlate with shifts in attention and are thought to be a consequence of optimal resource allocation for high-level tasks such as visual recognition. Models of attention, such as "saliency maps," are often built on the assumption that "early" features (color, contrast, orientation, motion, and so forth) drive attention directly. We explore an alternative hypothesis: Observers attend to "interesting" objects. To test this hypothesis, we measure the eye position of human observers while they inspect photographs of common natural scenes. Our observers perform different tasks: artistic evaluation, analysis of content, and search. Immediately after each presentation, our observers are asked to name objects they saw. Weighted with recall frequency, these objects predict fixations in individual images better than early saliency, irrespective of task. Also, saliency combined with object positions predicts which objects are frequently named. This suggests that early saliency has only an indirect effect on attention, acting through recognized objects. Consequently, rather than treating attention as mere preprocessing step for object recognition, models of both need to be integrated.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,702
The VideoToolbox software for visual psychophysics: transforming numbers into movies
Pelli, D. G. (1997) · Spatial Vision · doi:10.1163/156856897x00366 · Cited by 11,098
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
IARBUS, A. L. (1967) · Medical Entomology and Zoology · Cited by 3,319
Yarbus, A. L. (1967) · doi:10.1007/978-1-4899-5379-7 · Cited by 3,304
A saliency-based search mechanism for overt and covert shifts of visual attention
Itti, L., & Koch, C. (2000) · Vision Research · doi:10.1016/s0042-6989(99)00163-7 · Cited by 3,156
Fei-Fei, L., Fergus, R., & Perona, P. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.383 · Cited by 2,502
Self-citationTo See or not to See: The Need for Attention to Perceive Changes in Scenes
Rensink, R. A., O'Regan, J. K., & Clark, J. J. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00427.x · Cited by 2,338
Temporary suppression of visual processing in an RSVP task: An attentional blink?
Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.18.3.849 · Cited by 2,260
Object class recognition by unsupervised scale-invariant learning
Fergus, R., Perona, P., & Zisserman, A. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211479 · Cited by 2,023
Self-citationRizzolatti, G., Riggio, L., Dascola, I., & Umiltá, C. (1987) · Neuropsychologia · doi:10.1016/0028-3932(87)90041-8 · Cited by 1,812
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667
Temporary suppression of visual processing in an RSVP task: an attentional blink?
Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.18.3.849 · Cited by 1,649
Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.766 · Cited by 1,639
Hochstein (2002) · doi:10.1016/s0896-6273(02)01091-7 · Cited by 1,397
Modeling the role of salience in the allocation of overt visual attention
Parkhurst, D., Law, K., & Niebur, E. (2002) · Vision Research · doi:10.1016/s0042-6989(01)00250-4 · Cited by 1,388
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224
Modeling visual attention via selective tuning
Tsotsos, J. K., Culhane, S. M., Kei Wai, W. Y., Lai, Y., Davis, N., & Nuflo, F. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(95)00025-9 · Cited by 1,143
The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox
Cornelissen, F. W., Peters, E. M., & Palmer, J. (2002) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195489 · Cited by 1,041
Tatler, B. W. (2007) · Journal of Vision · doi:10.1167/7.14.4 · Cited by 1,028
In what ways do eye movements contribute to everyday activities?
Land, M. F., & Hayhoe, M. (2001) · Vision Research · doi:10.1016/s0042-6989(01)00102-x · Cited by 960
Cognitive determinants of fixation location during picture viewing.
Loftus, G. R., & Mackworth, N. H. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.4.4.565 · Cited by 849
Rapid natural scene categorization in the near absence of attention
Li, F. F., VanRullen, R., Koch, C., & Perona, P. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.092277599 · Cited by 833
Self-citationSelective looking: Attending to visually specified events
Neisser, U., & Becklen, R. (1975) · Cognitive Psychology · doi:10.1016/0010-0285(75)90019-5 · Cited by 784
Levy-Schoen, A. (1968) · Neuropsychologia · doi:10.1016/0028-3932(68)90012-2 · Cited by 782
Framing pictures: The role of knowledge in automatized encoding and memory for gist.
Friedman, A. (1979) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.108.3.316 · Cited by 741
Components of bottom-up gaze allocation in natural images
Peters, R. J., Iyer, A., Itti, L., & Koch, C. (2005) · Vision Research · doi:10.1016/j.visres.2005.03.019 · Cited by 739
The effects of semantic consistency on eye movements during complex scene viewing.
Henderson, J. M., Weeks, P. A., Jr., & Hollingworth, A. (1999) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.25.1.210 · Cited by 713
Visual correlates of fixation selection: effects of scale and time
Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2005) · Vision Research · doi:10.1016/j.visres.2004.09.017 · Cited by 700
(1991) · Comparative Biochemistry and Physiology Part A: Physiology · doi:10.1016/0300-9629(91)90509-b · Cited by 669
Repetition blindness: Type recognition without token individuation
Kanwisher, N. G. (1987) · Cognition · doi:10.1016/0010-0277(87)90016-3 · Cited by 664
A saliency map in primary visual cortex
Li, Z. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01817-9 · Cited by 660
Parallel and Serial Neural Mechanisms for Visual Search in Macaque Area V4
Bichot, N. P., Rossi, A. F., & Desimone, R. (2005) · Science · doi:10.1126/science.1109676 · Cited by 650
Accurate visual memory for previously attended objects in natural scenes.
Hollingworth, A., & Henderson, J. M. (2002) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.28.1.113 · Cited by 607
Is bottom-up attention useful for object recognition?
Rutishauser, U., Walther, D., Koch, C., & Perona, P. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315142 · Cited by 592
Self-citationAttentional capture and inattentional blindness
Simons, D. J. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01455-8 · Cited by 564
Scene Consistency in Object and Background Perception
Davenport, J. L., & Potter, M. C. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00719.x · Cited by 552
INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM
Wallis, G., & Rolls, E. T. (1997) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00054-8 · Cited by 538
Algorithms for defining visual regions-of-interest: comparison with eye fixations
Privitera, C. M., & Stark, L. W. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.877520 · Cited by 535
McMahon, A. P., & Buswell, G. T. (1935) · Parnassus · doi:10.2307/771371 · Cited by 485
Predicting human gaze using low-level saliency combined with face detection
Cerf, M., Harel, J., Einhäuser, W., & Koch, C. (2008) · CaltechAUTHORS (California Institute of Technology) · Cited by 465
Eye Guidance in Reading and Scene Perception
(1998) · Elsevier eBooks · doi:10.1016/b978-0-08-043361-5.x5000-7 · Cited by 449
Natural scene statistics at the centre of gaze
Reinagel, P., & Zador, A. M. (1999) · Network Computation in Neural Systems · doi:10.1088/0954-898x/10/4/304 · Cited by 434
A visual salience map in the primate frontal eye field
Thompson, K. G., & Bichot, N. P. (2005) · Progress in Brain Research · doi:10.1016/s0079-6123(04)47019-8 · Cited by 431
Visual saliency does not account for eye movements during visual search in real-world scenes
Henderson, J. M., Brockmole, J. R., Castelhano, M. S., & Mack, M. (2007) · Eye Movements · doi:10.1016/b978-008044980-7/50027-6 · Cited by 429
Neural mechanisms of object recognition
Riesenhuber, M., & Poggio, T. (2002) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(02)00304-5 · Cited by 377
Mel, B. W. (1997) · Neural Computation · doi:10.1162/neco.1997.9.4.777 · Cited by 341
Search Goal Tunes Visual Features Optimally
Navalpakkam, V., & Itti, L. (2007) · Neuron · doi:10.1016/j.neuron.2007.01.018 · Cited by 331
Parallel processing in high-level categorization of natural images
Rousselet, G. A., Fabre-Thorpe, M., & Thorpe, S. J. (2002) · Nature Neuroscience · doi:10.1038/nn866 · Cited by 302
At first sight: A high-level pop out effect for faces
Hershler, O., & Hochstein, S. (2005) · Vision Research · doi:10.1016/j.visres.2004.12.021 · Cited by 296
Einhäuser, W., Rutishauser, U., & Koch, C. (2008) · Journal of Vision · doi:10.1167/8.2.2 · Cited by 296
Interesting objects are visually salient
Elazary, L., & Itti, L. (2008) · Journal of Vision · doi:10.1167/8.3.3 · Cited by 283
Perception of Objects in Natural Scenes: Is It Really Attention Free?
Evans, K. K., & Treisman, A. (2005) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.31.6.1476 · Cited by 281
How People Look at Pictures, A Study of the Psychology of Perception in Art.
Langford, R. C. (1936) · Journal of Educational Psychology · doi:10.1037/h0050502 · Cited by 275
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · American Journal of Ophthalmology · doi:10.1016/s0002-9394(14)72148-8 · Cited by 271
Goal-Related Activity in V4 during Free Viewing Visual Search
Mazer, J. A., & Gallant, J. L. (2003) · Neuron · doi:10.1016/s0896-6273(03)00764-5 · Cited by 262
Eye movements in iconic visual search
Rao, R. P. N., Zelinsky, G. J., Hayhoe, M. M., & Ballard, D. H. (2002) · Vision Research · doi:10.1016/s0042-6989(02)00040-8 · Cited by 253
Changes in Visual Receptive Fields with Microstimulation of Frontal Cortex
Armstrong, K. M., Fitzgerald, J. K., & Moore, T. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.010 · Cited by 241
Does luminance‐contrast contribute to a saliency map for overt visual attention?
Einhäuser, W., & König, P. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.02508.x · Cited by 238
Visual causes versus correlates of attentional selection in dynamic scenes
Carmi, R., & Itti, L. (2006) · Vision Research · doi:10.1016/j.visres.2006.08.019 · Cited by 227
Ruddock, K. H., Wooding, D. S., & Mannan, S. K. (1996) · Spatial Vision · doi:10.1163/156856896x00123 · Cited by 204
Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics
Zetzsche, C., Schill, K., Krieger, G., Hauske, G., & Rentschler, I. (2000) · Spatial Vision · doi:10.1163/156856800741216 · Cited by 189
Cognitive Processes in Eye Guidance
(2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198566816.001.0001 · Cited by 183
The functional visual field during picture viewing.
Nelson, W. W., & Loftus, G. R. (1980) · Journal of Experimental Psychology: Human Learning & Memory · doi:10.1037//0278-7393.6.4.391 · Cited by 174
Memory for real-world scenes: The role of consistency with schema expectation.
Pezdek, K., Whetstone, T., Reynolds, K. A., & Askari, N. (1989) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.15.4.587 · Cited by 171
On second glance: Still no high-level pop-out effect for faces
VanRullen, R. (2006) · Vision Research · doi:10.1016/j.visres.2005.07.009 · Cited by 162
Eye movements during scene inspection: A test of the saliency map hypothesis
Underwood, G., Foulsham, T., van Loon, E. M., Humphreys, L., & Bloyce, J. (2006) · The European Journal of Cognitive Psychology · doi:10.1080/09541440500236661 · Cited by 139
Accurate visual memory for previously attended objects in natural scenes.
Hollingworth, A., & Henderson, J. M. (2002) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.28.1.113 · Cited by 124
The iconic bottleneck and the tenuous link between early visual processing and perception
Nakayama, K. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511626197.038 · Cited by 120
Underwood, G., Templeman, E., Lamming, L., & Foulsham, T. (2007) · Consciousness and Cognition · doi:10.1016/j.concog.2006.11.008 · Cited by 118
Active Object Recognition Integrating Attention and Viewpoint Control
Dickinson, S., Christensen, H. I., Tsotsos, J. K., & Olofsson, G. (1997) · Computer Vision and Image Understanding · doi:10.1006/cviu.1997.0532 · Cited by 111
Fixation Patterns Made during Brief Examination of Two-Dimensional Images
Mannan, S. K., Ruddock, K. H., & Wooding, D. S. (1997) · Perception · doi:10.1068/p261059 · Cited by 109
To See or Not to See: The Need for Attention to Perceive Changes in Scenes
Rensink, R. A., O'Regan, J. K., & Clark, J. J. (2003) · The MIT Press eBooks · doi:10.7551/mitpress/2834.003.0022 · Cited by 105
Picture processing during recognition.
Parker, R. (1978) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.4.2.284 · Cited by 103
Visual Memory for Objects in Natural Scenes: From Fixations to Object Files
Tatler, B. W., Gilchrist, I. D., & Land, M. F. (2004) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/02724980443000430 · Cited by 100
Saccadic selectivity in complex visual search displays
Pomplun, M. (2006) · Vision Research · doi:10.1016/j.visres.2005.12.003 · Cited by 97
Eye movements and picture processing during recognition
Henderson, J. M., Williams, C. C., Castelhano, M. S., & Falk, R. J. (2003) · Perception & Psychophysics · doi:10.3758/bf03194809 · Cited by 90
With a careful look: Still no low-level confound to face pop-out
Hershler, O., & Hochstein, S. (2006) · Vision Research · doi:10.1016/j.visres.2006.03.023 · Cited by 80
Differences of monkey and human overt attention under natural conditions
Einhäuser, W., Kruse, W., Hoffmann, K., & Peter König, B. (2005) · Vision Research · doi:10.1016/j.visres.2005.08.032 · Cited by 76
Image Analysis and Computer Vision: 1995
Rosenfeld, A. (1996) · Computer Vision and Image Understanding · doi:10.1006/cviu.1996.0041 · Cited by 71
Prioritizing new objects for eye fixation in real-world scenes: Effects of object–scene consistency
Brockmole, J. R., & Henderson, J. M. (2008) · Visual Cognition · doi:10.1080/13506280701453623 · Cited by 69
Ogawa, T., & Komatsu, H. (2006) · Experimental Brain Research · doi:10.1007/s00221-006-0362-5 · Cited by 60
Fixations in natural scenes: Interaction of image structure and image content
Kayser, C., Nielsen, K. J., & Logothetis, N. K. (2006) · Vision Research · doi:10.1016/j.visres.2006.02.003 · Cited by 53
The duration of the attentional blink in natural scenes depends on stimulus category
Einhäuser, W., Koch, C., & Makeig, S. (2007) · Vision Research · doi:10.1016/j.visres.2006.12.007 · Cited by 53
Learning viewpoint invariant object representations using a temporal coherence principle
Einhäuser, W., Hipp, J. F., Eggert, J. P., Körner, E., & Peter König, B. (2005) · Biological Cybernetics · doi:10.1007/s00422-005-0585-8 · Cited by 47
A bottom–up model of spatial attention predicts human error patterns in rapid scene recognition
Einhäuser, W., Mundhenk, T. N., Baldi, P., Koch, C., & Itti, L. (2007) · Journal of Vision · doi:10.1167/7.10.6 · Cited by 35
Semantic effects on object selection in real-world scene perception
Graef, P. D. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198566816.003.0008 · Cited by 25
Prefixational Object Perception in Scenes
Graef, P. D. (1998) · Elsevier eBooks · doi:10.1016/b978-008043361-5/50015-8 · Cited by 22
Examining art: dissociating pattern and perceptual influences on oculomotor behaviour
Kaulard, K., Tatler, B. W., & Wade, N. (2007) · Spatial Vision · doi:10.1163/156856807782753903 · Cited by 16
On the discriminability of hROIs, human visually selected regions-of-interest
Privitera, C. M., Fujita, T., Chernyak, D. A., & STARK, L. W. (2005) · Biological Cybernetics · doi:10.1007/s00422-005-0586-7 · Cited by 10
Cited by
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Five factors that guide attention in visual search
Wolfe, J. M., & Horowitz, T. S. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0058 · Cited by 875
What Do Different Evaluation Metrics Tell Us About Saliency Models?
Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., & Durand, F. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2815601 · Cited by 802
Eye guidance in natural vision: reinterpreting salience
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011) · Journal of Vision · doi:10.1167/11.5.5 · Cited by 775
Jiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729
Metadata
- Authors
- Einhauser, W., Spain, M., & Perona, P.
- Venue
- Journal of Vision
- Published
- 2008-11-01
- Volume
- 8
- Issue
- 14
- Pages
- 18-18
- ISSN
- 1534-7362 (ISSN-L)
- OpenAlex
- W2073767010
- Cited by
- 451
- References
- 93
- FWCI
- 31.98
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.