Task and context determine where you look
Open accessAbstract
The deployment of human gaze has been almost exclusively studied independent of any specific ongoing task and limited to two-dimensional picture viewing. This contrasts with its use in everyday life, which mostly consists of purposeful tasks where gaze is crucially involved. To better understand deployment of gaze under such circumstances, we devised a series of experiments, in which subjects navigated along a walkway in a virtual environment and executed combinations of approach and avoidance tasks. The position of the body and the gaze were monitored during the execution of the task combinations and dependence of gaze on the ongoing tasks as well as the visual features of the scene was analyzed. Gaze distributions were compared to a random gaze allocation strategy as well as a specific "saliency model." Gaze distributions showed high similarity across subjects. Moreover, the precise fixation locations on the objects depended on the ongoing task to the point that the specific tasks could be predicted from the subject's fixation data. By contrast, gaze allocation according to a random or a saliency model did not predict the executed fixations or the observed dependence of fixation locations on the specific task.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The value of an eye movement lies in reducing uncertainty and increasing the expected reward (probability of success) of a future action.
“An alternative explanation however emerges from studies of eye movements in natural behaviors, which suggests that the value of an eye movement lies in reducing uncertainty and increasing the expected reward (probability of success) of a future action (Ballard and Hayhoe, 2009; Hayhoe et al., 2012; Rothkopf et al., 2007; Tatler et al., 2011).”
From Attention, learning, and the value of information (2012) · cites this paper as Rothkopf et al., 2007
A model-free prediction error mechanism can be prohibitively slow in complex tasks and may erroneously assign credit to irrelevant steps.
“Such a mechanism can be prohibitively slow in complex tasks, and may erroneously assign credit to irrelevant steps (e.g., value the earlier decision to study for a final exam less than the more recent decision to wear a white shirt) (Rothkopf et al., 2007).”
From Attention, learning, and the value of information (2012) · cites this paper as Rothkopf et al., 2007
Cites
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Christof Koch, A., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,289 · see citing works
Marr, D. (1982) · Cumulative Index of Computer Aided Architectural Design · Cited by 5,733 · see citing works
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Christof Koch, A., & Ullman, S. (1987) · Springer eBooks · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,739 · see citing works
IARBUS, A. L. (1967) · Medical Entomology and Zoology · Cited by 3,319 · see citing works
A saliency-based search mechanism for overt and covert shifts of visual attention
Itti, L., & Koch, C. (2000) · Vision Research · doi:10.1016/s0042-6989(99)00163-7 · Cited by 3,137 · see citing works
A sensorimotor account of vision and visual consciousness
O'Regan, J. K., & Noë, A. (2001) · Behavioral and Brain Sciences · doi:10.1017/s0140525x01000115 · Cited by 3,126 · see citing works
Components of visual orienting
Posner, M. I. (1984) · Attention and performance · Cited by 2,980 · see citing works
Integration of Visual and Linguistic Information in Spoken Language Comprehension
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995) · Science · doi:10.1126/science.7777863 · Cited by 2,929 · see citing works
Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention
Chun, M. M., & Jiang, Y. (1998) · Cognitive Psychology · doi:10.1006/cogp.1998.0681 · Cited by 1,959 · see citing works
Torralba, A. B., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.766 · Cited by 1,646 · see citing works
Human gaze control during real-world scene perception
Henderson, J. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.09.006 · Cited by 1,409 · see citing works
Eye movements in natural behavior
Hayhoe, M., & Ballard, D. H. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.02.009 · Cited by 1,321 · see citing works
Self-citationMultiple reward signals in the brain
Schultz, W. (2000) · Nature Reviews Neuroscience · doi:10.1038/35044563 · Cited by 1,303 · see citing works
The role of attention in the programming of saccades
Kowler, E., Anderson, E., Dosher, B., & Blaser, E. (1995) · Vision Research · doi:10.1016/0042-6989(94)00279-u · Cited by 1,302 · see citing works
Change blindness: Past, present, and future
Simons, D. J., & Rensink, R. A. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.11.006 · Cited by 1,153 · see citing works
The Roles of Vision and Eye Movements in the Control of Activities of Daily Living
Land, M. F., Mennie, N. R., & Rusted, J. M. (1999) · Perception · doi:10.1068/p2935 · Cited by 1,096 · see citing works
Deictic codes for the embodiment of cognition
Ballard, D. H., Hayhoe, M. M., Pook, P. K., & Rao, R. P. N. (1997) · Behavioral and Brain Sciences · doi:10.1017/s0140525x97001611 · Cited by 1,082 · see citing works
Self-citationLand, M. F., & Lee, D. N. (1994) · Nature · doi:10.1038/369742a0 · Cited by 1,031 · see citing works
Ballard, D. H. (1991) · Artificial Intelligence · doi:10.1016/0004-3702(91)90080-4 · Cited by 1,022 · see citing works
Self-citationChalupa, L. M., Werner, J. S., & Barnstable, C. J. (2004) · Cited by 1,020 · see citing works
Henderson, J. M., & Hollingworth, A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.243 · Cited by 971 · see citing works
In what ways do eye movements contribute to everyday activities?
Land, M. F., & Hayhoe, M. (2001) · Vision Research · doi:10.1016/s0042-6989(01)00102-x · Cited by 966 · see citing works
Self-citationEye–Hand Coordination in Object Manipulation
Johansson, R. S., Westling, G., Bäckström, A., & Flanagan, J. R. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-17-06917.2001 · Cited by 873 · see citing works
Saccadic eye movements and cognition
Liversedge, S., & Findlay, J. M. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01418-7 · Cited by 818 · see citing works
Memory Representations in Natural Tasks
Ballard, D. H., Hayhoe, M. M., & Pelz, J. B. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.1.66 · Cited by 817 · see citing works
Self-citationOptic flow is used to control human walking
Warren, W. H., Kay, B. A., Zosh, W. D., Duchon, A. P., & Sahuc, S. (2001) · Nature Neuroscience · doi:10.1038/84054 · Cited by 749 · see citing works
Active vision: the psychology of looking and seeing
(2004) · Choice Reviews Online · doi:10.5860/choice.41-4646 · Cited by 731 · see citing works
From eye movements to actions: how batsmen hit the ball
Land, M. F., & McLeod, P. (2000) · Nature Neuroscience · doi:10.1038/81887 · Cited by 725 · see citing works
Action plans used in action observation
Flanagan, J. R., & Johansson, R. S. (2003) · Nature · doi:10.1038/nature01861 · Cited by 716 · see citing works
Visual correlates of fixation selection: effects of scale and time
Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2004) · Vision Research · doi:10.1016/j.visres.2004.09.017 · Cited by 700 · see citing works
Modeling the influence of task on attention
Navalpakkam, V., & Itti, L. (2004) · Vision Research · doi:10.1016/j.visres.2004.07.042 · Cited by 628 · see citing works
Visual memory and motor planning in a natural task
Hayhoe, M., Shrivastava, A., Mruczek, R. E. B., & Pelz, J. B. (2003) · Journal of Vision · doi:10.1167/3.1.6 · Cited by 585 · see citing works
Self-citationTop-down control of visual attention in object detection
Oliva, A., Torralba, A. B., Castelhano, M. S., & Henderson, J. M. (2004) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2003.1246946 · Cited by 474 · see citing works
Perceptual learning and top-down influences in primary visual cortex
Li, W., Piëch, V., & Gilbert, C. D. (2004) · Nature Neuroscience · doi:10.1038/nn1255 · Cited by 474 · see citing works
Diagnostic Colors Mediate Scene Recognition
Oliva, A., & Schyns, P. G. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0728 · Cited by 429 · see citing works
Natural scene statistics at the centre of gaze
Reinagel, P., & Zador, A. M. (1999) · Network Computation in Neural Systems · doi:10.1088/0954-898x_10_4_304 · Cited by 426 · see citing works
How Close Are We to Understanding V1?
Olshausen, B. A., & Field, D. J. (2005) · Neural Computation · doi:10.1162/0899766054026639 · Cited by 411 · see citing works
Involuntary attentional capture by abrupt onsets
Remington, R. W., Johnston, J. C., & Yantis, S. (1992) · Perception & Psychophysics · doi:10.3758/bf03212254 · Cited by 409 · see citing works
Visual motion and attentional capture
Hillstrom, A. P., & Yantis, S. (1994) · Perception & Psychophysics · doi:10.3758/bf03205298 · Cited by 382 · see citing works
Triesch, J., Ballard, D. H., Hayhoe, M., & Sullivan, B. T. (2003) · Journal of Vision · doi:10.1167/3.1.9 · Cited by 337 · see citing works
Self-citationQuantifying the contribution of low-level saliency to human eye movements in dynamic scenes
Itti, L. (2005) · Visual Cognition · doi:10.1080/13506280444000661 · Cited by 333 · see citing works
Scene content selected by active vision
Parkhurst, D. J., & Niebur, E. (2003) · Spatial Vision · doi:10.1163/15685680360511645 · Cited by 318 · see citing works
Vision Using Routines: A Functional Account of Vision
Hayhoe, M. (2000) · Visual Cognition · doi:10.1080/135062800394676 · Cited by 305 · see citing works
Self-citationT HE N EUROBIOLOGY OF V ISUAL -S ACCADIC D ECISION M AKING
Glimcher, P. W. (2003) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.010302.081134 · Cited by 291 · see citing works
The contributions of color to recognition memory for natural scenes.
Wichmann, F. A., Sharpe, L. T., & Gegenfurtner, K. R. (2002) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.28.3.509 · Cited by 288 · see citing works
Models of bottom-up and top-down visual attention
Koch, C., & Itti, L. (2000) · doi:10.7907/md7v-ne41 · Cited by 261 · see citing works
Does luminance‐contrast contribute to a saliency map for overt visual attention?
Einhäuser, W., & Peter König, B. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.02508.x · Cited by 235 · see citing works
Statistical decision theory and trade-offs in the control of motor response
Maloney, L. T., Landy, M. S., & Trommershäuser, J. (2003) · Spatial Vision · doi:10.1163/156856803322467527 · Cited by 225 · see citing works
How People Look At Pictures: A Study Of The Psychology Of Perception In Art
Buswell, G. T. (2012) · Cited by 210 · see citing works
Ruddock, K. H., Wooding, D. S., & Mannan, S. (1996) · Spatial Vision · doi:10.1163/156856896x00123 · Cited by 207 · see citing works
What controls attention in natural environments?
Shinoda, H., Hayhoe, M., & Shrivastava, A. (2001) · Vision Research · doi:10.1016/s0042-6989(01)00199-7 · Cited by 194 · see citing works
Self-citationObject and scene analysis by saccadic eye-movements: an investigation with higher-order statistics
Zetzsche, C., Schill, K., Krieger, G., Hauske, G., & Rentschler, I. (2000) · Spatial Vision · doi:10.1163/156856800741216 · Cited by 190 · see citing works
Baddeley, R. J., & Tatler, B. W. (2006) · Vision Research · doi:10.1016/j.visres.2006.02.024 · Cited by 159 · see citing works
Optic-flow and egocentric-direction strategies in walking: central vs peripheral visual field
Turano, K. A., Yu, D., Hao, L., & Hicks, J. C. (2005) · Vision Research · doi:10.1016/j.visres.2005.06.017 · Cited by 129 · see citing works
Task Demands Control Acquisition and Storage of Visual Information.
Droll, J. A., Hayhoe, M., Triesch, J., & Sullivan, B. T. (2005) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.31.6.1416 · Cited by 127 · see citing works
Self-citationProcessing of complex stimuli and natural scenes in the visual cortex
Kayser, C., Kording, K. P., & Peter König, B. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.06.002 · Cited by 127 · see citing works
Current oculomotor research : physiological and psychological aspects
Becker, W. S., Deubel, H., & Mergner, T. (1999) · Cited by 124 · see citing works
Coordination of hand movements and saccades: evidence for a common and a separate pathway
Frens, M. A., & Erkelens, C. J. (1991) · Experimental Brain Research · doi:10.1007/bf00231754 · Cited by 106 · see citing works
Eye Movements for Reward Maximization
Sprague, N. R., & Ballard, D. H. (2003) · Cited by 106 · see citing works
Self-citationAdvances in Relating Eye Movements and Cognition
Hayhoe, M. (2004) · Infancy · doi:10.1207/s15327078in0602_7 · Cited by 96 · see citing works
Self-citationSaccadic localization of eccentric forms
He, P., & Kowler, E. (1991) · Journal of the Optical Society of America A · doi:10.1364/josaa.8.000440 · Cited by 70 · see citing works
Perceiving visual expansion without optic flow
Schrater, P. R., Knill, D. C., & Simoncelli, E. P. (2001) · Nature · doi:10.1038/35071075 · Cited by 56 · see citing works
A probabilistic model of eye movements in concept formation
Nelson, J. D., & Cottrell, G. W. (2007) · Neurocomputing · doi:10.1016/j.neucom.2006.02.026 · Cited by 45 · see citing works
Portable eyetracking: a study of natural eye movements
Pelz, J. B., Canosa, R. L., Kucharczyk, D., Babcock, J. S., Silver, A., & Konno, D. (2000) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.387190 · Cited by 44 · see citing works
Land, M. F. (2003) · The MIT Press eBooks · doi:10.7551/mitpress/7131.003.0104 · Cited by 37 · see citing works
Oculomotor behavior in natural and man-made environments
Pelz, J. B., & Rothkopf, C. A. (2007) · Elsevier eBooks · doi:10.1016/b978-008044980-7/50033-1 · Cited by 35 · see citing works
Self-citationMiller, T. L. (2008) · Encyclopedia of Special Education · doi:10.1002/9780470373699.speced0807 · Cited by 24 · see citing works
Proceedings of SPIE: Human Vision and Electronic Imaging XVIII
Rogowlitz, B. E., & Pappas, T. N. (2009) · Cited by 14 · see citing works
Object Recognition and Goal-Directed Eye or Hand Movements are Coupled by Visual Attention
Paprotta, I., Deubel, H., & Schneider, W. X. (1999) · Springer eBooks · doi:10.1007/978-1-4757-3054-8_34 · Cited by 12 · see citing works
Deubel, H., Shimojo, S., & Paprotta, I. (1997) · Perception · doi:10.1068/v970060 · Cited by 2 · see citing works
Suprathreshold Motion Sensitivity and Ocular Dominance
Arnold-Schultz-Gahmen, B. E., Ehrenstein, W. H., & Schlykowa, L. (1997) · Perception · doi:10.1068/v970093 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Eye guidance in natural vision: reinterpreting salience
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011) · Journal of Vision · doi:10.1167/11.5.5 · Cited by 775 · see citing works
Self-citationLearning to Attend and to Ignore Is a Matter of Gains and Losses
Della Libera, C., & Chelazzi, L. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02360.x · Cited by 329 · see citing works
Basal Ganglia Circuits for Reward Value–Guided Behavior
Hikosaka, O., Kim, H. F., Yasuda, M., & Yamamoto, S. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-013924 · Cited by 259 · see citing works
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241 · see citing works
“Such a mechanism can be prohibitively slow in complex tasks, and may erroneously assign credit to irrelevant steps (Rothkopf et al., 2007).”
BackgroundAction perception as hypothesis testing
Donnarumma, F., Costantini, M., Ambrosini, E., Friston, K., & Pezzulo, G. (2017) · Cortex · doi:10.1016/j.cortex.2017.01.016 · Cited by 90 · see citing works
Get real! Resolving the debate about equivalent social stimuli
Birmingham, E., Bischof, W. F., & Kingstone, A. (2009) · Visual Cognition · doi:10.1080/13506280902758044 · Cited by 81 · see citing works
Tasks and their role in visual neuroscience
Kay, K., Bonnen, K., Denison, R. N., Arcaro, M. J., & Barack, D. L. (2023) · Neuron · doi:10.1016/j.neuron.2023.03.022 · Cited by 61 · see citing works
- Authors
- Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. M.
- Affiliations
- University of Rochester (US)
- The University of Texas at Austin (US)
- Venue
- Journal of Vision
- Published
- 2016-07-27
- Volume
- 7
- Issue
- 14
- Pages
- 16
- ISSN
- 1534-7362 (ISSN-L)
- OpenAlex
- W2139734816
- Cited by
- 366
- References
- 71
- Citations by year
- 2026: 10 · 2025: 11 · 2024: 11 · 2023: 12 · 2022: 12 · 2021: 28 · 2020: 20 · 2019: 11 · 2018: 29 · 2017: 24 · 2016: 31 · 2015: 19 · 2014: 32 · 2013: 19 · 2012: 23
- FWCI
- 35.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Journal of Vision (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Eye Institute (EY05729)
- National Center for Research Resources (RR-09283)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- visual attention, virtual navigation, fixation location, saliency models, gaze allocation
- MeSH terms
- Eye Movements, Fixation, Ocular, Humans, Models, Psychological, Psychomotor Performance, Task Performance and Analysis, User-Computer Interface, Walking, Eye Movement Measurements
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
