The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
How saliency, faces, and sound influence gaze in dynamic social scenes
Open accessAbstract
Conversation scenes are a typical example in which classical models of visual attention dramatically fail to predict eye positions. Indeed, these models rarely consider faces as particular gaze attractors and never take into account the important auditory information that always accompanies dynamic social scenes. We recorded the eye movements of participants viewing dynamic conversations taking place in various contexts. Conversations were seen either with their original soundtracks or with unrelated soundtracks (unrelated speech and abrupt or continuous natural sounds). First, we analyze how auditory conditions influence the eye movement parameters of participants. Then, we model the probability distribution of eye positions across each video frame with a statistical method (Expectation-Maximization), allowing the relative contribution of different visual features such as static low-level visual saliency (based on luminance contrast), dynamic low level visual saliency (based on motion amplitude), faces, and center bias to be quantified. Through experimental and modeling results, we show that regardless of the auditory condition, participants look more at faces, and especially at talking faces. Hearing the original soundtrack makes participants follow the speech turn-taking more closely. However, we do not find any difference between the different types of unrelated soundtracks. These eyetracking results are confirmed by our model that shows that faces, and particularly talking faces, are the features that best explain the gazes recorded, especially in the original soundtrack condition. Low-level saliency is not a relevant feature to explain eye positions made on social scenes, even dynamic ones. Finally, we propose groundwork for an audiovisual saliency model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738 · cited 1×
“…& Hochstein, 2005) and pre-attentive (Bindemann, Burton, Langton, Schweinberger, & Doherty, 2007; Crouzet, Kirchner, & Thorpe, 2010), and the brain structures specifically involved in face perception have been pointed out (Haxby, Hoffman, & Gobbini, 2000; Kanwisher, McDermott, & Chun, 1997).”
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Yarbus, A. L. (1967) · doi:10.1007/978-1-4899-5379-7 · Cited by 3,304
The Handbook of Multisensory Processes
(2004) · doi:10.7551/mitpress/3422.001.0001 · Cited by 948
Eye guidance in natural vision: reinterpreting salience
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011) · Journal of Vision · doi:10.1167/11.5.5 · Cited by 775 · cited 2×
“However, these models cannot be generalized to many experimental contexts, since the dynamic and social nature of visual perception are not taken into account (Tatler, Hayhoe, Land, & Ballard, 2011).”
Cited by
Perceptual image quality assessment: a survey
Zhai, G., & Min, X. (2020) · Science China Information Sciences · doi:10.1007/s11432-019-2757-1 · Cited by 655
A Multimodal Saliency Model for Videos With High Audio-Visual Correspondence
Min, X., Zhai, G., Zhou, J., Zhang, X.-P., Yang, X., & Guan, X. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2966082 · Cited by 295
Face perception influences the programming of eye movements.
Kauffmann, L., Peyrin, C., Chauvin, A., Entzmann, L., Breuil, C., & Guyader, N. (2019) · Scientific Reports · doi:10.1038/s41598-018-36510-0 · Cited by 28
Self-citation
Metadata
- Authors
- Coutrot, A., & Guyader, N.
- Venue
- Journal of Vision
- Published
- 2014-07-03
- Volume
- 14
- Issue
- 8
- Pages
- 5-5
- ISSN
- 1534-7362 (ISSN-L)
- OpenAlex
- W2155479901
- Cited by
- 158
- References
- 5
- FWCI
- 12.17
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.