CoChat
Sign Up
Cited by 221Multisensory perception and integrationNeurobiology of Language and BilingualismMultimodal Machine Learning Applications

When hearing the bark helps to identify the dog: semantically-congruent sounds modulate the identification of masked pictures

Chen, Yi-Chuan, Spence, Charles·2010·Cognition
DOI 10.1016/j.cognition.2009.10.012
Open this paper in CoChat — free account

Abstract

We report a series of experiments designed to assess the effect of audiovisual semantic congruency on the identification of visually-presented pictures. Participants made unspeeded identification responses concerning a series of briefly-presented, and then rapidly-masked, pictures. A naturalistic sound was sometimes presented together with the picture at a stimulus onset asynchrony (SOA) that varied between 0 and 533 ms (auditory lagging). The sound could be semantically congruent, semantically incongruent, or else neutral (white noise) with respect to the target picture. The results showed that when the onset of the picture and sound occurred simultaneously, a semantically-congruent sound improved, whereas a semantically-incongruent sound impaired, participants' picture identification performance, as compared to performance in the white-noise control condition. A significant facilitatory effect was also observed at SOAs of around 300 ms, whereas no such semantic congruency effects were observed at the longest interval (533 ms). These results therefore suggest that the neural representations associated with visual and auditory stimuli can interact in a shared semantic system. Furthermore, this crossmodal semantic interaction is not constrained by the need for the strict temporal coincidence of the constituent auditory and visual stimuli. We therefore suggest that audiovisual semantic interactions likely occur in a short-term buffer which rapidly accesses, and temporarily retains, the semantic representations of multisensory stimuli in order to form a coherent multisensory object representation. These results are explained in terms of Potter's (1993) notion of conceptual short-term memory.

Integrity signals

Registry

Crossref · 10.1016/j.cognition.2009.10.012

Cites

146 of 146 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Audiovisual priming studies with natural stimuli indicate a facilitatory role for visual perception.

    “Audiovisual priming studies with natural stimuli indicate a facilitatory role for visual perception [45].”

    From Decoding sound and imagery content in early visual cortex (2014) · cites this paper as [45]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Chen, Y.-C., & Spence, C.
Affiliations
University of Oxford (GB)
Corresponding author
Chen, Yi-Chuan
ORCID
Venue
Cognition
Publisher
Elsevier BV
Published
2010-03
Volume
114
Issue
3
Pages
389-404
ISSN
0010-0277 (ISSN-L)
DOI
10.1016/j.cognition.2009.10.012
OpenAlex
W2070012345
Semantic Scholar
0c2965bcb4e8a1b34531d57ba6297a8252def098
Cited by
221
References
146
Citations by year
2026: 10 · 2025: 17 · 2024: 19 · 2023: 10 · 2022: 14 · 2021: 16 · 2020: 14 · 2019: 16 · 2018: 13 · 2017: 15 · 2016: 11 · 2015: 13 · 2014: 14 · 2013: 17 · 2012: 6
FWCI
9.18
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Cognition (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
image recognition, stimulus onset asynchrony, multisensory integration
MeSH terms
Adult, Animals, Dogs, Female, Humans, Male, Perceptual Masking, Semantics, Vocalization, Animal, Signal Detection, Psychological

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review