Neural correlates of cross-modal binding
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Little is known about how the brain binds together signals from multiple sensory modalities to produce unified percepts of objects and events in the external world. Using event-related functional magnetic resonance imaging (fMRI) in humans, we measured transient brain responses to auditory/visual binding, as evidenced by a sound-induced change in visual motion perception. Identical auditory and visual stimuli were presented in all trials, but in some trials they were perceived to be bound together and in others they were perceived as unbound unimodal events. Cross-modal binding was associated with higher activity in multimodal areas, but lower activity in predominantly unimodal areas. This activation pattern suggests that a reciprocal and 'competitive' interaction between multimodal and unimodal areas underlies the perceptual interpretation of simultaneous signals from multiple sensory modalities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · cited 1×
“Studies of perceptual illusions such as the ventriloquist and McGurk effects (in which conflicting multisensory information is erroneously perceived to be bound together) suggest that cross-modal binding is a fast, pre-attentive process that occurs without the subject's awarenes”
BackgroundEvent-Related fMRI: Characterizing Differential Responses
Friston, K. J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M. D., & Turner, R. (1998) · NeuroImage · doi:10.1006/nimg.1997.0306 · Cited by 2,029 · cited 1×
“Data were analyzed using the general linear model (as implemented in SPM99) with a multiple regression design matrix including separate hemodynamic response functions and their temporal derivatives for each event (collide trials, pass trials and motor responses”
MethodWatson, J. D. G., Myers, R., Frackowiak, R. S. J., Hajnal, J. V., Woods, R. P., Mazziotta, J. C., Shipp, S., & Zeki, S. (1993) · Cerebral Cortex · doi:10.1093/cercor/3.2.79 · Cited by 1,234 · cited 1×
“As expected, we found significant activations in multimodal brain areas as well as auditory and visual areas, including a region in the occipitotemporal region previously identified as the 'human visual motion area”
BackgroundSound alters visual motion perception
Sekuler, R., Sekuler, A. B., & Lau, R. (1997) · Nature · doi:10.1038/385308a0 · Cited by 685
The mind's eye – precuneus activation in memory-related imagery
Fletcher, P. C., Frith, C. D., Baker, S. C., Shallice, T., Frackowiak, R. S. J., & Dolan, R. J. (1995) · NeuroImage · doi:10.1006/nimg.1995.1025 · Cited by 680 · cited 1×
“These were all in occipital and temporal areas, except for the precuneus—an area known to be active during visual imager”
BackgroundModulation of Human Visual Cortex by Crossmodal Spatial Attention
Macaluso, E., Frith, C. D., & Driver, J. (2000) · Science · doi:10.1126/science.289.5482.1206 · Cited by 613
Cross-modal and cross-temporal association in neurons of frontal cortex
Fuster, J. M., Bodner, M., & Kroger, J. K. (2000) · Nature · doi:10.1038/35012613 · Cited by 528
The cost of expecting events in the wrong sensory modality
Spence, C., Nicholls, M. E. R., & Driver, J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194473 · Cited by 467 · cited 1×
“Therefore, if the sound-induced change in visual perception shown in the current study was due to trial-to-trial attentional shifts to either modalit”
BackgroundCalvert, G. A., Hansen, P. C., Iversen, S. D., & Brammer, M. J. (2001) · NeuroImage · doi:10.1006/nimg.2001.0812 · Cited by 464
Event-related functional magnetic resonance imaging: modelling, inference and optimization
Josephs, O., & Henson, R. N. A. (1999) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1999.0475 · Cited by 425 · cited 1×
“Data were analyzed using the general linear model (as implemented in SPM99) with a multiple regression design matrix including separate hemodynamic response functions and their temporal derivatives for each event (collide trials, pass trials and motor responses”
Method
Cited by
Items 201–224 of 224
Tomaskovic, S. (2006) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 1
Temporal and Semantic Effects on Multisensory IntegrationPreprint
Vettel, J. M., Green, J. R., Heller, L. M., & Tarr, M. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05004 · Cited by 2
“Similar regions have been found in numerous studies using stimuli as diverse as vertical bars and beeps (Bushara et al., 2003), animals and novel objects (Hein and Knight, 2008), and articulating mouths (Miller and D'Esposito, 2005;Ojanen et al.”
BackgroundResultAudiovisual Processing and Integration in Amblyopia
Richards, M. (2018) · TSpace (University of Toronto) · Cited by 1
van de Ven, V. G., Kleuters, G., & Stuiver, J. (2021) · doi:10.31234/osf.io/xnqh9 · Cited by 1
Alonso, I. (2015) · theses.fr (ABES) · doi:10.70675/95112d1cz00bcz4123zb13bz371abb4613ee · Cited by 3
“For instance, the unified representation of faces and voices (Campanella and Belin, 2007), locations of different sounds (Maybery et al., 2009), moving objects and their sound (Bushara et al., 2003; Schmitz et al., 2013), or the illusory conjunction produced by ventriloquism (Bischoff et al., 2007).”
BackgroundCampanella, S., & Joassin, F. (2012) · Springer eBooks · doi:10.1007/978-1-4614-3585-3_8
Interactions of Attention, Stimulus Conflict, and Multisensory Processing
Donohue, S. E. (2012) · DukeSpace (Duke University)
Theta Frequency Background Tunes Transmission but Not Summation of Spiking Responses
Parameshwaran, D., & Bhalla, U. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0055607
Crossmodal binding in localizing objects outside the field of view
Piemo, A. C., Carìa, A., & Castiello, U. (2006) · Visual Cognition · doi:10.1080/13506280544000273
“These studies have provided evidence for neural and behavioural mechanisms in the processing of multisensory information (Bushara et al., 2003; Driver, 1996; Stein & Meredith, 1993).”
BackgroundFunctional implications of cortical damage
Rolheiser, T. M. (2008) · Scholars' Bank (University of Oregon)
Auditory processing in the brainstem and audiovisual integration in humans studied with fMRI
Slabu, L. (2008) · University of Groningen research database (University of Groningen / Centre for Information Technology)
Cannon, R. L. (2009) · Cited by 5
Jones, R. (2003) · Nature reviews. Neuroscience · doi:10.1038/nrn1039
Yuasa, M., Watanabe, N., & Toyomura, A. (2017) · IEEJ Transactions on Electronics Information and Systems · doi:10.1541/ieejeiss.137.724
The Posture of Putting One’s Palms Together Modulates Visual Motion Event Perception
Saito, G., & Gyoba, J. (2017) · Perception · doi:10.1177/0301006617745783
“For example, by means of functional magnetic resonance imaging, Bushara et al. (2003) found that the perception of bouncing produced by the interaction between a click sound and a visual motion stimulus was associated with increased activity in multisensory areas, whereas the perception of…”
BackgroundInfluentialBurke, K. M., Molholm, S., Butler, J. S., Ross, L. A., & Foxe, J. J. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/581777
Jenkins, D. (2019) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.17136149.v1
Neural correlates of temporal updating and reasoning in association with neuropsychiatric disorders
Ueda, N., & Hanakawa, T. (2019) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19000426
Self-citationRegional and Cross-Network Brain Dynamics in Acute and Chronic Pain
Rogachov, A. (2019) · TSpace (University of Toronto)
Watanabe, T., & Shibata, K. (2012) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.13435956.14810054
Farahani, N., Ghahari, S., Fatemizadeh, E., & Nasrabadi, A. M. (2022) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2022.2077235 · Cited by 1
Mise à jour en mémoire de travail et vieillissement normal de la mémoire épisodique
Boujut, A. (2016) · theses.fr (ABES) · doi:10.70675/a4f31e42zf86az4f4cz8f78z6216edb83e85 · Cited by 2
Metadata
- Authors
- Bushara, K. O., Hanakawa, T., Immisch, I., Toma, K., Kansaku, K., & Hallett, M.
- Affiliations
- Minneapolis VA Health Care System (US)
- National Institutes of Health (US)
- Corresponding author
- Bushara, Khalafalla O.
- Venue
- Nature Neuroscience
- Published
- 2002-12-23
- Volume
- 6
- Issue
- 2
- Pages
- 190-195
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2089488430
- Cited by
- 226
- References
- 16
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 4 · 2023: 6 · 2022: 10 · 2021: 4 · 2020: 11 · 2019: 12 · 2018: 10 · 2017: 12 · 2016: 10 · 2015: 7 · 2014: 6 · 2013: 12 · 2012: 11
- FWCI
- 18.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- cross-modal binding, multisensory integration, neural correlates, perceptual binding, event-related fMRI
- MeSH terms
- Acoustic Stimulation, Auditory Perception, Brain, Brain Mapping, Female, Humans, Functional Laterality, Magnetic Resonance Imaging, Male, Motion Perception, Nerve Net, Neural Pathways, Neurons, Neuropsychological Tests, Photic Stimulation, Psychomotor Performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.