The DOI matches its Crossref record; 87 of the 88 references it cites were found in a registry or an index.
Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study
Abstract
The aim of this study was (1) to provide behavioral evidence for multimodal feature integration in an object recognition task in humans and (2) to characterize the processing stages and the neural structures where multisensory interactions take place. Event-related potentials (ERPs) were recorded from 30 scalp electrodes while subjects performed a forced-choice reaction-time categorization task: At each trial, the subjects had to indicate which of two objects was presented by pressing one of two keys. The two objects were defined by auditory features alone, visual features alone, or the combination of auditory and visual features. Subjects were more accurate and rapid at identifying multimodal than unimodal objects. Spatiotemporal analysis of ERPs and scalp current densities revealed several auditory-visual interaction components temporally, spatially, and functionally distinct before 200 msec poststimulus. The effects observed were (1) in visual areas, new neural activities (as early as 40 msec poststimulus) and modulation (amplitude decrease) of the N185 wave to unimodal visual stimulus, (2) in the auditory cortex, modulation (amplitude increase) of subcomponents of the unimodal auditory N1 wave around 90 to 110 msec, and (3) new neural activity over the right fronto-temporal area (140 to 165 msec). Furthermore, when the subjects were separated into two groups according to their dominant modality to perform the task in unimodal conditions (shortest reaction time criteria), the integration effects were found to be similar for the two groups over the nonspecific fronto-temporal areas, but they clearly differed in the sensory-specific cortices, affecting predominantly the sensory areas of the nondominant modality. Taken together, the results indicate that multisensory integration is mediated by flexible, highly adaptive physiological processes that can take place very early in the sensory processing chain and operate in both sensory-specific and nonspecific cortical structures in different ways.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Multimodal stimuli are typically perceived more precisely and faster than unimodal stimuli (Doyle & Snowden, 2001;Forster, Cavina-Pratesi, Aglioti, & Berlucchi, 2002;Fort, Delpuech, Pernier, & Giard, 2002;G i a r d&P e r o n n e t , 1999).”
From Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review (2013) · cites this paper as Giard & Peronnet, 1999
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Ungerleider, L. G. (1982) · Medical Entomology and Zoology · Cited by 4,531
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
Visual Feature Integration and the Temporal Correlation Hypothesis
Singer, W., & Gray, C. M. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.003011 · Cited by 3,123
Spherical splines for scalp potential and current density mapping
Perrin, F., Pernier, J., Bertrand, O., & Echallier, J. F. (1989) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(89)90180-6 · Cited by 2,706
Society for Neuroscience 2007 abstracts
Wishart, T. M., Paterson, J. M., Short, D. M., Meredith, S., Robertson, K., Sutherland, C. D., Cousin, M. A., Dutia, M. B., & Gillingwater, T. H. (2007) · Cited by 2,094
(1994) · Choice Reviews Online · doi:10.5860/choice.31-4365 · Cited by 1,882
AN ANATOMICAL STUDY OF CONVERGING SENSORY PATHWAYS WITHIN THE CEREBRAL CORTEX OF THE MONKEY
JONES, E. G., & POWELL, T. P. S. (1970) · Brain · doi:10.1093/brain/93.4.793 · Cited by 1,711
Journal of the Washington Academy of Sciences.
(1918) · The Journal of Geology · doi:10.1086/622567 · Cited by 1,707
Logothetis, N. K., & Sheinberg, D. L. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.003045 · Cited by 1,351
Divided attention: Evidence for coactivation with redundant signals
Miller, J. (1982) · Cognitive Psychology · doi:10.1016/0010-0285(82)90010-x · Cited by 1,213
Meredith, M. A., & Stein, B. E. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.3.640 · Cited by 1,183
Treisman, A. (1996) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(96)80070-5 · Cited by 1,148
Visual dominance: An information-processing account of its origins and significance.
Posner, M. I., Nissen, M. J., & Klein, R. M. (1976) · Psychological Review · doi:10.1037//0033-295x.83.2.157 · Cited by 1,097
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Combined spatial and temporal imaging of brain activity during visual selective attention in humans
Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Münte, T. F., Gös, A., Scherg, M., Johannes, S., Hundeshagen, H., et al. (1994) · Nature · doi:10.1038/372543a0 · Cited by 914
Interactions Among Converging Sensory Inputs in the Superior Colliculus
Meredith, M. A., & Stein, B. E. (1983) · Science · doi:10.1126/science.6867718 · Cited by 841
Neural mechanisms of visual selective attention
MANGUN, G. R. (1995) · Psychophysiology · doi:10.1111/j.1469-8986.1995.tb03400.x · Cited by 839
Evoked dipole source potentials of the human auditory cortex
Scherg, M., & Von Cramon, D. (1986) · Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section · doi:10.1016/0168-5597(86)90014-6 · Cited by 707
The sources of auditory evoked responses recorded from the human scalp
Vaughan, H. G., & Ritter, W. (1970) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(70)90228-2 · Cited by 693
Sound alters visual motion perception
Sekuler, R., Sekuler, A. B., & Lau, R. (1997) · Nature · doi:10.1038/385308a0 · Cited by 685
Clark, V. P., & Hillyard, S. A. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.5.387 · Cited by 581
Identification of early visual evoked potential generators by retinotopic and topographic analyses
Clark, V. P., Fan, S., & Hillyard, S. A. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020306 · Cited by 560
Luck, S. J., Heinze, H. J., Mangun, G. R., & Hillyard, S. A. (1990) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(90)90139-b · Cited by 550
Audiovisual links in exogenous covert spatial orienting
Spence, C., & Driver, J. (1997) · Perception & Psychophysics · doi:10.3758/bf03206843 · Cited by 475
Multisensory perception: Beyond modularity and convergence
Driver, J., & Spence, C. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00740-5 · Cited by 465
Receptive fields in deep layers of cat superior colliculus.
Gordon, B. (1973) · Journal of Neurophysiology · doi:10.1152/jn.1973.36.2.157 · Cited by 448
Reaction time as a measure of intersensory facilitation.
Hershenson, M. (1962) · Journal of Experimental Psychology · doi:10.1037/h0039516 · Cited by 418
Representation and integration of multiple sensory inputs in primate superior colliculus
Wallace, M. T., Wilkinson, L. K., & Stein, B. E. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.2.1246 · Cited by 412
Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis
Stein, B. E., London, N., Wilkinson, L. K., & Price, D. D. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.6.497 · Cited by 404
Foxe, J. J., Morocz, I. A., Murray, M. M., Higgins, B. A., Javitt, D. C., & Schroeder, C. E. (2000) · Cognitive Brain Research · doi:10.1016/s0926-6410(00)00024-0 · Cited by 395
Stein, B. E., Meredith, M. A., Huneycutt, W. S., & McDade, L. (1989) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1989.1.1.12 · Cited by 394
Spatial factors determine the activity of multisensory neurons in cat superior colliculus
Alex Meredith, M., & Stein, B. E. (1986) · Brain Research · doi:10.1016/0006-8993(86)91648-3 · Cited by 388
Sensory conflict in judgments of spatial direction
Pick, H. L., Warren, D. H., & Hay, J. C. (1969) · Perception & Psychophysics · doi:10.3758/bf03207017 · Cited by 386
Cross-modal bias and perceptual fusion with auditory-visual spatial discordance
Bertelson, P., & Radeau, M. (1981) · Perception & Psychophysics · doi:10.3758/bf03207374 · Cited by 371
Seeing speech: visual information from lip movements modifies activity in the human auditory cortex
Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991) · Neuroscience Letters · doi:10.1016/0304-3940(91)90914-f · Cited by 367
Timecourse of coactivation in bimodal divided attention
Miller, J. (1986) · Perception & Psychophysics · doi:10.3758/bf03203025 · Cited by 366
On the detection of auditory deviations: A pre‐attentive activation model
SCHRÖGER, E. (1997) · Psychophysiology · doi:10.1111/j.1469-8986.1997.tb02395.x · Cited by 356
King, A. J., & Palmer, A. R. (1985) · Experimental Brain Research · doi:10.1007/bf00236934 · Cited by 353
Oram, M. W., & Perrett, D. I. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.1.109 · Cited by 345
Auditory receptive fields in primate superior colliculus shift with changes in eye position
Jay, M. F., & Sparks, D. L. (1984) · Nature · doi:10.1038/309345a0 · Cited by 340
Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements
Frens, M. A., Van Opstal, A. J., & Van Der Willigen, R. F. (1995) · Perception & Psychophysics · doi:10.3758/bf03206796 · Cited by 326
Anllo-Vento, L., & Hillyard, S. A. (1996) · Perception & Psychophysics · doi:10.3758/bf03211875 · Cited by 322
A temporal component of the auditory evoked response
Wolpaw, J. R., & Penry, J. K. (1975) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(75)90073-5 · Cited by 305
Hikosaka, K., Iwai, E., Saito, H., & Tanaka, K. (1988) · Journal of Neurophysiology · doi:10.1152/jn.1988.60.5.1615 · Cited by 296
Benevento, L. A., Fallon, J., Davis, B. J., & Rezak, M. (1977) · Experimental Neurology · doi:10.1016/0014-4886(77)90112-1 · Cited by 288
Audiovisual Integration of Letters in the Human Brain
Raij, T., Uutela, K., & Hari, R. (2000) · Neuron · doi:10.1016/s0896-6273(00)00138-0 · Cited by 282
Wallace, M. T., Meredith, M. A., & Stein, B. E. (1993) · Journal of Neurophysiology · doi:10.1152/jn.1993.69.6.1797 · Cited by 259
Visual-auditory interactions in sensorimotor processing: Saccades versus manual responses.
Hughes, H. C., Reuter‐Lorenz, P. A., Nozawa, G., & Fendrich, R. (1994) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.20.1.131 · Cited by 252
Neurons and behavior: the same rules of multisensory integration apply
Stein, B. E., Scott Huneycutt, W., & Alex Meredith, M. (1988) · Brain Research · doi:10.1016/0006-8993(88)91276-0 · Cited by 249
Integration of multiple sensory modalities in cat cortex
Wallace, M., Meredith, M. A., & Stein, B. (1992) · Experimental Brain Research · doi:10.1007/bf00227844 · Cited by 235
The physiology of coloured hearing A PET activation study of colour-word synaesthesia
Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J. D. G., Goldstein, L., Heather, J., Frackowiak, R. S. J., & Frith, C. D. (1995) · Brain · doi:10.1093/brain/118.3.661 · Cited by 214
Unit study of monkey frontal cortex: active localization of auditory and of visual stimuli
Vaadia, E., Benson, D. A., Hienz, R. D., & Goldstein, M. H. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.4.934 · Cited by 209
The time-course of intermodal binding between seeing and hearing affective information
Pourtois, G., de Gelder, B., Vroomen, J., Rossion, B., & Crommelinck, M. (2000) · NeuroReport · doi:10.1097/00001756-200004270-00036 · Cited by 197
Visual System's View of Acoustic Space
MORRELL, F. (1972) · Nature · doi:10.1038/238044a0 · Cited by 195
Walter, W. G. (1964) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1964.tb26760.x · Cited by 195
Speeded responses to audiovisual signal changes result from bimodal integration
SCHRÖGER, E., & WIDMANN, A. (1998) · Psychophysiology · doi:10.1017/s0048577298980714 · Cited by 192
Integration of auditory information in the cat's visual cortex
Fishman, M. C., & Michael, C. R. (1973) · Vision Research · doi:10.1016/0042-6989(73)90002-3 · Cited by 154
Temporal integration in auditory sensory memory: neuromagnetic evidence
Loveless, N. E., Levänen, S., Jousmäki, V., Sams, M. E., & Hari, R. (1996) · Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section · doi:10.1016/0168-5597(95)00271-5 · Cited by 140
Barth, D. S., Goldberg, N., Brett, B. A., & Di, S. (1995) · Brain Research · doi:10.1016/0006-8993(95)00182-p · Cited by 137
Word and Nonword Repetition Within- and Across-Modality: An Event-Related Potential Study
Rugg, M. D., Doyle, M. C., & Wells, T. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.2.209 · Cited by 129
Synesthesia-like Mappings of Lightness, Pitch, and Melodic Interval
Hubbard, T. L. (1996) · The American Journal of Psychology · doi:10.2307/1423274 · Cited by 127
Chapter 20 Comparisons of cross-modality integration in midbrain and cortex
Stein, B. E., & Wallace, M. T. (1996) · Progress in brain research · doi:10.1016/s0079-6123(08)63336-1 · Cited by 125
Slow vertex potentials: Interactions among auditory, tactile, electric and visual stimuli
Davis, H., Osterhammel, P. A., Wier, C. C., & Gjerdingen, D. (1972) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(72)90244-1 · Cited by 123
Gaffan, D., & HARRISON, S. (1991) · Brain · doi:10.1093/brain/114.5.2133 · Cited by 103
The influence of vision on the absolute identification of sound-source position
Shelton, B. R., & Searle, C. L. (1980) · Perception & Psychophysics · doi:10.3758/bf03198830 · Cited by 100
Shifts in Binaural Localization after Prolonged Exposures to Atypical Combinations of Stimuli
Held, R. M. (1955) · The American Journal of Psychology · doi:10.2307/1418782 · Cited by 99
Duhamel, J.-R. É., Colby, C., & Goldberg, M. E. (1991) · Brain And Space · doi:10.1093/oso/9780198542841.003.0012 · Cited by 99
Visual and somatosensory processing in the macaque temporal cortex: the role of ?expectation?
Mistlin, A. J., & Perrett, D. I. (1990) · Experimental Brain Research · doi:10.1007/bf00231263 · Cited by 81
Roda, J., & Reinoso-Su�rez, F. (1983) · Experimental Brain Research · doi:10.1007/bf00235548 · Cited by 76
Can we see and hear at the same time?
Bernstein, I. H. (1970) · Acta Psychologica · doi:10.1016/0001-6918(70)90119-8 · Cited by 66
Prefrontal unit activity of macaque monkeys during auditory and visual reaction time tasks
Ito, S.-I. (1982) · Brain Research · doi:10.1016/0006-8993(82)91025-3 · Cited by 66
Cross-modal effects on visual and auditory object perception
O’Leary, A., & Rhodes, G. (1984) · Perception & Psychophysics · doi:10.3758/bf03205954 · Cited by 61
Visual-auditory interactions in cat superior colliculus: their role in the control of gaze
Peck, C. K. (1987) · Brain Research · doi:10.1016/0006-8993(87)90253-8 · Cited by 57
Coactivation in the perception of redundant targets.
Fournier, L. R., & Eriksen, C. W. (1990) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.16.3.538 · Cited by 52
The locus of intersensory facilitation of reaction time
Schmidt, R. A., Gielen, S. C. A. M., & van den Heuvel, P. J. M. (1984) · Acta Psychologica · doi:10.1016/0001-6918(84)90040-4 · Cited by 49
Computer-assisted placement of electrodes on the human head
Echallier, J. F., Perrin, F., & Pernier, J. (1992) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(92)90161-a · Cited by 33
Effects of bisensory stimulation on reaction time and the evoked cortical potential
Andreassi, J. L., & Greco, J. R. (1975) · Physiological Psychology · doi:10.3758/bf03337503 · Cited by 31
Coactivation in the perception of redundant targets.
Fournier, L. R., & Eriksen, C. W. (1990) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.16.3.538 · Cited by 31
Watanabe, J., & Iwai, E. (1991) · Brain Research Bulletin · doi:10.1016/0361-9230(91)90099-6 · Cited by 30
A Cross-Modal Aftereffect: Auditory Displacement following Adaptation to Visual Motion
Ehrenstein, W. H., & Reinhardt-Rutland, A. H. (1996) · Perceptual and Motor Skills · doi:10.2466/pms.1996.82.1.23 · Cited by 25
Kimura, A., & Tamai, Y. (1992) · Brain Research · doi:10.1016/0006-8993(92)90078-n · Cited by 22
Discrepancy and nondiscrepancy methods of assessing visual-auditory interaction
Warren, D. H. D., McCarthy, T. J., & Welch, R. B. (1983) · Perception & Psychophysics · doi:10.3758/bf03202891 · Cited by 13
Dynamic interactions of evoked potentials in a polysensory cortex of the cat
Toldi, J., Fehér, O., & Feuer, L. (1984) · Neuroscience · doi:10.1016/0306-4522(84)90084-8 · Cited by 10
Neural mechanisms for integrating information from multiple senses
Stein, B. E., Laurienti, P. J., Stanford, T. R., & Wallace, M. T. (2002) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2000.869665 · Cited by 9
Walsh, V. (1996) · Current Biology · doi:10.1016/s0960-9822(02)00503-1 · Cited by 5
Crossmodal Interactions of Auditory Stimulus Presentation on the Visual Evoked Magnetic Response
Busch, C., Wilson, G. F., Orr, C., & Papanicolaou, A. C. (1989) · Springer eBooks · doi:10.1007/978-1-4613-0581-1_42 · Cited by 4
Research works printed but not linked: 6
- Bernstein I. H., 1970, Acta Physiologica Academiae Scientarum Hungaricae, 33, 21
- Costin D., 1991, Society for Neuroscience Abstracts, 17, 656
- Fisher B. D., 1994, Cahiers de Psychologie Cognitive, 13, 92
- 10.1037/0096-1523.20.1.131 · doi:10.1037/0096-1523.20.1.131 · Paper page
- 10.1037/0033-295X.83.2.157 · doi:10.1037/0033-295x.83.2.157 · Paper page
- Schroeder C. E., 1995, Society for Neuroscience Abstracts, 21, 694
Cited by
The mismatch negativity (MMN) in basic research of central auditory processing: A review
Näätänen, R., Paavilainen, P., Rinne, T., & Alho, K. (2007) · Clinical Neurophysiology · doi:10.1016/j.clinph.2007.04.026 · Cited by 2,644
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463
Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review
Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0333-8 · Cited by 1,416
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012
Metadata
- Authors
- Giard, M. H., & Peronnet, F.
- Affiliations
- Inserm (FR)
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 1999-09-01
- Volume
- 11
- Issue
- 5
- Pages
- 473-490
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2040510368
- Cited by
- 1,072
- References
- 88
- Citations by year
- 2026: 17 · 2025: 25 · 2024: 29 · 2023: 31 · 2022: 40 · 2021: 51 · 2020: 45 · 2019: 38 · 2018: 47 · 2017: 36 · 2016: 38 · 2015: 53 · 2014: 49 · 2013: 58 · 2012: 58
- FWCI
- 20.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Cognitive Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- audiovisual integration, multisensory integration, object recognition, multimodal feature integration, event-related potentials, auditory N1, early sensory processing
- MeSH terms
- Acoustic Stimulation, Adaptation, Physiological, Adult, Auditory Cortex, Auditory Perception, Brain Mapping, Cognition, Electroencephalography, Electrophysiology, Evoked Potentials, Auditory, Evoked Potentials, Visual, Female, Frontal Lobe, Humans, Male, Pattern Recognition, Visual, Photic Stimulation, Reaction Time, Visual Cortex, Visual Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.