Selective Attention and Audiovisual Integration: Is Attending to Both Modalities a Prerequisite for Early Integration?
Open accessAbstract
Interactions between multisensory integration and attention were studied using a combined audiovisual streaming design and a rapid serial visual presentation paradigm. Event-related potentials (ERPs) following audiovisual objects (AV) were compared with the sum of the ERPs following auditory (A) and visual objects (V). Integration processes were expressed as the difference between these AV and (A + V) responses and were studied while attention was directed to one or both modalities or directed elsewhere. Results show that multisensory integration effects depend on the multisensory objects being fully attended--that is, when both the visual and auditory senses were attended. In this condition, a superadditive audiovisual integration effect was observed on the P50 component. When unattended, this effect was reversed; the P50 components of multisensory ERPs were smaller than the unisensory sum. Additionally, we found an enhanced late frontal negativity when subjects attended the visual component of a multisensory object. This effect, bearing a strong resemblance to the auditory processing negativity, appeared to reflect late attention-related processing that had spread to encompass the auditory component of the multisensory object. In conclusion, our results shed new light on how the brain processes multisensory auditory and visual information, including how attention modulates multisensory integration processes.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The facilitation effects of multisensory integration on attentional capture partially rely on the stimuli to be attended.
“Previous work showed that the facilitation effects of the multisensory integration on attentional capture, that is the facilitation in the detection of target stimuli, seem to partially rely on the stimuli to be attended7.”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [7]
Previous studies report that a condition with the absence of sound elicits a lower sense of presence.
“Our result is consistent with previous studies reporting that a condition with the absence of sound elicits a lower sense of presence (e.g.,7,88,89).”
From The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence (2021) · cites this paper as [7]
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · see citing works
Näätänen, R., & Picton, T. (1987) · Psychophysiology · doi:10.1111/j.1469-8986.1987.tb00311.x · Cited by 3,321 · see citing works
(1994) · Choice Reviews Online · doi:10.5860/choice.31-4365 · Cited by 1,897 · see citing works
McCarthy, G., & Wood, C. C. (1985) · Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section · doi:10.1016/0168-5597(85)90015-2 · Cited by 1,765 · see citing works
Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1998.0281 · Cited by 1,201 · see citing works
Giard, M. H., & Peronnet, F. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563544 · Cited by 1,072 · see citing works
Motter, B. C. (1993) · Journal of Neurophysiology · doi:10.1152/jn.1993.70.3.909 · Cited by 1,055 · see citing works
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954 · see citing works
Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00066-6 · Cited by 894 · see citing works
Event-related potentials : a methods handbook
Handy, T. C. (2005) · Medical Entomology and Zoology · Cited by 753 · see citing works
Visual illusion induced by sound
Shams, L., Kamitani, Y., & Shimojo, S. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00069-1 · Cited by 668 · see citing works
Modulation of early sensory processing in human auditory cortex during auditory selective attention.
Woldorff, M. G., Gallen, C. C., Hampson, S. R., Hillyard, S. A., Pantev, C., Sobel, D. F., & Bloom, F. E. (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.18.8722 · Cited by 645 · see citing works
Self-citationHandbook of Cognitive Neuroscience
Gazzaniga, M. S. (1984) · Springer eBooks · doi:10.1007/978-1-4899-2177-2 · Cited by 460 · see citing works
van der Heijden, A. H. C. (2003) · Routledge eBooks · doi:10.4324/9780203359341 · Cited by 440 · see citing works
Talsma, D., & Woldorff, M. G. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929054475172 · Cited by 417 · see citing works
Self-citationEnhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis
Stein, B. E., London, N., Wilkinson, L. K., & Price, D. D. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.6.497 · Cited by 404 · see citing works
Foxe, J. J., Mórocz, I. Á., Murray, M. M., Higgins, B. A., Javitt, D. C., & Schroeder, C. E. (2000) · Cognitive Brain Research · doi:10.1016/s0926-6410(00)00024-0 · Cited by 391 · see citing works
Distortion of ERP averages due to overlap from temporally adjacent ERPs: Analysis and correction
Woldorff, M. G. (1993) · Psychophysiology · doi:10.1111/j.1469-8986.1993.tb03209.x · Cited by 391 · see citing works
Self-citationThe Effects of Channel‐Selective Attention on the Mismatch Negativity Wave Elicited by Deviant Tones
Woldorff, M. G., Hackley, S. A., & Hillyard, S. A. (1991) · Psychophysiology · doi:10.1111/j.1469-8986.1991.tb03384.x · Cited by 377 · see citing works
Self-citationDeactivation of Sensory-Specific Cortex by Cross-Modal Stimuli
Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y., Field, A. S., & Stein, B. E. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317361930 · Cited by 366 · see citing works
Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading
Driver, J. (1996) · Nature · doi:10.1038/381066a0 · Cited by 315 · see citing works
The spread of attention across modalities and space in a multisensory object
Busse, L., Roberts, K., Crist, R. E., Weissman, D. H., & Woldorff, M. G. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507704102 · Cited by 281 · see citing works
Self-citationDevelopment of Multisensory Neurons and Multisensory Integration in Cat Superior Colliculus
Wallace, M. T., & Stein, B. E. (1997) · Journal of Neuroscience · doi:10.1523/jneurosci.17-07-02429.1997 · Cited by 267 · see citing works
Reflexive Attention Modulates Processing of Visual Stimuli in Human Extrastriate Cortex
Hopfinger, J. B., & Mangun, G. R. (1998) · Psychological Science · doi:10.1111/1467-9280.00083 · Cited by 253 · see citing works
The ventriloquist effect does not depend on the direction of deliberate visual attention
Bertelson, P., Vroomen, J., de Gelder, B., & Driver, J. (2000) · Perception & Psychophysics · doi:10.3758/bf03205552 · Cited by 252 · see citing works
Auditory distraction: event-related potential and behavioral indices
Schröger, E., Giard, M. H., & Wolff, C. (2000) · Clinical Neurophysiology · doi:10.1016/s1388-2457(00)00337-0 · Cited by 239 · see citing works
Integration of multiple sensory modalities in cat cortex
Wallace, M. T., Meredith, M. A., & Stein, B. E. (1992) · Experimental Brain Research · doi:10.1007/bf00227844 · Cited by 239 · see citing works
Effects of Spatial Congruity on Audio-Visual Multimodal Integration
Teder-Sälejärvi, W. A., Russo, F. D., McDonald, J. J., & Hillyard, S. A. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929054985383 · Cited by 229 · see citing works
Kenemans, J. L., de Kok, A., & Smulders, F. T. Y. (1993) · Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section · doi:10.1016/0168-5597(93)90028-n · Cited by 228 · see citing works
Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli
Lewald, J., & Guski, R. (2003) · Cognitive Brain Research · doi:10.1016/s0926-6410(03)00074-0 · Cited by 218 · see citing works
Boutros, N. N., & Belger, A. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(98)00253-4 · Cited by 215 · see citing works
From retinotopy to recognition: fMRI in human visual cortex
Tootell, R. B. H., Hadjikhani, N. K., Mendola, J. D., Marrett, S., & Dale, A. M. (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01171-1 · Cited by 205 · see citing works
Hearing Lips and Seeing Voices
Boston, D. W. (1977) · British Journal of Audiology · doi:10.3109/03005367709078839 · Cited by 203 · see citing works
The ventriloquist effect does not depend on the direction of automatic visual attention
Vroomen, J., Bertelson, P., & de Gelder, B. (2001) · Perception & Psychophysics · doi:10.3758/bf03194427 · Cited by 196 · see citing works
Neural systems mediating selective attention.
Hillyard, S. A., Mangun, G. R., Woldorff, M. G., & Luck, S. J. (1995) · Cited by 193 · see citing works
Self-citationSound alters visual evoked potentials in humans
Shams, L. B., Kamitani, Y., Thompson, S., & Shinsuke Shimojo, A. (2001) · Neuroreport · doi:10.1097/00001756-200112040-00049 · Cited by 192 · see citing works
Theeuwes, J. (1993) · Acta Psychologica · doi:10.1016/0001-6918(93)90056-w · Cited by 190 · see citing works
ERP effects of intermodal attention and cross‐modal links in spatial attention
Eimer, M., & Schröger, E. (1998) · Psychophysiology · doi:10.1017/s004857729897086x · Cited by 185 · see citing works
Fort, A. (2002) · Cerebral Cortex · doi:10.1093/cercor/12.10.1031 · Cited by 179 · see citing works
Early auditory–visual interactions in human cortex during nonredundant target identification
Fort, A., Delpuech, C., Pernier, J., & Giard, M. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00058-7 · Cited by 176 · see citing works
Smid, H. G. O. M., Jakob, A., & Heinze, H. –. (1999) · Psychophysiology · doi:10.1017/s0048577299971135 · Cited by 167 · see citing works
Sensory and Multisensory Responses in the Newborn Monkey Superior Colliculus
Wallace, M. T., & Stein, B. E. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-22-08886.2001 · Cited by 133 · see citing works
Chapter 20 Comparisons of cross-modality integration in midbrain and cortex
Stein, B. E., & Wallace, M. T. (1996) · Progress in brain research · doi:10.1016/s0079-6123(08)63336-1 · Cited by 125 · see citing works
Multisensory processing and oscillatory gamma responses: effects of spatial selective attention
Senkowski, D., Talsma, D., Herrmann, C. S., & Woldorff, M. G. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2381-z · Cited by 119 · see citing works
Self-citationMacaluso, E., Martin Eimer, A., Frith, C., & Driver, J. (2003) · Experimental Brain Research · doi:10.1007/s00221-002-1335-y · Cited by 97 · see citing works
Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1999) · Attention, Space and Action · doi:10.1093/oso/9780198524694.003.0003 · Cited by 74 · see citing works
McCarthy, G., & Wood, C. C. (1985) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(85)90858-2 · Cited by 71 · see citing works
The role of position in object selection in vision
van der Heijden, A. H. C. (1993) · Psychological Research · doi:10.1007/bf00572132 · Cited by 69 · see citing works
Cortex Controls Multisensory Depression in Superior Colliculus
Jiang, W., & Stein, B. E. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00369.2003 · Cited by 69 · see citing works
Split-Second Sequential Selective Activation in Human Secondary Visual Cortex
Kenemans, J. L., Lijffijt, M., Camfferman, G., & Verbaten, M. N. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317205311 · Cited by 60 · see citing works
Talsma, D., & Kok, A. (2001) · Psychophysiology · doi:10.1111/1469-8986.3850736 · Cited by 60 · see citing works
Self-citationTalsma, D., & Kok, A. (2002) · Psychophysiology · doi:10.1017/s004857720200152x · Cited by 55 · see citing works
Self-citationDirecting spatial attention towards the illusory location of a ventriloquized sound
Vroomen, J., Bertelson, P., & de Gelder, B. (2001) · Acta Psychologica · doi:10.1016/s0001-6918(00)00068-8 · Cited by 52 · see citing works
Event-related potentials and audiovisual stimuli: multimodal interactions
Czigler, I., & Balázs, L. (2001) · Neuroreport · doi:10.1097/00001756-200102120-00009 · Cited by 13 · see citing works
Flagged first, then by citations
2 of the 56 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598 · see citing works
Driver, J., & Noesselt, T. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.013 · Cited by 940 · see citing works
The multifaceted interplay between attention and multisensory integration.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.06.008 · Cited by 828 · see citing works
Self-citationAuditory attention — focusing the searchlight on sound
Fritz, J. B., Elhilali, M., David, S. V., & Shamma, S. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.07.011 · Cited by 521 · see citing works
Visual Modulation of Neurons in Auditory Cortex
Kayser, C., Petkov, C. I., & Logothetis, N. K. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm187 · Cited by 502 · see citing works
Pip and pop: Nonspatial auditory signals improve spatial visual search.
Van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.34.5.1053 · Cited by 485 · see citing works
Development of multisensory integration from the perspective of the individual neuron
Stein, B. E., Stanford, T. R., & Rowland, B. A. (2014) · Nature reviews. Neuroscience · doi:10.1038/nrn3742 · Cited by 394 · see citing works
The Leading Sense: Supramodal Control of Neurophysiological Context by Attention
Lakatos, P., O’Connell, M. N., Barczak, A., Mills, A., Javitt, D. C., & Schroeder, C. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.10.014 · Cited by 381 · see citing works
Attention and the multiple stages of multisensory integration: A review of audiovisual studies
Koelewijn, T., Bronkhorst, A. W., & Theeuwes, J. (2010) · Acta Psychologica · doi:10.1016/j.actpsy.2010.03.010 · Cited by 301 · see citing works
Mishra, J., Martı́nez, A., B, T. J. S. A., & Hillyard, S. A. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.4912-06.2007 · Cited by 255 · see citing works
Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex
Kayser, C., Petkov, C. I., Augath, M. A., & Logothetis, N. K. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.4737-06.2007 · Cited by 231 · see citing works
Brandwein, A. B., Foxe, J. J., Butler, J. S., Russo, N. N., Altschuler, T. S., Gomes, H., & Molholm, S. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs109 · Cited by 236 · see citing works
Crossmodal influences on visual perception
Shams, L. B., & Kim, R. S. (2010) · Physics of Life Reviews · doi:10.1016/j.plrev.2010.04.006 · Cited by 229 · see citing works
Predictive coding and multisensory integration: an attentional account of the multisensory mind
Talsma, D. (2015) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2015.00019 · Cited by 219 · see citing works
Self-citationThe neural basis of multisensory integration in the midbrain: Its organization and maturation
Stein, B. E., Stanford, T. R., & Rowland, B. A. (2009) · Hearing Research · doi:10.1016/j.heares.2009.03.012 · Cited by 210 · see citing works
Senkowski, D., Saint‐Amour, D., Höfle, M., & Foxe, J. J. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.03.075 · Cited by 204 · see citing works
Assessing the Role of the ‘Unity Assumption’ on Multisensory Integration: A Review
Chen, Y., & Spence, C. J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00445 · Cited by 186 · see citing works
Marucci, M., Di Flumeri, G., Borghini, G., Sciaraffa, N., Scandola, M., Pavone, E. F., Babiloni, F., Betti, V., & Aricò, P. (2021) · Scientific Reports · doi:10.1038/s41598-021-84196-8 · Cited by 170 · see citing works
The interactions of multisensory integration with endogenous and exogenous attention
Tang, X., Wu, J., & Shen, Y. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.11.002 · Cited by 168 · see citing works
Early multisensory interactions affect the competition among multiple visual objects
der Burg, E. V., Talsma, D., Olivers, C. N. L., Hickey, C., & Theeuwes, J. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.12.068 · Cited by 157 · see citing works
Self-citationAttention to touch weakens audiovisual speech integration
Alsius, A., Navarra, J., & Soto‐Faraco, S. (2007) · Experimental Brain Research · doi:10.1007/s00221-007-1110-1 · Cited by 155 · see citing works
Budinger, E., & Scheich, H. (2009) · Hearing Research · doi:10.1016/j.heares.2009.04.021 · Cited by 148 · see citing works
Modality-specific selective attention attenuates multisensory integration
Mozolic, J. L., Hugenschmidt, C. E., Peiffer, A. M., & Laurienti, P. J. (2007) · Experimental Brain Research · doi:10.1007/s00221-007-1080-3 · Cited by 142 · see citing works
Mozolic, J. L., Long, A. B., Morgan, A. R., Rawley‐Payne, M., & Laurienti, P. J. (2009) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2009.04.013 · Cited by 138 · see citing works
Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites
Fairhall, S. L., & Macaluso, E. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06688.x · Cited by 137 · see citing works
A multisensory perspective of working memory
Quak, M., London, R. E., & Talsma, D. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00197 · Cited by 135 · see citing works
Self-citationCue combination in human spatial navigation
Chen, X., McNamara, T. P., Kelly, J. W., & Wolbers, T. (2017) · Cognitive Psychology · doi:10.1016/j.cogpsych.2017.04.003 · Cited by 122 · see citing works
Raij, T., Ahveninen, J., Lin, F., Witzel, T., Jääskeläinen, I. P., Letham, B., Israeli, E. R., Sahyoun, C. P., Vasios, C. E., Stufflebeam, S. M., et al. (2010) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07213.x · Cited by 132 · see citing works
Perceptual Inference, Learning, and Attention in a Multisensory World
Noppeney, U. (2021) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-100120-085519 · Cited by 133 · see citing works
Multisensory Integration and Aging
Mozolic, J. L., Hugenschmidt, C. E., Peiffer, A. M., & Laurienti, P. J. (2011) · Frontiers in neuroscience · doi:10.1201/b11092-25 · Cited by 117 · see citing works
The Role of the Posterior Superior Temporal Sulcus in Audiovisual Processing
Hocking, J., & Price, C. J. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn007 · Cited by 116 · see citing works
Reduced multisensory facilitation in persons with autism
Collignon, O., Charbonneau, G., Peters, F., Nassim, M., Lassonde, M., Lepore, F., Mottron, L., & Bertone, A. (2013) · Cortex · doi:10.1016/j.cortex.2012.06.001 · Cited by 114 · see citing works
Bottom-up and top-down modulation of multisensory integration
Choi, I., Lee, J. Y., & Lee, S. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.05.002 · Cited by 110 · see citing works
How prior expectations shape multisensory perception
Gau, R., & Noppeney, U. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.045 · Cited by 109 · see citing works
Salient Sounds Activate Human Visual Cortex Automatically
McDonald, J. J., Störmer, V. S., Martı́nez, A., Feng, W., & Hillyard, S. A. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5902-12.2013 · Cited by 109 · see citing works
Precision control for a flexible body representation
Limanowski, J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.10.023 · Cited by 107 · see citing works
Others' Actions Reduce Crossmodal Integration in Peripersonal Space
Heed, T., Habets, B., Sebanz, N., & Knoblich, G. (2010) · Current Biology · doi:10.1016/j.cub.2010.05.068 · Cited by 104 · see citing works
Top-down control and early multisensory processes: chicken vs. egg
Meo‐Monteil, R. D., Murray, M. M., Clarke, S., & Matusz, P. J. (2015) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2015.00017 · Cited by 107 · see citing works
Audiovisual events capture attention: Evidence from temporal order judgments
der Burg, E. V., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008) · Journal of Vision · doi:10.1167/8.5.2 · Cited by 106 · see citing works
Contextual modulation of primary visual cortex by auditory signals
Petro, L. S., Paton, A., & Muckli, L. F. (2017) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2016.0104 · Cited by 97 · see citing works
Molholm, S., Martı́nez, A., Shpaner, M., & Foxe, J. J. (2007) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05668.x · Cited by 98 · see citing works
Takagi, S., Hiramatsu, S., Tabei, K., & Tanaka, A. (2015) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2015.00001 · Cited by 99 · see citing works
Early, low-level auditory-somatosensory multisensory interactions impact reaction time speed
Sperdin, H. F. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.002.2009 · Cited by 97 · see citing works
Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search?
Klapetek, A., Ngo, M. K., & Spence, C. J. (2012) · Attention Perception & Psychophysics · doi:10.3758/s13414-012-0317-9 · Cited by 94 · see citing works
The Curious Incident of Attention in Multisensory Integration: Bottom-up vs. Top-down
Macaluso, E., Noppeney, U., Talsma, D., Vercillo, T., Hartcher-O’Brien, J., & Adam, R. (2016) · Multisensory Research · doi:10.1163/22134808-00002528 · Cited by 91 · see citing works
Self-citationElectrophysiological evidence for speech-specific audiovisual integration
Baart, M., Stekelenburg, J. J., & Vroomen, J. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.11.011 · Cited by 89 · see citing works
Suppression of multisensory integration by modality-specific attention in aging
Hugenschmidt, C. E., Mozolic, J. L., & Laurienti, P. J. (2009) · Neuroreport · doi:10.1097/wnr.0b013e328323ab07 · Cited by 89 · see citing works
Senkowski, D., Saint‐Amour, D., Kelly, S. P., & Foxe, J. J. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.01.053 · Cited by 89 · see citing works
Dual Mechanisms for the Cross-Sensory Spread of Attention: How Much Do Learned Associations Matter?
Fiebelkorn, I. C., Foxe, J. J., & Molholm, S. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhp083 · Cited by 89 · see citing works
Spatial Attention Evokes Similar Activation Patterns for Visual and Auditory Stimuli
Smith, D. V., Davis, B., Niu, K., Healy, E. W., Bonilha, L., Fridriksson, J., Morgan, P. S., & Rorden, C. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21241 · Cited by 86 · see citing works
Chen, Y., & Spence, C. J. (2011) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/a0025638 · Cited by 85 · see citing works
Besle, J., Bertrand, O., & Giard, M. (2009) · Hearing Research · doi:10.1016/j.heares.2009.06.016 · Cited by 83 · see citing works
Effect of attentional load on audiovisual speech perception: evidence from ERPs
Alsius, A., Möttönen, R., Sams, M. E., Soto‐Faraco, S., & Tiippana, K. M. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00727 · Cited by 83 · see citing works
Magnée, M. J. C. M., de Gelder, B., van Engeland, H., & Kemner, C. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2008.01902.x · Cited by 80 · see citing works
Neural basis of multisensory looming signals
Tyll, S., Bonath, B., Schoenfeld, M. A., Heinze, H.-J., Ohl, F. W., & Noesselt, T. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.056 · Cited by 75 · see citing works
Resolving multisensory and attentional influences across cortical depth in sensory cortices
Gau, R., Bazin, P., Trampel, R., Turner, R., & Noppeney, U. (2020) · eLife · doi:10.7554/elife.46856 · Cited by 76 · see citing works
Multi-sensory learning and learning to read
Blomert, L., & Froyen, D. (2010) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2010.06.025 · Cited by 81 · see citing works
The audio-visual integration effect on music emotion: Behavioral and physiological evidence
Pan, F., Zhang, L., Ou, Y., & Zhang, X. (2019) · PLoS ONE · doi:10.1371/journal.pone.0217040 · Cited by 75 · see citing works
Assessing the role of attention in the audiovisual integration of speech
Navarra, J., Alsius, A., Soto‐Faraco, S., & Spence, C. J. (2009) · Information Fusion · doi:10.1016/j.inffus.2009.04.001 · Cited by 76 · see citing works
Sambo, C. F., & Förster, B. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21109 · Cited by 74 · see citing works
Reliability-Weighted Integration of Audiovisual Signals Can Be Modulated by Top-down Attention
Rohe, T., & Noppeney, U. (2018) · eNeuro · doi:10.1523/eneuro.0315-17.2018 · Cited by 74 · see citing works
Poke and pop: Tactile–visual synchrony increases visual saliency
der Burg, E. V., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008) · Neuroscience Letters · doi:10.1016/j.neulet.2008.11.002 · Cited by 72 · see citing works
Badde, S., Navarro, K. T., & Landy, M. S. (2020) · Cognition · doi:10.1016/j.cognition.2019.104170 · Cited by 72 · see citing works
Empathy, motivation, and P300 BCI performance
Kleih, S. C., & Kübler, A. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00642 · Cited by 69 · see citing works
Audiovisual integration in the human brain: a coordinate-based meta-analysis
Gao, C., Green, J. J., Yang, X., Oh, S., Kim, J., & Shinkareva, S. V. (2022) · Cerebral Cortex · doi:10.1093/cercor/bhac443 · Cited by 67 · see citing works
Mishra, J., Martı́nez, A., & Hillyard, S. A. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21295 · Cited by 67 · see citing works
Why do we need to use a zero reference? Reference influences on the ERPs of audiovisual effects
Tian, Y., & Yao, D. (2013) · Psychophysiology · doi:10.1111/psyp.12130 · Cited by 65 · see citing works
Tanaka, E., Inui, K., Kida, T., Miyazaki, T., Takeshima, Y., & Kakigi, R. (2008) · BMC Neuroscience · doi:10.1186/1471-2202-9-116 · Cited by 66 · see citing works
Preattentive processing of audio-visual emotional signals
Föcker, J., Gondan, M., & Röder, B. (2011) · Acta Psychologica · doi:10.1016/j.actpsy.2011.02.004 · Cited by 63 · see citing works
Multisensory interactions in the depth plane in front and rear space: A review
der Stoep, N. V., Nijboer, T. C. W., der Stigchel, S. V., & Spence, C. J. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.12.007 · Cited by 63 · see citing works
Wahn, B., & Peter König, B. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01084 · Cited by 62 · see citing works
The role of emotion in dynamic audiovisual integration of faces and voices
Kokinous, J., Kotz, S. A., Tavano, A., & Schröger, E. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsu105 · Cited by 62 · see citing works
The angular gyrus is a supramodal comparator area in action–outcome monitoring
van Kemenade, B. M., Arikan, B. E., Kircher, T. T. J., & Straube, B. (2017) · Brain Structure and Function · doi:10.1007/s00429-017-1428-9 · Cited by 61 · see citing works
Multisensory enhancement of attention depends on whether you are already paying attention
Lunn, J., Sjoblom, A., Ward, J., Soto-Faraco, S., & Forster, S. (2019) · Cognition · doi:10.1016/j.cognition.2019.02.008 · Cited by 62 · see citing works
Magnée, M. J. C. M., de Gelder, B., van Engeland, H., & Kemner, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0024196 · Cited by 59 · see citing works
Audiovisual integration in high functioning adults with autism
Keane, B. P., Rosenthal, O., Chun, N. H., & Shams, L. B. (2009) · Research in Autism · doi:10.1016/j.rasd.2009.09.015 · Cited by 60 · see citing works
The effects of attention on the temporal integration of multisensory stimuli
Donohue, S. E., Green, J. J., & Woldorff, M. G. (2015) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2015.00032 · Cited by 59 · see citing works
Self-citationStekelenburg, J. J., & Vroomen, J. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00026 · Cited by 60 · see citing works
The effect of multisensory cues on attention in aging
Mahoney, J. R., Verghese, J., Dumas, K., Wang, C., & Holtzer, R. (2012) · Brain Research · doi:10.1016/j.brainres.2012.07.014 · Cited by 57 · see citing works
Giani, A. S., O, B. P. E., Belardinelli, P., Kleiner, M., Preissl, H., & Noppeney, U. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.114 · Cited by 57 · see citing works
Intermodal attention affects the processing of the temporal alignment of audiovisual stimuli
Talsma, D., Senkowski, D., & Woldorff, M. G. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1858-6 · Cited by 54 · see citing works
Self-citationThe COGs (context, object, and goals) in multisensory processing
ten Oever, S., Romei, V., van Atteveldt, N., Soto‐Faraco, S., Murray, M. M., & Matusz, P. J. (2016) · Experimental Brain Research · doi:10.1007/s00221-016-4590-z · Cited by 55 · see citing works
Wu, J., Li, Q., Bai, O., & Touge, T. (2009) · Journal of Clinical Neurophysiology · doi:10.1097/wnp.0b013e3181c298b1 · Cited by 54 · see citing works
Top-down and bottom-up modulation in processing bimodal face/voice stimuli
Latinus, M., Rufin Vanrullen, A., & Taylor, M. J. (2010) · BMC Neuroscience · doi:10.1186/1471-2202-11-36 · Cited by 55 · see citing works
Flavor binding: Its nature and cause
Stevenson, R. J. (2014) · Psychological Bulletin · doi:10.1037/a0033473 · Cited by 52 · see citing works
Selective Attention Modulates Early Human Evoked Potentials during Emotional Face–Voice Processing
Ho, H. T., Schröger, E., & Kotz, S. A. (2014) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00734 · Cited by 53 · see citing works
Attention Distributed across Sensory Modalities Enhances Perceptual Performance
Mishra, J., & Gazzaley, A. H. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0867-12.2012 · Cited by 53 · see citing works
Murray, M. M., & Spierer, L. (2009) · Hearing Research · doi:10.1016/j.heares.2009.04.022 · Cited by 50 · see citing works
Kayser, S. J., Philiastides, M. G., & Kayser, C. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.01.010 · Cited by 46 · see citing works
The neural basis of visual dominance in the context of audio-visual object processing
Schmid, C., Büchel, C., & Rose, M. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.11.051 · Cited by 48 · see citing works
Song, J., Lee, H., Kang, H., Lee, D. S., Chang, S. O., & Oh, S. H. (2014) · Brain Structure and Function · doi:10.1007/s00429-013-0704-6 · Cited by 48 · see citing works
Audiovisual interaction enhances auditory detection in late stage: an event-related potential study
Li, Q., Wu, J., & Touge, T. (2010) · Neuroreport · doi:10.1097/wnr.0b013e3283345f08 · Cited by 47 · see citing works
Non-spatial sounds regulate eye movements and enhance visual search
Zou, H., Müller, H. J., & Shi, Z. (2012) · Journal of Vision · doi:10.1167/12.5.2 · Cited by 47 · see citing works
Temporal Characteristics of Audiovisual Information Processing
Alpert, G. F., Hein, G., Tsai, N., Naumer, M. J., & Knight, R. T. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.5039-07.2008 · Cited by 46 · see citing works
Talsma, D., Mulckhuyse, M., Slagter, H. A., & Theeuwes, J. (2007) · Brain Research · doi:10.1016/j.brainres.2007.07.099 · Cited by 47 · see citing works
Self-citationA Novel Audiovisual Brain-Computer Interface and Its Application in Awareness Detection
Wang, F., He, Y., Pan, J., Xie, Q., Yu, R., Zhang, R., & Li, Y. (2015) · Scientific Reports · doi:10.1038/srep09962 · Cited by 47 · see citing works
Ren, Y., Ren, Y., Yang, W., Tang, X., Wu, F., Wu, Q., Takahashi, S., Ejima, Y., & Wu, J. (2017) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2017.12.004 · Cited by 45 · see citing works
Virtual Reality Adaptation Using Electrodermal Activity to Support the User Experience
Chiossi, F., Welsch, R., Villa, S., Chuang, L. L., & Mayer, S. (2022) · Big Data and Cognitive Computing · doi:10.3390/bdcc6020055 · Cited by 44 · see citing works
der Stoep, N. V., Spence, C. J., Nijboer, T. C. W., & der Stigchel, S. V. (2015) · Acta Psychologica · doi:10.1016/j.actpsy.2015.09.010 · Cited by 43 · see citing works
Exogenous spatial attention decreases audiovisual integration
der Stoep, N. V., der Stigchel, S. V., & Nijboer, T. C. W. (2014) · Attention Perception & Psychophysics · doi:10.3758/s13414-014-0785-1 · Cited by 43 · see citing works
- Authors
- Talsma, D., Doty, T. J., & Woldorff, M. G.
- Affiliations
- Duke University (US)
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2006-03-31
- Volume
- 17
- Issue
- 3
- Pages
- 679-690
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2156150357
- Cited by
- 434
- References
- 56
- Citations by year
- 2026: 13 · 2025: 20 · 2024: 15 · 2023: 25 · 2022: 32 · 2021: 31 · 2020: 24 · 2019: 20 · 2018: 20 · 2017: 22 · 2016: 17 · 2015: 25 · 2014: 22 · 2013: 25 · 2012: 20
- FWCI
- 24.38
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Cerebral Cortex (publishedVersion)
- PubMed (publishedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- University of Twente Research Information (submittedVersion, other-oa)
- Funders
- National Institute of Mental Health (R01-MH64015)
- National Institute of Neurological Disorders and Stroke (R01NS051048)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- multisensory integration, audiovisual integration, selective attention, P50, rapid serial visual presentation, audiovisual streaming, event-related potentials
- MeSH terms
- Acoustic Stimulation, Adolescent, Adult, Attention, Brain Mapping, Cognition, Discrimination, Psychological, Evoked Potentials, Auditory, Evoked Potentials, Visual, Female, Humans, Male, Photic Stimulation, Reaction Time, Scalp
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
