Disentangling unisensory and multisensory components in audiovisual integration using a novel multifrequency fMRI spectral analysis

Hertz, Uri; Amedi, Amir

doi:10.1016/j.neuroimage.2010.04.186

Cited by 43Multisensory perception and integrationNeuroscience and Music PerceptionTactile and Sensory Interactions

Disentangling unisensory and multisensory components in audiovisual integration using a novel multifrequency fMRI spectral analysis

Hertz, Uri, Amedi, Amir·2010·NeuroImage

DOI 10.1016/j.neuroimage.2010.04.186

Open this paper in CoChat — free account

Abstract

Events in the world are mediated through multiple sensory inputs which are processed separately in specific unisensory areas according to the division-of-labor principle, and then need to be integrated to create a unified percept. How this is done is still not clear. For instance, recent evidence showed crossmodal activation in primary areas. We developed a novel approach to study multisensory integration using multifrequency spectral analysis to investigate the processing of audio and visual streams in a multisensory context. Auditory and visual stimuli were delivered in the same experimental condition, each in different presentation frequencies, and thus could be detected by applying Fourier spectral analysis in their different presentation frequencies. The cochleotopic and retinotopic organization of primary auditory and visual areas were found to remain intact in spite of the multisensory context. Auditory responses were also found in the Precuneus, suggesting that it might be a new auditory area responsive to pure tone stimuli, and serving as one end of a novel sensory preference gradient stretching across POG to the calcarine sulcus. Additional audiovisual areal convergence was detected both in areas in the middle of sensory preference gradients, and in primary auditory areas. Interestingly, the in/out synchronization rate of the auditory and visual streams yielded a third interaction frequency, which could be analyzed independently to reveal higher-order audiovisual interaction responses. These results were detected in one compact and natural multisensory experimental condition, which has several advantages over previous approaches. The method can be further implemented to study any type of interaction, within and across sensory modalities.

Integrity signals

Registry

Crossref · 10.1016/j.neuroimage.2010.04.186

Cites

112 of 112 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The precuneus has been identified as an area responding to both visual and auditory stimuli and possibly serving as an audiovisual convergence area.
“The precuneus has been identified as an area responding to both visual and auditory stimuli and possibly serving as an audiovisual convergence area [25].”
From Decoding sound and imagery content in early visual cortex (2014) · cites this paper as [25]

Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Co-Planar Stereotaxic Atlas of the Human Brain
Talairach, J., & Tournoux, P. (1988) · Medical Entomology and Zoology · Cited by 11,798 · see citing works
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · see citing works
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923 · see citing works
Two cortical visual systems
Ungerleider, L. G. (1982) · Medical Entomology and Zoology · Cited by 4,531 · see citing works
Analysis of visual behavior
Ingle, D., Goodale, M. A., & Mansfield, R. J. W. (1982) · Medical Entomology and Zoology · Cited by 4,131 · see citing works
Visual Feature Integration and the Temporal Correlation Hypothesis
Singer, W., & Gray, C. M. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.003011 · Cited by 3,157 · see citing works
Borders of Multiple Visual Areas in Humans Revealed by Functional Magnetic Resonance Imaging
Sereno, M. I., Dale, A. M., Reppas, J. B., Kwong, K. K., Belliveau, J. W., Brady, T. J., Rosen, B. R., & Tootell, R. B. H. (1995) · Science · doi:10.1126/science.7754376 · Cited by 2,658 · see citing works
Social perception from visual cues: Role of the STS region
Allison, T., Puce, A., & McCarthy, G. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01501-1 · Cited by 2,461 · see citing works
The plastic human brain cortex
Pascual-Leone, A., Amedi, A., Fregni, F., & Merabet, L. B. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144216 · Cited by 1,929 · see citing works
Self-citation
Topics in Circular Statistics
Jammalamadaka, S. R., & SenGupta, A. (2001) · Series on multivariate analysis · doi:10.1142/4031 · Cited by 1,824 · see citing works
Retinotopic organization in human visual cortex and the spatial precision of functional MRI
Engel, S. A. (1997) · Cerebral Cortex · doi:10.1093/cercor/7.2.181 · Cited by 1,606 · see citing works
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · see citing works
Temporal binding and the neural correlates of sensory awareness.
Engel, A. K., & Singer, W. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01568-0 · Cited by 1,372 · see citing works
What you see is what you hear
Shams, L., Kamitani, Y., & Shimojo, S. (2000) · Nature · doi:10.1038/35048669 · Cited by 1,262 · see citing works
The Variability of Human, BOLD Hemodynamic Responses
Aguirre, G. K., Zarahn, E., & D’Esposito, M. (1998) · NeuroImage · doi:10.1006/nimg.1998.0369 · Cited by 1,224 · see citing works
THE HUMAN VISUAL CORTEX
Kalanit Grill-spector, A., & Malach, R. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144220 · Cited by 1,191 · see citing works
Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration
Meredith, M. A., & Stein, B. E. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.3.640 · Cited by 1,183 · see citing works
Cross-modal plasticity: where and how?
Bavelier, D., & Neville, H. J. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn848 · Cited by 1,040 · see citing works
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012 · see citing works
Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study.
Molholm, S., Ritter, W., Murray, M. M., Javitt, D., Schroeder, C., & Foxe, J. J. (2002) · Brain research. Cognitive brain research · doi:10.1016/s0926-6410(02)00066-6 · Cited by 980 · see citing works
fMRI of human visual cortex
Engel, S. A., Rumelhart, D. E., Wandell, B. A., Lee, A. T.-J., Glover, G. H., Chichilnisky, E. J., & Shadlen, M. N. (1994) · Nature · doi:10.1038/369525a0 · Cited by 977 · see citing works
Neuronal oscillations and multisensory interaction in primary auditory cortex
Lakatos, P., Chen, C.-M., O'Connell, M. N., Mills, A., & Schroeder, C. E. (2007) · Neuron · doi:10.1016/j.neuron.2006.12.011 · Cited by 961 · see citing works
Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954 · see citing works
Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments
Driver, J., & Noesselt, T. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.013 · Cited by 940 · see citing works
Processing of Complex Sounds in the Macaque Nonprimary Auditory Cortex
Rauschecker, J. P., Tian, B., & HAUSER, M. D. (1995) · Science · doi:10.1126/science.7701330 · Cited by 929 · see citing works
Functional specialisation in the visual cortex of the rhesus monkey
Zeki, S. M. (1978) · Nature · doi:10.1038/274423a0 · Cited by 874 · see citing works
Crossmodal Space and Crossmodal Attention
(2004) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198524861.001.0001 · Cited by 792 · see citing works
Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus
Beauchamp, M. S., Lee, K. E., Argall, B., & Martin, A. D. (2004) · Neuron · doi:10.1016/s0896-6273(04)00070-4 · Cited by 765 · see citing works
Center–periphery organization of human object areas
Levy, I., Hasson, U., Avidan, G., Hendler, T., & Malach, R. (2001) · Nature Neuroscience · doi:10.1038/87490 · Cited by 687 · see citing works
Early ‘visual’ cortex activation correlates with superior verbal memory performance in the blind
Amedi, A., Raz, N., Pianka, P., Malach, R., & Zohary, E. (2003) · Nature Neuroscience · doi:10.1038/nn1072 · Cited by 591 · see citing works
Self-citation
Multisensory convergence in calcarine visual areas in macaque monkey
Rockland, K. S., & Ojima, H. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00121-1 · Cited by 564 · see citing works
“What” and “where” in the human auditory system
Alain, C., Arnott, S. R., Hevenor, S. J., Graham, S. J., & Grady, C. L. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.211209098 · Cited by 557 · see citing works
Integration of Letters and Speech Sounds in the Human Brain
van Atteveldt, N., Formisano, E., Goebel, R., & Blomert, L. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.025 · Cited by 545 · see citing works
Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex
Ghazanfar, A. A., Maier, J. X., Hoffman, K. L., & Logothetis, N. K. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.0799-05.2005 · Cited by 531 · see citing works
Unraveling multisensory integration: patchy organization within human STS multisensory cortex
Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004) · Nature Neuroscience · doi:10.1038/nn1333 · Cited by 513 · see citing works
Do deaf individuals see better?
Bavelier, D., Dye, M., & Hauser, P. C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.09.006 · Cited by 507 · see citing works
Multisensory contributions to low-level, ‘unisensory’ processing
Schroeder, C. E., & Foxe, J. J. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.06.008 · Cited by 506 · see citing works
The structural components of music perception. A functional anatomical study
Platel, H. (1997) · Brain · doi:10.1093/brain/120.2.229 · Cited by 506 · see citing works
Superior Temporal Sulcus—It's My Area: Or Is It?
Hein, G., & Knight, R. T. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20148 · Cited by 486 · see citing works
Mirror-Symmetric Tonotopic Maps in Human Primary Auditory Cortex
Formisano, E., Kim, D., Salle, F. D., de Moortele, P. V., Uǧurbil, K., & Goebel, R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00669-x · Cited by 483 · see citing works
fMRI-Adaptation Reveals Dissociable Neural Representations of Identity and Expression in Face Perception
Winston, J. S., Henson, R. N., Fine-Goulden, M. R., & Dolan, R. J. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00155.2004 · Cited by 468 · see citing works
Hierarchical Organization of the Human Auditory Cortex Revealed by Functional Magnetic Resonance Imaging
Wessinger, C. M., VanMeter, J. W., Tian, B., Lare, J. V., Pekar, J. J., & Rauschecker, J. P. (2001) · Journal of Cognitive Neuroscience · doi:10.1162/089892901564108 · Cited by 453 · see citing works
Multisensory spatial interactions: a window onto functional integration in the human brain
Macaluso, E., & Driver, J. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.03.008 · Cited by 428 · see citing works
Multisensory integration: methodological approaches and emerging principles in the human brain.
Calvert, G. A., & Thesen, T. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.03.018 · Cited by 425 · see citing works
Grabbing Your Ear: Rapid Auditory–Somatosensory Multisensory Interactions in Low-level Sensory Cortices Are Not Constrained by Stimulus Alignment
Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh197 · Cited by 404 · see citing works
The Detection of Auditory Visual Desynchrony
Dixon, N. F., & Spitz, L. (1980) · Perception · doi:10.1068/p090719 · Cited by 391 · see citing works
Functional imaging of human crossmodal identification and object recognition
Amedi, A., von Kriegstein, K., van Atteveldt, N., Beauchamp, M. S., & Naumer, M. J. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2396-5 · Cited by 351 · see citing works
Self-citation
Tonotopic Organization in Human Auditory Cortex Revealed by Progressions of Frequency Sensitivity
Talavage, T. M., Sereno, M. I., Melcher, J. R., Ledden, P. J., Rosen, B. R., & Dale, A. M. (2004) · Journal of Neurophysiology · doi:10.1152/jn.01125.2002 · Cited by 342 · see citing works
Parallel Processing in the Auditory Cortex of Primates
Rauschecker, J. P. (1998) · Audiology and Neurotology · doi:10.1159/000013784 · Cited by 339 · see citing works
Sound alters activity in human V1 in association with illusory visual perception
Watkins, S., Shams, L. B., Tanaka, S., Haynes, J., & Rees, G. E. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.016 · Cited by 336 · see citing works
Large-scale mirror-symmetry organization of human occipito-temporal object areas
Hasson, U., Harel, M., Levy, I., & Malach, R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00144-2 · Cited by 336 · see citing works
What and where?
Szigetvári, P. (2008) · doi:10.1515/9783110211443.1.93 · Cited by 336 · see citing works
Perceptual Training Narrows the Temporal Window of Multisensory Binding
Powers, A. R., Hillock‐Dunn, A., & Wallace, M. T. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.3501-09.2009 · Cited by 331 · see citing works
A human parietal face area contains aligned head-centered visual and tactile maps
Sereno, M. I., & Huang, R.-S. (2006) · Nature Neuroscience · doi:10.1038/nn1777 · Cited by 322 · see citing works
Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions
Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W., & Perrett, D. I. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929053279586 · Cited by 320 · see citing works
Do early sensory cortices integrate cross-modal information?
Kayser, C., & Logothetis, N. (2007) · Brain Structure and Function · doi:10.1007/s00429-007-0154-0 · Cited by 319 · see citing works
Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech
Miller, L. M., & D’Esposito, M. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.0896-05.2005 · Cited by 318 · see citing works
Repetitive TMS over posterior STS disrupts perception of biological motion
Grossman, E. D., Battelli, L., & Pascual‐Leone, Á. (2005) · Vision Research · doi:10.1016/j.visres.2005.05.027 · Cited by 296 · see citing works
Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices
Noesselt, T., Rieger, J. W., Schoenfeld, M. A., Kanowski, M., Hinrichs, H., Heinze, H., & Driver, J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.2252-07.2007 · Cited by 282 · see citing works
Beyond superior temporal cortex: Intersubject correlations in narrative speech comprehension
Wilson, S. M., Molnar-Szakacs, I., & Iacoboni, M. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm049 · Cited by 281 · see citing works
Nonauditory Events of a Behavioral Procedure Activate Auditory Cortex of Highly Trained Monkeys
Brosch, M., Selezneva, E., & Scheich, H. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.1571-05.2005 · Cited by 279 · see citing works
Multisensory Interactions within Human Primary Cortices Revealed by BOLD Dynamics
Martuzzi, R., Murray, M. M., Michel, C. M., Thiran, J., Maeder, P. P., Clarke, S., & Meuli, R. A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl077 · Cited by 279 · see citing works
Integration of Local Features into Global Shapes
Kourtzi, Z., Tolias, A. S., Altmann, C. F., Augath, M. A., & Logothetis, N. K. (2003) · Neuron · doi:10.1016/s0896-6273(02)01174-1 · Cited by 278 · see citing works
Negative BOLD differentiates visual imagery and perception
Amedi, A., Malach, R., & Pascual-Leone, A. (2005) · Neuron · doi:10.1016/j.neuron.2005.10.032 · Cited by 271 · see citing works
Self-citation
Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion
Mishra, J., Martı́nez, A., B, T. J. S. A., & Hillyard, S. A. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.4912-06.2007 · Cited by 255 · see citing works
Multisensory Interaction in Saccadic Reaction Time: A Time-Window-of-Integration Model
Colonius, H., & Diederich, A. (2004) · Journal of Cognitive Neuroscience · doi:10.1162/0898929041502733 · Cited by 254 · see citing works
Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition
Stevenson, R. A., & James, T. W. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.09.034 · Cited by 252 · see citing works
Binding crossmodal object features in perirhinal cortex
Taylor, K. I., Moss, H. E., Stamatakis, E. A., & TYLER, L. K. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0509704103 · Cited by 247 · see citing works
Spatial and temporal factors during processing of audiovisual speech: a PET study
Macaluso, E., George, N., Dolan, R. J., Spence, C. J., & Driver, J. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.049 · Cited by 237 · see citing works
Functional Imaging Reveals Numerous Fields in the Monkey Auditory Cortex
Petkov, C. I., Kayser, C., Augath, M. A., & Logothetis, N. K. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040215 · Cited by 231 · see citing works
Statistical Criteria in fMRI Studies of Multisensory Integration
Beauchamp, M. S. (2005) · Neuroinformatics · doi:10.1385/ni:3:2:093 · Cited by 227 · see citing works
Cross-Modal Transfer of Information between the Tactile and the Visual Representations in the Human Brain: A Positron Emission Tomographic Study
Hadjikhani, N. K., & Roland, P. E. (1998) · Journal of Neuroscience · doi:10.1523/jneurosci.18-03-01072.1998 · Cited by 224 · see citing works
circular: Circular Statistics, from "Topics in circular Statistics" (2001) S. Rao Jammalamadaka and A. SenGupta, World Scientific.
Agostinelli, C., & Ulric, L. (2004) · ARCA (Università Ca' Foscari Venezia) · Cited by 214 · see citing works
Book Review: How Do We Tell Time?
Buonomano, D. V., & Karmarkar, U. R. (2002) · The Neuroscientist · doi:10.1177/107385840200800109 · Cited by 212 · see citing works
The Effect of Temporal Asynchrony on the Multisensory Integration of Letters and Speech Sounds
van Atteveldt, N., Formisano, E., Blomert, L., & Goebel, R. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl007 · Cited by 209 · see citing works
Touch, sound and vision in human superior temporal sulcus
Beauchamp, M. S., Yasar, N. E., Frye, R. E., & Ro, T. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.015 · Cited by 208 · see citing works
Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas
Hein, G., Doehrmann, O., Müller, N. G., Kaiser, J., Muckli, L. F., & Naumer, M. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1740-07.2007 · Cited by 203 · see citing works
Intermodal timing relations and audio-visual speech recognition by normal-hearing adults
McGrath, M., & Summerfield, Q. (1985) · The Journal of the Acoustical Society of America · doi:10.1121/1.392336 · Cited by 201 · cited 2× · see citing works
“…and visual stimuli is somewhat different from other multisensory studies, whether behavioral (Colonius and Diederich, 2004; Dixon and Spitz, 1980; McGrath and Summerfield, 1985; Powers et al., 2009) or imaging (Dhamala et al., 2007a; Noesselt et al., 2007; van Atteveldt et al., 2007a; Watkins et…”
Spatial attention and crossmodal interactions between vision and touch
Macaluso, E., & Driver, J. (2001) · Neuropsychologia · doi:10.1016/s0028-3932(01)00119-1 · Cited by 198 · see citing works
On the neuronal basis for multisensory convergence: a brief overview
Meredith, M. A. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00059-9 · Cited by 197 · see citing works
Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing.
Schroeder, C., Smiley, J., Fu, K., McGinnis, T., O'Connell, M. N., & Hackett, T. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00120-x · Cited by 193 · see citing works
Direct Temporal-Occipital Feedback Connections to Striate Cortex (V1) in the Macaque Monkey
Rockland, K. S., & Van Hoesen, G. W. (1994) · Cerebral Cortex · doi:10.1093/cercor/4.3.300 · Cited by 191 · see citing works
Visual Enhancement of the Information Representation in Auditory Cortex
Kayser, C., Logothetis, N. K., & Panzeri, S. (2010) · Current Biology · doi:10.1016/j.cub.2009.10.068 · Cited by 187 · see citing works
On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies
Laurienti, P. J., Perrault, T. J., Stanford, T. R., Wallace, M. T., & Stein, B. E. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2370-2 · Cited by 187 · see citing works
Auditory memory in congenitally blind adults: a behavioral-electrophysiological investigation
Röder, B., Rösler, F., & Neville, H. J. (2001) · Cognitive Brain Research · doi:10.1016/s0926-6410(01)00002-7 · Cited by 180 · see citing works
The Effect of Prior Visual Information on Recognition of Speech and Sounds
Noppeney, U., Josephs, O., Hocking, J., Price, C. J., & Friston, K. J. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm091 · Cited by 172 · see citing works
From monkeys to humans: what do we now know about brain homologies?
Sereno, M. I., & Tootell, R. B. H. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.03.014 · Cited by 170 · see citing works
Visual cortical activity during tactile perception in the sighted and the visually deprived
Sathian, K. (2005) · Developmental Psychobiology · doi:10.1002/dev.20056 · Cited by 165 · see citing works
Combined Activation and Deactivation of Visual Cortex During Tactile Sensory Processing
Merabet, L. B., Swisher, J. D., McMains, S. A., Halko, M. A., Amedi, A., Pascual‐Leone, Á., & Somers, D. C. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00806.2006 · Cited by 158 · see citing works
Self-citation
A Putative Model of Multisensory Object Representation
Lacey, S., Tal, N., Amedi, A., & Sathian, K. (2009) · Brain Topography · doi:10.1007/s10548-009-0087-4 · Cited by 139 · see citing works
Self-citation
Activity in human V1 follows multisensory perception
Watkins, S., Shams, L. B., Josephs, O., & Rees, G. E. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.05.027 · Cited by 138 · see citing works
Braille character discrimination in blindfolded human subjects
Kauffman, T., Théoret, H., & Pascual‐Leone, Á. (2002) · Neuroreport · doi:10.1097/00001756-200204160-00007 · Cited by 137 · see citing works
Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach
Tal, N., & Amedi, A. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1949-4 · Cited by 118 · see citing works
Self-citation
The Role of the Posterior Superior Temporal Sulcus in Audiovisual Processing
Hocking, J., & Price, C. J. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn007 · Cited by 116 · see citing works
Multisensory Processing in “Unimodal” Neurons: Cross-Modal Subthreshold Auditory Effects in Cat Extrastriate Visual Cortex
Allman, B. L., & Meredith, M. A. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00173.2007 · Cited by 115 · see citing works
Analysis of the Neuronal Selectivity Underlying Low fMRI Signals
Avidan, G., Hasson, U., Hendler, T., Zohary, E., & Malach, R. (2002) · Current Biology · doi:10.1016/s0960-9822(02)00872-2 · Cited by 114 · see citing works
Top–down task effects overrule automatic multisensory responses to letter–sound pairs in auditory association cortex
van Atteveldt, N., Formisano, E., Goebel, R., & Blomert, L. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.03.065 · Cited by 114 · see citing works
The Principle of Inverse Effectiveness in Multisensory Integration: Some Statistical Considerations
Holmes, N. P. (2009) · Brain Topography · doi:10.1007/s10548-009-0097-2 · Cited by 110 · see citing works
Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects
Stevenson, R. A., Geoghegan, M. L., & James, T. W. (2006) · Experimental Brain Research · doi:10.1007/s00221-006-0770-6 · Cited by 104 · see citing works
How the Blind “See” Braille: Lessons From Functional Magnetic Resonance Imaging
Sadato, N. (2005) · The Neuroscientist · doi:10.1177/1073858405277314 · Cited by 95 · see citing works
Cortical Plasticity of Audio-Visual Object Representations
Naumer, M. J., Doehrmann, O., Müller, N., Muckli, L. F., Kaiser, J., & Hein, G. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn200 · Cited by 80 · see citing works
Multisensory integration for timing engages different brain networks
Dhamala, M., Assisi, C. G., Jirsa, V., Steinberg, F. L., & Kelso, J. A. S. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.07.044 · Cited by 77 · see citing works
Function and Connectivity in Human Primary Auditory Cortex: A Combined fMRI and DTI Study at 3 Tesla
Upadhyay, J., Ducros, M. G., Knaus, T. A., Lindgren, K. A., Silver, A. M., Tager-Flusberg, H. B., & Kim, D. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl150 · Cited by 68 · see citing works
An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI
Stevenson, R. A., Kim, S., & James, T. W. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1783-8 · Cited by 68 · see citing works
fMR-adaptation indicates selectivity to audiovisual content congruency in distributed clusters in human superior temporal cortex
van Atteveldt, N., Blau, V. C., Blomert, L., & Goebel, R. (2010) · BMC Neuroscience · doi:10.1186/1471-2202-11-11 · Cited by 64 · see citing works
Multisensory Enhancement in the Optic Tectum of the Barn Owl: Spike Count and Spike Timing
Zahar, Y., Reches, A., & Gutfreund, Y. (2009) · Journal of Neurophysiology · doi:10.1152/jn.91193.2008 · Cited by 56 · see citing works
Utilizing the ventriloquism-effect to investigate audio-visual binding
Bischoff, M., Walter, B., Blecker, C. R., Morgen, K., Vaitl, D., & Sammer, G. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.03.008 · Cited by 53 · see citing works
Audio-visual crossmodal interactions in environmental perception: an fMRI investigation
Belardinelli, M. O., Sestieri, C., Matteo, R. D., Delogu, F., Gratta, C. D., Ferretti, A., Caulo, M., Tartaro, A., & Romani, G. L. (2004) · Cognitive Processing · doi:10.1007/s10339-004-0024-0 · Cited by 50 · see citing works
Retinotopic effects during spatial audio-visual integration
Meienbrock, A., Naumer, M. J., Doehrmann, O., Singer, W., & Muckli, L. F. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.05.018 · Cited by 45 · see citing works
The Integration of Colour and Motion by the Human Visual Brain
Self, M. W., & Zeki, S. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhi010 · Cited by 38 · see citing works
Negative BOLD in Sensory Cortices During Verbal Memory: A Component in Generating Internal Representations?
Azulay, H., Striem-Amit, E., & Amedi, A. (2009) · Brain Topography · doi:10.1007/s10548-009-0089-2 · Cited by 35 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

Decoding sound and imagery content in early visual cortex
Vetter, P., Smith, F. W., & Muckli, L. (2014) · Current Biology · doi:10.1016/j.cub.2014.04.020 · Cited by 323 · see citing works
Tonotopic mapping of human auditory cortex
Sàenz, M., & Langers, D. R. M. (2013) · Hearing Research · doi:10.1016/j.heares.2013.07.016 · Cited by 222 · see citing works
Functional connectivity of visual cortex in the blind follows retinotopic organization principles
Striem-Amit, E., Ovadia-Caro, S., Caramazza, A., Margulies, D. S., Villringer, A., & Amedi, A. (2015) · Brain · doi:10.1093/brain/awv083 · Cited by 194 · see citing works
Self-citation
Mapping the Tonotopic Organization in Human Auditory Cortex with Minimally Salient Acoustic Stimulation
Langers, D. R. M., & van Dijk, P. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr282 · Cited by 126 · see citing works
Intranasal oxytocin enhances connectivity in the neural circuitry supporting social motivation and social perception in children with autism
Gordon, I., Jack, A., Pretzsch, C. M., Vander Wyk, B., Leckman, J. F., Feldman, R., & Pelphrey, K. A. (2016) · Scientific Reports · doi:10.1038/srep35054 · Cited by 114 · see citing works
Contextual modulation of primary visual cortex by auditory signals
Petro, L. S., Paton, A., & Muckli, L. F. (2017) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2016.0104 · Cited by 97 · see citing works
Negative blood oxygenation level dependent homunculus and somatotopic information in primary motor cortex and supplementary motor area
Zeharia, N., Hertz, U., Flash, T., & Amedi, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1119125109 · Cited by 83 · see citing works
Self-citation
New Whole-Body Sensory-Motor Gradients Revealed Using Phase-Locked Analysis and Verified Using Multivoxel Pattern Analysis and Functional Connectivity
Zeharia, N., Hertz, U., Flash, T., & Amedi, A. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.4246-14.2015 · Cited by 57 · see citing works
Self-citation
Brain Plasticity Can Predict the Cochlear Implant Outcome in Adult-Onset Deafness
Han, J.-H., Lee, H., Kang, H., Oh, S. H., & Lee, D. S. (2019) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2019.00038 · Cited by 49 · see citing works
Intracortical depth analyses of frequency-sensitive regions of human auditory cortex using 7T fMRI
Ahveninen, J., Chang, W., Huang, S., Keil, B. R., Kopčo, N., Rossi, S., Bonmassar, G., Witzel, T., & Polimeni, J. R. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.09.010 · Cited by 45 · see citing works
Positive and Negative Somatotopic BOLD Responses in Contralateral Versus Ipsilateral Penfield Homunculus
Tal, Z., Geva, R., & Amedi, A. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx024 · Cited by 44 · see citing works
Self-citation
Audiovisual synchrony enhances BOLD responses in a brain network including multisensory STS while also enhancing target‐detection performance for both modalities
Agustus, J. L., Ruff, C. C., & Driver, J. (2011) · Human Brain Mapping · doi:10.1002/hbm.21278 · Cited by 42 · see citing works
Increased functional connectivity between language and visually deprived areas in late and partial blindness
Sabbah, N., Authié, C., Sanda, N., Mohand‐Saïd, S., Sahel, J., Safran, A. B., Habas, C., & Amedi, A. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.04.056 · Cited by 40 · see citing works
Self-citation
Reorganization of early visual cortex functional connectivity following selective peripheral and central visual loss
Sabbah, N., Sanda, N., Authié, C., Mohand‐Saïd, S., Sahel, J., Habas, C., Amedi, A., & Safran, A. B. (2017) · Scientific Reports · doi:10.1038/srep43223 · Cited by 38 · see citing works
Self-citation
Speech-Driven Spectrotemporal Receptive Fields Beyond the Auditory Cortex
Venezia, J. H., Richards, V. M., & Hickok, G. S. (2021) · Hearing Research · doi:10.1016/j.heares.2021.108307 · Cited by 37 · see citing works
Parameters of semantic multisensory integration depend on timing and modality order among people on the autism spectrum: evidence from event-related potentials
Russo, N., Mottron, L., Burack, J. A., & Jemel, B. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2012.05.003 · Cited by 35 · see citing works
Spoken language processing activates the primary visual cortex
Seydell‐Greenwald, A., Wang, X., Newport, E. L., Bi, Y., & Striem-Amit, E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289671 · Cited by 36 · see citing works
Multiple sensitivity profiles to diversity and transition structure in non-stationary input
Tobia, M. J., Iacovella, V., & Hasson, U. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.041 · Cited by 33 · see citing works
Cortical systems for local and global integration in discourse comprehension
Egidi, G., & Caramazza, A. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.01.003 · Cited by 33 · see citing works
Comparison of auditory spatial bisection and minimum audible angle in front, lateral, and back space
Aggius-Vella, E., Kolarik, A. J., Gori, M., Cirstea, S., Campus, C., Moore, B. C. J., & Pardhan, S. (2020) · Scientific Reports · doi:10.1038/s41598-020-62983-z · Cited by 27 · see citing works
A Whole-Body Sensory-Motor Gradient is Revealed in the Medial Wall of the Parietal Lobe
Zeharia, N., Hofstetter, S., Flash, T., & Amedi, A. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.0727-18.2019 · Cited by 25 · see citing works
Self-citation
Flexibility and Stability in Sensory Processing Revealed Using Visual-to-Auditory Sensory Substitution
Hertz, U., & Amedi, A. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhu010 · Cited by 22 · see citing works
Self-citation
Multisensory integration of redundant trisensory stimulation
Hagmann, C. E., & Russo, N. N. (2016) · Attention Perception & Psychophysics · doi:10.3758/s13414-016-1192-6 · Cited by 21 · see citing works
Audio-visual stimulation for visual compensatory functions in stroke survivors with visual field defect: a systematic review
Alwashmi, K., Meyer, G. F., & Rowe, F. J. (2022) · Neurological Sciences · doi:10.1007/s10072-022-05926-y · Cited by 20 · see citing works
Auditory-Cortex Short-Term Plasticity Induced by Selective Attention
Jääskeläinen, I. P., & Ahveninen, J. (2014) · Neural Plasticity · doi:10.1155/2014/216731 · Cited by 20 · see citing works
Neural correlates of audiovisual speech processing in a second language
Barrós‐Loscertales, A., Ventura‐Campos, N., Visser, M., Alsius, A., Pallier, C., Rivera, C. Á., & Soto‐Faraco, S. (2013) · Brain and Language · doi:10.1016/j.bandl.2013.05.009 · Cited by 19 · see citing works
Tonotopic organisation of the auditory cortex in sloping sensorineural hearing loss
Wolak, T. P., Cieśla, K., Lorens, A., Kochanek, K., Lewandowska, M., Rusiniak, M., Pluta, A., Wójcik, J., & Skarżyńśki, H. (2017) · Hearing Research · doi:10.1016/j.heares.2017.09.012 · Cited by 19 · see citing works
Audio-visual synchrony and spatial attention enhance processing of dynamic visual stimulation independently and in parallel: A frequency-tagging study
Čović, A., Keitel, C., Porcu, E., Schröger, E., & Müller, M. M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.022 · Cited by 19 · see citing works
Neuroimaging paradigms for tonotopic mapping (I): The influence of sound stimulus type
Langers, D. R. M., Krumbholz, K., Bowtell, R. W., & Hall, D. A. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.07.044 · Cited by 19 · see citing works
Neuroplasticity and crossmodal connectivity in the normal, healthy brain.
Karim, A. K. M. R., Proulx, M. J., de Sousa, A. A., & Likova, L. T. (2021) · Psychology & Neuroscience · doi:10.1037/pne0000258 · Cited by 20 · see citing works
Speech-Driven Spectrotemporal Receptive Fields Beyond the Auditory CortexPreprint
Venezia, J. H., Richards, V. M., & Hickok, G. S. (2021) · doi:10.31234/osf.io/nfh26 · Cited by 15 · see citing works
Auditory-visual convergence at the superior colliculus in rat using functional MRI
Lau, C., Manno, F. A. M., Dong, C. M., Chan, K. C., & Wu, E. X. (2018) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8513633 · Cited by 9 · see citing works
Human Navigation Without and With Vision - the Role of Visual Experience and Visual RegionsPreprint
Maidenbaum, S., Chebat, D., & Amedi, A. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/480558 · Cited by 9 · see citing works
Self-citation
Scaling effects in crossmodal improvement of visual perception by motor system stimulus
Gonzalo, I., & Porras, M. A. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.06.047 · Cited by 7 · see citing works
EFFECT OF SOUND INTENSITY ON LEVEL OF ACTIVATION IN AUDITORY CORTEX AS MEASURED BY FMRI
Wolak, T. P., Cieśla, K., Wójcik, J., & Skarżyńśki, H. (2017) · Journal of Hearing Science · doi:10.17430/1002807 · Cited by 2 · see citing works
Scaling Effects in Crossmodal Improvement of Visual Perception
Gonzalo, I., & Porras, M. A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21326-7_29 · Cited by 2 · see citing works
Modelling the compression of perceived time by a cerebral system with nervous excitability deficit and near the perception threshold
Gonzalo, I., & Porras, M. A. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.05.087 · Cited by 1 · see citing works
Deficitary Nervous Excitability and Subjective Contraction of Time: Time-Dispersive Model
Gonzalo, I., & Porras, M. A. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-38637-4_38 · Cited by 1 · see citing works
Report Decoding Sound and Imagery Content in Early Visual Cortex
Vetter, P., Smith, F. W., & Muckli, L. F. (2014) · Cited by 1 · see citing works
Justo Gonzalo (1910-1986) y su investigación sobre dinámica cerebral
Fonrodona, I. G. (2011) · Revista de historia de la psicología
fMRI Dependent Components Analysis Reveals Dynamic Relations Between Functional Large Scale Cortical NetworksPreprint
Hertz, U., Zoran, D., Weiss, Y., & Amedi, A. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/066282
Self-citation
Correlations between features of event-related potentials and Autism Spectrum Quotient scores
Wang, G., Kino, M., Yamauchi, K., & Wang, R. H. (2018) · Journal of Clinical Neuroscience · doi:10.1016/j.jocn.2018.10.070
Nervous Excitability dynamics in a multisensory syndrome and its similitude with a normal state. Scaling LawsPreprint
Gonzalo, I., & Porras, M. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01666

The Neural Correlates and Mechanisms Mediating the Integration of Auditory and Visual Information in the Human Brain
Werner, S. (2010) · MPG.PuRe (Max Planck Society)
Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses
Giani, A. S., O, B. P. E., Belardinelli, P., Kleiner, M., Preissl, H., & Noppeney, U. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.114 · Cited by 57 · see citing works
Top-down attention regulates the neural expression of audiovisual integration
Fernández, L. M., Visser, M., Ventura‐Campos, N., Ávila, C., & Soto‐Faraco, S. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.06.052 · Cited by 50 · see citing works
Multisensory Integration Is Modulated by Auditory Sound Frequency and Visual Spatial Frequency
Green, J. J., Pierce, A. M., & Adams, S. L. M. (2019) · Multisensory Research · doi:10.1163/22134808-20191402 · Cited by 3 · see citing works
An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI
Stevenson, R. A., Kim, S., & James, T. W. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1783-8 · Cited by 68 · see citing works

Authors: Hertz, U., & Amedi, A.
Affiliations: Hebrew University of Jerusalem (IL)
Corresponding author: Amedi, Amir
ORCID: Hertz, Uri
Amedi, Amir
Venue: NeuroImage
Publisher: Elsevier BV
Published: 2010-08-15
Volume: 52
Issue: 2
Pages: 617-632
ISSN: 1053-8119 (ISSN-L)
DOI: 10.1016/j.neuroimage.2010.04.186
OpenAlex: W2143708988
Cited by: 43
References: 112
Citations by year: 2023: 1 · 2022: 1 · 2021: 3 · 2020: 2 · 2019: 2 · 2018: 3 · 2017: 6 · 2016: 5 · 2015: 2 · 2014: 6 · 2013: 4 · 2012: 4
FWCI: 5.02
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: NeuroImage (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords: audiovisual integration, multisensory integration, primary auditory cortex, primary visual cortex, retinotopic organization, precuneus, calcarine sulcus, interaction frequency, division of labor
MeSH terms: Acoustic Stimulation, Adult, Auditory Pathways, Auditory Perception, Brain, Cochlea, Female, Fourier Analysis, Humans, Magnetic Resonance Imaging, Male, Neuropsychological Tests, Photic Stimulation, Retina, Signal Processing, Computer-Assisted, Visual Pathways, Visual Perception, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Disentangling unisensory and multisensory components in audiovisual integration using a novel multifrequency fMRI spectral analysis

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks