Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools
Open accessAbstract
Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
pSTS is implicated in audiovisual integration and has been shown to feed down information to primary visual and auditory cortices.
“pSTS is implicated in audiovisual integration and has been shown to feed down information to primary visual and auditory cortices [26].”
From Decoding sound and imagery content in early visual cortex (2014) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Co-Planar Stereotaxic Atlas of the Human Brain
Talairach, J., & Tournoux, P. (1988) · Medical Entomology and Zoology · Cited by 11,798 · see citing works
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986 · see citing works
Co-planar stereotaxic atlas of the human brain
Laitinen, L. (1989) · Clinical Neurology and Neurosurgery · doi:10.1016/0303-8467(89)90128-5 · Cited by 7,565 · see citing works
Functional connectivity in the resting brain: a network analysis of the default mode hypothesis
Greicius, M. D., Krasnow, B., Reiss, A. L., & Menon, V. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0135058100 · Cited by 6,358 · see citing works
Fast and robust fixed-point algorithms for independent component analysis
Hyvärinen, A. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.761722 · Cited by 6,239 · see citing works
Friston, K. J., Harrison, L., & Penny, W. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00202-7 · Cited by 4,583 · see citing works
Greicius, M. D., Srivastava, G., Reiss, A. L., & Menon, V. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0308627101 · Cited by 3,684 · see citing works
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202 · see citing works
Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1
Boynton, G., Engel, S., Glover, G., & Heeger, D. (1996) · Journal of Neuroscience · doi:10.1523/jneurosci.16-13-04207.1996 · Cited by 2,522 · see citing works
Functional Connectivity: The Principal-Component Analysis of Large (PET) Data Sets
Friston, K. J., Frith, C., Liddle, P., & Frackowiak, R. (1993) · Journal of Cerebral Blood Flow and Metabolism · doi:10.1038/jcbfm.1993.4 · Cited by 2,094 · see citing works
(1994) · Choice Reviews Online · doi:10.5860/choice.31-4365 · Cited by 1,897 · see citing works
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · see citing works
Validating the independent components of neuroimaging time series via clustering and visualization
Himberg, J., Hyvärinen, A., & Esposito, F. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.03.027 · Cited by 1,246 · see citing works
A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data
Calhoun, V. D., Liu, J., & Adalı, T. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.057 · Cited by 1,148 · see citing works
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012 · see citing works
Driver, J., & Noesselt, T. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.013 · Cited by 940 · see citing works
Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus
Beauchamp, M. S., Lee, K. E., Argall, B., & Martin, A. D. (2004) · Neuron · doi:10.1016/s0896-6273(04)00070-4 · Cited by 765 · see citing works
Tensorial extensions of independent component analysis for multisubject FMRI analysis
Beckmann, C. F., & Smith, S. M. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.10.043 · Cited by 748 · see citing works
Calhoun, V. D., Adalı, T., Pearlson, G. D., & Pekar, J. J. (2001) · Human Brain Mapping · doi:10.1002/hbm.1024 · Cited by 697 · see citing works
Marreiros, A. C., Stephan, K. E., & Friston, K. J. (2010) · Scholarpedia · doi:10.4249/scholarpedia.9568 · Cited by 565 · see citing works
Integration of Letters and Speech Sounds in the Human Brain
van Atteveldt, N., Formisano, E., Goebel, R., & Blomert, L. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.025 · Cited by 545 · see citing works
van de Ven, V. G., Formisano, E., Prvulovic, D., Röder, C. H., & Linden, D. E. J. (2004) · Human Brain Mapping · doi:10.1002/hbm.20022 · Cited by 538 · see citing works
Multisensory contributions to low-level, ‘unisensory’ processing
Schroeder, C. E., & Foxe, J. J. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.06.008 · Cited by 506 · see citing works
Assessing functional connectivity in the human brain by fMRI
Rogers, B. P., Morgan, V. L., Newton, A. T., & Gore, J. C. (2007) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2007.03.007 · Cited by 477 · see citing works
See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex
Beauchamp, M. S. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.03.011 · Cited by 403 · see citing works
Multisensory integration: psychophysics, neurophysiology, and computation
Angelaki, D. E., Gu, Y., & DeAngelis, G. C. (2009) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2009.06.008 · Cited by 383 · see citing works
The effect of model order selection in group PICA
Abou‐Elseoud, A., Starck, T., Remes, J. J., Nikkinen, J., Tervonen, O. A., & Kiviniemi, V. J. (2010) · Human Brain Mapping · doi:10.1002/hbm.20929 · Cited by 362 · see citing works
Independent component analysis of fMRI group studies by self-organizing clustering
Esposito, F., Scarabino, T., Hyvärinen, A., Himberg, J., Formisano, E., Comani, S., Tedeschi, G., Goebel, R., Seifritz, E., & Salle, F. D. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.10.042 · Cited by 357 · see citing works
Functional imaging of human crossmodal identification and object recognition
Amedi, A., von Kriegstein, K., van Atteveldt, N., Beauchamp, M. S., & Naumer, M. J. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2396-5 · Cited by 351 · see citing works
Self-citationDo early sensory cortices integrate cross-modal information?
Kayser, C., & Logothetis, N. (2007) · Brain Structure and Function · doi:10.1007/s00429-007-0154-0 · Cited by 319 · see citing works
Noesselt, T., Rieger, J. W., Schoenfeld, M. A., Kanowski, M., Hinrichs, H., Heinze, H., & Driver, J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.2252-07.2007 · Cited by 282 · see citing works
Noise Reduction in BOLD-Based fMRI Using Component Analysis
Thomas, C. G., Harshman, R. A., & Menon, R. S. (2002) · NeuroImage · doi:10.1006/nimg.2002.1200 · Cited by 280 · see citing works
Multisensory Interactions within Human Primary Cortices Revealed by BOLD Dynamics
Martuzzi, R., Murray, M. M., Michel, C. M., Thiran, J., Maeder, P. P., Clarke, S., & Meuli, R. A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl077 · Cited by 279 · see citing works
Statistical Criteria in fMRI Studies of Multisensory Integration
Beauchamp, M. S. (2005) · Neuroinformatics · doi:10.1385/ni:3:2:093 · Cited by 227 · see citing works
Semantics and the multisensory brain: How meaning modulates processes of audio-visual integration
Doehrmann, O., & Naumer, M. J. (2008) · Brain Research · doi:10.1016/j.brainres.2008.03.071 · Cited by 226 · see citing works
Self-citationHein, G., Doehrmann, O., Müller, N. G., Kaiser, J., Muckli, L. F., & Naumer, M. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1740-07.2007 · Cited by 203 · see citing works
Self-citationWerner, S., & Noppeney, U. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.5091-09.2010 · Cited by 197 · see citing works
Calhoun, V. D., Adalı, T., & Pekar, J. J. (2004) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2004.09.004 · Cited by 173 · see citing works
The Effect of Prior Visual Information on Recognition of Speech and Sounds
Noppeney, U., Josephs, O., Hocking, J., Price, C. J., & Friston, K. J. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm091 · Cited by 172 · see citing works
Werner, S., & Noppeney, U. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhp248 · Cited by 168 · see citing works
Multisensory Processing in Sensory-Specific Cortical Areas
Macaluso, E. (2006) · The Neuroscientist · doi:10.1177/1073858406287908 · Cited by 165 · see citing works
Perceptual Decisions Formed by Accumulation of Audiovisual Evidence in Prefrontal Cortex
Noppeney, U., Ostwald, D., & Werner, S. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0455-10.2010 · Cited by 157 · see citing works
Eckert, M. A., Kamdar, N. V., Chang, C. E., Beckmann, C. F., Greicius, M. D., & Menon, V. (2008) · Human Brain Mapping · doi:10.1002/hbm.20560 · Cited by 138 · see citing works
Towards natural stimulation in fMRI—Issues of data analysis
Malinen, S., Hlushchuk, Y., & Hari, R. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.11.015 · Cited by 128 · see citing works
Sadaghiani, S., Maier, J. X., & Noppeney, U. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5437-08.2009 · Cited by 117 · see citing works
Activity patterns in human motion-sensitive areas depend on the interpretation of global motion
Castelo-Branco, M. S., Formisano, E., Backes, W. H., Zanella, F. E., Neuenschwander, S., Singer, W., & Goebel, R. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.202049999 · Cited by 116 · see citing works
Self-citationThe Role of the Posterior Superior Temporal Sulcus in Audiovisual Processing
Hocking, J., & Price, C. J. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn007 · Cited by 116 · see citing works
Multisensory Object Perception in the Primate Brain
Kaiser, J., & Naumer, M. J. (2010) · Springer eBooks · doi:10.1007/978-1-4419-5615-6 · Cited by 107 · see citing works
Self-citationThe chronoarchitecture of the cerebral cortex
Bartels, A., & Zeki, S. (2005) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2005.1627 · Cited by 107 · see citing works
Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects
Stevenson, R. A., Geoghegan, M. L., & James, T. W. (2006) · Experimental Brain Research · doi:10.1007/s00221-006-0770-6 · Cited by 104 · see citing works
Baier, B., Kleinschmidt, A. K., & Müller, N. G. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.1457-06.2006 · Cited by 98 · see citing works
Cortical Plasticity of Audio-Visual Object Representations
Naumer, M. J., Doehrmann, O., Müller, N., Muckli, L. F., Kaiser, J., & Hein, G. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn200 · Cited by 80 · see citing works
Self-citationvan de Ven, V., Esposito, F., & Christoffels, I. K. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.05.057 · Cited by 72 · see citing works
Stevenson, R. A., Kim, S., & James, T. W. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1783-8 · Cited by 68 · see citing works
Estimation of the intrinsic dimensionality of fMRI data
Cordes, D., & Nandy, R. R. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.07.054 · Cited by 65 · see citing works
van Atteveldt, N., Blau, V. C., Blomert, L., & Goebel, R. (2010) · BMC Neuroscience · doi:10.1186/1471-2202-11-11 · Cited by 64 · see citing works
Power spectrum ranked independent component analysis of a periodic fMRI complex motor paradigm
Moritz, C. H., Rogers, B. P., & Meyerand, M. E. (2002) · Human Brain Mapping · doi:10.1002/hbm.10081 · Cited by 64 · see citing works
Meyer, G. F., Greenlee, M. W., & Wuerger, S. M. (2010) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21593 · Cited by 52 · see citing works
Audio-visual crossmodal interactions in environmental perception: an fMRI investigation
Belardinelli, M. O., Sestieri, C., Matteo, R. D., Delogu, F., Gratta, C. D., Ferretti, A., Caulo, M., Tartaro, A., & Romani, G. L. (2004) · Cognitive Processing · doi:10.1007/s10339-004-0024-0 · Cited by 50 · see citing works
Retinotopic effects during spatial audio-visual integration
Meienbrock, A., Naumer, M. J., Doehrmann, O., Singer, W., & Muckli, L. F. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.05.018 · Cited by 45 · see citing works
Self-citationDoehrmann, O., Weigelt, S., Altmann, C. F., Kaiser, J., & Naumer, M. J. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.5074-09.2010 · Cited by 42 · see citing works
Self-citationVisual target modulation of functional connectivity networks revealed by self‐organizing group ICA
van de Ven, V. G., Bledowski, C., Prvulovic, D., Goebel, R., Formisano, E., Salle, F. D., Linden, D. E. J., & Esposito, F. (2007) · Human Brain Mapping · doi:10.1002/hbm.20479 · Cited by 36 · see citing works
Exploratory analysis of brain connectivity with ICA
Rajapakse, J. C., Tan, C. L., Zheng, X., Mukhopadhyay, S. C., & Yang, K. (2006) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/memb.2006.1607674 · Cited by 29 · see citing works
Liao, R., McKeown, M. J., & Krolik, J. L. (2006) · Magnetic Resonance in Medicine · doi:10.1002/mrm.20893 · Cited by 23 · see citing works
Audio-Visual Perception of Everyday Natural Objects – Hemodynamic Studies in Humans
Lewis, J. W. (2010) · Springer eBooks · doi:10.1007/978-1-4419-5615-6_10 · Cited by 16 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Decoding sound and imagery content in early visual cortex
Vetter, P., Smith, F. W., & Muckli, L. (2014) · Current Biology · doi:10.1016/j.cub.2014.04.020 · Cited by 323 · see citing works
Self-citationLooming Signals Reveal Synergistic Principles of Multisensory Integration
Cappe, C., Thelen, A., Romei, V., Thut, G., & Murray, M. M. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5517-11.2012 · Cited by 107 · see citing works
- Authors
- Naumer, M. J., van den Bosch, J. J. F., Wibral, M., Kohler, A., Singer, W., Kaiser, J., van de Ven, V., & Muckli, L.
- Affiliations
- Goethe University Frankfurt (DE)
- University Hospital of Bern (CH)
- Max Planck Institute for Brain Research (DE)
- University of Glasgow (GB)
- Corresponding author
- Naumer, Marcus J.
- Venue
- Experimental Brain Research
- Published
- 2011-04-19
- Volume
- 213
- Issue
- 2-3
- Pages
- 309-320
- ISSN
- 0014-4819 (ISSN-L)
- 1432-1106
- PMC
- PMC3155044
- OpenAlex
- W2017527500
- Cited by
- 11
- References
- 66
- Citations by year
- 2025: 1 · 2021: 1 · 2017: 1 · 2015: 2 · 2014: 3 · 2012: 2
- FWCI
- 2.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- Experimental Brain Research (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, cc-by-nc)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by-nc)
- Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) (submittedVersion, cc-by-nc)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Open Access CRIS of the University of Bern
- Funders
- Bundesministerium für Bildung und Forschung (01GO0508)
- Max-Planck-Gesellschaft
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- audiovisual integration, multisensory integration, spatial independent component analysis, functional brain networks, auditory cortex, visual cortex, ventral occipitotemporal cortex, posterior parietal cortex, prefrontal cortex, superior temporal cortex, hypothesis generation, hypothesis testing, region of interest definition, functional connectivity
- MeSH terms
- Acoustic Stimulation, Adult, Auditory Perception, Brain, Brain Mapping, Female, Humans, Image Processing, Computer-Assisted, Functional Laterality, Magnetic Resonance Imaging, Male, Models, Neurological, Oxygen, Pattern Recognition, Visual, Photic Stimulation, Principal Component Analysis, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
