Multisensory Processes: A Balancing Act across the Lifespan
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Multisensory processes are fundamental in scaffolding perception, cognition, learning, and behavior. How and when stimuli from different sensory modalities are integrated rather than treated as separate entities is poorly understood. We review how the relative reliance on stimulus characteristics versus learned associations dynamically shapes multisensory processes. We illustrate the dynamism in multisensory function across two timescales: one long term that operates across the lifespan and one short term that operates during the learning of new multisensory relations. In addition, we highlight the importance of task contingencies. We conclude that these highly dynamic multisensory processes, based on the relative weighting of stimulus characteristics and learned associations, provide both stability and flexibility to brain functions over a wide range of temporal scales.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · cited 1×
“This traditional view has recently been complemented by evidence of a surprising degree of connectivity between areas traditionally deemed ‘unisensory’ (Figure 1) [83–87] (reviewed in [8,9]).”
BackgroundCross-modal plasticity: where and how?
Bavelier, D., & Neville, H. J. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn848 · Cited by 1,040 · cited 2×
“Extending this further, deaf individuals can learn to watch mouth movements to understand speech by lip reading, which results in extensive cross-modal plasticity within regions of the cerebral cortex [4].”
BackgroundThe bimodal perception of speech in infancy
Kuhl, P. K., & Meltzoff, A. N. (1982) · Science · doi:10.1126/science.7146899 · Cited by 919 · cited 1×
“For instance, starting at birth and continuing into the first several months of postnatal life, infants exhibit the ability to detect the synchrony of auditory and visual non-speech and speech information [32–34] as well as the equivalence of isolated visible and audible speech syllables and non-native faces and vocal…”
BackgroundAnatomical evidence of multimodal integration in primate striate cortex
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-13-05749.2002 · Cited by 892
The multifaceted interplay between attention and multisensory integration.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.06.008 · Cited by 828
Neural reorganization following sensory loss: the opportunity of change
Merabet, L. B., & Pascual-Leone, A. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2758 · Cited by 751
Visuo-haptic object-related activation in the ventral visual pathway
Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001) · Nature Neuroscience · doi:10.1038/85201 · Cited by 630 · cited 2×
“Furthermore, individuals do not activate the LO cortex when shape rendering is not required, such as when listening to the same sounds and performing an arbitrary auditory association naming task or performing an auditory object recognition task [67].”
BackgroundSelf-citationMultisensory convergence in calcarine visual areas in macaque monkey
Rockland, K. S., & Ojima, H. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00121-1 · Cited by 564
Intersensory redundancy guides attentional selectivity and perceptual learning in infancy
Bahrick, L. E., & Lickliter, R. (2000) · Developmental Psychology · doi:10.1037/0012-1649.36.2.190 · Cited by 554 · cited 1×
“For instance, starting at birth infants detect low-level, rudimentary multisensory correspondences based on simple cues such as intensity [27] and temporal synchrony [28,29] and fail to detect the specific multisensory identity of objects and events.”
BackgroundShape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex
Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., Rotman, S., Hemond, C., Meijer, P., & Pascual-Leone, A. (2007) · Nature Neuroscience · doi:10.1038/nn1912 · Cited by 451
Self-citationA ventral visual stream reading center independent of visual experience
Reich, L., Szwed, M., Cohen, L., & Amedi, A. (2011) · Current Biology · doi:10.1016/j.cub.2011.01.040 · Cited by 295
Self-citationThe multisensory function of the human primary visual cortex
Murray, M. M., Thelen, A., Thut, G., Romei, V., Martuzzi, R., & Matusz, P. J. (2016) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2015.08.011 · Cited by 219 · cited 1×
“This traditional view has recently been complemented by evidence of a surprising degree of connectivity between areas traditionally deemed ‘unisensory’ (Figure 1) [83–87] (reviewed in [8,9]).”
BackgroundSelf-citationFunctional connectivity of visual cortex in the blind follows retinotopic organization principles
Striem-Amit, E., Ovadia-Caro, S., Caramazza, A., Margulies, D. S., Villringer, A., & Amedi, A. (2015) · Brain · doi:10.1093/brain/awv083 · Cited by 194
Self-citationVisual experience is necessary for the development of multisensory integration
Wallace, M. T., Perrault, T. J., Hairston, W. D., & Stein, B. E. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.2535-04.2004 · Cited by 183
Self-citationMatching phonetic information in lips and voice is robust in 4.5-month-old infants
Patterson, M. L., & Werker, J. F. (1999) · Infant Behavior and Development · doi:10.1016/s0163-6383(99)00003-x · Cited by 177 · cited 1×
“For instance, starting at birth and continuing into the first several months of postnatal life, infants exhibit the ability to detect the synchrony of auditory and visual non-speech and speech information [32–34] as well as the equivalence of isolated visible and audible speech syllables and non-native faces and vocal…”
BackgroundInfant perception of audio-visual speech synchrony
Lewkowicz, D. J. (2010) · Developmental Psychology · doi:10.1037/a0015579 · Cited by 168 · cited 1×
“Furthermore, it has been shown that the TBW differs for different stimuli and tasks in infants [32,33], children [49], and adults [45], suggesting a dependence not only on physical stimulus characteristics but also on learned associations.”
BackgroundSelf-citationRicciardi, E., Bonino, D., Pellegrini, S., & Pietrini, P. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.10.006 · Cited by 154
Cortical activity during tactile exploration of objects in blind and sighted humans
Amedi, A., Raz, N., Azulay, H., Malach, R., & Zohary, E. (2010) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-2010-0503 · Cited by 125 · cited 1×
“This evidence comes from observations that both congenitally blind and sighted individuals activate the LO cortex in response to sounds, but only if these individuals learn to extract geometrical shapes from those sounds such as when using a visual-to-auditory SSD or when using touch to recognize objects [67,78].”
BackgroundDiffusion tensor imaging shows white matter tracts between human auditory and visual cortex
Beer, A. L., Plank, T., & Greenlee, M. W. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2715-y · Cited by 124
Looming Signals Reveal Synergistic Principles of Multisensory Integration
Cappe, C., Thelen, A., Romei, V., Thut, G., & Murray, M. M. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5517-11.2012 · Cited by 107
Self-citationWang, X., Peelen, M. V., Han, Z., He, C., Caramazza, A., & Bi, Y. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3914-14.2015 · Cited by 68
Beer, A. L., Plank, T., Meyer, G., & Greenlee, M. W. (2013) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2013.00005 · Cited by 53
Cited by
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Abstract Concepts and the Embodied Mind: Rethinking Grounded Cognition
Dove, G. (2022) · doi:10.1093/oso/9780190061975.001.0001 · Cited by 40
Multisensory Facilitation of Working Memory Training
Pahor, A., Collins, C., Smith-Peirce, R. N., Moon, A., Stavropoulos, T., Silva, I., Peng, E., Jaeggi, S. M., & Seitz, A. R. (2021) · Journal of Cognitive Enhancement · doi:10.1007/s41465-020-00196-y · Cited by 17
Metadata
- Authors
- Murray, M. M., Lewkowicz, D. J., Amedi, A., & Wallace, M. T.
- Venue
- Trends in Neurosciences
- Publisher
- Elsevier BV
- Published
- 2016-08
- Volume
- 39
- Issue
- 8
- Pages
- 567-579
- ISSN
- 0166-2236 (ISSN-L)
- OpenAlex
- W2411468300
- Cited by
- 240
- References
- 23
- FWCI
- 3.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.