The DOI matches its Crossref record.
The merging of the senses
Abstract
Bringing together neural, perceptual, and behavioral studies, The Merging of the Senses provides the first detailed review of how the brain assembles information from different sensory systems in order to produce a coherent view of the external world. Stein and Meredith marshall evidence from a broad array of species to show that interactions among senses are the most ancient scheme of sensory organization, an integrative system reflecting a general plan that supersedes structure and species. Most importantly, they explore what is known about the neural processes by which interactions among the senses take place at the level of the single cell.The authors draw on their own experiments to illustrate how sensory inputs converge (from visual, auditory, and somatosensory modalities, for instance) on individual neurons in different areas of the brain, how these neurons integrate their inputs, the principles by which this integration occurs, and what this may mean for perception and behavior. Neurons in the superior colliculus and cortex are emphasized as models of multiple sensory integrators.Barry E. Stein is Professor of Physiology and M. Alex Meredith is Associate Professor of Anatomy, both at the Medical College of Virginia, Virginia Commonwealth University.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Benefits of multisensory learning
Shams, L., & Seitz, A. R. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.07.006 · Cited by 1,178
Identification and classification of hubs in brain networks
Sporns, O., Honey, C. J., & Kötter, R. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001049 · Cited by 1,165
Giard, M. H., & Peronnet, F. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563544 · Cited by 1,072
Neuronal oscillations and multisensory interaction in primary auditory cortex
Lakatos, P., Chen, C.-M., O'Connell, M. N., Mills, A., & Schroeder, C. E. (2007) · Neuron · doi:10.1016/j.neuron.2006.12.011 · Cited by 961
Driver, J., & Noesselt, T. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.013 · Cited by 940
Anatomical evidence of multimodal integration in primate striate cortex
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-13-05749.2002 · Cited by 892
The multifaceted interplay between attention and multisensory integration.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.06.008 · Cited by 828
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
Odor/taste integration and the perception of flavor
Small, D. M., & Prescott, J. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2376-9 · Cited by 712
The multisensory perception of flavor
Auvray, M., & Spence, C. (2008) · Consciousness and Cognition · doi:10.1016/j.concog.2007.06.005 · Cited by 685
Oscillatory synchronization in large-scale cortical networks predicts perception
Hipp, J. F., Engel, A. K., & Siegel, M. (2011) · Neuron · doi:10.1016/j.neuron.2010.12.027 · Cited by 599
Compensatory plasticity and sensory substitution in the cerebral cortex
Rauschecker, J. P. (1995) · Trends in Neurosciences · doi:10.1016/0166-2236(95)93948-w · Cited by 590
Multisensory integration and the body schema: Close to hand and within reach
Maravita, A., Spence, C., & Driver, J. (2003) · Current Biology · doi:10.1016/s0960-9822(03)00449-4 · Cited by 578
Multisensory convergence in calcarine visual areas in macaque monkey
Rockland, K. S., & Ojima, H. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00121-1 · Cited by 564
The impact of perceptual interactions on perceived flavor
Delwiche, J. (2004) · Food Quality and Preference · doi:10.1016/s0950-3293(03)00041-7 · Cited by 563
Neural correlates of multisensory cue integration in macaque MSTd
Gu, Y., Angelaki, D. E., & DeAngelis, G. C. (2008) · Nature Neuroscience · doi:10.1038/nn.2191 · Cited by 554
Chapter 27 The metamodal organization of the brain
Pascual-Leone, A., & Hamilton, R. (2001) · Progress in Brain Research · doi:10.1016/s0079-6123(01)34028-1 · Cited by 540
Motor Functions of the Superior Colliculus
Gandhi, N. J., & Katnani, H. A. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113728 · Cited by 490
A multilevel multimodal circuit enhances action selection in drosophila
Ohyama, T., Schneider-Mizell, C. M., Fetter, R. D., Aleman, J. V., Franconville, R., Rivera-Alba, M., Mensh, B. D., Branson, K. M., Simpson, J. H., Truman, J. W., et al. (2015) · Nature · doi:10.1038/nature14297 · Cited by 502
Topographic Maps are Fundamental to Sensory Processing
Kaas, J. H. (1997) · Brain Research Bulletin · doi:10.1016/s0361-9230(97)00094-4 · Cited by 478
Reference frames for representing visual and tactile locations in parietal cortex
Avillac, M., Denève, S., Olivier, E., Pouget, A., & Duhamel, J.-R. (2005) · Nature Neuroscience · doi:10.1038/nn1480 · Cited by 464
Audiovisual links in exogenous covert spatial orienting
Spence, C., & Driver, J. (1997) · Perception & Psychophysics · doi:10.3758/bf03206843 · Cited by 475
Multisensory perception: Beyond modularity and convergence
Driver, J., & Spence, C. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00740-5 · Cited by 465
The role of auditory cues in modulating the perceived crispness and staleness of potato chips
ZAMPINI, M., & SPENCE, C. (2004) · Journal of Sensory Studies · doi:10.1111/j.1745-459x.2004.080403.x · Cited by 463
Neural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459
Multisensory spatial interactions: a window onto functional integration in the human brain
Macaluso, E., & Driver, J. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.03.008 · Cited by 428
Talsma, D., Doty, T. J., & Woldorff, M. G. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhk016 · Cited by 434
The development of intersensory temporal perception: an epigenetic systems/limitations view
Lewkowicz, D. J. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.2.281 · Cited by 403
Sensory integration and the perceptual experience of persons with autism
Iarocci, G., & McDonald, J. (2006) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-005-0044-3 · Cited by 376
Baum, S. H., Stevenson, R. A., & Wallace, M. T. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.007 · Cited by 370
Temporal binding, binocular rivalry, and consciousness
Engel, A. K., Fries, P., König, P., Brecht, M., & Singer, W. (1999) · Consciousness and Cognition · doi:10.1006/ccog.1999.0389 · Cited by 379
Krienen, F. M., Yeo, B. T. T., & Buckner, R. L. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0526 · Cited by 372
Can evoking nature in advertising mislead consumers? The power of ‘executional greenwashing'
Parguel, B., Benoit-Moreau, F., & Russell, C. A. (2015) · International Journal of Advertising · doi:10.1080/02650487.2014.996116 · Cited by 364
BIOLOGICALLY BASED ARTIFICIAL NAVIGATION SYSTEMS: REVIEW AND PROSPECTS
TRULLIER, O., WIENER, S. I., BERTHOZ, A., & MEYER, J.-A. (1997) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00060-3 · Cited by 363
Bahrick, L. E., Lickliter, R., & Flom, R. (2004) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2004.00283.x · Cited by 341
Vection and visually induced motion sickness: how are they related?
Keshavarz, B., Riecke, B. E., Hettinger, L. J., & Campos, J. L. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00472 · Cited by 342
Instructed learning in the auditory localization pathway of the barn owl
Knudsen, E. I. (2002) · Nature · doi:10.1038/417322a · Cited by 320
Evaluating the Operations Underlying Multisensory Integration in the Cat Superior Colliculus
Stanford, T. R., Quessy, S., & Stein, B. E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.5095-04.2005 · Cited by 314
A direct projection from superior colliculus to substantia nigra for detecting salient visual events
Comoli, E., Coizet, V., Boyes, J., Bolam, J. P., Canteras, N. S., Quirk, R. H., Overton, P. G., & Redgrave, P. (2003) · Nature Neuroscience · doi:10.1038/nn1113 · Cited by 295
Developmental mechanisms of topographic map formation and alignment
Cang, J., & Feldheim, D. A. (2013) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062012-170341 · Cited by 275
Negative BOLD differentiates visual imagery and perception
Amedi, A., Malach, R., & Pascual-Leone, A. (2005) · Neuron · doi:10.1016/j.neuron.2005.10.032 · Cited by 271
Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey.
Avillac, M., Ben Hamed, S., & Duhamel, J.-R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2646-06.2007 · Cited by 281
Audio–visual simultaneity judgments
Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2005) · Perception & Psychophysics · doi:10.3758/bf03193329 · Cited by 280
Integration of Visual and Tactile Signals From the Hand in the Human Brain: An fMRI Study
Gentile, G., Petkova, V. I., & Ehrsson, H. H. (2011) · Journal of Neurophysiology · doi:10.1152/jn.00840.2010 · Cited by 274
Stevenson, R. A., Zemtsov, R. K., & Wallace, M. T. (2012) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/a0027339 · Cited by 269
Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision
Spence, C., Nicholls, M. E. R., Gillespie, N., & Driver, J. (1998) · Perception & Psychophysics · doi:10.3758/bf03206045 · Cited by 271
Peripersonal Space and Body Schema: Two Labels for the Same Concept?
Cardinali, L., Brozzoli, C., & Farnè, A. (2009) · Brain Topography · doi:10.1007/s10548-009-0092-7 · Cited by 262
Merker, B. (2005) · Consciousness and Cognition · doi:10.1016/s1053-8100(03)00002-3 · Cited by 251
Multisensory In-Car Warning Signals for Collision Avoidance
Ho, C., Reed, N., & Spence, C. (2007) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x249965 · Cited by 235
Maidenbaum, S., Abboud, S., & Amedi, A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.11.007 · Cited by 230
Bayesian multisensory integration and cross-modal spatial links.
Deneve, S., & Pouget, A. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.03.011 · Cited by 230
Chen, Y.-C., & Spence, C. (2010) · Cognition · doi:10.1016/j.cognition.2009.10.012 · Cited by 221
The multisensory function of the human primary visual cortex
Murray, M. M., Thelen, A., Thut, G., Romei, V., Martuzzi, R., & Matusz, P. J. (2016) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2015.08.011 · Cited by 219
Serino, A., & Haggard, P. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.04.004 · Cited by 212
Touch, sound and vision in human superior temporal sulcus
Beauchamp, M. S., Yasar, N. E., Frye, R. E., & Ro, T. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.015 · Cited by 208
Spence, C., Pavani, F., Maravita, A., & Holmes, N. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.03.008 · Cited by 205
Spatial Sensory Organization and Body Representation in Pain Perception
Haggard, P., Iannetti, G. D., & Longo, M. R. (2013) · Current Biology · doi:10.1016/j.cub.2013.01.047 · Cited by 192
Multisensory Integration of Natural Odors and Sounds in the Auditory Cortex
Cohen, L., Rothschild, G., & Mizrahi, A. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.019 · Cited by 192
Effect of background noise on food perception
Woods, A. T., Poliakoff, E., Lloyd, D. M., Kuenzel, J., Hodson, R., Gonda, H., Batchelor, J., Dijksterhuis, G. B., & Thomas, A. (2011) · Food Quality and Preference · doi:10.1016/j.foodqual.2010.07.003 · Cited by 192
Neuropsychology of Learning and Memory in Teleost Fish
Salas, C., Broglio, C., Durán, E., Gómez, A., Ocaña, F. M., Jiménez-Moya, F., & Rodríguez, F. (2006) · Zebrafish · doi:10.1089/zeb.2006.3.157 · Cited by 189
Perspectives on sensory processing disorder: a call for translational research
Miller, L. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.022.2009 · Cited by 183
Visual experience is necessary for the development of multisensory integration
Wallace, M. T., Perrault, T. J., Hairston, W. D., & Stein, B. E. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.2535-04.2004 · Cited by 183
Movement Sonification: Effects on Perception and Action
Effenberg, A. O. (2005) · IEEE Multimedia · doi:10.1109/mmul.2005.31 · Cited by 177
Evolution of forebrain and spatial cognition in vertebrates: Conservation across diversity
Salas, C., Broglio, C., & Rodríguez, F. (2003) · Brain, Behavior and Evolution · doi:10.1159/000072438 · Cited by 174
Spence, C., Velasco, C., & Knoeferle, K. (2014) · Flavour · doi:10.1186/2044-7248-3-8 · Cited by 175
Multisensory cues capture spatial attention regardless of perceptual load.
Santangelo, V., & Spence, C. (2007) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.33.6.1311 · Cited by 170
Infant perception of audio-visual speech synchrony
Lewkowicz, D. J. (2010) · Developmental Psychology · doi:10.1037/a0015579 · Cited by 168
Feinberg, T. E., & Mallatt, J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00667 · Cited by 158
Multisensory integration of colors and scents: insights from bees and flowers
Leonard, A. S., & Masek, P. (2014) · Journal of Comparative Physiology A · doi:10.1007/s00359-014-0904-4 · Cited by 161
The gustatory cortex and multisensory integration
de Araujo, I. E., & Simon, S. A. (2009) · International Journal of Obesity · doi:10.1038/ijo.2009.70 · Cited by 150
Wang, Y., Celebrini, S., Trotter, Y., & Barone, P. (2008) · BMC Neuroscience · doi:10.1186/1471-2202-9-79 · Cited by 144
Modifying the multisensory perception of a carbonated beverage using auditory cues
Zampini, M., & Spence, C. (2005) · Food Quality and Preference · doi:10.1016/j.foodqual.2004.11.004 · Cited by 139
Hearing cheats touch, but less in congenitally blind than in sighted individuals
Hötting, K., & Röder, B. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01501010.x · Cited by 136
Cortical activity during tactile exploration of objects in blind and sighted humans
Amedi, A., Raz, N., Azulay, H., Malach, R., & Zohary, E. (2010) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-2010-0503 · Cited by 125
Multisensory Integration: How Visual Experience Shapes Spatial Perception
Eimer, M. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.018 · Cited by 122
Auditory and auditory-tactile processing in congenitally blind humans
Hötting, K., & Röder, B. (2009) · Hearing Research · doi:10.1016/j.heares.2009.07.012 · Cited by 120
The race that precedes coactivation: development of multisensory facilitation in children
Barutchu, A., Crewther, D. P., & Crewther, S. G. (2009) · Developmental Science · doi:10.1111/j.1467-7687.2008.00782.x · Cited by 113
Reduced multisensory facilitation in persons with autism
Collignon, O., Charbonneau, G., Peters, F., Nassim, M., Lassonde, M., Lepore, F., Mottron, L., & Bertone, A. (2013) · Cortex · doi:10.1016/j.cortex.2012.06.001 · Cited by 114
Temporal Multimodal Learning in Audiovisual Speech Recognition
Hu, D., Li, X., & Lu, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.389 · Cited by 108
Governing coordination: behavioural principles and neural correlates
Carson, R. G., & Kelso, J. A. S. (2004) · Experimental Brain Research · doi:10.1007/s00221-003-1726-8 · Cited by 107
Looming Signals Reveal Synergistic Principles of Multisensory Integration
Cappe, C., Thelen, A., Romei, V., Thut, G., & Murray, M. M. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5517-11.2012 · Cited by 107
BAHRICK, L. E., & LICKLITER, R. (2004) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.4.2.137 · Cited by 104
Diminished sensitivity of audiovisual temporal order in autism spectrum disorder
de Boer-Schellekens, L., Eussen, M., & Vroomen, J. (2013) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2013.00008 · Cited by 99
Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli
Ngo, M. K., & Spence, C. (2010) · Attention, Perception, & Psychophysics · doi:10.3758/app.72.6.1654 · Cited by 94
Multisensory warning signals: when spatial correspondence matters
Ho, C., Santangelo, V., & Spence, C. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-1778-5 · Cited by 89
Perceiving blocks of emotional pictures and sounds: effects on physiological variables.
Brouwer, A.-M., van Wouwe, N., Mühl, C., van Erp, J., & Toet, A. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00295 · Cited by 85
Panagiotidi, M., Overton, P. G., & Stafford, T. (2017) · Acta Psychologica · doi:10.1016/j.actpsy.2017.10.001 · Cited by 82
Neural basis of multisensory looming signals
Tyll, S., Bonath, B., Schoenfeld, M. A., Heinze, H.-J., Ohl, F. W., & Noesselt, T. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.056 · Cited by 75
Multisensory constraints on awareness
Deroy, O., Chen, Y.-C., & Spence, C. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0207 · Cited by 78
Stanney, K., Samman, S., Reeves, L., Hale, K., Buff, W., Bowers, C., Goldiez, B., Nicholson, D., & Lackey, S. (2004) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1702_7 · Cited by 65
Fini, C., Cardini, F., Tajadura-Jiménez, A., Serino, A., & Tsakiris, M. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00165 · Cited by 59
Charbonneau, G., Bertone, A., Lepore, F., Nassim, M., Lassonde, M., Mottron, L., & Collignon, O. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.02.009 · Cited by 54
Influence of Vibrotactile Feedback on Some Perceptual Features of Violins
Wollman, I., Fritz, C., & Poitevineau, J. (2014) · The Journal of the Acoustical Society of America · doi:10.1121/1.4889865 · Cited by 47
Stewart, C. R., Sanchez, S. S., Grenesko, E. L., Brown, C. M., Chen, C. P., Keehn, B., Velasquez, F., Lincoln, A. J., & Müller, R.-A. (2016) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-015-2367-z · Cited by 46
Identification and integration of sensory modalities: Neural basis and relation to consciousness
Pennartz, C. M. A. (2009) · Consciousness and Cognition · doi:10.1016/j.concog.2009.03.003 · Cited by 40
How the owl tracks its prey – II
Takahashi, T. T. (2010) · Journal of Experimental Biology · doi:10.1242/jeb.031195 · Cited by 34
Metadata
- Venue
- Choice Reviews Online
- Publisher
- American Library Association
- Published
- 1994-04-01
- Volume
- 31
- Issue
- 08
- Pages
- 31-4365-31-4365
- ISSN
- 0009-4978 (ISSN-L)
- 1523-8253
- OpenAlex
- W1523224409
- Cited by
- 1,882
- References
- 0
- Type
- book review
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.