Cited by 45Multisensory perception and integrationVisual perception and processing mechanismsNeural dynamics and brain function

Complementary congruent and opposite neurons achieve concurrent multisensory integration and segregation

Open access
Zhang, Wen-Hao, Wang, He, Chen, Aihua, Gu, Yong, Lee, Tai Sing, Wong, KY Michael, Wu, Si·2019·eLife

Published version of a preprint

Abstract

Abstract Our brain perceives the world by exploiting multiple sensory modalities to extract information about various aspects of external stimuli. If these sensory cues are from the same stimulus of interest, they should be integrated to improve perception; otherwise, they should be segregated to distinguish different stimuli. In reality, however, the brain faces the challenge of recognizing stimuli without knowing in advance whether sensory cues come from the same or different stimuli. To address this challenge and to recognize stimuli rapidly, we argue that the brain should carry out multisensory integration and segregation concurrently with complementary neuron groups. Studying an example of inferring heading-direction via visual and vestibular cues, we develop a concurrent multisensory processing neural model which consists of two reciprocally connected modules, the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP), and that at each module, there exists two distinguishing groups of neurons, congruent and opposite neurons. Specifically, congruent neurons implement cue integration, while opposite neurons compute the cue disparity, both optimally as described by Bayesian inference. The two groups of neurons provide complementary information which enables the neural system to assess the validity of cue integration and, if necessary, to recover the lost information associated with individual cues without re-gathering new inputs. Through this process, the brain achieves rapid stimulus perception if the cues come from the same stimulus of interest, and differentiates and recognizes stimuli based on individual cues with little time delay if the cues come from different stimuli of interest. Our study unveils the indispensable role of opposite neurons in multisensory processing and sheds light on our understanding of how the brain achieves multisensory processing efficiently and rapidly. Significance Statement Our brain perceives the world by exploiting multiple sensory cues. These cues need to be integrated to improve perception if they come from the same stimulus and otherwise be segregated. To address the challenge of recognizing whether sensory cues come from the same or different stimuli that are unknown in advance, we propose that the brain should carry out multisensory integration and segregation concurrently with two different neuron groups. Specifically, congruent neurons implement cue integration, while opposite neurons compute the cue disparity, and the interplay between them achieves rapid stimulus recognition without information loss. We apply our model to the example of inferring heading-direction based on visual and vestibular cues and reproduce the experimental data successfully.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.43753

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Previous theoretical modeling work indicated that opposite neurons in macaque MSTd may be involved in segregating sensory cues.

    “Previous theoretical modeling work has indicated that opposite neurons in macaque MSTd may be involved in segregating sensory cues (22, 32, 33).”

    From How multisensory neurons solve causal inference. (2021) · cites this paper as [33]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Zhang, W.-H., Wang, H., Chen, A., Gu, Y., Lee, T. S., Wong, K. M., & Wu, S.
Affiliations
  • Department of Physics, Hong Kong University of Science and Technology, Hong Kong
  • Hong Kong University of Science and Technology (HK)
  • East China Normal University (CN)
  • Chinese Academy of Sciences (CN)
  • Center for the Neural Basis of Cognition (US)
Corresponding author
Wu, Si
Venue
eLife
Published
2019-05-23
Volume
8
ISSN
2050-084X (ISSN-L)
OpenAlex
W2946207583
Cited by
45
References
57
Citations by year
2026: 4 · 2025: 4 · 2024: 5 · 2023: 5 · 2022: 10 · 2021: 7 · 2020: 6 · 2019: 3 · 2018: 1
FWCI
7.66
Type
journal article
Language
en
Open access
Locations
  • eLife (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders
  • Research Grants Council, University Grants Committee (N_HKUST606/12, 605813, 16322616, 16306817)
  • National Basic Research Program of China (2014CB846101)
  • Natural Science Foundation of China (31261160495)
  • National Science Foundation (1816568)
  • Intelligence Advanced Research Projects Activity (D16PC00007)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
Keywords
multisensory integration, visual-vestibular integration, reciprocal connections, neural network model
MeSH terms
Animals, Brain, Humans, Macaca mulatta, Motion Perception, Neurons, Photic Stimulation, Temporal Lobe, Vestibule, Labyrinth, Visual Perception, Head Movements

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review