How prior expectations shape multisensory perception
Abstract
The brain generates a representation of our environment by integrating signals from a common source, but segregating signals from different sources. This fMRI study investigated how the brain arbitrates between perceptual integration and segregation based on top-down congruency expectations and bottom-up stimulus-bound congruency cues. Participants were presented audiovisual movies of phonologically congruent, incongruent or McGurk syllables that can be integrated into an illusory percept (e.g. "ti" percept for visual «ki» with auditory /pi/). They reported the syllable they perceived. Critically, we manipulated participants' top-down congruency expectations by presenting McGurk stimuli embedded in blocks of congruent or incongruent syllables. Behaviorally, participants were more likely to fuse audiovisual signals into an illusory McGurk percept in congruent than incongruent contexts. At the neural level, the left inferior frontal sulcus (lIFS) showed increased activations for bottom-up incongruent relative to congruent inputs. Moreover, lIFS activations were increased for physically identical McGurk stimuli, when participants segregated the audiovisual signals and reported their auditory percept. Critically, this activation increase for perceptual segregation was amplified when participants expected audiovisually incongruent signals based on prior sensory experience. Collectively, our results demonstrate that the lIFS combines top-down prior (in)congruency expectations with…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Expectations could modulate reinforcement signals, and if such manipulations affect the reinforcement signal used in calculating prediction errors, it would short-circuit learning by suppressing cue-inconsistent prediction errors.
“If these manipulations affect the reinforcement signal used in calculating prediction errors, it would short-circuit learning by suppressing cue-inconsistent prediction errors.”
From Social information creates self-fulfilling prophecies in judgments of pain, vicarious pain, and cognitive effort (2026) · cites this paper as [42]
Cites
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,308 · see citing works
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · see citing works
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844 · see citing works
Kleiner, M., Brainard, D., & Pelli, D. G. (2007) · MPG.PuRe (Max Planck Society) · Cited by 3,751 · see citing works
Anterior Cingulate Conflict Monitoring and Adjustments in Control
Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004) · Science · doi:10.1126/science.1089910 · Cited by 2,891 · see citing works
Common regions of the human frontal lobe recruited by diverse cognitive demands
Duncan, J., & Owen, A. M. (2000) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01633-7 · Cited by 2,498 · see citing works
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321 · see citing works
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · see citing works
Causal Inference in Multisensory Perception
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000943 · Cited by 1,069 · see citing works
Action sets and decisions in the medial frontal cortex
RUSHWORTH, M., WALTON, M., KENNERLEY, S., & BANNERMAN, D. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.07.009 · Cited by 1,004 · see citing works
Learned predictions of error likelihood in the anterior cingulate cortex
Brown, J. W., & Braver, T. S. (2005) · Science · doi:10.1126/science.1105783 · Cited by 928 · see citing works
Visual speech speeds up the neural processing of auditory speech
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0408949102 · Cited by 843 · see citing works
Ross, L. A., Saint‐Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl024 · Cited by 650 · see citing works
Temporal window of integration in auditory-visual speech perception
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.01.001 · Cited by 622 · see citing works
How robust is the language architecture? The case of mood
Berkum, J. J. A. V., Goede, D. D., van Alphen, P. M., Mulder, E. R., & Kerstholt, J. H. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00505 · Cited by 562 · see citing works
Talsma, D., Doty, T. J., & Woldorff, M. G. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhk016 · Cited by 434 · see citing works
Causal inference in perception
Shams, L., & Beierholm, U. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.07.001 · Cited by 403 · see citing works
Audiovisual Integration of Speech Falters under High Attention Demands
Alsius, A., Navarra, J., Campbell, R., & Soto‐Faraco, S. (2005) · Current Biology · doi:10.1016/j.cub.2005.03.046 · Cited by 380 · see citing works
Skipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl147 · Cited by 368 · see citing works
Temporal constraints on the McGurk effect
Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996) · Perception & Psychophysics · doi:10.3758/bf03206811 · Cited by 358 · see citing works
Methods and Software for fMRI Analysis of Clinical Subjects
Mazaika, P. K., Hoeft, F., Glover, G. H., & Reiss, A. L. (2009) · NeuroImage · doi:10.1016/s1053-8119(09)70238-1 · Cited by 356 · see citing works
Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception
Rohe, T., & Noppeney, U. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002073 · Cited by 344 · see citing works
Self-citationPerceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech
Miller, L. M., & D’Esposito, M. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.0896-05.2005 · Cited by 318 · see citing works
Semantic congruence is a critical factor in multisensory behavioral performance
Laurienti, P. J., Kraft, R., Maldjian, J. A., Burdette, J. H., & Wallace, M. T. (2004) · Experimental Brain Research · doi:10.1007/s00221-004-1913-2 · Cited by 303 · see citing works
A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion
Nath, A. R., & Beauchamp, M. S. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.024 · Cited by 282 · see citing works
Foxe, J. J., Molholm, S., Del Bene, V. A., Frey, H.-P., Russo, N. N., Blanco, D., Saint-Amour, D., & Ross, L. A. (2015) · Cerebral Cortex · doi:10.1093/cercor/bht213 · Cited by 263 · see citing works
Beauchamp, M. S., Nath, A. R., & Pasalar, S. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4865-09.2010 · Cited by 240 · see citing works
Roach, N. W., Heron, J., & McGraw, P. V. (2006) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2006.3578 · Cited by 210 · see citing works
Hein, G., Doehrmann, O., Müller, N. G., Kaiser, J., Muckli, L. F., & Naumer, M. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1740-07.2007 · Cited by 203 · see citing works
State-related and item-related neural correlates of successful memory encoding
Otten, L. J., Henson, R. N., & Rugg, M. D. (2002) · Nature Neuroscience · doi:10.1038/nn967 · Cited by 196 · see citing works
Long-term music training tunes how the brain temporally binds signals from multiple senses
Lee, H. L., & Noppeney, U. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1115267108 · Cited by 190 · see citing works
Self-citationThe Effect of Prior Visual Information on Recognition of Speech and Sounds
Noppeney, U., Josephs, O., Hocking, J., Price, C. J., & Friston, K. J. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm091 · Cited by 172 · see citing works
Self-citationLewis, R., & Noppeney, U. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.5745-09.2010 · Cited by 167 · see citing works
Self-citationProcessing of audiovisual speech in Broca's area
Ojanen, V., Möttönen, R., Pekkola, J., Jääskeläinen, I. P., Joensuu, R., Autti, T. H., & Sams, M. E. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.001 · Cited by 160 · see citing works
Perceptual Decisions Formed by Accumulation of Audiovisual Evidence in Prefrontal Cortex
Noppeney, U., Ostwald, D., & Werner, S. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0455-10.2010 · Cited by 157 · see citing works
Self-citationMcGurk effects in cochlear-implanted deaf subjects
Rouger, J., Fraysse, B., Déguine, O., & Barone, P. (2007) · Brain Research · doi:10.1016/j.brainres.2007.10.049 · Cited by 146 · see citing works
On the Variability of the McGurk Effect: Audiovisual Integration Depends on Prestimulus Brain States
KEIL, J. E., Müller, N., Ihssen, N., & Weisz, N. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr125 · Cited by 130 · see citing works
Sensory reliability shapes perceptual inference via two mechanisms
Rohe, T., & Noppeney, U. (2015) · Journal of Vision · doi:10.1167/15.5.22 · Cited by 129 · see citing works
Self-citationStevenson, R. A., Fister, J. K., Barnett, Z. P., Nidiffer, A. R., & Wallace, M. T. (2012) · Experimental Brain Research · doi:10.1007/s00221-012-3072-1 · Cited by 119 · see citing works
van Atteveldt, N., Formisano, E., Goebel, R., & Blomert, L. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.03.065 · Cited by 114 · see citing works
Looming Signals Reveal Synergistic Principles of Multisensory Integration
Cappe, C., Thelen, A., Romei, V., Thut, G., & Murray, M. M. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5517-11.2012 · Cited by 107 · see citing works
Pekkola, J., Laasonen, M. R., Ojanen, V., Autti, T. H., Jääskeläinen, I. P., Kujala, T., & Sams, M. E. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.09.069 · Cited by 97 · see citing works
Causal inference of asynchronous audiovisual speech
Magnotti, J. F., Ma, W. J., & Beauchamp, M. S. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00798 · Cited by 96 · see citing works
Abstract Coding of Audiovisual Speech: Beyond Sensory Representation
Hasson, U., Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2007) · Neuron · doi:10.1016/j.neuron.2007.09.037 · Cited by 88 · see citing works
Primary and multisensory cortical activity is correlated with audiovisual percepts
Benoit, M. M., Raij, T., Lin, F., Jääskeläinen, I. P., & Stufflebeam, S. M. (2009) · Human Brain Mapping · doi:10.1002/hbm.20884 · Cited by 88 · see citing works
Speech through ears and eyes: interfacing the senses with the supramodal brain
van Wassenhove, V. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00388 · Cited by 82 · see citing works
Cortical Plasticity of Audio-Visual Object Representations
Naumer, M. J., Doehrmann, O., Müller, N., Muckli, L. F., Kaiser, J., & Hein, G. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn200 · Cited by 80 · see citing works
Anterior Cingulate Cortex Makes 2 Contributions to Minimizing Distraction
Orr, J. M., & Weissman, D. H. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn119 · Cited by 75 · see citing works
Binding and unbinding the auditory and visual streams in the McGurk effect
Nahorna, O., Berthommier, F., & Schwartz, J. (2012) · The Journal of the Acoustical Society of America · doi:10.1121/1.4728187 · Cited by 75 · see citing works
Temporal prediction errors in visual and auditory cortices
Lee, H. L., & Noppeney, U. (2014) · Current Biology · doi:10.1016/j.cub.2014.02.007 · Cited by 68 · see citing works
Self-citationThe neural mechanisms of reliability weighted integration of shape information from vision and touch
Helbig, H. B., Ernst, M. O., Ricciardi, E., Pietrini, P., Thielscher, A., Mayer, K. M., Schultz, J., & Noppeney, U. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.072 · Cited by 67 · see citing works
Self-citationSchwartz, J. (2010) · The Journal of the Acoustical Society of America · doi:10.1121/1.3293001 · Cited by 61 · see citing works
Green, K. P., & Gerdman, A. (1995) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.21.6.1409 · Cited by 57 · see citing works
Senkowski, D., Saint‐Amour, D., Gruber, T., & Foxe, J. J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.06.046 · Cited by 55 · see citing works
Audio-visual crossmodal interactions in environmental perception: an fMRI investigation
Belardinelli, M. O., Sestieri, C., Matteo, R. D., Delogu, F., Gratta, C. D., Ferretti, A., Caulo, M., Tartaro, A., & Romani, G. L. (2004) · Cognitive Processing · doi:10.1007/s10339-004-0024-0 · Cited by 50 · see citing works
Examining the McGurk illusion using high-field 7 Tesla functional MRI
Szycik, G. R., Stadler, J., Tempelmann, C., & Münte, T. F. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00095 · Cited by 46 · see citing works
Prior auditory information shapes visual category-selectivity in ventral occipito-temporal cortex
Adam, R., & Noppeney, U. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.05.002 · Cited by 40 · see citing works
Self-citationNahorna, O., Berthommier, F., & Schwartz, J. (2015) · The Journal of the Acoustical Society of America · doi:10.1121/1.4904536 · Cited by 35 · see citing works
Supramodal processing optimizes visual perceptual learning and plasticity
Zilber, N., Ciuciu, P., Gramfort, A., Azizi, L., & van Wassenhove, V. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.02.017 · Cited by 33 · see citing works
Bonath, B., Noesselt, T., Krauel, K., Tyll, S., Tempelmann, C., & Hillyard, S. A. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.04.077 · Cited by 29 · see citing works
Green, K. P., & Gerdman, A. (1995) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.21.6.1409 · Cited by 28 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cognition does not affect perception: Evaluating the evidence for “top-down” effects
Firestone, C., & Scholl, B. J. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15000965 · Cited by 1,200 · see citing works
The neural dynamics of hierarchical Bayesian causal inference in multisensory perception
Rohe, T., Ehlis, A.-C., & Noppeney, U. (2019) · Nature Communications · doi:10.1038/s41467-019-09664-2 · Cited by 187 · see citing works
Self-citationYazdanpanah, A., Jung, H., Soltani, A., & Wager, T. D. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2513856123 · Cited by 1 · see citing works
- Authors
- Gau, R., & Noppeney, U.
- Affiliations
- University of Birmingham (GB)
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2016-01
- Volume
- 124
- Issue
- Pt A
- Pages
- 876-886
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W1769676696
- Cited by
- 109
- References
- 61
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 10 · 2023: 11 · 2022: 10 · 2021: 16 · 2020: 15 · 2019: 12 · 2018: 8 · 2017: 6 · 2016: 7
- FWCI
- 10.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Max Planck Digital Library (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- European Research Council (ERC-STG 309349-multsens)
- Bundesministerium für Bildung und Forschung (01GQ1002)
- Max-Planck-Gesellschaft
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Keywords
- multisensory integration, perceptual segregation, McGurk effect, audiovisual speech perception, top-down expectations, bottom-up cues, inferior frontal sulcus, illusory perception, audiovisual integration
- MeSH terms
- Acoustic Stimulation, Adult, Auditory Perception, Cues, Female, Humans, Illusions, Magnetic Resonance Imaging, Male, Middle Aged, Noise, Perception, Photic Stimulation, Psychometrics, Reaction Time, Set, Psychology, Speech Perception, Visual Perception, Prefrontal Cortex, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
