The Fusion of Mental Imagery and Sensation in the Temporal Association Cortex
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
It is well understood that the brain integrates information that is provided to our different senses to generate a coherent multisensory percept of the world around us (Stein and Stanford, 2008), but how does the brain handle concurrent sensory information from our mind and the external world? Recent behavioral experiments have found that mental imagery—the internal representation of sensory stimuli in one's mind—can also lead to integrated multisensory perception (Berger and Ehrsson, 2013); however, the neural mechanisms of this process have not yet been explored. Here, using functional magnetic resonance imaging and an adapted version of a well known multisensory illusion (i.e., the ventriloquist illusion; Howard and Templeton, 1966), we investigated the neural basis of mental imagery-induced multisensory perception in humans. We found that simultaneous visual mental imagery and auditory stimulation led to an illusory translocation of auditory stimuli and was associated with increased activity in the left superior temporal sulcus (L. STS), a key site for the integration of real audiovisual stimuli (Beauchamp et al., 2004a, 2010; Driver and Noesselt, 2008; Ghazanfar et al., 2008; Dahl et al., 2009). This imagery-induced ventriloquist illusion was also associated with increased effective connectivity between the L. STS and the auditory cortex. These findings suggest an important role of the temporal association cortex in integrating imagined visual stimuli with real auditory stimuli, and further suggest that connectivity between the STS and auditory cortex plays a modulatory role in spatially localizing auditory stimuli in the presence of imagined visual stimuli.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neurophysiological investigation of the basis of the fMRI signal
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001) · Nature · doi:10.1038/35084005 · Cited by 6,551 · cited 1×
“Participants were scanned using a 3T General Electric 750, MR scanner equipped with an eight-channel head coil to acquire gradient-echo T2*-weighted echo-planar images with BOLD contrast as an index of local increases in synaptic activity (Logothetis et al., 2001; Magri et al., 2012).”
MethodDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454 · cited 1×
“While there has been a great deal of research examining the effects of attention, expectation, and prior knowledge on multisensory integration (Engel et al., 2001, 2012; Talsma et al., 2010), our results sug- gest that coherent multisensory representations of external objects are not only modulated…”
BackgroundPsychophysiological and Modulatory Interactions in Neuroimaging
Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., & Dolan, R. J. (1997) · NeuroImage · doi:10.1006/nimg.1997.0291 · Cited by 3,097 · cited 4×
“STS) and remote brain areas were assessed in a psychophysiological interaction (PPI) analysis (Friston et al., 1997) by defining a seed region for each participant centered on the peak voxel found within an 8 mm sphere centered on the group peak for the contrast AVi synchrony greater than AVi asynchrony.”
MethodInfluentialThe Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · cited 2×
“It is important to note that many experiments on the ventriloquist effect have successfully demonstrated that the ventriloquist illusion reflects a genuine perceptual phenomenon that cannot merely be explained by cognitive bias or postperceptual decisions (Bertelson and Aschersleben, 1998; Bertelson et al., 2000, 2006…”
BackgroundKosslyn, S. M., Ganis, G., & Thompson, W. L. (2001) · Nature Reviews Neuroscience · doi:10.1038/35090055 · Cited by 1,894 · cited 5×
“For example, it has been found that visual imagery activates the primary visual cortex (Kosslyn et al., 2001; Kamitani and Tong, 2005), and that visual imagery of objects from different categories selectively activates the corresponding parts of the visual cortex associated with perceiving objects…”
BackgroundInfluentialDecoding the visual and subjective contents of the human brain
Kamitani, Y., & Tong, F. (2005) · Nature Neuroscience · doi:10.1038/nn1444 · Cited by 1,829 · cited 2×
“For example, it has been found that visual imagery activates the primary visual cortex (Kosslyn et al., 2001; Kamitani and Tong, 2005), and that visual imagery of objects from different categories selectively activates the corresponding parts of the visual cortex associated with perceiving objects from those categorie…”
BackgroundMultisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598
That's my hand! Activity in premotor cortex reflects feeling of ownership of a limb
Ehrsson, H. H., Spence, C., & Passingham, R. E. (2004) · Science · doi:10.1126/science.1097011 · Cited by 1,348
Self-citationCalvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954 · cited 6×
“These conditions were chosen for our main analysis in light of several previous neuroimaging studies, which found robust multisensory integration effects by manipulating the temporal correspondence between audiovisual stimuli (Calvert et al., 2000; Bischoff et al., 2007; Noesselt et al., 2007; Lewis and Noppeney, 2010…”
BackgroundMethodResultInfluentialDriver, J., & Noesselt, T. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.013 · Cited by 940
The multifaceted interplay between attention and multisensory integration.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.06.008 · Cited by 828
Brain areas underlying visual mental imagery and visual perception: an fMRI study
Ganis, G., Thompson, W. L., & Kosslyn, S. M. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.02.012 · Cited by 765
The neurological basis of mental imagery: a componential analysis.
Farah, M. J. (1984) · Cognition · doi:10.1016/0010-0277(84)90026-x · Cited by 587
Unraveling multisensory integration: patchy organization within human STS multisensory cortex
Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004) · Nature Neuroscience · doi:10.1038/nn1333 · Cited by 513 · cited 2×
“…an illusory translocation of auditory stimuli and was associated with increased activity in the left superior temporal sulcus (L. STS), a key site for the integration of real audiovisual stimuli (Beauchamp et al., 2004a, 2010; Driver and Noesselt, 2008; Ghazanfar et al., 2008; Dahl et al., 2009).”
BackgroundNeural decoding of visual imagery during sleep
Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013) · Science · doi:10.1126/science.1234330 · Cited by 498
Multisensory spatial interactions: a window onto functional integration in the human brain
Macaluso, E., & Driver, J. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.03.008 · Cited by 428 · cited 2×
“…association cortices, and to subcortical structures, such as the superior colliculus and putamen (Calvert et al., 2000; Bushara et al., 2003; Macaluso and Driver, 2005; Bischoff et al., 2007; Stein and Stanford, 2008), it remains unknown whether imagery-related multisensory integration…”
BackgroundTop-down activation of shape-specific population codes in visual cortex during mental imagery
Stokes, M., Thompson, R., Cusack, R., & Duncan, J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4657-08.2009 · Cited by 323 · cited 1×
“Recent studies using brain-decoding techniques have also shown that the fine-grained patterns of activity in sensory areas when imagining a stimulus are similar to those when perceiving it (Thirion et al., 2006; Stokes et al., 2009; Cichy et al., 2012; Horikawa et al., 2013).”
BackgroundImagery and perception share cortical representations of content and location
Cichy, R. M., Heinzle, J., & Haynes, J.-D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr106 · Cited by 242
Beauchamp, M. S., Nath, A. R., & Pasalar, S. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4865-09.2010 · Cited by 240
Neural correlates of cross-modal binding
Bushara, K. O., Hanakawa, T., Immisch, I., Toma, K., Kansaku, K., & Hallett, M. (2003) · Nature Neuroscience · doi:10.1038/nn993 · Cited by 226 · cited 4×
“, 2004a, 2004b) and in studies on the perceptual effects of audiovisual integration (Bushara et al., 2003; Bischoff et al., 2007; Stevenson and James, 2009; Werner and Noppeney, 2010b; Marchant et al., 2012).”
BackgroundInfluentialGentile, G., Guterstam, A., Brozzoli, C., & Ehrsson, H. H. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.1363-13.2013 · Cited by 221 · cited 2×
“Consistent with previous studies that have manipulated the temporal correspondence between audiovisual stimuli to investigate the neural correlates of multisensory integration (Calvert et al., 2000; Ehrsson et al., 2004; Bischoff et al., 2007; Noesselt et al., 2007; Driver and Noesselt, 2008; Marchant et al., 2012; Ge…”
BackgroundResultSelf-citation
Research works printed but not linked: 44
- 10.1038/ncomms3585 · DOI
- 10.1016/S0960-9822(04)00043-0 · DOI
- 10.1007/s00221-012-3020-0 · DOI
- 10.1016/S0896-6273(04)00070-4 · DOI · Paper page
- 10.1016/j.cub.2013.06.012 · DOI
- 10.3758/BF03208826 · DOI · Paper page
- 10.3758/BF03205552 · DOI · Paper page
- 10.3758/BF03193687 · DOI
- 10.1016/j.neuropsychologia.2006.03.008 · DOI
- 10.1016/j.cub.2007.08.050 · DOI
- Brett, 2002, Region of interest analysis using an SPM toolbox [abstract]. 8th Int Conf Funct Mapp Hum Brain (OHBM), June 2–6; Sendai, Japan, Neuroimage, 16, 497
- Bruce, 1981, Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque, J Neurophysiol, 46, 369, 10.1152/jn.1981.46.2.369 · DOI · Paper page
- 10.1016/j.neuroimage.2005.03.013 · DOI · Paper page
- 10.1523/JNEUROSCI.3437-09.2009 · DOI · Paper page
- 10.1152/jn.01113.2002 · DOI · Paper page
- Engel AK Senkowski D Schneider TR (2012) in The neural bases of multisensory processes, Multisensory integration through neural coherence, eds Murray M Wallace M (CRC, Boca Raton, FL). · doi:10.1201/9781439812174-10
- 10.1016/0166-2236(89)90079-9 · DOI · Paper page
- 10.1523/JNEUROSCI.0799-05.2005 · DOI · Paper page
- 10.1523/JNEUROSCI.0541-08.2008 · DOI · Paper page
- Howard IP Templeton WB (1966) Human spatial orientation (Wiley, London).
- Kaas J Collins C (2004) in The handbook of multisensory processes, The resurrection of multisensory cortex in primates: connection patterns that integrate modalities, eds Calvert G Spence C Stein B (Bradford, Cambridge, MA), pp 285–294. · DOI
- 10.1016/j.cub.2009.10.068 · DOI · Paper page
- 10.1523/JNEUROSCI.5745-09.2010 · DOI · Paper page
- 10.1038/nn.2108 · DOI · Paper page
- 10.1016/S0896-6273(02)00580-9 · DOI
- 10.1523/JNEUROSCI.3985-11.2012 · DOI · Paper page
- 10.1002/hbm.21278 · DOI
- 10.1523/JNEUROSCI.2252-07.2007 · DOI · Paper page
- 10.1523/JNEUROSCI.0455-10.2010 · DOI · Paper page
- 10.1162/08989290051137549 · DOI · Paper page
- 10.1162/jocn_a_00280 · DOI
- 10.1016/j.jneumeth.2006.11.017 · DOI · Paper page
- 10.3389/neuro.11.010.2008 · DOI · Paper page
- 10.1523/JNEUROSCI.2805-13.2014 · DOI
- 10.1097/00001756-199605170-00012 · DOI · Paper page
- 10.1016/S0926-6410(02)00073-3 · DOI · Paper page
- 10.1002/cne.903430308 · DOI · Paper page
- 10.1016/j.neuroimage.2008.09.034 · DOI · Paper page
- 10.1016/j.neuroimage.2006.06.062 · DOI · Paper page
- 10.1126/science.1058911 · DOI · Paper page
- 10.3758/BF03194427 · DOI · Paper page
- 10.1073/pnas.0305697101 · DOI · Paper page
- 10.1523/JNEUROSCI.5091-09.2010 · DOI · Paper page
- 10.1093/cercor/bhp248 · DOI · Paper page
Cited by
Predictive coding and multisensory integration: an attentional account of the multisensory mind
Talsma, D. (2015) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2015.00019 · Cited by 222
“Thelen et al. (2012) have provided evidence for the role of memory in the formation of a multisensory representation.”
BackgroundThe roles of sensory perceptions and mental imagery in consumer decision-making
Kim, M., Kim, J.-H., Park, M., & Yoo, J. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102517 · Cited by 97
Dreams and Psychedelics: Neurophenomenological Comparison and Therapeutic Implications
Kraehenmann, R. (2017) · Current Neuropharmacology · doi:10.2174/1573413713666170619092629 · Cited by 92
“The neurocognitive model of dreaming is supported by evidence indicating that brain areas supporting sensory processing are activated during mental imagery in the absence of perceptual inputs [27, 28] and from lesion studies showing that lesions in the temporoparieto-occipital junction affect both dreaming and mental…”
BackgroundNielsen, T. (2017) · Neuroscience of Consciousness · doi:10.1093/nc/nix001 · Cited by 56
“A similar perspective has been supported for interactions between MSI and intentionally generated, waking state, auditory and visual images (Berger and Ehrsson, 2013; Berger and Ehrsson, 2014; Vetter et al., 2014).”
Background
Metadata
- Authors
- Berger, C. C., & Ehrsson, H. H.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2014-10-08
- Volume
- 34
- Issue
- 41
- Pages
- 13684-13692
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W1974359008
- Cited by
- 79
- References
- 21
- FWCI
- 6.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.