Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Visual mental imagery is the quasi-perceptual experience of "seeing in the mind's eye". While a tight correspondence between imagery and perception in terms of subjective experience is well established, their correspondence in terms of neural representations remains insufficiently understood. In the present study, we exploit the high spatial resolution of functional magnetic resonance imaging (fMRI) at 7T, the retinotopic organization of early visual cortex, and machine-learning techniques to investigate whether visual imagery of letter shapes preserves the topographic organization of perceived shapes. Sub-millimeter resolution fMRI images were obtained from early visual cortex in six subjects performing visual imagery of four different letter shapes. Predictions of imagery voxel activation patterns based on a population receptive field-encoding model and physical letter stimuli provided first evidence in favor of detailed topographic organization. Subsequent visual field reconstructions of imagery data based on the inversion of the encoding model further showed that visual imagery preserves the geometric profile of letter shapes. These results open new avenues for decoding, as we show that a denoising autoencoder can be used to pretrain a classifier purely based on perceptual data before fine-tuning it on imagery data. Finally, we show that the autoencoder can project imagery-related voxel activations onto their perceptual counterpart allowing for visually recognizable reconstructions even at the single-trial level. The latter may eventually be utilized for the development of content-based BCI letter-speller systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Andersson, J. L. R., Skare, S., & Ashburner, J. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00336-7 · Cited by 3,469
Moeller, S., Yacoub, E., Olman, C. A., Auerbach, E., Strupp, J., Harel, N., & Uğurbil, K. (2010) · Magnetic Resonance in Medicine · doi:10.1002/mrm.22361 · Cited by 1,586 · cited 2×
“For all functional runs we acquired high-resolution gradient echo (T2* weighted) echo-planar imaging (Moeller et al. 2010) data (TE = 26 ms, TR = 3000 ms, generalized auto-calibrating partially parallel acquisitions (GRAPPA) factor = 3, multi-band factor = 2, nominal flip angle = 55°, number of…”
MethodA spelling device for the paralysed
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., & Flor, H. (1999) · Nature · doi:10.1038/18581 · Cited by 1,577 · cited 2×
“, mental spatial navigation, mental calculation, mental drawing, or inner speech), and hence originating from distinct neural substrates, onto letters of the alphabet (Birbaumer et al. 1999; Sorger et al. 2012).”
BackgroundDecoding reveals the contents of visual working memory in early visual areas
Harrison, S. A., & Tong, F. (2009) · Nature · doi:10.1038/nature07832 · Cited by 1,347 · cited 2×
“Applying multi-voxel pattern analyses (MVPA) furthermore enabled the decoding of feature-specific imagery content related to orientations (Harrison and Tong 2009; Albers et al. 2013), motion (Emmerling et al. 2016), objects (Reddy et al. 2010; Cichy et al. 2012; Lee et al. 2012), shapes (Stokes et…”
MethodMental Imagery: Functional Mechanisms and Clinical Applications
Pearson, J., Naselaris, T., Holmes, E. A., & Kosslyn, S. M. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.08.003 · Cited by 1,087 · cited 2×
“While we anticipated this given findings that visual mental imagery exhibits retinotopic organization in early visual cortex (Slotnick et al. 2005; Albers et al. 2013; Pearson et al. 2015), these results were, nonetheless, exciting, because the previous reconstructions of mental imagery based on retinotopy did not pre…”
ResultGoebel, R., Esposito, F., & Formisano, E. (2006) · Human Brain Mapping · doi:10.1002/hbm.20249 · Cited by 1,053 · cited 1×
“A region-growing approach analyzed local intensity histograms, corrected topological errors of the segmented grey/white matter border and finally reconstructed meshes of the cortical surfaces (Kriegeskorte and Goebel 2001; Goebel et al. 2006).”
MethodSelf-citationKosslyn, S. M., Thompson, W. L., & Ganis, G. (2006) · doi:10.1093/acprof:oso/9780195179088.001.0001 · Cited by 909
Brain areas underlying visual mental imagery and visual perception: an fMRI study
Ganis, G., Thompson, W. L., & Kosslyn, S. M. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.02.012 · Cited by 765
Metamers of the ventral stream
Freeman, J., & Simoncelli, E. P. (2011) · Nature Neuroscience · doi:10.1038/nn.2889 · Cited by 667 · cited 1×
“Receptive field size exhibits a linear relationship with eccentricity with the exact slope depending on the visual area (Freeman and Simoncelli 2011).”
Background
Cited by
Visual mental imagery in typical imagers and in aphantasia: A millimeter-scale 7-T fMRI study
Liu, J., Zhan, M., Hajhajate, D., Spagna, A., Dehaene, S., Cohen, L., & Bartolomeo, P. (2023) · doi:10.1101/2023.06.14.544909 · Cited by 94
Neural dynamics of perceptual inference and its reversal during imagery
Dijkstra, N., Ambrogioni, L., Vidaurre, D., & van Gerven, M. (2020) · eLife · doi:10.7554/elife.53588 · Cited by 89
Spagna, A., Hajhajate, D., Liu, J., & Bartolomeo, P. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.029 · Cited by 33
Reading Imagined Letter Shapes from the Mind’s Eye Using Real-time 7 Tesla fMRI
Goebel, R., van Hoof, R., Bhat, S., Luhrs, M., & Senden, M. (2022) · 2022 10th International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci53720.2022.9735031 · Cited by 8
“The classifier was trained on reconstructions of imagery trials (down sampled to 30x30) of letters 'H' and 'T' from 5 of the 6 subjects in [1].”
MethodInfluentialSelf-citationThe brain as image processor and generator: towards function-restoring brain-computer-interfaces
van Hoof, R. (2022) · doi:10.26481/dis.20220603rh · Cited by 1
Metadata
- Authors
- Senden, M., Emmerling, T. C., van Hoof, R., Frost, M. A., & Goebel, R.
- Venue
- Brain Structure and Function
- Published
- 2019-01-14
- Volume
- 224
- Issue
- 3
- Pages
- 1167-1183
- ISSN
- 1863-2653 (ISSN-L)
- 1863-2661
- OpenAlex
- W2908704712
- Cited by
- 59
- References
- 18
- FWCI
- 3.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception. https://app.cochat.ai/papers/10.1007/s00429-019-01828-6
- The paper
- Senden, M., Emmerling, T. C., van Hoof, R., Frost, M. A., & Goebel, R. (2019). Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception. Brain Structure and Function. https://doi.org/10.1007/s00429-019-01828-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.