The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes
Open accessAbstract
Recent multi-voxel pattern classification (MVPC) studies have shown that in early visual cortex patterns of brain activity generated during mental imagery are similar to patterns of activity generated during perception. This finding implies that low-level visual features (e.g., space, spatial frequency, and orientation) are encoded during mental imagery. However, the specific hypothesis that low-level visual features are encoded during mental imagery is difficult to directly test using MVPC. The difficulty is especially acute when considering the representation of complex, multi-object scenes that can evoke multiple sources of variation that are distinct from low-level visual features. Therefore, we used a voxel-wise modeling and decoding approach to directly test the hypothesis that low-level visual features are encoded in activity generated during mental imagery of complex scenes. Using fMRI measurements of cortical activity evoked by viewing photographs, we constructed voxel-wise encoding models of tuning to low-level visual features. We also measured activity as subjects imagined previously memorized works of art. We then used the encoding models to determine if putative low-level visual features encoded in this activity could pick out the imagined artwork from among thousands of other randomly selected images. We show that mental images can be accurately identified in this way; moreover, mental image identification accuracy depends upon the degree of tuning to low-level visual features in the voxels selected for decoding. These results directly confirm the hypothesis that low-level visual features are encoded during mental imagery of complex scenes. Our work also points to novel forms of brain-machine interaction: we provide a proof-of-concept demonstration of an internet image search guided by mental imagery.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ambrose, A., Wittgenstein, L., & Anscombe, G. E. M. (1954) · Philosophy and Phenomenological Research · doi:10.2307/2103745 · Cited by 12,192
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783
Ryle, G. (2006) · Theories of Mind · doi:10.5040/9798216413257.ch-2 · Cited by 6,809
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · cited 2×
“, V1 and V2), and provide an efficient basis for representing complex natural scenes (Olshausen and Field, 1996).”
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 2×
“One compelling answer has been advanced by the predictive coding theory of vision (Gregory, 1980; Rao and Ballard, 1999; Lee and Mumford, 2003; Yuille and Kersten, 2006; Bastos et al., 2012). ing to this theory feedback corresponds to the outputs of an internal, hierarchical model of the visual…”
Stanford Encyclopedia of Philosophy
(2015) · Choice Reviews Online · doi:10.5860/choice.187763 · Cited by 5,309
Preface: Cerebral Cortex Has Come of Age
Goldman-Rakic, P. S., & Rakic, P. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1-a · Cited by 3,393
Psychology as the behaviorist views it
Watson, J. B. (1913) · Psychological Review · doi:10.1037/h0074428 · Cited by 3,230
Canonical Microcircuits for Predictive Coding
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., & Friston, K. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.038 · Cited by 2,421 · cited 2×
“…has been advanced by the predictive coding theory of vision (Gregory, 1980; Rao and Ballard, 1999; Lee and Mumford, 2003; Yuille and Kersten, 2006; Bastos et al., 2012). ing to this theory feedback corresponds to the outputs of an internal, hierarchical model of the visual environment (see Berkes…”
Nipype: A Flexible, Lightweight and Extensible Neuroimaging Data Processing Framework in Python
Gorgolewski, K., Burns, C. D., Madison, C., Clark, D., Halchenko, Y. O., Waskom, M. L., & Ghosh, S. S. (2011) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2011.00013 · Cited by 2,186
Decoding the visual and subjective contents of the human brain
Kamitani, Y., & Tong, F. (2005) · Nature Neuroscience · doi:10.1038/nn1444 · Cited by 1,829 · cited 2×
“Additional sources of variation include attention (Kamitani and Tong, 2005), reward expectation (Serences, 2008), the perception of coherent shape (Murray et al., 2002), global context (Joo et al., 2012), and even auditory stimulation (Vetter et al., 2014).”
Psychology as the behaviorist views it.
Watson, J. B. (1994) · Psychological Review · doi:10.1037//0033-295x.101.2.248 · Cited by 1,735
Retinotopic organization in human visual cortex and the spatial precision of functional MRI
Engel, S. (1997) · Cerebral Cortex · doi:10.1093/cercor/7.2.181 · Cited by 1,621
Hierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 3×
“One compelling answer has been advanced by the predictive coding theory of vision (Gregory, 1980; Rao and Ballard, 1999; Lee and Mumford, 2003; Yuille and Kersten, 2006; Bastos et al., 2012). ing to this theory feedback corresponds to the outputs of an internal, hierarchical model of the visual…”
Decoding reveals the contents of visual working memory in early visual areas
Harrison, S. A., & Tong, F. (2009) · Nature · doi:10.1038/nature07832 · Cited by 1,347
What the Mind’s Eye Tells the Mind’s Brain: A Critique of Mental Imagery
Pylyshyn, Z. W. (1973) · Images, Perception, and Knowledge · doi:10.1007/978-94-010-1193-8_1 · Cited by 1,342
Identifying natural images from human brain activity
Kay, K. N., Naselaris, T., Prenger, R. J., & Gallant, J. L. (2008) · Nature · doi:10.1038/nature06713 · Cited by 1,286
Self-citationWhat the mind's eye tells the mind's brain: A critique of mental imagery.
Pylyshyn, Z. W. (1973) · Psychological Bulletin · doi:10.1037/h0034650 · Cited by 1,191
Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey
Rockland, K. S., & Pandya, D. N. (1979) · Brain Research · doi:10.1016/0006-8993(79)90485-2 · Cited by 1,060
Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions
O'Craven, K. M., & Kanwisher, N. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/08989290051137549 · Cited by 1,055
Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies
Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011) · Current Biology · doi:10.1016/j.cub.2011.08.031 · Cited by 933
Self-citationVisual images preserve metric spatial information: Evidence from studies of image scanning.
Kosslyn, S. M., Ball, T. M., & Reiser, B. J. (1978) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.4.1.47 · Cited by 927
Kosslyn, S. M., Thompson, W. L., & Ganis, G. (2006) · doi:10.1093/acprof:oso/9780195179088.001.0001 · Cited by 909
Naselaris, T., Kay, K. N., Nishimoto, S., & Gallant, J. L. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.07.073 · Cited by 905
Self-citationMemory's echo: vivid remembering reactivates sensory-specific cortex
Wheeler, M. E., Petersen, S. E., & Buckner, R. L. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.20.11125 · Cited by 865
Vision as Bayesian inference: analysis by synthesis?
Yuille, A., & Kersten, D. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.002 · Cited by 849
Gregory, R. L. (1980) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1980.0090 · Cited by 789
Circuits for local and global signal integration in primary visual cortex
Angelucci, A., Levitt, J. B., Walton, E. J. S., Hupé, J.-M., Bullier, J., & Lund, J. S. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-19-08633.2002 · Cited by 789
Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment
Berkes, P., Orbán, G., Lengyel, M., & Fiser, J. (2011) · Science · doi:10.1126/science.1195870 · Cited by 782
Brain areas underlying visual mental imagery and visual perception: an fMRI study
Ganis, G., Thompson, W. L., & Kosslyn, S. M. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.02.012 · Cited by 765
Anatomy of hierarchy: Feedforward and feedback pathways in macaque visual cortex
Markov, N. T., Vezoli, J., Chameau, P., Falchier, A., Quilodran, R., Huissoud, C., Lamy, C., Misery, P., Giroud, P., Ullman, S., et al. (2014) · Journal of Comparative Neurology · doi:10.1002/cne.23458 · Cited by 759
Mental imagery: In search of a theory
Pylyshyn, Z. W. (2002) · Behavioral and Brain Sciences · doi:10.1017/s0140525x02000043 · Cited by 722
Kosslyn, S. M. (1990) · Cited by 715
Cortical high-density counterstream architectures
Markov, N. T., Ercsey-Ravasz, M., Van Essen, D. C., Knoblauch, K., Toroczkai, Z., & Kennedy, H. (2013) · Science · doi:10.1126/science.1238406 · Cited by 616 · cited 2×
“Anatomical studies have revealed that EVA receives feedback projections from higher-order visual areas (Rockland and Pandya, 1979; Felleman and Van Essen, 1991; Markov et al., 2014a).”
Distributed Neural Systems for the Generation of Visual Images
Ishai, A., Ungerleider, L. G., & Haxby, J. V. (2000) · Neuron · doi:10.1016/s0896-6273(00)00168-9 · Cited by 579 · cited 2×
“At least eight studies reported no significant activity above baseline in early visual cortex duringmental imagery (D'Esposito et al., 1997; Ishai et al., 2000; Knauff et al., 2000; Trojano et al., 2000; Wheeler et al., 2000; Formisano et al., 2002; Sack et al., 2002; Daselaar et al., 2010), while at least twelve repo…”
Shared representations for working memory and mental imagery in early visual cortex
Albers, A. M., Kok, P., Toni, I., Dijkerman, H. C., & de Lange, F. P. (2013) · Current Biology · doi:10.1016/j.cub.2013.05.065 · Cited by 528
When is early visual cortex activated during visual mental imagery?
Kosslyn, S. M., & Thompson, W. L. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.723 · Cited by 515 · cited 2×
“Recent evidence suggests that the discrepancy can be explained by differences in experimental factors (Kosslyn and Thompson, 2003) and variation in the vividness of mental imagery across individuals (Cui et al., 2007).”
Bayesian Reconstruction of Natural Images from Human Brain Activity
Naselaris, T., Prenger, R. J., Kay, K. N., Oliver, M., & Gallant, J. L. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.006 · Cited by 511
Self-citationNeural decoding of visual imagery during sleep
Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013) · Science · doi:10.1126/science.1234330 · Cited by 498
Shape perception reduces activity in human primary visual cortex
Murray, S. O., Kersten, D., Olshausen, B. A., Schrater, P., & Woods, D. L. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.192579399 · Cited by 496 · cited 3×
“For example, activity in V1 in response to simple line elements can vary significantly depending upon whether the stimuli is perceived as a coherent object or a collection of independent lines (Murray et al., 2002).”
Vividness of mental imagery: Individual variability can be measured objectively
Cui, X., Jeter, C. B., Yang, D., Montague, P. R., & Eagleman, D. M. (2007) · Vision Research · doi:10.1016/j.visres.2006.11.013 · Cited by 478
Goebel, R., Khorram‐Sefat, D., Muckli, L., Hacker, H., & Singer, W. (1998) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.1998.00181.x · Cited by 474
Visual mental imagery induces retinotopically organized activation of early visual areas
Slotnick, S. D., Thompson, W. L., & Kosslyn, S. M. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhi035 · Cited by 458 · cited 2×
“Most studies on mental imagery have in fact adopted this approach, using extremely reduced visual stimuli such as blobs (Thirion et al., 2006), wedges (Slotnick et al., 2005), or gratings (Albers et al., 2013) for their experiments .”
A functional MRI study of mental image generation
D'Esposito, M., Detre, J. A., Aguirre, G. K., Stallcup, M., Alsop, D. C., Tippet, L. J., & Farah, M. J. (1997) · Neuropsychologia · doi:10.1016/s0028-3932(96)00121-2 · Cited by 453
Imaging brain function in humans at 7 Tesla
Yacoub, E., Shmuel, A., Pfeuffer, J., Van De Moortele, P., Adriany, G., Andersen, P., Vaughan, J. T., Merkle, H., Ugurbil, K., & Hu, X. (2001) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1080 · Cited by 448
Inverse retinotopy: Inferring the visual content of images from brain activation patterns
Thirion, B., Duchesnay, E., Hubbard, E., Dubois, J., Poline, J.-B., Lebihan, D., & Dehaene, S. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.06.062 · Cited by 363
Value-Based Modulations in Human Visual Cortex
Serences, J. T. (2008) · Neuron · doi:10.1016/j.neuron.2008.10.051 · Cited by 355
Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI
Ishai, A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1330 · Cited by 332
Decoding sound and imagery content in early visual cortex
Vetter, P., Smith, F. W., & Muckli, L. (2014) · Current Biology · doi:10.1016/j.cub.2014.04.020 · Cited by 323
Top-down activation of shape-specific population codes in visual cortex during mental imagery
Stokes, M., Thompson, R., Cusack, R., & Duncan, J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4657-08.2009 · Cited by 323
Reading the mind's eye: Decoding category information during mental imagery
Reddy, L., Tsuchiya, N., & Serre, T. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.084 · Cited by 304
Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study.
Le Bihan, D., Turner, R., Zeffiro, T. A., Cuénod, C. A., Jezzard, P., & Bonnerot, V (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.24.11802 · Cited by 290
Negative BOLD differentiates visual imagery and perception
Amedi, A., Malach, R., & Pascual-Leone, A. (2005) · Neuron · doi:10.1016/j.neuron.2005.10.032 · Cited by 271
Disentangling visual imagery and perception of real-world objects
Lee, S.-H., Kravitz, D. J., & Baker, C. I. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.10.055 · Cited by 263
Imagery and perception share cortical representations of content and location
Cichy, R. M., Heinzle, J., & Haynes, J.-D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr106 · Cited by 242
Tracking the Mind's Image in the Brain I
Formisano, E., Linden, D. E. J., Di Salle, F., Trojano, L., Esposito, F., Sack, A. T., Grossi, D., Zanella, F. E., & Goebel, R. (2002) · Neuron · doi:10.1016/s0896-6273(02)00747-x · Cited by 240
Klein, I., Paradis, A.-L., Poline, J.-B., Kosslyn, S. M., & Le Bihan, D. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/089892900564037 · Cited by 175
Human primary visual cortex and lateral geniculate nucleus activation during visual imagery
Chen, W., Kato, T., Zhu, X.-H., Ogawa, S., Tank, D. W., & Ugurbil, K. (1998) · NeuroReport · doi:10.1097/00001756-199811160-00019 · Cited by 169
Self-citationFunctional representations common to visual perception and imagination.
Podgorny, P., & Shepard, R. N. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.4.1.21 · Cited by 167
Cortical activation evoked by visual mental imagery as measured by fMRI
Knauff, M., Kassubek, J., Mulack, T., & Greenlee, M. W. (2000) · Neuroreport · doi:10.1097/00001756-200012180-00011 · Cited by 166
Imagery and visual working memory: one and the same?
Tong, F. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.08.005 · Cited by 163
Albright, T. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.001 · Cited by 162
Tracking the Mind's Image in the Brain II
Sack, A. T., Sperling, J. M., Prvulovic, D., Formisano, E., Goebel, R., Salle, F. D., Dierks, T., & Linden, D. E. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00745-6 · Cited by 155
A Real-Time fMRI-Based Spelling Device Immediately Enabling Robust Motor-Independent Communication
Sorger, B., Reithler, J., Dahmen, B., & Goebel, R. (2012) · Current Biology · doi:10.1016/j.cub.2012.05.022 · Cited by 144
Modality-specific and modality-independent components of the human imagery system
Daselaar, S. M., Porat, Y., Huijbers, W., & Pennartz, C. M. A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.239 · Cited by 143
Trojano, L. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.5.473 · Cited by 120
Decoding individual natural scene representations during perception and imagery
Johnson, M. R., & Johnson, M. K. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00059 · Cited by 119
Decoding Working Memory of Stimulus Contrast in Early Visual Cortex
Xing, Y., Ledgeway, T., McGraw, P. V., & Schluppeck, D. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.3754-12.2013 · Cited by 84
Functional Magnetic Resonance Imaging at 1.5 T during Sensorimotor and Cognitive Task
Sabbah, P., Simond, G., Levrier, O., Habib, M., Trabaud, V., Murayama, N., Mazoyer, B. M., Briant, J. F., Raybaud, C., & Salamon, G. (2008) · European Neurology · doi:10.1159/000117108 · Cited by 76
Functional Subdomains within Human FFA
Çukur, T., Huth, A. G., Nishimoto, S., & Gallant, J. L. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.1259-13.2013 · Cited by 72
Self-citationLong-Range, Pattern-Dependent Contextual Effects in Early Human Visual Cortex
Joo, S. J., Boynton, G. M., & Murray, S. O. (2012) · Current Biology · doi:10.1016/j.cub.2012.02.067 · Cited by 60
High-Field fMRI for Human Applications: An Overview of Spatial Resolution and Signal Specificity
Olman, C. A., & Yacoub, E. S. (2011) · The Open Neuroimaging Journal · doi:10.2174/1874440001105010074 · Cited by 56
Self-citationLambert, S., Sampaio, E., Scheiber, C., & Mauss, Y. (2002) · Brain Research · doi:10.1016/s0006-8993(01)03232-2 · Cited by 50 · cited 1×
“…o m / l o c a t e / y n i m g activity (Le Bihan et al., 1993; Sabbah et al., 1995; Goebel et al., 1998; Chen et al., 1998; Klein et al., 2000; O'Craven and Kanwisher, 2000; Ishai et al., 2002; Lambert et al., 2002; Ganis et al., 2004; Handy et al., 2004; Amedi et al., 2005; Cui et al., 2007).”
Bullier, J. (2006) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195148220.003.0006 · Cited by 37
Visual imagery and memory: Do retrieval strategies affect what the mind's eye sees?
Handy, T., Miller, M., Schott, B., Shroff, N., Janata, P., Van Horn, J., Inati, S., Grafton, S., & Gazzaniga, M. (2004) · European Journal of Cognitive Psychology · doi:10.1080/09541440340000457 · Cited by 24
On the function of mental imagery
Waltz, D. L. (1979) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00064529 · Cited by 5
Cited by
Mental Imagery: Functional Mechanisms and Clinical Applications
Pearson, J., Naselaris, T., Holmes, E. A., & Kosslyn, S. M. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.08.003 · Cited by 1,087
Self-citationThe human imagination: the cognitive neuroscience of visual mental imagery
Pearson, J. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0202-9 · Cited by 820
Generic decoding of seen and imagined objects using hierarchical visual features
Horikawa, T., & Kamitani, Y. (2017) · Nature Communications · doi:10.1038/ncomms15037 · Cited by 516
Shared Neural Mechanisms of Visual Perception and Imagery
Dijkstra, N., Bosch, S. E., & van Gerven, M. A. J. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.004 · Cited by 439
Semantic reconstruction of continuous language from non-invasive brain recordings
Tang, J., LeBel, A., Jain, S., & Huth, A. G. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01304-9 · Cited by 442
“For the decoder to transfer across tasks, the target task must share representations with the training task 1,31–33 .”
Metadata
- Authors
- Naselaris, T., Olman, C. A., Stansbury, D. E., Ugurbil, K., & Gallant, J. L.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2015-01
- Volume
- 105
- Pages
- 215-228
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2012027725
- Cited by
- 356
- References
- 76
- FWCI
- 9.70
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.