Neural portraits of perception: Reconstructing face images from evoked brain activity
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Recent neuroimaging advances have allowed visual experience to be reconstructed from patterns of brain activity. While neural reconstructions have ranged in complexity, they have relied almost exclusively on retinotopic mappings between visual input and activity in early visual cortex. However, subjective perceptual information is tied more closely to higher-level cortical regions that have not yet been used as the primary basis for neural reconstructions. Furthermore, no reconstruction studies to date have reported reconstructions of face images, which activate a highly distributed cortical network. Thus, we investigated (a) whether individual face images could be accurately reconstructed from distributed patterns of neural activity, and (b) whether this could be achieved even when excluding activity within occipital cortex. Our approach involved four steps. (1) Principal component analysis (PCA) was used to identify components that efficiently represented a set of training faces. (2) The identified components were then mapped, using a machine learning algorithm, to fMRI activity collected during viewing of the training faces. (3) Based on activity elicited by a new set of test faces, the algorithm predicted associated component scores. (4) Finally, these scores were transformed into reconstructed images. Using both objective and subjective validation measures, we show that our methods yield strikingly accurate neural reconstructions of faces even when excluding occipital cortex. This methodology not only represents a novel and promising approach for investigating face perception, but also suggests avenues for reconstructing ‘offline’ visual experiences—including dreams, memories, and imagination—which are chiefly represented in higher-level cortical areas.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-01-00411.1998 · Cited by 2,205 · cited 1×
“Furthermore, other non-occipital regions have been implicated in the processing of relatively subjective face properties such as race (Hart et al., 2000) and emotional expression (Whalen et al., 1998).”
Predicting Human Brain Activity Associated with the Meanings of Nouns
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A. (2008) · Science · doi:10.1126/science.1152876 · Cited by 1,180
Neural systems for recognition of familiar faces
Gobbini, M. I., & Haxby, J. V. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.04.015 · Cited by 958 · cited 1×
“These regions correspond to a broader network of areas that have been associated with various aspects of face processing (Fox et al., 2009b; Gobbini and Haxby, 2007).”
Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies
Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011) · Current Biology · doi:10.1016/j.cub.2011.08.031 · Cited by 933
The fusiform “face area” is part of a network that processes faces at the individual level
Gauthier, I., Tarr, M. J., Moylan, J., Skudlarski, P., Gore, J. C., & Anderson, A. W. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/089892900562165 · Cited by 859
Category-Specific Cortical Activity Precedes Retrieval During Memory Search
Polyn, S. M., Natu, V. S., Cohen, J. D., & Norman, K. A. (2005) · Science · doi:10.1126/science.1117645 · Cited by 690 · cited 1×
“Indeed, recent studies have found that the visual content of imagery (Stokes et al., 2011), memory retrieval (Kuhl et al., 2011; Polyn et al., 2005), and dreams (Horikawa et al., 2013) are represented in higher-level visual areas— areas that overlap with those that supported face reconstruction in…”
BackgroundMiyawaki, Y., Uchida, H., Yamashita, O., Sato, M.-A., Morito, Y., Tanabe, H. C., Sadato, N., & Kamitani, Y. (2008) · Neuron · doi:10.1016/j.neuron.2008.11.004 · Cited by 543 · cited 4×
“That is, rather than using a manually selected local image basis such as a set of binary patches (Miyawaki et al., 2008) or Gabor filters (Naselaris et al., 2009; Nishimoto et al., 2011), and without applying any semantic labels to the images (Naselaris et al., 2009), we identified components that…”
InfluentialNeural decoding of visual imagery during sleep
Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013) · Science · doi:10.1126/science.1234330 · Cited by 498
Cited by
The Code for Facial Identity in the Primate Brain
Chang, L., & Tsao, D. Y. (2017) · Cell · doi:10.1016/j.cell.2017.05.011 · Cited by 644
Deep image reconstruction from human brain activity
Shen, G., Horikawa, T., Majima, K., & Kamitani, Y. (2019) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006633 · Cited by 315
Progress and challenges in probing the human brain
Poldrack, R. A., & Farah, M. J. (2015) · Nature · doi:10.1038/nature15692 · Cited by 280
A cryptography-based approach for movement decoding
Dyer, E. L., Gheshlaghi Azar, M., Perich, M. G., Fernandes, H. L., Naufel, S., Miller, L. E., & Körding, K. P. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-017-0169-7 · Cited by 70
From remembering to reconstruction: The transformative neural representation of episodic memory.
Xue, G. (2022) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2022.102351 · Cited by 49
Metadata
- Authors
- Cowen, A. S., Chun, M. M., & Kuhl, B. A.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2014-07
- Volume
- 94
- Pages
- 12-22
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W1979733564
- Cited by
- 113
- References
- 9
- FWCI
- 5.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Neural portraits of perception: Reconstructing face images from evoked brain activity. https://app.cochat.ai/papers/10.1016/j.neuroimage.2014.03.018
- The paper
- Cowen, A. S., Chun, M. M., & Kuhl, B. A. (2014). Neural portraits of perception: Reconstructing face images from evoked brain activity. NeuroImage. https://doi.org/10.1016/j.neuroimage.2014.03.018
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.