Hierarchical Neural Representation of Dreamed Objects Revealed by Brain Decoding with Deep Neural Network Features
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Dreaming is generally thought to be generated by spontaneous brain activity during sleep with patterns common to waking experience. This view is supported by a recent study demonstrating that dreamed objects can be predicted from brain activity during sleep using statistical decoders trained with stimulus-induced brain activity. However, it remains unclear whether and how visual image features associated with dreamed objects are represented in the brain. In this study, we used a deep neural network (DNN) model for object recognition as a proxy for hierarchical visual feature representation, and DNN features for dreamed objects were analyzed with brain decoding of fMRI data collected during dreaming. The decoders were first trained with stimulus-induced brain activity labeled with the feature values of the stimulus image from multiple DNN layers. The decoders were then used to decode DNN features from the dream fMRI data, and the decoded features were compared with the averaged features of each object category calculated from a large-scale image database. We found that the feature values decoded from the dream fMRI data positively correlated with those associated with dreamed object categories at mid- to high-level DNN layers. Using the decoded features, the dreamed object category could be identified at above-chance levels by matching them to the averaged features for candidate categories. The results suggest that dreaming recruits hierarchical visual feature representations associated with objects, which may support phenomenal aspects of dream experience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 2×
“We first computed visual feature vectors for all images of categories in the ImageNet database (50 test categories for Horikawa and Kamitani (2015), and 15,314 candidate categories; Deng et al., 2009) and for images used in the perception test experiment in Horikawa et al. (2013) (26 and 16 dreamed…”
MethodThe Science of Mind Wandering: Empirically Navigating the Stream of Consciousness
Smallwood, J., & Schooler, J. W. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015331 · Cited by 1,685
Regional cerebral blood flow throughout the sleep-wake cycle. An H2(15)O PET study
Braun, A. (1997) · Brain · doi:10.1093/brain/120.7.1173 · Cited by 1,268
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Functional neuroanatomy of human rapid-eye-movement sleep and dreaming
Maquet, P., Péters, J.-M., Aerts, J., Delfiore, G., Degueldre, C., Luxen, A., & Franck, G. (1996) · Nature · doi:10.1038/383163a0 · Cited by 1,168 · cited 1×
“…of studies have sought to address the neural commonalities and differences of these contrasting states by analyses of regional brain activations (Maquet et al., 1996; Braun et al., 1997, 1998; Maquet, 2000; Hong et al., 2009; Dresler et al., 2011), and brain activity patterns for specific…”
BackgroundNeural correlates of the LSD experience revealed by multimodal neuroimaging
Carhart-Harris, R. L., Muthukumaraswamy, S., Roseman, L., Kaelen, M., Droog, W., Murphy, K., Tagliazucchi, E., Schenberg, E. E., Nest, T., Orban, C., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1518377113 · Cited by 859 · cited 2×
“Taken together with the generalizability of the generic decoding from task-induced brain activity to spontaneous brain activity, we may be able to expect that the generic decoding approach via visual feature prediction is also applicable to decoding of visual information from other types of spontaneously generated sub…”
BackgroundDreaming and the brain: from phenomenology to neurophysiology
Nir, Y., & Tononi, G. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.12.001 · Cited by 595 · cited 3×
“Although REM sleep and its underlying neurophysiological mechanisms were originally believed to be indispensable for dreaming, there has been accumulating evidence that dreaming is dissociable from REM sleep and can be experienced during NREM sleep periods (Nir and Tononi, 2009).”
BackgroundNeural decoding of visual imagery during sleep
Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013) · Science · doi:10.1126/science.1234330 · Cited by 498
Self-citationBraun, A. R., Balkin, T. J., Wesensten, N. J., Gwadry, F., Carson, R. E., Varga, M., Baldwin, P., Belenky, G., & Herscovitch, P. (1998) · Science · doi:10.1126/science.279.5347.91 · Cited by 474
Dreamed movement elicits activation in the sensorimotor cortex
Dresler, M., Koch, S. P., Wehrle, R., Spoormaker, V. I., Holsboer, F., Steiger, A., Sämann, P. G., Obrig, H., & Czisch, M. (2011) · Current Biology · doi:10.1016/j.cub.2011.09.029 · Cited by 168
fMRI evidence for multisensory recruitment associated with rapid eye movements during sleep.
Hong, C. C., Harris, J. C., Pearlson, G. D., Kim, J., Calhoun, V. D., Fallon, J. H., Golay, X., Gillen, J. S., Simmonds, D. J., van Zijl, P. C. M., et al. (2009) · Human Brain Mapping · doi:10.1002/hbm.20635 · Cited by 124 · cited 5×
“From the collected reports, words describing visual objects or scenes were manually extracted and mapped to WordNet, a lexical database in which semantically similar words are grouped together as synsets (categories), in a hierarchical structure (Fellbaum, 1998).”
BackgroundMethodInfluential
Cited by
Deep Learning: The Good, the Bad, and the Ugly
Serre, T. (2019) · Annual Review of Vision Science · doi:10.1146/annurev-vision-091718-014951 · Cited by 267
Altszyler, E., Ribeiro, S., Sigman, M., & Fernández Slezak, D. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.09.004 · Cited by 128
“These methods rely on the idea that words with similar meanings tend to occur in similar contexts (Harris, 1954).”
MethodWen, D., Wei, Z., Zhou, Y., Li, G., Zhang, X., & Han, W. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2018.00023 · Cited by 93
“For example, CNN was used to effectively classify fMRI data (Horikawa and Kamitani, 2017a) and to classify and diagnose Alzheimer’s disease (AD) (Sarraf and Tofighi, 2016) and aMCI (Meszlényi et al., 2017).”
MethodAre we really unconscious in “unconscious” states? Common assumptions revisited
Sevenius Nilsen, A., Juel, B. E., Thürer, B., Aamodt, A., & Storm, J. F. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.987051 · Cited by 49
Wong, W., Noreika, V., Móró, L., Revonsuo, A., Windt, J., Valli, K., & Tsuchiya, N. (2020) · Neuroscience of Consciousness · doi:10.1093/nc/niaa006 · Cited by 40
“Some studies have investigated neural correlates of specific dream content (Dresler et al. 2012; Horikawa et al. 2013; Horikawa and Kamitani 2017; Siclari et al. 2017), but reconstruction of the full phenomenal level would be a step for the distant future.”
Background
Metadata
- Authors
- Horikawa, T., & Kamitani, Y.
- Venue
- Frontiers in Computational Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2017-01-31
- Volume
- 11
- Pages
- 4-4
- ISSN
- 1662-5188 (ISSN-L)
- OpenAlex
- W2557547398
- Cited by
- 111
- References
- 13
- FWCI
- 4.29
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.