Convolutional neural network-based encoding and decoding of visual object recognition in space and time
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Representations learned by deep convolutional neural networks (CNNs) for object recognition are a widely investigated model of the processing hierarchy in the human visual system. Using functional magnetic resonance imaging, CNN representations of visual stimuli have previously been shown to correspond to processing stages in the ventral and dorsal streams of the visual system. Whether this correspondence between models and brain signals also holds for activity acquired at high temporal resolution has been explored less exhaustively. Here, we addressed this question by combining CNN‐based encoding models with magnetoencephalography (MEG). Human participants passively viewed 1,000 images of objects while MEG signals were acquired. We modelled their high temporal resolution source‐reconstructed cortical activity with CNNs, and observed a feed‐forward sweep across the visual hierarchy between 75 and 200 ms after stimulus onset. This spatiotemporal cascade was captured by the network layer representations, where the increasingly abstract stimulus representation in the hierarchical network model was reflected in different parts of the visual cortex, following the visual ventral stream. We further validated the accuracy of our encoding model by decoding stimulus identity in a left‐out validation set of viewed objects, achieving state‐of‐the‐art decoding accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 1×
“Neural networks have been shown to strongly bene fi t from biologically inspired mechanisms such as local convolutional operations (LeCun and Bengio (1995)), DropOut (Hinton et al., 2012) or ReLU non-linearities (He et al., 2015).”
BackgroundFukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · cited 1×
“CNN-like architectures have been motivated from a neuroscienti fi c point of view with the introduction of the Neocognitron by Fukushima (1980).”
The WU-Minn Human Connectome Project: An overview
Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E. J., Yacoub, E., & Ugurbil, K. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.041 · Cited by 5,651
A multi-modal parcellation of human cerebral cortex
Glasser, M. F., Coalson, T. S., Robinson, E. C., Hacker, C. D., Harwell, J., Yacoub, E., Ugurbil, K., Andersson, J., Beckmann, C. F., Jenkinson, M., et al. (2016) · Nature · doi:10.1038/nature18933 · Cited by 5,111
Receptive fields of single neurones in the cat's striate cortex
Hubel, D. H., & Wiesel, T. N. (1959) · The Journal of Physiology · doi:10.1113/jphysiol.1959.sp006308 · Cited by 4,653
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
The period of susceptibility to the physiological effects of unilateral eye closure in kittens
Hubel, D. H., & Wiesel, T. N. (1970) · The Journal of Physiology · doi:10.1113/jphysiol.1970.sp009022 · Cited by 3,030 · cited 1×
“Similarly, in biological systems even the earliest cortical receptive fi elds will only exist if the organism has been exposed to certain visual environments in a critical learning period after birth (Blakemore and Cooper, 1970; Hubel and Wiesel, 1970).”
BackgroundVan Veen, B. D., Van Drongelen, W., Yuchtman, M., & Suzuki, A. (1997) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.623056 · Cited by 2,561 · cited 1×
“Single-trial time courses of source activity were computed using Linearly-Constrained Minimum Variance (LCMV) Beamformers (Van Veen et al., 1997) using FieldTrip.”
Method
Cited by
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557
Shared Neural Mechanisms of Visual Perception and Imagery
Dijkstra, N., Bosch, S. E., & van Gerven, M. A. J. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.004 · Cited by 439
Self-citationRajalingham, R., Issa, E. B., Bashivan, P., Kar, K., Schmidt, K., & DiCarlo, J. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0388-18.2018 · Cited by 358
The neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233
Self-citationSpeech synthesis from ECoG using densely connected 3D convolutional neural networks
Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0c59 · Cited by 225
“More recently, initial studies have successfully applied deep learning methods to brain data [33, 34, 35, 36] and BCI applications [37, 38, 39, 40, 41, 42].”
Background
Metadata
- Authors
- Seeliger, K., Fritsche, M., Güçlü, U., Schoenmakers, S., Schoffelen, J.-M., Bosch, S. E., & van Gerven, M. A. J.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2018-10
- Volume
- 180
- Issue
- Pt A
- Pages
- 253-266
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2736245388
- Cited by
- 121
- References
- 21
- FWCI
- 7.44
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Convolutional neural network-based encoding and decoding of visual object recognition in space and time. https://app.cochat.ai/papers/10.1016/j.neuroimage.2017.07.018
- The paper
- Seeliger, K., Fritsche, M., Güçlü, U., Schoenmakers, S., Schoffelen, J.-M., Bosch, S. E., & van Gerven, M. A. J. (2018). Convolutional neural network-based encoding and decoding of visual object recognition in space and time. NeuroImage. https://doi.org/10.1016/j.neuroimage.2017.07.018
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.