Interpreting encoding and decoding models
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Encoding and decoding models are widely used in systems, cognitive, and computational neuroscience to make sense of brain-activity data. However, the interpretation of their results requires care. Decoding models can help reveal whether particular information is present in a brain region in a format the decoder can exploit. Encoding models make comprehensive predictions about representational spaces. In the context of sensory experiments, where stimuli are experimentally controlled, encoding models enable us to test and compare brain-computational theories. Encoding and decoding models typically include fitted linear-model components. Sometimes the weights of the fitted linear combinations are interpreted as reflecting, in an encoding model, the contribution of different sensory features to the representation or, in a decoding model, the contribution of different measured brain responses to a decoded feature. Such interpretations can be problematic when the predictor variables or their noise components are correlated and when priors (or penalties) are used to regularize the fit. Encoding and decoding models are evaluated in terms of their generalization performance. The correct interpretation depends on the level of generalization a model achieves (e.g. to new response measurements for the same stimuli, to new stimuli from the same population, or to stimuli from a different population). Significant decoding or encoding performance of a single model (at whatever level of generality) does not provide strong constraints for theory. Many models must be tested and inferentially compared for analyses to drive theoretical progress.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783
Distributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940
Representational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
Self-citationThe information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626 · cited 1×
“A more general notion is that information is re-coded through a sequence of stages, possibly compressing it for more efficient representation [8], but also discarding unneeded information and converting needed information into a format that enables it to be exploited to control behavior [9,10].”
BackgroundNatural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · cited 1×
“A more general notion is that information is re-coded through a sequence of stages, possibly compressing it for more efficient representation [8], but also discarding unneeded information and converting needed information into a format that enables it to be exploited to control behavior [9,10].”
BackgroundBeyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Decoding the visual and subjective contents of the human brain
Kamitani, Y., & Tong, F. (2005) · Nature Neuroscience · doi:10.1038/nn1444 · Cited by 1,829 · cited 1×
“It might decode a continuous variable (e.g. the orientation of the stimulus; Kamitani & Tong 2005), or it might output a complex multidimensional description.”
BackgroundHow does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790
Neural correlations, population coding and computation
Averbeck, B. B., Latham, P. E., & Pouget, A. (2006) · Nature reviews. Neuroscience · doi:10.1038/nrn1888 · Cited by 1,743 · cited 1×
“However, a univariate encoding model would in general have less sensitivity, because it does not account for the noise correlations between different response channels [32].”
BackgroundMachine learning classifiers and fMRI: A tutorial overview
Pereira, F., Mitchell, T., & Botvinick, M. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.11.007 · Cited by 1,624
Using goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Self-citationPredicting Human Brain Activity Associated with the Meanings of Nouns
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A. (2008) · Science · doi:10.1126/science.1152876 · Cited by 1,180 · cited 2×
“In sensory neuroscience, the experimental conditions correspond to sensory stimuli, and so an encoding model maps sensory stimuli to their encodings in sensory regions of the brain [5 ,58 ,59 ,65].”
BackgroundKell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544 · cited 1×
“images or sounds; [5 ,31]; KhalighRazavi and Kriegeskorte, 2014; [60,66]; for recent reviews see Refs.”
BackgroundMiyawaki, Y., Uchida, H., Yamashita, O., Sato, M.-A., Morito, Y., Tanabe, H. C., Sadato, N., & Kamitani, Y. (2008) · Neuron · doi:10.1016/j.neuron.2008.11.004 · Cited by 543 · cited 1×
“The most ambitious variant of decoding is stimulus reconstruction, which provides a richer picture of the information a brain region contains about the stimulus (Thirion et al. 2006, Miyawaki et al. 2008, Mesgarani et al. 2009, Naselaris et al. 2009, Santoro et al. 2017).”
BackgroundA Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives
Haynes, J.-D. (2015) · Neuron · doi:10.1016/j.neuron.2015.05.025 · Cited by 402
Statistical learning and selective inference
Taylor, J., & Tibshirani, R. J. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1507583112 · Cited by 398 · cited 1×
“A related problem is performing statistical inference to test hypotheses about the weights [81].”
BackgroundDecoding Patterns of Human Brain Activity
Tong, F., & Pratte, M. S. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100412 · Cited by 375
Microstimulation of inferotemporal cortex influences face categorization
Afraz, S.-R., Kiani, R., & Esteky, H. (2006) · Nature · doi:10.1038/nature04982 · Cited by 363 · cited 2×
“(2) Models are fit at many locations and voxels highlighted on the basis of model fits.”
BackgroundSeeing it all: Convolutional network layers map the function of the human visual system
Eickenberg, M., Gramfort, A., Varoquaux, G., & Thirion, B. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.10.001 · Cited by 338
Top-down activation of shape-specific population codes in visual cortex during mental imagery
Stokes, M., Thompson, R., Cusack, R., & Duncan, J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4657-08.2009 · Cited by 323 · cited 1×
“Testing decoders for cross-generalization between different domains is useful to address a host of questions about the invariances of the encoding, for example its consistency between imagery and perception [45,46] and its stability across latencies after stimulus onset [47].”
BackgroundImagery and perception share cortical representations of content and location
Cichy, R. M., Heinzle, J., & Haynes, J.-D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr106 · Cited by 242
Cited by
Machine Learning for Neural Decoding
Glaser, J. I., Benjamin, A. S., Chowdhury, R. H., Perich, M. G., Miller, L. E., & Kording, K. P. (2020) · eneuro · doi:10.1523/eneuro.0506-19.2020 · Cited by 319
Neural Decoding of EEG Signals with Machine Learning: A Systematic Review
Saeidi, M., Karwowski, W., Farahani, F. V., Fiok, K., Taiar, R., Hancock, P. A., & Al-Juaid, A. (2021) · Brain Sciences · doi:10.3390/brainsci11111525 · Cited by 258
Neural tuning and representational geometry
Kriegeskorte, N., & Wei, X.-X. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00502-3 · Cited by 187
Self-citationDynamic and reversible remapping of network representations in an unchanging environment
Low, I. I. C., Williams, A. H., Campbell, M. G., Linderman, S. W., & Giocomo, L. M. (2021) · Neuron · doi:10.1016/j.neuron.2021.07.005 · Cited by 70
Tasks and their role in visual neuroscience
Kay, K., Bonnen, K., Denison, R. N., Arcaro, M. J., & Barack, D. L. (2023) · Neuron · doi:10.1016/j.neuron.2023.03.022 · Cited by 61
Metadata
- Authors
- Kriegeskorte, N., & Douglas, P. K.
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2019-04
- Volume
- 55
- Pages
- 167-179
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W2902687238
- Cited by
- 230
- References
- 25
- FWCI
- 2.83
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.