Sparse coding of sensory inputs
Abstract
Several theoretical, computational, and experimental studies suggest that neurons encode sensory information using a small number of active neurons at any given point in time. This strategy, referred to as 'sparse coding', could possibly confer several advantages. First, it allows for increased storage capacity in associative memories; second, it makes the structure in natural signals explicit; third, it represents complex data in a way that is easier to read out at subsequent levels of processing; and fourth, it saves energy. Recent physiological recordings from sensory neurons have indicated that sparse coding could be a ubiquitous strategy employed in several different modalities across different organisms.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many brain regions exhibit population activity where only a fraction of neurons are active simultaneously, with spike timing locked to ongoing oscillatory rhythms.
“Many brain regions exhibit population activity where only of neurons are active simultaneously, with spike timing locked to ongoing oscillatory rhythms (Fries 2015; Olshausen and Field 2004).”
From The myth of the Bayesian brain (2025) · cites this paper as Olshausen and Field 2004
Encoding and maintaining full posterior distributions in sparse neural systems would require coordination and representational bandwidth not supported by current neurophysiological data.
“Encoding and maintaining full posterior distributions in such systems would require coordination and representational bandwidth not supported by current neurophysiological data (Olshausen and Field 2004).”
From The myth of the Bayesian brain (2025) · cites this paper as Olshausen and Field 2004
Actual neural activity is highly sparse and temporally structured.
“In contrast, empirical evidence suggests that actual neural activity is highly sparse and temporally structured.”
From The myth of the Bayesian brain (2025) · cites this paper as Olshausen and Field 2004
Computational neuroscience has developed mathematical models of elementary computational components including sensory coding, normalization, working memory, evidence accumulation and decision mechanisms, and motor control.
“These include components for sensory coding29,30, normalization31, working memory32, evidence accumulation and decision mechanisms33–35, and motor control36.”
From Cognitive computational neuroscience (2018) · cites this paper as [29]
Cites
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · see citing works
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · see citing works
Self-citationThe Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,913 · see citing works
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642 · see citing works
Self-citationAn Energy Budget for Signaling in the Grey Matter of the Brain
Attwell, D., & Laughlin, S. B. (2001) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1097/00004647-200110000-00001 · Cited by 3,543 · see citing works
Relations between the statistics of natural images and the response properties of cortical cells
Field, D. J. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.002379 · Cited by 3,241 · see citing works
Self-citationNatural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · see citing works
Self-citationSingle Units and Sensation: A Neuron Doctrine for Perceptual Psychology?
Barlow, H. B. (1972) · Perception · doi:10.1068/p010371 · Cited by 1,714 · see citing works
The “independent components” of natural scenes are edge filters
Bell, A. J., & Sejnowski, T. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00121-1 · Cited by 1,592 · see citing works
Slow Feature Analysis: Unsupervised Learning of Invariances
Wiskott, L., & B, T. J. S. A. (2002) · Neural Computation · doi:10.1162/089976602317318938 · Cited by 1,351 · see citing works
Shiftable multiscale transforms
Simoncelli, E. P., Freeman, W. T., Adelson, E. H., & Heeger, D. J. (1992) · IEEE Transactions on Information Theory · doi:10.1109/18.119725 · Cited by 1,335 · see citing works
Sparse Coding and Decorrelation in Primary Visual Cortex During Natural Vision
Vinje, W. E., & Gallant, J. L. (2000) · Science · doi:10.1126/science.287.5456.1273 · Cited by 1,288 · see citing works
What Is the Goal of Sensory Coding?
Field, D. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.4.559 · Cited by 1,183 · see citing works
Self-citationThe cost of cortical computation.
Lennie, P. (2003) · Current Biology · doi:10.1016/s0960-9822(03)00135-0 · Cited by 1,134 · see citing works
Learning Overcomplete Representations
Lewicki, M. S., & B, T. J. S. A. (2000) · Neural Computation · doi:10.1162/089976600300015826 · Cited by 1,127 · see citing works
Independent component filters of natural images compared with simple cells in primary visual cortex
van Hateren, J. H., & van der Schaaf, A. (1998) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.1998.0303 · Cited by 1,106 · see citing works
Chalupa, L. M., Werner, J. S., & Barnstable, C. J. (2004) · Cited by 1,020 · see citing works
An ultra-sparse code underliesthe generation of neural sequences in a songbird
Hahnloser, R. H. R., Kozhevnikov, A. A., & Fee, M. S. (2002) · Nature · doi:10.1038/nature00974 · Cited by 986 · see citing works
Non-Holographic Associative Memory
WILLSHAW, D. J., BUNEMAN, O. P., & LONGUET-HIGGINS, H. C. (1969) · Nature · doi:10.1038/222960a0 · Cited by 979 · see citing works
The statistics of natural images
Ruderman, D. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x_5_4_006 · Cited by 945 · see citing works
Oscillations and Sparsening of Odor Representations in the Mushroom Body
Pérez-Orive, J., Mazor, O., Turner, G. C., Cassenaer, S., Wilson, R. I., & Laurent, G. (2002) · Science · doi:10.1126/science.1070502 · Cited by 792 · see citing works
Olfactory network dynamics and the coding of multidimensional signals
Laurent, G. (2002) · Nature reviews. Neuroscience · doi:10.1038/nrn964 · Cited by 733 · see citing works
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (2011) · Network: Computation in Neural Systems · doi:10.3109/0954898x.2011.638888 · Cited by 679 · see citing works
Efficient coding of natural sounds
Lewicki, M. S. (2002) · Nature Neuroscience · doi:10.1038/nn831 · Cited by 667 · see citing works
Sparse Population Coding of Faces in the Inferotemporal Cortex
Young, M. P., & Yamane, S. (1992) · Science · doi:10.1126/science.1598577 · Cited by 595 · see citing works
Hyvärinen, A., & Hoyer, P. O. (2000) · Neural Computation · doi:10.1162/089976600300015312 · Cited by 533 · see citing works
Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex
Rolls, E. T., & Tovée, M. J. (1995) · Journal of Neurophysiology · doi:10.1152/jn.1995.73.2.713 · Cited by 500 · see citing works
The structure and precision of retinal spike trains
Berry, M. J., Warland, D. K., & Meister, M. (1997) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.94.10.5411 · Cited by 490 · see citing works
Levy, W. B., & Baxter, R. A. (1996) · Neural Computation · doi:10.1162/neco.1996.8.3.531 · Cited by 477 · see citing works
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (1992) · Network Computation in Neural Systems · doi:10.1088/0954-898x_3_2_009 · Cited by 448 · see citing works
Palm, G. (1980) · Biological Cybernetics · doi:10.1007/bf00337019 · Cited by 439 · see citing works
Responses of neurons in primary and inferior temporal visual cortices to natural scenes
Baddeley, R. J., Abbott, L. F., Booth, M. C. A., Sengpiel, F., Freeman, T., Wakeman, E. A., & Rolls, E. T. (1997) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.1997.0246 · Cited by 431 · see citing works
Whisker movements evoked by stimulation of single pyramidal cells in rat motor cortex
Brecht, M., Schneider, M., Sakmann, B. J., & Margrie, T. W. (2004) · Nature · doi:10.1038/nature02266 · Cited by 429 · see citing works
van Hateren, J. H., & Ruderman, D. (1998) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.1998.0577 · Cited by 407 · see citing works
Origins of scaling in natural images
Ruderman, D. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00008-4 · Cited by 358 · see citing works
Binary Spiking in Auditory Cortex
DeWeese, M. R., Wehr, M., & Zador, A. M. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-21-07940.2003 · Cited by 355 · see citing works
Brecht, M., & Sakmann, B. J. (2002) · The Journal of Physiology · doi:10.1113/jphysiol.2002.018465 · Cited by 341 · see citing works
Place cells and silent cells in the hippocampus of freely-behaving rats
Thompson, L. T., & Best, P. (1989) · Journal of Neuroscience · doi:10.1523/jneurosci.09-07-02382.1989 · Cited by 337 · see citing works
Domany, E., Hammen, J. L. V., & Schulten, K. (1991) · CERN Document Server (European Organization for Nuclear Research) · doi:10.1007/978-1-4612-0723-8 · Cited by 334 · see citing works
Vision and the statistics of the visual environment
Simoncelli, E. P. (2003) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(03)00047-3 · Cited by 305 · see citing works
A theory of maximizing sensory information
van Hateren, J. H. (1992) · Biological Cybernetics · doi:10.1007/bf00203134 · Cited by 267 · see citing works
Characterizing the sparseness of neural codes
Willmore, B. D. B., & Tolhurst, D. J. (2001) · Network Computation in Neural Systems · doi:10.1088/0954-898x/12/3/302 · Cited by 264 · see citing works
Vinje, W. E., & Gallant, J. L. (2002) · Journal of Neuroscience · doi:10.1523/jneurosci.22-07-02904.2002 · Cited by 245 · see citing works
The Nonlinear Statistics of High-Contrast Patches in Natural Images
Lee, A. B., Pedersen, K. S., & Mumford, D. (2003) · International Journal of Computer Vision · doi:10.1023/a:1023705401078 · Cited by 234 · see citing works
Characterizing the sparseness of neural codes
Willmore, B. D. B., & Tolhurst, D. J. (2001) · Network Computation in Neural Systems · doi:10.1080/net.12.3.255.270 · Cited by 229 · see citing works
Parallel Processing in Neural Systems and Computers
Eckmiller, R., Hartmann, G., & Hauska, G. (1990) · Cited by 195 · see citing works
Sparse coding in the primate cortex
Földiák, P., & Young, M. P. (1998) · St Andrews Research Repository (St Andrews Research Repository) · Cited by 184 · see citing works
Associative neural memories : theory and implementation
Hassoun, M. H. (1993) · Oxford University Press eBooks · Cited by 179 · see citing works
Coding of Natural Scenes in Primary Visual Cortex
Weliky, M., Fiser, J., Hunt, R. H., & Wagner, D. N. (2003) · Neuron · doi:10.1016/s0896-6273(03)00022-9 · Cited by 142 · see citing works
Internal representations for associative memory
Baum, E. B., Moody, J. E., & Wilczek, F. A. (1988) · Biological Cybernetics · doi:10.1007/bf00332910 · Cited by 129 · see citing works
Activity of different classes of neurons of the motor cortex during locomotion
Beloozerova, I. N., Sirota, M. G., & Swadlow, H. A. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-03-01087.2003 · Cited by 126 · see citing works
Sparse distributed memory and related models
Kanerva, P. (1993) · NASA Technical Reports Server (NASA) · Cited by 107 · see citing works
Principles of Image Representation in Visual Cortex
Olshausen, B. A. (2003) · The MIT Press eBooks · doi:10.7551/mitpress/7131.003.0123 · Cited by 105 · see citing works
Self-citationFiete, I., Hahnloser, R. H. R., Fee, M. S., et al. (2004) · Journal of Neurophysiology · doi:10.1152/jn.01133.2003 · Cited by 89 · see citing works
Associative Data Storage and Retrieval in Neural Networks
Palm, G., & Sommer, F. T. (1996) · Physics of neural networks · doi:10.1007/978-1-4612-0723-8_3 · Cited by 78 · see citing works
How do visual neurons respond in the real world?
Reinagel, P. (2001) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00231-2 · Cited by 61 · see citing works
The atoms of vision: Cartesian or polar?
Zetzsche, C., Krieger, G., & Wegmann, B. (1999) · Journal of the Optical Society of America A · doi:10.1364/josaa.16.001554 · Cited by 51 · see citing works
Theunissen, F. E. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(02)00016-4 · Cited by 32 · see citing works
Olshausen, B. A., & O’Connor, K. N. (2002) · Nature Neuroscience · doi:10.1038/nn0402-292 · Cited by 24 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790 · see citing works
Energy limitation as a selective pressure on the evolution of sensory systems
Niven, J. E., & Laughlin, S. B. (2008) · Journal of Experimental Biology · doi:10.1242/jeb.017574 · Cited by 1,056 · see citing works
Haider, B., Duque, A., Hasenstaub, A. R., & McCormick, D. A. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.5297-05.2006 · Cited by 962 · see citing works
Lieder, F., & Griffiths, T. L. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1900061x · Cited by 892 · see citing works
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 736 · see citing works
Nir, Y., Fisch, L., Mukamel, R., Gelbard-Sagiv, H., Arieli, A., Fried, I., & Malach, R. (2007) · Current Biology · doi:10.1016/j.cub.2007.06.066 · Cited by 621 · see citing works
Cognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535 · see citing works
Neuromechanics of muscle synergies for posture and movement
Ting, L. H., & McKay, J. L. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.01.002 · Cited by 514 · see citing works
Projections from neocortex mediate top-down control of memory retrieval
Rajasethupathy, P., Sankaran, S., Marshel, J. H., Kim, C. K., Ferenczi, E., Lee, S. Y., Berndt, A., Ramakrishnan, C., Jaffe, A., Lo, M., et al. (2015) · Nature · doi:10.1038/nature15389 · Cited by 509 · see citing works
Concept cells: the building blocks of declarative memory functions
Quiroga, R. Q. (2012) · Nature reviews. Neuroscience · doi:10.1038/nrn3251 · Cited by 472 · see citing works
Silver, R. A. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2864 · Cited by 481 · see citing works
Prior expectation mediates neural adaptation to repeated sounds in the auditory cortex: an MEG study
Todorovic, A., van Ede, F., Maris, E., & de Lange, F. P. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.1425-11.2011 · Cited by 391 · see citing works
Rapid Neocortical Dynamics: Cellular and Network Mechanisms
Haider, B., & McCormick, D. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.04.008 · Cited by 441 · see citing works
Experimental evidence for sparse firing in the neocortex
Barth, A. L., & Poulet, J. F. A. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.03.008 · Cited by 364 · see citing works
Heterosynaptic plasticity underlies aversive olfactory learning in Drosophila
Hige, T., Aso, Y., Modi, M. N., Rubin, G. M., & Turner, G. C. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.003 · Cited by 354 · see citing works
Oxytocin enhances social recognition by modulating cortical control of early olfactory processing
Oettl, L.-L., Ravi, N., Schneider, M., Scheller, M. F., Schneider, P., Mitre, M., da Silva Gouveia, M., Froemke, R. C., Chao, M. V., Young, W. S., et al. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.033 · Cited by 359 · see citing works
Decision theory: what “should” the nervous system do?
Körding, K. (2007) · Science · doi:10.1126/science.1142998 · Cited by 313 · see citing works
Circuit level defects in the developing neocortex of fragile X mice
Gonçalves, J. T., Anstey, J. E., Golshani, P., & Portera-Cailliau, C. (2013) · Nature Neuroscience · doi:10.1038/nn.3415 · Cited by 279 · see citing works
Internally Mediated Developmental Desynchronization of Neocortical Network Activity
Golshani, P., Gonçalves, J. T., Khoshkhoo, S., Mostany, R., Smirnakis, S., & Portera-Cailliau, C. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2012-09.2009 · Cited by 298 · see citing works
Stowell, D., & Plumbley, M. D. (2014) · PeerJ · doi:10.7717/peerj.488 · Cited by 295 · see citing works
Active inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226 · see citing works
“The idea that neural representations weigh complexity against accuracy underwrites the imperative to find the most accurate explanation for sensory observations that is minimally complex, which has been leveraged by things like Horace Barlow’s principle of minimum redundancy [97] and subsequently supported empirically…”
BackgroundCircuit Mechanisms of Sensorimotor Learning
Makino, H., Hwang, E. J., Hedrick, N. G., & Komiyama, T. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.029 · Cited by 228 · see citing works
Paying the brain's energy bill
Padamsey, Z., & Rochefort, N. L. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2022.102668 · Cited by 114 · see citing works
Cell Type-Specific Structural Organization of the Six Layers in Rat Barrel Cortex
Narayanan, R. T., Udvary, D., & Oberlaender, M. (2017) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2017.00091 · Cited by 100 · see citing works
Abstract representations emerge naturally in neural networks trained to perform multiple tasks
Johnston, W. J., & Fusi, S. (2023) · Nature Communications · doi:10.1038/s41467-023-36583-0 · Cited by 89 · see citing works
Connecting neural codes with behavior in the auditory system of Drosophila
Clemens, J., Girardin, C. C., Coen, P., Guan, X.-J., Dickson, B. J., & Murthy, M. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.014 · Cited by 69 · see citing works
Contrastive Learning Models for Sentence Representations
Xu, L., Xie, H., Li, Z., Wang, F. L., Wang, W., & Li, Q. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3593590 · Cited by 50 · see citing works
Feed-Forward versus Feedback Inhibition in a Basic Olfactory Circuit
Kee, T., Sanda, P., Gupta, N., Stopfer, M., & Bazhenov, M. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004531 · Cited by 40 · see citing works
Thompson, A. W., & Scott, E. K. (2016) · Scientific Reports · doi:10.1038/srep34887 · Cited by 26 · see citing works
Predicting individual neuron responses with anatomically constrained task optimization
Mano, O., Creamer, M. S., Badwan, B. A., & Clark, D. A. (2021) · Current Biology · doi:10.1016/j.cub.2021.06.090 · Cited by 24 · see citing works
The myth of the Bayesian brain
Mangalam, M. (2025) · European Journal of Applied Physiology · doi:10.1007/s00421-025-05855-6 · Cited by 14 · see citing works
“Encoding and maintaining full posterior distributions in such systems would require coordination and representational bandwidth not supported by current neurophysiological data (Olshausen and Field 2004).”
- Authors
- Olshausen, B. A., & Field, D. J.
- Affiliations
- University of California, Davis (US)
- Cornell University (US)
- Corresponding author
- Olshausen, Bruno A
- ORCID
- Olshausen, Bruno A
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2004-08
- Volume
- 14
- Issue
- 4
- Pages
- 481-487
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W2074376560
- Cited by
- 1,404
- References
- 59
- Citations by year
- 2026: 40 · 2025: 79 · 2024: 70 · 2023: 62 · 2022: 62 · 2021: 93 · 2020: 68 · 2019: 68 · 2018: 75 · 2017: 73 · 2016: 88 · 2015: 78 · 2014: 77 · 2013: 79 · 2012: 54
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Current Opinion in Neurobiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Physical Sciences › Engineering › Computational Mechanics › Sparse and Compressive Sensing Techniques
- Keywords
- sparse coding, sensory neurons, neural coding, sparse representation, energy efficiency, sensory modalities, neural activity patterns
- MeSH terms
- Action Potentials, Afferent Pathways, Animals, Brain, Humans, Models, Neurological, Neurons, Afferent, Sensation, Visual Cortex, Signal Transduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
