The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Hierarchical temporal memory (HTM) provides a theoretical framework that models several key computational principles of the neocortex. In this paper, we analyze an important component of HTM, the HTM spatial pooler (SP). The SP models how neurons learn feedforward connections and form efficient representations of the input. It converts arbitrary binary input patterns into sparse distributed representations (SDRs) using a combination of competitive Hebbian learning rules and homeostatic excitability control. We describe a number of key properties of the SP, including fast adaptation to changing input statistics, improved noise robustness through learning, efficient use of cells, and robustness to cell death. In order to quantify these properties we develop a set of metrics that can be directly computed from the SP outputs. We show how the properties are met using these metrics and targeted artificial simulations. We then demonstrate the value of the SP in a complete end-to-end real-world HTM system. We discuss the relationship with neuroscience and previous studies of sparse coding. The HTM spatial pooler represents a neurally inspired algorithm for learning sparse representations from noisy data streams in an online fashion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“MNIST We trained a SP without topology on the MNIST database of handwritten digits (Lecun et al., 1998).”
MethodEmergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · cited 3×
“When applied to natural images, these techniques lead to receptive fields that resemble those of V1 neurons (Olshausen and Field, 1996a; Lee et al., 2006), suggesting that the functionality of early sensory neurons can be explained by the sparse coding framework.”
BackgroundMethodSparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Interneurons of the neocortical inhibitory system
Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1519 · Cited by 2,954 · cited 1×
“Some classes, such as bipolar cells and double bouquet cells exclusively innervate cells within a cortical mini-column (Markram et al., 2004; Wonders and Anderson, 2006).”
BackgroundThe action potential in mammalian central neurons
Bean, B. P. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2148 · Cited by 1,785 · cited 2×
“A cortical neuron recognizes presynaptic input patterns by initiating nonlinear dendritic spikes (Major et al., 2013) or somatic action potentials (Bean, 2007), with thresholds depending on intrinsic cellular properties.”
BackgroundPyramidal neurons: dendritic structure and synaptic integration
Spruston, N. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2286 · Cited by 1,688
Two networks of electrically coupled inhibitory neurons in neocortex
Gibson, J. R., Beierlein, M., & Connors, B. W. (1999) · Nature · doi:10.1038/47035 · Cited by 1,404 · cited 2×
“It is well-documented that feedforward thalamacortical input strongly activates specific subtypes of inhibitory neurons (Gibson et al., 1999; Porter et al., 2001; Swadlow, 2002; Kremkow et al., 2016).”
BackgroundSparse coding of sensory inputs
Olshausen, B. A., & Field, D. J. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.07.007 · Cited by 1,404
The origin and specification of cortical interneurons
Wonders, C. P., & Anderson, S. A. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1954 · Cited by 980 · cited 1×
“Some classes, such as bipolar cells and double bouquet cells exclusively innervate cells within a cortical mini-column (Markram et al., 2004; Wonders and Anderson, 2006).”
BackgroundNon-Holographic Associative Memory
WILLSHAW, D. J., BUNEMAN, O. P., & LONGUET-HIGGINS, H. C. (1969) · Nature · doi:10.1038/222960a0 · Cited by 979 · cited 1×
“, 2014) and associative memory models (Willshaw et al., 1969; Hecht-Nielsen, 1990; Bibbig et al., 1995).”
BackgroundHOMEOSTATIC CONTROL OF NEURAL ACTIVITY: From Phenomenology to Molecular Design
Davis, G. W. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135751 · Cited by 543
Recovery after brain injury: mechanisms and principles
Nudo, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00887 · Cited by 524
Topographic Maps are Fundamental to Sensory Processing
Kaas, J. H. (1997) · Brain Research Bulletin · doi:10.1016/s0361-9230(97)00094-4 · Cited by 478
Efficient codes and balanced networks
Denève, S., & Machens, C. K. (2016) · Nature Neuroscience · doi:10.1038/nn.4243 · Cited by 471
The decade of the dendritic nmda spike
Antic, S. D., Zhou, W., Moore, A. R., Short, S. M., & Ikonomu, K. D. (2010) · Journal of Neuroscience Research · doi:10.1002/jnr.22444 · Cited by 222
Activity-dependent synaptogenesis in the adult Mammalian cortex
Zito, K., & Svoboda, K. (2002) · Neuron · doi:10.1016/s0896-6273(02)00903-0 · Cited by 134
Thalamocortical control of feed–forward inhibition in awake somatosensory ‘barrel’ cortex
Swadlow, H. A. (2002) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1156 · Cited by 107
Cited by
Unsupervised real-time anomaly detection for streaming data
Ahmad, S., Lavin, A., Purdy, S., & Agha, Z. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.04.070 · Cited by 995
Self-citationDeep Learning for Time Series Anomaly Detection: A Survey
Zamanzadeh Darban, Z., Webb, G. I., Pan, S., Aggarwal, C., & Salehi, M. (2025) · ACM Computing Surveys · doi:10.1145/3691338 · Cited by 613
EEG Based Emotion Recognition: A Tutorial and Review
Li, X., Zhang, Y., Tiwari, P., Song, D., Hu, B., Yang, M., Zhao, Z., Kumar, N., & Marttinen, P. (2023) · ACM Computing Surveys · doi:10.1145/3524499 · Cited by 508
A thousand brains: toward biologically constrained AI
Hole, K. J., & Ahmad, S. (2021) · SN Applied Sciences · doi:10.1007/s42452-021-04715-0 · Cited by 42
“The use of SDRs makes networks robust to noisy input, natural variations, and neuron failures [35, 37].”
BackgroundMethodSelf-citationHierarchical Temporal Memory Continuous Learning Algorithms for Fire State Determination
Ryder, N. L., Geiman, J. A., & Weckman, E. J. (2021) · Fire Technology · doi:10.1007/s10694-020-01055-0 · Cited by 13
Metadata
- Authors
- Cui, Y., Ahmad, S., & Hawkins, J.
- Venue
- Frontiers in Computational Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2017-11-29
- Volume
- 11
- Pages
- 111-111
- ISSN
- 1662-5188 (ISSN-L)
- OpenAlex
- W2950860789
- Cited by
- 132
- References
- 17
- FWCI
- 8.20
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.