Transient localist representations in critical branching networks
The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
There is a long-standing debate between localist and distributed representations in neural network research. Different connectionist models have employed one or the other type of representation, but all such models assume that representations are either always stable, or stable after learning converges. The present article explores an alternate possibility, which is that representations continually shift and change, even on relatively fast timescales, and even after learning has stabilised. This possibility of transient representation is demonstrated in a spiking neural network model, along with supporting neuroscientific evidence. The model learns localist representations that change over a wide range of timescales. Representations are made transient because synaptic connections are continually enabled and disabled to regulate spike propagation to a homeostasis of critical branching. Experiments and models suggest that so-called grandmother cells may shift on relatively fast timescales between different mental representations in different contexts and conditions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Thus, in addition to having desirable properties for a cognitive architecture, reservoir networks with local homeostatic control resemble biological dynamics in important ways.”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Rodny et al., 2017
In the citing paper
“However, previous work by Rodny et al. (2017) found the presence of localist representations in critical branching networks that drifted over time, similar to demonstrations of representational drift in mice (Rule et al., 2019).”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Rodny et al. (2017)
Citations
Cites
Ambrose, A., Wittgenstein, L., & Anscombe, G. E. M. (1954) · Philosophy and Phenomenological Research · doi:10.2307/2103745 · Cited by 12,192
Receptive fields and functional architecture of monkey striate cortex
Hubel, D. H., & Wiesel, T. N. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008455 · Cited by 6,568 · cited 1×
“…that neurons in primary visual cortex have well-defined receptive fields described by a direct correspondence between a neuron’s firing rate and the presence of, for example, a luminance contrast of a particular orientation at a particular location in the visual field (Hubel & Wiesel, 1959, 1968).”
BackgroundMcClelland, J. L., & Rumelhart, D. E. (1981) · Psychological Review · doi:10.1037/0033-295x.88.5.375 · Cited by 4,889 · cited 1×
“From Rosenblatt’s perceptron (Rosenblatt, 1961) and Selfridge’s pandemonium (Selfridge, 1958), to adaptive resonance (Grossberg, 1976) and interactive activation (McClelland & Rumelhart, 1981), it has been nearly universal to formulate units in neural network models with stable representations.”
MethodReceptive fields of single neurones in the cat's striate cortex
Hubel, D. H., & Wiesel, T. N. (1959) · The Journal of Physiology · doi:10.1113/jphysiol.1959.sp006308 · Cited by 4,653 · cited 1×
“…that neurons in primary visual cortex have well-defined receptive fields described by a direct correspondence between a neuron’s firing rate and the presence of, for example, a luminance contrast of a particular orientation at a particular location in the visual field (Hubel & Wiesel, 1959, 1968).”
BackgroundSparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642 · cited 1×
“Localist and distributed representations highlight one dimension of the relationship between neural and mental activity, and other dimensions are highlighted by other representations, such as population codes (Erickson, 2001) and sparse codes (Olshausen & Field, 1997).”
BackgroundReservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956 · cited 1×
“Review of critical branching network with learning Rodny and Kello (2014) presented a spiking reservoir network (Lukoševičius & Jaeger, 2009) with three groups of units: A group of excitatory source units, a group of excitatory and inhibitory reservoir units, and a group of sink units.”
BackgroundNeuronal Avalanches in Neocortical Circuits
Beggs, J. M., & Plenz, D. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-35-11167.2003 · Cited by 2,325 · cited 1×
“Evidence for critical branching started with a study by Beggs and Plenz (2003), in which the authors recorded intrinsic bursts of spike activity in slice preparations of rat somatosensory cortex.”
BackgroundInvariant visual representation by single neurons in the human brain
Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005) · Nature · doi:10.1038/nature03687 · Cited by 1,988 · cited 1×
“The Quiroga et al. (2005) study is illustrative because the authors assumed stable representations, and any apparent transience was attributed to unwanted changes in recording conditions, such as electrode slippage.”
BackgroundGrossberg, S. (1976) · Biological Cybernetics · doi:10.1007/bf00344744 · Cited by 1,909 · cited 2×
“From Rosenblatt’s perceptron (Rosenblatt, 1961) and Selfridge’s pandemonium (Selfridge, 1958), to adaptive resonance (Grossberg, 1976) and interactive activation (McClelland & Rumelhart, 1981), it has been nearly universal to formulate units in neural network models with stable representations.”
MethodPrinciples of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
White, B. W., & Rosenblatt, F. (1963) · The American Journal of Psychology · doi:10.2307/1419730 · Cited by 1,792
PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS
ROSENBLATT, F. (1961) · doi:10.21236/ad0256582 · Cited by 1,691
Internally Generated Cell Assembly Sequences in the Rat Hippocampus
Pastalkova, E., Itskov, V., Amarasingham, A., & Buzsáki, G. (2008) · Science · doi:10.1126/science.1159775 · Cited by 1,301
How Does a Brain Build a Cognitive Code?
Grossberg, S. (1982) · Boston Studies in the Philosophy of Science · doi:10.1007/978-94-009-7758-7_1 · Cited by 1,192
Proceedings of the 35th Annual Meeting of the Cognitive Science Society
Culbertson, J., & Wilson, C. (2013) · Cited by 1,139
Transient and persistent dendritic spines in the neocortex in vivo
Holtmaat, A. J. G. D., Trachtenberg, J. T., Wilbrecht, L., Shepherd, G. M., Zhang, X., Knott, G. W., & Svoboda, K. (2005) · Neuron · doi:10.1016/j.neuron.2005.01.003 · Cited by 1,131
How does a brain build a cognitive code?
Grossberg, S. (1980) · Psychological Review · doi:10.1037/0033-295x.87.1.1 · Cited by 1,094
Hinton, G. E. (2018) · KiltHub Repository · doi:10.1184/r1/6604925 · Cited by 1,025
Cortical coordination dynamics and cognition
Bressler, S. L., & Kelso, J. A. S. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01564-3 · Cited by 786 · cited 1×
“The ability of a system to dynamically reconfigure relationships between neural and mental activity has been associated with metastability (Bressler & Kelso, 2001; Kello, Anderson, Holden, & Van Orden, 2008), which is a computationally adaptive property (and thus potentially beneficial) associated…”
BackgroundNonlinear Dynamics, Psychology and Life Sciences
Cecen, A. A. (2010) · Cited by 785
Tognoli, E., & Kelso, J. A. S. (2014) · Neuron · doi:10.1016/j.neuron.2013.12.022 · Cited by 718 · cited 1×
“This balance can promote the spontaneous formation of transient spike patterns, and also increase and even maximise the number of different patterns that spontaneously emerge (Tognoli & Kelso, 2014).”
BackgroundEmergence of a stable cortical map for neuroprosthetic control
Ganguly, K., & Carmena, J. M. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000153 · Cited by 578 · cited 2×
“suring in this regard – Ganguly and Carmena (2009) found population codes in macaque primary motor cortex that were fairly stable over multiple week”
BackgroundKwok, T.-Y., & Yeung, D.-Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572102 · Cited by 493
Haldeman, C., & Beggs, J. M. (2005) · Physical Review Letters · doi:10.1103/physrevlett.94.058101 · Cited by 469 · cited 3×
“For instance, Haldeman and Beggs (2005) presented a feedforward, stochastic model of spike propagation that was simple enough to provide a clear demonstration of how critical branching can maximise pattern formation.”
BackgroundThe probability of neurotransmitter release: variability and feedback control at single synapses
Branco, T., & Staras, K. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2634 · Cited by 457 · cited 2×
“In addition to structural changes, synaptic transmission can be dynamically modulated by neurotransmitter release at the synapses, causing some synapses to trigger postsynaptic potentials more reliably than others (Branco & Staras, 2009).”
BackgroundPerson, R. S., & Kudina, L. P. (1972) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(72)90058-2 · Cited by 410
The organizing principles of neuronal avalanches: cell assemblies in the cortex?
Plenz, D., & Thiagarajan, T. C. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.01.005 · Cited by 375 · cited 1×
“The associations between neuronal avalanches, critical branching, and metastability may not be readily apparent (see Plenz & Thiagarajan, 2007).”
BackgroundTransient Cognitive Dynamics, Metastability, and Decision Making
Rabinovich, M. I., Huerta, R., Varona, P., & Afraimovich, V. S. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000072 · Cited by 349
Sparse but not ‘Grandmother-cell’ coding in the medial temporal lobe
Quiroga, R. Q., Kreiman, G., Koch, C., & Fried, I. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.12.003 · Cited by 291
Motor Learning with Unstable Neural Representations
Rokni, U., Richardson, A. G., Bizzi, E., & Seung, H. S. (2007) · Neuron · doi:10.1016/j.neuron.2007.04.030 · Cited by 269
Connectionist modelling in psychology: A localist manifesto
Page, M. (2000) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00003356 · Cited by 267 · cited 2×
“one hand, the activity of a given neuron might contribute to a single representation at some level, that is, localist representation (Page, 2000).”
BackgroundDynamic Sensory Representations in the Olfactory Bulb: Modulation by Wakefulness and Experience
Kato, H. K., Chu, M. W., Isaacson, J. S., & Komiyama, T. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.037 · Cited by 261
Perge, J. A., Homer, M. L., Malik, W. Q., Cash, S., Eskandar, E., Friehs, G., Donoghue, J. P., & Hochberg, L. R. (2013) · Journal of Neural Engineering · doi:10.1088/1741-2560/10/3/036004 · Cited by 243
Rossini, P. M., Martino, G., Narici, L., Pasquarelli, A., Peresson, M., Pizzella, V., Tecchio, F., Torrioli, G., & Romani, G. L. (1994) · Brain Research · doi:10.1016/0006-8993(94)90919-9 · Cited by 205 · cited 2×
“…in primary motor cortex to drift over the course of days and even hours (for similar results see János et al., 2013), and similar evidence for transience have been found in studies of sensorimotor maps (Rossini et al., 1994) and olfactory representations (Kato, Chu, Isaacson, & Komiyama, 2012).”
BackgroundResultBowers, J. S. (2009) · Psychological Review · doi:10.1037/a0014462 · Cited by 191 · cited 2×
“neurons are sometimes referred to as grandmother cells (Bowers, 2009), based on the illustrative example that the spiking of a given neuron might always contribute”
BackgroundCan Power-Law Scaling and Neuronal Avalanches Arise from Stochastic Dynamics?
Touboul, J., & Destexhe, A. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008982 · Cited by 190
Metastability of Active CA3 Networks
Sasaki, T., Matsuki, N., & Ikegaya, Y. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4514-06.2007 · Cited by 180
Single-Neuron Stability during Repeated Reaching in Macaque Premotor Cortex
Chestek, C. A., Batista, A. P., Santhanam, G., Yu, B. M., Afshar, A., Cunningham, J. P., Gilja, V., Ryu, S. I., Churchland, M. M., & Shenoy, K. V. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0959-07.2007 · Cited by 174 · cited 2×
“periods, and Chestek et al. (2007) found evidence for stable localist representations over two days in macaque premotor cortex.”
BackgroundThe pervasiveness of 1/f scaling in speech reflects the metastable basis of cognition
Kello, C. T., Anderson, G. G., Holden, J. G., & Van Orden, G. C. (2008) · Cognitive Science · doi:10.1080/03640210801944898 · Cited by 151
Self-citationChristensen, T. A., Pawlowski, W. M., Lei, H., & Hildebrand, J. G. (2000) · Nature Neuroscience · doi:10.1038/78840 · Cited by 128
The heavy tail of the human brain
Roberts, J. A., Boonstra, T. W., & Breakspear, M. (2015) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.10.014 · Cited by 93
De Pittà, M., Volman, V., Berry, H., Parpura, V., Volterra, A., & Ben-Jacob, E. (2012) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2012.00098 · Cited by 92 · cited 1×
“Also, astrocytes seem to support an additional as yet unknown mechanism of synaptic plasticity, and evidence suggests they can rapidly and dynamically modulate neurotransmitter transmission at synaptic clefts over time (De Pitta et al., 2012).”
BackgroundComputational significance of transient dynamics in cortical networks
Durstewitz, D., & Deco, G. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05976.x · Cited by 92
Soft-assembly of sensorimotor function.
Kello, C. T., & Orden, G. C. V. (2009) · PubMed · Cited by 69
Self-citationCatastrophic interference in neural networks
Lewandowsky, S., & Li, S. (1995) · Elsevier eBooks · doi:10.1016/b978-012208930-5/50011-8 · Cited by 67 · cited 1×
“This phenomenon is known as “catastrophic interference” (Lewandowsky & Li, 1995), and it is a long-standing issue in neural network research with many hypotheses on how it can be addressed and mitigated.”
BackgroundMeasuring sparseness in the brain: Comment on Bowers (2009).
Quian Quiroga, R., & Kreiman, G. (2010) · Psychological Review · doi:10.1037/a0016917 · Cited by 67 · cited 1×
“They interpreted their results as indicative of a sparse code (Quiroga, Kreiman, Koch, & Fried, 2008), which is a coding scheme relatively similar to localist representation with respect to the distributed-localist debate (Quiroga & Kreiman, 2010).”
BackgroundCritical branching neural networks.
Kello, C. T. (2013) · Psychological Review · doi:10.1037/a0030970 · Cited by 63 · cited 15×
“Kello (2013) showed that the algorithm can bring a spiking network to its critical branching”
BackgroundMethodResultInfluentialSelf-citationFreeman, W. (1994) · Integrative Psychological and Behavioral Science · doi:10.1007/bf02691333 · Cited by 55 · cited 2×
“One benefit of transient representations may be to support the context dependency of perceptual and cognitive function (Freeman, 1994).”
BackgroundChapter 2 The evolution and implications of population and modular neural coding ideas
Erickson, R. P. (2001) · Progress in brain research · doi:10.1016/s0079-6123(01)30003-1 · Cited by 31 · cited 2×
“sions are highlighted by other representations, such as population codes (Erickson, 2001) and sparse codes (Olshausen & Field, 1997).”
BackgroundThematic Minireview Series: Molecular Mechanisms of Synaptic Plasticity
Colbran, R. J. (2015) · Journal of Biological Chemistry · doi:10.1074/jbc.r115.696468 · Cited by 28 · cited 2×
“There are now many known mechanisms of plasticity that modify synaptic strengths, and combined, they can make relatively large changes in short periods of time (Colbran, 2015).”
BackgroundOptimization via Intermittency with a Self-Organizing Neural Network
Kwok, T., & Smith‐Miles, K. (2005) · Neural Computation · doi:10.1162/0899766054796860 · Cited by 21 · cited 2×
“Kwok and Smith (2005) implemented transient representations in a network of threshold gate units and showed that transience can serve to dynamically search a problem space to find multiple solutions, rather than settling on just one.”
BackgroundLearning and Variability in Spiking Neural Networks - eScholarship
Rodny, J., & Kello, C. T. (2014) · Proceedings of the Annual Meeting of the Cognitive Science Society · Cited by 2
Self-citation
Research works printed but not linked: 1
- Hinton G. E., 1986, Parallel distributed processing: Explorations in the microstructure of cognition, 1, 77
Cited by
Grandmother cells and localist representations: a review of current thinking
Bowers, J. S. (2017) · Language Cognition and Neuroscience · doi:10.1080/23273798.2016.1267782 · Cited by 45
“Theorists from psychology associated with the PDP perspective It is unfortunate that many theorists associated with the PDP perspective declined to contribute, but I am pleased that Rodny, Shea, and Kello (2016) as well as Thomas and French (2016) have contributed.”
BackgroundFalandays, J. B., Yoshimi, J., Warren, W. H., & Spivey, M. J. (2024) · Cognitive Neurodynamics · doi:10.1007/s11571-023-09988-2 · Cited by 19
“Work by Kello and colleagues (Kello, 2013; Kello et al., 2011; Rodny et al., 2017; Szary et al., 2011) has shown that several interesting, biologically-realistic phenomena emerge when a reservoir network is endowed with homeostatic control.”
Who You Are: The Science of Connectedness
Spivey, M. J. (2020) · The MIT Press eBooks · doi:10.7551/mitpress/12818.001.0001 · Cited by 41
Editor's Introduction and Review: Coordination and Context in Cognitive Science
Kello, C. T. (2017) · Topics in Cognitive Science · doi:10.1111/tops.12302 · Cited by 3
Self-citation
Metadata
- Authors
- Rodny, J. J., Shea, T. M., & Kello, C. T.
- Affiliations
- University of California, Merced (US)
- Corresponding author
- Kello, Christopher T.
- Venue
- Language, Cognition and Neuroscience
- Publisher
- Informa UK Limited
- Published
- 2016-10-18
- Volume
- 32
- Issue
- 3
- Pages
- 330-341
- ISSN
- 2327-3798 (ISSN-L)
- 2327-3801
- OpenAlex
- W2536572249
- Cited by
- 4
- References
- 51
- Citations by year
- 2023: 1 · 2020: 1 · 2017: 2
- FWCI
- 0.43
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Language Cognition and Neuroscience (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- localist representations, distributed representations, critical branching, spiking neural networks, synaptic homeostasis, grandmother cells, spike propagation, representation stability, neural network learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.