Emergence of Complex Computational Structures From Chaotic Neural Networks Through Reward-Modulated Hebbian Learning
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
This paper addresses the question how generic microcircuits of neurons in different parts of the cortex can attain and maintain different computational specializations. We show that if stochastic variations in the dynamics of local microcircuits are correlated with signals related to functional improvements of the brain (e.g. in the control of behavior), the computational operation of these microcircuits can become optimized for specific tasks such as the generation of specific periodic signals and task-dependent routing of information. Furthermore, we show that working memory can autonomously emerge through reward-modulated Hebbian learning, if needed for specific tasks. Altogether, our results suggest that reward-modulated synaptic plasticity can not only optimize the network parameters for specific computational tasks, but also initiate a functional rewiring that re-programs microcircuits, thereby generating diverse computational functions in different generic cortical microcircuits. On a more general level, this work provides a new perspective for a standard model for computations in generic cortical microcircuits (liquid computing model). It shows that the arguably most problematic assumption of this model, the postulate of a teacher that trains neural readouts through supervised learning, can be eliminated. We show that generic networks of neurons can learn numerous biologically relevant computations through trial and error.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Hopfield, J. J. (1984) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.81.10.3088 · Cited by 6,918
A mechanism for cognitive dynamics: neuronal communication through neuronal coherence.
Fries, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.08.011 · Cited by 4,284 · cited 1×
“Additionally , several models have been introduced where synchronous activity is utilized for information routing in neural networks (Crick and Koch 1990; Salinas and Sejnowski 2001; Fries 2005; Akam and Kullmann 2010).”
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 4×
“It has already been shown that networks of spiking neurons with stereotypical connection probabilities can in principle support a large variety of computational tasks (Maass et al. 2002; Haeusler and Maass 2007; Haeusler et al. 2008; Buonomano and Maass 2009).”
InfluentialSelf-citationHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · cited 2×
“…been shown that if such supervised learning is applied to readout neurons whose axons are also connected to other neurons within the network, such generic networks can also learn to generate periodic patterns and working memory (Jaeger and Haas 2004; Maass et al. 2007; Sussillo and Abbott 2009).”
The role of the medial frontal cortex in cognitive control
Ridderinkhof, K. R., Ullsperger, M., Crone, E. A., & Nieuwenhuis, S. (2004) · Science · doi:10.1126/science.1100301 · Cited by 2,974
Cellular basis of working memory
Goldman-Rakic, P. S. (1995) · Neuron · doi:10.1016/0896-6273(95)90304-6 · Cited by 2,663
Neuron Activity Related to Short-Term Memory
Fuster, J. M., & Alexander, G. E. (1971) · Science · doi:10.1126/science.173.3997.652 · Cited by 2,085 · cited 2×
“Traces of persistent memory, reflected in sustained firing activity of single neurons in response to specific visual stimuli, have been recorded in the prefrontal cortex (Fuster and Alexander 1971; Goldman-Rakic 1995; Miller et al. 1996).”
Training and plasticity of working memory
Klingberg, T. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.05.002 · Cited by 1,585 · cited 2×
“Repeated training of working memory tasks can improve performance (Klingberg et al. 2002) and this improvement can transfer to tasks that were not a part of the training program (Klingberg 2010).”
Correlated neuronal activity and the flow of neural information
Salinas, E., & Sejnowski, T. J. (2001) · Nature Reviews Neuroscience · doi:10.1038/35086012 · Cited by 1,333 · cited 1×
“Additionally , several models have been introduced where synchronous activity is utilized for information routing in neural networks (Crick and Koch 1990; Salinas and Sejnowski 2001; Fries 2005; Akam and Kullmann 2010).”
Increased prefrontal and parietal activity after training of working memory
Olesen, P. J., Westerberg, H., & Klingberg, T. (2004) · Nature Neuroscience · doi:10.1038/nn1165 · Cited by 1,317
Training of Working Memory in Children With ADHD
Klingberg, T., Forssberg, H., & Westerberg, H. (2002) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.24.6.781.8395 · Cited by 1,225 · cited 2×
“Repeated training of working memory tasks can improve performance (Klingberg et al. 2002) and this improvement can transfer to tasks that were not a part of the training program (Klingberg 2010).”
Variability, compensation and homeostasis in neuron and network function
Marder, E., & Goaillard, J.-M. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1949 · Cited by 1,212
Schultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028
Olshausen, B., Anderson, C., & Van Essen, D. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-11-04700.1993 · Cited by 908
A cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879 · cited 2×
“A large set of experimental data shows that neuromodulators, such as dopamine, implement such a third factor in biological networks of neurons (Reynolds et al. 2001; Reynolds and Wickens 2002; Pawlak et al. 2010), but also a modulatory signal in the form of synaptic input can influence the…”
Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory
Bailey, C. H., Giustetto, M., Huang, Y.-Y., Hawkins, R. D., & Kandel, E. R. (2000) · Nature Reviews Neuroscience · doi:10.1038/35036191 · Cited by 382
Functional network reorganization during learning in a brain-computer interface paradigm
Jarosiewicz, B., Chase, S. M., Fraser, G. W., Velliste, M., Kass, R. E., & Schwartz, A. B. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0808113105 · Cited by 302
A reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292
Oscillations and Filtering Networks Support Flexible Routing of Information
Akam, T., & Kullmann, D. M. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.019 · Cited by 245
Neurons in Monkey Prefrontal Cortex That Track Past or Predict Future Performance
Hasegawa, R. P., Blitz, A. M., Geller, N. L., & Goldberg, M. E. (2000) · Science · doi:10.1126/science.290.5497.1786 · Cited by 71 · cited 1×
“Many studies indicate that the frontal cortex contains neurons that are sensitive to errors (Ridderinkhof et al. 2004) as well as neurons that track past or future performance (Hasegawa et al. 2000).”
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Recent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
Hierarchical process memory: memory as an integral component of information processing
Hasson, U., Chen, J., & Honey, C. J. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.04.006 · Cited by 749
Integration of nanoscale memristor synapses in neuromorphic computing architectures
Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G., & Prodromakis, T. (2013) · Nanotechnology · doi:10.1088/0957-4484/24/38/384010 · Cited by 613
Self-citationOptimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements
Hennequin, G., Vogels, T. P., & Gerstner, W. (2014) · Neuron · doi:10.1016/j.neuron.2014.04.045 · Cited by 346
Metadata
- Authors
- Hoerzer, G. M., Legenstein, R., & Maass, W.
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-11-11
- Volume
- 24
- Issue
- 3
- Pages
- 677-690
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2172115975
- Cited by
- 139
- References
- 21
- FWCI
- 4.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.