Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Neural activity during cognitive tasks exhibits complex dynamics that flexibly encode task-relevant variables. Chaotic recurrent networks, which spontaneously generate rich dynamics, have been proposed as a model of cortical computation during cognitive tasks. However, existing methods for training these networks are either biologically implausible, and/or require a continuous, real-time error signal to guide learning. Here we show that a biologically plausible learning rule can train such recurrent networks, guided solely by delayed, phasic rewards at the end of each trial. Networks endowed with this learning rule can successfully learn nontrivial tasks requiring flexible (context-dependent) associations, memory maintenance, nonlinear mixed selectivities, and coordination among multiple outputs. The resulting networks replicate complex dynamics previously observed in animal cortex, such as dynamic encoding of task features and selective integration of sensory inputs. We conclude that recurrent neural networks offer a plausible model of cortical dynamics during both learning and performance of flexible behavior.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 5×
“Models based on recurrent neural networks (RNN), operating in the near-chaotic regime, seem well-suited to capture similar dynamics (Jaeger 2001; Maass, Natschläger, and Markram 2002; Buonomano and Maass 2009; Sussillo and Abbott 2009).”
BackgroundMethodInfluentialReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · cited 1×
“Interestingly, this method is largely similar to the well-known REINFORCE algorithm, which is widely used in reinforcement learning (Williams 1992; Mnih et al. 2014; Peters and Schaal 2008; Kober, Bagnell, and Peters 2013) (see in particular Eq.”
MethodThe importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639
Chaos in random neural networks
Sompolinsky, H., Crisanti, A., & Sommers, H. J. (1988) · Physical Review Letters · doi:10.1103/physrevlett.61.259 · Cited by 1,061 · cited 7×
“Note that the latter value places the networks in the early chaotic regime, where the long-term behavior generally remains non-periodic (Sompolinsky et al., 1988).”
BackgroundMethodInfluential‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework
Stokes, M. G. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.05.004 · Cited by 850
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
Dynamic Coding for Cognitive Control in Prefrontal Cortex
Stokes, M. G., Kusunoki, M., Sigala, N., Nili, H., Gaffan, D., & Duncan, J. (2013) · Neuron · doi:10.1016/j.neuron.2013.01.039 · Cited by 711 · cited 4×
“Thus, the same stimulus could entail different appropriate responses depending on current context (i.e. which modality - color or motion - was relevant for this trial).”
BackgroundMethodInfluentialRobust timing and motor patterns by taming chaos in recurrent neural networks
Laje, R., & Buonomano, D. V. (2013) · Nature Neuroscience · doi:10.1038/nn.3405 · Cited by 454
Song, H. F., Yang, G. R., & Wang, X.-J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004792 · Cited by 281 · cited 8×
“, 2013), and also used by Song and colleagues (Song et al., 2016) (see Figure 5).”
BackgroundMethodInfluentialFiete, I. R., Fee, M. S., & Seung, H. S. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01311.2006 · Cited by 167
Cited by
Towards the neural population doctrine
Saxena, S., & Cunningham, J. P. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.02.002 · Cited by 321
Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits
Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A., & Naud, R. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00857-x · Cited by 292
Learning from Action: Reconsidering Movement Signaling in Midbrain Dopamine Neuron Activity
Coddington, L. T., & Dudman, J. T. (2019) · Neuron · doi:10.1016/j.neuron.2019.08.036 · Cited by 150
Motor primitives in space and time via targeted gain modulation in cortical networks
Stroud, J. P., Porter, M. A., Hennequin, G., & Vogels, T. P. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0276-0 · Cited by 119
Trujillo, C. A., Adams, J. W., Negraes, P. D., Carromeu, C., Tejwani, L., Acab, A., Tsuda, B., Thomas, C. A., Sodhi, N., Fichter, K. M., et al. (2021) · EMBO Molecular Medicine · doi:10.15252/emmm.202012523 · Cited by 101
Metadata
- Authors
- Miconi, T.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2017-02-23
- Volume
- 6
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2953293408
- Cited by
- 126
- References
- 11
- FWCI
- 5.87
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.