Prospective Coding by Spiking Neurons
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Animals learn to make predictions, such as associating the sound of a bell with upcoming feeding or predicting a movement that a motor command is eliciting. How predictions are realized on the neuronal level and what plasticity rule underlies their learning is not well understood. Here we propose a biologically plausible synaptic plasticity rule to learn predictions on a single neuron level on a timescale of seconds. The learning rule allows a spiking two-compartment neuron to match its current firing rate to its own expected future discounted firing rate. For instance, if an originally neutral event is repeatedly followed by an event that elevates the firing rate of a neuron, the originally neutral event will eventually also elevate the neuron's firing rate. The plasticity rule is a form of spike timing dependent plasticity in which a presynaptic spike followed by a postsynaptic spike leads to potentiation. Even if the plasticity window has a width of 20 milliseconds, associations on the time scale of seconds can be learned. We illustrate prospective coding with three examples: learning to predict a time varying input, learning to predict the next stimulus in a delayed paired-associate task and learning with a recurrent network to reproduce a temporally compressed version of a sequence. We discuss the potential role of the learning mechanism in classical trace conditioning. In the special case that the signal to be predicted encodes reward, the neuron learns to predict the discounted future reward and learning is closely related to the temporal difference learning algorithm TD(λ).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 1×
“The short-term memory network can be seen as a liquid state machine [42] or echo state machine [43] and the ramping activity is learned as readout from this activity traces.”
BackgroundHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · cited 1×
“The short-term memory network can be seen as a liquid state machine [42] or echo state machine [43] and the ramping activity is learned as readout from this activity traces.”
BackgroundStimulus onset quenches neural variability: A widespread cortical phenomenon
Churchland, M. M., Yu, B. M., Cunningham, J. P., Sugrue, L. P., Cohen, M. R., Corrado, G. S., Newsome, W. T., Clark, A. M., Hosseini, P., Scott, B. B., et al. (2010) · Nature Neuroscience · doi:10.1038/nn.2501 · Cited by 1,084
Hippocampal replay of extended experience
Davidson, T. J., Kloosterman, F., & Wilson, M. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.027 · Cited by 806 · cited 1×
“The same fast replay of a previously induced slower activity sequence was also observed in the rat primary visual cortex [33] and it is as well studied as compressed hippocampal replay of a spatial sequence [34].”
BackgroundNeural organization for the long-term memory of paired associates
Sakai, K., & Miyashita, Y. (1991) · Nature · doi:10.1038/354152a0 · Cited by 743 · cited 3×
“This proposition, however, fails to explain the experimentally observed activity ramps prior to predictable events [1, 2, 6 – 11] The origin of the ramping activity observed in experiments is not yet fully understood.”
BackgroundSolomon, P. R., Vander Schaaf, E. R., Thompson, R. F., & Weisz, D. J. (1986) · Behavioral Neuroscience · doi:10.1037/0735-7044.100.5.729 · Cited by 724 · cited 1×
“The usage of these short-term memory neurons is motivated by the observation that hippocampal activity is needed to overcome the temporal discontiguity in trace conditioning [4, 27].”
BackgroundA critical time window for dopamine actions on the structural plasticity of dendritic spines
Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014) · Science · doi:10.1126/science.1255514 · Cited by 614
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565 · cited 1×
“In rats that navigate a maze towards the learned position of a chocolate milk reward, the activity of striatal neurons increases while the rat approaches the reward position [7, 8].”
BackgroundDorsal raphe neurons signal reward through 5-HT and glutamate
Liu, Z., Zhou, J., Li, Y., Hu, F., Lu, Y., Ma, M., Feng, Q., Zhang, J.-E., Wang, D., Zeng, J., et al. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.010 · Cited by 469
Robust timing and motor patterns by taming chaos in recurrent neural networks
Laje, R., & Buonomano, D. V. (2013) · Nature Neuroscience · doi:10.1038/nn.3405 · Cited by 454 · cited 1×
“It should also be possible to model the memory traces with “dynamical attractors” in recurrent networks of rate neurons [30] or with long and stable transient dynamics in balanced networks [31].”
BackgroundProspective coding for objects in primate prefrontal cortex
Rainer, G., Rao, S. C., & Miller, E. K. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-13-05493.1999 · Cited by 432 · cited 5×
“Prospective coding in neurons of the prefrontal cortex was observed in an experiment with monkeys performing a delayed paired-associate task [1].”
BackgroundMethodInfluentialSingle-trial spike trains in parietal cortex reveal discrete steps during decision-making
Latimer, K. W., Yates, J. L., Meister, M. L. R., Huk, A. C., & Pillow, J. W. (2015) · Science · doi:10.1126/science.aaa4056 · Cited by 306 · cited 1×
“Data analysis of recordings in the macaque lateral intraparietal area revealed yet another candidate explanation: single neuron activity profiles could follow a step-like time course, while the averaged activity is a ramp, if the steps occur at different points in time [46].”
BackgroundPredictive information in a sensory population
Palmer, S. E., Marre, O., Berry, M. J., & Bialek, W. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1506855112 · Cited by 257
Learning by the Dendritic Prediction of Somatic Spiking
Urbanczik, R., & Senn, W. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.030 · Cited by 230 · cited 7×
“These terms would cancel each other in the case of Urbanczik & Senn [18] where α = λ = 1 and g PSPi 1⁄4 PSPi.”
BackgroundMethodResultInfluentialSelf-citationTrace Conditioning and the Hippocampus: The Importance of Contiguity
Bangasser, D. A., Waxler, D. E., Santollo, J., & Shors, T. J. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1742-06.2006 · Cited by 185
Cited by
Gerstner, W., Lehmann, M., Liakoni, V., Corneil, D., & Brea, J. (2018) · Frontiers in Neural Circuits · doi:10.3389/fncir.2018.00053 · Cited by 290
Self-citationThe successor representation: Its computational logic and neural substrates
Gershman, S. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0151-18.2018 · Cited by 191
“Brea et al. (2016) showed that such prospective coding is mathematically equivalent to the SR, and is consistent with a number of neurophysiological observations.”
ResultToward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79
Episodic retrieval for model-based evaluation in sequential decision tasks
Zhou, C. Y., Talmi, D., Daw, N., & Mattar, M. G. (2023) · doi:10.31234/osf.io/3sqjh · Cited by 21
Metadata
- Authors
- Brea, J., Gaál, A. T., Urbanczik, R., & Senn, W.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-06-24
- Volume
- 12
- Issue
- 6
- Pages
- e1005003
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2468211122
- Cited by
- 40
- References
- 15
- FWCI
- 3.13
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.