The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
A Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback
Open accessAbstract
Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how behaviorally relevant adaptive changes in complex networks of spiking neurons could be achieved in a self-organizing manner through local synaptic plasticity. However, the capabilities and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allows us to predict under which conditions reward-modulated STDP will achieve a desired learning effect. These analytical results imply that neurons can learn through reward-modulated STDP to classify not only spatial but also temporal firing patterns of presynaptic neurons. They also can learn to respond to specific presynaptic firing patterns with particular spike patterns. Finally, the resulting learning theory predicts that even difficult credit-assignment problems, where it is very hard to tell which synaptic weights should be modified in order to increase the global reward for the system, can be solved in a self-organizing manner through reward-modulated STDP. This yields an explanation for a fundamental experimental result on biofeedback in monkeys by Fetz and Baker. In this experiment monkeys were rewarded for increasing the firing rate of a particular neuron in the cortex and were able to solve this extremely difficult credit assignment problem. Our model for this experiment relies on a combination of reward-modulated STDP with variable spontaneous firing activity. Hence it also provides a possible functional explanation for trial-to-trial variability, which is characteristic for cortical networks of neurons but has no analogue in currently existing artificial computing systems. In addition our model demonstrates that reward-modulated STDP can be applied to all synapses in a large recurrent neural network without endangering the stability of the network dynamics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Self-citationSpiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
Competitive Hebbian learning through spike-timing-dependent synaptic plasticity
Song, S., Miller, K. D., & Abbott, L. F. (2000) · Nature Neuroscience · doi:10.1038/78829 · Cited by 2,729
Synaptic plasticity: taming the beast
Abbott, L. F., & Nelson, S. B. (2000) · Nature Neuroscience · doi:10.1038/81453 · Cited by 2,263
Reliability of spike timing in neocortical neurons
Mainen, Z. F., & Sejnowski, T. J. (1995) · Science · doi:10.1126/science.7770778 · Cited by 1,950
Gerstner, W. (2009) · Encyclopedia of Neuroscience · doi:10.1016/b978-008045046-9.01405-4 · Cited by 1,199
Differential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102
Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex
Gupta, A., Wang, Y., & Markram, H. (2000) · Science · doi:10.1126/science.287.5451.273 · Cited by 1,036
Schultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028
A combined SVM and LDA approach for classification
Xiong, T., & Cherkassky, V. (2006) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1556089 · Cited by 980
A cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879
The tempotron: a neuron that learns spike timing–based decisions
Gütig, R., & Sompolinsky, H. (2006) · Nature Neuroscience · doi:10.1038/nn1643 · Cited by 815
Visual input evokes transient and strong shunting inhibition in visual cortical neurons
Borg-Graham, L. J., Monier, C., & Frégnac, Y. (1998) · Nature · doi:10.1038/30735 · Cited by 669
Hebbian learning and spiking neurons
Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999) · Physical Review E · doi:10.1103/physreve.59.4498 · Cited by 664
Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling
Izhikevich, E. M. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl152 · Cited by 657
Fluctuating synaptic conductances recreate in vivo-like activity in neocortical neurons
Destexhe, A., Rudolph, M., Fellous, J.-M., & Sejnowski, T. J. (2001) · Neuroscience · doi:10.1016/s0306-4522(01)00344-x · Cited by 642
Operant Conditioning of Cortical Unit Activity
Fetz, E. E. (1969) · Science · doi:10.1126/science.163.3870.955 · Cited by 627
Cortical remodelling induced by activity of ventral tegmental dopamine neurons
Bao, S., Chan, V. T., & Merzenich, M. M. (2001) · Nature · doi:10.1038/35083586 · Cited by 569
Dopamine-dependent plasticity of corticostriatal synapses
Reynolds, J. N. J., & Wickens, J. R. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00045-x · Cited by 535
Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity
Gu, Q. (2002) · Neuroscience · doi:10.1016/s0306-4522(02)00026-x · Cited by 518
Input synchrony and the irregular firing of cortical neurons
Stevens, C. F., & Zador, A. M. (1998) · Nature Neuroscience · doi:10.1038/659 · Cited by 465
A computational model of filtering, detection, and compression in the cochlea
Lyon, R. (2005) · ICASSP '82. IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1982.1171644 · Cited by 411
Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory
Bailey, C. H., Giustetto, M., Huang, Y.-Y., Hawkins, R. D., & Kandel, E. R. (2000) · Nature Reviews Neuroscience · doi:10.1038/35036191 · Cited by 382
Spike-Timing-Dependent Plasticity in Balanced Random Networks
Morrison, A., Aertsen, A., & Diesmann, M. (2007) · Neural Computation · doi:10.1162/neco.2007.19.6.1437 · Cited by 365
Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity
Florian, R. V. (2007) · Neural Computation · doi:10.1162/neco.2007.19.6.1468 · Cited by 358
Synaptic Integration in Striate Cortical Simple Cells
Hirsch, J. A., Alonso, J.-M., Reid, R. C., & Martinez, L. M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-22-09517.1998 · Cited by 324
Plasticity in single neuron and circuit computations
Destexhe, A., & Marder, E. (2004) · Nature · doi:10.1038/nature03011 · Cited by 274
Volitional control of neural activity: implications for brain–computer interfaces
Fetz, E. E. (2007) · The Journal of Physiology · doi:10.1113/jphysiol.2006.127142 · Cited by 265
Isolated word recognition with the Liquid State Machine: a case study
Verstraeten, D., Schrauwen, B., Stroobandt, D., & Van Campenhout, J. (2005) · Information Processing Letters · doi:10.1016/j.ipl.2005.05.019 · Cited by 262
What is a moment? Transient synchrony as a collective mechanism for spatiotemporal integration
Hopfield, J. J., & Brody, C. D. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.98.3.1282 · Cited by 248
Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning
Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.6.1318 · Cited by 245
Computational Aspects of Feedback in Neural Circuits
Maass, W., Joshi, P., & Sontag, E. D. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020165 · Cited by 242
Self-citationHaeusler, S., & Maass, W. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj132 · Cited by 221
Self-citationStimulus dependence of two-state fluctuations of membrane potential in cat visual cortex
Anderson, J., Lampl, I., Reichova, I., Carandini, M., & Ferster, D. (2000) · Nature Neuroscience · doi:10.1038/75797 · Cited by 219
Fetz, E. E., & Baker, M. A. (1973) · Journal of Neurophysiology · doi:10.1152/jn.1973.36.2.179 · Cited by 199
Fetz, E. E., & Finocchio, D. V. (1975) · Experimental Brain Research · doi:10.1007/bf00239736 · Cited by 183
Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning
Kempter, R., Gerstner, W., & Hemmen, J. L. V. (2001) · Neural Computation · doi:10.1162/089976601317098501 · Cited by 171
Solving the distal reward problem through linkage of STDP and dopamine signaling
Izhikevich, E. M. (2007) · BMC Neuroscience · doi:10.1186/1471-2202-8-s2-s15 · Cited by 163
A neuronal analogue of state-dependent learning
Shulz, D. E., Sosnik, R., Ego‐Stengel, V., Haidarliu, S., & Ahissar, E. (2000) · Nature · doi:10.1038/35000586 · Cited by 156
Spike Timing-Dependent Synaptic Depression in theIn VivoBarrel Cortex of the Rat
Jacob, V., Brasier, D. J., Erchova, I. A., Feldman, D., & Shulz, D. E. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.4264-06.2007 · Cited by 147
BSA, a fast and accurate spike train encoding scheme
Schrauwen, B., & Campenhout, J. V. (2004) · doi:10.1109/ijcnn.2003.1224019 · Cited by 147
Reinforcement Learning With Modulated Spike Timing–Dependent Synaptic Plasticity
Farries, M. A., & Fairhall, A. L. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00364.2007 · Cited by 136
Cholinergic Modulation of Experience-Dependent Plasticity in Human Auditory Cortex
Thiel, C. M., Friston, K. J., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00801-2 · Cited by 134
Dynamics of Population Rate Codes in Ensembles of Neocortical Neurons
Silberberg, G., Bethge, M., Markram, H., Pawelzik, K. R., & Tsodyks, M. V. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00415.2003 · Cited by 133
Gradient Learning in Spiking Neural Networks by Dynamic Perturbation of Conductances
Fiete, I., & Seung, H. S. (2006) · Physical Review Letters · doi:10.1103/physrevlett.97.048104 · Cited by 128
Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons
Brunel, N. (2000) · Journal of Physiology-Paris · doi:10.1016/s0928-4257(00)01084-6 · Cited by 114
Fading memory and kernel properties of generic cortical microcircuit models
Maass, W., Natschläger, T., & Markram, H. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.020 · Cited by 97
Self-citationDirect gradient-based reinforcement learning
Baxter, J., & Bartlett, P. L. (2002) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2000.856049 · Cited by 94
Synapses as dynamic memory buffers
Maass, W., & Markram, H. (2002) · Neural Networks · doi:10.1016/s0893-6080(01)00144-7 · Cited by 77
Self-citationReinforcement Learning, Spike-Time-Dependent Plasticity, and the BCM Rule
Baras, D., & Meir, R. (2007) · Neural Computation · doi:10.1162/neco.2007.19.8.2245 · Cited by 72
Temporal dynamics of information content carried by neurons in the primary visual cortex
Nikolić, D., Haeusler, S., Singer, W., & Maass, W. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0135 · Cited by 47
Self-citationShulz, D. E., Ego‐Stengel, V., & Ahissar, E. (2003) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.01.001 · Cited by 19
Cited by
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
State-dependent computations: spatiotemporal processing in cortical networks
Buonomano, D. V., & Maass, W. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2558 · Cited by 926
Self-citationIntegration of nanoscale memristor synapses in neuromorphic computing architectures
Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G., & Prodromakis, T. (2013) · Nanotechnology · doi:10.1088/0957-4484/24/38/384010 · Cited by 613
Self-citationPonulak, F., & Kasiński, A. (2010) · Neural Computation · doi:10.1162/neco.2009.11-08-901 · Cited by 573
EEG-Neurofeedback as a Tool to Modulate Cognition and Behavior: A Review Tutorial
Enriquez-Geppert, S., Huster, R. J., & Herrmann, C. S. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00051 · Cited by 515
Metadata
- Authors
- Legenstein, R., Pecevski, D., & Maass, W.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2008-10-10
- Volume
- 4
- Issue
- 10
- Pages
- e1000180
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2138214290
- Cited by
- 267
- References
- 53
- FWCI
- 8.12
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.