The DOI matches its Crossref record; 181 of the 181 references it cites were found in a registry or an index.
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules
Open accessAbstract
Classical Hebbian learning puts the emphasis on joint pre- and postsynaptic activity, but neglects the potential role of neuromodulators. Since neuromodulators convey information about novelty or reward, the influence of neuromodulators on synaptic plasticity is useful not just for action learning in classical conditioning, but also to decide "when" to create new memories in response to a flow of sensory stimuli. In this review, we focus on timing requirements for pre- and postsynaptic activity in conjunction with one or several phasic neuromodulatory signals. While the emphasis of the text is on conceptual models and mathematical theories, we also discuss some experimental evidence for neuromodulation of Spike-Timing-Dependent Plasticity. We highlight the importance of synaptic mechanisms in bridging the temporal gap between sensory stimulation and neuromodulatory signals, and develop a framework for a class of neo-Hebbian three-factor learning rules that depend on presynaptic activity, postsynaptic variables as well as the influence of neuromodulators.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
A synaptic model of memory: long-term potentiation in the hippocampus
Bliss, T. V. P., & Collingridge, G. L. (1993) · Nature · doi:10.1038/361031a0 · Cited by 11,566
Hebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Kohonen, T. (1990) · Proceedings of the IEEE · doi:10.1109/5.58325 · Cited by 8,414
Bliss, T. V. P., & Gardner‐Medwin, A. R. (1973) · The Journal of Physiology · doi:10.1113/jphysiol.1973.sp010274 · Cited by 7,437
indicated, N. A. (1950) · Journal of Consulting Psychology · doi:10.1037/h0049612 · Cited by 6,923
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
The hippocampus as a cognitive map
Passingham, R. E. (1979) · Neuroscience · doi:10.1016/0306-4522(79)90015-0 · Cited by 5,128
Bi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs
Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997) · Science · doi:10.1126/science.275.5297.213 · Cited by 3,832
Morris, R. G. M., Anderson, E., Lynch, G. S., & Baudry, M. (1986) · Nature · doi:10.1038/319774a0 · Cited by 3,640
Dopamine receptors: from structure to function
MISSALE, C., NASH, S. R., ROBINSON, S. W., JABER, M., & CARON, M. G. (1998) · Physiological Reviews · doi:10.1152/physrev.1998.78.1.189 · Cited by 3,633
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Synaptic Plasticity and Memory: An Evaluation of the Hypothesis
Martin, S. J., Grimwood, P. D., & Morris, R. G. M. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.649 · Cited by 2,794 · cited 8×
“Spike-Timing-Dependent Plasticity (STDP; Gerstner et al., 1996; Markram et al., 1997; Bi and Poo, 1998; Sjöström et al., 2001, can be considered as a temporally precise form of Hebbian synaptic plasticity, induced by isolated spikes in pre- and postsynaptic neurons (for reviews see e.g., Abbott and Nelson, 2000; Bi an…”
BackgroundMethodInfluentialLong-Term Potentiation--A Decade of Progress?
Malenka, R. C., & Nicoll, A. R. A. (1999) · Science · doi:10.1126/science.285.5435.1870 · Cited by 2,729
Competitive Hebbian learning through spike-timing-dependent synaptic plasticity
Song, S., Miller, K. D., & Abbott, L. F. (2000) · Nature Neuroscience · doi:10.1038/78829 · Cited by 2,729
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Synaptic plasticity: taming the beast
Abbott, L. F., & Nelson, S. B. (2000) · Nature Neuroscience · doi:10.1038/81453 · Cited by 2,263 · cited 3×
“…or LTD, depending on the relative firing frequency and timing of pre- and postsynaptic neurons (Levy and Stewart, 1983; Malenka and Nicoll, 1999; Abbott and Nelson, 2000; Bi and Poo, 2001; Markram et al., 2011) and voltage of the postsynaptic neuron (Artola and Singer, 1993; Sjöström et al.,…”
BackgroundMemory deficits associated with senescence: A neurophysiological and behavioral study in the rat.
Barnes, C. A. (1979) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0077579 · Cited by 2,088
Spike Timing–Dependent Plasticity: A Hebbian Learning Rule
Caporale, N., & Dan, Y. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.31.060407.125639 · Cited by 1,997 · cited 1×
“…be considered as a temporally precise form of Hebbian synaptic plasticity, induced by isolated spikes in pre- and postsynaptic neurons (for reviews see e.g., Abbott and Nelson, 2000; Bi and Poo, 2001; Caporale and Dan, 2008; Sjöström et al., 2008; Sjöström and Gerstner, 2010; Markram et al., 2011).”
BackgroundPlace Cells, Grid Cells, and the Brain's Spatial Representation System
Moser, E. I., Kropff, E., & Moser, M.-B. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.31.061307.090723 · Cited by 1,934
The locus coeruleus and noradrenergic modulation of cognition
Sara, S. J. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2573 · Cited by 1,848
Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
van Vreeswijk, C., & Sompolinsky, H. (1996) · Science · doi:10.1126/science.274.5293.1724 · Cited by 1,811
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“Reward, novelty or surprise are correlated with neuromodulatory signals, such as dopamine, acetylcholine or noradrenaline (Schultz, 2002; Ranganath and Rainer, 2003; Yu and Dayan, 2005).”
BackgroundSynaptic tagging and long-term potentiation
Frey, U., & Morris, R. G. M. (1997) · Nature · doi:10.1038/385533a0 · Cited by 1,736 · cited 7×
“The biological plausibility of eligibility traces and its potential relation to synaptic “tags” (Frey and Morris, 1997; Bailey et al., 2000; Redondo and Morris, 2011) will be explored in the discussion section.”
BackgroundInfluentialDynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons
Brunel, N. (2000) · Journal of Computational Neuroscience · doi:10.1023/a:1008925309027 · Cited by 1,726
The role of acetylcholine in learning and memory
Hasselmo, M. E. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.09.002 · Cited by 1,693
Behavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637 · cited 2×
“…field of neuromodulation, plasticity, and behavioral learning has been reviewed before (Martin et al., 2000; Gu, 2002; Reynolds and Wickens, 2002; Schultz, 2002, 2006; Hasselmo, 2006; Shohamy and Adcock, 2010; Lisman et al., 2011; Nadim and Bucher, 2014), this review mainly focuses on the case…”
BackgroundNeuronal Dynamics: From Single Neurons to Networks and Models of Cognition
Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014) · doi:10.1017/cbo9781107447615 · Cited by 1,515
Synaptic Modification by Correlated Activity: Hebb's Postulate Revisited
Bi, G.-Q., & Poo, M.-M. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.139 · Cited by 1,489
Dynamic Programming: Deterministic and Stochastic Models
Bertsekas, D. P. (1987) · Medical Entomology and Zoology · Cited by 1,325
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
The ART of adaptive pattern recognition by a self-organizing neural network
Carpenter, G. A., & Grossberg, S. (1988) · Computer · doi:10.1109/2.33 · Cited by 1,287
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266
Cortical Map Reorganization Enabled by Nucleus Basalis Activity
Kilgard, M. P., & Merzenich, M. M. (1998) · Science · doi:10.1126/science.279.5357.1714 · Cited by 1,195
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190
Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity
Sjöström, P. J., Turrigiano, G. G., & Nelson, S. B. (2001) · Neuron · doi:10.1016/s0896-6273(01)00542-6 · Cited by 1,174
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,117
A neuronal learning rule for sub-millisecond temporal coding
Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner, H. (1996) · Nature · doi:10.1038/383076a0 · Cited by 1,110
Self-citationDopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens
Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.185 · Cited by 931
State-dependent computations: spatiotemporal processing in cortical networks
Buonomano, D. V., & Maass, W. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2558 · Cited by 926
Neuromodulation of neuronal circuits: back to the future
Marder, E. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.010 · Cited by 904
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 869
Freund, T. F., Powell, J. F., & Smith, A. D. (1984) · Neuroscience · doi:10.1016/0306-4522(84)90294-x · Cited by 865
Dopamine-mediated regulation of corticostriatal synaptic plasticity
Calabresi, P., Picconi, B., Tozzi, A., & Di Filippo, M. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.001 · Cited by 865
Artola, A., & Singer, W. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90081-v · Cited by 795
Making memories last: the synaptic tagging and capture hypothesis
Redondo, R. L., & Morris, R. G. M. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn2963 · Cited by 792 · cited 2×
“The biological plausibility of eligibility traces and its potential relation to synaptic “tags” (Frey and Morris, 1997; Bailey et al., 2000; Redondo and Morris, 2011) will be explored in the discussion section.”
BackgroundNeural mechanisms for detecting and remembering novel events
Ranganath, C., & Rainer, G. (2003) · Nature Reviews Neuroscience · doi:10.1038/nrn1052 · Cited by 779
Levy, W. B., & Steward, O. (1983) · Neuroscience · doi:10.1016/0306-4522(83)90010-6 · Cited by 766
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
States of Curiosity Modulate Hippocampus-Dependent Learning via the Dopaminergic Circuit
Gruber, M. J., Gelman, B. D., & Ranganath, C. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.060 · Cited by 734
Dendritic Excitability and Synaptic Plasticity
Sjöström, P. J., Rancz, E. A., Roth, A., & Häusser, M. (2008) · Physiological Reviews · doi:10.1152/physrev.00016.2007 · Cited by 725
Rational regulation of learning dynamics by pupil-linked arousal systems
Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012) · Nature Neuroscience · doi:10.1038/nn.3130 · Cited by 722
Shohamy, D., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 707
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Synaptic tagging: implications for late maintenance of hippocampal long-term potentiation
Frey, U., & Morris, R. G. M. (1998) · Trends in Neurosciences · doi:10.1016/s0166-2236(97)01189-2 · Cited by 689
Reversing pathological neural activity using targeted plasticity
Engineer, N. D., Riley, J. R., Seale, J. D., Vrana, W. A., Shetake, J. A., Sudanagunta, S. P., Borland, M. S., & Kilgard, M. P. (2011) · Nature · doi:10.1038/nature09656 · Cited by 668
Hebbian learning and spiking neurons
Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999) · Physical Review E · doi:10.1103/physreve.59.4498 · Cited by 664
Self-citationSolving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling
Izhikevich, E. M. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl152 · Cited by 657
Packard, M. G., & McGaugh, J. L. (1992) · Behavioral Neuroscience · doi:10.1037//0735-7044.106.3.439 · Cited by 641
Dendritic spikes as a mechanism for cooperative long-term potentiation
Golding, N. L., Staff, N. P., & Spruston, N. (2002) · Nature · doi:10.1038/nature00854 · Cited by 621
Heterosynaptic depression: a postsynaptic correlate of long-term potentiation
LYNCH, G. S., DUNWIDDIE, T., & GRIBKOFF, V. (1977) · Nature · doi:10.1038/266737a0 · Cited by 615
Connectivity reflects coding: a model of voltage-based STDP with homeostasis
Clopath, C., Büsing, L., Vasilaki, E., & Gerstner, W. (2010) · Nature Neuroscience · doi:10.1038/nn.2479 · Cited by 607
Self-citationHökfelt, T., Broberger, C., Xu, Z.-Q. D., Sergeyev, V., Ubink, R., & Diez, M. (2000) · Neuropharmacology · doi:10.1016/s0028-3908(00)00010-1 · Cited by 604
Phenomenological models of synaptic plasticity based on spike timing
Morrison, A., Diesmann, M., & Gerstner, W. (2008) · Biological Cybernetics · doi:10.1007/s00422-008-0233-1 · Cited by 593
Self-citationDopamine: a potential substrate for synaptic plasticity and memory mechanisms
Jay, T. M. (2003) · Progress in Neurobiology · doi:10.1016/s0301-0082(03)00085-6 · Cited by 577
Curious model-building control systems
Schmidhuber, J. (1991) · [Proceedings] 1991 IEEE International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.170605 · Cited by 541
Dopamine-dependent plasticity of corticostriatal synapses
Reynolds, J. N. J., & Wickens, J. R. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00045-x · Cited by 535
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525
Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity
Gu, Q. (2002) · Neuroscience · doi:10.1016/s0306-4522(02)00026-x · Cited by 518
Udenfriend, S. (1966) · Pharmacological Reviews · doi:10.1016/s0031-6997(25)07173-x · Cited by 501
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Vogels, T. P., Rajan, K., & Abbott, L. F. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135637 · Cited by 497
Cortical Development and Remapping through Spike Timing-Dependent Plasticity
Song, S., & Abbott, L. F. (2001) · Neuron · doi:10.1016/s0896-6273(01)00451-2 · Cited by 476
A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP
Lisman, J., Grace, A. A., & Duzel, E. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.07.006 · Cited by 465 · cited 13×
“Since three-factor rules are a modern generalization of the original concept of Hebb, they are also called “neo-Hebbian” (Lisman et al., 2011).”
BackgroundMethodInfluentialA history of spike-timing-dependent plasticity
Markram, H. (2011) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2011.00004 · Cited by 454
Robust timing and motor patterns by taming chaos in recurrent neural networks
Laje, R., & Buonomano, D. V. (2013) · Nature Neuroscience · doi:10.1038/nn.3405 · Cited by 454
Sjöström, P. J., & Häusser, M. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.017 · Cited by 439
Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex.
Goldman-Rakic, P. S., Leranth, C., Williams, S. M., Mons, N., & Geffard, M. (1989) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.86.22.9015 · Cited by 433
Neuromodulators control the polarity of spike-timing-dependent synaptic plasticity
Seol, G. H., Ziburkus, J., Huang, S., Song, L., Kim, I. T., Takamiya, K., Huganir, R. L., Lee, H.-K., & Kirkwood, A. (2007) · Neuron · doi:10.1016/j.neuron.2007.08.013 · Cited by 429
Dopamine gates LTP induction in lateral amygdala by suppressing feedforward inhibition
Bissière, S., Humeau, Y., & Lüthi, A. (2003) · Nature Neuroscience · doi:10.1038/nn1058 · Cited by 418
Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent Plasticity
Pawlak, V., & Kerr, J. N. D. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4402-07.2008 · Cited by 412
Mathematical formulations of Hebbian learning
Gerstner, W., & Kistler, W. M. (2002) · Biological Cybernetics · doi:10.1007/s00422-002-0353-y · Cited by 408
Self-citationKerr, J. N. D., & Wickens, J. R. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.85.1.117 · Cited by 383
Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory
Bailey, C. H., Giustetto, M., Huang, Y.-Y., Hawkins, R. D., & Kandel, E. R. (2000) · Nature Reviews Neuroscience · doi:10.1038/35036191 · Cited by 382 · cited 3×
“The biological plausibility of eligibility traces and its potential relation to synaptic “tags” (Frey and Morris, 1997; Bailey et al., 2000; Redondo and Morris, 2011) will be explored in the discussion section.”
BackgroundMoncada, D., & Viola, H. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1083-07.2007 · Cited by 370
Responses to reward in monkey dorsal and ventral striatum
Apicella, P., Ljungberg, T., Scarnati, E., & Schultz, W. (1991) · Experimental Brain Research · doi:10.1007/bf00231732 · Cited by 369
Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity
Florian, R. V. (2007) · Neural Computation · doi:10.1162/neco.2007.19.6.1468 · Cited by 358
Optimal Control of Transient Dynamics in Balanced Networks Supports Generation of Complex Movements
Hennequin, G., Vogels, T. P., & Gerstner, W. (2014) · Neuron · doi:10.1016/j.neuron.2014.04.045 · Cited by 346
Self-citationInfinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343
Learning in spiking neural networks by reinforcement of stochastic synaptic transmission
Seung, H. S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00761-x · Cited by 340 · cited 2×
“A similar argument, albeit with a weaker theoretical foundation, is made in the case of rewardmodulated STDP with eligibility traces (Klopf, 1982; Seung, 2003; Farries and Fairhall, 2007; Florian, 2007; Izhikevich, 2007): synapses that underwent pairing just before a reward should get a stronger…”
BackgroundNeuromodulation of neurons and synapses
Nadim, F., & Bucher, D. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.05.003 · Cited by 338
Gain Modulation by Nicotine in Macaque V1
Disney, A. A., Aoki, C., & Hawken, M. J. (2007) · Neuron · doi:10.1016/j.neuron.2007.09.034 · Cited by 331
Reymann, K. G., & Frey, J. U. (2007) · Neuropharmacology · doi:10.1016/j.neuropharm.2006.07.026 · Cited by 331
Advances in Neural Information Processing Systems 23 (NIPS 2010)
Seth, S., Park, I. M., Brockmeier, A. J., Semework, M., Choi, J. S., Francis, J. T., & Príncipe, J. C. (2010) · Cited by 318
Suri, R. E., & Schultz, W. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00697-6 · Cited by 294
A model of hippocampally dependent navigation, using the temporal difference learning rule
Foster, D. J., Morris, R. G. M., & Dayan, P. (2000) · Hippocampus · doi:10.1002/(sici)1098-1063(2000)10:1<1::aid-hipo1>3.0.co;2-1 · Cited by 290
Timing-Dependent Septal Cholinergic Induction of Dynamic Hippocampal Synaptic Plasticity
Gu, Z., & Yakel, J. L. (2011) · Neuron · doi:10.1016/j.neuron.2011.04.026 · Cited by 285
Nessler, B., Pfeiffer, M., Buesing, L., & Maass, W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003037 · Cited by 281
Introduction to the Theory: Coming to Grips with Loss
Cummings, K. (2015) · Constructing Knowledge: Curriculum Studies In Action · doi:10.1007/978-94-6300-250-9_1 · Cited by 276
Emotional enhancement of memory: how norepinephrine enables synaptic plasticity
Tully, K., & Bolshakov, V. Y. (2010) · Molecular Brain · doi:10.1186/1756-6606-3-15 · Cited by 275
The convergence of TD(?) for general ?
Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992701 · Cited by 275
Unsupervised Learning: Foundations of Neural Computation
(1999) · The MIT Press eBooks · doi:10.7551/mitpress/7011.001.0001 · Cited by 272
Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity
Arleo, A., & Gerstner, W. (2000) · Biological Cybernetics · doi:10.1007/s004220000171 · Cited by 270
Self-citationLegenstein, R., Pecevski, D., & Maass, W. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000180 · Cited by 267
The role of acetylcholine in cortical synaptic plasticity
Rasmusson, D. D. (2000) · Behavioural Brain Research · doi:10.1016/s0166-4328(00)00259-x · Cited by 265
Timing is not everything: neuromodulation opens the STDP gate
Pawlak, V. (2010) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2010.00146 · Cited by 255
Couey, J. J., Meredith, R. M., Spijker, S., Poorthuis, R. B., Smit, A. B., Brussaard, A. B., & Mansvelder, H. D. (2007) · Neuron · doi:10.1016/j.neuron.2007.03.006 · Cited by 253
Spike-timing dependent plasticity
Sjöström, J., & Gerstner, W. (2010) · Scholarpedia · doi:10.4249/scholarpedia.1362 · Cited by 253
Self-citationNeural circuits as computational dynamical systems
Sussillo, D. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.008 · Cited by 247
Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning
Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.6.1318 · Cited by 245
Self-citationLong-term potentiation: outstanding questions and attempted synthesis
Lisman, J. (2003) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1242 · Cited by 240
The cholinergic system and hippocampal plasticity
Drever, B. D., Riedel, G., & Platt, B. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.11.037 · Cited by 240
Theta Phase Precession in Rat Ventral Striatum Links Place and Reward Information
van der Meer, M. A. A., & Redish, A. D. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4869-10.2011 · Cited by 233
Zhang, J.-C., Lau, P.-M., & Bi, G.-Q. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0900546106 · Cited by 231
Limits on the memory storage capacity of bounded synapses
Fusi, S., & Abbott, L. F. (2007) · Nature Neuroscience · doi:10.1038/nn1859 · Cited by 224
Conditional modulation of spike-timing-dependent plasticity for olfactory learning
Cassenaer, S., & Laurent, G. (2012) · Nature · doi:10.1038/nature10776 · Cited by 216
Reinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons
Frémaux, N., Sprekeler, H., & Gerstner, W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003024 · Cited by 209
Self-citationCalabresi, P., Maj, R., Mercuri, N. B., & Bernardi, G. (1992) · Neuroscience Letters · doi:10.1016/0304-3940(92)90628-k · Cited by 202
Laje, R., & Buonomano, D. V. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.2104 · Cited by 200
Learning of sequential movements by neural network model with dopamine-like reinforcement signal
Suri, R. E., & Schultz, W. (1998) · Experimental Brain Research · doi:10.1007/s002210050467 · Cited by 197
Buchanan, K. A., Petrovic, M. M., Chamberlain, S. E. L., Marrion, N. V., & Mellor, J. R. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.018 · Cited by 184
Schmitzer-Torbert, N., & Redish, A. D. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00687.2003 · Cited by 175
Xu, T., & Yao, W. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1004108107 · Cited by 167
A cholinergic trigger drives learning-induced plasticity at hippocampal synapses
Mitsushima, D., Sano, A., & Takahashi, T. (2013) · Nature Communications · doi:10.1038/ncomms3760 · Cited by 166
Temporal Difference Model Reproduces Anticipatory Neural Activity
Suri, R. E., & Schultz, W. (2001) · Neural Computation · doi:10.1162/089976601300014376 · Cited by 166
The role of NPY in learning and memory
Gøtzsche, C. R., & Woldbye, D. P. D. (2016) · Neuropeptides · doi:10.1016/j.npep.2015.09.010 · Cited by 165
Cortical Synapses and Reinforcement: a Hypothesis
CROW, T. J. (1968) · Nature · doi:10.1038/219736a0 · Cited by 159
An experimental analysis of surprise
Meyer, W.-U., Niepel, M., Rudolph, U., & Schützwohl, A. (1991) · Cognition & Emotion · doi:10.1080/02699939108411042 · Cited by 159
Learning in neural networks by reinforcement of irregular spiking
Xie, X. S., & Seung, H. S. (2004) · Physical Review E · doi:10.1103/physreve.69.041909 · Cited by 157
Seidenbecher, T., Reymann, K. G., & Balschun, D. (1997) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.94.4.1494 · Cited by 154
Reinforcement learning: Computational theory and biological mechanisms
Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246 · Cited by 150
Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity
Frémaux, N., Sprekeler, H., & Wulfram Gerstner, B. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6249-09.2010 · Cited by 148
Self-citationReinforcement Learning With Modulated Spike Timing–Dependent Synaptic Plasticity
Farries, M. A., & Fairhall, A. L. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00364.2007 · Cited by 148
Tag-Trigger-Consolidation: A Model of Early and Late Long-Term-Potentiation and Depression
Clopath, C., Ziegler, L., Vasilaki, E., Buesing, L., & Wulfram Gerstner, B. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000248 · Cited by 147
Is there a geometric module for spatial orientation? Insights from a rodent navigation model.
Sheynikhovich, D., Chavarriaga, R., Strösslin, T., Arleo, A., & Wulfram Gerstner, B. (2009) · Psychological Review · doi:10.1037/a0016170 · Cited by 143
A computational analysis of the neural bases of Bayesian inference
Kolossa, A., Kopp, B., & Fingscheidt, T. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.11.007 · Cited by 133
Matching Recall and Storage in Sequence Learning with Spiking Neural Networks
Brea, J. M., Senn, W., & Pfister, J.-P. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4098-12.2013 · Cited by 132
Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates
Fusi, S. (2002) · Biological Cybernetics · doi:10.1007/s00422-002-0356-8 · Cited by 131
Vasilaki, E., Frémaux, N., Urbanczik, R., Senn, W., & Wulfram Gerstner, B. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000586 · Cited by 130
Self-citationLin, Y., Min, M., Chiu, T., & Yang, H. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-10-04173.2003 · Cited by 126
A Kinetic Model of Dopamine- and Calcium-Dependent Striatal Synaptic Plasticity
Nakano, T., Doi, T., Yoshimoto, J., & Doya, K. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000670 · Cited by 122
Loewenstein, Y., & Seung, H. S. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0505220103 · Cited by 113
A Spiking Neural Network Model of an Actor-Critic Learning Agent
Potjans, W., Morrison, A., & Diesmann, M. (2008) · Neural Computation · doi:10.1162/neco.2008.08-07-593 · Cited by 111
Introduction. Long-term potentiation and structure of the issue.
Bliss, T., Collingridge, G., & Morris, R. (2003) · Philosophical transactions of the Royal Society of London. Series B, Biological sciences · doi:10.1098/rstb.2003.1282 · Cited by 110
Emotional tagging—A simple hypothesis in a complex reality
Bergado, J. A., Lucas, M., & Richter‐Levin, G. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.03.004 · Cited by 104
Arleo, A., Smeraldi, F., & Wulfram Gerstner, B. (2004) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.826221 · Cited by 101
Disney, A. A., Domakonda, K. V., & Aoki, C. (2006) · The Journal of Comparative Neurology · doi:10.1002/cne.21096 · Cited by 99
Neuromodulation: The biochemical control of neuronal excitability
Kennedy, M. B. (1987) · Cell · doi:10.1016/0092-8674(87)90487-9 · Cited by 97
Reinforcement learning in populations of spiking neuronsPreprint
Urbanczik, R., & Senn, W. (2008) · Nature Precedings · doi:10.1038/nn.2264 · Cited by 94
State Based Model of Long-Term Potentiation and Synaptic Tagging and Capture
Barrett, A. B., Billings, G. O., Morris, R., & van Rossum, M. C. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000259 · Cited by 89
Boosting visual cortex function and plasticity with acetylcholine to enhance visual perception
Kang, J. I., Huppé‐Gourgues, F., & Vaucher, E. (2014) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2014.00172 · Cited by 86
Stochastic variational learning in recurrent spiking networks
Rezende, D. J., & Wulfram Gerstner, B. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00038 · Cited by 78
The possible role of the adrenergic systems of the cortex in learning.
Ss, K. (1972) · PubMed · Cited by 77
White, R. H. (1991) · IJCNN-91-Seattle International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.155554 · Cited by 76
Reinforcement Learning, Spike-Time-Dependent Plasticity, and the BCM Rule
Baras, D., & Meir, R. (2007) · Neural Computation · doi:10.1162/neco.2007.19.8.2245 · Cited by 72
Ma, X., & Suga, N. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0503851102 · Cited by 71
Spatio-Temporal Credit Assignment in Neuronal Population Learning
Friedrich, J. H., Urbanczik, R., & Senn, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002092 · Cited by 68
Cholinergic modulation on spike timing-dependent plasticity in hippocampal CA1 network
Sugisaki, E., Fukushima, Y., Tsukada, M., & Aihara, T. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.06.064 · Cited by 60
An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning
Potjans, W., Diesmann, M., & Morrison, A. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001133 · Cited by 60
Svensson, E., Grillner, S. E., & Parker, D. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-16-05984.2001 · Cited by 54
Toyoizumi, T., Pfister, J.-P., Aihara, K., & Wulfram Gerstner, B. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.639 · Cited by 50
Bliss, T. V. P. (1990) · Science · doi:10.1126/science.2396098 · Cited by 50
Li, N., & DiCarlo, J. J. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.3786-11.2012 · Cited by 49
Bohté, S. M., & Mozer, M. C. (2007) · Neural Computation · doi:10.1162/neco.2007.19.2.371 · Cited by 47
Sussillo, D., & Abbott, L. F. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037372 · Cited by 45
Cortico-Striatal Spike-Timing Dependent Plasticity After Activation of Subcortical Pathways
Schulz, J. M., Redgrave, P., & Reynolds, J. N. (2010) · Front. Syn. Neurosci. · doi:10.3389/fnsyn.2010.00023 · Cited by 42
Perceptual learning, roving and the unsupervised bias
Herzog, M. H., Aberg, K. C., Frémaux, N., Wulfram Gerstner, B., & Sprekeler, H. (2011) · Vision Research · doi:10.1016/j.visres.2011.11.001 · Cited by 30
Self-citationRobustness of Learning That Is Based on Covariance-Driven Synaptic Plasticity
Loewenstein, Y. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000007 · Cited by 28
Poole, D. L., & Mackworth, A. K. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009258227.018 · Cited by 24
Temporal Difference Based Actor Critic Learning - Convergence and Neural Implementation
Castro, D. D., Volkinshtein, D., & Meir, R. (2008) · neural information processing systems · Cited by 15
On the Maximization of Information Flow Between Spiking Neurons
Parra, L. C., Beck, J. M., & Bell, A. J. (2009) · Neural Computation · doi:10.1162/neco.2009.04-06-184 · Cited by 11
Dimensional reduction for reward-based learning
Swinehart, C. D., & Abbott, L. F. (2006) · Network Computation in Neural Systems · doi:10.1080/09548980600773215 · Cited by 9
Cited by
Tang, J., Yuan, F., Shen, X., Wang, Z., Rao, M., He, Y., Sun, Y., Li, X., Zhang, W., Li, Y., et al. (2019) · Advanced Materials · doi:10.1002/adma.201902761 · Cited by 834
Synaptic Plasticity Forms and Functions
Magee, J. C., & Grienberger, C. (2020) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-090919-022842 · Cited by 806
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., & Tian, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00266 · Cited by 605
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
Zenke, F., & Ganguli, S. (2018) · Neural Computation · doi:10.1162/neco_a_01086 · Cited by 515
A solution to the learning dilemma for recurrent networks of spiking neurons
Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020) · Nature Communications · doi:10.1038/s41467-020-17236-y · Cited by 506
Metadata
- Authors
- Frémaux, N., & Gerstner, W.
- Venue
- Frontiers in Neural Circuits
- Publisher
- Frontiers Media SA
- Published
- 2016-01-19
- Volume
- 9
- Pages
- 85-85
- ISSN
- 1662-5110 (ISSN-L)
- OpenAlex
- W2271476098
- Cited by
- 413
- References
- 181
- FWCI
- 16.94
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.