Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates
Abstract
Synaptic plasticity is believed to underlie the formation of appropriate patterns of connectivity that stabilize stimulus-selective reverberations in the cortex. Here we present a general quantitative framework for studying the process of learning and memorizing of patterns of mean spike rates. General considerations based on the limitations of material (biological or electronic) synaptic devices show that most learning networks share the palimpsest property: old stimuli are forgotten to make room for the new ones. In order to prevent too-fast forgetting, one can introduce a stochastic mechanism for selecting only a small fraction of synapses to be changed upon the presentation of a stimulus. Such a mechanism can be easily implemented by exploiting the noisy fluctuations in the pre- and postsynaptic activities to be encoded. The spike-driven synaptic dynamics described here can implement such a selection mechanism to achieve slow learning, which is shown to maximize the performance of the network as an associative memory.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
We used binary synapses that undergo a stochastic Hebbian learning rule, namely that synaptic plasticity depends on coactivation of presynaptic and postsynaptic neurons and takes place probabilistically.
“Specifically, we used binary synapses that undergo a stochastic Hebbian learning rule, namely that synaptic plasticity depends on coactivation of presynaptic and postsynaptic neurons and takes place probabilistically (Fusi, 2002; Fusi et al., 2007).”
From Decision Making in Recurrent Neuronal Circuits (2008) · cites this paper as Fusi, 2002
Cites
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270 · see citing works
A synaptic model of memory: long-term potentiation in the hippocampus
Bliss, T. V. P., & Collingridge, G. L. (1993) · Nature · doi:10.1038/361031a0 · Cited by 11,566 · see citing works
The Organization of Behavior; A Neuropsychological Theory.
Attneave, F., B., M., & Hebb, D. O. (1950) · The American Journal of Psychology · doi:10.2307/1418888 · Cited by 5,146 · see citing works
Amit, D. J. (1989) · Cambridge University Press eBooks · doi:10.1017/cbo9780511623257 · Cited by 2,279 · see citing works
Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity
van Vreeswijk, C., & Sompolinsky, H. (1996) · Science · doi:10.1126/science.274.5293.1724 · Cited by 1,831 · see citing works
Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons
Brunel, N. (2000) · Journal of Computational Neuroscience · doi:10.1023/a:1008925309027 · Cited by 1,733 · see citing works
Softky, W., & Koch, C. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-01-00334.1993 · Cited by 1,689 · see citing works
Introduction to Theoretical Neurobiology
Tuckwell, H. C. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511623202 · Cited by 1,174 · see citing works
Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity
Sjöström, P. J., Turrigiano, G. G., & Nelson, S. B. (2001) · Neuron · doi:10.1016/s0896-6273(01)00542-6 · Cited by 1,174 · see citing works
Amit, D. (1997) · Cerebral Cortex · doi:10.1093/cercor/7.3.237 · Cited by 1,061 · see citing works
Synaptic reverberation underlying mnemonic persistent activity
Wang, X.-J. (2001) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01868-3 · Cited by 1,059 · see citing works
A critical window for cooperation and competition among developing retinotectal synapses
Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., & Poo, M. (1998) · Nature · doi:10.1038/25665 · Cited by 895 · see citing works
Storing covariance with nonlinearly interacting neurons
Sejnowski, T. J. (1977) · Journal of Mathematical Biology · doi:10.1007/bf00275079 · Cited by 546 · see citing works
Fuster, J. M. (1999) · Cited by 456 · see citing works
Inferior Temporal Cortex: Where Visual Perception Meets Memory
Miyashita, Y. (1993) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.16.030193.001333 · Cited by 446 · see citing works
Diffusion Processes and Related Topics in Biology
Ricciardi, L. M. (1977) · Lecture notes in biomathematics · doi:10.1007/978-3-642-93059-1 · Cited by 407 · see citing works
The Hebbian paradigm reintegrated: local reverberations as internal representations
Amit, D. J. (1995) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00040164 · Cited by 388 · see citing works
Learning in Neural Networks with Material Synapses
Amit, D., & Fusi, S. (1994) · Neural Computation · doi:10.1162/neco.1994.6.5.957 · Cited by 299 · see citing works
Self-citationDynamics of a recurrent network of spiking neurons before and following learning
Amit, D. J., & Brunel, N. (1997) · Network Computation in Neural Systems · doi:10.1088/0954-898x_8_4_003 · Cited by 241 · see citing works
Spike-Driven Synaptic Plasticity: Theory, Simulation, VLSI Implementation
Fusi, S., Annunziato, M., Badoni, D., Salamon, A., & Amit, D. J. (2000) · Neural Computation · doi:10.1162/089976600300014917 · Cited by 227 · see citing works
Self-citationNetworks of Formal Neurons and Memory Palimpsests
Nadal, J., Toulouse, G., Changeux, J., & Dehaene, S. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/10/008 · Cited by 213 · see citing works
Parisi, G. (1986) · Journal of Physics A Mathematical and General · doi:10.1088/0305-4470/19/10/011 · Cited by 172 · see citing works
Collective Behavior of Networks with Linear (VLSI) Integrate-and-Fire Neurons
Fusi, S., & Mattia, M. (1999) · Neural Computation · doi:10.1162/089976699300016601 · Cited by 163 · see citing works
Self-citationNeural representation of visual objects: encoding and top-down activation
Miyashita, Y. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00071-4 · Cited by 140 · see citing works
Yakovlev, V., Fusi, S., Berman, E., & Zohary, E. (1998) · Nature Neuroscience · doi:10.1038/1131 · Cited by 108 · see citing works
Self-citationLearning processes in neural networks
Heskes, T., & Kappen, B. (1991) · Physical Review A · doi:10.1103/physreva.44.2718 · Cited by 105 · see citing works
Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network
Brunel, N., Carusi, F., & Fusi, S. (1998) · Network Computation in Neural Systems · doi:10.1088/0954-898x_9_1_007 · Cited by 78 · see citing works
Self-citationSpike-Driven Synaptic Dynamics Generating Working Memory States
Amit, D. J., & Mongillo, G. (2003) · Neural Computation · doi:10.1162/089976603321192086 · Cited by 74 · see citing works
Giudice, P. D., Fusi, S., & Mattia, M. (2003) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.01.021 · Cited by 74 · see citing works
Self-citationParadigmatic Working Memory (Attractor) Cell in IT Cortex
Amit, D. J., Fusi, S., & Yakovlev, V. (1997) · Neural Computation · doi:10.1162/neco.1997.9.5.1071 · Cited by 60 · see citing works
Self-citationConstraints on learning in dynamic synapses
Amit, D. J., & Fusi, S. (1992) · Network Computation in Neural Systems · doi:10.1088/0954-898x_3_4_008 · Cited by 55 · see citing works
Self-citationBeyond spike timing: the role of nonlinear plasticity and unreliable synapses
Senn, W. (2002) · Biological Cybernetics · doi:10.1007/s00422-002-0350-1 · Cited by 49 · see citing works
Electronic implementation of an analogue attractor neural network with stochastic learning
Badoni, D., Bertazzoni, S., Buglioni, S., Salina, G., Amit, D. J., & Fusi, S. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x_6_2_002 · Cited by 41 · see citing works
Self-citationLong and short-term synaptic plasticity and the formation of working memory: A case study
Giudice, P. D., & Mattia, M. (2001) · Neurocomputing · doi:10.1016/s0925-2312(01)00557-4 · Cited by 24 · see citing works
Stochastic synaptic plasticity in deterministic aVLSI networks of spiking neurons
Chicca, E., & Fusi, S. (2001) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 21 · see citing works
Self-citationLearning attractors in an asynchronous, stochastic electronic neural network
Giudice, P. D., Fusi, S., Badoni, D., et al. (1998) · Network Computation in Neural Systems · doi:10.1088/0954-898x/9/2/003 · Cited by 11 · see citing works
Self-citationLong term memory: Encoding and storing strategies of the brain
Fusi, S. (2001) · Neurocomputing · doi:10.1016/s0925-2312(01)00571-9 · Cited by 11 · see citing works
Self-citationStochastic learning in a neural network with adapting synapses
Lattanzi, G., Nardulli, G., Pasquariello, G., et al. (1997) · Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics · doi:10.1103/physreve.56.4567 · Cited by 10 · see citing works
LEARNING IN NEURAL NETWORKS WITH PARTIALLY STRUCTURED SYNAPTIC TRANSITIONS
Battaglia, F., & Fusi, S. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x_6_2_007 · Cited by 7 · see citing works
Self-citationPrototype extraction in material attractor neural networks with stochastic dynamic learning
Fusi, S. (1995) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.205099 · Cited by 6 · see citing works
Self-citationSpike- Driven Synaptic Plasticity for Learning Correlated Patterns of Asynchronous Activity
Fusi, S. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46084-5_40 · Cited by 5 · see citing works
Self-citationA memory model with novel behaviour in sequential learning
Kahn, P. E., Wong, K. Y. M., & Sherrington, D. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x_6_3_007 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Decision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679 · see citing works
“Specifically, we used binary synapses that undergo a stochastic Hebbian learning rule, namely that synaptic plasticity depends on coactivation of presynaptic and postsynaptic neurons and takes place probabilistically (Fusi, 2002; Fusi et al., 2007).”
MethodFusi, S., Asaad, W. F., Miller, E. K., & Wang, X.-J. (2007) · Neuron · doi:10.1016/j.neuron.2007.03.017 · Cited by 199 · see citing works
Self-citationTheoretical Neuroscience Rising
Abbott, L. F. (2008) · Neuron · doi:10.1016/j.neuron.2008.10.019 · Cited by 163 · see citing works
Neural mechanism for stochastic behaviour during a competitive game
Soltani, A., Lee, D., & Wang, X.-J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.044 · Cited by 77 · see citing works
Long-Term Memory Stabilized by Noise-Induced Rehearsal
Wei, Y., & Koulakov, A. A. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3929-12.2014 · Cited by 29 · see citing works
- Authors
- Fusi, S.
- Affiliations
- University of Bern (CH)
- Corresponding author
- Fusi, Stefano
- ORCID
- Fusi, Stefano
- Venue
- Biological Cybernetics
- Published
- 2002-12-01
- Volume
- 87
- Issue
- 5-6
- Pages
- 459-470
- ISSN
- 0340-1200 (ISSN-L)
- OpenAlex
- W2092001091
- Cited by
- 131
- References
- 42
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 5 · 2023: 3 · 2022: 5 · 2021: 3 · 2020: 2 · 2018: 2 · 2017: 3 · 2016: 12 · 2015: 4 · 2014: 9 · 2013: 6 · 2012: 3
- FWCI
- 4.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Biological Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- Hebbian learning, associative memory, forgetting, slow learning, pattern memorization
- MeSH terms
- Action Potentials, Animals, Learning, Mathematics, Memory, Nerve Net, Neuronal Plasticity, Neurons, Synapses, Time Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
