A biologically plausible supervised learning method for spiking neural networks using the symmetric STDP rule
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Spiking neural networks (SNNs) possess energy-efficient potential due to event-based computation. However, supervised training of SNNs remains a challenge as spike activities are non-differentiable. Previous SNNs training methods can be generally categorized into two basic classes, i.e., backpropagation-like training methods and plasticity-based learning methods. The former methods are dependent on energy-inefficient real-valued computation and non-local transmission, as also required in artificial neural networks (ANNs), whereas the latter are either considered to be biologically implausible or exhibit poor performance. Hence, biologically plausible (bio-plausible) high-performance supervised learning (SL) methods for SNNs remain deficient. In this paper, we proposed a novel bio-plausible SNN model for SL based on the symmetric spike-timing dependent plasticity (sym-STDP) rule found in neuroscience. By combining the sym-STDP rule with bio-plausible synaptic scaling and intrinsic plasticity of the dynamic threshold, our SNN model implemented SL well and achieved good performance in the benchmark recognition task (MNIST dataset). To reveal the underlying mechanism of our SL model, we visualized both layer-based activities and synaptic weights using the t-distributed stochastic neighbor embedding (t-SNE) method after training and found that they were well clustered, thereby demonstrating excellent classification ability. Furthermore, to verify the robustness of our model, we trained it on another more realistic dataset (Fashion-MNIST), which also showed good performance. As the learning rules were bio-plausible and based purely on local spike events, our model could be easily applied to neuromorphic hardware for online training and may be helpful for understanding SL information processing at the synaptic level in biological neural systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“, long short-term memory, LSTM) [7, 8] as their individual neurons generally have no intrinsic temporal characteristics and only generate output responses based on current inputs at each step.”
BackgroundLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 1×
“, long short-term memory, LSTM) [7, 8] as their individual neurons generally have no intrinsic temporal characteristics and only generate output responses based on current inputs at each step.”
BackgroundFashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108
Bi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 2×
“The LSM is a classic model of reservoir computing (RC) [67, 68, 69, 70, 64], which”
BackgroundMethodHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · cited 1×
“The LSM is a classic model of reservoir computing (RC) [67, 68, 69, 70, 64], which”
MethodDopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Activity-dependent scaling of quantal amplitude in neocortical neurons
Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., & Nelson, S. B. (1998) · Nature · doi:10.1038/36103 · Cited by 2,480 · cited 2×
“In our proposed model, a three-layer feedforward SNN was trained by DA-STDP combined with synaptic scaling [37, 38, 39] and dynamic threshold, which are two homeostatic plasticity mechanisms for stabilizing and specializing the network response under supervised signals.”
BackgroundMethodSpike Timing–Dependent Plasticity: A Hebbian Learning Rule
Caporale, N., & Dan, Y. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.31.060407.125639 · Cited by 1,997 · cited 2×
“Moreover, although great progress has been made in understanding the physiological mechanisms for the modification of synapses at the microscopic level [4, 5], how teacher learning at the macroscopic behavioral level is realized by changes in synapses at the microscopic level, i.”
BackgroundThe self-tuning neuron: synaptic scaling of excitatory synapses
Turrigiano, G. G. (2008) · Cell · doi:10.1016/j.cell.2008.10.008 · Cited by 1,533
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
Dopaminergic modulation of synaptic transmission in cortex and striatum
Tritsch, N. X., & Sabatini, B. L. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.023 · Cited by 756
Unraveling mechanisms of homeostatic synaptic plasticity
Pozo, K., & Goda, Y. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.028 · Cited by 565
HOMEOSTATIC CONTROL OF NEURAL ACTIVITY: From Phenomenology to Molecular Design
Davis, G. W. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135751 · Cited by 543
The BCM theory of synapse modification at 30: interaction of theory with experiment
Cooper, L. N., & Bear, M. F. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3353 · Cited by 395 · cited 1×
“The dynamic threshold is the intrinsic plasticity of a neuron, and is found in different neural systems [41, 42, 43, 44, 45].”
BackgroundMultiple Modes of Network Homeostasis in Visual Cortical Layer 2/3
Maffei, A., & Turrigiano, G. G. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.5298-07.2008 · Cited by 282 · cited 1×
“Synaptic scaling is a homeostatic plasticity mechanism observed in many experiments [44, 37, 47], especially in visual systems [48, 49, 50] and the neocortex [38].”
Background
Metadata
- Authors
- Hao, Y., Huang, X., Dong, M., & Xu, B.
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2020-01
- Volume
- 121
- Pages
- 387-395
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2905533880
- Cited by
- 187
- References
- 21
- FWCI
- 5.20
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.