The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
Open accessAbstract
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in silico. Here we revisit the problem of supervised learning in temporally coding multilayer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric, and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike time patterns.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309 · cited 1×
“5 was motivated by previous work on rapid compensatory processes as an effective way to constrain neuronal activity through linear scaling of afferent weights (Oja, 1982; Zenke et al., 2015).”
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition
Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014) · doi:10.1017/cbo9781107447615 · Cited by 1,515
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 2×
“…efficient auto-differentiation tools are not available for most event-based spiking neural network frameworks, and (ii) the method of weight updates implemented by the standard back-propagation of error algorithm (Backprop) is thought to be biologically implausible (Grossberg, 1987; Crick, 1989).”
MethodUnsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
The Spike-Timing Dependence of Plasticity
Feldman, D. E. (2012) · Neuron · doi:10.1016/j.neuron.2012.08.001 · Cited by 1,071
Error-backpropagation in temporally encoded networks of spiking neurons
Bohte, S. M., Kok, J. N., & La Poutré, H. (2002) · Neurocomputing · doi:10.1016/s0925-2312(01)00658-0 · Cited by 1,021
A Large-Scale Model of the Functioning Brain
Eliasmith, C., Stewart, T. C., Choo, X., Bekolay, T., DeWolf, T., Tang, Y., & Rasmussen, D. (2012) · Science · doi:10.1126/science.1225266 · Cited by 943
Artola, A., Bröcher, S., & Singer, W. (1990) · Nature · doi:10.1038/347069a0 · Cited by 879
The tempotron: a neuron that learns spike timing–based decisions
Gütig, R., & Sompolinsky, H. (2006) · Nature Neuroscience · doi:10.1038/nn1643 · Cited by 815
Random synaptic feedback weights support error backpropagation for deep learning
Lillicrap, T. P., Cownden, D., Tweed, D. B., & Akerman, C. J. (2016) · Nature Communications · doi:10.1038/ncomms13276 · Cited by 812
The recent excitement about neural networks
Crick, F. (1989) · Nature · doi:10.1038/337129a0 · Cited by 714
Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling
Izhikevich, E. M. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl152 · Cited by 657
Rapid Neural Coding in the Retina with Relative Spike Latencies
Gollisch, T., & Meister, M. (2008) · Science · doi:10.1126/science.1149639 · Cited by 651
Ponulak, F., & Kasiński, A. (2010) · Neural Computation · doi:10.1162/neco.2009.11-08-901 · Cited by 573
Rossum, M. C. W. V. (2001) · Neural Computation · doi:10.1162/089976601300014321 · Cited by 519
Efficient codes and balanced networks
Denève, S., & Machens, C. K. (2016) · Nature Neuroscience · doi:10.1038/nn.4243 · Cited by 471
Metric-space analysis of spike trains: theory, algorithms and application
Victor, J. D., & Purpura, K. P. (1997) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_8_2_003 · Cited by 426
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules
Frémaux, N., & Gerstner, W. (2016) · Frontiers in Neural Circuits · doi:10.3389/fncir.2015.00085 · Cited by 413
Towards deep learning with segregated dendrites
Guerguiev, J., Lillicrap, T. P., & Richards, B. A. (2017) · eLife · doi:10.7554/elife.22901 · Cited by 371
Zenke, F., Agnes, E. J., & Gerstner, W. (2015) · Nature Communications · doi:10.1038/ncomms7922 · Cited by 370
Self-citationDeep Learning Models of the Retinal Response to Natural Scenes
McIntosh, L., Maheswaranathan, N., Nayebi, A., Ganguli, S., & Baccus, S. A. (2017) · PubMed · doi:10.48550/arxiv.1702.01825 · Cited by 289
Self-citationSPAN: SPIKE PATTERN ASSOCIATION NEURON FOR LEARNING SPATIO-TEMPORAL SPIKE PATTERNS
MOHEMMED, A., SCHLIEBS, S., MATSUDA, S., & KASABOV, N. (2012) · International Journal of Neural Systems · doi:10.1142/s0129065712500128 · Cited by 270
Timing is not everything: neuromodulation opens the STDP gate
Pawlak, V. (2010) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2010.00146 · Cited by 255
Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning
Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.6.1318 · Cited by 245
The chronotron: a neuron that learns to fire temporally-precise spike patterns
Florian, R. (2010) · Nature Precedings · doi:10.1038/npre.2010.5190.1 · Cited by 204
Building functional networks of spiking model neurons
Abbott, L. F., DePasquale, B., & Memmesheimer, R.-M. (2016) · Nature Neuroscience · doi:10.1038/nn.4241 · Cited by 204
A learning rule for very simple universal approximators consisting of a single layer of perceptrons
Auer, P., Burgsteiner, H., & Maass, W. (2008) · Neural Networks · doi:10.1016/j.neunet.2007.12.036 · Cited by 179
Learning with three factors: modulating Hebbian plasticity with errors
Kuśmierz, Ł., Isomura, T., & Toyoizumi, T. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.08.020 · Cited by 174
Supervised Learning in Multilayer Spiking Neural Networks
Sporea, I., & Grüning, A. (2013) · Neural Computation · doi:10.1162/neco_a_00396 · Cited by 159
Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity
Frémaux, N., Sprekeler, H., & Wulfram Gerstner, B. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6249-09.2010 · Cited by 148
Learning Precisely Timed Spikes
Memmesheimer, R.-M., Rubin, R., Ölveczky, B. P., & Sompolinsky, H. (2014) · Neuron · doi:10.1016/j.neuron.2014.03.026 · Cited by 145
Spiking neurons can discover predictive features by aggregate-label learning
Gütig, R. (2016) · Science · doi:10.1126/science.aab4113 · Cited by 141
Matching Recall and Storage in Sequence Learning with Spiking Neural Networks
Brea, J. M., Senn, W., & Pfister, J.-P. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4098-12.2013 · Cited by 132
Fast Modifications of the SpikeProp Algorithm
McKennoch, S., Liu, D., & Bushnell, L. G. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246918 · Cited by 129
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network
Gilra, A., & Wulfram Gerstner, B. (2017) · eLife · doi:10.7554/elife.28295 · Cited by 100
Learning Universal Computations with Spikes
Thalmeier, D., Uhlmann, M., Kappen, H. J., & Memmesheimer, R.-M. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004895 · Cited by 99
Limits to high-speed simulations of spiking neural networks using general-purpose computers
Zenke, F., & Wulfram Gerstner, B. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00076 · Cited by 88
Self-citationStochastic variational learning in recurrent spiking networks
Rezende, D. J., & Wulfram Gerstner, B. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00038 · Cited by 78
Adaptive learning rate of SpikeProp based on weight convergence analysis
Shrestha, S. B., & Song, Q. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.12.001 · Cited by 65
(2017) · doi:10.3389/fncom.2016.00131 · Cited by 59
Error-Backpropagation in Networks of Fractionally Predictive Spiking Neurons
Bohté, S. M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21735-7_8 · Cited by 50
Learning Spatiotemporally Encoded Pattern Transformations in Structured Spiking Neural Networks
Brian, G., Sporea, I., & Grüning, A. (2015) · Neural Computation · doi:10.1162/neco_a_00790 · Cited by 49
Supervised Learning in Spiking Neural Networks for Precise Temporal Encoding
Brian, G., & Grüning, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0161335 · Cited by 49
Bohté, S. M., & Mozer, M. C. (2007) · Neural Computation · doi:10.1162/neco.2007.19.2.371 · Cited by 47
A Gradient Learning Rule for the Tempotron
Urbanczik, R., & Senn, W. (2008) · Neural Computation · doi:10.1162/neco.2008.09-07-605 · Cited by 25
Learning of Precise Spike Times with Homeostatic Membrane Potential Dependent Synaptic Plasticity
Albers, C., Westkott, M., & Pawelzik, K. R. (2016) · PLoS ONE · doi:10.1371/journal.pone.0148948 · Cited by 21
Banerjee, A. (2016) · Neural Computation · doi:10.1162/neco_a_00829 · Cited by 10
Cited by
Neftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538
Self-citationDeep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
Brian 2, an intuitive and efficient neural simulator
Stimberg, M., Brette, R., & Goodman, D. F. (2019) · eLife · doi:10.7554/elife.47314 · Cited by 749
Spiking Neural Networks and Their Applications: A Review
Yamazaki, K., Vo-Ho, V.-K., Bulsara, D., & Le, N. (2022) · Brain Sciences · doi:10.3390/brainsci12070863 · Cited by 626
Metadata
- Authors
- Zenke, F., & Ganguli, S.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2018-06
- Volume
- 30
- Issue
- 6
- Pages
- 1514-1541
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2620474507
- Cited by
- 515
- References
- 55
- FWCI
- 13.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.