The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
Open accessAbstract
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in silico. Here we revisit the problem of supervised learning in temporally coding multilayer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric, and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike time patterns.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 2×
“Technically, this computation requires either backpropagation through time through the PSP kernel or the computation of all relevant quantities online as in the case of RTRL.”
BackgroundMaass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 1×
“For instance, Maass et al. (2002) have used the p-delta rule (Auer et al., 2008) to train the readout layer of a liquid state machine.”
MethodSpeed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309 · cited 1×
“5 was motivated by previous work on rapid compensatory processes as an effective way to constrain neuronal activity through linear scaling of afferent weights (Oja, 1982; Zenke et al., 2015).”
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition
Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014) · doi:10.1017/cbo9781107447615 · Cited by 1,515 · cited 1×
“To further compute the derivative ∂Ui/∂wij in the expression above, we exploit the fact that for current-based LIF models the membrane potential Ui(t) can be written in integral form as a spike response model (SRM0 (Gerstner et al., 2014)):”
MethodCompetitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 2×
“…efficient auto-differentiation tools are not available for most event-based spiking neural network frameworks, and (ii) the method of weight updates implemented by the standard back-propagation of error algorithm (Backprop) is thought to be biologically implausible (Grossberg, 1987; Crick, 1989).”
Method
Cited by
Neftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538
Self-citationDeep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
“To leverage the temporal dimension of SNN, some learning algorithms aim to generate single or multiple spikes at desired times, which have been applied in classification tasks 79–83 .”
MethodBrian 2, an intuitive and efficient neural simulator
Stimberg, M., Brette, R., & Goodman, D. F. (2019) · eLife · doi:10.7554/elife.47314 · Cited by 749
Spiking Neural Networks and Their Applications: A Review
Yamazaki, K., Vo-Ho, V.-K., Bulsara, D., & Le, N. (2022) · Brain Sciences · doi:10.3390/brainsci12070863 · Cited by 626
A solution to the learning dilemma for recurrent networks of spiking neurons
Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020) · Nature Communications · doi:10.1038/s41467-020-17236-y · Cited by 506
A review of learning in biologically plausible spiking neural networks
Taherkhani, A., Belatreche, A., Li, Y., Cosma, G., Maguire, L. P., & McGinnity, T. M. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.09.036 · Cited by 422
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420
“…(e.g., handwritten digit classi fi cation) [12 – 14, The criticism of weight symmetry has been addressed by demonstrating that even if the errors in arti fi cial neural networks are back-propagated by random connections, good performance in classi fi cation tasks can still be achieved [21,23 – 27].”
BackgroundArtificial Neural Networks for Neuroscientists: A Primer
Yang, G. R., & Wang, X.-J. (2020) · Neuron · doi:10.1016/j.neuron.2020.09.005 · Cited by 376
Neuromorphic Engineering: From Biological to Spike‐Based Hardware Nervous Systems
Yang, J., Wang, R., Ren, Y., Mao, J., Wang, Z., Zhou, Y., & Han, S. (2020) · Advanced Materials · doi:10.1002/adma.202003610 · Cited by 350
Mehonic, A., Sebastian, A., Rajendran, B., Simeone, O., Vasilaki, E., & Kenyon, A. J. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000085 · Cited by 331
The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks
Cramer, B., Stradmann, Y., Schemmel, J., & Zenke, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3044364 · Cited by 288
Self-citationSynaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
Kaiser, J., Mostafa, H., & Neftci, E. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00424 · Cited by 288
Supervised learning in spiking neural networks: A review of algorithms and evaluations
Wang, X., Lin, X., & Dang, X. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.02.011 · Cited by 277
Spiking Neural Networks Hardware Implementations and Challenges: A Survey
Bouvier, M., Valentian, A., Mesquida, T., Rummens, F., Reyboz, M., Vianello, E., & Beigne, E. (2019) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3304103 · Cited by 267
Exploring Neuromorphic Computing Based on Spiking Neural Networks: Algorithms to Hardware
Rathi, N., Chakraborty, I., Kosta, A., Sengupta, A., Ankit, A., Panda, P., & Roy, K. (2023) · ACM Computing Surveys · doi:10.1145/3571155 · Cited by 267
Zenke, F., & Vogels, T. P. (2021) · Neural Computation · doi:10.1162/neco_a_01367 · Cited by 261
Self-citationAccurate and efficient time-domain classification with adaptive spiking recurrent neural networks
Yin, B., Corradi, F., & Bohté, S. M. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00397-w · Cited by 259
The neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233
New light on cortical neuropeptides and synaptic network plasticity
Smith, S. J., Hawrylycz, M., Rossier, J., & Sümbül, U. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.04.002 · Cited by 26
Metadata
- Authors
- Zenke, F., & Ganguli, S.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2018-06
- Volume
- 30
- Issue
- 6
- Pages
- 1514-1541
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2620474507
- Cited by
- 515
- References
- 55
- FWCI
- 13.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.