The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
Open accessAbstract
A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, addresses the similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks. The challenge preventing this is largely caused by the discrepancy between the dynamical properties of synaptic plasticity and the requirements for gradient backpropagation. Learning algorithms that approximate gradient backpropagation using local error functions can overcome this challenge. Here, we introduce Deep Continuous Local Learning (DECOLLE), a spiking neural network equipped with local error functions for online learning with no memory overhead for computing gradients. DECOLLE is capable of learning deep spatio temporal representations from spikes relying solely on local information, making it compatible with neurobiology and neuromorphic hardware. Synaptic plasticity rules are derived systematically from user-defined cost functions and neural dynamics by leveraging existing autodifferentiation methods of machine learning frameworks. We benchmark our approach on the event-based neuromorphic dataset N-MNIST and DvsGesture, on which DECOLLE performs comparably to the state-of-the-art. DECOLLE networks provide continuously learning machines that are relevant to biology and supportive of event-based, low-power computer vision architectures matching the accuracies of conventional computers on tasks where temporal precision and speed are essential.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
Proceedings of IEEE International Conference on Computer Vision
(1995) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466933 · Cited by 12,860
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 3×
“Existing rules of this sort require dedicated state variables for every synapse, thus scaling at least quadratically with the number of neurons [26, 27].”
BackgroundMethodA million spiking-neuron integrated circuit with a scalable communication network and interface
Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Guo, C., Nakamura, Y., et al. (2014) · Science · doi:10.1126/science.1254642 · Cited by 4,147
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 2×
“This is achieved using a local rate-based cost function reminiscent of readout mechanisms used in liquid state machines [15], but where the readout is performed over a fixed random combination of the neuron outputs.”
MethodLoihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,897
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor
Lichtsteiner, P., Posch, C., & Delbruck, T. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2007.914337 · Cited by 2,485
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106
Automatic Differentiation in Machine Learning: a Survey
Baydin, A. G., Pearlmutter, B. A., Radul, A., & Siskind, J. M. (2015) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 2,029
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Wulfram Gerstner, B., & Kistler, W. M. (2002) · Cited by 2,028
Neftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538
Self-citationNeuronal Dynamics: From Single Neurons to Networks and Models of Cognition
Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014) · doi:10.1017/cbo9781107447615 · Cited by 1,515
Generating Coherent Patterns of Activity from Chaotic Neural Networks
Sussillo, D., & Abbott, L. F. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.018 · Cited by 1,092
A Low Power, Fully Event-Based Gesture Recognition System
Amir, A., Taba, B., Berg, D., Melano, T., McKinstry, J., Di Nolfo, C., Nayak, T., Andreopoulos, A., Garreau, G., Mendoza, M., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.781 · Cited by 1,017
Training Deep Spiking Neural Networks Using Backpropagation
Lee, J. H., Delbruck, T., & Pfeiffer, M. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00508 · Cited by 976
Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades
Orchard, G., Jayawant, A., Cohen, G. K., & Thakor, N. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00437 · Cited by 882
Simulation of networks of spiking neurons: A review of tools and strategies
Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J. M., Diesmann, M., Morrison, A., Goodman, P. H., Harris, F. C., et al. (2007) · Journal of Computational Neuroscience · doi:10.1007/s10827-007-0038-6 · Cited by 822
The tempotron: a neuron that learns spike timing–based decisions
Gütig, R., & Sompolinsky, H. (2006) · Nature Neuroscience · doi:10.1038/nn1643 · Cited by 815
Random synaptic feedback weights support error backpropagation for deep learning
Lillicrap, T. P., Cownden, D., Tweed, D. B., & Akerman, C. J. (2016) · Nature Communications · doi:10.1038/ncomms13276 · Cited by 812
Convolutional networks for fast, energy-efficient neuromorphic computing
Esser, S. K., Merolla, P. A., Arthur, J. V., Cassidy, A. S., Appuswamy, R., Andreopoulos, A., Berg, D. J., McKinstry, J. L., Melano, T., Barch, D. R., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1604850113 · Cited by 781
Connectivity reflects coding: a model of voltage-based STDP with homeostasis
Clopath, C., Büsing, L., Vasilaki, E., & Gerstner, W. (2010) · Nature Neuroscience · doi:10.1038/nn.2479 · Cited by 607
Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems
Chicca, E., Stefanini, F., Bartolozzi, C., & Indiveri, G. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2313954 · Cited by 552
Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems
(2004) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.826381 · Cited by 543
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
Zenke, F., & Ganguli, S. (2018) · Neural Computation · doi:10.1162/neco_a_01086 · Cited by 515
SLAYER: Spike Layer Error Reassignment in Time
Shrestha, S. B., & Orchard, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08646 · Cited by 444
Gradient-based learning algorithms for recurrent networks and their computational complexity
Williams, R. J., & Zipser, D. (1995) · Cited by 424
Decoupled Neural Interfaces using Synthetic GradientsPreprint
Jaderberg, M., Czarnecki, W. M., Osindero, S., Vinyals, O., Graves, A., Silver, D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.05343 · Cited by 410
Neural engineering: Computation, representation, and dynamics in neurobiological systems
(2003) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(03)90181-7 · Cited by 399
Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics
Brader, J. M., Senn, W., & Fusi, S. (2007) · Neural Computation · doi:10.1162/neco.2007.19.11.2881 · Cited by 371
Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and ArchitecturesPreprint
Bartunov, S., Santoro, A., Richards, B. A., Marris, L., Hinton, G. E., & Lillicrap, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04587 · Cited by 310
Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning
Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.6.1318 · Cited by 245
Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines
Neftci, E. O., Augustine, C., Paul, S., & Detorakis, G. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00324 · Cited by 237
Self-citationLearning by the Dendritic Prediction of Somatic Spiking
Urbanczik, R., & Senn, W. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.030 · Cited by 230 · cited 1×
“These types of rules are often termed three-factor rules, which have been shown to be consistent with biology [19] while being compatible with a wide number of unsupervised, supervised and reinforcement learning paradigms [25].”
BackgroundGradient Descent for Spiking Neural NetworksPreprint
Huh, D., & B, T. J. S. A. (2017) · arXiv (Cornell University) · doi:10.12751/nncn.bc2017.0088 · Cited by 212
SpikeProp: backpropagation for networks of spiking neurons.
Bohté, S. M., Kok, J. N., & Poutré, J. A. L. (2000) · The European Symposium on Artificial Neural Networks · Cited by 161
Deep Supervised Learning Using Local Errors
Mostafa, H., Ramesh, V., & Cauwenberghs, G. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00608 · Cited by 147
Self-citationSurrogate Gradient Learning in Spiking Neural NetworksPreprint
Neftci, E., Mostafa, H., & Zenke, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09948 · Cited by 144
Self-citationEvent-driven random backpropagation: Enabling neuromorphic deep learning machines
Neftci, E. O., Augustine, C., Paul, S., & Detorakis, G. (2017) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2017.8050529 · Cited by 118
Self-citationBellec, G., Scherr, F., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2019) · TUGraz OPEN Library (Graz University of Technology) · doi:10.12751/nncn.bc2019.0013 · Cited by 115
Random feedback weights support learning in deep neural networksPreprint
Lillicrap, T., Cownden, D., Tweed, D., & Akerman, C. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.0247 · Cited by 89
Iyer, L., Chua, Y., & Li, H. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.608567 · Cited by 89
A Systematic Method for Configuring VLSI Networks of Spiking Neurons
Neftci, E., Chicca, E., Indiveri, G., & Douglas, R. J. (2011) · Neural Computation · doi:10.1162/neco_a_00182 · Cited by 55
Self-citationLearning in the machine: The symmetries of the deep learning channel
Baldi, P., Sadowski, P., & Lu, Z. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.08.008 · Cited by 37
Huayaney, F. L. M., Nease, S., & Chicca, E. (2016) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2016.2616169 · Cited by 35
Error-triggered Three-Factor Learning Dynamics for Crossbar Arrays
Payvand, M., Fouda, M. E., Kurdahi, F. J., Eltawil, A. M., & Neftci, E. (2020) · IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas48895.2020.9073998 · Cited by 33
Self-citationSMPLR: Deep SMPL reverse for 3D human pose and shape recoveryPreprint
Madadi, M., Bertiche, H., & Escalera, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.10766 · Cited by 20
Bartolozzi, C., & Indiveri, G. (2006) · Brain Inspired Cognitive Systems · Cited by 13
Cited by
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
Training Spiking Neural Networks Using Lessons From Deep Learning
Eshraghian, J. K., Ward, M., Neftci, E. O., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D. S., & Lu, W. D. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3308088 · Cited by 782
Self-citationEmbedded Devices for Neuromorphic Time-Series Assessment
Mazumder, A. N., Hosseini, M., & Mohsenin, T. (2022) · Research Explorer (The University of Manchester) · doi:10.13016/m2ksy0-wxr3 · Cited by 652
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., & Tian, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00266 · Cited by 605
A solution to the learning dilemma for recurrent networks of spiking neurons
Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020) · Nature Communications · doi:10.1038/s41467-020-17236-y · Cited by 506
Metadata
- Authors
- Kaiser, J., Mostafa, H., & Neftci, E.
- Venue
- Frontiers in Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2020-05-12
- Volume
- 14
- Pages
- 424-424
- ISSN
- 1662-453X (ISSN-L)
- OpenAlex
- W3025773901
- Cited by
- 288
- References
- 54
- FWCI
- 14.43
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.