The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines
Open accessAbstract
An ongoing challenge in neuromorphic computing is to devise general and computationally efficient models of inference and learning which are compatible with the spatial and temporal constraints of the brain. One increasingly popular and successful approach is to take inspiration from inference and learning algorithms used in deep neural networks. However, the workhorse of deep learning, the gradient descent Gradient Back Propagation (BP) rule, often relies on the immediate availability of network-wide information stored with high-precision memory during learning, and precise operations that are difficult to realize in neuromorphic hardware. Remarkably, recent work showed that exact backpropagated gradients are not essential for learning deep representations. Building on these results, we demonstrate an event-driven random BP (eRBP) rule that uses an error-modulated synaptic plasticity for learning deep representations. Using a two-compartment Leaky Integrate & Fire (I&F) neuron, the rule requires only one addition and two comparisons for each synaptic weight, making it very suitable for implementation in digital or mixed-signal neuromorphic hardware. Our results show that using eRBP, deep representations are rapidly learned, achieving classification accuracies on permutation invariant datasets comparable to those obtained in artificial neural network simulations on GPUs, while being robust to neural and synaptic state quantizations during learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
McClelland, J. L., Rumelhart, D. E., & AU (1987) · doi:10.7551/mitpress/5237.001.0001 · Cited by 3,315
Neuromorphic electronic systems
Mead, C. (1990) · Proceedings of the IEEE · doi:10.1109/5.58356 · Cited by 2,368 · cited 2×
“…this is true for some cases and on metrics based on absolute accuracy at some standardized benchmark task, the premise of neuromorphic engineering, i.e. that electronic and biological share similar constraints on communication, power and reliability [Mead, 1990], extend to the algorithmic domain.”
BackgroundCompetitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 2×
“The implementation of gradient BP on a neural substrate is even more challenging [Baldi et al., 2016; Grossberg, 1987; Lee et al., 2016] because it requires 1) using synaptic weights that are identical with 1 a r X i v : 1612 .”
BackgroundThe dynamic brain: from spiking neurons to neural masses and cortical fields
Deco, G., Jirsa, V. K., Robinson, P. A., Breakspear, M., & Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000092 · Cited by 1,084 · cited 2×
“Activation Function of Spiking Neurons with Background Poisson Noise and Its Derivative The dynamics of spiking neural circuits driven by Poisson spike trains is often studied in the diffusion approximation (Wang, 1999; Brunel and Hakim, 1999; Fusi and Mattia, 1999; Brunel, 2000; Renart et al., 2003; Tuckwell, 2005; D…”
BackgroundGraves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 2×
“…such as normalization techniques for improving deep learning [Ioffe and Szegedy, 2015; Ren et al., 2016], attention, short-term memory for learning complex tasks [Graves et al., 2014], and memory consolidation through fast replays for reinforcement learning [Kumaran et al., 2016; Mnih et al., 2015].”
BackgroundMethodSynaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory
Wang, X.-J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-21-09587.1999 · Cited by 859 · cited 2×
“Activation Function of Spiking Neurons with Background Poisson Noise and Its Derivative The dynamics of spiking neural circuits driven by Poisson spike trains is often studied in the diffusion approximation (Wang, 1999; Brunel and Hakim, 1999; Fusi and Mattia, 1999; Brunel, 2000; Renart et al., 2003; Tuckwell, 2005; D…”
BackgroundWhat Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 759
Nessler, B., Pfeiffer, M., Buesing, L., & Maass, W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003037 · Cited by 281 · cited 2×
“STDP has been shown to be very powerful in a number of different models and tasks [Neftci et al., 2014a; Nessler et al., 2013; Thorpe et al., 2001].”
BackgroundLearning by the Dendritic Prediction of Somatic Spiking
Urbanczik, R., & Senn, W. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.030 · Cited by 230 · cited 6×
“The two compartment neuron model used in this work is motivated from conductance-based dynamics in [Urbanczik and Senn, 2014] and previous neuromorphic realizations of two compartment mixed signal spiking neurons [Park et al., 2014].”
BackgroundMethodInfluential
Cited by
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,897
Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3008413 · Cited by 2,566
Towards spike-based machine intelligence with neuromorphic computing
Roy, K., Jaiswal, A., & Panda, P. (2019) · Nature · doi:10.1038/s41586-019-1677-2 · Cited by 2,190
Neftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538
Self-citationDeep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Metadata
- Authors
- Neftci, E. O., Augustine, C., Paul, S., & Detorakis, G.
- Venue
- Frontiers in Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2017-06-21
- Volume
- 11
- Pages
- 324-324
- ISSN
- 1662-453X (ISSN-L)
- OpenAlex
- W2565565355
- Cited by
- 237
- References
- 14
- FWCI
- 12.53
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.