An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms for SNNs, however, lags behind hardware implementations: most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Inspired by the dynamic spiking neurons in the human brain, the energy consumption and latency of edge AI can be significantly reduced by processing the learning tasks in an event-driven manner [62], [63].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [62]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 2×
“In terms of algorithms, the doubly stochastic SGD approach reviewed here for ML training can be extended and improved by leveraging an alternative estimator of the ELBO and its gradients with respect to the variational parameters that is known as the reparameterization trick [28].”
MethodLoihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,897 · cited 11×
“The design of training algorithms for SNNs, however, lags behind hardware implementations: most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding.”
BackgroundInfluentialNetworks of spiking neurons: The third generation of neural network models
Maass, W. (1997) · Neural Networks · doi:10.1016/s0893-6080(97)00011-7 · Cited by 3,481 · cited 1×
“Inspired by this observation, as illustrated in Figure 1, SNNs have been introduced in the theoretical neuroscience literature as networks of dynamic spiking neurons [3].”
MethodSpiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
Bienenstock, E., Cooper, L., & Munro, P. (1982) · The Journal of Neuroscience · doi:10.1523/jneurosci.02-01-00032.1982 · Cited by 2,896 · cited 1×
“This update can be related to the standard spike-timingdependent plasticity (STDP) rule [10], [16], [25].”
BackgroundA database for handwritten text recognition research
Hull, J. J. (1994) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.291440 · Cited by 2,244 · cited 1×
“Batch learning examples As an example of batch learning, we consider the standard handwritten digit classification task on the USPS data set [35].”
MethodNeftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538 · cited 1×
“When including loops, deterministic SNNs can also implement recurrent neural networks (RNNs) [20].”
BackgroundSpatio-temporal correlations and visual signalling in a complete neuronal population
Pillow, J. W., Shlens, J., Paninski, L., Sher, A., Litke, A. M., Chichilnisky, E. J., & Simoncelli, E. P. (2008) · Nature · doi:10.1038/nature07140 · Cited by 1,311 · cited 2×
“Examples of feedforward/feedback kernels: (a) an exponentially decaying feedforward kernel , at (b) an exponentially decaying feedback kernel , bt (c) raised cosine basis functions a , k t in [14], and (d) spike-timing-dependent plasticity basis functions a , k t for long-term potentiation (LTP) and longterm depressio…”
BackgroundSpatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks
Wu, Y., Deng, L., Li, G., Zhu, J., & Shi, L. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00331 · Cited by 1,196 · cited 1×
“To obviate this problem, a number of existing learning rules approximate the derivative by smoothing out the membrane potential as a function of the weights [7]–[9].”
BackgroundTraining Deep Spiking Neural Networks Using Backpropagation
Lee, J. H., Delbruck, T., & Pfeiffer, M. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00508 · Cited by 976 · cited 1×
“To obviate this problem, a number of existing learning rules approximate the derivative by smoothing out the membrane potential as a function of the weights [7]–[9].”
BackgroundLong short-term memory and learning-to-learn in networks of spiking neuronsPreprint
Bellec, G., Salaj, D., Subramoney, A., Legenstein, R. A., & Maass, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09574 · Cited by 683 · cited 2×
“, [41]); the design of distributed learning techniques; and the definition of clear use cases and applications with the quantification of advantages in terms of power efficiency [42].”
BackgroundConnectionist learning of belief networks
Neal, R. M. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90065-6 · Cited by 513 · cited 1×
“i SNNs modeled according to the described GLM framework can be thought of as a generalization of dynamic models of belief networks [17], and they can also be interpreted as a discrete-time version of Hawkes processes [18].”
MethodNeuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules
Frémaux, N., & Gerstner, W. (2016) · Frontiers in Neural Circuits · doi:10.3389/fncir.2015.00085 · Cited by 413 · cited 1×
“Three-factor rule While the details differ for each learning rule and task, a general form of the learning rule for the synaptic weights follows the three-factor rule [21], [22].”
BackgroundBuesing, L., Bill, J., Nessler, B., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002211 · Cited by 384
Computing with Spiking Neuron Networks
Paugam-Moisy, H., & Bohte, S. (2012) · Handbook of Natural Computing · doi:10.1007/978-3-540-92910-9_10 · Cited by 272 · cited 1×
“The human brain is capable of performing general and complex tasks via continuous adaptation at a minute fraction of the power required by state-of-the-art supercomputers and ANN-based models [2].”
BackgroundUnsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks
Bohte, S. M., La Poutre, H., & Kok, J. N. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/72.991428 · Cited by 239 · cited 3×
“For time coding, each of the NX visible neurons is associated with a different shifted, truncated Gaussian receptive field [37].”
BackgroundMethodNoise as a Resource for Computation and Learning in Networks of Spiking Neurons
Maass, W. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2310593 · Cited by 198 · cited 1×
“Furthermore, while the models studied in this article encode information in the temporal behavior of the network within a given interval of time, information can also be retrieved from the asymptotic steady-state spiking rates, which define a joint probability distribution [4], [40], [41].”
BackgroundBenchmarking Keyword Spotting Efficiency on Neuromorphic Hardware
Blouw, P., Choo, X., Hunsberger, E., & Eliasmith, C. (2019) · Proceedings of the 7th Annual Neuro-inspired Computational Elements Workshop · doi:10.1145/3320288.3320304 · Cited by 192 · cited 1×
“, [41]); the design of distributed learning techniques; and the definition of clear use cases and applications with the quantification of advantages in terms of power efficiency [42].”
BackgroundHomeostatic plasticity and STDP: keeping a neuron's cool in a fluctuating world
Watt, A. J. (2010) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2010.00005 · Cited by 186
Conversion of analog to spiking neural networks using sparse temporal coding
Rueckauer, B., & Liu, S.-C. (2018) · 2018 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2018.8351295 · Cited by 177 · cited 2×
“This has motivated a popular line of work that aims at converting a pretrained ANN into a potentially more efficient SNN implementation (see [6] and the “Models” section for further details).”
BackgroundRajendran, B., Sebastian, A., Schmuker, M., Srinivasa, N., & Eleftheriou, E. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2933719 · Cited by 177 · cited 3×
“Given the extremely low idle energy consumption, the energy spent by SNNs for learning and inference is essentially proportional to the number of spikes processed and communicated by the neurons, with the energy per spike being as low as a few picojoules [5].”
BackgroundNeural Variational Inference and Learning in Belief Networks
Mnih, A., & Gregor, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0030 · Cited by 176 · cited 1×
“This can be done by replacing the learning signal in (14b) with the centered learning signal ( , ) , x h , T T , , # # i z r where the baseline ,r is calculated as a moving average of learning signals computed at previous iterations [22], [27], [29].”
To spike or not to spike: A probabilistic spiking neuron model
Kasabov, N. K. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.08.010 · Cited by 174 · cited 1×
“In terms of models, while randomness is defined here at the level of neurons’ outputs, alternative models introduce randomness at the level of synapses or thresholds [38], [39].”
BackgroundA Brief Introduction to Machine Learning for Engineers
Simeone, O. (2018) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000102 · Cited by 154
Self-citationSurrogate Gradient Learning in Spiking Neural NetworksPreprint
Neftci, E., Mostafa, H., & Zenke, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09948 · Cited by 144 · cited 1×
“When including loops, deterministic SNNs can also implement recurrent neural networks (RNNs) [20].”
BackgroundNetwork Plasticity as Bayesian Inference
Kappel, D., Habenschuss, S., Legenstein, R., & Maass, W. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004485 · Cited by 126 · cited 2×
“Furthermore, similar techniques can be developed to tackle other training criteria, such as Bayesian optimal inference [31], reward maximization in reinforcement learning [23], and mutual information maximization for representation learning (see [12] for a discussion in the context of general probabilistic models).”
BackgroundMethodMatching Recall and Storage in Sequence Learning with Spiking Neural Networks
Brea, J., Senn, W., & Pfister, J.-P. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4098-12.2013 · Cited by 124 · cited 4×
“Three-factor rule While the details differ for each learning rule and task, a general form of the learning rule for the synaptic weights follows the three-factor rule [21], [22].”
BackgroundInfluentialDeep Spiking NetworksPreprint
O’Connor, P., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.08323 · Cited by 93
Gerhard, F., Deger, M., & Truccolo, W. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005390 · Cited by 83 · cited 2×
“i SNNs modeled according to the described GLM framework can be thought of as a generalization of dynamic models of belief networks [17], and they can also be interpreted as a discrete-time version of Hawkes processes [18].”
MethodStochastic variational learning in recurrent spiking networks
Jimenez Rezende, D., & Gerstner, W. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00038 · Cited by 81 · cited 2×
“This can be done by replacing the learning signal in (14b) with the centered learning signal ( , ) , x h , T T , , # # i z r where the baseline ,r is calculated as a moving average of learning signals computed at previous iterations [22], [27], [29].”
BackgroundLearning Spatiotemporally Encoded Pattern Transformations in Structured Spiking Neural Networks
Brian, G., Sporea, I., & Grüning, A. (2015) · Neural Computation · doi:10.1162/neco_a_00790 · Cited by 50 · cited 2×
“When taking the limit , u 0 " D we obtain the deterministic integrateand-fire model [19].”
BackgroundMethodHinton, G. E., & Brown, A. D. (1999) · neural information processing systems · Cited by 42
A Reward-Maximizing Spiking Neuron as a Bounded Rational Decision Maker
Leibfried, F., & Braun, D. A. (2015) · Neural Computation · doi:10.1162/neco_a_00758 · Cited by 16 · cited 1×
“This regularizing term, which penalizes variational distributions far from the baseline distribution, can also act as a regularizer to minimize overfitting by enforcing a bounded rationality constraint [32].”
BackgroundLearning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients
Rosenfeld, B., Simeone, O., & Rajendran, B. (2019) · IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2019.8815546 · Cited by 16 · cited 2×
“Furthermore, similar techniques can be developed to tackle other training criteria, such as Bayesian optimal inference [31], reward maximization in reinforcement learning [23], and mutual information maximization for representation learning (see [12] for a discussion in the context of general probabilistic models).”
BackgroundMethodSelf-citationA Learning Framework for Winner-Take-All Networks with Stochastic Synapses
Mostafa, H., & Cauwenberghs, G. (2018) · Neural Computation · doi:10.1162/neco_a_01080 · Cited by 14 · cited 1×
“In terms of models, while randomness is defined here at the level of neurons’ outputs, alternative models introduce randomness at the level of synapses or thresholds [38], [39].”
BackgroundBoltzmann machines for time-seriesPreprint
Osogami, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.06004 · Cited by 9 · cited 6×
“i V ! When the filter is of finite duration , x computing the feedforward trace s , i t v requires keeping track of the window { , , , } s s s , , , ( ) i t i t i t 1 1 f x - - of prior synaptic inputs as part of the neuron’s state [16].”
BackgroundMethodInfluentialLow-Power Neuromorphic Hardware for Signal Processing ApplicationsPreprint
Rajendran, B., Sebastian, A., Schmuker, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.03690 · Cited by 7 · cited 3×
“Given the extremely low idle energy consumption, the energy spent by SNNs for learning and inference is essentially proportional to the number of spikes processed and communicated by the neurons, with the energy per spike being as low as a few picojoules [5].”
BackgroundJang, H., & Simeone, O. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682960 · Cited by 7
Self-citation
Research works printed but not linked: 7
- dau, 2018, The UCR time series classification archive
- jang, 2019, An introduction to spiking neural networks Probabilistic models learning rules and applications [supplementary material]
- dayan, 2001, Theoretical Neuroscience Computational and Mathematical Modeling of Neural Systems
- eliasmith, 2004, Neural Engineering Computation Representation and Dynamics in Neurobiological Systems
- sutton, 2018, Reinforcement Learning An Introduction
- welling, 2018, Intelligence per kilowatt-hour
- goodfellow, 2016, Deep Learning
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Inspired by the dynamic spiking neurons in the human brain, the energy consumption and latency of edge AI can be significantly reduced by processing the learning tasks in an event-5 driven manner [62], [63].”
MethodAdvancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook
Davies, M., Wild, A., Orchard, G., Sandamirskaya, Y., Guerra, G. A. F., Joshi, P., Plank, P., & Risbud, S. R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3067593 · Cited by 688
“Applying methods from probabilistic signal processing, stochastic SNNs can be designed to tackle a variety of inference and learning problems that optimally leverage their time-coding capabilities [99].”
MethodMehonic, A., Sebastian, A., Rajendran, B., Simeone, O., Vasilaki, E., & Kenyon, A. J. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000085 · Cited by 331
Self-citationThe Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks
Cramer, B., Stradmann, Y., Schemmel, J., & Zenke, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3044364 · Cited by 288
Green Edge AI: A Contemporary Survey
Mao, Y., Yu, X., Huang, K., Zhang, Y. A., & Zhang, J. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3437365 · Cited by 184
“They differ from the conventional DNNs by encoding input as time series and using discrete spikes to transfer information between spiking neurons, each of which can fire a spike with sufficient input strength called membrane potential [298].”
Background
Metadata
- Authors
- Jang, H., Simeone, O., Gardner, B., & Gruning, A.
- Affiliations
- King's College London (GB)
- University of Surrey (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2019-11
- Volume
- 36
- Issue
- 6
- Pages
- 64-77
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W2985873131
- Cited by
- 101
- References
- 35
- Citations by year
- 2026: 8 · 2025: 12 · 2024: 21 · 2023: 19 · 2022: 16 · 2021: 18 · 2020: 7
- FWCI
- 3.68
- Type
- journal article
- Language
- en
- Open access
- green
- kclpure.kcl.ac.uk
- Licence
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Research Portal (King's College London) (publishedVersion, other-oa)
- arXiv (Cornell University) (acceptedVersion)
- arXiv (Cornell University)
- Funders
- European Union Horizon 2020 (725731)
- U.S. National Science Foundation (1710009)
- European Union Horizon 2020 Framework Programme (785907 (Human Brain Project SGA2))
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- spiking neural networks, probabilistic models, learning rules, energy-efficient hardware, event-driven computing, training algorithms, biological plausibility, neuromorphic computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.