Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity
Open accessThe DOI matches its Crossref record; 121 of the 121 references it cites were found in a registry or an index.
Abstract
The principles by which networks of neurons compute, and how spike-timing dependent plasticity (STDP) of synaptic weights generates and maintains their computational function, are unknown. Preceding work has shown that soft winner-take-all (WTA) circuits, where pyramidal neurons inhibit each other via interneurons, are a common motif of cortical microcircuits. We show through theoretical analysis and computer simulations that Bayesian computation is induced in these network motifs through STDP in combination with activity-dependent changes in the excitability of neurons. The fundamental components of this emergent Bayesian computation are priors that result from adaptation of neuronal excitability and implicit generative models for hidden causes that are created in the synaptic weights through STDP. In fact, a surprising result is that STDP is able to approximate a powerful principle for fitting such implicit generative models to high-dimensional spike inputs: Expectation Maximization. Our results suggest that the experimentally observed spontaneous activity and trial-to-trial variability of cortical neurons are essential features of their information processing capability, since their functional role is to represent probability distributions rather than static neural codes. Furthermore it suggests networks of Bayesian computation modules as a new model for distributed information processing in the cortex.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



![Figure 4 Relationship between the continuous-time SEM model and experimental data on synaptic plasticity. A–C: The effect of the continuous-time plasticity rule in Eq. (18) at a single synapse for different stimulation frequencies and different time-differences between pre- and post-synaptic spike pairs. Only time-intervals without overlapping pairs are shown. A: For very low stimulation frequencies (1 Hz) the standard shape of the complex learning rule from Fig. 2 is recovered. B: At a stimulation frequency of 20 Hz the plasticity curve shifts more towards LTP, and depression is no longer time independent, due to overlapping EPSPs. C: At high stimulation frequencies of 40 Hz or above, the STDP curve shifts towards only LTP, and thus becomes similar to a rate-based Hebbian learning rule. D: Cumulative effect of pre- and post-synaptic burst stimulation (50 Hz bursts of 5 pre-synaptic and 4 post-synaptic spikes) with different onset delays of -120, -60, 10, 20, 30, 80 and 140 ms (time difference between the onsets of the post- and pre-synaptic bursts). As in [55], the amount of overlap between bursts determines the magnitude of LTP, rather than the exact temporal order of spikes.](https://figures.cochat.ai/paper-figures/c8793663018c4fae8d611dbd2370c7d4/bayesian-computation-emerges-in-generic-cortical-microcircuits-through-spike-figure-4.jpg)





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The EM algorithm that is used for learning is a local update mechanism analogous to spike timing–dependent plasticity (24, 25).”
From Space is a latent sequence: A theory of the hippocampus (2024) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“6 through an application to a computational task that is substantially more difficult than previously considered tasks for networks of spiking neurons: We show that a simple network of spiking neurons can learn without any supervision to discriminate handwritten digits from the MNIST benchmark dataset [64] consisting…”
BackgroundMaximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · cited 4×
“The fact that STDP approximates EM is remarkable, since it is known from machine learning that EM can solve a fundamental chicken-and-egg problem of unsupervised learning systems [31]: To detect - without a teacher - hidden causes for complex input data, and to induce separate learning agents to specialize each on one…”
BackgroundMethodInfluentialBook Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 2×
“or semi-supervised machine learning algorithms [27,65], or to evaluate unsupervised feature learning approaches [66,67].”
BackgroundA model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271 · cited 1×
“These ubiquitous network motifs were called soft winner-take-all (WTA) circuits, and have been suggested as neural network models for implementing functions like non-linear selection [16,17], normalization [18], selective attention [19], decision making [20,21], or as primitives for general purpose computation [22,23].”
BackgroundHebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · cited 2×
“Our model shows that a WTA circuit could learn orientation selectivity from such simple bar-like inputs, but does not provide an alternative explanation to the results of studies like [61] using natural image stimuli.”
BackgroundResultBi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744 · cited 2×
“standard models of STDP and observed biological data [43,54].”
Result
Research works printed but not linked: 2
- Bishop CM (2006) Pattern Recognition and Machine Learning. New York: Springer.
- G Celeux, 1985, The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem, Comput Statist Quater, 2, 73
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Unsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
“The second STDP rule uses an exponential weight dependence (Nessler et al., 2013; Querlioz et al., 2013) to compute the weight change where β determines the strength of the weight dependence.”
MethodDeep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
“(2013) [71] extended their network by using an”
InfluentialNeuromorphic computing using non-volatile memory
Burr, G. W., Shelby, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K., Ishii, M., Narayanan, P., Fumarola, A., et al. (2017) · Advances in Physics: X · doi:10.1080/23746149.2016.1259585 · Cited by 1,107
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
Memory and Information Processing in Neuromorphic Systems
Indiveri, G., & Liu, S.-C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2444094 · Cited by 1,037
“, the neuron’s membrane potential) [96], [97], and/or the current value of the synaptic weights [98], [99].”
BackgroundDeep Learning With Spiking Neurons: Opportunities and Challenges
Pfeiffer, M., & Pfeil, T. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00774 · Cited by 766
“In competitive networks, STDP is capable of solving unsupervised learning tasks such as clustering (Masquelier et al., 2009; Nessler et al., 2013).”
BackgroundSelf-citationIncorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., & Tian, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00266 · Cited by 605
“Existing approaches exploited the self-organization principle [56, 11, 29], and STDP-based expectation-maximization algorithm [43, 17].”
MethodFlexible Neuromorphic Electronics for Computing, Soft Robotics, and Neuroprosthetics
Park, H., Lee, Y., Kim, N., Seo, D., Go, G., & Lee, T. (2020) · Advanced Materials · doi:10.1002/adma.201903558 · Cited by 533
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules
Frémaux, N., & Gerstner, W. (2016) · Frontiers in Neural Circuits · doi:10.3389/fncir.2015.00085 · Cited by 413
“For example, for the learning of binary decisions in populations of spiking neurons, a neuromodulatory signal proportional to the population activity has been suggested (Urbanczik and Senn, 2009; Friedrich et al., 2011).”
BackgroundReal-time classification and sensor fusion with a spiking deep belief network
O'Connor, P., Neil, D., Liu, S.-C., Delbruck, T., & Pfeiffer, M. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00178 · Cited by 408
“Nessler et al. (2013) have shown that learning via STDP in cortical microcircuits can lead to the emergence of Bayesian computation for the detection of hidden causes of inputs.”
BackgroundSelf-citationPhased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Neil, D., Pfeiffer, M., & Liu, S. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.48550/arxiv.1610.09513 · Cited by 319
“An inspiration for using oscillations in recurrent networks comes from computational neuroscience [3], where rhythms have been shown to play important roles for synchronization and plasticity [22].”
BackgroundSelf-citationSpin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems
Vincent, A. F., Larroque, J., Locatelli, N., Ben Romdhane, N., Bichler, O., Gamrat, C., Zhao, W. S., Klein, J.-O., Galdin-Retailleau, S., & Querlioz, D. (2015) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2015.2414423 · Cited by 360
“The learning rule is also reminiscent of the simplified STDP proposed for computational neuroscience studies in [59], but in a binary and probabilistic version.”
Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines
Neftci, E. O., Augustine, C., Paul, S., & Detorakis, G. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00324 · Cited by 237
“STDP has been shown to be very powerful in a number of different models and tasks [Neftci et al., 2014a; Nessler et al., 2013; Thorpe et al., 2001].”
BackgroundPlasticity in memristive devices for spiking neural networks
Saïghi, S., Mayr, C. G., Serrano-Gotarredona, T., Schmidt, H., Lecerf, G., Tomas, J., Grollier, J., Boyn, S., Vincent, A. F., Querlioz, D., et al. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00051 · Cited by 238
Spiking Deep Networks with LIF Neurons
Hunsberger, E., & Eliasmith, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.08829 · Cited by 203
“Networks could also be trained offline as described here and then fine-tuned online using an STDP rule, such as the one described in [22], to help further reduce errors associated with converting from rate-based to spike-based networks, while avoiding difficulties with training a network in spiking…”
MethodSpiking Neural Networks: A Survey
Nunes, J. D., Carvalho, M., Carneiro, D., & Cardoso, J. S. (2022) · IEEE Access · doi:10.1109/access.2022.3179968 · Cited by 215
“On one hand, they added an exponential weight dependence to the previous strategy of synaptic traces [119, 120] (see Equation 10).”
BackgroundNoise as a Resource for Computation and Learning in Networks of Spiking Neurons
Maass, W. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2310593 · Cited by 198
“This figure was previously published in [15] under the CCAL.”
BackgroundMethodSelf-citationMagnetic Tunnel Junction Mimics Stochastic Cortical Spiking Neurons
Sengupta, A., Panda, P., Wijesinghe, P., Kim, Y., & Roy, K. (2016) · Scientific Reports · doi:10.1038/srep30039 · Cited by 193
The computational nature of memory modification
Gershman, S. J., Monfils, M.-H., Norman, K. A., & Niv, Y. (2017) · eLife · doi:10.7554/elife.23763 · Cited by 172
“For example, Friston (2005) suggests that it is a basic motif for synaptic plasticity in the cortex, and biologically plausible spiking neuron implementations have been put forth by Deneve (2008) and Nessler et al. (2013) .”
BackgroundPathways to efficient neuromorphic computing with non-volatile memory technologies
Chakraborty, I., Jaiswal, A., Saha, A. K., Gupta, S. K., & Roy, K. (2020) · Applied Physics Reviews · doi:10.1063/1.5113536 · Cited by 168
Lieder, F., Griffiths, T. L., & Hsu, M. (2018) · Psychological Review · doi:10.1037/rev0000074 · Cited by 166
Sengupta, A., & Roy, K. (2017) · Applied Physics Reviews · doi:10.1063/1.5012763 · Cited by 158
Bioinspired Programming of Memory Devices for Implementing an Inference Engine
Querlioz, D., Bichler, O., Vincent, A. F., & Gamrat, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2437616 · Cited by 146
“In this work, we abandon biological realism for a highly simplified version of STDP [36], shown in Fig.”
BackgroundMethodInfluentialPhase-change memtransistive synapses for mixed-plasticity neural computations
Sarwat, S. G., Kersting, B., Moraitis, T., Jonnalagadda, V. P., & Sebastian, A. (2022) · Nature Nanotechnology · doi:10.1038/s41565-022-01095-3 · Cited by 156
Training Spiking Deep Networks for Neuromorphic HardwarePreprint
Hunsberger, E., & Eliasmith, C. (2016) · arXiv (Cornell University) · doi:10.13140/rg.2.2.10967.06566 · Cited by 185
“Networks could also be trained offline as described here and then fine-tuned online using an STDP rule [22, 23] to help further reduce errors associated with converting from rate-based to spike-based networks, while avoiding difficulties with training a network in spiking neurons from scratch.”
MethodCarbon Nanotube Synaptic Transistor Network for Pattern Recognition
Kim, S., Yoon, J., Kim, H., & Choi, S. (2015) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.5b08541 · Cited by 140
Memristors Empower Spiking Neurons With Stochasticity
Al-Shedivat, M., Naous, R., Cauwenberghs, G., & Salama, K. N. (2015) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2015.2435512 · Cited by 128
“The function of inhibition is important for the network to operate in a balanced regime and for controlling the WTA sampling rate [8].”
BackgroundMethodResultInfluentialNetwork Plasticity as Bayesian Inference
Kappel, D., Habenschuss, S., Legenstein, R., & Maass, W. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004485 · Cited by 126
“The factor wi = exp(θi − θ0) had been discussed in [36], because it is compatible with the underlying generative model, but provides in addition a better fit to the experimental data of [41].”
BackgroundMethodInfluentialSelf-citationMemristor for computing: Myth or reality?
Hamdioui, S., Kvatinsky, S., Cauwenberghs, G., Xie, L., Wald, N., Joshi, S., Mostafa, H., Corporaal, H., & Bertels, K. (2017) · Design, Automation, and Test in Europe · doi:10.23919/date.2017.7927083 · Cited by 122
“Learning rules based on STDP have been used to train spiking networks on image classification tasks [62] and on probabilistic inference tasks [63].”
MethodSpiking neurons can discover predictive features by aggregate-label learning
Gütig, R. (2016) · Science · doi:10.1126/science.aab4113 · Cited by 141
“…in spiking neural networks have typically relied on spatiotemporally local correlations between the pre-and postsynaptic activity of individual neurons to detect structure within a networks input activity ( 27 ) — for instance, throughspike-timing – dependentsynapticplastic-ity ( 25 , 26 , 28 ).”
BackgroundAn Accelerated LIF Neuronal Network Array for a Large-Scale Mixed-Signal Neuromorphic Architecture
Aamir, S. A., Stradmann, Y., Müller, P. F., Pehle, C., Hartel, A., Grübl, A., Schemmel, J., & Meier, K. (2018) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2018.2840718 · Cited by 125
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning
Kappel, D., Nessler, B., & Maass, W. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003511 · Cited by 119
“The same STDP rule, that leads to the emergence of sparse codes for individual input patterns in the absence of lateral excitatory connections [13] now leads to the emergence of context specific neural codes and even predictions for temporal sequences of such patterns.”
BackgroundMethodInfluentialSelf-citationGeorge, D., Rikhye, R. V., Gothoskar, N., Guntupalli, J. S., Dedieu, A., & Lázaro-Gredilla, M. (2021) · Nature Communications · doi:10.1038/s41467-021-22559-5 · Cited by 109
Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays
Shi, Y., Nguyen, L., Oh, S., Liu, X., Koushan, F., Jameson, J. R., & Kuzum, D. (2018) · Nature Communications · doi:10.1038/s41467-018-07682-0 · Cited by 106
Experience-dependent contextual codes in the hippocampus
Plitt, M. H., & Giocomo, L. M. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00816-6 · Cited by 101
A compound memristive synapse model for statistical learning through STDP in spiking neural networks
Bill, J., & Legenstein, R. A. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00412 · Cited by 117
“…attracted increasing attention in theoretical studies on statistical learning (Habenschuss et al., 2012; Keck et al., 2012; Habenschuss et al., 2013; Nessler et al., 2013; Kappel et al., 2014) because their comparatively simple network dynamics facilitate a comprehensive mathematical treatment.”
BackgroundMethodResultInfluentialEmergence of Dynamic Memory Traces in Cortical Microcircuit Models through STDP
Klampfl, S., & Maass, W. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5044-12.2013 · Cited by 99
“Furthermore it is shown in (Nessler et al., 2013), see Figure 8 of that article and the surrounding discussion, that the dependence of weight updates on the current weight in (5) is qualitatively consistent with published experimental data (see also the discussion in section 2.”
BackgroundMethodResultInfluentialSelf-citationA Low-Cost High-Speed Neuromorphic Hardware Based on Spiking Neural Network
Farsa, E. Z., Ahmadi, A., Maleki, M. A., Gholami, M., & Rad, H. N. (2019) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2019.2890846 · Cited by 115
“2890846 have also been developed in spiking neural networks studies based on Spike Timing Dependent Plasticity (STDP) [6], [7], and [8].”
MethodA neuromorphic network for generic multivariate data classification
Schmuker, M., Pfeil, T., & Nawrot, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1303053111 · Cited by 106
Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition
Xin, W., Saxena, V., & Zhu, K. (2015) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2015.2433552 · Cited by 96
“Theoretical studies have suggested STDP can be used to train spiking neural networks (SNNs) in-situ without trading-off their parallelism [9]–[12].”
BackgroundMethodInfluentialThreshold switching memristor-based stochastic neurons for probabilistic computing
Wang, K., Hu, Q., Gao, B., Lin, Q., Zhuge, F., Zhang, D., Wang, L., He, Y., Scheicher, R. H., Tong, H., et al. (2020) · Materials Horizons · doi:10.1039/d0mh01759k · Cited by 94
Review of advances in neural networks: Neural design technology stack
Almási, A.-D., Woźniak, S., Cristea, V., Leblebici, Y., & Engbersen, T. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.092 · Cited by 90
“Another connection to biological models is an equivalence between the expectation maximization algorithm (EM) and STDP, under certain conditions – for a spiking neural network arranged as a winnertake-all circuit [52].”
MethodEvent-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP
Thiele, J. C., Bichler, O., & Dupret, A. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00046 · Cited by 87
“The rule (2) is qualitatively very similar to the optimal STDP rule for stochastic neurons introduced by Nessler et al. (2013) and explored in more detail by Habenschuss et al. (2012) and Habenschuss et al. (2013).”
ResultComputational Foundations of Natural Intelligence
van Gerven, M. A. J. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00112 · Cited by 91
“In the same vein, researchers are exploring how probabilistic computations can be performed in neural networks (Pouget et al., 2013; Nessler et al., 2013; Orhan and Ma, 2016; Heeger, 2017) and deriving new biologically plausible synaptic plasticity rules (Schiess et al., 2016; Brea et al., 2016;…”
BackgroundShamsi, J., Mohammadi, K., & Shokouhi, S. B. (2018) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2815025 · Cited by 72
“STDP rule is usually applied to adjust the synaptic weights in a spiking WTA [5], [11], [25]–[31].”
MethodNeuromorphic computing enabled by physics of electron spins: Prospects and perspectives
Sengupta, A., & Roy, K. (2018) · Applied Physics Express · doi:10.7567/apex.11.030101 · Cited by 65
Efficient Design of Spiking Neural Network With STDP Learning Based on Fast CORDIC
Wu, J., Zhan, Y., Peng, Z., Ji, X., Yu, G., Zhao, R. J., & Wang, C. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3061766 · Cited by 63
“There are two major encoding strategies for image encoding, i.e., spiking rate coding (e.g., the Poisson coding) [29] and spiking latency coding [30]–[33].”
Ye, W., Chen, Y., & Liu, Y. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3179246 · Cited by 67
“Commonly, there are three training methods for SNNs, including unsupervised spike timing dependent plasticity (STDP) [31]–[34], the ANN-to-SNN [10], [35], and spikebased backpropagation (BP) [11], [12].”
MethodHartmann, C., Lazăr, A., Nessler, B., & Triesch, J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004640 · Cited by 70
“Most prominently, the authors of [61] showed in a rigorous analysis that in a feed-forward-like structure with winner-take-all circuits, STDP can be used to approximate a sampling-version of expectation maximization and thereby learn a generative model of the input.”
BackgroundMethodSelf-citationLegenstein, R., & Maass, W. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003859 · Cited by 58
“It has been shown that hidden-cause representations that require such synaptic efficacies can be learned in spiking neural networks with lateral inhibition through spike-timing-dependent (STDP)-like synaptic plasticity rules [34,35].”
BackgroundSelf-citationA framework for plasticity implementation on the SpiNNaker neural architecture
Galluppi, F., Lagorce, X., Stromatias, E., Pfeiffer, M., Plana, L. A., Furber, S. B., & Benosman, R. B. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00429 · Cited by 55
“Examples include supervised learning methods for classification of visual (e.g., Brader et al., 2007; Beyeler et al., 2013), or auditory stimuli (Sheik et al., 2012), and unsupervised learning methods like Expectation Maximization (Nessler et al., 2013; Kappel et al., 2014), Independent Component Analysis (Savin et al.…”
BackgroundMethodSelf-citationItinerancy between attractor states in neural systems
Miller, P. C. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.05.005 · Cited by 64
A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection
Chakraborty, B., She, X., & Mukhopadhyay, S. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3122092 · Cited by 77
“• We adapt the Monte Carlo dropout methodology [12] to quantify uncertainty for FSHNN and use that to discriminate true positives from false-positive.”
MethodBio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP LearningPreprint
Tavanaei, A., & Maida, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03000 · Cited by 88
“Nessler et al (2009 and 2013) showed that their STDP variation implements a stochastic online EM algo-rithm [40, 41].”
BackgroundSpatiotemporal features for asynchronous event-based data
Lagorce, X., Ieng, S.-H., Clady, X., Pfeiffer, M., & Benosman, R. B. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00046 · Cited by 55
“Nessler et al. (2009) and Nessler et al. (2013) have proven that this can be implemented in spiking neural networks, using a soft WTA circuit to compute the E-step, and an STDP learning rule to implement the M-step.”
MethodSelf-citationTestolin, A., & Zorzi, M. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00073 · Cited by 59
“Could deep learning systems exhibit the same universal resilience patterns observed in other types of complex networks (Gao et al., 2016)? other physiological mechanisms such as background oscillations and synchronous activity (Nessler et al., 2013).”
BackgroundMethodRedundancy in synaptic connections enables neurons to learn optimally
Hiratani, N., & Fukai, T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803274115 · Cited by 67
“Previous theoretical studies revealed that Hebbian-type plasticity rules eventually enable neural circuits to perform optimal computation under appropriate normalization (Soltani and Wang, 2010; Nessler et al., 2013).”
BackgroundMethodResultResearch Progress of spiking neural network in image classification: a review
Niu, L.-Y., Wei, Y. J., Liu, W., Long, J.-Y., & Xue, T.-H. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04553-0 · Cited by 60
Song, S., Choi, C., Ahn, J., Lee, J., Jang, J., Yu, B., Hong, J. P., Ryu, Y., Kim, Y., & Hwang, D. K. (2023) · InfoMat · doi:10.1002/inf2.12479 · Cited by 51
Synaptic and nonsynaptic plasticity approximating probabilistic inference
Tully, P. J., Hennig, M. H., & Lansner, A. (2014) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2014.00008 · Cited by 60
“Coupled synaptic plasticity and synaptic scaling (Keck et al., 2012) along with coupled STDP and homeostatic intrinsic excitability (Nessler et al., 2013) have been proposed in the context of the expectation maximization algorithm, whereas a model with coupled synaptic and intrinsic plasticity has…”
BackgroundMethodInfluentialSpin-transfer torque magnetic memory as a stochastic memristive synapse
Vincent, A. F., Larroque, J., Zhao, W. S., Romdhane, N. B., Bichler, O., Gamrat, C., Klein, J., Galdin-Retailleau, S., & Querlioz, D. (2014) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2014.6865325 · Cited by 61
Stable spike-timing dependent plasticity rule for multilayer unsupervised and supervised learning
Shrestha, A., Ahmed, K., Wang, Y., & Qiu, Q. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966096 · Cited by 51
“As the theoretical results remain true [20] for any , we choose .”
BackgroundMethodAttractive and repulsive effects of sensory history concurrently shape visual perception
Moon, J., & Kwon, O. (2022) · BMC Biology · doi:10.1186/s12915-022-01444-7 · Cited by 67
A detailed comparison of optimality and simplicity in perceptual decision making.
Shen, S., & Ma, W. J. (2016) · Psychological Review · doi:10.1037/rev0000028 · Cited by 46
Magnetic tunnel junction enabled all-spin stochastic spiking neural network
Srinivasan, G., Sengupta, A., & Roy, K. (2017) · Design, Automation, and Test in Europe · doi:10.23919/date.2017.7927045 · Cited by 47
“Recent theoretical research efforts have demonstrated the applicability of stochastic SNNs in complex applications including Bayesian inference [7], [8].”
BackgroundSynaptic Transmission Optimization Predicts Expression Loci of Long-Term Plasticity
Costa, R. P., Padamsey, Z., D’amour, J. A., Emptage, N. J., Froemke, R. C., & Vogels, T. P. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.021 · Cited by 53
“introduced for specific aspects of long-term synaptic plasticity, namely spike timing (Lengyel et al., 2005; Pfister et al., 2006; Brea et al., 2013; Nessler et al., 2013), but also probability distributions over synaptic weights (Lengyel et al.”
Acquisition of visual features through probabilistic spike-timing-dependent plasticity
Tavanaei, A., Masquelier, T., & Maida, A. S. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727213 · Cited by 44
“To interpret the behavior of the STDP rule probabilistically, we follow the approach of Nessler et al. (2013), p. 21, formula 28 [12].”
BackgroundMethodInfluentialSpintronic Integrate-Fire-Reset Neuron with Stochasticity for Neuromorphic Computing
Qu, Y., Mishra, R., Cen, Y., Shi, G., Sharma, R., Fong, X., & Yang, H. (2022) · Nano Letters · doi:10.1021/acs.nanolett.2c02409 · Cited by 46
In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory
Li, Y., Xiao, T. P., Bennett, C. H., Isele, E., Melianas, A., Tao, H., Marinella, M. J., Salleo, A., Fuller, E. J., & Talin, A. A. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.636127 · Cited by 53
“While STDP or STPD-like rules do not directly implement stochastic gradient descent backpropagation, they effectively sample an input and may approximate a statistical method known as expectation maximization (Nessler et al., 2013).”
MethodBinas, J., Rutishauser, U., Indiveri, G., & Pfeiffer, M. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00068 · Cited by 48
“In addition, various forms of learning rely on balanced WTA competition (Masquelier et al., 2009; Habenschuss et al., 2012; Nessler et al., 2013), which requires an adaptation of the gain as the excitatory connections into the circuit undergo plasticity.”
BackgroundSelf-citationSpike-Based Bayesian-Hebbian Learning of Temporal Sequences
Tully, P. J., Lindén, H., Hennig, M. H., & Lansner, A. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004954 · Cited by 45
“We matched the shape of the learning kernel to the shape of the EPSP (Fig 2) following previous studies [66 – 68], and also acknowledge this parameterization is sufficient but not necessary for temporal sequence recall in our model.”
BackgroundMethodSaxena, V., Xin, W., Srivastava, I., & Zhu, K. (2018) · Journal of Low Power Electronics and Applications · doi:10.3390/jlpea8040034 · Cited by 40
“Recently, it was analytically shown that the WTA with exponential STDP realizes a powerful unsupervised learning motif that implements expectation maximization; network weights converge to the log probability of the hidden input cause [7,42].”
BackgroundShi, Y., Nguyen, L., Oh, S., Liu, X., & Kuzum, D. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00405 · Cited by 57
“As can be seen from the table, multilayer SNNs (Diehl and Cook, 2015a; Kheradpisheh et al., 2017; Tavanaei and Maida, 2017; Ferré et al., 2018) generally have higher accuracy than single layer SNNs.”
BackgroundMethodResultInfluentialAn Online Unsupervised Structural Plasticity Algorithm for Spiking Neural Networks
Roy, S., & Basu, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582517 · Cited by 38
Hierarchical Bayesian Inference and Learning in Spiking Neural Networks
Guo, S., Yu, Z., Deng, F., Hu, X., & Chen, F. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2768554 · Cited by 45
“[22] built a two layer spiking neural architecture”
InfluentialPerspective: Stochastic magnetic devices for cognitive computing
Roy, K., Sengupta, A., & Shim, Y. S. (2018) · Journal of Applied Physics · doi:10.1063/1.5020168 · Cited by 40
Benchmarking Spike-Based Visual Recognition: A Dataset and Evaluation
Liu, Q., García, G. P., Stromatias, E., Serrano‐Gotarredona, T., & Furber, S. B. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00496 · Cited by 42
“Theoretical analysis (Nessler et al., 2013) showed that unsupervised STDP was able to approximate a stochastic version of Expectation Maximization, a powerful learning algorithm in machine learning.”
MethodSomatodendritic consistency check for temporal feature segmentation
Asabuki, T., & Fukai, T. (2020) · Nature Communications · doi:10.1038/s41467-020-15367-w · Cited by 52
Local dendritic balance enables learning of efficient representations in networks of spiking neurons
Mikulasch, F. A., Rudelt, L., & Priesemann, V. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2021925118 · Cited by 48
“, in winnertake-all circuits with extremely strong inhibition (15), or when the PSP is extremely short (14).”
Neural implementations of Bayesian inference
Sohn, H., & Narain, D. (2021) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2021.09.008 · Cited by 50
Dendritic error backpropagation in deep cortical microcircuitsPreprint
Sacramento, J., Costa, R. P., Bengio, Y., & Senn, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.00062 · Cited by 56
“One such type are the PV (parvalbumin-positive) cells, which have been postulated to mediate a somatic excitation-inhibition balance (Vogels et al., 2011; Froemke, 2015) and competition (Masquelier and Thorpe, 2007; Nessler et al., 2013).”
AnRAD: A Neuromorphic Anomaly Detection Framework for Massive Concurrent Data Streams
Chen, Q., Luley, R. S., Wu, Q., Bishop, M. A., Linderman, R., & Qiu, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2676110 · Cited by 39
Scalable NoC-based Neuromorphic Hardware Learning and Inference
Fang, H., Shrestha, A., Ma, D., & Qiu, Q. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489619 · Cited by 39
“We utilize a simplified version [32] of the neuron model proposed in [33].”
MethodMasquelier, T. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2017.06.032 · Cited by 37
“…selective to arbitrary repeating spike patterns, even without supervision [Gilson et al., 2011, Humble et al., 2012, Hunzinger et al., 2012, Kasabov et al., 2013, Klampfl and Maass, 2013, Krunglevicius, 2015, Masquelier et al., 2008, 2009, Nessler et al., 2013, Sun et al., 2016, Yger et al., 2015].”
Belief states as a framework to explain extra-retinal influences in visual cortex
Nienborg, H., & Roelfsema, P. R. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.10.013 · Cited by 35
Feedback Inhibition Shapes Emergent Computational Properties of Cortical Microcircuit Motifs
Jonke, Z., Legenstein, R. A., Habenschuss, S., & Maass, W. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2078-16.2017 · Cited by 33
“Model M is also somewhat similar to other models (Habenschuss et al., 2013a; Nessler et al., 2013; Kappel et al., 2014).”
MethodResultInfluentialSelf-citationEmergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All
Yu, Z., Guo, S., Deng, F., Yan, Q., Huang, K., Liu, J. K., & Chen, F. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2871144 · Cited by 37
“Note that the WTA model used in this paper is a soft WTA model [2], [39], it is interesting to see if other WTA models also work [46], [47].”
MethodEnergy-Efficient CMOS Memristive Synapses for Mixed-Signal Neuromorphic System-on-a-Chip
Saxena, V., Xin, W., & Zhu, K. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351766 · Cited by 33
Evaluation of ambiguous associations in the amygdala by learning the structure of the environment
Madarász, T. J., Díaz-Mataix, L., Akhand, O., Ycu, E. A., LeDoux, J. E., & Johansen, J. P. (2016) · Nature Neuroscience · doi:10.1038/nn.4308 · Cited by 32
Deep Convolutional Spiking Neural Networks for Image Classification
Vaila, R. (2021) · ScholarWorks (Boise State University) · doi:10.18122/td.1782.boisestate · Cited by 39
SoftHebb: Bayesian inference in unsupervised Hebbian soft winner-take-all networks
Moraitis, T., Toichkin, D., Journé, A., Chua, Y., & Guo, Q. (2022) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/aca710 · Cited by 54
Vohra, S. K., Thomas, S. A., Sakare, M., & Das, D. M. (2023) · Integration · doi:10.1016/j.vlsi.2023.102122 · Cited by 30
Space is a latent sequence: A theory of the hippocampus
Raju, R. V., Guntupalli, J. S., Zhou, G., Wendelken, C., Lázaro-Gredilla, M., & George, D. (2024) · Science Advances · doi:10.1126/sciadv.adm8470 · Cited by 25
Developmental Self-Construction and -Configuration of Functional Neocortical Neuronal Networks
Bauer, R., Zubler, F., Pfister, S., Hauri, A., Pfeiffer, M., Muir, D. R., & Douglas, R. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003994 · Cited by 32
“This fundamental characteristic is applicable in various contexts, and so many studies modeling cortical function are based on WTA modules [8–15].”
BackgroundSelf-citationScaling Deep Spiking Neural Networks with Binary Stochastic Activations
Roy, D., Chakraborty, I., & Roy, K. (2019) · IEEE International Conference on Cognitive Computing (ICCC) · doi:10.1109/iccc.2019.00020 · Cited by 35
“Several works [11], [12] indicate that cortical microcircuits of pyramidal neurons perform Bayesian computations, and their switching behavior can be modelled by a sigmoid [13].”
Racing to learn: statistical inference and learning in a single spiking neuron with adaptive kernels
Afshar, S., George, L., Tapson, J. C., André, V. S., & Hamilton, T. J. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00377 · Cited by 29
“…algorithms utilize multi neuron Winner-Take-All layers with real valued rate based inhibitory signals to prevent correlated spiking and maximize the network learning capacity (Gupta and Long, 2009; Nessler et al., 2013), in a SKAN network a simple global inhibitory OR gate serves the same function.”
MethodSpatio-temporal Spike Pattern Classification in Neuromorphic Systems
Sheik, S., Pfeiffer, M., Stefanini, F., & Indiveri, G. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39802-5_23 · Cited by 28
Self-citationRate-distortion theory of neural coding and its implications for working memory
Jakob, A. M. V., & Gershman, S. J. (2023) · eLife · doi:10.7554/elife.79450 · Cited by 35
Metadata
- Authors
- Nessler, B., Pfeiffer, M., Buesing, L., & Maass, W.
- Affiliations
- Graz University of Technology (AT)
- University of Zurich (CH)
- Corresponding author
- Nessler, Bernhard
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-04-25
- Volume
- 9
- Issue
- 4
- Pages
- e1003037
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3636028
- OpenAlex
- W2003400208
- Semantic Scholar
- 24d54b44151bc1161f66249f75d049a88f22e0c7
- Cited by
- 281
- References
- 121
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 18 · 2023: 17 · 2022: 16 · 2021: 25 · 2020: 18 · 2019: 21 · 2018: 25 · 2017: 35 · 2016: 37 · 2015: 28 · 2014: 18 · 2013: 5
- FWCI
- 16.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Samsung Advanced Institute of Technology
- European Commission (269921, 243914, 506778, 216593)
- Universität Zürich
- Samsung
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neuropharmacology Research
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- Bayesian computation, spike-timing-dependent plasticity, cortical microcircuits, winner-take-all circuit, implicit generative models, expectation-maximization algorithm, spontaneous activity, trial-to-trial variability, distributed information processing, pyramidal neurons, interneurons
- MeSH terms
- Action Potentials, Animals, Bayes Theorem, Brain, Computer Simulation, Humans, Models, Neurological, Nerve Net, Synaptic Transmission, Neuronal Plasticity, Neurons, Probability, Synapses, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.