Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity
Open accessThe DOI matches its Crossref record; 121 of the 121 references it cites were found in a registry or an index.
Abstract
The principles by which networks of neurons compute, and how spike-timing dependent plasticity (STDP) of synaptic weights generates and maintains their computational function, are unknown. Preceding work has shown that soft winner-take-all (WTA) circuits, where pyramidal neurons inhibit each other via interneurons, are a common motif of cortical microcircuits. We show through theoretical analysis and computer simulations that Bayesian computation is induced in these network motifs through STDP in combination with activity-dependent changes in the excitability of neurons. The fundamental components of this emergent Bayesian computation are priors that result from adaptation of neuronal excitability and implicit generative models for hidden causes that are created in the synaptic weights through STDP. In fact, a surprising result is that STDP is able to approximate a powerful principle for fitting such implicit generative models to high-dimensional spike inputs: Expectation Maximization. Our results suggest that the experimentally observed spontaneous activity and trial-to-trial variability of cortical neurons are essential features of their information processing capability, since their functional role is to represent probability distributions rather than static neural codes. Furthermore it suggests networks of Bayesian computation modules as a new model for distributed information processing in the cortex.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



![Figure 4 Relationship between the continuous-time SEM model and experimental data on synaptic plasticity. A–C: The effect of the continuous-time plasticity rule in Eq. (18) at a single synapse for different stimulation frequencies and different time-differences between pre- and post-synaptic spike pairs. Only time-intervals without overlapping pairs are shown. A: For very low stimulation frequencies (1 Hz) the standard shape of the complex learning rule from Fig. 2 is recovered. B: At a stimulation frequency of 20 Hz the plasticity curve shifts more towards LTP, and depression is no longer time independent, due to overlapping EPSPs. C: At high stimulation frequencies of 40 Hz or above, the STDP curve shifts towards only LTP, and thus becomes similar to a rate-based Hebbian learning rule. D: Cumulative effect of pre- and post-synaptic burst stimulation (50 Hz bursts of 5 pre-synaptic and 4 post-synaptic spikes) with different onset delays of -120, -60, 10, 20, 30, 80 and 140 ms (time difference between the onsets of the post- and pre-synaptic bursts). As in [55], the amount of overlap between bursts determines the magnitude of LTP, rather than the exact temporal order of spikes.](https://figures.cochat.ai/paper-figures/c8793663018c4fae8d611dbd2370c7d4/bayesian-computation-emerges-in-generic-cortical-microcircuits-through-spike-figure-4.jpg)





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The EM algorithm that is used for learning is a local update mechanism analogous to spike timing–dependent plasticity (24, 25).”
From Space is a latent sequence: A theory of the hippocampus (2024) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“6 through an application to a computational task that is substantially more difficult than previously considered tasks for networks of spiking neurons: We show that a simple network of spiking neurons can learn without any supervision to discriminate handwritten digits from the MNIST benchmark dataset [64] consisting…”
BackgroundMaximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · cited 4×
“The fact that STDP approximates EM is remarkable, since it is known from machine learning that EM can solve a fundamental chicken-and-egg problem of unsupervised learning systems [31]: To detect - without a teacher - hidden causes for complex input data, and to induce separate learning agents to specialize each on one…”
BackgroundMethodInfluentialBook Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 2×
“or semi-supervised machine learning algorithms [27,65], or to evaluate unsupervised feature learning approaches [66,67].”
BackgroundA model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271 · cited 1×
“These ubiquitous network motifs were called soft winner-take-all (WTA) circuits, and have been suggested as neural network models for implementing functions like non-linear selection [16,17], normalization [18], selective attention [19], decision making [20,21], or as primitives for general purpose computation [22,23].”
BackgroundHebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · cited 2×
“Our model shows that a WTA circuit could learn orientation selectivity from such simple bar-like inputs, but does not provide an alternative explanation to the results of studies like [61] using natural image stimuli.”
BackgroundResultBi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744 · cited 2×
“standard models of STDP and observed biological data [43,54].”
ResultTheoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 4,099
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
Faisal, A. A., Selen, L. P. J., & Wolpert, D. M. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2258 · Cited by 2,883
Competitive Hebbian learning through spike-timing-dependent synaptic plasticity
Song, S., Miller, K. D., & Abbott, L. F. (2000) · Nature Neuroscience · doi:10.1038/78829 · Cited by 2,729 · cited 3×
“[54] initiated the investigation of STDP in the context of unsupervised competitive Hebbian learning and demonstrated that correlations of input spike trains can be learned in this way.”
BackgroundMethodHierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.181 · Cited by 2,673
Salakhutdinov, R., & Hinton, G. E. (2009) · International Conference on Artificial Intelligence and Statistics · Cited by 2,605
Synaptic plasticity: taming the beast
Abbott, L. F., & Nelson, S. B. (2000) · Nature Neuroscience · doi:10.1038/81453 · Cited by 2,263 · cited 3×
“excitability through homeostatic mechanisms were observed in experiments [13,46].”
BackgroundA View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 1×
“Stochastic approximation algorithms for expectation maximization [31] were first considered in [79], incremental and on-line EM algorithms with soft-max competition in [80–82].”
MethodBayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004 · cited 1×
“Following the ‘‘probabilistic turn’’ in cognitive science [3,4,69] and related hypotheses in computational neuroscience [1,2,5], probabilistic inference has become very successful in explaining behavioral data on human reasoning and other brain functions.”
BackgroundSpike Timing–Dependent Plasticity: A Hebbian Learning Rule
Caporale, N., & Dan, Y. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.31.060407.125639 · Cited by 1,997 · cited 2×
“Several experimental studies have already confirmed, that the intrinsic excitability of neurons does in fact increase when they are more frequently activated [45], see [97], [14] and [39] for reviews.”
BackgroundNormalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906 · cited 2×
“Inhibition in the WTA accounts for normalization [18], and in addition controls the rate at which samples are generated.”
BackgroundNEURONAL CIRCUITS OF THE NEOCORTEX
Douglas, R. J., & Martin, K. A. C. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144152 · Cited by 1,886 · cited 5×
“Our model consists of a network of spiking neurons, arranged in a WTA circuit, which is one of the most frequently studied connectivity patterns (or network motifs) of cortical microcircuits [16].”
BackgroundMethodResultInfluentialStochastic Approximation and Recursive Algorithms and Applications
(2003) · Stochastic Modelling and Applied Probability · doi:10.1007/b97441 · Cited by 1,838 · cited 5×
“(5) implements a stochastic version of the generalized EM algorithm; instead, we have to resort to the theory of stochastic approximation algorithms as presented in [52].”
BackgroundMethodInfluentialNeuromorphic Silicon Neuron Circuits
Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Schaik, A. V., Etienne-Cummings, R., Delbruck, T., Liu, S.-C., Dudek, P., Häfliger, P., Renaud, S., et al. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00073 · Cited by 1,821 · cited 1×
“This could benefit the field of neuromorphic engineering [74–76], which develops dedicated massively parallel and very efficient hardware for emulating spiking neural networks and suitable plasticity rules.”
BackgroundBayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit
Hahnloser, R. H. R., Sarpeshkar, R., Mahowald, M. A., Douglas, R. J., & Seung, H. S. (2000) · Nature · doi:10.1038/35016072 · Cited by 1,293 · cited 1×
“These ubiquitous network motifs were called soft winner-take-all (WTA) circuits, and have been suggested as neural network models for implementing functions like non-linear selection [16,17], normalization [18], selective attention [19], decision making [20,21], or as primitives for general purpose computation [22,23].”
BackgroundGerstner, W. (2009) · Encyclopedia of Neuroscience · doi:10.1016/b978-008045046-9.01405-4 · Cited by 1,199 · cited 2×
“The individual units zk are modeled by a simplified Spike Response Model [37] in which the membrane potential is computed as the difference between the excitatory input uk(t) and the common inhibition term I(t).”
MethodRate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity
Sjöström, P. J., Turrigiano, G. G., & Nelson, S. B. (2001) · Neuron · doi:10.1016/s0896-6273(01)00542-6 · Cited by 1,174 · cited 8×
“Many different versions of STDP rules have emerged from experimental data [8,39,40].”
BackgroundResultInfluentialUnsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F. J., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,086 · cited 1×
“or semi-supervised machine learning algorithms [27,65], or to evaluate unsupervised feature learning approaches [66,67].”
BackgroundDeep, Big, Simple Neural Nets for Handwritten Digit Recognition
Cireşan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010) · Neural Computation · doi:10.1162/neco_a_00052 · Cited by 1,078 · cited 1×
“or semi-supervised machine learning algorithms [27,65], or to evaluate unsupervised feature learning approaches [66,67].”
BackgroundThe Spike-Timing Dependence of Plasticity
Feldman, D. E. (2012) · Neuron · doi:10.1016/j.neuron.2012.08.001 · Cited by 1,071 · cited 1×
“The synaptic connections are subject to various forms of plasticity, and recent experimental results have emphasized the role of STDP, which constantly modifies synaptic strengths (weights) in dependence of the difference between the firing times of the pre- and postsynaptic neurons (see [8,9] for reviews).”
ResultFeature Discovery by Competitive Learning*
Rumelhart, D. E., & Zipser, D. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_5 · Cited by 1,057 · cited 1×
“A first model for competitive Hebbian learning paradigm in non-spiking networks of neurons had been introduced in [77].”
BackgroundSpike Timing-Dependent Plasticity of Neural Circuits
Dan, Y., & Poo, M.-M. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.007 · Cited by 1,040
Spike-timing-dependent synaptic modification induced by natural spike trains
Froemke, R. C., & Dan, Y. (2002) · Nature · doi:10.1038/416433a · Cited by 803 · cited 2×
“Experimental evidence shows that the time constants of the LTP learning window are usually smaller than the time constants of the LTD window ([40,60]), which will further enhance the specificity of the LTP learning as opposed to the LTD part that computes the average over a longer window.”
BackgroundNeural synchrony in cortical networks: history, concept and current status
Uhlhaas, P. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.017.2009 · Cited by 758 · cited 1×
“This has already been shown for neurons in close proximity [107] but also for neurons in different cortical areas [108,109].”
BackgroundNeural reorganization following sensory loss: the opportunity of change
Merabet, L. B., & Pascual-Leone, A. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2758 · Cited by 751 · cited 1×
“Furthermore in the case of sensory deprivation (see [106]) our model predicts that neurons that used to encode stimuli which no longer occur will start to participate in the encoding of other stimuli.”
MethodInstantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities
Okun, M., & Lampl, I. (2008) · Nature Neuroscience · doi:10.1038/nn.2105 · Cited by 693 · cited 1×
“Such instantaneous balance between excitation and inhibition was described by [34].”
BackgroundThe Neural Basis of Perceptual Learning
Gilbert, C. D., Sigman, M., & Crist, R. E. (2001) · Neuron · doi:10.1016/s0896-6273(01)00424-x · Cited by 691
Development and plasticity of the primary visual cortex
Espinosa, J. S., & Stryker, M. P. (2012) · Neuron · doi:10.1016/j.neuron.2012.06.009 · Cited by 672
Statistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672 · cited 2×
“important task of probabilistic perception [6], in which hidden”
BackgroundToo many cooks? Intrinsic and synaptic homeostatic mechanisms in cortical circuit refinement
Turrigiano, G. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-060909-153238 · Cited by 671 · cited 1×
“excitability through homeostatic mechanisms were observed in experiments [13,46].”
BackgroundHebbian learning and spiking neurons
Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999) · Physical Review E · doi:10.1103/physreve.59.4498 · Cited by 664 · cited 1×
“Functional consequences of STDP can resemble those of rate-based Hebbian models [10], but may also lead to the emergence of temporal coding [11] and ratenormalization [12,13].”
Bayesian Rationality: The Probabilistic Approach to Human Reasoning
Oaksford, M., & Chater, N. (2007) · Cited by 655
Fluctuating synaptic conductances recreate in vivo-like activity in neocortical neurons
Destexhe, A., Rudolph, M., Fellous, J.-M., & Sejnowski, T. J. (2001) · Neuroscience · doi:10.1016/s0306-4522(01)00344-x · Cited by 642 · cited 2×
“(3) are firing already on their own - even without any excitatory drive from the input neuron y (due to assumed background synaptic inputs; modeled in our simulations by an Ornstein-Uhlenbeck process, as suggested by in-vivo data [110]).”
BackgroundMethodDecorrelated neuronal firing in cortical microcircuits
Ecker, A. S., Berens, P., Keliris, G. A., Bethge, M., Logothetis, N. K., & Tolias, A. S. (2010) · Science · doi:10.1126/science.1179867 · Cited by 640
Optimal Predictions in Everyday Cognition
Griffiths, T. L., & Tenenbaum, J. B. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01780.x · Cited by 639 · cited 1×
“Following the ‘‘probabilistic turn’’ in cognitive science [3,4,69] and related hypotheses in computational neuroscience [1,2,5], probabilistic inference has become very successful in explaining behavioral data on human reasoning and other brain functions.”
BackgroundConnectivity reflects coding: a model of voltage-based STDP with homeostasis
Clopath, C., Büsing, L., Vasilaki, E., & Gerstner, W. (2010) · Nature Neuroscience · doi:10.1038/nn.2479 · Cited by 607 · cited 1×
“Other phenomenological [56,57] or mechanistic models of STDP [58] also show some of these characteristics, but come without such theoretical properties.”
BackgroundSelf-citationPhenomenological models of synaptic plasticity based on spike timing
Morrison, A., Diesmann, M., & Gerstner, W. (2008) · Biological Cybernetics · doi:10.1007/s00422-008-0233-1 · Cited by 593 · cited 1×
“Other phenomenological [56,57] or mechanistic models of STDP [58] also show some of these characteristics, but come without such theoretical properties.”
BackgroundFace-Selective Cells in the Temporal Cortex of Monkeys
Desimone, R. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.1 · Cited by 565 · cited 1×
“face-selective cells in IT, [71]).”
Basal forebrain activation enhances cortical coding of natural scenes
Goard, M., & Dan, Y. (2009) · Nature Neuroscience · doi:10.1038/nn.2402 · Cited by 556 · cited 1×
“These predictions are consistent with a number of experimental data related to perceptual learning [102,103], and with the evolution of neuronal responses to natural scenes that were shown repeatedly in conjunction with nucleus basalis stimulation [104].”
ResultOaksford, M., & Chater, N. (2007) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198524496.001.0001 · Cited by 549
Long-Term Plasticity of Intrinsic Excitability: Learning Rules and Mechanisms
Daoudal, G., & Debanne, D. (2003) · Learning & Memory · doi:10.1101/lm.64103 · Cited by 543 · cited 2×
“Several experimental studies have already confirmed, that the intrinsic excitability of neurons does in fact increase when they are more frequently activated [45], see [97], [14] and [39] for reviews.”
BackgroundDense Inhibitory Connectivity in Neocortex
Fino, E., & Yuste, R. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.025 · Cited by 532 · cited 1×
“[16,34–36].”
Neural synchrony and the development of cortical networks
Uhlhaas, P. J., Roux, F., Rodriguez, E., Rotarska-Jagiela, A., & Singer, W. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.12.002 · Cited by 532 · cited 1×
“This has already been shown for neurons in close proximity [107] but also for neurons in different cortical areas [108,109].”
BackgroundBayesian Brain: Probabilistic Approaches to Neural Coding
(2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262042383.001.0001 · Cited by 511
Griffiths, T. L., Kemp, C., & Tenenbaum, J. B. (2001) · The Cambridge Handbook of Computational Psychology · doi:10.1017/cbo9780511816772.006 · Cited by 507 · cited 1×
“Following the ‘‘probabilistic turn’’ in cognitive science [3,4,69] and related hypotheses in computational neuroscience [1,2,5], probabilistic inference has become very successful in explaining behavioral data on human reasoning and other brain functions.”
BackgroundCortical Development and Remapping through Spike Timing-Dependent Plasticity
Song, S., & Abbott, L. F. (2001) · Neuron · doi:10.1016/s0896-6273(01)00451-2 · Cited by 476
How Close Are We to Understanding V1?
Olshausen, B. A., & Field, D. J. (2005) · Neural Computation · doi:10.1162/0899766054026639 · Cited by 414 · cited 1×
“A number of experimental studies confirm this predictions (see [98] for a review).”
ResultGraupner, M., & Brunel, N. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1109359109 · Cited by 390 · cited 3×
“When an output neuronzk fired, on average only 8.6% of the input neuronsyi had fired during the preceding 10 ms (the time window for potentiation accordingto the STDP rule inEq.”
BackgroundInfluentialWeight consistency specifies regularities of macaque cortical networks
Markov, N. T., Misery, P., Falchier, A., Lamy, C., Vezoli, J., Quilodran, R., Gariel, M. A., Giroud, P., Ercsey-Ravasz, M., Pilaz, L. J., et al. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhq201 · Cited by 386
Buesing, L., Bill, J., Nessler, B., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002211 · Cited by 384 · cited 2×
“In particular, [24,25] have recently shown how recurrently connected neurons can be designed to perform neural sampling, an approach in which time-independent probability distributions can be represented through spiking activity in recurrent neural networks.”
BackgroundSelf-citationOn the Computational Power of Winner-Take-All
Maass, W. (2000) · Neural Computation · doi:10.1162/089976600300014827 · Cited by 355 · cited 2×
“Such networks can be composed out of interconnected soft WTA circuits, which has been shown to be a powerful principle for designing neural networks that can solve arbitrary deterministic or stochastic computations [22,23,25].”
BackgroundSelf-citationSum-Product Networks: A New Deep Architecture
Poon, H., & Domingos, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1202.3732 · Cited by 352 · cited 1×
“The hierarchical structure, however, that would result from such deeply organized SEM-modules is more reminiscent of a Deep Sum-Product Network [72], a recently presented new architecture, which has a much simpler learning dynamics but arguably a similar expressive power as DBM.”
ResultCompetitive STDP-Based Spike Pattern Learning
Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2009) · Neural Computation · doi:10.1162/neco.2008.06-08-804 · Cited by 340 · cited 2×
“Competitive pattern learning through STDP has recently been studied in [68].”
BackgroundResultPrécis of Bayesian Rationality: The Probabilistic Approach to Human Reasoning
Oaksford, M., & Chater, N. (2009) · Behavioral and Brain Sciences · doi:10.1017/s0140525x09000284 · Cited by 335 · cited 1×
“Following the ‘‘probabilistic turn’’ in cognitive science [3,4,69] and related hypotheses in computational neuroscience [1,2,5], probabilistic inference has become very successful in explaining behavioral data on human reasoning and other brain functions.”
BackgroundSum-product networks: A new deep architecture
Poon, H., & Domingos, P. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130310 · Cited by 315 · cited 1×
“The hierarchical structure, however, that would result from such deeply organized SEM-modules is more reminiscent of a Deep Sum-Product Network [72], a recently presented new architecture, which has a much simpler learning dynamics but arguably a similar expressive power as DBM.”
ResultBayesian Spiking Neurons I: Inference
Deneve, S. (2008) · Neural Computation · doi:10.1162/neco.2008.20.1.91 · Cited by 313 · cited 1×
“An interesting complementary approach is presented in [90,91], where a single neuron is modeled as hidden Markov model with two possible states.”
BackgroundSpike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains
Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001377 · Cited by 312 · cited 6×
“Such time-warped versions of these patterns had never been presented during learning via STDP.”
BackgroundResultInfluentialOn-line EM Algorithm for the Normalized Gaussian Network
Sato, M.-A., & Ishii, S. (2000) · Neural Computation · doi:10.1162/089976600300015853 · Cited by 276 · cited 1×
“In contrast to previous rate-based models for probabilistic inference [28–30] every spike in our model has a clear semantic interpretation: one spike indicates the instantaneous assignment of a certain value to an abstract variable represented by the firing neuron.”
ResultSpatial Organization of Neuronal Population Responses in Layer 2/3 of Rat Barrel Cortex
Kerr, J. N. D., de Kock, C. P. J., Greenberg, D. S., Bruno, R. M., Sakmann, B., & Helmchen, F. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2210-07.2007 · Cited by 255 · cited 1×
“[99], Fig.”
Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning
Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.6.1318 · Cited by 245 · cited 1×
“LTP curves that mirror the EPSP shape are in accordance with previous studies, which analyzed optimal shapes of STDP curves under different mathematical criteria [41,42].”
ResultGenerative models for discovering sparse distributed representations
Hinton, G. E., & Ghahramani, Z. (1997) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1997.0101 · Cited by 220 · cited 1×
“The concept of a generative model is well known in theoretical neuroscience [26,27], but it has so far primarily been applied in the context of an abstract non-spiking neural circuit architecture.”
BackgroundDirect measurement of quantal changes underlying long-term potentiation in CA1 hippocampus
Liao, D., Jones, A., & Malinow, R. (1992) · Neuron · doi:10.1016/0896-6273(92)90068-o · Cited by 219
Predicting spike timing of neocortical pyramidal neurons by simple threshold models
Jolivet, R., Rauch, A., Lüscher, H.-R., & Gerstner, W. (2006) · Journal of Computational Neuroscience · doi:10.1007/s10827-006-7074-5 · Cited by 214 · cited 2×
“This result is then transformed into an instantaneous firing rate, assuming an exponential relationship between rate and the membrane potential [38].”
BackgroundResultGjorgjieva, J., Clopath, C., Audet, J., & Pfister, J.-P. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1105933108 · Cited by 213 · cited 2×
“[53]), we can study empirically the effect of a modification of the frequency of spike-pairing.”
BackgroundThe development of direction selectivity in ferret visual cortex requires early visual experience
Li, Y., Fitzpatrick, D., & White, L. E. (2006) · Nature Neuroscience · doi:10.1038/nn1684 · Cited by 204 · cited 1×
“direction selectivity, exists in V1 neurons even before eye opening [62,63].”
BackgroundDistributed Fading Memory for Stimulus Properties in the Primary Visual Cortex
Nikolić, D., Häusler, S., Singer, W., & Maass, W. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000260 · Cited by 202 · cited 1×
“1B in [100]; a review is provided in [101]).”
BackgroundSelf-citationIntrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning
Kempter, R., Gerstner, W., & Hemmen, J. L. V. (2001) · Neural Computation · doi:10.1162/089976601317098501 · Cited by 171 · cited 1×
“Functional consequences of STDP can resemble those of rate-based Hebbian models [10], but may also lead to the emergence of temporal coding [11] and ratenormalization [12,13].”
Montgomery, J. M., Pavlidis, P., & Madison, D. V. (2001) · Neuron · doi:10.1016/s0896-6273(01)00244-6 · Cited by 170 · cited 1×
“1 in [96] for other protocols for LTP induction.”
BackgroundProbabilistic Models of the Brain: Perception and Neural Function
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/5583.001.0001 · Cited by 160
Masquelier, T., Hugues, E., Deco, G., & Thorpe, S. J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2207-09.2009 · Cited by 159 · cited 5×
“Such time-warped versions of these patterns had never been presented during learning via STDP.”
BackgroundResultInfluentialLong-Term Potentiation of Intrinsic Excitability in LV Visual Cortical Neurons
Cudmore, R. H., & Turrigiano, G. G. (2004) · Journal of Neurophysiology · doi:10.1152/jn.01059.2003 · Cited by 154 · cited 1×
“Several experimental studies have already confirmed, that the intrinsic excitability of neurons does in fact increase when they are more frequently activated [45], see [97], [14] and [39] for reviews.”
BackgroundPerceptual learning and adult cortical plasticity.
Gilbert, C. D., Li, W., & Piech, V. (2009) · The Journal of Physiology · doi:10.1113/jphysiol.2009.171488 · Cited by 151 · cited 1×
“These predictions are consistent with a number of experimental data related to perceptual learning [102,103], and with the evolution of neuronal responses to natural scenes that were shown repeatedly in conjunction with nucleus basalis stimulation [104].”
ResultMaximum Likelihood Competitive Learning
Nowlan, S. J. (1989) · Cited by 138
Nowlan, S. J. (1991) · Cited by 124
Synchronization of Neuronal Responses in Primary Visual Cortex of Monkeys Viewing Natural Images
Maldonado, P., Babul, C., Singer, W., Rodriguez, E., Berger, D., & Grün, S. (2008) · Journal of Neurophysiology · doi:10.1152/jn.00076.2008 · Cited by 124 · cited 1×
“This has already been shown for neurons in close proximity [107] but also for neurons in different cortical areas [108,109].”
BackgroundLifelong plasticity in the rat auditory cortex
de Villers‐Sidani, E., & Merzenich, M. M. (2011) · Progress in brain research · doi:10.1016/b978-0-444-53752-2.00009-6 · Cited by 123
Toyoizumi, T., Pfister, J.-P., Aihara, K., & Gerstner, W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0500495102 · Cited by 123 · cited 1×
“LTP curves that mirror the EPSP shape are in accordance with previous studies, which analyzed optimal shapes of STDP curves under different mathematical criteria [41,42].”
ResultKobayashi, K., & Poo, M. (2004) · Neuron · doi:10.1016/s0896-6273(03)00873-0 · Cited by 121 · cited 3×
“Another effect that is observed in hippocampal synapses when two neurons are stimulated with bursts, is that the magnitude of LTP is determined mostly by the amount of overlap between the pre- and post-synaptic bursts, rather than the exact timing of spikes [55].”
BackgroundResultInfluentialMicrocircuits: The Interface between Neurons and Global Brain Function
Function, G. B., Grillner, S., & Graybiel, A. M. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/4596.001.0001 · Cited by 115
Computation with Spikes in a Winner-Take-All Network
Oster, M., Douglas, R. J., & Liu, S. (2009) · Neural Computation · doi:10.1162/neco.2009.07-08-829 · Cited by 114 · cited 2×
“Such WTA competition through lateral inhibition has been studied extensively in the literature [16,33].”
BackgroundMethodPecevski, D., Buesing, L., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002294 · Cited by 109 · cited 4×
“In particular, [24,25] have recently shown how recurrently connected neurons can be designed to perform neural sampling, an approach in which time-independent probability distributions can be represented through spiking activity in recurrent neural networks.”
BackgroundInfluentialSelf-citationMicrocircuits : the interface between neurons and global brain function
Function, G. B., Grillner, S., & Graybiel, A. M. (2006) · Medical Entomology and Zoology · Cited by 107
(2009) · Cited by 107
Self-citationIndependent Component Analysis in Spiking Neurons
Savin, C., Joshi, P., & Triesch, J. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000757 · Cited by 105 · cited 1×
“[83] present a network and learning model that is designed to perform Independent Component Analysis (ICA) with spiking neurons through STDP and intrinsic plasticity.”
MethodCharacter Recognition using Spiking Neural Networks
Gupta, A., & Long, L. N. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4370930 · Cited by 105 · cited 4×
“In [85] the 4 output neurons learn to differentiate the 4 presented patterns and smoothly interpolate new rotated input patterns, whereas in [86] 48 neurons learn to differentiate characters in a small pixel raster.”
BackgroundMethodInfluentialModeling Synaptic Plasticity within Networks of Highly Accelerated I&F Neurons
Schemmel, J., Brüderle, D., Meier, K., & Ostendorf, B. (2007) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2007.378289 · Cited by 103 · cited 1×
“This could benefit the field of neuromorphic engineering [74–76], which develops dedicated massively parallel and very efficient hardware for emulating spiking neural networks and suitable plasticity rules.”
BackgroundEffects of grouping in contextual modulation
Herzog, M. H., & Fahle, M. W. (2002) · Nature · doi:10.1038/415433a · Cited by 96
Stereotypical Bouton Clustering of Individual Neurons in Cat Primary Visual Cortex
Binzegger, T., Douglas, R. J., & Martin, K. A. C. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3753-07.2007 · Cited by 92 · cited 1×
“This consequence is consistent with the surprisingly weak input from the LGN to area V1 [16,111,112].”
ResultModeling Spiking Neural Networks on SpiNNaker
Jin, X., Lujan, M., Plana, L. A., Davies, S., Temple, S., & Furber, S. B. (2010) · Computing in Science & Engineering · doi:10.1109/mcse.2010.112 · Cited by 86 · cited 1×
“This could benefit the field of neuromorphic engineering [74–76], which develops dedicated massively parallel and very efficient hardware for emulating spiking neural networks and suitable plasticity rules.”
BackgroundTime-Warp–Invariant Neuronal Processing
Gütig, R., & Sompolinsky, H. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000141 · Cited by 84 · cited 2×
“In [92] it was shown that a suitable rule for supervised spikebased learning (the Tempotron learning rule) can be used to train a network to recognize spatio-temporal spike patterns.”
ResultState-Dependent Computation Using Coupled Recurrent Networks
Rutishauser, U., & Douglas, R. J. (2009) · Neural Computation · doi:10.1162/neco.2008.03-08-734 · Cited by 82 · cited 3×
“Such networks can be composed out of interconnected soft WTA circuits, which has been shown to be a powerful principle for designing neural networks that can solve arbitrary deterministic or stochastic computations [22,23,25].”
BackgroundHierarchical Bayesian Inference in Networks of Spiking Neurons
Rao, R. P. N. (2004) · neural information processing systems · Cited by 81
Bayesian spiking neurons II: learning
Deneve, S. (2008) · Neural Computation · doi:10.1162/neco.2008.20.1.118 · Cited by 71 · cited 1×
“An interesting complementary approach is presented in [90,91], where a single neuron is modeled as hidden Markov model with two possible states.”
BackgroundDebanne, D. (2010) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2010.00021 · Cited by 68 · cited 1×
“Several experimental studies have already confirmed, that the intrinsic excitability of neurons does in fact increase when they are more frequently activated [45], see [97], [14] and [39] for reviews.”
BackgroundProbabilistic Models of the Brain.
Long, D. M. (2002) · Neurosurgery Quarterly · doi:10.1097/00013414-200212000-00005 · Cited by 67 · cited 1×
“Following the ‘‘probabilistic turn’’ in cognitive science [3,4,69] and related hypotheses in computational neuroscience [1,2,5], probabilistic inference has become very successful in explaining behavioral data on human reasoning and other brain functions.”
BackgroundHebbian learning with winner take all for spiking neural networks
Gupta, A., & Long, L. N. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178751 · Cited by 51 · cited 3×
“In [85] the 4 output neurons learn to differentiate the 4 presented patterns and smoothly interpolate new rotated input patterns, whereas in [86] 48 neurons learn to differentiate characters in a small pixel raster.”
BackgroundMethodSTDP Allows Fast Rate-Modulated Coding with Poisson-Like Spike Trains
Gilson, M., Masquelier, T., & Hugues, E. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002231 · Cited by 47 · cited 6×
“Such time-warped versions of these patterns had never been presented during learning via STDP.”
BackgroundResultInfluentialReward-Modulated Hebbian Learning of Decision Making
Pfeiffer, M., Nessler, B., Douglas, R. J., & Maass, W. (2010) · Neural Computation · doi:10.1162/neco.2010.03-09-980 · Cited by 42 · cited 2×
“In previous studies [21,73] we have shown how optimal strategies can be learned very fast by simple local learning rules for reinforcement learning or categorization, if a preprocessing of input signals based on probabilistic dependencies is performed.”
BackgroundSelf-citationVariational Learning for Recurrent Spiking Networks
Rezende, D. J., Wierstra, D., & Wulfram Gerstner, B. (2011) · neural information processing systems · Cited by 40
Habenschuss, S., Bill, J., & Nessler, B. (2012) · neural information processing systems · Cited by 39
Self-citationFeedforward Inhibition and Synaptic Scaling – Two Sides of the Same Coin?
Keck, C., Savin, C., & Lücke, J. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002432 · Cited by 37 · cited 1×
“In contrast to previous rate-based models for probabilistic inference [28–30] every spike in our model has a clear semantic interpretation: one spike indicates the instantaneous assignment of a certain value to an abstract variable represented by the firing neuron.”
ResultSequence learning with hidden units in spiking neural networks
Brea, J., Senn, W., & Pfister, J.-P. (2011) · Bern Open Repository and Information System (University of Bern) · Cited by 37
Jank, W. (2006) · Operations research, computer science. Interface series · doi:10.1007/978-0-387-39934-8_21 · Cited by 33 · cited 1×
“Even though additional stochastic fluctuations are introduced due to the stochastic sampling process, this stochastic EM algorithm will also converge to a stable result in the limit of infinite iterations, if the number of samples T is increased with every iteration [51].”
BackgroundProbabilistic Computation in Spiking Populations
Zemel, R. S., Huys, Q. J. M., Natarajan, R., & Dayan, P. (2004) · UCL Discovery (University College London) · Cited by 31
Mixture Models for Learning from Incomplete Data
(1997) · The MIT Press eBooks · doi:10.7551/mitpress/2016.003.0010 · Cited by 28 · cited 2×
“This corresponds to learning with missing attribute values for xj , which is known to impair learning performance in graphical models [48].”
BackgroundNeural Models of Bayesian Belief Propagation
Rao, R. P. N. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/1535.003.0017 · Cited by 27
Hebbian Learning of Bayes Optimal Decisions
Nessler, B., Pfeiffer, M., & Maass, W. (2008) · neural information processing systems · Cited by 17
Self-citationLegenstein, R. A., Chase, S. M., Schwartz, A. B., & Maass, W. (2009) · PubMed · Cited by 12
Self-citationSpike timing-dependent plasticity as dynamic filter
Schmiedt, J. T., Albers, C., & Pawelzik, K. R. (2010) · neural information processing systems · Cited by 6
Greiner, R., Petsche, T., & Hanson, S. J. (1997) · Conference on Learning Theory · Cited by 4
Research works printed but not linked: 2
- Bishop CM (2006) Pattern Recognition and Machine Learning. New York: Springer.
- G Celeux, 1985, The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem, Comput Statist Quater, 2, 73
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Unsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
“The second STDP rule uses an exponential weight dependence (Nessler et al., 2013; Querlioz et al., 2013) to compute the weight change where β determines the strength of the weight dependence.”
MethodDeep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
“(2013) [71] extended their network by using an”
InfluentialNeuromorphic computing using non-volatile memory
Burr, G. W., Shelby, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K., Ishii, M., Narayanan, P., Fumarola, A., et al. (2017) · Advances in Physics: X · doi:10.1080/23746149.2016.1259585 · Cited by 1,107
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
Metadata
- Authors
- Nessler, B., Pfeiffer, M., Buesing, L., & Maass, W.
- Affiliations
- Graz University of Technology (AT)
- University of Zurich (CH)
- Corresponding author
- Nessler, Bernhard
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-04-25
- Volume
- 9
- Issue
- 4
- Pages
- e1003037
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3636028
- OpenAlex
- W2003400208
- Semantic Scholar
- 24d54b44151bc1161f66249f75d049a88f22e0c7
- Cited by
- 281
- References
- 121
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 18 · 2023: 17 · 2022: 16 · 2021: 25 · 2020: 18 · 2019: 21 · 2018: 25 · 2017: 35 · 2016: 37 · 2015: 28 · 2014: 18 · 2013: 5
- FWCI
- 16.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Samsung Advanced Institute of Technology
- European Commission (269921, 243914, 506778, 216593)
- Universität Zürich
- Samsung
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neuropharmacology Research
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- Bayesian computation, spike-timing-dependent plasticity, cortical microcircuits, winner-take-all circuit, implicit generative models, expectation-maximization algorithm, spontaneous activity, trial-to-trial variability, distributed information processing, pyramidal neurons, interneurons
- MeSH terms
- Action Potentials, Animals, Bayes Theorem, Brain, Computer Simulation, Humans, Models, Neurological, Nerve Net, Synaptic Transmission, Neuronal Plasticity, Neurons, Probability, Synapses, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.