Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows ("explaining away") and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 1×
“Another spiking neural implementation of the sum-product belief propagation algorithm was proposed in [36], where the calculation and passing of the messages was achieved in a recurrent network of interconnected liquid state machines [49].”
BackgroundSelf-citationThe Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · cited 1×
“…p , for example the computation of marginal probabilities for specific RVs, can be reduced to counting the number of spikes of specific neurons for a behaviorally relevant time span of a few hundred ms, similarly as in previously proposed mechanisms for evidence accumulation in neural systems [4].”
BackgroundA learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
An introduction to MCMC for machine learning.
Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003) · Machine Learning · doi:10.1023/a:1020281327116 · Cited by 2,451
Markov Chains and Mixing Times
Levin, D., Peres, Y., & Wilmer, E. (2008) · doi:10.1090/mbk/058 · Cited by 2,241
How to grow a mind: Statistics, structure, and abstraction
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011) · Science · doi:10.1126/science.1192788 · Cited by 1,815
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Raichle, M. E. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.008 · Cited by 1,111 · cited 1×
“It has been partially confirmed through more detailed analyses, which show that spontaneous brain activity shows many characteristic features of brain responses to natural external stimuli ([3,58,59]).”
BackgroundLearning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037
Spontaneously emerging cortical representations of visual attributes
Kenet, T., Bibitchkov, D., Tsodyks, M., Grinvald, A., & Arieli, A. (2003) · Nature · doi:10.1038/nature02078 · Cited by 924 · cited 1×
“It has been partially confirmed through more detailed analyses, which show that spontaneous brain activity shows many characteristic features of brain responses to natural external stimuli ([3,58,59]).”
BackgroundSpontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment
Berkes, P., Orbán, G., Lengyel, M., & Fiser, J. (2011) · Science · doi:10.1126/science.1195870 · Cited by 782
Statistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672 · cited 1×
“…abstract computational models of information processing in the brain that use MCMC algorithms for probabilistic inference on one hand, and neuroscientific data about neural structures and neural processes on the other hand, has been pointed out and emphasized by several studies [12,13,54,55].”
BackgroundCompartmentalized dendritic plasticity and input feature storage in neurons
Losonczy, A., Makara, J. K., & Magee, J. C. (2008) · Nature · doi:10.1038/nature06725 · Cited by 598 · cited 6×
“Further, in [22] it is shown that the latency of an action potential evoked by a strong dendritic spike, calculated with respect to the time of the activation of the synaptic input at the dendritic branch, is slightly below 2 ms, supporting the assumption of fast propagation of the dendritic spike…”
BackgroundInfluentialBuesing, L., Bill, J., Nessler, B., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002211 · Cited by 384 · cited 10×
“Before the first use of this strategy in networks of spiking neurons in [1], MCMC sampling had already been studied in the context of artificial neural networks, so-called Boltzmann machines [15].”
BackgroundMethodInfluentialSelf-citationBayesian Spiking Neurons I: Inference
Deneve, S. (2008) · Neural Computation · doi:10.1162/neco.2008.20.1.91 · Cited by 313 · cited 6×
“It uses a population code for encoding the log-posterior estimation of the time varying hidden stimulus, which is modeled as a continuous RV instead of the binary hidden state used in [17].”
BackgroundMethodInfluentialBayesian Computation in Recurrent Neural Circuits
Rao, R. P. N. (2004) · Neural Computation · doi:10.1162/08997660460733976 · Cited by 252
An embedded subnetwork of highly active neurons in the neocortex
Yassin, L., Benedetti, B. L., Jouhanneau, J.-S., Wen, J. A., Poulet, J. F. A., & Barth, A. L. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.029 · Cited by 227 · cited 1×
“Such strong differences in the firing activity of neurons has already been found in some experimental studies, see [64,65].”
Result
Cited by
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
“Pecevski et al. 101 used neuromorphic hardware to perform inference and sampling on general graphical structures, such as Bayes’s nets, which is NP complete for random variables with probabilities not bounded away from zero 102 .”
MethodStochastic phase-change neurons
Tuma, T., Pantazi, A., Le Gallo, M., Sebastian, A., & Eleftheriou, E. (2016) · Nature Nanotechnology · doi:10.1038/nnano.2016.70 · Cited by 1,109
Integration of nanoscale memristor synapses in neuromorphic computing architectures
Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G., & Prodromakis, T. (2013) · Nanotechnology · doi:10.1088/0957-4484/24/38/384010 · Cited by 613
Metadata
- Authors
- Pecevski, D., Buesing, L., & Maass, W.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-12-15
- Volume
- 7
- Issue
- 12
- Pages
- e1002294
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2163179684
- Cited by
- 109
- References
- 18
- FWCI
- 3.17
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.