Network Plasticity as Bayesian Inference
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
General results from statistical learning theory suggest to understand not only brain computations, but also brain plasticity as probabilistic inference. But a model for that has been missing. We propose that inherently stochastic features of synaptic plasticity and spine motility enable cortical networks of neurons to carry out probabilistic inference by sampling from a posterior distribution of network configurations. This model provides a viable alternative to existing models that propose convergence of parameters to maximum likelihood values. It explains how priors on weight distributions and connection probabilities can be merged optimally with learned experience, how cortical networks can generalize learned information so well to novel experiences, and how they can compensate continuously for unforeseen disturbances of the network. The resulting new theory of network plasticity explains from a functional perspective a number of experimental data on stochastic aspects of synaptic plasticity that previously appeared to be quite puzzling.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 3×
“For the special case of a restricted Boltzmann machine (RBM), maximum likelihood learning of these parameters can be done efficiently [27], and therefore RBM’s are typically used for deep learning.”
BackgroundMethodInfluentialBi, G.-Q., & Poo, M.-M. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-24-10464.1998 · Cited by 4,744
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Interneurons of the neocortical inhibitory system
Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1519 · Cited by 2,954 · cited 2×
“However also a substantial variability within the same type of synaptic connections, had been found [62].”
BackgroundExperience-dependent structural synaptic plasticity in the mammalian brain
Holtmaat, A., & Svoboda, K. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2699 · Cited by 1,934 · cited 1×
“Other experimental data point to surprising ongoing fluctuations in dendritic spines and spine volumes, to some extent even in the adult brain [1] and in the absence of synaptic activity [2].”
BackgroundLong-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex
Trachtenberg, J. T., Chen, B. E., Knott, G. W., Feng, G., Sanes, J. R., Welker, E., & Svoboda, K. (2002) · Nature · doi:10.1038/nature01273 · Cited by 1,927
Emergent Properties of Networks of Biological Signaling Pathways
Bhalla, U. S., & Iyengar, R. (1999) · Science · doi:10.1126/science.283.5400.381 · Cited by 1,686 · cited 1×
“Furthermore the stochastically varying internal states of multiple interacting biochemical signaling pathways in the postsynaptic neuron are likely to effect synaptic transmission and plasticity [22].”
BackgroundHighly nonrandom features of synaptic connectivity in local cortical circuits
Song, S., Sjöström, P. J., Reigl, M., Nelson, S., & Chklovskii, D. B. (2005) · PLoS Biology · doi:10.1371/journal.pbio.0030068 · Cited by 1,512
Variability, compensation and homeostasis in neuron and network function
Marder, E., & Goaillard, J.-M. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1949 · Cited by 1,212
Stably maintained dendritic spines are associated with lifelong memories
Yang, G., Pan, F., & Gan, W.-B. (2009) · Nature · doi:10.1038/nature08577 · Cited by 1,182 · cited 8×
“experimental results which revealed a quick decay of transient dendritic spines [30, 31, 33].”
BackgroundMethodResultInfluentialLong-term dynamics of CA1 hippocampal place codes
Ziv, Y., Burns, L. D., Cocker, E. D., Hamel, E. O., Ghosh, K. K., Kitch, L. J., Gamal, A. E., & Schnitzer, M. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3329 · Cited by 1,155
Transient and persistent dendritic spines in the neocortex in vivo
Holtmaat, A. J. G. D., Trachtenberg, J. T., Wilbrecht, L., Shepherd, G. M., Zhang, X., Knott, G. W., & Svoboda, K. (2005) · Neuron · doi:10.1016/j.neuron.2005.01.003 · Cited by 1,131
Differential signaling via the same axon of neocortical pyramidal neurons
Markram, H., Wang, Y., & Tsodyks, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.9.5323 · Cited by 1,102
Similar network activity from disparate circuit parameters
Prinz, A. A., Bucher, D., & Marder, E. (2004) · Nature Neuroscience · doi:10.1038/nn1352 · Cited by 1,091 · cited 2×
“The need to rethink observed parameter values and plasticity processes in biological networks of neurons in a way which takes into account their astounding variability and compensation capabilities has been emphasized by Eve Marder (see e.g. [11,47,60]) and others.”
BackgroundDevelopment of long-term dendritic spine stability in diverse regions of cerebral cortex
Zuo, Y., Lin, A., Chang, P., & Gan, W.-B. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.001 · Cited by 684
Statistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672 · cited 1×
“The results reported in [56] suggest that learned neural representations integrate experience with a priori beliefs about the sensory environment.”
BackgroundProbabilistic brains: knowns and unknowns
Pouget, A., Beck, J. M., Ma, W. J., & Latham, P. E. (2013) · Nature Neuroscience · doi:10.1038/nn.3495 · Cited by 615 · cited 5×
“Hence this model provides a solution to the challenge of [8] to understand how posterior distributions of weights can be represented and learned by networks of neurons in the brain.”
BackgroundMethodInfluentialExperience leaves a lasting structural trace in cortical circuits
Hofer, S. B., Mrsic-Flogel, T. D., Bonhoeffer, T., & Hübener, M. (2009) · Nature · doi:10.1038/nature07487 · Cited by 514
Experience-dependent structural plasticity in the adult human brain
May, A. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.08.002 · Cited by 512 · cited 2×
“Both structural plasticity (see [10, 11] for reviews) and synaptic plasticity (STDP) are integrated into the resulting theory of network plasticity.”
BackgroundStructural plasticity upon learning: regulation and functions
Caroni, P., Donato, F., & Muller, D. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3258 · Cited by 491 · cited 2×
“Both structural plasticity (see [10, 11] for reviews) and synaptic plasticity (STDP) are integrated into the resulting theory of network plasticity.”
BackgroundMultiple models to capture the variability in biological neurons and networks
Marder, E., & Taylor, A. L. (2011) · Nature Neuroscience · doi:10.1038/nn.2735 · Cited by 451
Impermanence of dendritic spines in live adult CA1 hippocampus
Attardo, A., Fitzgerald, J. E., & Schnitzer, M. J. (2015) · Nature · doi:10.1038/nature14467 · Cited by 399
Nessler, B., Pfeiffer, M., Buesing, L., & Maass, W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003037 · Cited by 281 · cited 5×
“The factor wi = exp(θi − θ0) had been discussed in [36], because it is compatible with the underlying generative model, but provides in addition a better fit to the experimental data of [41].”
BackgroundMethodInfluentialSelf-citationAxons and synaptic boutons are highly dynamic in adult visual cortex
Stettler, D. D., Yamahachi, H., Li, W., Denk, W., & Gilbert, C. D. (2006) · Neuron · doi:10.1016/j.neuron.2006.02.018 · Cited by 256 · cited 1×
“Also a significant portion of axonal side branches and axonal boutons were found to appear and disapper within a week in adult visual cortex, even in the absence of imposed learning and lesions [5].”
BackgroundFrom the stochasticity of molecular processes to the variability of synaptic transmission
Ribrault, C., Sekimoto, K., & Triller, A. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3025 · Cited by 198 · cited 2×
“These fluctuations may have numerous causes, from noise in the external environment over noise and fluctuations of internal states in sensory neurons and brain networks, to noise in the pre- and postsynaptic molecular machinery that implements changes in synaptic efficacies on various time scales [18].”
BackgroundRapid axonal sprouting and pruning accompany functional reorganization in primary visual cortex
Yamahachi, H., Marik, S. A., McManus, J. N. J., Denk, W., & Gilbert, C. D. (2009) · Neuron · doi:10.1016/j.neuron.2009.11.026 · Cited by 193 · cited 5×
“istributed increments Wt i W s i ˘Normal(0;t s) between times tand s). Thus in an approximation of (3) for discrete time steps tthe term dW i can be replaced by Gaussian noise with variance t(see Eq. (7)). Note that Eq. (3) does not have a single solution i(t), but a continuum of stochastic sample paths (see Fig. 1F f…”
BackgroundInfluentialPrecise temperature compensation of phase in a rhythmic motor pattern
Tang, L. S., Goeritz, M. L., Caplan, J. S., Taylor, A. L., Fisek, M., & Marder, E. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000469 · Cited by 156
Cited by
Dynamic Reorganization of Neuronal Activity Patterns in Parietal Cortex
Driscoll, L. N., Pettit, N. L., Minderer, M., Chettih, S. N., & Harvey, C. D. (2017) · Cell · doi:10.1016/j.cell.2017.07.021 · Cited by 445
“…models propose that continuous change in neuronal circuits can optimize a tradeoff between stability and flexibility by sampling frommultiple solutions of activity patterns and connectivity that similarly convey relevant information (Ajemian et al., 2013; Kappel et al., 2015; Rokni et al., 2007).”
Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures
Lee, C., Sarwar, S. S., Panda, P., Srinivasan, G., & Roy, K. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00119 · Cited by 434
Serb, A., Bill, J., Khiat, A., Berdan, R., Legenstein, R., & Prodromakis, T. (2016) · Nature Communications · doi:10.1038/ncomms12611 · Cited by 339
Self-citationCauses and consequences of representational drift
Rule, M. E., O’Leary, T., & Harvey, C. D. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.08.005 · Cited by 227
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
Metadata
- Authors
- Kappel, D., Habenschuss, S., Legenstein, R., & Maass, W.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-11-06
- Volume
- 11
- Issue
- 11
- Pages
- e1004485
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2107244895
- Cited by
- 126
- References
- 29
- FWCI
- 6.45
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.