Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Recent experiments have shown that animals and humans have a remarkable ability to adapt their learning rate according to the volatility of the environment. Yet the neural mechanism responsible for such adaptive learning has remained unclear. To fill this gap, we investigated a biophysically inspired, metaplastic synaptic model within the context of a well-studied decision-making network, in which synapses can change their rate of plasticity in addition to their efficacy according to a reward-based learning rule. We found that our model, which assumes that synaptic plasticity is guided by a novel surprise detection system, captures a wide range of key experimental findings and performs as well as a Bayes optimal model, with remarkably little parameter tuning. Our results further demonstrate the computational power of synaptic plasticity, and provide insights into the circuit-level computation which underlies adaptive decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 3×
“…has long been studied in relation to the modulation of learning rates in reinforcement learning theory and Bayesian optimal learning theory ( Pearce and Hall, 1980 ; Adams and MacKay, 2007 ; Dayan et al., 2000 ; Gallistel et al., 2001 ; Courville et al., 2006 ; Yu and Dayan, 2005 ; Behrens…”
BackgroundSynaptic Plasticity and Memory: An Evaluation of the Hypothesis
Martin, S. J., Grimwood, P. D., & Morris, R. G. M. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.649 · Cited by 2,794 · cited 1×
“…and experimental studies strongly suggest that activity-dependent synaptic plasticity plays a crucial role in learning and adaptation in general ( Martin et al., 2000 ; Kandel et al., 2000 ; Dayan and Abbott, 2001 ), the neural mechanisms behind flexible learning, especially in the case of…”
BackgroundA theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Synaptic Plasticity: Multiple Forms, Functions, and Mechanisms
Citri, A., & Malenka, R. C. (2008) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301559 · Cited by 2,323 · cited 1×
“It is however known that there are various chemical cascade processes taking place in synapses that affect synaptic plasticity ( Citri and Malenka, 2008 ; Kotaleski and Blackwell, 2010 ).”
BackgroundRELATIVE AND ABSOLUTE STRENGTH OF RESPONSE AS A FUNCTION OF FREQUENCY OF REINFORCEMENT 1, 2
Herrnstein, R. J. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-267 · Cited by 2,281
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 5×
“We also related our computation to the notions of unexpected and expected uncertainties, which have been suggested to be correlated with NE and Acetylcholine (Ach) release, respectively ( Yu and Dayan, 2005 ).”
BackgroundInfluentialThe importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 2×
“In fact, it has been shown that introducing neurons that are randomly connected to neurons in the decision making network can solve context dependent decision-making tasks ( Rigotti et al., 2010 ; Barak et al., 2013 ).”
BackgroundProbabilistic decision making by slow reverberation in cortical circuits
Wang, X.-J. (2002) · Neuron · doi:10.1016/s0896-6273(02)01092-9 · Cited by 1,242
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“…well known that the phasic activity of dopaminergic neurons reflects a reward prediction error ( Schultz et al., 1997 ), while tonic dopamine levels may reflect reward rates ( Niv et al., 2007 ); these signals could also play crucial roles in our multiple timescales of reward integration process.”
Background
Cited by
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
Adaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286
Iigaya, K., Fonseca, M. S., Murakami, M., Mainen, Z. F., & Dayan, P. (2018) · Nature Communications · doi:10.1038/s41467-018-04840-2 · Cited by 120
Self-citationBalancing new against old information: the role of puzzlement surprise in learning
Faraji, M., Preuschoff, K., & Gerstner, W. (2018) · Neural Computation · doi:10.1162/neco_a_01025 · Cited by 68
“…Li, Daw, & Schoenbaumn, 2012), risk prediction errors (Preuschoff & Bossaerts, 2007; Preuschoff, Quartz, & Bossaerts, 2008), uncertainty-based jump detection (Nassar, Wilson, Heasly, & Gold, 2010; Payzan-Nestour & Bossaerts, 2011), or multi-timescale reward estimator comparison (Iigaya, 2016).”
Metadata
- Authors
- Iigaya, K.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2016-08-09
- Volume
- 5
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2514548734
- Cited by
- 43
- References
- 31
- FWCI
- 3.13
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system. https://app.cochat.ai/papers/10.7554/elife.18073
- The paper
- Iigaya, K. (2016). Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system. eLife. https://doi.org/10.7554/elife.18073
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.