Uncertainty and Learning
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
It is a commonplace in statistics that uncertainty about parameters drives learning. Indeed one of the most influential models of behavioural learning has uncertainty at its heart. However, many popular theoretical models of learning focus exclusively on error, and ignore uncertainty. Here we review the links between learning and uncertainty from three perspectives: statistical theories such as the Kalman filter, psychological models in which differential attention is paid to stimuli with an effect on the speed of learning associated with those stimuli, and neurobiological data on the influence of the neuromodulators acetylcholine and norepinephrine on learning and inference.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“This theory complements those of the way that two of the other major neuromodulators (dopamine [ 12 , 13] and serotonin [31]) interact to specify the prediction error component for learning.”
BackgroundSelf-citationToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 1×
“Rescorla & Wagner [8] described the best known model of this, which was shown [ 9 ] to be equivalent to the well-known delta rule from engineering [10], and has formed the basis of much subsequent theoretical work on accounts of both psychological [11] and neural [12n14] phenomena.”
BackgroundNeuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427
Role of locus coeruleus in attention and behavioral flexibility
Aston-Jones, G., Rajkowski, J., & Cohen, J. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(99)00140-7 · Cited by 900
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855
Self-citationOpponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Self-citationMetalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Amygdala circuitry in attentional and representational processes
Holland, P. C., & Gallagher, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01271-6 · Cited by 618
Differential regulation of neocortical synapses by neuromodulators and activity
Gil, Z., Connors, B. W., & Amitai, Y. (1997) · Neuron · doi:10.1016/s0896-6273(00)80380-3 · Cited by 553 · cited 2×
“In this model, as in the electrophysiological data, ACh comparatively boosts bottom-up input to the cortex (via nicotinic receptors [ 47 ]) and suppresses recurrent and top-down input (via muscarinic receptors [48]).”
BackgroundMethodNeuromodulatory transmitter systems in the cortex and their role in cortical plasticity
Gu, Q. (2002) · Neuroscience · doi:10.1016/s0306-4522(02)00026-x · Cited by 518
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495 · cited 4×
“There is not a great deal of evidence about the synaptic representation of the uncertainties 2 1 NtO; 2 2 NtO; and certainly nothing to support the suggestion [ 28 ] of the involvement of the nucleus accumbens in the competition between dierent stimuli to take responsibility for any prediction error.”
BackgroundMethodInfluentialSelf-citationNoradrenergic Suppression of Synaptic Transmission May Influence Cortical Signal-to-Noise Ratio
Hasselmo, M. E., Linster, C., Patil, M., Ma, D., & Cekic, M. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.77.6.3326 · Cited by 229 · cited 1×
“Contrary to this, there is evidence that NE and ACh have very similar eects on cortical processing in promoting stimulus-evoked activities at the expense of top-down and recurrent processing [ 53 , 54], as well as some less well-understood mutual interactions.”
BackgroundHsieh, C. Y., Cruikshank, S. J., & Metherate, R. (2000) · Brain Research · doi:10.1016/s0006-8993(00)02766-9 · Cited by 165
Kimura, F., Fukuda, M., & Tsumoto, T. (1999) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.1999.00779.x · Cited by 165 · cited 1×
“In this model, as in the electrophysiological data, ACh comparatively boosts bottom-up input to the cortex (via nicotinic receptors [47]) and suppresses recurrent and top-down input (via muscarinic receptors [ 48 ]).”
MethodSelective suppression of horizontal propagation in rat visual cortex by norepinephrine
Kobayashi, M., Imamura, K., Sugai, T., Onoda, N., Yamamoto, M., Komai, S., & Watanabe, Y. (2000) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2000.00917.x · Cited by 63 · cited 1×
“Contrary to this, there is evidence that NE and ACh have very similar eects on cortical processing in promoting stimulus-evoked activities at the expense of top-down and recurrent processing [53, 54 ], as well as some less well-understood mutual interactions.”
Background
Cited by
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774
Self-citationGreat Expectations: Is there Evidence for Predictive Coding in Auditory Cortex?
Heilbron, M., & Chait, M. (2018) · Neuroscience · doi:10.1016/j.neuroscience.2017.07.061 · Cited by 428
A unified model for perceptual learning
SEITZ, A., & WATANABE, T. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.05.010 · Cited by 356
Phasic norepinephrine: A neural interrupt signal for unexpected events
Dayan, P., & Yu, A. J. (2006) · Network: Computation in Neural Systems · doi:10.1080/09548980601004024 · Cited by 299
Self-citationTemporal Differences in the Role of Marketing Communication in New Product Categories
Narayanan, S., Manchanda, P., & Chintagunta, P. K. (2005) · Journal of Marketing Research · doi:10.1509/jmkr.2005.42.3.278 · Cited by 254
Metadata
- Authors
- Dayan, P., & Jyu, A.
- Venue
- IETE Journal of Research
- Publisher
- Informa UK Limited
- Published
- 2003-03
- Volume
- 49
- Issue
- 2-3
- Pages
- 171-181
- ISSN
- 0377-2063 (ISSN-L)
- 0974-780X
- OpenAlex
- W1965274432
- Cited by
- 105
- References
- 19
- FWCI
- 2.10
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.