Neural Mechanisms of Updating under Reducible and Irreducible Uncertainty
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Adaptive decision making depends on an agent9s ability to use environmental signals to reduce uncertainty. However, because of multiple types of uncertainty, agents must take into account not only the extent to which signals violate prior expectations but also whether uncertainty can be reduced in the first place. Here we studied how human brains of both sexes respond to signals under conditions of reducible and irreducible uncertainty. We show behaviorally that subjects9 value updating was sensitive to the reducibility of uncertainty, and could be quantitatively characterized by a Bayesian model where agents ignore expectancy violations that do not update beliefs or values. Using fMRI, we found that neural processes underlying belief and value updating were separable from responses to expectancy violation, and that reducibility of uncertainty in value modulated connections from belief-updating regions to value-updating regions. Together, these results provide insights into how agents use knowledge about uncertainty to make better decisions while ignoring mere expectancy violation. SIGNIFICANCE STATEMENT To make good decisions, a person must observe the environment carefully, and use these observations to reduce uncertainty about consequences of actions. Importantly, uncertainty should not be reduced purely based on how surprising the observations are, particularly because in some cases uncertainty is not reducible. Here we show that the human brain indeed reduces uncertainty adaptively by taking into account the nature of uncertainty and ignoring mere surprise. Behaviorally, we show that human subjects reduce uncertainty in a quasioptimal Bayesian manner. Using fMRI, we characterize brain regions that may be involved in uncertainty reduction, as well as the network they constitute, and dissociate them from brain regions that respond to mere surprise.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
The reorienting system of the human brain: from environment to theory of mind
Corbetta, M., Patel, G., & Shulman, G. L. (2008) · Neuron · doi:10.1016/j.neuron.2008.04.017 · Cited by 3,889 · cited 2×
“This localization of expectancy violation is consistent with previous reports that the AI responds to salient events in various domains (Corbetta et al., 2008; Singer et al., 2009; Menon and Uddin, 2010).”
ResultPearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 1×
“Distinction between uncertainty reduction and expectancy violation is not clear in some traditionally prevalent frameworks, such as RL and Pearce-Hall (Rescorla and Wagner, 1972; Pearce and Hall, 1980; Sutton and Barto, 1998; Pearce and Bouton, 2001; Roesch et al., 2012).”
BackgroundCircular analysis in systems neuroscience: the dangers of double dipping
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2009) · Nature Neuroscience · doi:10.1038/nn.2303 · Cited by 2,696
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Bayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004
Bartra, O., McGuire, J. T., & Kable, J. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.063 · Cited by 1,927 · cited 1×
“These regions have been long associated with value-related processing, such as valuation of choices or RL (Bartra et al., 2013).”
BackgroundRecent developments in modeling preferences: Uncertainty and ambiguity
Camerer, C., & Weber, M. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122575 · Cited by 1,795 · cited 3×
“Second, decision theory has long emphasized a distinction between reducible and irreducible uncertainty, often referred to as ambiguity and risk respectively (Keynes, 1921; Knight, 1921; Ellsberg, 1961; Camerer and Weber, 1992).”
BackgroundMethodUncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“It is an open question how agents estimate the reducibility of uncertainty, particularly in nonstationary environments (Yu and Dayan, 2005; Behrens et al., 2007; Payzan-LeNestour and Bossaerts, 2011).”
BackgroundBayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423
Neural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411
Self-citationThe Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390 · cited 3×
“We speculate that connections from premotor/FEF and the IPS to the MPFC could be mediated by anterior–posterior connections within cingulate cortex (Margulies et al., 2007) or by corticostriatal connections (Balleine et al., 2007; Di Martino et al., 2008).”
BackgroundA common role of insula in feelings, empathy and uncertainty
Singer, T., Critchley, H. D., & Preuschoff, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.05.001 · Cited by 1,270
Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations
Woo, C.-W., Krishnan, A., & Wager, T. D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.12.058 · Cited by 1,253 · cited 1×
“…clusters defined by voxel-level threshold p 0.001 (uncorrected), cluster-level p values with whole-brain correction for familywise error (FWE) were calculated using nonparametric permutation in SnPM13 package (Nichols and Holmes, 2002; Hayasaka and Nichols, 2003; Woo et al., 2014; RRID:SCR_002092).”
MethodInfluentialBayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Functional connectivity of human striatum: a resting state FMRI study
Di Martino, A., Scheres, A., Margulies, D. S., Kelly, A. M. C., Uddin, L. Q., Shehzad, Z., Biswal, B., Walters, J. R., Castellanos, F. X., & Milham, M. P. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn041 · Cited by 1,090
Higher nervous functions: the orienting reflex
Sokolov, E. N. (1963) · Annual Review of Physiology · doi:10.1146/annurev.ph.25.030163.002553 · Cited by 1,026
Stimulus novelty, task relevance and the visual evoked potential in man
Courchesne, E., Hillyard, S. A., & Galambos, R. (1975) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(75)90003-6 · Cited by 1,012
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925 · cited 3×
“There is growing evidence that the human brain constructs and makes use of beliefs about the external world (Behrens et al., 2007, 2008; Gläscher et al., 2010; Daw et al., 2011), and that it represents risk and ambiguity (Hsu et al., 2005; Huettel et al., 2006; Bach et al., 2011).”
BackgroundBeckmann, M., Johansen-Berg, H., & Rushworth, M. F. S. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3328-08.2009 · Cited by 789 · cited 2×
“Although previous studies have found anatomical connections from premotor/FEF and the IPS to the cingulate motor area, connections to the more anterior portion of the MPFC, where our value-updating ROIs are located, have not been reported (Tomassini et al., 2007; Beckmann et al., 2009; Mars et al., 2011; Eradath et al…”
BackgroundResultComparing families of dynamic causal models
Penny, W. D., Stephan, K. E., Daunizeau, J., Rosa, M. J., Friston, K. J., Schofield, T. M., & Leff, A. P. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000709 · Cited by 688 · cited 3×
“…procedure takes into account both goodness of fit and model complexity, aggregates performance across all models in each family, and calculates exceedance probability, i.e., probability in which each family was better than the other two families (Stephan et al., 2009; Penny et al., 2010).”
BackgroundResultProbabilistic brains: knowns and unknowns
Pouget, A., Beck, J. M., Ma, W. J., & Latham, P. E. (2013) · Nature Neuroscience · doi:10.1038/nn.3495 · Cited by 615 · cited 2×
“Representation of uncertainty has been shown in Bayesian sensory and sensorimotor literature (Ernst and Banks, 2002; Körding and Wolpert, 2004; Pouget et al., 2013; Ma and Jazayeri, 2014), and our results suggest that such representation also plays a key role in the processes that link perception of signals to valuati…”
BackgroundValidating cluster size inference: random field and permutation methods
Hayasaka, S. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.08.003 · Cited by 491
Neural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290 · cited 4×
“Such agents may possess representation of uncertainty, by which uncertainty reduction could be decoupled from expectancy violation (Behrens et al., 2007; Itti and Baldi, 2009; Nassar et al., 2010; Payzan-LeNestour and Bossaerts, 2011; O’Reilly et al., 2013; Ma and Jazayeri, 2014).”
BackgroundInfluentialOrthogonalization of regressors in fMRI models
Mumford, J. A., Poline, J.-B., & Poldrack, R. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0126255 · Cited by 276 · cited 2×
“To adjust for the correlation between belief updating and expectancy violation, we included both parametric modulators in the GLMs so that their coefficients only captured variance uniquely explained by each of them, not the shared variance (Mumford et al., 2015).”
MethodRisk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267 · cited 3×
“Such agents may possess representation of uncertainty, by which uncertainty reduction could be decoupled from expectancy violation (Behrens et al., 2007; Itti and Baldi, 2009; Nassar et al., 2010; Payzan-LeNestour and Bossaerts, 2011; O’Reilly et al., 2013; Ma and Jazayeri, 2014).”
BackgroundTask and Content Modulate Amygdala-Hippocampal Connectivity in Emotional Retrieval
Smith, A. P. R., Stephan, K. E., Rugg, M. D., & Dolan, R. J. (2006) · Neuron · doi:10.1016/j.neuron.2005.12.025 · Cited by 252
Cited by
Levy, I., & Schiller, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.11.007 · Cited by 95
Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty
Sandhu, T. R., Xiao, B., & Lawson, R. P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105123 · Cited by 51
Noradrenergic and Cholinergic Modulation of Belief Updating
Jepma, M., Brown, S. B. R. E., Murphy, P. R., Koelewijn, S. C., de Vries, B., van den Maagdenberg, A. M., & Nieuwenhuis, S. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01317 · Cited by 39
“However, these two types of expected uncertainty differently affect learning rate and are represented in partially different brain regions (Kobayashi & Hsu, 2017; Payzan-LeNestour et al., 2013).”
BackgroundNeural and psychophysiological markers of intolerance of uncertainty.
Morriss, J., Abend, R., Zika, O., Bradford, D. E., & Mertens, G. (2023) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2023.01.003 · Cited by 38
Metadata
- Authors
- Kobayashi, K., & Hsu, M.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2017-06-16
- Volume
- 37
- Issue
- 29
- Pages
- 6972-6982
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2625431615
- Cited by
- 52
- References
- 29
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.