Multiple Mechanisms for Processing Reward Uncertainty in the Primate Basal Forebrain
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
UNLABELLED: The ability to use information about the uncertainty of future outcomes is critical for adaptive behavior in an uncertain world. We show that the basal forebrain (BF) contains at least two distinct neural-coding strategies to support this capacity. The dorsal-lateral BF, including the ventral pallidum (VP), contains reward-sensitive neurons, some of which are selectively suppressed by uncertain-reward predictions (U(-)). In contrast, the medial BF (mBF) contains reward-sensitive neurons, some of which are selectively enhanced (U(+)) by uncertain-reward predictions. In a two-alternative choice-task, U(-) neurons were selectively suppressed while monkeys chose uncertain options over certain options. During the same choice-epoch, U(+) neurons signaled the subjective reward value of the choice options. Additionally, after the choice was reported, U(+) neurons signaled reward uncertainty until the choice outcome. We suggest that uncertainty-related suppression of VP may participate in the mediation of uncertainty-seeking actions, whereas uncertainty-related enhancement of the mBF may direct cognitive resources to monitor and learn from uncertain-outcomes. SIGNIFICANCE STATEMENT: To survive in an uncertain world, we must approach uncertainty and learn from it. Here we provide evidence for two mostly distinct mechanisms for processing uncertainty about rewards within different subregions of the primate basal forebrain (BF). We found that uncertainty suppressed the representation of certain (or safe) reward values by some neurons in the dorsal-lateral BF, in regions occupied by the ventral pallidum. This uncertainty-related suppression was evident as monkeys made risky choices. We also found that uncertainty-enhanced the activity of many medial BF neurons, most prominently after the monkeys' choices were completed (as they awaited uncertain outcomes). Based on these findings, we propose that different subregions of the BF could support action and learning under uncertainty in distinct but complimentary manners.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · cited 1×
“…integration of value and uncertainty in BF subregions is supported by monoamine transmission (and other inputs) from the brainstem and hypothalamus (Fiorillo et al., 2003; Watabe-Uchida et al., 2012; Ogawa et al., 2014; Cohen et al., 2015), but more anatomical studies are required to support or…”
BackgroundMesulam, M. ‐., Mufson, E. J., Levey, A. I., & Wainer, B. H. (1983) · Journal of Comparative Neurology · doi:10.1002/cne.902140206 · Cited by 1,962
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“Learning and memory are enhanced by surprising events and uncertain contexts (Pearce and Hall, 1980; Yu and Dayan, 2005; Courville et al., 2006; Esber and Haselgrove, 2011; Le Pelley et al., 2011; Bach and Dolan, 2012) and are thought to be in part dependent on the BF (Damasio et al., 1985; Everitt…”
BackgroundNeural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · cited 1×
“To quantify irregularity of spiking activity, we used the irregularity index developed by Davies et al. (Davies et al., 2006; Nakamura et al., 2008; Matsumoto and Hikosaka, 2009).”
MethodCENTRAL CHOLINERGIC SYSTEMS AND COGNITION
Everitt, B. J., & Robbins, T. W. (1997) · Annual Review of Psychology · doi:10.1146/annurev.psych.48.1.649 · Cited by 1,348 · cited 12×
“…to sensory and/or attention-related cortical areas (Mesulam et al., 1983) and the amygdala (Koliatsos et al., 1988), and has been implicated in salience-driven behavioral control (Everitt and Robbins, 1997; Lin and Nicolelis, 2008; Avila and Lin, 2014; Peck and Salzman, 2014; Hangya et al., 2015).”
BackgroundInfluentialHuman insula activation reflects risk prediction errors as well as risk
Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4286-07.2008 · Cited by 784
Neural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692 · cited 1×
“How these BF subregions interact with other brain areas that are modulated by reward uncertainty (Hsu et al., 2005; McCoy and Platt, 2005; Preuschoff et al., 2006, 2008; Preuschoff and Bossaerts, 2007; O’Neill and Schultz, 2010; Burke and Tobler, 2011a; Schultz et al., 2011; Monosov and Hikosaka,…”
BackgroundKolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
The orbital and medial prefrontal circuit through the primate basal ganglia
Haber, S., Kunishio, K., Mizobuchi, M., & Lynd-Balta, E. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-07-04851.1995 · Cited by 605
Risky business: the neuroeconomics of decision making under uncertainty
Platt, M. L., & Huettel, S. A. (2008) · Nature Neuroscience · doi:10.1038/nn2062 · Cited by 556
Disentangling pleasure from incentive salience and learning signals in brain reward circuitry
Smith, K. S., Berridge, K. C., & Aldridge, J. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1101920108 · Cited by 464 · cited 3×
“As observed in previous studies (Smith et al., 2011; Tachibana and Hikosaka, 2012), these reward value signals persisted until the trial’s outcome (Figs.”
BackgroundResultBayesian theories of conditioning in a changing world
Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.004 · Cited by 457
Central Cholinergic Neurons Are Rapidly Recruited by Reinforcement Feedback
Hangya, B., Ranade, S. P., Lorenc, M., & Kepecs, A. (2015) · Cell · doi:10.1016/j.cell.2015.07.057 · Cited by 399
Serotonergic neurons signal reward and punishment on multiple timescales
Cohen, J. Y., Amoroso, M. W., & Uchida, N. (2015) · eLife · doi:10.7554/elife.06346 · Cited by 388
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379 · cited 1×
“, 1983), regions involved in learning and memory (Suzuki and Clayton, 2000; Suzuki and Eichenbaum, 2000; Suzuki, 2007; Bryden et al., 2011; Hayden et al., 2011).”
BackgroundCognitive functions of the basal forebrain
Baxter, M. G., & Chiba, A. A. (1999) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(99)80024-5 · Cited by 351
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339 · cited 3×
“Uncertainty about important future events, such as rewards, influences a wide range of behavioral states (Loewenstein et al., 2001; Bach and Dolan, 2012).”
BackgroundDopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257 · cited 1×
“6B) (Padoa-Schioppa and Cai, 2011; Monosov and Hikosaka, 2013; Lak et al., 2014; Monosov et al., 2015).”
Organization of monosynaptic inputs to the serotonin and dopamine neuromodulatory systems
Ogawa, S. K., Cohen, J. Y., Hwang, D., Uchida, N., & Watabe-Uchida, M. (2014) · Cell Reports · doi:10.1016/j.celrep.2014.06.042 · Cited by 249 · cited 1×
“…and uncertainty in BF subregions is supported by monoamine transmission (and other inputs) from the brainstem and hypothalamus (Fiorillo et al., 2003; Watabe-Uchida et al., 2012; Ogawa et al., 2014; Cohen et al., 2015), but more anatomical studies are required to support or exclude this hypothesis.”
BackgroundAdding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133 · cited 1×
“…interact with other brain areas that are modulated by reward uncertainty (Hsu et al., 2005; McCoy and Platt, 2005; Preuschoff et al., 2006, 2008; Preuschoff and Bossaerts, 2007; O’Neill and Schultz, 2010; Burke and Tobler, 2011a; Schultz et al., 2011; Monosov and Hikosaka, 2012, 2013; Monosov…”
Background
Cited by
Anterior cingulate is a source of valence-specific information about value and uncertainty
Monosov, I. E. (2017) · Nature Communications · doi:10.1038/s41467-017-00072-y · Cited by 163
Self-citationA neural network for information seeking
White, J. K., Bromberg-Martin, E. S., Heilbronner, S. R., Zhang, K., Pai, J., Haber, S. N., & Monosov, I. E. (2019) · Nature Communications · doi:10.1038/s41467-019-13135-z · Cited by 129
Self-citationAnterior Cingulate Cortex and the Control of Dynamic Behavior in Primates
Monosov, I. E., Haber, S. N., Leuthardt, E. C., & Jezzini, A. (2020) · Current Biology · doi:10.1016/j.cub.2020.10.009 · Cited by 118
“And, that neuromodulators, in particular the basal forebrain, that carry signals about behavioral importance [41] and about the total value of choices or contexts [40], may directly impact how ACC impacts action related processes [139,140], particularly during foraging and/or while under uncertainty.”
BackgroundSelf-citationA neural mechanism for conserved value computations integrating information and rewards
Bromberg-Martin, E. S., Feng, Y.-Y., Ogasawara, T., White, J. K., Zhang, K., & Monosov, I. E. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01511-4 · Cited by 43
Self-citationHuman decisions about when to act originate within a basal forebrain–nigral circuit
Khalighinejad, N., Priestley, L., Jbabdi, S., & Rushworth, M. F. S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1921211117 · Cited by 29
Metadata
- Authors
- Ledbetter, N. M., Chen, C. D., & Monosov, I. E.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2016-07-27
- Volume
- 36
- Issue
- 30
- Pages
- 7852-7864
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2505254434
- Cited by
- 33
- References
- 21
- FWCI
- 1.96
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.