Subcortical Substrates of Explore-Exploit Decisions in Primates
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Summary The explore-exploit dilemma refers to the challenge of deciding when to forego immediate rewards and explore new opportunities that could lead to greater rewards in the future. While motivational neural circuits facilitate reinforcement learning based on past choices and outcomes, it is unclear if they also support computations relevant for deciding when to explore. We recorded neural activity in the amygdala and ventral striatum of rhesus macaques, as they solved a task that required them to balance novelty driven exploration with exploitation of what they had already learned. Using a partially-bserved Markov decision process model to quantify explore-exploit trade-offs, we identified that the ventral striatum and amygdala differ in how they represent the immediate value of exploitative choices and the future value of exploratory choices. These findings show that subcortical motivational circuits are important in guiding exploratory decisions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“There has been considerable focus on the striatum and dopamine system in RL (Costa et al., 2015; O'Doherty et al., 2004; Parker et al., 2016; Schultz, 2015).”
BackgroundPreliminary analysis of functions of the temporal lobes in monkeys
KLÜVER, H. (1939) · Archives of Neurology And Psychiatry · doi:10.1001/archneurpsyc.1939.02270240017001 · Cited by 1,696 · cited 1×
“, 2008) and monkeys with amygdala lesions more readily approach novel stimuli (Burns et al., 1996; Kluver and Bucy, 1939; Mason et al., 2006).”
BackgroundThe importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 1×
“, 2018), similar to coding schemes in cortical areas (Rigotti et al., 2013).”
BackgroundOlejnik, S., & Algina, J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.434 · Cited by 1,622
Bandit processes and dynamic allocation indices
Gittins, J. C. (1979) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1979.tb01068.x · Cited by 1,487 · cited 1×
“A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript (RL) and while theoretical solutions exist (Averbeck, 2015; Gittins, 1979; Wilson et al., 2014a), their neural implementation has received limited attention (Cohen et al., 2007; Daw et al., 2006; Frank et al., 2009).”
BackgroundNeural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Cited by
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
The anterior cingulate cortex directs exploration of alternative strategies
Tervo, D. G. R., Kuleshova, E., Manakov, M., Proskurin, M., Karlsson, M., Lustig, A., Behnam, R., & Karpova, A. Y. (2021) · Neuron · doi:10.1016/j.neuron.2021.03.028 · Cited by 97
Trudel, N., Scholl, J., Klein-Flügge, M. C., Fouragnan, E., Tankelevitch, L., Wittmann, M. K., & Rushworth, M. F. S. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0929-3 · Cited by 74
Curiosity: primate neural circuits for novelty and information seeking
Monosov, I. E. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00784-9 · Cited by 44
“…context-dependent modulation of value by novelty afforded by this architecture can help to solve some forms of exploration– exploitation dilemmas 3,12,19,62,73 , particularly in contexts in which rewards are sparse, by allowing for flexible boosting (or bonusing) of RPEs by novelty to modulate…”
Metadata
- Authors
- Costa, V. D., Mitz, A. R., & Averbeck, B. B.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2019-08
- Volume
- 103
- Issue
- 3
- Pages
- 533-545.e5
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2952908749
- Cited by
- 133
- References
- 32
- FWCI
- 7.84
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Subcortical Substrates of Explore-Exploit Decisions in Primates. https://app.cochat.ai/papers/10.1016/j.neuron.2019.05.017
- The paper
- Costa, V. D., Mitz, A. R., & Averbeck, B. B. (2019). Subcortical Substrates of Explore-Exploit Decisions in Primates. Neuron. https://doi.org/10.1016/j.neuron.2019.05.017
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.