The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Open accessAbstract
The reward value of a stimulus can be learned through two distinct mechanisms: reinforcement learning through repeated stimulus-reward pairings and abstract inference based on knowledge of the task at hand. The reinforcement mechanism is often identified with midbrain dopamine neurons. Here we show that a neural pathway controlling the dopamine system does not rely exclusively on either stimulus-reward pairings or abstract inference but instead uses a combination of the two. We trained monkeys to perform a reward-biased saccade task in which the reward values of two saccade targets were related in a systematic manner. Animals used each trial's reward outcome to learn the values of both targets: the target that had been presented and whose reward outcome had been experienced (experienced value) and the target that had not been presented but whose value could be inferred from the reward statistics of the task (inferred value). We then recorded from three populations of reward-coding neurons: substantia nigra dopamine neurons; a major input to dopamine neurons, the lateral habenula; and neurons that project to the lateral habenula, located in the globus pallidus. All three populations encoded both experienced values and inferred values. In some animals, neurons encoded experienced values more strongly than inferred values, and the animals showed behavioral evidence of learning faster from experience than from inference. Our data indicate that the pallidus-habenula-dopamine pathway signals reward values estimated through both experience and inference.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 2×
“An influential distinction is between two learning mechanisms: reinforcement learning by repeated stimulus-reward pairings (Bayley et al. 2005; Knowlton et al. 1996; Wise 2004) and abstract inference using task-specific rules (Daw et al. 2005; Dayan and Niv 2008; Hampton et al. 2006).”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378
Self-citationLateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
Self-citationHampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
Ji, H., & Shepard, P. D. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.0958-07.2007 · Cited by 388
The globus pallidus sends reward-related signals to the lateral habenula
Hong, S., & Hikosaka, O. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.035 · Cited by 346
Self-citation
Cited by
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
Self-citationModel-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
“Following Bromberg-Martin et al.(18), we trained our agent on a reversal task, in which the values of the two stimulus cues were anticorrelated, with one being rewarded and the other not rewarded.”
MethodInfluentialNeural Basis of Reinforcement Learning and Decision Making
Lee, D., Seo, H., & Jung, M. W. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150512 · Cited by 546
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 438
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
The place of dopamine in the cortico-basal ganglia circuit
Haber, S. N. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.10.008 · Cited by 364
“The main inputs to the LHb in primates are from forebrain areas; the lateral hypothalamus, the perifornical area, and from the GPi and the VP (Parent et al., 1981; Haber et al., 1993; Hong and Hikosaka, 2008; Bromberg-Martin et al., 2010, {Bromberg-Martin, 2010 #13259}).”
The ventral pallidum: Subregion-specific functional anatomy and roles in motivated behaviors
Root, D. H., Melendez, R. I., Zaborszky, L., & Napier, T. C. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.03.005 · Cited by 353
Single rodent mesohabenular axons release glutamate and GABA
Root, D. H., Mejias-Aponte, C. A., Zhang, S., Wang, H.-L., Hoffman, A. F., Lupica, C. R., & Morales, M. (2014) · Nature Neuroscience · doi:10.1038/nn.3823 · Cited by 349
Dopamine transients are sufficient and necessary for acquisition of model-based associations
Sharpe, M. J., Chang, C. Y., Liu, M. A., Batchelor, H. M., Mueller, L. E., Jones, J. L., Niv, Y., & Schoenbaum, G. (2017) · Nature Neuroscience · doi:10.1038/nn.4538 · Cited by 300
Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236
Dopamine Neuron-Specific Optogenetic Stimulation in Rhesus Macaques
Stauffer, W. R., Lak, A., Yang, A., Borel, M., Paulsen, O., Boyden, E. S., & Schultz, W. (2016) · Cell · doi:10.1016/j.cell.2016.08.024 · Cited by 257
Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170
Dopamine neurons respond to errors in the prediction of sensory features of expected rewards
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.025 · Cited by 200
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183
Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement
Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. S. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08023.x · Cited by 169
Humans primarily use model-based inference in the two-stage task
Feher da Silva, C., & Hare, T. A. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0905-y · Cited by 153
Generalization of value in reinforcement learning by humans
Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08017.x · Cited by 111
“options in a serial reversal task (Bromberg-Martin et al., 2010).”
BackgroundResultInfluentialBelief state representation in the dopamine system
Babayan, B. M., Uchida, N., & Gershman, S. J. (2018) · Nature Communications · doi:10.1038/s41467-018-04397-0 · Cited by 101
Dopamine neuron ensembles signal the content of sensory prediction errors
Stalnaker, T. A., Howard, J. D., Takahashi, Y. K., Gershman, S. J., Kahnt, T., & Schoenbaum, G. (2019) · eLife · doi:10.7554/elife.49315 · Cited by 74
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
McGinty, V. B., Hayden, B. Y., Heilbronner, S. R., Dumont, E. C., Graves, S. M., Mirrione, M. M., du Hoffmann, J., Sartor, G. C., España, R. A., Millan, E. Z., et al. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2011.07.036 · Cited by 36
“Finally, a pair of recent studies n nonhuman primates has suggested that the VP may contribute to eward prediction error signaling via its input to the LHb [35,43].”
Background
Metadata
- Authors
- Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O.
- Venue
- Journal of Neurophysiology
- Publisher
- American Physiological Society
- Published
- 2010-08
- Volume
- 104
- Issue
- 2
- Pages
- 1068-1076
- ISSN
- 0022-3077 (ISSN-L)
- 1522-1598
- OpenAlex
- W2170171432
- Cited by
- 186
- References
- 13
- FWCI
- 3.75
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.