Overlapping Prediction Errors in Dorsal Striatum During Instrumental Learning With Juice and Money Reward in the Human Brain
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Prediction error signals have been reported in human imaging studies in target areas of dopamine neurons such as ventral and dorsal striatum during learning with many different types of reinforcers. However, a key question that has yet to be addressed is whether prediction error signals recruit distinct or overlapping regions of striatum and elsewhere during learning with different types of reward. To address this, we scanned 17 healthy subjects with functional magnetic resonance imaging while they chose actions to obtain either a pleasant juice reward (1 ml apple juice), or a monetary gain (5 cents) and applied a computational reinforcement learning model to subjects' behavioral and imaging data. Evidence for an overlapping prediction error signal during learning with juice and money rewards was found in a region of dorsal striatum (caudate nucleus), while prediction error signals in a subregion of ventral striatum were significantly stronger during learning with money but not juice reward. These results provide evidence for partially overlapping reward prediction signals for different types of appetitive reinforcers within the striatum, a finding with important implications for understanding the nature of associative encoding in the striatum as a function of reinforcer type.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“These neurons demonstrate a response profile that resembles 58 the difference between expected and actual rewards associated with the presentation of 59 particular stimuli and/or performance of specific actions (Mirenowicz and Schultz 1994; 60 Montague et al. 1996; Roesch et al. 2007; Schultz 1998).”
BackgroundDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationActions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
Neural Economics and the Biological Substrates of Valuation
Montague, P. R., & Berns, G. S. (2002) · Neuron · doi:10.1016/s0896-6273(02)00974-1 · Cited by 994
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 4×
“…stimulus-outcome associations in 423 humans, as prediction error signals in this area have been reported during studies of 424 appetitive classical conditioning with a range of different reinforcer types (O’Doherty et 425 al., 2003; McClure et al., 2003; Kim et al., 2006; Bray and O’Doherty, 2007).”
BackgroundInfluentialOptimized EPI for fMRI studies of the orbitofrontal cortex
Deichmann, R., Gottfried, J. A., Hutton, C., & Turner, R. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00073-9 · Cited by 821
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
Self-citationPrediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004) · Nature Neuroscience · doi:10.1038/nn1279 · Cited by 779 · cited 2×
“51 52 3 Introduction 53 54 Accumulating evidence from neurophysiological recording studies in nonhuman primates 55 and rodents implicates the phasic responses of dopamine neurons in encoding 56 prediction errors for reward (Mirenowicz and Schultz 1994; Morris et al. 2006; Roesch et 57 al.”
BackgroundRoesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 569
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · cited 1×
“51 52 3 Introduction 53 54 Accumulating evidence from neurophysiological recording studies in nonhuman primates 55 and rodents implicates the phasic responses of dopamine neurons in encoding 56 prediction errors for reward (Mirenowicz and Schultz 1994; Morris et al. 2006; Roesch et 57 al.”
BackgroundHaruno, M., & Kawato, M. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00382.2005 · Cited by 433
Prediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376
O'Doherty, J. P., Buchanan, T. W., Seymour, B., & Dolan, R. J. (2006) · Neuron · doi:10.1016/j.neuron.2005.11.014 · Cited by 322
Self-citation
Cited by
Bartra, O., McGuire, J. T., & Kable, J. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.063 · Cited by 1,927
“The present investigation extends previous work by examining all four of the above questions simultaneously across the entire brain in a large corpus of published studies.”
Anatomically distinct dopamine release during anticipation and experience of peak emotion to music
Salimpoor, V. N., Benovoy, M., Larcher, K., Dagher, A., & Zatorre, R. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2726 · Cited by 1,828
Gujar, N., Yoo, S.-S., Hu, P., & Walker, M. P. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3220-10.2011 · Cited by 491
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 438
Levy, D. J., & Glimcher, P. W. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.2218-11.2011 · Cited by 358
Metadata
- Authors
- Valentin, V. V., & O'Doherty, J. P.
- Venue
- Journal of Neurophysiology
- Publisher
- American Physiological Society
- Published
- 2009-12
- Volume
- 102
- Issue
- 6
- Pages
- 3384-3391
- ISSN
- 0022-3077 (ISSN-L)
- 1522-1598
- OpenAlex
- W1968266853
- Cited by
- 117
- References
- 19
- FWCI
- 4.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.