Value and probability coding in a feedback-based learning task utilizing food rewards
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
For the consequences of our actions to guide behavior, the brain must represent different types of outcome-related information. For example, an outcome can be construed as negative because an expected reward was not delivered or because an outcome of low value was delivered. Thus behavioral consequences can differ in terms of the information they provide about outcome probability and value. We investigated the role of the striatum in processing probability-based and value-based negative feedback by training participants to associate cues with food rewards and then employing a selective satiety procedure to devalue one food outcome. Using functional magnetic resonance imaging, we examined brain activity related to receipt of expected rewards, receipt of devalued outcomes, omission of expected rewards, omission of devalued outcomes, and expected omissions of an outcome. Nucleus accumbens activation was greater for rewarding outcomes than devalued outcomes, but activity in this region did not correlate with the probability of reward receipt. Activation of the right caudate and putamen, however, was largest in response to rewarding outcomes relative to expected omissions of reward. The dorsal striatum (caudate and putamen) at the time of feedback also showed a parametric increase correlating with the trialwise probability of reward receipt. Our results suggest that the ventral striatum is sensitive to the motivational relevance, or subjective value, of the outcome, while the dorsal striatum codes for a more complex signal that incorporates reward probability. Value and probability information may be integrated in the dorsal striatum, to facilitate action planning and allocation of effort.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“If there is a continuum with the ventromedial striatum more sensitive to subjective value and the dorsolateral striatum more sensitive to probability, it is possible that since the caudate is in the middle of this continuum we happened to find more evidence for value coding in the left caudate and…”
BackgroundHaber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Neural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165
Functional neuroanatomy of emotions: A meta-analysis
MURPHY, F. C., NIMMO-SMITH, I., & LAWRENCE, A. D. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.3.207 · Cited by 1,135 · cited 1×
“…stronger dopaminergic reward response in the right striatum than the left (Cannon et al. 2009; Martin-Soelch et al. 2011; van Dyck et al. 2002; Vernaleken et al. 2007) and other work suggesting stronger left-lateralized activity supporting approach processes (Murphy et al. 2003; Tomer et al. 2014).”
BackgroundTracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · cited 4×
“…of valence, with increases in activation for rewards, such as monetary gains, and decreases in activation for punishments, such as monetary losses (Coricelli et al. 2005; Delgado et al. 2000, 2003; Yacubian et al. Right Caudate Nucleus Response at Outcome -0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02”
InfluentialDistributed Neural Representation of Expected Value
Knutson, B., Taylor, J., Kaufman, M., Peterson, R., & Glover, G. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0642-05.2005 · Cited by 955
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · cited 1×
“Thus the dorsal striatum may play an important role in cementing the learned relationship between a given action and a desired outcome (Balleine et al. 2009; Haruno and Kawato 2006; Samejima et al. 2005).”
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 1×
“Prediction error responses in the striatum tend to be found in the ventral striatum or the putamen, rather than the caudate (Abler et al. 2006; Hare et al. 2008; McClure et al. 2003; O'Doherty et al. 2003; Yacubian et al. 2006).”
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
Kringelbach, M. L. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.10.1064 · Cited by 824
Optimized EPI for fMRI studies of the orbitofrontal cortex
Deichmann, R., Gottfried, J. A., Hutton, C., & Turner, R. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00073-9 · Cited by 821
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769 · cited 3×
“At the time of cue presentation, activation of the putamen was parametrically modulated by the expected value of the upcoming outcome.”
BackgroundMethodSelf-citationValue, pleasure and choice in the ventral prefrontal cortex
Grabenhorst, F., & Rolls, E. T. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.12.004 · Cited by 766
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642 · cited 3×
“9 VALUE AND PROBABILITY REPRESENTATION 2006).”
InfluentialYacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
The principled control of false positives in neuroimaging
Bennett, C. M., Wolford, G. L., & Miller, M. B. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp053 · Cited by 486 · cited 1×
“As this tool provides information on the type I error rate across the whole brain, it constitutes a principled correction and is compliant with recent recommendations to avoid arbitrary cluster thresholds (Bennett et al. 2009).”
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472 · cited 4×
“, reward magnitude probability) on activation of the dorsal striatum (Hsu et al. 2005; Tobler et al. 2007).”
InfluentialNeural evidence for inequality-averse social preferences
Tricomi, E., Rangel, A., Camerer, C. F., & O’Doherty, J. P. (2010) · Nature · doi:10.1038/nature08785 · Cited by 462 · cited 1×
“…by the subjective value of outcomes based on a host of factors, such as counterfactual comparisons (Breiter et al. 2001), delay in reward delivery (Kable and Glimcher 2007; Peters and Buchel 2009), risk and ambiguity aversion (Levy et al. 2010), and fairness considerations (Tricomi et al. 2010).”
BackgroundSelf-citationYin, H. H., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04219.x · Cited by 447 · cited 2×
“Its activity in humans is modulated by perceived action-outcome contingency (Tricomi et al. 2004; Yin et al. 2005).”
BackgroundHaruno, M., & Kawato, M. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00382.2005 · Cited by 433
Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations
DELGADO, M. R., LOCKE, H. M., STENGER, V. A., & FIEZ, J. A. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.1.27 · Cited by 431
Neural representations of subjective reward value
Peters, J., & Büchel, C. (2010) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.04.031 · Cited by 405 · cited 4×
“The OFC has been shown to track decision value, goal value, and outcome value (de Wit et al. 2009; Hare et al. 2008; Peters and Buchel 2010).”
InfluentialChoice selection and reward anticipation: an fMRI study
Ernst, M., Nelson, E. E., McClure, E. B., Monk, C. S., Munson, S., Eshel, N., Zarahn, E., Leibenluft, E., Zametkin, A., Towbin, K., et al. (2004) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2004.05.011 · Cited by 377
Prediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376 · cited 1×
“Prediction error responses in the striatum tend to be found in the ventral striatum or the putamen, rather than the caudate (Abler et al. 2006; Hare et al. 2008; McClure et al. 2003; O'Doherty et al. 2003; Yacubian et al. 2006).”
The integrative function of the basal ganglia in instrumental conditioning
Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.10.034 · Cited by 356
Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions
Tanaka, S. C., Balleine, B. W., & O'Doherty, J. P. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1808-08.2008 · Cited by 268 · cited 4×
“The caudate has also been shown to be sensitive to action-outcome contingency (Elliott et al. 2004; O’Doherty et al. 2004; Tanaka et al. 2008; Tricomi et al. 2004; Zink et al. 2004).”
InfluentialMotivation-dependent responses in the human caudate nucleus.
Delgado, M. R. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh062 · Cited by 246 · cited 2×
“…coefficients for all conditions in the caudate and nucleus accumbens, which were extracted based on a priori cubic regions of 8 mm 3 , centered on coordinates from previous work (Bischoff-Grethe et al. 2009; Cools et al. 2002; Delgado et al. 2000, 2004; Tricomi et al. 2004; Zink et al. 2004).”
Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216
The role of ventral frontostriatal circuitry in reward-based learning in humans.
Galvan, A., Hare, T. A., Davidson, M., Spicer, J., Glover, G., & Casey, B. J. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.2431-05.2005 · Cited by 207 · cited 1×
“This fits with previous evidence indicating a role for the striatum in anticipation of upcoming rewards (Bjork and Hommer 2007; Ernst et al. 2004; Galvan et al. 2005; Gottfried et al. 2003; Haruno and Kawato 2006; Knutson et al. 2005; Yacubian et al. 2006) and in reinforcement learning processes…”
Resultde Wit, S., Corlett, P. R., Aitken, M. R., Dickinson, A., & Fletcher, P. C. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1639-09.2009 · Cited by 200
Cited by
for the ENIGMA-Major Depressive Disorder Working Group, Schmaal, L., Veltman, D. J., van Erp, T. G. M., Sämann, P. G., Frodl, T., Jahanshad, N., Loehrer, E., Tiemeier, H., Hofman, A., et al. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.69 · Cited by 1,172
Reduced dorsal striatal gray matter volume predicts implicit suicidal ideation in adolescents
Ho, T. C., Cichocki, A. C., Gifuni, A. J., Catalina Camacho, M., Ordaz, S. J., Singh, M. K., & Gotlib, I. H. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy089 · Cited by 48
Intelligence mindset shapes neural learning signals and memory
Bejjani, C., DePasque, S., & Tricomi, E. (2019) · Biological Psychology · doi:10.1016/j.biopsycho.2019.06.003 · Cited by 19
Self-citation
Metadata
- Authors
- Tricomi, E., & Lempert, K. M.
- Venue
- Journal of Neurophysiology
- Publisher
- American Physiological Society
- Published
- 2015-01-01
- Volume
- 113
- Issue
- 1
- Pages
- 4-13
- ISSN
- 0022-3077 (ISSN-L)
- 1522-1598
- OpenAlex
- W2144873902
- Cited by
- 36
- References
- 35
- FWCI
- 1.70
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.