How instructed knowledge modulates the neural systems of reward learning
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Recent research in neuroeconomics has demonstrated that the reinforcement learning model of reward learning captures the patterns of both behavioral performance and neural responses during a range of economic decision-making tasks. However, this powerful theoretical model has its limits. Trial-and-error is only one of the means by which individuals can learn the value associated with different decision options. Humans have also developed efficient, symbolic means of communication for learning without the necessity for committing multiple errors across trials. In the present study, we observed that instructed knowledge of cue-reward probabilities improves behavioral performance and diminishes reinforcement learning-related blood-oxygen level-dependent (BOLD) responses to feedback in the nucleus accumbens, ventromedial prefrontal cortex, and hippocampal complex. The decrease in BOLD responses in these brain regions to reward-feedback signals was functionally correlated with activation of the dorsolateral prefrontal cortex (DLPFC). These results suggest that when learning action values, participants use the DLPFC to dynamically adjust outcome responses in valuation regions depending on the usefulness of action-outcome information.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
The cognitive control of emotion
OCHSNER, K., & GROSS, J. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.03.010 · Cited by 4,495
The neural basis of economic decision-making in the Ultimatum Game.
Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003) · Science · doi:10.1126/science.1082976 · Cited by 3,375
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 2×
“Although the current study demonstrates the importance of this DLPFC, reward-related structure circuitry for learning and reward processing, previous neuroeconomic research has outlined a similar circuitry across a range of decision-making tasks in which preexisting reward values that are represented in valuation regi…”
BackgroundResultDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 2×
“The brainarbitrates betweendifferent representation systemsaccording to theuncertainty estimated fromeach system(38, 39, 55).”
BackgroundA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Abstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
Self-Control in Decision-Making Involves Modulation of the vmPFC Valuation System
Hare, T. A., Camerer, C. F., & Rangel, A. (2009) · Science · doi:10.1126/science.1168450 · Cited by 1,928 · cited 2×
“Although the current study demonstrates the importance of this DLPFC, reward-related structure circuitry for learning and reward processing, previous neuroeconomic research has outlined a similar circuitry across a range of decision-making tasks in which preexisting reward values that are represented in valuation regi…”
BackgroundCognitive neuroscience of emotional memory
LaBar, K. S., & Cabeza, R. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1825 · Cited by 1,834
The Neural Basis of Altruistic Punishment
de Quervain, D. J.-F., Fischbacher, U., Treyer, V., Schellhammer, M., Schnyder, U., Buck, A., & Fehr, E. (2004) · Science · doi:10.1126/science.1100735 · Cited by 1,768
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Getting to Know You: Reputation and Trust in a Two-Person Economic Exchange
King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., & Montague, P. R. (2005) · Science · doi:10.1126/science.1108062 · Cited by 1,345
Dissociation of reward anticipation and outcome with event-related fMRI
Knutson, B., Fong, G. W., Adams, C. M., Varner, J. L., & Hommer, D. (2001) · Neuroreport · doi:10.1097/00001756-200112040-00016 · Cited by 1,300
The Neural Basis of Financial Risk Taking
Kuhnen, C. M., & Knutson, B. (2005) · Neuron · doi:10.1016/j.neuron.2005.08.008 · Cited by 1,182
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116
Self-citationBeautiful Faces Have Variable Reward Value
Aharon, I., Etcoff, N., Ariely, D., Chabris, C. F., O'Connor, E., & Breiter, H. C. (2001) · Neuron · doi:10.1016/s0896-6273(01)00491-3 · Cited by 1,087
Dissociated neural representations of intensity and valence in human olfaction
Anderson, A. K., Christoff, K., Stappen, I., Panitz, D., Ghahremani, D. G., Glover, G., Gabrieli, J. D. E., & Sobel, N. (2003) · Nature Neuroscience · doi:10.1038/nn1001 · Cited by 1,054
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
Reward‐Related Responses in the Human Striatum
DELGADO, M. R. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.002 · Cited by 736
Self-citationMotivation and cognitive control in the human prefrontal cortex
Kouneiher, F., Charron, S., & Koechlin, E. (2009) · Nature Neuroscience · doi:10.1038/nn.2321 · Cited by 590
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · cited 2×
“The brainarbitrates betweendifferent representation systemsaccording to theuncertainty estimated fromeach system(38, 39, 55).”
BackgroundSchönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Frontal cortex subregions play distinct roles in choices between actions and stimuli
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3541-08.2008 · Cited by 350
Regulating the expectation of reward via cognitive strategies
Delgado, M. R., Gillis, M. M., & Phelps, E. A. (2008) · Nature Neuroscience · doi:10.1038/nn.2141 · Cited by 215 · cited 1×
“Although the current study demonstrates the importance of this DLPFC, reward-related structure circuitry for learning and reward processing, previous neuroeconomic research has outlined a similar circuitry across a range of decision-making tasks in which preexisting reward values that are represented in valuation regi…”
BackgroundSelf-citationResisting the Power of Temptations: The Right Prefrontal Cortex and Self‐Control
KNOCH, D., & FEHR, E. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.004 · Cited by 208
Cited by
Mechanisms of Social Cognition
Frith, C. D., & Frith, U. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100449 · Cited by 973
“…(Cunningham et al. 2004) Overcoming response to unfairness (Kirk et al. 2011) Overriding trial-and-error learning of reputation by instructed knowledge (Li et al. 2011) dlPFC and ACC Managing conflicting information about emotional states (Zaki et al. 2010) (Continued ) 290 Frith · Frith A nn u.”
BackgroundThe neuroscience of placebo effects: connecting context, learning and health
Wager, T. D., & Atlas, L. Y. (2015) · Nature Reviews Neuroscience · doi:10.1038/nrn3976 · Cited by 844
Emotion and the prefrontal cortex: an integrative review
Dixon, M. L., Thiruchselvam, R., Todd, R., & Christoff, K. (2017) · Psychological Bulletin · doi:10.1037/bul0000096 · Cited by 745
Action, Outcome, and Value: A Dual-System Framework for Morality
Cushman, F. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313495594 · Cited by 473
Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies
Garrison, J., Erdeniz, B., & Done, J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.023 · Cited by 438
Metadata
- Authors
- Li, J., Delgado, M. R., & Phelps, E. A.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2010-12-20
- Volume
- 108
- Issue
- 1
- Pages
- 55-60
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W1990247995
- Cited by
- 155
- References
- 30
- FWCI
- 6.52
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.