Neural coding of distinct statistical properties of reward information in humans
Abstract
Brain processing of reward information is essential for complex functions such as learning and motivation. Recent primate electrophysiological studies using concepts from information, economic and learning theories indicate that the midbrain may code two statistical parameters of reward information: a transient reward error prediction signal that varies linearly with reward probability and a sustained signal that varies highly non-linearly with reward probability and that is highest with maximal reward uncertainty (reward probability = 0.5). Here, using event-related functional magnetic resonance imaging, we disentangled these two signals in humans using a novel paradigm that systematically varied monetary reward probability, magnitude and expected reward value. The midbrain was activated both transiently with the error prediction signal and in a sustained fashion with reward uncertainty. Moreover, distinct activity dynamics were observed in post-synaptic midbrain projection sites: the prefrontal cortex responded to the transient error prediction signal while the ventral striatum covaried with the sustained reward uncertainty signal. These data suggest that the prefrontal cortex may generate the reward prediction while the ventral striatum may be involved in motivational processes that are useful when an organism needs to obtain more information about its environment. Our results indicate that distinct functional brain networks code different aspects of the statistical properties of reward information in humans.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The VS is consistently associated with all phases of reward processing, including detection, anticipation, and outcome.
“The VS is a region that has been consistently associated with all phases of the processing of rewards, including detection, anticipation, and outcome (Bjork et al., 2004; Dreher et al., 2006; Galvan et al., 2006; Knutson et al., 2001; Schultz et al., 1992)”
From Developmental changes in brain function underlying the influence of reward processing on inhibitory control (2011) · cites this paper as Dreher et al., 2006
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Neurophysiological investigation of the basis of the fMRI signal
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001) · Nature · doi:10.1038/35084005 · Cited by 6,542 · see citing works
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127 · see citing works
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · see citing works
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · see citing works
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
Bayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004 · see citing works
The Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, E., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,922 · see citing works
Neural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · see citing works
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · see citing works
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404 · see citing works
Multiple reward signals in the brain
Schultz, W. (2000) · Nature Reviews Neuroscience · doi:10.1038/35044563 · Cited by 1,303 · see citing works
Object perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269 · see citing works
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076 · see citing works
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · see citing works
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · see citing works
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855 · see citing works
Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning
Schoenbaum, G. M., Chiba, A. A., & Gallagher, M. (1998) · Nature Neuroscience · doi:10.1038/407 · Cited by 849 · see citing works
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711 · see citing works
Choice and the relative pleasure of consequences.
Mellers, B. A. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.6.910 · Cited by 575 · see citing works
Modulation of Caudate Activity by Action Contingency
Tricomi, E. M., Delgado, M. R., & Fiez, J. A. (2004) · Neuron · doi:10.1016/s0896-6273(03)00848-1 · Cited by 575 · see citing works
Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals
Holroyd, C. B., Nieuwenhuis, S., Yeung, N., Nystrom, L. E., Mars, R. B., COLES, M. G. H., & Cohen, J. D. (2004) · Nature Neuroscience · doi:10.1038/nn1238 · Cited by 487 · see citing works
Ito, R., Dalley, J. W., Robbins, T. W., & Everitt, B. J. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-14-06247.2002 · Cited by 463 · see citing works
Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine
Salamone, J. D., Correa, M., Mingote, S., & Weber, S. (2004) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2004.09.004 · Cited by 454 · see citing works
Activity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451 · see citing works
Anatomical Substrates for Glutamate‐Dopamine Interactions
Sesack, S., Carr, D. B., Omelchenko, N., & Pinto, A. (2003) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1300.066 · Cited by 348 · see citing works
Nishijo, H., Ono, T., & Nishino, H. (1988) · Journal of Neuroscience · doi:10.1523/jneurosci.08-10-03570.1988 · Cited by 333 · see citing works
Neuronal activity in monkey striatum related to the expectation of predictable environmental events
Apicella, P., Scarnati, E., Ljungberg, T., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.68.3.945 · Cited by 305 · see citing works
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298 · see citing works
T HE N EUROBIOLOGY OF V ISUAL -S ACCADIC D ECISION M AKING
Glimcher, P. W. (2003) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.010302.081134 · Cited by 291 · see citing works
Monitoring and Control of Action by the Frontal Lobes
Schall, J. D., Stuphorn, V., & Brown, J. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00964-9 · Cited by 280 · see citing works
Human Midbrain Sensitivity to Cognitive Feedback and Uncertainty During Classification Learning
Aron, A. R., Daphna Shohamy, A., Clark, J. A., Myers, C. E., Gluck, M. A., & Poldrack, R. A. (2004) · Journal of Neurophysiology · doi:10.1152/jn.01209.2003 · Cited by 265 · see citing works
Fletcher, P. C., Anderson, J. M., Shanks, D. R., Honey, R., Carpenter, T. A., Donovan, T., Papadakis, N., & Bullmore, E. T. (2001) · Nature Neuroscience · doi:10.1038/nn733 · Cited by 238 · see citing works
Influence of Reward Expectation on Visuospatial Processing in Macaque Lateral Prefrontal Cortex
Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M., & Hikosaka, O. (2002) · Journal of Neurophysiology · doi:10.1152/jn.00472.2001 · Cited by 206 · see citing works
Shidara, M., Aigner, T. G., & Richmond, B. J. (1998) · Journal of Neuroscience · doi:10.1523/jneurosci.18-07-02613.1998 · Cited by 205 · see citing works
The Roles of Timing and Task Order during Task Switching
Dreher, J., Koechlin, É., Ali, O., & Grafman, J. H. (2002) · NeuroImage · doi:10.1006/nimg.2002.1169 · Cited by 164 · see citing works
Self-citationCoding and Monitoring of Motivational Context in the Primate Prefrontal Cortex
Watanabe, M., Hikosaka, K., Sakagami, M., & Shirakawa, S. (2002) · Journal of Neuroscience · doi:10.1523/jneurosci.22-06-02391.2002 · Cited by 159 · see citing works
Taste-related activity in the human dorsolateral prefrontal cortex
Kringelbach, M. L., de Araújo, I. E., & Rolls, E. T. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.063 · Cited by 156 · see citing works
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153 · see citing works
Prediction Error during Retrospective Revaluation of Causal Associations in Humans
Corlett, P. R., Aitken, M. R. F., Dickinson, A., Shanks, D. R., Honey, G. D., Honey, R. A. E., Robbins, T. W., Bullmore, E. T., & Fletcher, P. C. (2004) · Neuron · doi:10.1016/j.neuron.2004.11.022 · Cited by 144 · see citing works
Prediction error for free monetary reward in the human prefrontal cortex
Ramnani, N., Elliott, R., Athwal, B. S., & Passingham, R. E. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.028 · Cited by 130 · see citing works
Elliott, R., Newman, J. L., Longe, O. A., & Deakin, J. F. W. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.10.010 · Cited by 127 · see citing works
Reward-dependent learning in neuronal networks for planning and decision making
Dehaene, S., & Changeux, J. (2000) · Progress in brain research · doi:10.1016/s0079-6123(00)26016-0 · Cited by 115 · see citing works
Medial Prefrontal Cortex and Error Potentials
Holroyd, C. B., COLES, M. G. H., & Nieuwenhuis, S. (2002) · Science · doi:10.1126/science.296.5573.1610 · Cited by 104 · see citing works
Acute and Chronic Dopamine Dynamics in a Nonhuman Primate Model of Recreational Cocaine Use
Bradberry, C. W. (2000) · Journal of Neuroscience · doi:10.1523/jneurosci.20-18-07109.2000 · Cited by 97 · see citing works
An integrative theory of the phasic and tonic modes of dopamine modulation in the prefrontal cortex
Dreher, J., & Burnod, Y. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00051-5 · Cited by 66 · see citing works
Self-citationPaulus, M. P., Feinstein, J. S., Tapert, S. F., & Liu, T. T. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.060 · Cited by 49 · see citing works
A Model of Prefrontal Cortex Dopaminergic Modulation during the Delayed Alternation Task
Dreher, J., Guigon, E., & Burnod, Y. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902760191081 · Cited by 45 · see citing works
Self-citationIn Vivo Atlas of Deep Brain Structures
Lucerna, S., Salpietro, F. M., Alafaci, C., & Tomasello, F. F. (2002) · Springer eBooks · doi:10.1007/978-3-642-56381-2 · Cited by 25 · see citing works
Improved Shim by Subject Head Positioning
Heberlein, K., & Hu, X. (2001) · Cited by 7 · see citing works
Uncertainty-based competition between prefrontal and striatal systems for behavioural control
Daw, N., Niv, Y., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718 · see citing works
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · see citing works
Treadway, M. T., Buckholtz, J. W., Schwartzman, A. N., Lambert, W. E., & Zald, D. H. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006598 · Cited by 769 · see citing works
Neural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692 · see citing works
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661 · see citing works
Menstrual cycle phase modulates reward-related neural function in women
Dreher, J.-C., Schmidt, P. J., Kohn, P., Furman, D., Rubinow, D., & Berman, K. F. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0605569104 · Cited by 567 · see citing works
Self-citationSeparate Valuation Subsystems for Delay and Effort Decision Costs
Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2752-10.2010 · Cited by 499 · see citing works
Self-citationSchott, B. H., Minuzzi, L., Krebs, R. M., Elmenhorst, D., Lang, M., Winz, O. H., Seidenbecher, C. I., Coenen, H. H., Heinze, H.-J., Zilles, K., et al. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2058-08.2008 · Cited by 496 · see citing works
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472 · see citing works
Mohr, P. N. C., Biele, G., & Heekeren, H. R. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0003-10.2010 · Cited by 411 · see citing works
Prediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376 · see citing works
Variation in dopamine genes influences responsivity of the human reward system
Dreher, J.-C., Kohn, P., Kolachana, B., Weinberger, D. R., & Berman, K. F. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0805517106 · Cited by 376 · see citing works
Self-citationKnowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339 · see citing works
The Architecture of Reward Value Coding in the Human Orbitofrontal Cortex
Sescousse, G., Redouté, J., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.3501-10.2010 · Cited by 353 · see citing works
Self-citationAttention and associative learning in humans: An integrative review.
Le Pelley, M. E., Mitchell, C. J., Beesley, T., George, D. N., & Wills, A. J. (2016) · Psychological Bulletin · doi:10.1037/bul0000064 · Cited by 329 · see citing works
Learning-related changes in reward expectancy are reflected in the feedback-related negativity
Bellebaum, C., & Daum, I. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06138.x · Cited by 299 · see citing works
Age-related changes in midbrain dopaminergic regulation of the human reward system
Dreher, J.-C., Meyer-Lindenberg, A., Kohn, P., & Berman, K. F. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0802127105 · Cited by 230 · see citing works
Self-citationFrom prediction error to psychosis: Ketamine as a pharmacological model of delusions
Corlett, P. R., Honey, G. D., & Fletcher, P. C. (2007) · Journal of Psychopharmacology · doi:10.1177/0269881107077716 · Cited by 227 · see citing works
Abnormal Reward System Activation in Mania
Abler, B., Greenhouse, I., Ongur, D., Walter, H., & Heckers, S. (2008) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301620 · Cited by 216 · see citing works
Padmanabhan, A., Geier, C. F., Ordaz, S. J., Teslovich, T., & Luna, B. (2011) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2011.06.004 · Cited by 202 · see citing works
Corlett, P. R., Honey, G. D., Aitken, M. R. F., Dickinson, A., Shanks, D. R., Absalom, A. R., Lee, M., Pomarol-Clotet, E., Murray, G. K., McKenna, P. J., et al. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.6.611 · Cited by 182 · see citing works
Satterthwaite, T. D., Ruparel, K., Loughead, J., Elliott, M. A., Gerraty, R. T., Calkins, M. E., Hakonarson, H., Gur, R. C., Gur, R. E., & Wolf, D. H. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.03.060 · Cited by 148 · see citing works
Cerebral Correlates of Salient Prediction Error for Different Rewards and Punishments
Metereau, E., & Dreher, J.-C. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs037 · Cited by 114 · see citing works
Self-citationLevy, I., & Schiller, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.11.007 · Cited by 95 · see citing works
Modulation of corticospinal excitability by reward depends on task framing
Mooshagian, E., Keisler, A., Zimmermann, T., Schweickert, J. M., & Wassermann, E. M. (2015) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.12.021 · Cited by 24 · see citing works
Update on pathological gambling
Grant, J. E., Williams, K. A., & Kim, S. W. (2006) · Current Psychiatry Reports · doi:10.1007/s11920-006-0081-4 · Cited by 22 · see citing works
- Authors
- Dreher, J.-C., Kohn, P., & Berman, K. F.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Dreher, Jean-Claude
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2005-07-20
- Volume
- 16
- Issue
- 4
- Pages
- 561-573
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2023876982
- Cited by
- 180
- References
- 55
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 3 · 2022: 6 · 2021: 6 · 2020: 7 · 2019: 5 · 2018: 6 · 2017: 6 · 2016: 7 · 2015: 2 · 2014: 15 · 2013: 11 · 2012: 12
- FWCI
- 7.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Cerebral Cortex (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reward coding, reward prediction error, reward uncertainty, reward probability, midbrain, prefrontal cortex, ventral striatum, monetary rewards, motivation
- MeSH terms
- Adult, Conditioning, Operant, Discrimination Learning, Evoked Potentials, Female, Humans, Male, Mesencephalon, Psychomotor Performance, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
