Neural differentiation of expected reward and risk in human subcortical structures

Preuschoff, Kerstin; Bossaerts, Peter; Quartz, Steven R.

doi:10.1016/j.neuron.2006.06.024

Cited by 692Neural and Behavioral Psychology StudiesNeurotransmitter Receptor Influence on BehaviorDecision-Making and Behavioral Economics

Neural differentiation of expected reward and risk in human subcortical structures

Open access

Preuschoff, Kerstin, Bossaerts, Peter, Quartz, Steven R.·2006·Neuron

DOI 10.1016/j.neuron.2006.06.024

Open this paper in CoChat — free account Open PDF

Abstract

In decision-making under uncertainty, economic studies emphasize the importance of risk in addition to expected reward. Studies in neuroscience focus on expected reward and learning rather than risk. We combined functional imaging with a simple gambling task to vary expected reward and risk simultaneously and in an uncorrelated manner. Drawing on financial decision theory, we modeled expected reward as mathematical expectation of reward, and risk as reward variance. Activations in dopaminoceptive structures correlated with both mathematical parameters. These activations differentiated spatially and temporally. Temporally, the activation related to expected reward was immediate, while the activation related to risk was delayed. Analyses confirmed that our paradigm minimized confounds from learning, motivation, and salience. These results suggest that the primary task of the dopaminergic system is to convey signals of upcoming stochastic rewards, such as expected reward and risk, beyond its role in learning, motivation, and salience.

Integrity signals

Registry

Crossref · 10.1016/j.neuron.2006.06.024

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

When some amount of uncertainty is expected, learning rates should decrease.
“When some amount of uncertainty is expected (also referred to as outcome variance or risk), learning rates should decrease.10–14”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [10]
With monetary rewards, ventral striatum activity can signal both anticipated reward magnitude and probability.
“With monetary rewards, research shows that ventral striatum activity can signal both anticipated reward magnitude and probability 56, 57.”
From The social brain and reward: Social information processing in the human striatum (2014) · cites this paper as [57]

The pricing of options and corporate liabilities
Black, F., & Scholes, M. (1973) · Journal of Political Economy · doi:10.1086/260062 · Cited by 29,074 · see citing works
CAPITAL ASSET PRICES: A THEORY OF MARKET EQUILIBRIUM UNDER CONDITIONS OF RISK*
Sharpe, W. F. (1964) · The Journal of Finance · doi:10.1111/j.1540-6261.1964.tb02865.x · Cited by 19,441 · see citing works
Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk
Sharpe, W. F. (1964) · The Journal of Finance · doi:10.2307/2977928 · Cited by 7,556 · see citing works
The Valuation of Risk Assets and the Selection of Risky Investments in Stock Portfolios and Capital Budgets
Lintner, J. (1965) · The Review of Economics and Statistics · doi:10.2307/1924119 · Cited by 7,065 · see citing works
THE VALUATION OF RISK ASSETS AND THE SELECTION OF RISKY INVESTMENTS IN STOCK PORTFOLIOS AND CAPITAL BUDGETS**This paper is another in a series of interrelated theoretical and statistical studies of corporate financial and investment policies being made under grants from the Rockefeller Foundation, and more recently the Ford Foundation, to the Harvard Business School. The generous support for this work is most gratefully acknowledged. The author is also much indebted to his colleagues Professors Bishop, Christenson, Kahr, Raiffa, and (especially) Schlaifer, for extensive discussion and commentary on an earlier draft of this paper; but responsibility for any errors or imperfections remains strictly his own.[Professor Sharpe's paper, “Capital Asset Prices: A Theory of Market Equilibrium Under Conditions of Risk” (Journal of Finance, September 1964) appeared after this paper was in final form and on its way to the printers. My first section, which parallels the first half of his paper (with corresponding conclusions), sets the algebraic framework for sections II, III and VI, (which have no counterpart in his paper) and for section IV on the equilibrium prices of risk assets, concerning which our results differ significantly for reasons which will be explored elsewhere. Sharpe does not take up the capital budgeting problem developed in section V below.]
Lintner, J. (1975) · Elsevier eBooks · doi:10.1016/b978-0-12-780850-5.50018-6 · Cited by 6,747 · see citing works
Status quo bias in decision making
Samuelson, W., & Zeckhauser, R. (1988) · Journal of Risk and Uncertainty · doi:10.1007/bf00055564 · Cited by 5,897 · see citing works
Risk Aversion and Incentive Effects
Holt, C. A., & Laury, S. K. (2002) · American Economic Review · doi:10.1257/000282802762024700 · Cited by 5,829 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Equilibrium in a Capital Asset Market
Mossin, J. (1966) · Econometrica · doi:10.2307/1910098 · Cited by 4,910 · see citing works
Liquidity Preference as Behavior Towards Risk
Tobin, J. G. (1958) · The Review of Economic Studies · doi:10.2307/2296205 · Cited by 3,954 · see citing works
Deciding advantageously before knowing the advantageous strategy
Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997) · Science · doi:10.1126/science.275.5304.1293 · Cited by 3,547 · see citing works
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · see citing works
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
Asset Prices under Habit Formation and Catching up with the Joneses
Abel, A. B. (1990) · National Bureau of Economic Research · doi:10.3386/w3279 · Cited by 2,080 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Several Tests for Model Specification in the Presence of Alternative Hypotheses
Davidson, R., & MacKinnon, J. G. (1981) · Econometrica · doi:10.2307/1911522 · Cited by 1,747 · see citing works
FMRI visualization of brain activity during a monetary incentive delay task
Knutson, B., Westdorp, A., Kaiser, E., & Hommer, D. (2000) · NeuroImage · doi:10.1006/nimg.2000.0593 · Cited by 1,414 · see citing works
Neural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411 · see citing works
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404 · see citing works
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · see citing works
Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · see citing works
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,103 · see citing works
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076 · see citing works
The somatic marker hypothesis: A critical evaluation
Dunn, B. D., Dalgleish, T., & Lawrence, A. D. (2006) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2005.07.001 · Cited by 894 · see citing works
A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI
Knutson, B. D., Fong, G., Bennett, S. M., Adams, C. M., & Hommer, D. W. (2003) · NeuroImage · doi:10.1016/s1053-8119(02)00057-5 · Cited by 850 · see citing works
Dissociable Neural Responses in Human Reward Systems
Elliott, R., Friston, K. J., & Dolan, R. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-16-06159.2000 · Cited by 740 · see citing works
Neural Signatures of Economic Preferences for Risk and Ambiguity
Huettel, S. A., Stowe, C. J., Gordon, E. M., Warner, B. T., & Platt, M. L. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.024 · Cited by 713 · see citing works
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711 · see citing works
Neural activity in the human brain relating to uncertainty and arousal during anticipation.
Critchley, H. D., Mathias, C. J., & Dolan, R. J. (2001) · Neuron · doi:10.1016/s0896-6273(01)00225-2 · Cited by 681 · see citing works
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 544 · see citing works
A reexamination of the evidence for the somatic marker hypothesis: What participants really know in the Iowa gambling task
Maia, T. V., & McClelland, J. L. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0406666101 · Cited by 541 · see citing works
Differential Response Patterns in the Striatum and Orbitofrontal Cortex to Financial Reward in Humans: A Parametric Functional Magnetic Resonance Imaging Study
Elliott, R., Newman, J. L., Longe, O. A., & Deakin, J. F. W. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-01-00303.2003 · Cited by 515 · see citing works
Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices
Huettel, S. A., Song, A. W., & McCarthy, G. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.5070-04.2005 · Cited by 457 · see citing works
Human Striatal Responses to Monetary Reward Depend On Saliency
Zink, C. F., Pagnoni, G., Martin-Skurski, M. E., Chappelow, J., & Berns, G. S. (2004) · Neuron · doi:10.1016/s0896-6273(04)00183-7 · Cited by 432 · see citing works
Frontal lobe dysfunction in pathological gambling patients
Cavedini, P., Riboldi, G., Keller, R., D’Annucci, A., & Bellodi, L. (2002) · Biological Psychiatry · doi:10.1016/s0006-3223(01)01227-6 · Cited by 418 · see citing works
A neural correlate of response bias in monkey caudate nucleus
Lauwereyns, J., Watanabe, K., Coe, B. C., & Hikosaka, O. (2002) · Nature · doi:10.1038/nature00892 · Cited by 415 · see citing works
Animal choice behavior and the evolution of cognitive architecture
Real, L. A. (1991) · Science · doi:10.1126/science.1887231 · Cited by 377 · see citing works
Choice selection and reward anticipation: an fMRI study
Ernst, M., Nelson, E. E., McClure, E. B., Monk, C. S., Munson, S., Eshel, N., Zarahn, E., Leibenluft, E., Zametkin, A., Towbin, K., et al. (2004) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2004.05.011 · Cited by 377 · see citing works
Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements
Romo, R., & Schultz, W. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.3.592 · Cited by 353 · cited 1× · see citing works
“…previous neuroscience studies have focused on the learning aspect of reward anticipation (Hollerman and Schultz, 1998; Mirenowicz and Schultz, 1994; Romo and Schultz, 1990), leaving it unclear whether activation related to reward expectation and risk in subcortical structures requires learning and…”
Background
A Neural Representation of Categorization Uncertainty in the Human Brain
Grinband, J., Hirsch, J., & Ferrera, V. P. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.032 · Cited by 330 · see citing works
A Neuropsychological Investigation of Prefrontal Cortex Involvement in Acute Mania
Clark, L., Iversen, S. D., & Goodwin, G. M. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.10.1605 · Cited by 277 · see citing works
Human Midbrain Sensitivity to Cognitive Feedback and Uncertainty During Classification Learning
Aron, A. R., Daphna Shohamy, A., Clark, J. A., Myers, C. E., Gluck, M. A., & Poldrack, R. A. (2004) · Journal of Neurophysiology · doi:10.1152/jn.01209.2003 · Cited by 265 · see citing works
Functional activity related to risk anticipation during performance of the Iowa gambling task
Fukui, H., Murai, T., Fukuyama, H., Hayashi, T., & Hanakawa, T. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.08.028 · Cited by 256 · see citing works
Schizophrenia patients demonstrate a distinctive pattern of decision-making impairment on the Iowa Gambling Task
Shurman, B., Horan, W. P., & Nuechterlein, K. H. (2004) · Schizophrenia Research · doi:10.1016/j.schres.2004.03.020 · Cited by 237 · see citing works
Reliability and information transmission in spiking neurons.
Bialek, W., & Rieke, F. (1992) · Trends in Neurosciences · doi:10.1016/0166-2236(92)90005-s · Cited by 200 · see citing works
Saccade Reward Signals in Posterior Cingulate Cortex
McCoy, A. N., Crowley, J. C., Haghighian, G., Dean, H. L., & Platt, M. L. (2003) · Neuron · doi:10.1016/s0896-6273(03)00719-0 · Cited by 193 · see citing works
The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms.Preprint
Montague, P. R., & Sejnowski, T. J. (1994) · Learning & Memory · doi:10.1101/lm.1.1.1 · Cited by 185 · see citing works
Neural coding of distinct statistical properties of reward information in humans
Dreher, J.-C., Kohn, P., & Berman, K. F. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj004 · Cited by 180 · see citing works
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153 · see citing works
Basic Principles of Asset Pricing Theory: Evidence from Large-Scale Experimental Financial Markets
Bossaerts, P. L., & Plott, C. R. (2004) · European Finance Review · doi:10.1023/b:eufi.0000035190.24818.e5 · Cited by 143 · see citing works
Self-citation
Reward-Dependent Spatial Selectivity of Anticipatory Activity in Monkey Caudate Neurons
Takikawa, Y., Kawagoe, R., & Hikosaka, O. (2002) · Journal of Neurophysiology · doi:10.1152/jn.00288.2001 · Cited by 124 · see citing works
A brain imaging study of the choice procedure
Rustichini, A., Dickhaut, J. W., Ghirardato, P., et al. (2004) · Games and Economic Behavior · doi:10.1016/j.geb.2004.08.005 · Cited by 74 · see citing works
Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than backpropagating TD errors
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-7 · Cited by 65 · see citing works
Increased sensitivity to error during decision-making in bipolar disorder patients with acute mania
Minassian, A., Paulus, M. P., & Perry, W. (2004) · Journal of Affective Disorders · doi:10.1016/j.jad.2003.11.010 · Cited by 41 · see citing works

Flagged first, then by citations

Sorted by citations

The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718 · see citing works
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · see citing works
The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value
Bartra, O., McGuire, J. T., & Kable, J. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.063 · Cited by 1,927 · see citing works
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126 · see citing works
Behavioral dopamine signals.
Schultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028 · see citing works
Common and distinct networks underlying reward valence and processing stages: A meta-analysis of functional neuroimaging studies
Liu, X., Hairston, J., Schrier, M., & Fan, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.012 · Cited by 916 · see citing works
Processing of Social and Monetary Rewards in the Human Striatum
Izuma, K., Saito, D. N., & Sadato, N. (2008) · Neuron · doi:10.1016/j.neuron.2008.03.020 · Cited by 855 · see citing works
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866 · see citing works
Human insula activation reflects risk prediction errors as well as risk
Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4286-07.2008 · Cited by 784 · see citing works
Self-citation
Social neuroeconomics: the neural circuitry of social preferences
Fehr, E., & Camerer, C. F. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.09.002 · Cited by 779 · see citing works
Differential effects of insular and ventromedial prefrontal cortex lesions on risky decision-making
Clark, L., Bechara, A., Damasio, H., Aitken, M. R. F., Sahakian, B. J., & Robbins, T. W. (2008) · Brain · doi:10.1093/brain/awn066 · Cited by 613 · see citing works
Risky business: the neuroeconomics of decision making under uncertainty
Platt, M. L., & Huettel, S. A. (2008) · Nature Neuroscience · doi:10.1038/nn2062 · Cited by 556 · see citing works
Separate Valuation Subsystems for Delay and Effort Decision Costs
Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2752-10.2010 · Cited by 499 · see citing works
Neural correlates of value, risk, and risk aversion contributing to decision making under risk
Christopoulos, G. I., Tobler, P. N., Bossaerts, P., Dolan, R. J., & Schultz, W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2614-09.2009 · Cited by 451 · see citing works
Self-citation
Neural processing of risk
Mohr, P. N. C., Biele, G., & Heekeren, H. R. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0003-10.2010 · Cited by 411 · see citing works
Toward a neurobiology of delusions
Corlett, P. R., Taylor, J. R., Wang, X.-J., Fletcher, P. C., & Krystal, J. H. (2010) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2010.06.007 · Cited by 394 · see citing works
Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356 · see citing works
Confidence as Bayesian probability: From neural origins to behavior
Meyniel, F., Sigman, M., & Mainen, Z. F. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.039 · Cited by 349 · see citing works
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339 · see citing works
The Right and the Good: Distributive Justice and Neural Encoding of Equity and Efficiency
Hsu, M., Anen, C., & Quartz, S. R. (2008) · Science · doi:10.1126/science.1153651 · Cited by 354 · see citing works
Self-citation
Neural mechanisms of the influence of popularity on adolescent ratings of music
Berns, G. S., Capra, C. M., Moore, S., & Noussair, C. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.10.070 · Cited by 312 · see citing works
Rewards, aversions and affect in adolescence: emerging convergences across laboratory animal and human data
Spear, L. P. (2011) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2011.08.001 · Cited by 292 · see citing works
Neural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290 · see citing works
Dissociating valuation and saliency signals during decision-making
Litt, A., Plassmann, H., Shiv, B., & Rangel, A. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhq065 · Cited by 272 · see citing works
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267 · see citing works
A Key Role for Similarity in Vicarious Reward
Mobbs, D., Yu, R., Meyer, M., Passamonti, L., Seymour, B., Calder, A. J., Schweizer, S., Frith, C. D., & Dalgleish, T. (2009) · Science · doi:10.1126/science.1170539 · Cited by 270 · see citing works
Triangulating the Neural, Psychological, and Economic Bases of Guilt Aversion
Chang, L. J., Smith, A., Dufwenberg, M., & Sanfey, A. G. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.056 · Cited by 260 · see citing works
Testing the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244 · see citing works
The social brain and reward: Social information processing in the human striatum
Bhanji, J. P., & Delgado, M. R. (2014) · WIREs Cognitive Science · doi:10.1002/wcs.1266 · Cited by 256 · see citing works
The impact of prior risk experiences on subsequent risky decision-making: The role of the insula
Xue, G., Lu, Z., Levin, I. P., & Bechara, A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.097 · Cited by 249 · see citing works
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241 · see citing works
“…in multiple structures (den Ouden et al., 2010; Fiorillo et al., 2003; Kepecs et al., 2008; McCoy and Platt, 2005; O’Neill and Schultz, 2010; Preuschoff et al., 2006, 2008; Schultz et al., 2008; So and Stuphorn, 2012; Tobler et al., 2009) and have been linked with variables such as…”
Background
Explicit neural signals reflecting reward uncertainty
Schultz, W., Preuschoff, K., Camerer, C., Hsu, M., Fiorillo, C. D., Tobler, P. N., & Bossaerts, P. (2008) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2008.0152 · Cited by 239 · see citing works
Self-citation
Corticostriatal Interactions during Learning, Memory Processing, and Decision Making
Pennartz, C. M. A., Berke, J. D., Graybiel, A. M., Ito, R., Lansink, C. S., van der Meer, M., Redish, A. D., Smith, K. S., & Voorn, P. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3177-09.2009 · Cited by 213 · see citing works
Dopamine agonists and risk: impulse control disorders in Parkinson's; disease
Voon, V., Gao, J., Brezing, C., Symmonds, M., Ekanayake, V., Fernandez, H., Dolan, R. J., & Hallett, M. (2011) · Brain · doi:10.1093/brain/awr080 · Cited by 209 · see citing works
Mesolimbic dopamine release is linked to symptom severity in pathological gambling
Joutsa, J., Johansson, J., Niemelä, S., Ollikainen, A., Hirvonen, M. M., Piepponen, P., Arponen, E., Alho, H., Voon, V., Rinne, J. O., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.006 · Cited by 209 · see citing works
Computational substrates of norms and their violations during social exchange
Xiang, T., Lohrenz, T., & Montague, P. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.1642-12.2013 · Cited by 197 · see citing works
Risk taking and the adolescent reward system: A potential common link to substance abuse
Schneider, S., Peters, J., Bromberg, U., Brassen, S., Miedl, S. F., Banaschewski, T., Barker, G. J., Conrod, P., Flor, H., Garavan, H., et al. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11030489 · Cited by 186 · see citing works
Overlapping Responses for the Expectation of Juice and Money Rewards in Human Ventromedial Prefrontal Cortex
Kim, H., Shimojo, S., & O'Doherty, J. P. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhq145 · Cited by 176 · see citing works
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164 · see citing works
Engagement in digital interventions.
Nahum-Shani, I., Shaw, S. D., Carpenter, S. M., Murphy, S. A., & Yoon, C. (2022) · American Psychologist · doi:10.1037/amp0000983 · Cited by 177 · see citing works
A neurological study of compulsive buying behaviour
Raab, G., Elger, C. E., Neuner, M., & Weber, B. (2011) · Journal of Consumer Policy · doi:10.1007/s10603-011-9168-3 · Cited by 177 · see citing works
Uncertainty increases pain: evidence for a novel mechanism of pain modulation involving the periaqueductal gray
Yoshida, W., Seymour, B., Koltzenburg, M., & Dolan, R. J. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4984-12.2013 · Cited by 165 · see citing works
Reward uncertainty enhances incentive salience attribution as sign-tracking
Anselme, P., Robinson, M. J. F., & Berridge, K. C. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2012.10.006 · Cited by 170 · see citing works
Decreasing ventromedial prefrontal cortex activity during sequential risk-taking: an fMRI investigation of the balloon analog risk task
Schonberg, T., Fox, C. R., Mumford, J. A., Congdon, E., Trepel, C., & Poldrack, R. A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00080 · Cited by 162 · see citing works
Neural mechanisms regulating different forms of risk-related decision-making: Insights from animal models
Orsini, C. A., Moorman, D. E., Young, J. W., Setlow, B., & Floresco, S. B. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.04.009 · Cited by 153 · see citing works
Adaptive effort investment in cognitive and physical tasks: A neurocomputational model
Verguts, T., Vassena, E., & Silvetti, M. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00057 · Cited by 118 · see citing works
Adding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133 · see citing works
Roles of the different sub-regions of the insular cortex in various phases of the decision-making process
Droutman, V., Bechara, A., & Read, S. J. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00309 · Cited by 131 · see citing works
Functional neuroimaging of intertemporal choice models: a review
Carter, R. M., Meyer, J. R., & Huettel, S. A. (2010) · Journal of Neuroscience Psychology and Economics · doi:10.1037/a0018046 · Cited by 124 · see citing works
Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum
Diederen, K. M. J., Ziauddeen, H., Vestergaard, M. D., Spencer, T., Schultz, W., & Fletcher, P. C. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.1979-16.2016 · Cited by 110 · see citing works
Human VMPFC encodes early signatures of confidence in perceptual decisions
Gherman, S., & Philiastides, M. G. (2018) · eLife · doi:10.7554/elife.38293 · Cited by 107 · see citing works
Contextual novelty changes reward representations in the striatum
Guitart-Masip, M., Bunzeck, N., Stephan, K. E., Dolan, R. J., & Düzel, E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5331-09.2010 · Cited by 104 · see citing works
Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum
Gregorios-Pippas, L., Tobler, P. N., & Schultz, W. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90730.2008 · Cited by 101 · see citing works
Neural computations of threat
Levy, I., & Schiller, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.11.007 · Cited by 95 · see citing works
Striatal dopamine release codes uncertainty in pathological gambling
Linnet, J., Mouridsen, K., Peterson, E., Møller, A., Doudet, D. J., & Gjedde, A. (2012) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2012.04.012 · Cited by 82 · see citing works
Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning
Hart, A. S., Clark, J. J., & Phillips, P. E. M. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.07.010 · Cited by 82 · see citing works
Striatal structure and function predict individual biases in learning to avoid pain
Eldar, E., Hauser, T. U., Dayan, P., & Dolan, R. J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1519829113 · Cited by 79 · see citing works
Anterior insula reflects surprise in value-based decision-making and perception
Loued-Khenissi, L., Pfeuffer, A., Einhäuser, W., & Preuschoff, K. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116549 · Cited by 66 · see citing works
Self-citation
Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans
Fouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G. (2017) · Scientific Reports · doi:10.1038/s41598-017-04507-w · Cited by 61 · see citing works
P300 amplitude variations, prior probabilities, and likelihoods: a Bayesian ERP study
Kopp, B., Seer, C., Lange, F., Kluytmans, A., Kolossa, A., Fingscheidt, T., & Hoijtink, H. (2016) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-016-0442-3 · Cited by 60 · see citing works
Studying the neural representations of uncertainty
Walker, E. Y., Pohl, S., Denison, R. N., Barack, D. L., Lee, J., Block, N., Ma, W. J., & Meyniel, F. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01444-y · Cited by 56 · see citing works
Identification and disruption of a neural mechanism for accumulating prospective metacognitive information prior to decision-making
Miyamoto, K., Trudel, N., Kamermans, K., Lim, M. C., Lazari, A., Verhagen, L., Wittmann, M. K., & Rushworth, M. F. S. (2021) · Neuron · doi:10.1016/j.neuron.2021.02.024 · Cited by 45 · see citing works
Aberrant orbitofrontal cortex reactivity to erotic cues in compulsive sexual behavior disorder
Golec, K., Draps, M., Stark, R., Pluta, A., & Gola, M. (2021) · Journal of Behavioral Addictions · doi:10.1556/2006.2021.00051 · Cited by 36 · see citing works
Available alternative incentives modulate anticipatory nucleus accumbens activation
Cooper, J. C., Hollon, N. G., Wimmer, G. E., & Knutson, B. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp031 · Cited by 27 · see citing works
Variables influencing the neural correlates of perceived risk of physical harm
Coaster, M., Rogers, B. P., Jones, O. D., Viscusi, W. K., Merkle, K. L., Zald, D. H., & Gore, J. C. (2011) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-011-0047-9 · Cited by 24 · see citing works
Brain Activity Foreshadows Stock Price Dynamics
Stallen, M., Borg, N., & Knutson, B. (2021) · The Journal of Neuroscience · doi:10.1523/jneurosci.1727-20.2021 · Cited by 18 · see citing works
Persistent Ventral Anterior Cingulate Cortex and Resolved Amygdala Hyper-responses to Negative Outcomes After Depression Remission: A Combined Cross-sectional and Longitudinal Study
Zhang, X., Wang, X., Dong, D., Sun, X., Zhong, X., Xiong, G., Cheng, C., Lei, H., Chai, Y., Yu, M., et al. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.09.003 · Cited by 11 · see citing works

Stress and the dopaminergic reward system
Baik, J. (2020) · Experimental & Molecular Medicine · doi:10.1038/s12276-020-00532-4 · Cited by 446 · see citing works
Hedonic and incentive signals for body weight control
Egecioglu, E., Skibicka, K. P., Hansson, C., Álvarez-Crespo, M., Friberg, P. A., Jerlhag, E., Engel, J. A., & Dickson, S. L. (2011) · Reviews in Endocrine and Metabolic Disorders · doi:10.1007/s11154-011-9166-4 · Cited by 173 · see citing works
“Liking” and “wanting” food rewards: Brain substrates and roles in eating disorders
Berridge, K. C. (2009) · Physiology & Behavior · doi:10.1016/j.physbeh.2009.02.044 · Cited by 1,147 · see citing works
The psychology and neurobiology of addiction: an incentive–sensitization view
Robinson, T. E., & Berridge, K. C. (2000) · Addiction · doi:10.1046/j.1360-0443.95.8s2.19.x · Cited by 1,507 · see citing works
Endocannabinoid Regulation of Reward and Reinforcement through Interaction with Dopamine and Endogenous Opioid Signaling
Wenzel, J. M., & Cheer, J. F. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.126 · Cited by 154 · see citing works

Authors: Preuschoff, K., Bossaerts, P., & Quartz, S. R.
Affiliations: California Institute of Technology (US)
Corresponding author: Preuschoff, Kerstin
ORCID: Preuschoff, Kerstin
Bossaerts, Peter
Venue: Neuron
Publisher: Elsevier BV
Published: 2006-08
Volume: 51
Issue: 3
Pages: 381-390
ISSN: 0896-6273 (ISSN-L)
DOI: 10.1016/j.neuron.2006.06.024
OpenAlex: W2139635035
Cited by: 692
References: 55
Citations by year: 2026: 14 · 2025: 18 · 2024: 25 · 2023: 30 · 2022: 28 · 2021: 34 · 2020: 27 · 2019: 27 · 2018: 30 · 2017: 27 · 2016: 34 · 2015: 37 · 2014: 37 · 2013: 51 · 2012: 42
FWCI: 21.27
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
www.elsevier.com/legal/tdmrep-license (tdm)
www.elsevier.com/open-access/userlicense/1.0/ (vor, 2578 d)
doi.org/10.15223/policy-017 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
doi.org/10.15223/policy-012 (stm-asf)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-004 (stm-asf)
Locations: Neuron (publishedVersion)
PubMed (publishedVersion)
CaltechAUTHORS (California Institute of Technology) (submittedVersion)
Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
Funders: David and Lucile Packard Foundation
Gordon and Betty Moore Foundation
National Science Foundation (0093757)
Topics: Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Keywords: expected reward, risk, decision making under uncertainty, gambling task, dopaminergic system, subcortical structures, reward variance, expected value, stochastic rewards
MeSH terms: Adolescent, Adult, Brain, Brain Mapping, Discrimination Learning, Female, Humans, Male, Psychomotor Performance, Reward, Risk-Taking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Neural differentiation of expected reward and risk in human subcortical structures

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks