Dopamine prediction error responses integrate subjective value from different reward dimensions
Open accessAbstract
Significance: Most real-world rewards have multiple dimensions, such as amount, risk, and type. Here we show that within a bounded set of such multidimensional rewards monkeys’ choice behavior fulfilled several core tenets of rational choice theory; namely, their choices were stochastically complete and transitive. As such, in selecting between rewards, the monkeys behaved as if they maximized value on a common scale. Dopamine neurons encoded prediction errors that reflected that scale. A particular reward dimension influenced dopamine activity only to the extent that it influenced choice. Thus, vastly different reward types such as juice and food activated dopamine neurons in accordance with subjective value derived from the different rewards. This neuronal signal could serve to update value signals for economic choice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Dopamine neurons code subjective rather than objective reward value.
“Dopamine neurons show higher activations in response to the preferred juice, and their activity also correlates with the indifference amounts in choices between risky and safe rewards, indicating that the neurons consistently code subjective rather than objective reward value9.”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [9]
Stimuli of all sensory modalities are effective in eliciting the initial response component.
“Stimuli of all sensory modalities are effective in eliciting this initial response component; those tested include loud sounds, intense light flashes, rapidly moving visual objects and liquids touching the mouth2,9,16,19,20,35.”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [9]
These slow changes fail to occur with more-controlled arm, mouth and eye movements in a considerable range of studies.
“these changes fail to occur with more-controlled arm, mouth and eye movements in a considerable range of studies2,5,9,13,151,152,154”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [9]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationIndividual Choice Behavior: A Theoretical Analysis.
Cane, V. R., & Luce, R. D. (1960) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343282 · Cited by 2,598 · see citing works
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · see citing works
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
Self-citationRegulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry
Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., & Kreitzer, A. C. (2010) · Nature · doi:10.1038/nature09159 · Cited by 1,885 · see citing works
A Discounting Framework for Choice With Delayed and Probabilistic Rewards.
Green, L., & Myerson, J. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.5.769 · Cited by 1,676 · see citing works
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · see citing works
Modulation of memory fields by dopamine Dl receptors in prefrontal cortex
Williams, G. V., & Goldman-Rakic, P. S. (1995) · Nature · doi:10.1038/376572a0 · Cited by 1,593 · see citing works
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · see citing works
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · see citing works
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291 · see citing works
Self-citationAdaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · see citing works
Self-citationPhasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266 · see citing works
Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons
Watabe‐Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., & Uchida, N. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.017 · Cited by 1,221 · see citing works
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190 · see citing works
PEST: Efficient Estimates on Probability Functions
Taylor, M. M., & Creelman, C. D. (1967) · The Journal of the Acoustical Society of America · doi:10.1121/1.1910407 · Cited by 1,173 · see citing works
Individual choice behavior: A theoretical analysis.
Luce, R. D. (2005) · doi:10.1037/14396-000 · Cited by 1,157 · see citing works
Matching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030 · see citing works
Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory
Vijayraghavan, S., Wang, M., Birnbaum, S. G., Williams, G. V., & Arnsten, A. F. T. (2007) · Nature Neuroscience · doi:10.1038/nn1846 · Cited by 1,013 · see citing works
Neural Economics and the Biological Substrates of Valuation
Montague, P. R., & Berns, G. S. (2002) · Neuron · doi:10.1016/s0896-6273(02)00974-1 · Cited by 994 · see citing works
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · see citing works
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K. A., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 879 · see citing works
A cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879 · see citing works
Gonon, F. G. (1988) · Neuroscience · doi:10.1016/0306-4522(88)90307-7 · Cited by 676 · see citing works
GABA Neurons of the VTA Drive Conditioned Place Aversion
Tan, K. R., Yvon, C., Turiault, M., Mirzabekov, J. J., Doehner, J., Labouèbe, G., Deisseroth, K. A., Tye, K. M., & Lüscher, C. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.015 · Cited by 635 · see citing works
Activation of VTA GABA Neurons Disrupts Reward Consumption
van Zessen, R., Phillips, J. L., Budygin, E. A., & Stuber, G. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.016 · Cited by 609 · see citing works
Neurobiology of economic choice: a good-based model
Padoa-Schioppa, C. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113648 · Cited by 600 · see citing works
Handbook of Mathematical Psychology
McGinnis, R., Luce, R. D., Bush, R. R., & Galanter, E. (1967) · The American Journal of Psychology · doi:10.2307/1421001 · Cited by 551 · see citing works
Free choice activates a decision circuit between frontal and parietal cortex
Pesaran, B., Nelson, M., & Andersen, R. (2008) · Nature · doi:10.1038/nature06849 · Cited by 479 · see citing works
Double dissociation of value computations in orbitofrontal and anterior cingulate neurons
Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2961 · Cited by 452 · see citing works
Medial prefrontal cell activity signaling prediction errors of action values
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007) · Nature Neuroscience · doi:10.1038/nn1890 · Cited by 440 · see citing works
Influence of Reward Delays on Responses of Dopamine Neurons
Kobayashi, S., & Schultz, W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 431 · see citing works
Self-citationTransient stimulation of distinct subpopulations of striatal neurons mimics changes in action value
Tai, L.-H., Lee, A. M., Benavidez, N., Bonci, A., & Wilbrecht, L. (2012) · Nature Neuroscience · doi:10.1038/nn.3188 · Cited by 399 · see citing works
Utility of Gains and Losses: Measurement-Theoretical and Experimental Approaches
Luce, R. D. (2000) · Medical Entomology and Zoology · Cited by 396 · see citing works
Hong, S., Jhou, T. C., Smith, M., Saleem, K. S., & Hikosaka, O. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1384-11.2011 · Cited by 390 · see citing works
Lewis, D. A., Melchitzky, D. S., Sesack, S. R., Whitehead, R. E., Auh, S., & Sampson, A. (2001) · Journal of Comparative Neurology · doi:10.1002/cne.1092 · Cited by 352 · see citing works
Reward value-based gain control: divisive normalization in parietal cortex
Louie, K., Grattan, L. E., & Glimcher, P. W. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.1237-11.2011 · Cited by 308 · see citing works
Williams, S. M., & Goldman-Rakic, P. S. (1993) · Cerebral Cortex · doi:10.1093/cercor/3.3.199 · Cited by 284 · see citing works
Dopamine Neurons Can Represent Context-Dependent Prediction Error
Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., & Hikosaka, O. (2004) · Neuron · doi:10.1016/s0896-6273(03)00869-9 · Cited by 276 · see citing works
Burton, M. J., Rolls, E. T., & Mora, F. (1976) · Experimental Neurology · doi:10.1016/0014-4886(76)90189-8 · Cited by 252 · see citing works
The organization of midbrain projections to the ventral striatum in the primate
Lynd‐Balta, E., & Haber, S. N. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90181-3 · Cited by 169 · see citing works
The Role of Prefrontal Dopamine D1 Receptors in the Neural Mechanisms of Associative Learning
Puig, M. V., & Miller, E. K. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.018 · Cited by 146 · see citing works
The primate amygdala combines information about space and value
Peck, C. J., Lau, B., & Salzman, C. D. (2013) · Nature Neuroscience · doi:10.1038/nn.3328 · Cited by 115 · see citing works
EXPERIMENTAL TESTS OF A STOCHASTIC DECISION THEORYPreprint
Davidson, D., & Marschak, J. (1958) · RePEc: Research Papers in Economics · Cited by 110 · see citing works
Multiple Timescales of Memory in Lateral Habenula and Dopamine Neurons
Bromberg-Martin, E. S., Matsumoto, M., Nakahara, H., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.031 · Cited by 91 · see citing works
Hypothalamic neuron involvement in integration of reward, aversion, and cue signals
Ono, T., Nakamura, K., Nishijo, H., & Fukuda, M. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.1.63 · Cited by 87 · see citing works
Bidirectional Modulation of Substantia Nigra Activity by Motivational State
Rossi, M. A., Fan, D., Barter, J. W., & Yin, H. H. (2013) · PLoS ONE · doi:10.1371/journal.pone.0071598 · Cited by 43 · see citing works
An, X., Bandler, R., �ng�r, D., & Price, J. L. (1998) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19981130)401:4
Flagged first, then by citations
Cited by
Sorted by citations
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126 · see citing works
Self-citationDopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036 · see citing works
Self-citationDopamine reward prediction error coding
Schultz, W. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/wschultz · Cited by 597 · see citing works
Self-citationDopamine and Cognitive Control in Prefrontal Cortex
Ott, T., & Nieder, A. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.006 · Cited by 530 · see citing works
Neural Circuitry of Reward Prediction Error
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437 · see citing works
Varazzani, C., San‐Galli, A., Gilardeau, S., & Bouret, S. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0454-15.2015 · Cited by 446 · see citing works
A Corticostriatal Path Targeting Striosomes Controls Decision-Making under Conflict
Friedman, A., Homma, D., Gibb, L. G., Amemori, K.-I., Rubin, S. J., Hood, A. S., Riad, M. H., & Graybiel, A. M. (2015) · Cell · doi:10.1016/j.cell.2015.04.049 · Cited by 309 · see citing works
Blanchard, T. C., Hayden, B. Y., & Bromberg-Martin, E. S. (2015) · Neuron · doi:10.1016/j.neuron.2014.12.050 · Cited by 301 · see citing works
Dopamine Neuron-Specific Optogenetic Stimulation in Rhesus Macaques
Stauffer, W. R., Lak, A., Yang, A., Borel, M., Paulsen, O., Boyden, E. S., & Schultz, W. (2016) · Cell · doi:10.1016/j.cell.2016.08.024 · Cited by 256 · see citing works
Self-citationReward functions of the basal ganglia
Schultz, W. (2016) · Journal of Neural Transmission · doi:10.1007/s00702-016-1510-0 · Cited by 248 · see citing works
Self-citationDopamine release in the nucleus accumbens core signals perceived saliency
Kutlu, M. G., Zachry, J. E., Melugin, P. R., Cajigas, S. A., Chevee, M. F., Kelly, S. J., Kutlu, B., Tian, L., Siciliano, C. A., & Calipari, E. S. (2021) · Current Biology · doi:10.1016/j.cub.2021.08.052 · Cited by 238 · see citing works
Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases
Pessiglione, M., Vinckier, F., Bouret, S., Jean Daunizeau, A., & Bouc, R. L. (2017) · Brain · doi:10.1093/brain/awx278 · Cited by 230 · see citing works
Rodeberg, N. T., Sandberg, S. G., Johnson, J. A., Phillips, P. E. M., & Wightman, R. M. (2017) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.6b00393 · Cited by 226 · see citing works
Stress-induced plasticity and functioning of ventral tegmental dopamine neurons
Douma, E. H., & de Kloet, E. R. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.10.015 · Cited by 226 · see citing works
Dopamine Reward Prediction Error Responses Reflect Marginal Utility
Stauffer, W. R., Lak, A., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.064 · Cited by 204 · see citing works
Self-citationDopamine neurons respond to errors in the prediction of sensory features of expected rewards
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.025 · Cited by 200 · see citing works
Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170 · see citing works
Self-citationModel-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161 · see citing works
Chronic Stress Alters Striosome-Circuit Dynamics, Leading to Aberrant Decision-Making
Friedman, A., Homma, D., Bloem, B., Gibb, L. G., Amemori, K.-I., Hu, D., Delcasso, S., Truong, T. F., Yang, J., Hood, A. S., et al. (2017) · Cell · doi:10.1016/j.cell.2017.10.017 · Cited by 162 · see citing works
Schultz, W., Stauffer, W. R., & Lak, A. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.03.013 · Cited by 164 · see citing works
Self-citationDiCarlo, G. E., Aguilar, J. I., Matthies, H. J. G., Harrison, F. E., Bundschuh, K. E., West, A. M., Hashemi, P., Herborg, F., Rickhag, M., Chen, H., et al. (2019) · Journal of Clinical Investigation · doi:10.1172/jci127411 · Cited by 156 · see citing works
Hosking, J. G., Floresco, S., & Winstanley, C. A. (2014) · Neuropsychopharmacology · doi:10.1038/npp.2014.285 · Cited by 154 · see citing works
The Psychological and Neural Basis of Loss Aversion
Sokol‐Hessner, P., & Rutledge, R. B. (2018) · Current Directions in Psychological Science · doi:10.1177/0963721418806510 · Cited by 150 · see citing works
Advances from neuroimaging studies in eating disorders
Frank, G. K. W. (2015) · CNS Spectrums · doi:10.1017/s1092852915000012 · Cited by 144 · see citing works
Functional architecture of reward learning in mushroom body extrinsic neurons of larval Drosophila
Saumweber, T., Rohwedder, A., Schleyer, M. H., Eichler, K., Chen, Y., Aso, Y., Cardona, A., Eschbach, C., Kobler, O., Voigt, A., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-03130-1 · Cited by 154 · see citing works
Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework
Sadacca, B. F., Jones, J. L., & Schoenbaum, G. M. (2016) · eLife · doi:10.7554/elife.13665 · Cited by 137 · see citing works
Moral Learning: Conceptual foundations and normative relevance
Railton, P. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.015 · Cited by 130 · see citing works
GURNEY, K. N., Humphries, M. D., & Redgrave, P. (2015) · PLoS Biology · doi:10.1371/journal.pbio.1002034 · Cited by 123 · see citing works
O’Doherty, J. P. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.03.027 · Cited by 129 · see citing works
Nonhuman Primate Optogenetics: Recent Advances and Future Directions
Galván, A., Stauffer, W. R., Acker, L., El-Shamayleh, Y., Inoue, K., Ohayon, S., & Schmid, M. C. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.1839-17.2017 · Cited by 127 · see citing works
Self-citationHebscher, M., & Gilboa, A. (2016) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2016.05.003 · Cited by 124 · see citing works
Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation
Collins, A. L., Greenfield, V. Y., Bye, J. K., Linker, K. E., Wang, A. S., & Wassum, K. M. (2016) · Scientific Reports · doi:10.1038/srep20231 · Cited by 116 · see citing works
Saddoris, M. P., Sugam, J. A., Stuber, G. D., Witten, I. B., Deisseroth, K. A., & Carelli, R. M. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.10.024 · Cited by 116 · see citing works
Dopamine, Prediction Error and Beyond
Diederen, K. M. J., & Fletcher, P. C. (2021) · The Neuroscientist · doi:10.1177/1073858420907591 · Cited by 106 · see citing works
Post-learning hippocampal replay selectively reinforces spatial memory for highly rewarded locations
Michon, F., Sun, J.-J., Kim, C. Y., Ciliberti, D., & Kloosterman, F. (2019) · Current Biology · doi:10.1016/j.cub.2019.03.048 · Cited by 86 · see citing works
A comparison of neural responses to appetitive and aversive stimuli in humans and other mammals
Hayes, D. J., Duncan, N. W., Xu, J., & Northoff, G. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.06.018 · Cited by 98 · see citing works
Transcriptional and anatomical diversity of medium spiny neurons in the primate striatum
He, J., Kleyman, M., Chen, J., Alikaya, A., Rothenhoefer, K. M., Öztürk, B. E., Wirthlin, M. E., Bostan, A. C., Fish, K. N., Byrne, L. C., et al. (2021) · Current Biology · doi:10.1016/j.cub.2021.10.015 · Cited by 96 · see citing works
Self-citationA primate temporal cortex–zona incerta pathway for novelty seeking
Ogasawara, T., Sogukpinar, F., Zhang, K., Feng, Y.-Y., Pai, J., Jezzini, A., & Monosov, I. E. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00950-1 · Cited by 96 · see citing works
Dopamine neurons learn relative chosen value from probabilistic rewards
Lak, A., Stauffer, W. R., & Schultz, W. (2016) · eLife · doi:10.7554/elife.18044 · Cited by 87 · see citing works
Self-citationPhasic dopamine signals: from subjective reward value to formal economic utility
Schultz, W., Carelli, R. M., & Wightman, R. M. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.09.006 · Cited by 88 · see citing works
Self-citationThe Dopamine Prediction Error: Contributions to Associative Models of Reward Learning
Nasser, H. M., Calu, D. J., Schoenbaum, G. M., & Sharpe, M. J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00244 · Cited by 89 · see citing works
Recent advances in understanding the role of phasic dopamine activity
Schultz, W. (2019) · F1000Research · doi:10.12688/f1000research.19793.1 · Cited by 89 · see citing works
Self-citationSocial reward monitoring and valuation in the macaque brain
Noritake, A., Ninomiya, T., & Isoda, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0229-7 · Cited by 86 · see citing works
Anticipating cognitive effort: roles of perceived error-likelihood and time demands
Dunn, T. L., Inzlicht, M., & Risko, E. F. (2017) · Psychological Research · doi:10.1007/s00426-017-0943-x · Cited by 80 · see citing works
Dopamine and the interdependency of time perception and reward
Fung, B. J., Sutlief, E., & Shuler, M. G. H. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.02.030 · Cited by 79 · see citing works
Economic Choices Reveal Probability Distortion in Macaque Monkeys
Stauffer, W. R., Lak, A., Bossaerts, P. L., & Schultz, W. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3653-14.2015 · Cited by 80 · see citing works
Self-citationA decade of decoding reward-related fMRI signals and where we go from here
Kahnt, T. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.067 · Cited by 75 · see citing works
Lateral Orbitofrontal Inactivation Dissociates Devaluation-Sensitive Behavior and Economic Choice
Gardner, M., Conroy, J. S., Shaham, M. H., Styer, C. V., & Schoenbaum, G. M. (2017) · Neuron · doi:10.1016/j.neuron.2017.10.026 · Cited by 75 · see citing works
Mesolimbic Dopamine Encodes Prediction Errors in a State-Dependent Manner
Papageorgiou, G., Baudonnat, M., Cucca, F., & Walton, M. E. (2016) · Cell Reports · doi:10.1016/j.celrep.2016.03.031 · Cited by 76 · see citing works
Correlates of economic decisions in the dorsal and subgenual anterior cingulate cortices
Azab, H., & Hayden, B. Y. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.13865 · Cited by 71 · see citing works
Components and characteristics of the dopamine reward utility signal
Stauffer, W. R., Lak, A., Kobayashi, S., & Schultz, W. (2015) · The Journal of Comparative Neurology · doi:10.1002/cne.23880 · Cited by 62 · see citing works
Self-citationStriosomes Mediate Value-Based Learning Vulnerable in Age and a Huntington’s Disease Model
Friedman, A., Hueske, E., Drammis, S. M., Arana, S. E. T., Nelson, E. D., Carter, C. W., Delcasso, S., Rodriguez, R. X., Lutwak, H., DiMarco, K. S., et al. (2020) · Cell · doi:10.1016/j.cell.2020.09.060 · Cited by 61 · see citing works
Pastor‐Bernier, A., Stasiak, A., & Schultz, W. (2019) · Nature Communications · doi:10.1038/s41467-019-12792-4 · Cited by 61 · see citing works
Self-citationLearning the specific quality of taste reinforcement in larval Drosophila
Schleyer, M. H., Miura, D., Tanimura, T., & Gerber, B. (2015) · eLife · doi:10.7554/elife.04711 · Cited by 70 · see citing works
Benarroch, E. E. (2015) · Neurology · doi:10.1212/wnl.0000000000001937 · Cited by 58 · see citing works
Konanur, V., Hsu, T. M., Kanoski, S. E., Hayes, M. R., & Roitman, M. F. (2019) · Physiology & Behavior · doi:10.1016/j.physbeh.2019.112771 · Cited by 58 · see citing works
Toward sophisticated basal ganglia neuromodulation: Review on basal ganglia deep brain stimulation
Cunha, C. D., Boschen, S. L., Gómez‐A, A., Ross, E. K., Gibson, W. S. J., Min, H.-K. P., Lee, K. H., & Blaha, C. D. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.02.003 · Cited by 58 · see citing works
Monetary, Food, and Social Rewards Induce Similar Pavlovian-to-Instrumental Transfer Effects
Lehner, R., Balsters, J. H., Herger, A., Hare, T. A., & Wenderoth, N. (2017) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2016.00247 · Cited by 56 · see citing works
Novelty, Salience, and Surprise Timing Are Signaled by Neurons in the Basal Forebrain
Zhang, K., Chen, C. D., & Monosov, I. E. (2018) · Current Biology · doi:10.1016/j.cub.2018.11.012 · Cited by 48 · see citing works
Utility functions predict variance and skewness risk preferences in monkeys
Genest, W., Stauffer, W. R., & Schultz, W. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602217113 · Cited by 48 · see citing works
Self-citationEndogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability
Chew, B., Hauser, T. U., Papoutsi, M., Magerkurth, J., Dolan, R. J., & Rutledge, R. B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900872116 · Cited by 47 · see citing works
Dopamine-associated cached values are not sufficient as the basis for action selection
Hollon, N. G., Arnold, M. M., Gan, J. O., Walton, M. E., & Phillips, P. E. M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1419770111 · Cited by 52 · see citing works
A benefit–cost framework of motivation for a specific activity
Studer, B., & Knecht, S. (2016) · Progress in brain research · doi:10.1016/bs.pbr.2016.06.014 · Cited by 51 · see citing works
Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation
Kato, A., & Morita, K. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005145 · Cited by 57 · see citing works
Thirst recruits phasic dopamine signaling through subfornical organ neurons
Hsu, T. M., Bazzino, P., Hurh, S. J., Konanur, V., Roitman, J. D., & Roitman, M. F. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2009233117 · Cited by 50 · see citing works
Schelp, S. A., Pultorak, K. J., Rakowski, D. R., Gomez, D. M., Krzystyniak, G., Das, R., & Oleson, E. B. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1706969114 · Cited by 50 · see citing works
Insular neural system controls decision-making in healthy and methamphetamine-treated rats
Mizoguchi, H., Katahira, K., Inutsuka, A., Fukumoto, K., Nakamura, A., Wang, T., Nagai, T., Sato, J., Sawada, M., Ohira, H., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1418014112 · Cited by 49 · see citing works
The Attraction Effect Modulates Reward Prediction Errors and Intertemporal Choices
Gluth, S., Hotaling, J. M., & Rieskamp, J. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.2532-16.2016 · Cited by 46 · see citing works
Neurobiology of Craving: Current Findings and New Directions
Ray, L. A., & Roche, D. J. O. (2018) · Current Addiction Reports · doi:10.1007/s40429-018-0202-2 · Cited by 49 · see citing works
Planning activity for internally generated reward goals in monkey amygdala neurons
Hernádi, I., Fabian Grabenhorst, B., & Schultz, W. (2015) · Nature Neuroscience · doi:10.1038/nn.3925 · Cited by 46 · see citing works
Self-citationBalasubramani, P. P., Chakravarthy, V. S., Ravindran, B., & Moustafa, A. A. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00076 · Cited by 42 · see citing works
A dopamine mechanism for reward maximization
Schultz, W. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2316658121 · Cited by 44 · see citing works
Self-citationCampos, L., Arokiaraj, C. M., Chuapoco, M. R., Chen, X., Goeden, N., Gradinaru, V., & Fox, A. S. (2023) · Current Research in Neurobiology · doi:10.1016/j.crneur.2023.100086 · Cited by 41 · see citing works
Overlapping representations of food and social stimuli in mouse VTA dopamine neurons
Willmore, L., Minerva, A. R., Engelhard, B., Murugan, M., McMannon, B., Oak, N., Thiberge, S. Y., Peña, C. J., & Witten, I. B. (2023) · Neuron · doi:10.1016/j.neuron.2023.08.003 · Cited by 42 · see citing works
Takahashi, Y. K., Stalnaker, T. A., Mueller, L. E., Harootonian, S. K., Langdon, A., & Schoenbaum, G. M. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01310-x · Cited by 41 · see citing works
Cremer, A., Kalbe, F., Müller, J. C., Wiedemann, K., & Schwabe, L. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-022-01517-9 · Cited by 39 · see citing works
The influence of contextual reward statistics on risk preference
Rigoli, F., Rutledge, R. B., Dayan, P., & Dolan, R. J. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.12.016 · Cited by 39 · see citing works
Pastor‐Bernier, A., Plott, C. R., & Schultz, W. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1612010114 · Cited by 39 · see citing works
Self-citationRecent Advances in Neuroimaging to Model Eating Disorder Neurobiology
Frank, G. (2015) · Current Psychiatry Reports · doi:10.1007/s11920-015-0559-z · Cited by 35 · see citing works
Could Dopamine Agonists Aid in Drug Development for Anorexia Nervosa?
Frank, G. (2014) · Frontiers in Nutrition · doi:10.3389/fnut.2014.00019 · Cited by 37 · see citing works
Acute Stress Enhances Associative Learning via Dopamine Signaling in the Ventral Lateral Striatum
Stelly, C. E., Tritley, S. C., Rafati, Y., & Wanat, M. J. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.3003-19.2020 · Cited by 36 · see citing works
Kutlu, M. G., Tat, J., Christensen, B. A., Zachry, J. E., & Calipari, E. S. (2023) · Cell Reports · doi:10.1016/j.celrep.2023.112948 · Cited by 34 · see citing works
Suarez, J. A., Howard, J. D., Schoenbaum, G. M., & Kahnt, T. (2019) · eLife · doi:10.7554/elife.43962 · Cited by 34 · see citing works
Multiple Mechanisms for Processing Reward Uncertainty in the Primate Basal Forebrain
Ledbetter, N. M., Chen, C. D., & Monosov, I. E. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1123-16.2016 · Cited by 33 · see citing works
Humans Integrate Monetary and Liquid Incentives to Motivate Cognitive Task Performance
Yee, D. M., Krug, M. K., Allen, A. Z., & Braver, T. S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.02037 · Cited by 31 · see citing works
Tanaka, S., O'Doherty, J., & Sakagami, M. (2019) · Nature Communications · doi:10.1038/s41467-019-11334-2 · Cited by 30 · see citing works
Neuronal Circuits for Social Decision-Making and Their Clinical Implications
Báez-Mendoza, R., Vázquez, Y., Mastrobattista, E. P., & Williams, Z. M. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.720294 · Cited by 32 · see citing works
Cell-type and projection-specific dopaminergic encoding of aversive stimuli in addiction
Thibeault, K. C., Kutlu, M. G., Sanders, C. M., & Calipari, E. S. (2018) · Brain Research · doi:10.1016/j.brainres.2018.12.024 · Cited by 27 · see citing works
Oleson, E. B., Hamilton, L. R., & Gomez, D. M. (2021) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2021.660218 · Cited by 27 · see citing works
Seitz, B. M., Hoang, I. B., DiFazio, L. E., Blaisdell, A. P., & Sharpe, M. J. (2022) · Current Biology · doi:10.1016/j.cub.2022.06.035 · Cited by 27 · see citing works
Intrinsic rewards explain context-sensitive valuation in reinforcement learning
Molinaro, G., & Collins, A. G. E. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002201 · Cited by 29 · see citing works
Neural substrates of updating the prediction through prediction error during decision making
Wang, Y., Ma, N., He, X., Li, N., Wei, Z., Yang, L., Zha, R., Han, L., Li, X., Zhang, D., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.05.041 · Cited by 27 · see citing works
A Transient Dopamine Signal Represents Avoidance Value and Causally Influences the Demand to Avoid
Pultorak, K. J., Schelp, S. A., Isaacs, D. P., Krzystyniak, G., & Oleson, E. B. (2018) · eNeuro · doi:10.1523/eneuro.0058-18.2018 · Cited by 27 · see citing works
Camerer, C. F. (2014) · Current Biology · doi:10.1016/j.cub.2014.07.040 · Cited by 27 · see citing works
Cross-talk between the epigenome and neural circuits in drug addiction
Mews, P., & Calipari, E. S. (2017) · Progress in brain research · doi:10.1016/bs.pbr.2017.08.012 · Cited by 25 · see citing works
A neuronal reward inequity signal in primate striatum
Báez-Mendoza, R., van Coeverden, C., & Schultz, W. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00321.2015 · Cited by 26 · see citing works
Self-citationArtificial Intelligence and Behavioral Economics: A Bibliographic Analysis of Research Field
Aoujil, Z., Hanine, M., Flores, E. S., Samad, M. A., & Ashraf, I. (2023) · IEEE Access · doi:10.1109/access.2023.3339778 · Cited by 47 · see citing works
Evaluation of the hypothesis that phasic dopamine constitutes a cached-value signal
Sharpe, M. J., & Schoenbaum, G. M. (2017) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2017.12.002 · Cited by 21 · see citing works
Neural correlates of subjective CS/UCS association in appetitive conditioning
León, I. T., Kruse, O., Tobias Stalder, A., Stark, R. M., & Klucken, T. (2018) · Human Brain Mapping · doi:10.1002/hbm.23940 · Cited by 22 · see citing works
Ferrari-Toniolo, S., Bujold, P. M., Fabian Grabenhorst, B., Báez-Mendoza, R., & Schultz, W. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.0955-20.2020 · Cited by 23 · see citing works
Self-citation
- Authors
- Lak, A., Stauffer, W. R., & Schultz, W.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2014-01-22
- Volume
- 111
- Issue
- 6
- Pages
- 2343-2348
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC3926061
- OpenAlex
- W2010297727
- Cited by
- 257
- References
- 51
- Citations by year
- 2026: 8 · 2025: 19 · 2024: 17 · 2023: 23 · 2022: 16 · 2021: 24 · 2020: 15 · 2019: 22 · 2018: 24 · 2017: 26 · 2016: 26 · 2015: 24 · 2014: 13
- FWCI
- 13.55
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Licence
- other-oa
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- dopamine, prediction error, subjective value, economic choice, reward magnitude, risk, reward type, single-unit recording, primate decision making, value integration, reinforcement learning
- MeSH terms
- Animals, Dopamine, Macaca mulatta, Male, Neurons, Probability, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
