The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Adaptive Coding of Reward Value by Dopamine Neurons
Abstract
It is important for animals to estimate the value of rewards as accurately as possible. Because the number of potential reward values is very large, it is necessary that the brain's limited resources be allocated so as to discriminate better among more likely reward outcomes at the expense of less likely outcomes. We found that midbrain dopamine neurons rapidly adapted to the information provided by reward-predicting stimuli. Responses shifted relative to the expected reward value, and the gain adjusted to the variance of reward value. In this way, dopamine neurons maintained their reward sensitivity over a large range of reward values.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Neurons in MBD nuclei are believed to encode reward prediction errors (RPEs).45”
From Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning (2026) · cites this paper as [45]
In the citing paper
“As is the case for dopamine neurons, 46 reward-related BOLD signals should maintain their sensitivity over a large range of reward values.”
From Increased neural responses to reward in adolescents and young adults with attention-deficit/hyperactivity disorder and their unaffected siblings (2015) · cites this paper as [46]
In the citing paper
“Indeed, reward-related responses adapt to the context in which a reward is presented 46, in the sense that they depend on a combination of the overall expected value and their variance rather than the absolute magnitude of reward value.”
From Increased neural responses to reward in adolescents and young adults with attention-deficit/hyperactivity disorder and their unaffected siblings (2015) · cites this paper as [46]
In the citing paper
“Yet, both reward magnitude and reward probability are coded relatively rather than absolutely. 46”
From Increased neural responses to reward in adolescents and young adults with attention-deficit/hyperactivity disorder and their unaffected siblings (2015) · cites this paper as [46]
In the citing paper
“Activity in individual neurons in both habenula and VTA reflects both the value of a stimulus in terms of the reward that it predicts58–60 as well as VoI21,22,27,61”
From Interpretable abstractions of artificial neural networks predict behavior and neural activity during human information gathering (2026) · cites this paper as [59]
In the citing paper
“For example, the gain of RPE signals produced by dopamine neurons in the ventral tegmental area is modulated by the variance of reward value.100,101”
From Serotonin neurons modulate learning rate through uncertainty (2022) · cites this paper as [101]
In the citing paper
“Consistent with a prediction error signal, spiking activity of dopamine neurons increases with both reward magnitude and probability (Fiorillo et al., 2003; Tobler et al., 2005; Roesch et al., 2007).”
From Decision Making in Recurrent Neuronal Circuits (2008) · cites this paper as Tobler et al., 2005
In the citing paper
“Research in non-human primates, confirmed in humans, indicates that differences the probability with which a cue predicts a reward produce a quantitative change in the engagement of dopaminergic systems.46,47”
From Striatal sensitivity during reward processing in attention-deficit/hyperactivity disorder (2012) · cites this paper as [46]
Citations
Cites
Plum, F. (1960) · Archives of Neurology · doi:10.1001/archneur.1960.03840090124020 · Cited by 10,388
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationThe neural basis of drug craving: An incentive-sensitization theory of addiction
Robinson, T. (1993) · Brain Research Reviews · doi:10.1016/0165-0173(93)90013-p · Cited by 7,346
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationClassical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Taussig, M. (1991) · Routledge eBooks · doi:10.4324/9780203397244 · Cited by 2,403
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
Self-citationNeurobehavioural mechanisms of reward and motivation
Robbins, T. W., & Everitt, B. J. (1996) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(96)80077-8 · Cited by 1,187
Animal intelligence; experimental studies
Thorndike, E. L. (1911) · doi:10.5962/bhl.title.55072 · Cited by 1,100
Predictive coding: a fresh view of inhibition in the retina
Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1982.0085 · Cited by 1,042
Research works printed but not linked: 3
- I. P. Pavlov Conditional Reflexes (Oxford Univ. Press Oxford 1927).
- H. B. Barlow, in Sensory Communication, W. A. Rosenblith, Ed. (MIT Press, Cambridge, MA, 1961), pp. 217–234.
- S. B. Laughlin, Z. Naturforsch. Teil C Biochem. Biophys. Biol. Virol.36, 910 (1981).
Cited by
Sorted by citations
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
“DA neurons can also adapt their reward signals based on higherorder statistics of the reward distribution, such as scaling prediction error signals based on their expected variance (Tobler et al., 2005) and ‘‘spontaneously recovering’’ their responses to extinguished reward cues (Pan et al., 2008).”
BackgroundThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
“…models of reinforcement learning, which incorporates time more explicitly into expectations of reward as a series of future events ðV s t ð Þ ¼ h P i¼0 g i r tþ1 iÞ, (Bayer and Glimcher 2005; Daw et al. 2005; Dayan and Balleine 2002; Montague et al. 2004; Schultz 2002, 2006; Tobler et al. 2005a,b).”
BackgroundA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
“Similar risk and expected-value signals have been found in the midbrain dopamine system in electrophysiology studies in non-human primate”
BackgroundHabits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697
“…error) and motivational state in a context-dependent manner (Bayer & Glimcher 2005, Morris et al. 2004, Nakahara et al. 2004, Satoh et al. 2003, Tobler et al. 2005, Waelti et al. 2001), that they are specialized to respond in relation to positive but not aversive reinforcements, and that they…”
BackgroundBehavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637
Self-citationMultiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415
“It covaries with reward probability (Fiorillo et al. 2003, Morris et al. 2004) and the expected value of the reward distribution (Tobler et al. 2005) and is modulated by the motivation of the animal (Satoh et al. 2003), the time course of predictions (Nakahara et al. 2004), and the animal’s choice…”
BackgroundSelf-citationIs there a common molecular pathway for addiction?
Nestler, E. J. (2005) · Nature Neuroscience · doi:10.1038/nn1578 · Cited by 1,388
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
“Their activations disappear gradually with extinction (598), possibly due to inhibition from neighboring pars reticulata neurons activated during extinction (408).”
BackgroundInfluentialSelf-citationTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
“Larger phasic prediction-error signals for stimuli previously associated with higher rewards (Fiorillo et al. 2003; Tobler et al. 2005) would then result in faster responding to these cues.”
Dopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036
Self-citationSchultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028
Self-citationUnique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976
“Schultz and colleagues have successfully defined the computational roles of tonic and phasic discharge patterns of conventional DA midbrain neurons in reward-prediction error and uncertainty signaling (Fiorillo et al., 2003; Schultz, 2007; Tobler et al., 2005).”
The Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
“Additional studies have demonstrated that, when conditioned cues predict rewards with different magnitudes or probabilities, the cue-elicited dopaminergic response scales with magnitude or probability, as expected if it represents a cue-elicited prediction error (Fiorillo et al., 2003; Tobler et al., 2005).”
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
“Such signals are encoded quantitatively, whether the magnitud”
BackgroundThe short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
Corticostriatal-hypothalamic circuitry and food motivation: Integration of energy, action and reward
Kelley, A. E., Baldo, B. A., Pratt, W. E., & Will, M. J. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.066 · Cited by 752
Jhou, T. C., Fields, H. L., Baxter, M. G., Saper, C. B., & Holland, P. C. (2009) · Neuron · doi:10.1016/j.neuron.2009.02.001 · Cited by 696
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 775
“Impressively, even in sophisticated conditioning tasks such as blocking and appetitive conditioned inhibition, dopaminergic responses are in line with the predictions of TD learning (Tobler et al., 2003, 2005; Waelti, Dickinson, & Schultz, 2001).”
ResultTemporally precise in vivo control of intracellular signalling
Airan, R. D., Thompson, K. R., Fenno, L. E., Bernstein, H. L., & Deisseroth, K. A. (2009) · Nature · doi:10.1038/nature07926 · Cited by 715
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 733
Neural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692
Metal–organic frameworks as photoluminescent biosensing platforms: mechanisms and applications
Zhao, Y., Zeng, H., Zhu, X., Lu, W., & Li, D. (2021) · Chemical Society Reviews · doi:10.1039/d0cs00955e · Cited by 693
Decision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679
“Consistent with a prediction error signal, spiking activity of dopamine neurons increases with both reward magnitude and probability (Fiorillo et al., 2003; Tobler et al., 2005; Roesch et al., 2007).”
BackgroundUpdating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651
“Dopamine responses are influenced by previously acquired information about reward probability distributions [13].”
BackgroundSelf-citationDopamine signals for reward value and risk: basic and recent data
Schultz, W. (2010) · Behavioral and Brain Functions · doi:10.1186/1744-9081-6-24 · Cited by 653
“First, it could influence the scaling of the immediately following prediction error response by standard deviation immediately after the reward [23].”
InfluentialSelf-citationRegret and its avoidance: a neuroimaging study of choice behavior
Coricelli, G., Critchley, H., Joffily, M., O’Doherty, J. P., Angela Sirigu, B., & Dolan, R. J. (2005) · Nature Neuroscience · doi:10.1038/nn1514 · Cited by 673
Monitoring Rapid Chemical Communication in the Brain
Robinson, D. L., Hermans, A., Seipel, A. T., & Wightman, R. M. (2008) · Chemical Reviews · doi:10.1021/cr068081q · Cited by 672
The basal ganglia: learning new tricks and loving it
Graybiel, A. M. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.10.006 · Cited by 678
Leptin Regulation of the Mesoaccumbens Dopamine Pathway
Fulton, S., Pissios, P., Manchon, R. P., Stiles, L., Frank, L., Pothos, E. N., Maratos-Flier, E., & Flier, J. S. (2006) · Neuron · doi:10.1016/j.neuron.2006.09.006 · Cited by 644
“…incentive motivation (Berridge and Robinson, 1998; Kelley et al., 2005; Robinson et al., 2005; Salamone et al., 2005) and reward-related learning (Tobler et al., 2005), and is a major substrate underlying the locomotor-activating, rewarding and addictive effects of drugs of abuse (Wise, 2002;…”
Placebo and Nocebo Effects Are Defined by Opposite Opioid and Dopaminergic Responses
Scott, D. J., Stohler, C. S., Egnatuk, C. M., Wang, H., Koeppe, R. A., & Zubieta, J.-K. (2008) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2007.34 · Cited by 637
“In animal models, the tonic firing of mesolimbic DA cells increases with the expectation of a positive outcome (the receipt of a reward larger than expected) and is reduced when the expected outcome is less prominent than that predicted by initial cues.(23) Similar effects have been observed in recent work examining t…”
BackgroundMethodResultVentral Tegmental Area Neurons in Learned Appetitive Behavior and Positive Reinforcement
Fields, H. L., Hjelmstad, G. O., Margolis, E. B., & Nicola, S. M. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094341 · Cited by 632
“This increase in firing scales with the magnitude of reward received (e.g., vol- ume of juice; Tobler et al. 2005).”
BackgroundActivation of VTA GABA Neurons Disrupts Reward Consumption
van Zessen, R., Phillips, J. L., Budygin, E. A., & Stuber, G. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.016 · Cited by 646
“…display a transient high frequency burst of activity in response to the onset of reward predictive cues, followed by low frequency firing following reward delivery (Day et al., 2007; Mirenowicz and Schultz, 1996; Pan et al., 2005; Stuber et al., 2008; Tobler et al., 2005; Waelti et al., 2001).”
Rapid signalling in distinct dopaminergic axons during locomotion and reward
Howe, M. W., & Dombeck, D. A. (2016) · Nature · doi:10.1038/nature18942 · Cited by 618
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
“2b ) were analyzed for features characteristic of dopamine neuron”
MethodThe role of D2-autoreceptors in regulating dopamine neuron activity and transmission
Ford, C. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.01.025 · Cited by 630
“As both drugs of abuse and natural rewards increase dopamine firing and release (Phillips et al., 2003, Tobler et al., 2005, Bromberg-Martin and Hikosaka, 2009, Cohen et al., 2012), changes in chronic food availability also promote D2-receptor desensitization in the midbrain (Branch et al., 2013).”
BackgroundFrank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 596
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565
Representation of negative motivational value in the primate lateral habenula
Matsumoto, M., & Hikosaka, O. (2009) · Nature Neuroscience · doi:10.1038/nn.2233 · Cited by 547
More attention must be paid: The neurobiology of attentional effort
Sarter, M., Gehring, W. J., & Kozak, R. (2006) · Brain Research Reviews · doi:10.1016/j.brainresrev.2005.11.002 · Cited by 552
Neuroeconomics: cross-currents in research on decision-making
Sanfey, A. G., Loewenstein, G. F., McClure, S. M., & Cohen, J. D. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.01.009 · Cited by 571
Walsh, M. M., & Anderson, J. R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.008 · Cited by 577
“Neural firing rates and BOLD responses adapt to the range of outcomes such that maximum deviations from baseline remain constant regardless of absolute reward values (Bunzek et al., 2010; Nieuwenhuis et al., 2005b; Tobler et al., 2005).”
BackgroundAbsolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA
Bunzeck, N., & Düzel, E. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.021 · Cited by 503
Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making
Preuschoff, K. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00115 · Cited by 538
“, 2009), electrophysiological data of adaptive coding (Tobler et al., 2005) as well as computational models of risk learning (Dayan and Yu, 2006; Preuschoff and Bossaerts, 2007).”
BackgroundNeuronal basis of sequential foraging decisions in a patchy environment
Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2011) · Nature Neuroscience · doi:10.1038/nn.2856 · Cited by 547
“We speculate that such control may be implemented through neuromodulatory inputs to the dACC, perhaps via dopamine or norepinephrin”
BackgroundTheory of Mind: A Neural Prediction Problem
Koster-Hale, J., & Saxe, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.020 · Cited by 513
“…example of a neural “error” code: the rate of firing corresponds to any currently “new” (i.e., previously unpredictable) information about the value of coming reward, not to the actual value of any currently perceived stimulus (Bayer and Glimcher, 2005; Nakahara et al., 2004; Tobler et al., 2005).”
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472
Self-citationNeural Circuitry of Reward Prediction Error
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437
“…encode various features that determine reward value: probability (Fiorillo et al. 2003), magnitude (Bayer & Glimcher 2005, Bayer et al. 2007, Tobler et al. 2005), timing (Fiorillo et al. 2008, Hollerman et al. 1998, Kobayashi & Schultz 2008), and even subjective preference (Lak et al.…”
Individual differences in reward responding explain placebo-induced expectations and effects
Scott, D. J., Stohler, C. S., Egnatuk, C. M., Wang, H., Koeppe, R. A., & Zubieta, J.-K. (2007) · Neuron · doi:10.1016/j.neuron.2007.06.028 · Cited by 466
Double dissociation of value computations in orbitofrontal and anterior cingulate neurons
Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2961 · Cited by 485
“This feature of PFC neurons is in stark contrast to activity measured, for example, in dopamine neurons, which encode reward and reward prediction error at predictable times, using a unified coding schem”
MethodHow Placebos Change the Patient's Brain
Benedetti, F., Carlino, E., & Pollo, A. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.81 · Cited by 446
Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons
Ungless, M. A., & Grace, A. A. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.02.003 · Cited by 450
Val‐Laillet, D., Aarts, E., Weber, B., Ferrari, M., Quaresima, V., Stoeckel, L. E., Alonso‐Alonso, M., Audette, M. A., Malbert, C., & Stice, E. M. (2015) · NeuroImage Clinical · doi:10.1016/j.nicl.2015.03.016 · Cited by 466
“According to animal studies, firing of striatal and ventral pallidum DA neurons initially occurs in response to receipt of a novel palatable food, but after repeated pairings of palatable food intake and cues that signal impending receipt of that food, DA neurons begin firing in response to reward-predictive cues and…”
A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages
Sambrook, T. D., & Goslin, J. (2015) · Psychological Bulletin · doi:10.1037/bul0000006 · Cited by 411
Influence of Reward Delays on Responses of Dopamine Neurons
Kobayashi, S., & Schultz, W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 440
“Dopamine responses to conditioned stimuli Previous studies showed that dopamine neurons change their response to conditioned stimuli proportional to magnitude and probability of the associated reward (Fiorillo et al., 2003; Tobler et al., 2005).”
BackgroundResultSelf-citationYacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
“Prediction error signal scaled by magnitude Primate data have suggested that dopaminergic midbrain neurons should be able to signal the magnitude of a prediction error (Tobler et al., 2005).”
BackgroundLinking context with reward: a functional circuit from hippocampal CA3 to ventral tegmental area
Luo, A. H., Tahsili-Fahadan, P., Wise, R. A., Lupica, C. R., & Aston-Jones, G. (2011) · Science · doi:10.1126/science.1204622 · Cited by 425
Foster, D. J. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031538 · Cited by 375
“Both these effects resemble the pattern of dopamine signaling in response to rewards [prediction error effects (Schultz et al. 1997) and menu effects (Tobler et al. 2005), respectively], suggesting that dopamine might regulate replay, and reverse replay specifically.”
BackgroundNeural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356
“At stimulus onset, in the absence of reward and assuming that the prior expectation V(tstimulus 1) is zero (a common assumption based on recordings from dopaminergic neurons, which show that the neural activity at stimulus onset reflects a prediction error comparing the value of that stimulus to zero, rather than to t…”
BackgroundBeyond stereotypes of adolescent risk taking: Placing the adolescent brain in developmental context
Römer, D., Reyna, V. F., & Satterthwaite, T. D. (2017) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.07.007 · Cited by 466
“Adults experience a domain-general decline in verbatim cognitive skills starting in the third decade of life (Tucker-Drob, 2011; Lachman et al., 2014), although gist memory is conserved (e.g., Brainerd et al., 2009; Reyna, 2012).”
Basal ganglia orient eyes to reward
Hikosaka, O., Nakamura, K., & Nakahara, H. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00458.2005 · Cited by 377
The representation of economic value in the orbitofrontal cortex is invariant for changes of menu
Padoa‐Schioppa, C., & Assad, J. A. (2007) · Nature Neuroscience · doi:10.1038/nn2020 · Cited by 407
Arithmetic and local circuitry underlying dopamine prediction errors
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N. (2015) · Nature · doi:10.1038/nature14855 · Cited by 401
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
Adamantidis, A. R., Tsai, H.-C., Boutrel, B., Zhang, F., Stuber, G. D., Budygin, E. A., Tourino, C., Bonci, A., Deisseroth, K., & de Lecea, L. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.2246-11.2011 · Cited by 357
“Those signals update the predictive value of stimuli during associative learning for future reinforcement of behavioral actions (Fiorillo et al., 2003; Bayer and Glimcher, 2005; Tobler et al., 2005; Day et al., 2007; Schultz, 2007).”
BackgroundPrediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376
Dissecting the diversity of midbrain dopamine neurons
Roeper, J. (2013) · Trends in Neurosciences · doi:10.1016/j.tins.2013.03.003 · Cited by 395
IMAGING VALUATION MODELS IN HUMAN CHOICE
Montague, P. R., King‐Casas, B., & Cohen, J. D. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112903 · Cited by 382
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339
Irritability in Youths: A Translational Model
Brotman, M. A., Kircanski, K., Stringaris, A. K., Pine, D. S., & Leibenluft, E. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.16070839 · Cited by 381
Event-related potential studies of outcome processing and feedback-guided learning
San Martín, R. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00304 · Cited by 354
“This theory is based on research that implicates the basal ganglia and the midbrain dopamine (DA) system in reward prediction and reinforcement learning (Barto, 1995; Montague et al., 1996; Schultz et al., 1997; Schultz and Dickinson, 2000; Tobler et al., 2005; for a review see Schultz, 2002).”
BackgroundStamatakis, A. M., Jennings, J. H., Ung, R. L., Blair, G. A., Weinberg, R. J., Neve, R. L., Boyce, F., Mattis, J., Ramakrishnan, C., Deisseroth, K., et al. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.023 · Cited by 364
The nucleus accumbens as part of a basal ganglia action selection circuit
Nicola, S. M. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0510-4 · Cited by 383
“Third, dopamine neurons fire in response to stimuli that predict reward, and their firing is correlated with the degree to which the stimulus predicts reward magnitude or probability (Fiorillo et al. 2003; Kawagoe et al. 2004; Morris et al. 2004; Satoh et al. 2003; Schultz 1998; Takikawa et al. 2004; Tobler et al. 200…”
BackgroundMidbrain dopamine neurons control judgment of time
Soares, S., Atallah, B. V., & Paton, J. J. (2016) · Science · doi:10.1126/science.aah5234 · Cited by 365
Stocco, A., Lebiere, C. J., & Anderson, J. R. (2010) · Psychological Review · doi:10.1037/a0019077 · Cited by 383
The good, the bad and the neutral: Electrophysiological responses to feedback stimuli
Holroyd, C. B., Hajcak, G., & Larsen, J. T. (2006) · Brain Research · doi:10.1016/j.brainres.2005.12.015 · Cited by 350
Reverse replay of hippocampal place cells is uniquely modulated by changing reward
Ambrose, R. E., Pfeiffer, B. E., & Foster, D. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.07.047 · Cited by 292
“Interestingly, dopamine neurons of the ventral tegmental area display the same dynamical adaptation of neural output to the current range of reward magnitude as we observed in reverse replay (Tobler et al., 2005).”
Normalization is a general neural mechanism for context-dependent decision making
Louie, K., Khaw, M. W., & Glimcher, P. W. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1217854110 · Cited by 340
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340
How the Opinion of Others Affects Our Valuation of Objects
Campbell-Meiklejohn, D. K., Bach, D. R., Roepstorff, A., Dolan, R. J., & Frith, C. D. (2010) · Current Biology · doi:10.1016/j.cub.2010.04.055 · Cited by 339
The role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328
“That is, DA responds to the earliest predictor of the reward, a signal that can be modulated by magnitude (Tobler et al. 2005) and probability (Fiorillo et al.”
BackgroundDiekhof, E. K., Kaps, L., Falkai, P., & Gruber, O. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2012.02.007 · Cited by 332
Risk-Sensitivity in Sensorimotor Control
Braun, D. A., Nagengast, A. J., & Wolpert, D. M. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00001 · Cited by 330
“In support of the mean–variance approach, separate encoding of reward magnitude and risk has been reported in humans (Preuschoff et al., 2006; Tobler et al., 2007, 2009) as well as in non-human primates (Tobler et al., 2005).”
BackgroundEffects of Expectation on Placebo-Induced Dopamine Release in Parkinson Disease
Lidstone, S. C., Schulzer, M., Dinelle, K., Mak, E., Sossi, V., Ruth, T. J., de la Fuente‐Fernández, R., Phillips, A. G., & Stoessl, A. J. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.88 · Cited by 325
Neural systems implicated in delayed and probabilistic reinforcement
Cardinal, R. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.004 · Cited by 330
“DA neurons that innervate the Acb may fire in a manner related to reward probability (Fiorillo et al., 2003, 2005; Niv et al., 2005; Tobler et al., 2005) and the midbrain, the site of the cell bodies of these neurons, responds to stimulus uncertainty in humans (Aron et al.”
BackgroundDopamine neurons share common response function for reward prediction error
Eshel, N., Tian, J., Bukwich, M., & Uchida, N. (2016) · Nature Neuroscience · doi:10.1038/nn.4239 · Cited by 321
“Consistent with previous reports 23,29 , dopamine neurons monotonically increased their responses with increasing reward (Fig.”
Identifying the neural circuitry of alcohol craving and relapse vulnerability
Heinz, A., Beck, A., Grüsser, S. M., Grace, A. A., & Wrase, J. (2008) · Addiction Biology · doi:10.1111/j.1369-1600.2008.00136.x · Cited by 326
“The nucleus accumbens may thus act as a ‘sensory motor gateway’ (Tobler et al. 2005), which controls the effects of salient environmental stimuli on brain areas that regulate motor behaviour.”
BackgroundThe temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310
Self-citationBrain, emotion and decision making: the paradigmatic example of regret
Coricelli, G., Dolan, R. J., & Sirigu, A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.04.003 · Cited by 325
Reward-Predictive Cues Enhance Excitatory Synaptic Strength onto Midbrain Dopamine Neurons
Stuber, G. D., Klanker, M., de Ridder, B., Bowers, M. S., Joosten, R. N. J. M. A., Feenstra, M. G. P., & Bonci, A. (2008) · Science · doi:10.1126/science.1160873 · Cited by 319
Range-adapting representation of economic value in the orbitofrontal cortex
Padoa-Schioppa, C. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3751-09.2009 · Cited by 276
“neurons (Tobler et al., 2005) and in the anterior cingulate”
BackgroundScott, D., Heitzeg, M. M., Koeppe, R. A., Stohler, C. S., & Zubieta, J. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.2577-06.2006 · Cited by 298
“In this context, mesolimbic DA cells are engaged in the encoding of both reward value and its uncertainty, to appropriately orient the responses of the organism (Fiorillo et al., 2003; Hsu et al., 2005; Tobler et al., 2005).”
BackgroundDoes the brain calculate value?
Vlaev, I., Chater, N., Stewart, N., & Brown, G. D. A. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.09.008 · Cited by 312
Nucleus accumbens and impulsivity
Başar, K., Sesia, T., Groenewegen, H. J., Steinbusch, H. W. M., Visser-Vandewalle, V. E. R. M., & Temel, Y. (2010) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2010.08.007 · Cited by 289
“The firing of dopaminergic neurons has been shown to quantitatively code the magnitude, probability, and the combination of these features of the outcome (Samejima et al., 2005; Satoh et al., 2003; Tobler et al., 2005).”
Neurobiological Mechanisms of Placebo Responses
Zubieta, J., & Stohler, C. S. (2009) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.04424.x · Cited by 277
“It is also thought to respond to updates in reward expectations that depend on the emotional response to changing environmental information (so-called counterfactual comparisons) through its connections with the OFC and the AMY (Schultz 2006; Tobler et al. 2005; Tom et al. 2007).”
BackgroundSteiger, T. K., & Bunzeck, N. (2017) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2017.00001 · Cited by 275
“Finally, the SN/VTA and the NAcc are sensitive to reward magnitude, with higher rewards leading to more neural activation and more efficient behavior, such as faster RTs, in contrast to lower rewards (Knutson et al., 2001b; Tobler et al., 2005).”
BackgroundFood reward system: current perspectives and future research needs
Alonso‐Alonso, M., Woods, S. C., Pelchat, M. L., Grigson, P. S., Stice, E. M., Farooqi, I. S., Khoo, C. S., Mattes, R. D., & Beauchamp, G. K. (2015) · Nutrition Reviews · doi:10.1093/nutrit/nuv002 · Cited by 293
Anticipation of novelty recruits reward system and hippocampus while promoting recollection
Wittmann, B. C., Bunzeck, N., Dolan, R. J., & Düzel, E. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.06.038 · Cited by 265
“Another property of SN/VTA neuronal coding of reward outcome is adaptive coding (Tobler et al., 2005), which is characterized by a different level of responding to the same expected reward value depending on the alternative rewards available in each context.”
Dopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257
Self-citation
Metadata
- Authors
- Tobler, P. N., Fiorillo, C. D., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Schultz, Wolfram
- Venue
- Science
- Published
- 2005-03-11
- Volume
- 307
- Issue
- 5715
- Pages
- 1642-1645
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2098534820
- Cited by
- 1,276
- References
- 24
- Citations by year
- 2026: 36 · 2025: 32 · 2024: 47 · 2023: 56 · 2022: 53 · 2021: 72 · 2020: 57 · 2019: 69 · 2018: 63 · 2017: 65 · 2016: 74 · 2015: 58 · 2014: 72 · 2013: 76 · 2012: 72
- FWCI
- 22.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- reroDoc Digital Library (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, adaptive coding, midbrain, reward prediction, gain modulation, variance, reward sensitivity, neural adaptation
- MeSH terms
- Adaptation, Physiological, Animals, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Neurons, Photic Stimulation, Probability, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
