Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 801–900 of 1,007
Wiederhofer, J. (2024) · Springer eBooks · doi:10.1007/978-3-658-46275-8_2 · Cited by 42
A stable, distributed code for cue value in mouse cortex during reward learning
Ottenheimer, D. J., Hjort, M. M., Bowen, A. J., Steinmetz, N. A., & Stuber, G. D. (2023) · eLife · doi:10.7554/elife.84604 · Cited by 44
Neuronal signals for reward risk in frontal cortex
Schultz, W., O’Neill, M., Tobler, P. N., & Kobayashi, S. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06256.x · Cited by 36
Self-citationIoannou, A., Demetriou, S., & Mama, M. (2014) · American Journal of Distance Education · doi:10.1080/08923647.2014.926780 · Cited by 45
Lole, L., Gonsalvez, C. J., & Barry, R. J. (2014) · Clinical Neurophysiology · doi:10.1016/j.clinph.2014.10.011 · Cited by 45
“Within this system, the basal ganglia are hypothesised to monitor outcomes based on previously learned expectations, and in turn, stimulate dopamine (DA) levels; accordingly, rewarding outcomes cause DA to be released, leading to feelings of pleasure, whereas worse-than-expected outcomes are associated with reduced DA…”
BackgroundImpact of Size and Delay on Neural Activity in the Rat Limbic Corticostriatal System
Roesch, M. R., & Bryden, D. W. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00130 · Cited by 42
“…the presentation of unconditioned and conditioned stimuli (Mirenowicz and Schultz, 1994; Montague et al., 1996; Hollerman and Schultz, 1998a,b; Waelti et al., 2001; Fiorillo et al., 2003; Tobler et al., 2003; Nakahara et al., 2004; Bayer and Glimcher, 2005; Pan et al., 2005; Morris et al., 2006).”
BackgroundSurprise! Dopamine signals mix action, value and error
Collins, A. G. E., & Frank, M. J. (2015) · Nature Neuroscience · doi:10.1038/nn.4207 · Cited by 44
Linnet, J. (2019) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2019.109802 · Cited by 49
Tank, A., & Farrell, A. M. (2021) · European Accounting Review · doi:10.1080/09638180.2020.1866634 · Cited by 48
“(Fiorillo et al., 2003) d Managerial accounting Comparison and evaluation of expected (i.”
BackgroundNavigating complex decision spaces: Problems and paradigms in sequential choice.
Walsh, M. M., & Anderson, J. R. (2014) · Psychological Bulletin · doi:10.1037/a0033455 · Cited by 38
Challenges of Interpreting Frontal Neurons during Value-Based Decision-Making
Wallis, J. D., & Rich, E. L. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00124 · Cited by 40
“A similar approach helped to delimit prediction errors in dopamine neurons (Fiorillo et al., 2003).”
BackgroundResultTan, C. O., & Bullock, D. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.0259-08.2008 · Cited by 41
Nonhuman gamblers: lessons from rodents, primates, and robots
Paglieri, F., Addessi, E., Petrillo, F. D., Laviola, G., Mirolli, M., Parisi, D., Petrosino, G., Ventricelli, M., Zoratto, F., & Adriani, W. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00033 · Cited by 49
“…best of our knowledge, no neural models have been developed so far to explain pathological gambling, although there is evidence of a role of midbrain dopamine in the coding of reward uncertainty (Fiorillo et al., 2003), thus suggesting an influence of the dopaminergic system on risk-taking behavior.”
BackgroundNeural Evidence for Adaptive Strategy Selection in Value-Based Decision-Making
Gluth, S., Rieskamp, J., & Büchel, C. (2013) · Cerebral Cortex · doi:10.1093/cercor/bht049 · Cited by 46
“the dopaminoceptive system (Fiorillo et al. 2003; Preuschoff et al. 2006; Bach and Dolan 2012).”
Dopamine prevents muscarinic-induced decrease of glutamate release in the auditory cortex
Atzori, M., Kanold, P. O., Pineda, J. C., Flores‐Hernández, J., & Paz, R. (2005) · Neuroscience · doi:10.1016/j.neuroscience.2005.05.005 · Cited by 41
Dopamine, inference, and uncertainty
Gershman, S. J. (2017) · Neural Computation · doi:10.1162/neco_a_01023 · Cited by 40
Goldstein Ferber, S., Shoval, G., Zalsman, G., Mikulincer, M., & Weller, A. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.680403 · Cited by 40
Reinforcement learning detuned in addiction: integrative and translational approaches
Groman, S. M., Thompson, S. L., Lee, D., & Taylor, J. R. (2021) · Trends in Neurosciences · doi:10.1016/j.tins.2021.11.007 · Cited by 42
Gheza, D., Paul, K., & Pourtois, G. (2017) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2017.11.013 · Cited by 41
“Although remaining largely speculative, this break-up effect might be caused by a phasic dip or transient pausing in dopaminergic firing, as the RL-theory would suggest (Fiorillo et al., 2003; Schultz, 2013; Warren and Holroyd, 2012).”
Lin, W. C., Liu, C. J., Kosillo, P., Tai, L.-H., Galarce, E. M., Bateup, H. S., Lammel, S., & Wilbrecht, L. (2022) · Current Biology · doi:10.1016/j.cub.2022.06.089 · Cited by 42
Dopaminergic Balance between Reward Maximization and Policy Complexity
Parush, N., Tishby, N., & Bergman, H. (2011) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2011.00022 · Cited by 39
“…that phasic dopamine activity encodes the mismatch between prediction and reality or the RL temporal difference (TD) error signal (Schultz et al., 1997; Dayan and Balleine, 2002; Fiorillo et al., 2003; Satoh et al., 2003; Morris et al., 2004; Nakahara et al., 2004; Bayer and Glimcher, 2005).”
BackgroundNeuroethology of reward and decision making
Watson, K. K., & Platt, M. L. (2008) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2008.0159 · Cited by 42
“Moreover, dopamine neuron responses scale with both reward magnitude and reward probability (Fiorillo et al. 2003; Tobler et al. 2005).”
BackgroundA behavioral genetics approach to understanding D1 receptor involvement in phasic dopamine signaling
Wall, V., Parker, J. G., Fadok, J. P., Darvas, M., Zweifel, L. S., & Palmiter, R. D. (2010) · Molecular and Cellular Neuroscience · doi:10.1016/j.mcn.2010.09.011 · Cited by 43
“The magnitude of the cue-elicited phasic DA response is influenced by the size of the reward, the time to reward delivery and the probability of receiving a reward (Fiorillo et al., 2003, 2008).”
BackgroundNeural processing of reward magnitude under varying attentional demands
Stoppel, C. M., Boehler, C. N., Strumpf, H., Heinze, H., Hopf, J., & Schoenfeld, M. A. (2011) · Brain Research · doi:10.1016/j.brainres.2011.01.095 · Cited by 45
“…also conveys information about the magnitude (D'Ardenne et al., 2008; Knutson et al., 2001; Tobler et al., 2005), probability (Abler et al., 2006; Fiorillo et al., 2003) and delay (Fiorillo et al., 2008; McClure et al., 2007) of the expected reward. ico.boehler@duke.edu (C.N. Boehler), d.ovgu.de…”
Pavlovian-Instrumental Interaction in ‘Observing Behavior’
Beierholm, U. R., & Dayan, P. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000903 · Cited by 35
“Figure 4b shows modelled dopamine activity in the variable engagement temporal difference model (here, negative prediction errors have been compressed compared with positive ones, see methods; [33,34]).”
MethodAhn, S., & Phillips, A. G. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0526-9 · Cited by 39
“This degree of uncertainty may be highly compatible with the optimal conditions for activating midbrain DA neurons (Fiorillo et al. 2003), which in turn would result in a sustained increase in DA release in the NAc throughout a period of random reinforcement.”
BackgroundRisk and Reward Preferences under Time Pressure
Nursimulu, A., & Bossaerts, P. L. (2013) · European Finance Review · doi:10.1093/rof/rft013 · Cited by 49
Hippocampal neural activity reflects the economy of choices during goal‐directed navigation
Tryon, V. L., Penner, M. R., Heide, S. W., King, H. O., Larkin, J., & Mizumori, S. J. Y. (2017) · Hippocampus · doi:10.1002/hipo.22720 · Cited by 39
“For instance, dopamine cells signal the novel presentation of rewards, as well as the cues that predict future rewards (Schultz et al., 1997; Fiorillo et al., 2003).”
InfluentialHigh-frequency, short-latency disinhibition bursting of midbrain dopaminergic neurons
Lobb, C. J., Wilson, C. J., & Paladini, C. A. (2011) · Journal of Neurophysiology · doi:10.1152/jn.01076.2010 · Cited by 38
Working Memory and Anticipatory Set Modulate Midbrain and Putamen Activity
Yu, Y., FitzGerald, T. H. B., & Friston, K. J. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.1176-13.2013 · Cited by 36
“Fiorillo et al. (2003) have demonstrated that the level of tonic dopamine firing varies with the uncertainty about future events.”
BackgroundOngoing behavior predicts perceptual report of interval duration
Gouvêa, T. S., Monteiro, T., Soares, S., Atallah, B. V., & Paton, J. J. (2014) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2014.00010 · Cited by 49
“…parietal cortex (Janssen and Shadlen, 2005), prefrontal cortex (Fuster, 2001; Kim et al., 2013), the basal ganglia (Maricq and Church, 1983; Matell and Meck, 2000, 2004; Fiorillo et al., 2003; Meck et al., 2008; Jin et al., 2009; Adler et al., 2012), and the cerebellum (Ivry and Keele, 1989).”
BackgroundCognitive impulsivity in Parkinson's disease patients: Assessment and pathophysiology
Robert, G. H., Drapier, D., Vérin, M., Millet, B., Azulay, J., & Blin, O. (2009) · Movement Disorders · doi:10.1002/mds.22836 · Cited by 40
Neural and neurochemical basis of reinforcement-guided decision making
Khani, A., & Rainer, G. (2016) · Journal of Neurophysiology · doi:10.1152/jn.01113.2015 · Cited by 35
“…range from motivation (Satoh et al. 2003), reward magnitude (Calaminus and Hauber 2009), valence (Koob 1996), and probability and uncertainty (Fiorillo et al. 2003) to reward memory (Yamagata et al. 2015) and are integrated with various other information regarding other aspects of the…”
BackgroundCategory representation and generalization in the prefrontal cortex
Pan, X., & Sakagami, M. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07981.x · Cited by 44
The Role of the Rat Medial Prefrontal Cortex in Adapting to Changes in Instrumental Contingency
Coutureau, E., Esclassan, F., Di Scala, G., & Marchand, A. R. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033302 · Cited by 30
“Moreover, variability of delays between action and reward induces ramping of dopamine activity [44] and the mPFC may be especially important in detecting these signals.”
BackgroundDopamine Modulates the Neural Representation of Subjective Value of Food in Hungry Subjects
Medic, N., Ziauddeen, H., Vestergaard, M. D., Henning, E., Schultz, W., Farooqi, I. S., & Fletcher, P. C. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.2051-14.2014 · Cited by 38
Self-citationBayesian Models of Brain and Behaviour
Penny, W. (2012) · ISRN Biomathematics · doi:10.5402/2012/785791 · Cited by 35
Alves, F. D. S., Bakker, G., Schmitz, N., Abeling, N. G. G. M., Hasler, G., van der Meer, J. N., Nederveen, A. J., de Haan, L., LINSZEN, D. H., & van Amelsvoort, T. (2013) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2013.06.008 · Cited by 38
Bradberry, C. W., & Rubino, S. R. (2004) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300386 · Cited by 41
“The work of Schultz and co-workers (Schultz, 1998; Tremblay et al, 1998; Schultz and Dickinson, 2000) has highlighted the important role of DA neuronal activation during contingency learning or when the outcome is not certain (Fiorillo et al, 2003).”
BackgroundExpected Reward Value and Reward Uncertainty Have Temporally Dissociable Effects on Memory Formation
Stanek, J. K., Dickerson, K. C., Chiew, K. S., Clement, N. J., & Adcock, R. A. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01411 · Cited by 39
“, 2003; Tobler, Fiorillo, & Schultz, 2005), whereas a slower, anticipatory sustained response has been reported to be associated with reward uncertainty (Fiorillo et al., 2003).”
BackgroundInfluentialSmith, A. J., Becker, S., & Kapur, S. (2005) · Neural Computation · doi:10.1162/0899766053011546 · Cited by 37
Dealing with uncertain situations
Alquist, J. L., & Baumeister, R. F. (2023) · The Journal of Positive Psychology · doi:10.1080/17439760.2023.2282781 · Cited by 42
“Fiorillo et al. (2003) trained monkeys to recognize visual cues associated with different probabilities of receiving a liquid reward.”
Ting, C.-C., Yu, C.-C., Maloney, L. T., & Wu, S.-W. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.3161-14.2015 · Cited by 39
Ishii, H., Ohara, S., Tobler, P. N., Tsutsui, K., & Iijima, T. (2014) · Neuroscience Research · doi:10.1016/j.neures.2014.11.009 · Cited by 41
Self-citationKoenig, S., Uengoer, M., & Lachnit, H. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00266 · Cited by 39
“There is physiological evidence from single cell recordings in the monkey that the dopaminergic midbrain computes values of both, expectancy (associative strength) and uncertainty (prediction error) in order to engage in learning and decision making (Fiorillo et al., 2003; Schultz et al., 2008).”
BackgroundBehavioral Modulation by Spontaneous Activity of Dopamine Neurons
Ichinose, T., Tanimoto, H., & Yamagata, N. (2017) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2017.00088 · Cited by 38
“These dopamine fluctuations are suggested to represent animal states, such as sleep/wake, or motivational state of the animal (Fiorillo et al., 2003; Dahan et al., 2007; Hamid et al., 2016).”
BackgroundSerenko, A., & Turel, O. (2020) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/3410977.3410982 · Cited by 44
“Subsequently, an individual starts paying closer attention to the stimuli and subsequent actions, which produces a stronger stimuli-behavioral response link (Fiorillo, Tobler, & Schultz, 2003).”
BackgroundEffects of methylphenidate on reinforcement learning depend on working memory capacity
Kandroodi, M. R., Cook, J. L., Swart, J. C., Froböse, M. I., Geurts, D. E. M., Vahabie, A., Ahmadabadi, M. N., Cools, R., & den Ouden, H. E. M. (2021) · Psychopharmacology · doi:10.1007/s00213-021-05974-w · Cited by 38
Surgeons’ Attitudes Are Associated With Reoperation and Readmission Rates
Kadzielski, J. J., McCormick, F., Herndon, J. H., Rubash, H. E., & Ring, D. (2014) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-014-3687-6 · Cited by 37
Pankevich, D. E., Cherry, J. A., & Baum, M. J. (2006) · Physiology & Behavior · doi:10.1016/j.physbeh.2006.01.020 · Cited by 37
“Dopamine neurons are also activated by conditioned cues that predict a reward [47] [14], and show graded responses to rewards of different value [48].”
BackgroundA neuroeconomic theory of the decision process
Dickhaut, J. W., Rustichini, A., & Smith, V. L. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912500106 · Cited by 45
“Arriving at internal representations of elements of NTCP, such as priors and revised probabilities, has been found to engage the limbic system and orbital frontal cortex (14, 15).”
BackgroundNair, A. G., Bhalla, U. S., & Kotaleski, J. H. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005080 · Cited by 38
“Amplitude of the striatal dopamine represents the reward prediction error and it could vary depending on the difference between the expected and the acquired reward [62].”
BackgroundManagement of impulse control disorders with deep brain stimulation: A double-edged sword
Kasemsuk, C., Oyama, G., & Hattori, N. (2017) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2017.01.019 · Cited by 40
Vorhold, V. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1399.006 · Cited by 38
McGinty, V. B., Hayden, B. Y., Heilbronner, S. R., Dumont, E. C., Graves, S. M., Mirrione, M. M., du Hoffmann, J., Sartor, G. C., España, R. A., Millan, E. Z., et al. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2011.07.036 · Cited by 36
Aberrant orbitofrontal cortex reactivity to erotic cues in compulsive sexual behavior disorder
Golec, K., Draps, M., Stark, R., Pluta, A., & Gola, M. (2021) · Journal of Behavioral Addictions · doi:10.1556/2006.2021.00051 · Cited by 36
Yohn, S. E., Galbraith, J., Calipari, E. S., & Conn, P. J. (2019) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.8b00601 · Cited by 36
Implicit Value Updating Explains Transitive Inference Performance: The Betasort Model
Jensen, G., Muñoz, F., Alkan, Y., Ferrera, V. P., & Terrace, H. S. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004523 · Cited by 38
What is dopamine doing in model-based reinforcement learning?
Akam, T., & Walton, M. E. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.10.010 · Cited by 46
Bauch, E. M., Rausch, V. H., & Bunzeck, N. (2014) · Human Brain Mapping · doi:10.1002/hbm.22497 · Cited by 39
“3A), which mirrors well-known coding strategies of midbrain dopamine neurons during reward anticipation (Fiorillo et al., 2003; Schultz et al., 1997; Tobler et al., 2005).”
BackgroundThe effect of exposure to drugs on the processing of natural rewards
Anselme, P. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2008.10.002 · Cited by 43
“Therefore, uncertain events are predictable (they can be anticipated), with a probability of occurrence greater than zero and less than one, and there is evidence that they stimulate dopamine production in the ventral midbrain area (Fiorillo et al., 2003).”
BackgroundStriatal topography of probability and magnitude information for decisions under uncertainty
Berns, G. S., & Bell, E. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.11.008 · Cited by 39
“How do individual decision makers combine the two dimensions of magnitude and probability? The single system hypothesis suggests that a combined metric of expected utility, including both probability and magnitude, exists throughout brain systems that play a role in decision makingbut particularly in reward-related re…”
BackgroundLuque, D., Vadillo, M. A., Pelley, M. E. L., & Beesley, T. (2017) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2016.1188407 · Cited by 39
Accumbal Dopamine Release Tracks the Expectation of Dopamine Neuron-Mediated Reinforcement
Covey, D. P., & Cheer, J. F. (2019) · Cell Reports · doi:10.1016/j.celrep.2019.03.055 · Cited by 41
“Violations of expectations, such as a change in the magnitude or probability of reward, are reflected in reward-and cue-evoked DA signals (Beyene et al., 2010; Day et al., 2010; Fiorillo et al., 2003; Gan et al., 2010; Tobler et al., 2005).”
Coello, Y., Quesque, F., Gigliotti, M.-F., Ott, L., & Bruyelle, J.-L. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196874 · Cited by 35
Language statistical learning responds to reinforcement learning principles rooted in the striatum
Orpella, J., Herrero, E. M., Ripollés, P., Marco‐Pallarés, J., & de Diego‐Balaguer, R. (2021) · PLoS Biology · doi:10.1371/journal.pbio.3001119 · Cited by 43
“Under this light, our reported pattern of activity in the ventral striatum is consistent with the gradual transfer over learning of prediction error–related dopaminergic responses from rewarding to predictive stimuli as found in classic conditioning paradigms [53,54].”
ResultRational inattention and tonic dopamine
Mikhael, J. G., Lai, L., & Gershman, S. J. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008659 · Cited by 40
A reinforcement learning approach to gait training improves retention
Hasson, C. J., Manczurowsky, J., & Yen, S. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00459 · Cited by 39
“Dopamine release increases with greater reward uncertainty in both monkeys (Fiorillo et al., 2003) and humans (Preuschoff et al.”
BackgroundFörderung früher mathematischer Kompetenzen
Hauser, B., Vogt, F., Стеблер, Р., & Rechsteiner, K. (2014) · Frühe Bildung · doi:10.1026/2191-9186/a000144 · Cited by 35
A Comparison of the Subsecond Dynamics of Neurotransmission of Dopamine and Serotonin
Jennings, K. A. (2013) · ACS Chemical Neuroscience · doi:10.1021/cn4000605 · Cited by 36
Lole, L., Gonsalvez, C. J., Barry, R. J., & Blasio, F. M. D. (2013) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2013.06.011 · Cited by 38
“…system, the basal ganglia monitor outcomes and stimulate dopamine levels: accordingly, rewarding outcomes cause dopamine to be released, leading to feelings of pleasure, whereas worse-than-expected outcomes are associated with reduced dopamine transmission (Fiorillo et al., 2003; Schultz, 2007).”
The placebo response as a reward mechanism
Lidstone, S. C., de la Fuente‐Fernández, R., & Stoessl, A. J. (2005) · Seminars in Pain Medicine · doi:10.1016/j.spmd.2005.02.004 · Cited by 38
The Brain as the Original Accounting Institution
Dickhaut, J. W. (2009) · The Accounting Review · doi:10.2308/accr.2009.84.6.1703 · Cited by 35
An Open Source Syringe Pump Controller for Fluid Delivery of Multiple Volumes
Amarante, L. M., Newport, J. M., Mitchell, M., Wilson, J., & Laubach, M. (2019) · eNeuro · doi:10.1523/eneuro.0240-19.2019 · Cited by 35
“As such, published neuroscience studies of reward magnitude coding have confounded fluid volume with reward delivery time (Fiorillo et al., 2003; Bermudez and Schultz, 2010) through using longer solenoid valve openings to give rewards of different sizes to animals.”
BackgroundNeural architecture of choice behaviour in a concurrent interval schedule
Kalenscher, T., Diekamp, B., & Güntürkün, O. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.03006.x · Cited by 44
From uncertainty to reward: BOLD characteristics differentiate signaling pathways
Abler, B., Herrnberger, B., Grön, G., & Spitzer, M. (2009) · BMC Neuroscience · doi:10.1186/1471-2202-10-154 · Cited by 37
“Particularly the tonic dopamine response linked to reward uncertainty by Fiorillo et al [3] is modulated by limbic inputs from hippocampus and basolateral amygdala [25,26].”
BackgroundResultInfluentialBeyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning
Ballard, I. C., Miller, E. M., Piantadosi, S. T., Goodman, N. D., & McClure, S. M. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx259 · Cited by 38
“2015) but has never been able to account for the full repertoire of midbrain dopamine neuronal firing patterns (Redgrave et al. 1999; Fiorillo et al. 2003; Bromberg-Martin and Hikosaka 2009; Bromberg-Martin et al. 2010; Berridge 2012).”
BackgroundHeterogeneity of dopamine release sites in health and degeneration
Lebowitz, J. J., & Khoshbouei, H. (2019) · Neurobiology of Disease · doi:10.1016/j.nbd.2019.104633 · Cited by 36
Dunsmoor, J. E., & LaBar, K. S. (2012) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2012.02.003 · Cited by 38
Ray, N. J., & Strafella, A. P. (2010) · Clinical EEG and Neuroscience · doi:10.1177/155005941004100208 · Cited by 36
From biophysics to cognition: reward-dependent adaptive choice behavior
Soltani, A., & Wang, X. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.003 · Cited by 34
“In value-based choice tasks, many studies have shown that prior knowledge about reward probability of response alternatives modulates neural activity in multiple brain regions, including midbrain DA neurons [53], caudate [54], posterior parietal cortex (area LIP) [55], and various”
BackgroundWealth, consumption and happiness
Ahuvia, A. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511490118.009 · Cited by 38
Multiple Mechanisms for Processing Reward Uncertainty in the Primate Basal Forebrain
Ledbetter, N. M., Chen, C. D., & Monosov, I. E. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.1123-16.2016 · Cited by 33
“…integration of value and uncertainty in BF subregions is supported by monoamine transmission (and other inputs) from the brainstem and hypothalamus (Fiorillo et al., 2003; Watabe-Uchida et al., 2012; Ogawa et al., 2014; Cohen et al., 2015), but more anatomical studies are required to support or…”
BackgroundDissecting Impulsivity: Brain Mechanisms and Neuropsychiatric Implications
Robbins, T. W., & Dalley, J. W. (2017) · Nebraska Symposium on Motivation/The Nebraska symposium on motivation · doi:10.1007/978-3-319-51721-6_7 · Cited by 37
Cheng, J., & Feenstra, M. G. P. (2006) · Learning & Memory · doi:10.1101/lm.1806 · Cited by 36
Dopaminergic medication increases reliance on current information in Parkinson’s disease
Vilares, I., & Kording, K. P. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0129 · Cited by 33
Why is the Rescorla-Wagner model so influential?
Soto, F. A., Vogel, E. H., Uribe-Bahamonde, Y. E., & Perez, O. D. (2023) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2023.107794 · Cited by 36
Bakhurin, K. I., Hughes, R. N., Jiang, Q., Hossain, M. A., Gutkin, B. S., Fallon, I. P., & Yin, H. H. (2025) · Nature Communications · doi:10.1038/s41467-025-64132-4 · Cited by 36
“PhasicDA signaling is known to change systematically with rewardsize manipulations 29,30 .”
Risk-taking and pathological gambling behavior in Huntington’s disease
Kalkhoven, C., Sennef, C., Peeters, A., & van den Bos, R. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00103 · Cited by 38
“It should be noted that outcome variability, including both wins and losses, may be crucial to the development of gambling addiction, as it presents a variable intermittent pattern of reinforcement, which is the most powerful form of instrumental/classic conditioning (Sharpe, 2002; Fiorillo et al., 2003).”
BackgroundKoenig, S., Kadel, H., Uengoer, M., Schubö, A., & Lachnit, H. (2017) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2017.00128 · Cited by 35
“, 2001) and code for prediction errors, reward value, and reward uncertainty (Fiorillo et al., 2003; Schultz et al., 2008; Schultz, 2016).”
BackgroundNoradrenergic contributions to cue-driven risk-taking and impulsivity
Chernoff, C. S., Hynes, T. J., & Winstanley, C. A. (2021) · Psychopharmacology · doi:10.1007/s00213-021-05806-x · Cited by 34
Sex Differences in Behavioral Responding and Dopamine Release during Pavlovian Learning
Lefner, M. J., Dejeux, M. I., & Wanat, M. J. (2022) · eNeuro · doi:10.1523/eneuro.0050-22.2022 · Cited by 36
“, 2010), reward probability (Fiorillo et al., 2003; Hart et al., 2015), and reward rate (Fonzi et al.”
BackgroundIntegrating explanations of tax evasion and avoidance
Braithwaite, V. A., & Wenzel, M. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511490118.013 · Cited by 45
Neuroeconomics: making risky choices in the brain
Lee, D. (2005) · Nature Neuroscience · doi:10.1038/nn0905-1129 · Cited by 38
Delayed monitoring of accuracy errors compared to commission errors in ACC
Bediou, B., Koban, L., Rosset, S., Pourtois, G., & Sander, D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.010 · Cited by 36
“Such a quantitative coding of the magnitude of an error would be consistent with a fi ne-tuned prediction error coding system (Fiorillo et al., 2003; Potts et al., 2010; Yasuda et al., 2004) and may depend on fl exible reinforcement learning brain mechanisms (Holroyd et al., 2002; Nieuwenhuis et al.,…”
Mitz, A. R. (2005) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2005.07.012 · Cited by 35
Goh, J. O. S., Su, Y.-S., Tang, Y.-J., McCarrey, A. C., Tereshchenko, A., Elkins, W., & Resnick, S. M. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1386-16.2016 · Cited by 34
“Our results are consistent with an increase in neural activity to increasing stimulus value in the frontostriatal areas (Schultz et al., 1997; Breiter et al., 2001; Knutson et al., 2001; Fiorillo et al., 2003; Preuschoff et al., 2006; Tobler et al., 2007; Tom et al., 2007).”
BackgroundResultTzur, G., & Berger, A. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.11.041 · Cited by 35
Neural Systems Underlying Reward and Approach Behaviors in Childhood and Adolescence
Galván, A. (2013) · Current topics in behavioral neurosciences · doi:10.1007/7854_2013_240 · Cited by 41
Grossberg, S., Bullock, D., & Dranias, M. R. (2008) · Behavioral Neuroscience · doi:10.1037/a0012808 · Cited by 32
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.