Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 901–1,000 of 1,007
Striatal action-learning based on dopamine concentration
Morris, G., Schmidt, R., & Bergman, H. (2009) · Experimental Brain Research · doi:10.1007/s00221-009-2060-6 · Cited by 36
“were performed with primates (Fiorillo et al. 2003; Nakahara et al. 2004; Morris et al. 2004, 2006; Tobler et al. 2005; Bayer and Glimcher 2005), and rats (Roesch et al.”
MethodAlves, F. D. S., Schmitz, N., Figee, M., Abeling, N. G. G. M., Hasler, G., van der Meer, J. N., Nederveen, A. J., de Haan, L., LINSZEN, D. H., & van Amelsvoort, T. (2010) · Journal of Psychopharmacology · doi:10.1177/0269881110367731 · Cited by 36
“…practice trials in each session, was set such that each subject would succeed on approximately 66% of his or her target responses, a rate based upon the conditioned reinforcement rate needed to maintain firing activity of dopamine neurons in monkeys performing similar tasks (Fiorillo et al., 2003).”
BackgroundElectrophysiological correlates of abused drugs: Relation to natural rewards
Deadwyler, S. A. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.05155.x · Cited by 33
The Neurobiological Foundations of Valuation in Human Decision Making Under Uncertainty
Bossaerts, P. L., Preuschoff, K., & Hsu, M. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00023-3 · Cited by 35
Feedback control of learning by the cerebello-olivary pathway
Rasmussen, A., & Hesslow, G. (2014) · Progress in Brain Research · doi:10.1016/b978-0-444-63356-9.00005-4 · Cited by 31
Lateral habenula deep brain stimulation for personalized treatment of drug addiction
Yadid, G., Gispan, I. H., & Lax, E. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00806 · Cited by 35
“Conversely, low frequency inputs from other brain circuits, including the VTA dopaminergic system, inhibit LHb activity following positive reward expectation (Fiorillo et al., 2003).”
BackgroundSensory regulation of dopaminergic cell activity: Phenomenology, circuitry and function
Overton, P. G., Vautrelle, N., & Redgrave, P. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.01.023 · Cited by 36
“However, most recent studies in the monkey have abandoned the use of simple visual stimuli in favor of colored, geometrically complex visual stimuli, to which midbrain DA neurons respond with activation when the stimuli are paired with reward (Waelti et al., 2001; Fiorillo et al., 2003; Morris et al., 2004; Tobler et…”
BackgroundInfluentialDichotomy between motor and cognitive functions of midbrain cholinergic neurons
Gut, N. K., & Mena‐Segovia, J. (2018) · Neurobiology of Disease · doi:10.1016/j.nbd.2018.09.008 · Cited by 32
Neural arbitration between social and individual learning systems
Diaconescu, A. O., Stecy, M., Kasper, L., Burke, C. J., Nagy, Z., Mathys, C. D., & Tobler, P. N. (2020) · eLife · doi:10.7554/elife.54051 · Cited by 34
“Primate studies found that sustained dopamine neuron activity signalled expected uncertainty (Fiorillo, 2003; Schultz, 2010; Schultz et al., 2008).”
BackgroundSelf-citationNeural Plasticity, Human Genetics, and Risk for Alcohol Dependence
Hill, S. Y. (2010) · International review of neurobiology · doi:10.1016/s0074-7742(10)91003-9 · Cited by 34
Addictive behaviors and Parkinson's disease
Witjas, T., Eusébio, A., Fluchère, F., & Azulay, J. (2012) · Revue Neurologique · doi:10.1016/j.neurol.2012.06.014 · Cited by 31
Oyarzún, J. P., Packard, P. A., de Diego‐Balaguer, R., & Fuentemilla, L. (2016) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2016.05.005 · Cited by 33
Dobbs, L. K., & Mark, G. P. (2008) · Neuroscience · doi:10.1016/j.neuroscience.2008.07.052 · Cited by 34
Dopaminergic Reward Pathways and Effects of Stress
Marinelli, M. (2007) · Elsevier eBooks · doi:10.1016/b978-012370632-4/50006-1 · Cited by 36
How is your mind-set? Proof of concept for the measurement of the level of emotional development
Sappok, T., Böhm, J., Birkner, J., Roth, G., & Heinrich, M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215474 · Cited by 29
Nigrostriatal dopamine signals sequence-specific action-outcome prediction errors
Hollon, N. G., Williams, E., Howard, C. D., Li, H., Traut, T. I., & Jin, X. (2021) · Current Biology · doi:10.1016/j.cub.2021.09.040 · Cited by 31
Culture, Mind, and Brain: Emerging Concepts, Models, and Applications
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108695374 · Cited by 32
Bratec, S. M., Xie, X., Schmid, G., Doll, A., Schilbach, L., Zimmer, C., Wohlschläger, A. M., Riedl, V., & Sorg, C. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.08.038 · Cited by 32
“In addition, given the role of prediction errors in attention allocation (Corlett et al., 2007; Fiorillo et al., 2003; Pearce and Hall, 1980), current results suggest that cognitive emotion regulation might enable us to enhance our attention to the relevant stimuli while decreasing aversive…”
A role for the lateral dorsal tegmentum in memory and decision neural circuitry
Redila, V. A., Kinzel, C. C., Jo, Y. S., Puryear, C. B., & Mizumori, S. J. Y. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.05.009 · Cited by 35
Learning to represent reward structure: A key to adapting to complex environments
Nakahara, H., & Hikosaka, O. (2012) · Neuroscience Research · doi:10.1016/j.neures.2012.09.007 · Cited by 25
Schulz, D., Sergeeva, O. A., Ianovskii, E., Luhmann, H. J., Haas, H. L., & Huston, J. P. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03234.x · Cited by 32
A Longitudinal Study of the Effects of Uncertainty on Reproductive Behaviors
Davis, J., & Werre, D. (2008) · Human Nature · doi:10.1007/s12110-008-9052-2 · Cited by 30
“In summary, researchers have found that uncertainty about future outcomes strongly influences the activation of specific neural structures (Pouget et al. 2003; Smith et al. 2003) and neurohormones (Hills 2006 ;M anuck et al. 2003, 2005) which, in turn, have strong associations with impulsiveness, heightened aggression…”
BackgroundFrank, L. E., Preston, A. R., & Zeithamová, D. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-019-00700-8 · Cited by 30
Reyes, F. D., Mozzachiodi, R., Baxter, D. A., & Byrne, J. H. (2005) · Learning & Memory · doi:10.1101/lm.92905 · Cited by 30
“Moreover, the activity of dopaminergic neurons is correlated with changes in the parameters of reinforcing stimuli such as probability, uncertainty, and magnitude (Cromwell and Schultz 2003; Martin-Soelch et al. 2003; Fiorillo et al. 2003).”
BackgroundDopamine Triggers Heterosynaptic Plasticity
Ishikawa, M., Otaka, M., Huang, Y. H., Neumann, P. A., Winters, B. D., Grace, A. A., Schlüter, O. M., & Dong, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4694-12.2013 · Cited by 32
“This form of LTD relieves NAc neurons from inhibition upon receipt of phasic input from dopamine neurons that has been shown to be triggered by reward or reward-associated cues (Fiorillo et al., 2003).”
Models of neuromodulation for computational psychiatry
Iglesias, S., Tomiello, S., Schneebeli, M., & Stephan, K. E. (2017) · WIREs Cognitive Science · doi:10.1002/wcs.1420 · Cited by 26
Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations
Farries, M. A., Faust, T. W., Mohebi, A., & Berke, J. D. (2023) · Current Biology · doi:10.1016/j.cub.2023.08.042 · Cited by 31
“33 We therefore directly compared the average activity of Slow Pace-makers to identified dopamine cells recorded in the lateral ventral tegmental area (VTA) 10 during the same task and reward schedule (Figure 2E).”
Traditional and Innovative Experimental and Clinical Trial Designs and Their Advantages and Pitfalls
Weimer, K., & Enck, P. (2014) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-44519-8_14 · Cited by 32
Kim, Y., Matthews, M., & Moghaddam, B. (2010) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07371.x · Cited by 32
Murty, V. P., Ballard, I. C., & Adcock, R. A. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhw005 · Cited by 31
“Future studies are needed to investigate the dynamics of this system in behavioral contexts more likely to evoke prolonged regulation of the VTA by the PFC, such as cueevoked target detection (Fiorillo et al. 2003; Totah et al. 2013) or the anticipation of distant rewards (Howe et al. 2013).”
BackgroundInfluentialCholinergic tone in ventral tegmental area: Functional organization and behavioral implications
Zhang, C., Liu, X., Zhou, P., Zhang, J., He, W., & Yuan, T. (2018) · Neurochemistry International · doi:10.1016/j.neuint.2018.02.003 · Cited by 35
“VTA dopaminergic neurons also encode information related to uncertainty-seeking (Fiorillo et al., 2003).”
Culture in Mind – An Enactivist Account: Not Cognitive Penetration but Cultural Permeation
Hutto, D. D., Gallagher, S., Ilundáin‐Agurruza, J., & Hipólito, I. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108695374.009 · Cited by 40
Neural coding of computational factors affecting decision making
Dreher, J. (2013) · Progress in brain research · doi:10.1016/b978-0-444-62604-2.00016-2 · Cited by 29
Electroencephalographic Topography Measures of Experienced Utility
Pedroni, A. F., Langer, N., Koenig, T. W., Allemand, M., & Jäncke, L. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.5488-10.2011 · Cited by 30
“Because we ensured that these stimuli solely differed with respect to the magnitude of the outcome, it is conceivable that the response of brain activity (measured at the ERP topography) corresponds to the experienced utility or regret of a more or less favorable outcome.”
BackgroundNeural correlates of taste reward value across eating disorders
Olsavsky, A. K., Shott, M. E., DeGuzman, M. C., & Frank, G. (2018) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2018.08.010 · Cited by 28
Kalmbach, A., Winiger, V., Jeong, N., Asok, A., Gallistel, C. R., Balsam, P. D., & Simpson, E. H. (2022) · Nature Communications · doi:10.1038/s41467-022-31377-2 · Cited by 28
Reward prediction error in learning-related behaviors
Yujun, D., Song, D. F., Ni, J., Qing, H., & Quan, Z. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1171612 · Cited by 34
“This signal reflects the difference between a received and a predicted reward (Waelti et al., 2001; Tobler et al., 2003; Bayer and Glimcher, 2005; Pan et al., 2005; Lak et al., 2014; Ergo et al., 2020), which is extremely different from the slower dopamine activity (Fiorillo et al., 2003).”
BackgroundDopaminergic therapy in aphasia
Gill, S. K., & Leff, A. P. (2013) · Aphasiology · doi:10.1080/02687038.2013.802286 · Cited by 29
A biologically plausible embodied model of action discovery
Bolado-Gomez, R., & GURNEY, K. N. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00004 · Cited by 30
“Thus, several studies (Fiorillo et al., 2003; Tobler et al., 2005; Morris et al., 2006; Roesch et al., 2007) have shown that, with well trained animals, size of reward or its probability of delivery reward associated with unpredictable phasic cues produced phasic dopamine responses which reflected…”
BackgroundVitay, J., & Hamker, F. H. (2014) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2014.00004 · Cited by 31
van der Plasse, G., La Fors, S. S. B. M., Meerkerk, D. T. J., Joosten, R. N. J. M. A., Uylings, H. B. M., & Feenstra, M. G. P. (2007) · Psychopharmacology · doi:10.1007/s00213-007-0917-6 · Cited by 28
“Direct evidence for DA involvement comes from measurements of DA ventral tegmental area neurons in non-human primates that show reactivity to reward (Schultz 1998), probability of reward (Fiorillo et al. 2003) and adaptive neural firing in response to reward-predicting stimuli (Tobler et al. 2005).”
BackgroundWITHDRAWN: Neurofeedback-mediated self-regulation of the dopaminergic midbrain
Sulzer, J. S., Sitaram, R., Blefari, M. L., Kollias, S., Birbaumer, N., Stephan, K. E., Luft, A. R., & Gassert, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.041 · Cited by 29
RetractedStriatal Dopamine Signals and Reward Learning
Bech, P., Crochet, S., Dard, R. F., Ghaderi, P., Liu, Y., Malekzadeh, M., Petersen, C. C. H., Pulin, M., Renard, A., & Sourmpis, C. (2023) · Function · doi:10.1093/function/zqad056 · Cited by 34
“7 , 26 What dri v es RPE signals in midbrain dopamine neur ons?”
BackgroundThe neural basis of the insight memory advantage
Becker, M., & Cabeza, R. E. L. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.01.001 · Cited by 29
“DA neurons exhibit phasic excitation in response to rewards, and the magnitude is proportional to PE – responding most strongly to unexpected rewards, minimally to fully anticipated rewards, and decreasing fi ring for rewards below expectations [80].”
Neuroeducational research in the design and use of a learning technology
Howard‐Jones, P. A., Holmes, W., Demetriou, S., Jones, C., Tanimoto, E., Morgan, O., Perkins, D., & Davies, N. (2014) · Learning Media and Technology · doi:10.1080/17439884.2014.943237 · Cited by 34
“…of the teacher scaffolding learning just prior to students gaming their points through a combination of educational insight (feedback from teacher and students in the classroom) and neuroscienti fi c understanding (in terms of the ‘ ramping up ’ of dopamine shown in Figure 1 Fiorillo 2003).”
BackgroundInfluentialDoes reward unpredictability reflect risk?
Anselme, P. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2014.12.003 · Cited by 28
The Expected Utility of Movement
Trommershäuser, J., Maloney, L. T., & Landy, M. S. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00008-7 · Cited by 33
“Single-cell activity in response to stochastic variations of reward has been found in ventral midbrain areas (Fiorillo et al., 2003).”
BackgroundThe Iowa Gambling Task and the three fallacies of dopamine in gambling disorder
Linnet, J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00709 · Cited by 30
“Based on suggestions that risk and outcome uncertainty is associated with dopamine release in gambling disorder (Fiorillo et al., 2003), it was further hypothesized that dopamine release in the ventral striatum of gambling disorder sufferers would be associated with more risky decision-making,…”
BackgroundOyama, K., Tateyama, Y., Hernádi, I., Tobler, P. N., Iijima, T., & Tsutsui, K. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00097.2015 · Cited by 29
Self-citationTremblay, M., Barrus, M. M., Cocker, P. J., Baunez, C., & Winstanley, C. A. (2019) · Psychopharmacology · doi:10.1007/s00213-019-5173-z · Cited by 28
History- and Current Instruction-Based Coding of Forthcoming Behavioral Outcomes in the Striatum
Yamada, H., Matsumoto, N., & Kimura, M. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00779.2007 · Cited by 30
HORVITZ, J. C., CHOI, W. Y., MORVAN, C., EYNY, Y., & BALSAM, P. D. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.017 · Cited by 26
Berthet, P., Kotaleski, J. H., & Lansner, A. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00065 · Cited by 29
“In accordance with the biology (Aosaki et al., 1994; Calabresi et al., 1996; Schultz, 1998; Reynolds and Wickens, 2000, 2002; Kerr and Wickens, 2001; Fiorillo et al., 2003; Surmeier et al., 2007; Matsuda et al., 2009; Pawlak et al., 2010) and in a similar way to previous computational studies…”
ResultUncertainty modulates value-driven attentional capture
Cho, S. A., & Cho, Y. S. (2020) · Attention Perception & Psychophysics · doi:10.3758/s13414-020-02171-3 · Cited by 28
Dopamine error signal to actively cope with lack of expected reward
Ishino, S., Kamada, T., Sarpong, G. A., Kitano, J., Tsukasa, R., Mukohira, H., Sun, F., Li, Y., Kobayashi, K., Naoki, H., et al. (2023) · Science Advances · doi:10.1126/sciadv.ade5420 · Cited by 26
“Although the RPE-type DA signal has been reported in Pavlovian tasks many times (8, 9, 14, 16, 18), type 2 DA error signals in the tasks are unknown.”
BackgroundMany hats: intratrial and reward level-dependent BOLD activity in the striatum and premotor cortex
Peterson, E. J., & Seger, C. A. (2013) · Journal of Neurophysiology · doi:10.1152/jn.00164.2012 · Cited by 28
The Role of Risk Aversion in Non-Conscious Decision Making
Wang, S., Krajbich, I. M., Adolphs, R., & Tsuchiya, N. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00050 · Cited by 30
“…of probability and reward, influence the activity of midbrain dopamine neurons as well as the activation of ventral prefrontal, insular, and cingulate cortices (Bechara et al., 1999; Critchley et al., 2001; Smith et al., 2002; Fiorillo et al., 2003; Tobler et al., 2007; Christopoulos et al., 2009).”
BackgroundPetykó, Z., Gálosi, R., Tóth, A., Máté, K., Szabó, I., Szabó, I., Karádi, Z., & Lénárd, L. (2015) · Behavioural Brain Research · doi:10.1016/j.bbr.2015.03.034 · Cited by 28
Goñi, J., Aznárez‐Sanado, M., Arrondo, G., Fernández‐Seara, M. A., Loayza, F. R., Heukamp, F. H., & Pastor, M. A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0017408 · Cited by 28
“Single-neuron recordings in macaque studies have provided preliminary evidence that such a property may better fit the behavior of dopaminergic neurons under uncertainty paradigms [13,31].”
BackgroundCumulative gains enhance striatal response to reward opportunities in alcohol-dependent patients
Gilman, J. M., Smith, A. R., Bjork, J. M., Ramchandani, V. A., Momenan, R., & Hommer, D. W. (2014) · Addiction Biology · doi:10.1111/adb.12147 · Cited by 26
“This modulation during risky choice is also consistent with studies showing that dopamine neurons exhibit a sustained mode of activity during reward uncertainty (maximal when reward probability = 0.5) (Fiorillo et al. 2003).”
InfluentialChiew, K. S., Stanek, J. K., & Adcock, R. A. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00555 · Cited by 26
“On this timescale, DA neurons have been shown to exhibit gradually increasing (ramping) anticipatory activity, scaling with reward uncertainty (Fiorillo et al., 2003).”
BackgroundInfluentialHétu, S., Luo, Y., D’Ardenne, K., Lohrenz, T. M., & Montague, P. R. (2017) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsx097 · Cited by 27
“Data from animal (Fiorillo et al. , 2003) and human (Dreher et al. , 2006; Preuschoff et al. , 2006) studies have shown that activity in the SN/VTA varies as a function of uncertainty/risk but no study has specifically looked at its potential role in the updating/learning of this estimate.”
BackgroundInfluentialFlannery, J. S., Riedel, M. C., Bottenhorn, K. L., Poudel, R., Salo, T., Hill-Bowen, L. D., Laird, A. R., & Sutherland, M. T. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-019-00763-7 · Cited by 27
Funamizu, A., Ito, M., Doya, K., Kanzaki, R., & Takahashi, H. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08025.x · Cited by 28
“risk values than did the choices with the reward probabilities of 0 and 100% (Fiorillo et al., 2003; Rushworth & Behrens, 2008; Schultz et al., 2008).”
BackgroundNeural Representation of Reward Probability: Evidence from the Illusion of Control
Kool, W., Getz, S. J., & Botvinick, M. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00369 · Cited by 29
Human preferences are biased towards associative information
Trapp, S., Shenhav, A., Bitzer, S., & Bar, M. (2014) · Cognition & Emotion · doi:10.1080/02699931.2014.966064 · Cited by 27
Brain-behaviour relationships in latent inhibition: a computational model
Schmajuk, N. A. (2005) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2005.02.005 · Cited by 26
Tanaka, N., Takahashi, H., Kitano, H., Matsuoka, M., Akao, S., Uchimiya, H., & Komatsu, S. (2005) · Journal of Proteome Research · doi:10.1021/pr050114f · Cited by 26
Doyon, W. M., Ramachandra, V. S., Samson, H. H., Czachowski, C. L., & Gonzales, R. A. (2004) · Alcohol · doi:10.1016/j.alcohol.2004.10.004 · Cited by 26
The curious case of dopaminergic prediction errors and learning associative information beyond value
Kahnt, T., & Schoenbaum, G. M. (2025) · Nature reviews. Neuroscience · doi:10.1038/s41583-024-00898-8 · Cited by 28
Neural correlates of reward-directed action and inhibition of action
Le, T. M., Zhang, S., Zhornitsky, S., Wang, W., & Li, C. R. (2019) · Cortex · doi:10.1016/j.cortex.2019.10.007 · Cited by 26
Aberg, K. C., Kramer, E. E., & Schwartz, S. (2020) · Nature Communications · doi:10.1038/s41467-020-15542-z · Cited by 26
Winkelmeier, L., Filosa, C., Hartig, R., Scheller, M. F., Sack, M., Reinwald, J., Becker, R., Wolf, D., Gerchen, M. F., Sartorius, A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-30978-1 · Cited by 26
“One-way ANOVA on Tu history and satiety combined (left graph) during: CS, F(3,28)= 0.”
MethodLeib, R., Howard, I. S., Millard, M., & Franklin, D. W. (2023) · Comprehensive physiology · doi:10.1002/cphy.c220032 · Cited by 27
Early adversity disrupts the adult use of aversive prediction errors to reduce fear in uncertainty
Wright, K. M., DiLeo, A., & McDannald, M. A. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00227 · Cited by 25
“Long-studied in the context of reward (Schultz et al., 1997; Fiorillo et al., 2003, 2008; Roesch et al., 2007; Niv and Schoenbaum, 2008), recent work highlights the role of prediction errors in aversive learning (McNally and Cole, 2006; McNally et al., 2011; Berg et al., 2014; Li and McNally, 2014;…”
BackgroundMai, B., Sommer, S., & Hauber, W. (2015) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyv043 · Cited by 26
“For instance, midbrain DA neurons are activated by stimuli predicting risky rewards and generate a neuronal signal that varies monotonically with risk (Fiorillo et al., 2003).”
BackgroundNeural dynamics of reward probability coding: a Magnetoencephalographic study in humans
Thomas, J., Vanni-Mercier, G., & Dreher, J. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00214 · Cited by 28
Impulse control disorders in Parkinson’s disease: an overview from neurobiology to treatment
Maréchal, E., Denoiseux, B., Thys, E., Crosiers, D., Pickut, B. A., & Cras, P. P. (2014) · Journal of Neurology · doi:10.1007/s00415-014-7361-4 · Cited by 25
“This activity is increased when patients are treated with L-dopa and dopamine agonists [22].”
BackgroundInfluentialMuzik, O., & Diwadkar, V. A. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.08.015 · Cited by 27
Betts, G., Hynes, T. J., & Winstanley, C. A. (2021) · Journal of Psychopharmacology · doi:10.1177/0269881120972421 · Cited by 25
“…with the hypothesis that repeated engagement with heavily cued probabilistic reinforcement schedules alters the sensitivity of the DA system, thereby accelerating the development of addiction-related behaviors (Fiorillo et al., 2003; Singer et al., 2012; Zack et al., 2014; Zeeb et al., 2017).”
BackgroundD1 receptors involved in the acquisition of sexual experience in male rats
Biały, M., Kalata, U., Nikolaev-Diak, A., & Nikolaev, E. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2009.09.008 · Cited by 28
How Costs Influence Decision Values for Mixed Outcomes
Talmi, D., & Pine, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00146 · Cited by 26
“In a prior study (Fiorillo et al., 2003) the FIGURE 4 | Separate and partially overlapping striatal regions encoding unique valuation components of probabilistic rewards.”
BackgroundSchmajuk, N. A., Larrauri, J. A., la Casa, L. G. D., & Levin, E. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.08.030 · Cited by 27
Modulation of corticospinal excitability by reward depends on task framing
Mooshagian, E., Keisler, A., Zimmermann, T., Schweickert, J. M., & Wassermann, E. M. (2015) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.12.021 · Cited by 24
“Single-cell recordings in monkeys show that midbrain dopamine (DA) neurons encode reward probability and uncertainty (Fiorillo et al., 2003).”
BackgroundInfluentialNeural circuits associated with positive and negative self-appraisal
Brühl, A. B., Rufer, M., Kaffenberger, T., Baur, V., & Herwig, U. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.01.053 · Cited by 24
Goedhoop, J., Arbab, T., & Willuhn, I. (2023) · The Journal of Neuroscience · doi:10.1523/jneurosci.1527-22.2023 · Cited by 24
“Next, since previous work demonstrated that the dopamine system is sensitive to reward uncertainty (Kobayashi and Schultz, 2008; Fiorillo et al., 2003; de Lafuente and Romo, 2011; Starkweather et al., 2017; Lak et al., 2017), we rendered food delivery probabilistic (only 50% of trials rewarded,…”
BackgroundCandia‐Rivera, D., Vidailhet, M., Chávez, M., & Fallani, F. D. V. (2024) · Human Brain Mapping · doi:10.1002/hbm.26668 · Cited by 25
“Numerous studies have shown that dopamine encodes learning and reward prediction (Fiorillo et al., 2003; Hollerman & Schultz, 1998; Mirenowicz & Schultz, 1994; Pessiglione et al., 2006), further supporting this idea.”
BackgroundAcetylcholine and noradrenaline enhance foraging optimality in humans
Doren, N., Chung, H.-K., Grueschow, M., Quednow, B. B., Hayward-Koennecke, H. K., Jetter, A., & Tobler, P. N. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2305596120 · Cited by 25
“Based on the role of dopamine in reward processing (11, 25, 26), effort exertion (27), and response vigour (28), and the putative requirement of these functions for calculating the optimal policy, dopamine has been hypothesized to be involved in foraging.”
Self-citationWiehler, A., & Peters, J. (2024) · Computational Psychiatry · doi:10.5334/cpsy.104 · Cited by 24
Ten challenges for decision neuroscience
Huettel, S. A. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00171 · Cited by 27
“Yet, risk also influences activation in other regions seemingly associated with simpler sensory, motor, or attentional processes (McCoy and Platt, 2005), as well as in the brain’s reward system directly (Berns et al., 2001; Fiorillo et al., 2003).”
BackgroundRisk, Reward, and Decision-Making in a Rodent Model of Cognitive Aging
Gilbert, R. J., Mitchell, M. R., Simon, N. W., Bañuelos, C., Setlow, B., & Bizon, J. L. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00144 · Cited by 24
“In addition, the activity of midbrain dopamine neurons encodes information regarding both reward probability and delay (Fiorillo et al., 2003; Kobayashi and Schultz, 2008), suggesting that this neurochemical system processes information regarding outcome costs.”
BackgroundDifferential coding of uncertain reward in rat insular and orbitofrontal cortex
Jo, S., & Jung, M. W. (2016) · Scientific Reports · doi:10.1038/srep24085 · Cited by 28
Ward, R. D., Winiger, V., Higa, K. K., Kahn, J. B., Kandel, E. R., Balsam, P. D., & Simpson, E. H. (2015) · Behavioral Neuroscience · doi:10.1037/bne0000051 · Cited by 32
“Indeed several studies have demonstrated an involvement of dopamine signaling in the modulation of behavioral performance by signaled-reward probability (Day et al., 2010; Gan et al., 2010; Fiorillo et al., 2003).”
BackgroundCost–Benefit Decision Circuitry
Fobbs, W. C., & Mizumori, S. J. Y. (2014) · Progress in molecular biology and translational science · doi:10.1016/b978-0-12-420170-5.00009-x · Cited by 25
Evaluation of the hypothesis that phasic dopamine constitutes a cached-value signal
Sharpe, M. J., & Schoenbaum, G. M. (2017) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2017.12.002 · Cited by 24
The cutaneous borders of interoception: Active and social inference of pain and pleasure on the skin
Mohr, M. V., & Fotopoulou, A. (2018) · Oxford University Press eBooks · doi:10.1093/oso/9780198811930.003.0006 · Cited by 32
Rubin, J. E., Vich, C., Clapp, M., Noneman, K. K., & Verstynen, T. (2020) · European Journal of Neuroscience · doi:10.1111/ejn.14745 · Cited by 28
Salazar, R. F., König, P., & Kayser, C. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03579.x · Cited by 24
“To facilitate learning and synaptic plasticity, higher areas may generate expectations and predictions about future sensory events (Schultz, 1998; Waelti et al., 2001; Schultz, 2002; Fiorillo et al., 2003), which are transmitted to sensory areas in a top-down manner.”
BackgroundStein, D. J., & Mayberg, H. S. (2005) · CNS Spectrums · doi:10.1017/s1092852900023117 · Cited by 23
Update on pathological gambling
Grant, J. E., Williams, K. A., & Kim, S. W. (2006) · Current Psychiatry Reports · doi:10.1007/s11920-006-0081-4 · Cited by 22
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.