Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 401–500 of 1,007
Dopamine, reward learning, and active inference
FitzGerald, T. H. B., Dolan, R. J., & Friston, K. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00136 · Cited by 112
“[In fact overt conditioned behavioral responses is typically a precondition for recording meaningful data (Fiorillo et al., 2003)].”
BackgroundIndependent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons
Arkadir, D., Morris, G., Vaadia, E., & Bergman, H. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.2583-04.2004 · Cited by 109
“…Activity in these structures is assumed to be modulated by reward-related activity of the midbrain dopaminergic neurons (Hollerman and Schultz, 1998; Fiorillo et al., 2003; Satoh et al., 2003; Morris et al., 2004) and striatal tonically active neurons (TANs; presumably striatal cholinergic…”
BackgroundCowen, S. L., Davis, G. A., & Nitz, D. A. (2012) · Journal of Neurophysiology · doi:10.1152/jn.01012.2011 · Cited by 107
Instrumental uncertainty as a determinant of behavior under interval schedules of reinforcement
DeRusso, A. L. (2010) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2010.00017 · Cited by 111
“Nor, for that matter, is the food automatically delivered into the animal’s mouth, as in laboratory experiments using Pavlovian conditioning procedures (Fiorillo et al., 2003).”
Noradrenaline and Dopamine Elevations in the Rat Prefrontal Cortex in Spatial Working Memory
Rossetti, Z. L., & Carboni, S. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3038-04.2005 · Cited by 105
“The maximal activation of DA neurons occurs under conditions of maximal uncertainty of the outcome (Fiorillo et al., 2003); that is, when p 50%, as in the present and other tasks of working memory.”
Anticipating instrumentally obtained and passively-received rewards: A factorial fMRI investigation
Bjork, J. M., & Hommer, D. W. (2006) · Behavioural Brain Research · doi:10.1016/j.bbr.2006.10.034 · Cited by 119
“Notably, Fiorillo et al demonstrated that anticipatory single-unit activity in mesolimbic dopaminergic neurons is maximal when the probability of payoff for the instrumental response is maximally uncertain (50%) [13].”
BackgroundReinforcement learning and human behavior
Shteingart, H., & Loewenstein, Y. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.004 · Cited by 125
Howes, O. D., Hird, E. J., Adams, R. A., Corlett, P. R., & McGuire, P. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.03.012 · Cited by 110
Ziegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
What motivates gambling behavior? Insight into dopamine's role
Anselme, P., & Robinson, M. J. F. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00182 · Cited by 106
“Studies have shown that reward uncertainty rather than reward per se, will magnify mesolimbic DA, both in monkeys (Fiorillo et al., 2003; de Lafuente and Romo, 2011) and healthy human participants (Preuschoff et al., 2006).”
BackgroundDavis, C. (2014) · Substance Abuse and Rehabilitation · doi:10.2147/sar.s56835 · Cited by 104
“dopamine release is an important learning/teaching signal, and therefore has a multiple and essential role in the acquisition of survival behaviors.(29) In essence, dopamine is, as well as other things, a signal of “surprise”, because its release is triggered by unexpected rewards to a greater degree than when the rew…”
How foraging works: Uncertainty magnifies food-seeking motivation
Anselme, P., & Güntürkün, O. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000948 · Cited by 117
St. Onge, J. R., Ahn, S., Phillips, A. G., & Floresco, S. B. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3807-12.2012 · Cited by 107
“Stimuli predictive of potential rewards evoke sustained increases in neural firing of VTA DA neurons that are maximal during periods of greatest reward uncertainty (Fiorillo et al., 2003).”
BackgroundFoti, D., & Hajcak, G. (2012) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01343.x · Cited by 102
“, maximum uncertainty) and a decreasing response for more predictable outcomes (Fiorillo et al., 2003).”
BackgroundNeurobiology of placebo effects: expectations or learning?
Peciña, M., Stohler, C. S., & Zubieta, J.-K. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst079 · Cited by 62
“Consistently with the role of DA in prediction error signal (Schultz, 1998; Fiorillo et al., 2003; Tobler et al., 2005; Fiorillo et al., 2008), we have previously shown that the introduction of a placebo during pain anticipation was associated with the activation of DA neurotransmission and D2/D3…”
Dopamine and Acetylcholine, a Circuit Point of View in Parkinson’s Disease
Rizzi, G., & Tan, K. R. (2017) · Frontiers in Neural Circuits · doi:10.3389/fncir.2017.00110 · Cited by 121
“The response of both neurons encodes the probability of reward: the DA neurons respond to reward mismatch between expectation and outcome, while the cholinergic pause represents the expected timing of presentation reward (Fiorillo et al., 2003; Morris et al., 2004).”
BackgroundThe Cognitive Neuroscience of Motivation and Learning
Daw, N. D., & Shohamy, D. (2008) · Social Cognition · doi:10.1521/soco.2008.26.5.593 · Cited by 102
The risky business of dopamine agonists in Parkinson disease and impulse control disorders.
Claassen, D. O., van den Wildenberg, W. P. M., Ridderinkhof, K. R., Jessup, C. K., Harrison, M. B., Wooten, G. F., & Wylie, S. A. (2011) · Behavioral Neuroscience · doi:10.1037/a0023795 · Cited by 104
Expected reward modulates encoding-related theta activity before an event
Gruber, M. J., Watrous, A. J., Ekstrom, A. D., Ranganath, C., & Otten, L. J. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.064 · Cited by 108
“Results from studies of rats and monkeys indicate that midbrain dopamine neurons show increased firing during anticipation of rewards (Fiorillo et al., 2003; Ljungberg et al., 1992; Schultz, 1998, 2002; Tabuchi et al., 2000) and, consistent with these data, human fMRI studies have shown that…”
ResultZack, M., Featherstone, R. E., Mathewson, S., & Fletcher, P. J. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00036 · Cited by 108
Dopamine Neurons Reflect the Uncertainty in Fear Generalization
Jo, Y. S., Heymann, G., & Zweifel, L. S. (2018) · Neuron · doi:10.1016/j.neuron.2018.09.028 · Cited by 114
“These findings are remarkably similar to those reported for dopamine neuron responses to CS cues in a probabilistic reinforcement task (Fiorillo et al., 2003).”
Glutamate neurons are intermixed with midbrain dopamine neurons in nonhuman primates and humans
Root, D. H., Wang, H.-L., Liu, B., Barker, D. J., Mód, L., Szocsics, P., Silva, A. C., Maglóczky, Z., & Morales, M. (2016) · Scientific Reports · doi:10.1038/srep30615 · Cited by 101
Macey, J., & Hamari, J. (2022) · New Media & Society · doi:10.1177/14614448221083903 · Cited by 127
“…of a choice or action are not known beforehand, it is comprised of a set of neural processes concerning risk assessment and decision-making and has been particularly associated with learning, memory formation and economic behaviour (Fiorillo et al., 2003; Schultz et al., 2008; Stanek et al., 2019).”
BackgroundBaxter, D. A., & Byrne, J. H. (2006) · Learning & Memory · doi:10.1101/lm.339206 · Cited by 113
Delays Conferred by Escalating Costs Modulate Dopamine Release to Rewards But Not Their Predictors
Wanat, M. J., Kuhnen, C. M., & Phillips, P. E. M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2691-10.2010 · Cited by 100
“There is increasing experimental evidence that dopamine encodes reward-related information in the execution of economic cost–benefit decisions (Fiorillo et al., 2003, 2008; Phillips et al., 2003b, 2007; Roitman et al., 2004; Tobler et al., 2005; Roesch et al., 2007; Kobayashi and Schultz, 2008;…”
BackgroundStructural changes of the brain in rheumatoid arthritis
Wartolowska, K. A., Hough, M., Jenkinson, M., Andersson, J., Wordsworth, B. P., & Tracey, I. (2012) · Arthritis & Rheumatism · doi:10.1002/art.33326 · Cited by 117
Harnett, M. T., Bernier, B. E., Ahn, K.-C., & Morikawa, H. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.011 · Cited by 103
Magical Flight and Monstrous Stress: Technologies of Absorption and Mental Wellness in Azeroth
Snodgrass, J. G., Lacy, M. G., Dengah, H. J. F., Fagan, J. M., & Most, D. E. (2010) · Culture Medicine and Psychiatry · doi:10.1007/s11013-010-9197-4 · Cited by 118
“Circulating cortisol can lead to increased bursts of dopamine, with stress in uncertain situations promoting even greater releases of this chemical transmitter (Fiorillo et al. 2003; Phillips et al. 2003; Sapolsky 2003; Schultz et al. 2000; Waelti et al. 2001).”
BackgroundHypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction
Veissière, S. P. L., & Stendel, M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00141 · Cited by 105
“At a 50% delivery rate, a reward schedule is still predictable enough to be enticing, but unpredictable enough to be anxiety-inducing (Fiorillo et al., 2003).”
BackgroundContextual novelty changes reward representations in the striatum
Guitart-Masip, M., Bunzeck, N., Stephan, K. E., Dolan, R. J., & Düzel, E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.5331-09.2010 · Cited by 104
“…projection sites of the midbrain dopamine system, also respond to reward and reward-predictive cues in classical conditioning paradigms (Delgado et al., 2000; Knutson et al., 2000; Fiorillo et al., 2003; Knutson et al., 2005; Tobler et al., 2005; Wittmann et al., 2005; D’Ardenne et al., 2008).”
BackgroundObservational learning computations in neurons of the human anterior cingulate cortex
Hill, M. R., Boorman, E. D., & Fried, I. (2016) · Nature Communications · doi:10.1038/ncomms12722 · Cited by 102
Shindou, T., Shindou, M., Watanabe, S., & Wickens, J. (2019) · European Journal of Neuroscience · doi:10.1111/ejn.13921 · Cited by 94
How pupil responses track value-based decision-making during and after reinforcement learning
Slooten, J. C. V., Jahfari, S., Knapen, T. H. J., & Theeuwes, J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006632 · Cited by 116
Winstanley, C. A., Cocker, P. J., & Rogers, R. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.230 · Cited by 113
Leyton, M., aan het Rot, M., Booij, L., Baker, G. B., Young, S. N., & Benkelfat, C. (2007) · Journal of Psychiatry and Neuroscience · doi:10.1139/jpn.0713 · Cited by 100
“Craving for drug reward, in comparison, might emerge from increases in both phasic DA cell firing and tonic DA release.(38,43,90,91,95,99) Fifth, APTD might affect NE synthesis(19,24); however, accumulating evidence suggests that, to the extent that effects on NE synthesis occur, they are insufficient to alter NE rele…”
BackgroundOpriș, I. C., Lebedev, M., & Nelson, R. J. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00061 · Cited by 105
“…the activation of midbrain’s dopaminergic projections to fronto-parietal cortex and dorsal striatum that track successful and erroneous behaviors and the contingencies between the behaviors and rewards (Romo and Schultz, 1990; Gaspar et al., 1992; Kiyatkin, and Rebec, 1996; Fiorillo et al., 2003).”
BackgroundUncertainty and engagement with learning games
Howard‐Jones, P. A., & Demetriou, S. (2008) · Instructional Science · doi:10.1007/s11251-008-9073-6 · Cited by 117
Pharmacological and Psychological Effects of Caffeine Ingestion in 40-km Cycling Performance
FOAD, A. J., BEEDIE, C. J., & COLEMAN, D. A. (2008) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3181593e02 · Cited by 91
Koch, K., Schachtzabel, C., Wagner, G., Schikora, J., Schultz, C., Reichenbach, J. R., Sauer, H., & Schlösser, R. G. M. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.031 · Cited by 101
State- and trait-greed, its impact on risky decision-making and underlying neural mechanisms
Mussel, P., Reiter, A. M. F., Osinsky, R., & Hewig, J. S. (2014) · Social Neuroscience · doi:10.1080/17470919.2014.965340 · Cited by 108
Friedman, A., Lax, E., Dikshtein, Y., Abraham, L., Flaumenhaft, Y., Sudai, E., Ben-Tzion, M., & Yadid, G. (2011) · Neuropharmacology · doi:10.1016/j.neuropharm.2010.10.006 · Cited by 97
“Dopaminergic and serotonergic neurons that project to the LHb respond to reward-predicting cues by firing at a frequency of approximately 8e12 Hz (Fiorillo et al., 2003; Nakamura et al., 2008), which inhibits LHb activity.”
BackgroundIntegrating rewards and cognition in the frontal cortex
Gilbert, A. M., & Fiez, J. A. (2004) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.4.4.540 · Cited by 108
“For instance, presentation of a conditioned cue associated with future delivery of a juice reward will initiate licking behavior in monkeys ( Fiorillo, Tobler, & Schultz, 2003 ).”
BackgroundCognitive Neurostimulation: Learning to Volitionally Sustain Ventral Tegmental Area Activation
MacInnes, J. J., Dickerson, K. C., Chen, N.-K., & Adcock, R. A. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.002 · Cited by 101
“…view of dopamine neuron physiology as either tonic or phasic has guided the field for many years, converging evidence across methodologies (Fiorillo et al., 2003; Howe et al., 2013; Totah et al., 2013) suggests a third type of dopamine response profile: a sustained, ramping signal evident…”
InfluentialThe microcircuits of striatum in silico
Hjorth, J. J. J., Kozlov, A., Carannante, I., Nylén, J. F., Lindroos, R., Johansson, Y., Tokarska, A., Dorst, M. C., Suryanarayana, S. M., Silberberg, G., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2000671117 · Cited by 112
Diversity and Homogeneity in Responses of Midbrain Dopamine Neurons
Fiorillo, C. D., Yun, S. R., & Song, M. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3886-12.2013 · Cited by 99
“A lack of clusters was also found in previous studies in behaving monkeys (Fiorillo et al., 2003; Joshua et al., 2008; Matsumoto and Hikosaka, 2009).”
BackgroundResultSelf-citationCognitive reappraisal modulates expected value and prediction error encoding in the ventral striatum
Staudinger, M. R., Erk, S., Abler, B., & Walter, H. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.095 · Cited by 104
“, 2006), the probability (Fiorillo et al., 2003; Abler et al., 2006), and even their mathematical product, the expected value (EV) (Tobler et al.”
Damiano, C. R., Aloi, J., Treadway, M., Bodfish, J. W., & Dichter, G. S. (2012) · Journal of Neurodevelopmental Disorders · doi:10.1186/1866-1955-4-13 · Cited by 90
“More specifically, animal studies suggest that dopaminergic activity in the ventral striatum (and more specifically the nucleus accumbens (NAc)) mediates how expected reward value and effort expenditure influence behavioral choices [22-24].”
BackgroundLearning Reward Uncertainty in the Basal Ganglia
Mikhael, J. G., & Bogacz, R. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005062 · Cited by 95
Modulation of Dorsolateral Prefrontal Delay Activity during Self-Organized Behavior
Procyk, E., & Goldman-Rakic, P. S. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2157-06.2006 · Cited by 91
“Mesencephalic potentially dopaminergic neurons can show sustained activity that varies according to uncertainty of obtaining a particular reward (Fiorillo et al., 2003).”
Fiorillo, C. D., Song, M. R., & Yun, S. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3883-12.2013 · Cited by 98
“…a unified framework of RPE (e.g., Kakade and Dayan, 2002; Fiorillo et al., 2008; Nomoto et al., 2010), others have proposed a multiplicity of dopamine signals that could serve distinct functions (e.g., Fiorillo et al., 2003; Schultz, 2007; Brischoux et al., 2009; Bromberg-Martin et al., 2010).”
BackgroundSelf-citationLesaint, F., Sigaud, O., Flagel, S. B., Robinson, T. E., & Khamassi, M. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003466 · Cited by 95
“– an outcome that is better or worse than expected – to the time of the conditioned stimulus that precedes it, which, in Pavlovian conditioning experiments, is fully predictive of the reward [12,13].”
BackgroundContextual Factors Explain Risk-Seeking Preferences in Rhesus Monkeys
Heilbronner, S. R., & Hayden, B. Y. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00007 · Cited by 92
“These results also tacitly endorse the validity of animal models for studies of risk preferences, and provide a foundation for neuroeconomic studies of risky choice (Platt and Glimcher, 1999; Fiorillo et al., 2003; McCoy and Platt, 2005).”
ResultThe Dopamine Prediction Error: Contributions to Associative Models of Reward Learning
Nasser, H. M., Calu, D. J., Schoenbaum, G., & Sharpe, M. J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00244 · Cited by 86
“Further, the magnitude of phasic activity correlates with the size of the unexpected reward (Hollerman and Schultz, 1998; Fiorillo et al., 2003; Roesch et al., 2007; Stauffer et al., 2014; Eshel et al., 2016) in a manner that reflects the value of the reward (Lak et al., 2014), value of the future…”
BackgroundInfluence of overanxious disorder of childhood on the expression of anorexia nervosa
Raney, T. J., Thornton, L. M., Berrettini, W., Brandt, H., Crawford, S., Fichter, M. M., Halmi, K. A., Johnson, C., Kaplan, A. S., LaVia, M., et al. (2008) · International Journal of Eating Disorders · doi:10.1002/eat.20508 · Cited by 97
Limited Encoding of Effort by Dopamine Neurons in a Cost–Benefit Trade-off Task
Pasquereau, B., & Turner, R. S. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4619-12.2013 · Cited by 94
“…from previous work that the dopamine response to a sensory cue is modulated by any information signaled by the cue about the temporal delay to reward (Roesch et al., 2007; Fiorillo et al., 2008; Kobayashi and Schultz, 2008) or the probability of reward (Fiorillo et al., 2003; Morris et al., 2006).”
BackgroundInfluentialDistinct roles of dopamine and subthalamic nucleus in learning and probabilistic decision making
Coulthard, E. J., Bogacz, R., Javed, S., Mooney, L., Murphy, G., Keeley, S., & Whone, A. L. (2012) · Brain · doi:10.1093/brain/aws273 · Cited by 105
“Midbrain dopaminergic neurons facilitate adaptation of behaviour according to upcoming reward or task demands (Hollerman and Schultz, 1998; Cools et al., 2001, 2003; Fiorillo et al., 2003).”
BackgroundDopamine Encoding of Pavlovian Incentive Stimuli Diminishes with Extended Training
Clark, J. J., Collins, A. L., Sanford, C. A., & Phillips, P. E. M. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.5119-12.2013 · Cited by 94
“In experimental paradigms involving probabilistic rewards, there is evidence that cue-evoked dopamine signaling scales with the probability of reward delivery (Fiorillo et al., 2003).”
BackgroundThe effect of uncertainty on learning in game-like environments
Özçelik, E., Çağıltay, N. E., & Ozcelik, N. S. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.02.009 · Cited by 119
Multiple Timescales of Memory in Lateral Habenula and Dopamine Neurons
Bromberg-Martin, E. S., Matsumoto, M., Nakahara, H., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.031 · Cited by 94
“This is unexpected based on previous studies, which largely emphasized phasic activations to task events (but see Schultz, 1986; Fiorillo et al., 2003, 2008).”
Distribution of Power Output During Cycling: Impact and Mechanisms
Atkinson, G., Peacock, O. J., Gibson, A. S. C., & Tucker, R. (2007) · Sports Medicine · doi:10.2165/00007256-200737080-00001 · Cited by 121
Volkow, N. D., Wang, G., Ma, Y., Fowler, J. S., Wong, C., Jayne, M. C., Telang, F., & Swanson, J. M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.04.192 · Cited by 101
Reward Prediction Error Coding in Dorsal Striatal Neurons
Oyama, K., Hernádi, I., Iijima, T., & Tsutsui, K.-I. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1719-10.2010 · Cited by 93
“Fiorillo et al. (2003) did not have any delay between the CS offset and the reward onset (delay conditioning procedure) in regular recording sessions.”
BackgroundResultInfluentialThe placebo-reward hypothesis: dopamine and the placebo effect
de la Fuente-Fernández, R. (2009) · Parkinsonism & Related Disorders · doi:10.1016/s1353-8020(09)70785-0 · Cited by 95
“Animal experiments have shown that there are two different patterns of dopamine cell activation in relation to reward processing, (1) a phasic response, which signals reward occurrence, and (2) a tonic response, which precedes reward occurrence and apparently encodes reward uncertainty [13].”
BackgroundCANALES, J. (2005) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2004.10.006 · Cited by 95
Neural correlates of risk prediction error during reinforcement learning in humans
d'Acremont, M., Lu, Z.-L., Li, X., Van der Linden, M., & Bechara, A. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.096 · Cited by 94
“Single cell recording studies in monkeys have highlighted a subsample of neurons that increases their activities after the presentation of conditioned stimuli only when these stimuli predict variable outcomes (Fiorillo et al., 2003).”
Incentive Motivation: The Missing Piece between Learning and Behavior
Anselme, P., & Robinson, M. J. F. (2019) · Cambridge University Press eBooks · doi:10.1017/9781316823279.009 · Cited by 101
Recent advances in understanding the role of phasic dopamine activity
Schultz, W. (2019) · F1000Research · doi:10.12688/f1000research.19793.1 · Cited by 99
Self-citationBarrus, M. M., & Winstanley, C. A. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.2225-15.2016 · Cited by 93
“Although cues predictive of reward increase the release of dopamine (Schultz, 1998), dopaminergic neurons appear to fire more strongly when reward delivery is probabilistic, with the greatest increase occurring when uncertainty is maximized (Fiorillo et al., 2003).”
BackgroundExposure to conditions of uncertainty promotes the pursuit of amphetamine
Mascia, P., Neugebauer, N. M., Brown, J., Bubula, N., Nesbitt, K. M., Kennedy, R. T., & Vezina, P. (2018) · Neuropsychopharmacology · doi:10.1038/s41386-018-0099-4 · Cited by 95
Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking
Collins, A. L., & Saunders, B. T. (2020) · Journal of Neuroscience Research · doi:10.1002/jnr.24587 · Cited by 95
Higher-Dimensional Neurons Explain the Tuning and Dynamics of Working Memory Cells
Singh, R., & Eliasmith, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4864-05.2006 · Cited by 90
“Indeed, it is possible that both the F(t) and T(t) signals are external to the population in the PFC.”
BackgroundInfluentialMolecular basis of dopamine replacement therapy and its side effects in Parkinson’s disease
You, H., Mariani, L., Mangone, G., de Nailly, D. L. F., Charbonnier‐Beaupel, F., & Corvol, J. (2018) · Cell and Tissue Research · doi:10.1007/s00441-018-2813-2 · Cited by 97
Dickhaut, J. W., Basu, S., McCabe, K., & Waymire, G. (2010) · Accounting Horizons · doi:10.2308/acch.2010.24.2.221 · Cited by 111
The role of learning-related dopamine signals in addiction vulnerability
Huys, Q. J. M., Tobler, P. N., Hasler, G., & Flagel, S. B. (2014) · Progress in brain research · doi:10.1016/b978-0-444-63425-2.00003-9 · Cited by 107
“The occurrence of reward or reward prediction (positive prediction error) or their omission (negative prediction error) activates or depresses dopamine neurons in an inverse monotonic function of probability, such that the more unpredicted the event the stronger the response (de Lafuente and Romo, 2011; Enomoto et al.…”
BackgroundSelf-citationEvolved to satisfy our immediate needs: Self-control and the rewarding properties of food
van den Bos, R., & de Ridder, D. (2006) · Appetite · doi:10.1016/j.appet.2006.02.008 · Cited by 91
“The contrast between small and big rewards under conditions of uncertainty strongly enhances tonic activity in the dopaminergic system (Fiorillo et al., 2003), which may contribute to the observed increase in exploratory or risk-taking behaviour and concomitant loss of self-control (Van den Bos et…”
“Stopping for knowledge”: The sense of beauty in the perception-action cycle
Sarasso, P., Neppi-Mòdona, M., Sacco, K., & Ronga, I. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.09.004 · Cited by 97
Neural Correlates of Anticipation Risk Reflect Risk Preferences
Rudorf, S., Preuschoff, K., & Weber, B. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4235-11.2012 · Cited by 89
Illicit dopamine transients: reconciling actions of abused drugs
Covey, D. P., Roitman, M. F., & Garris, P. A. (2014) · Trends in Neurosciences · doi:10.1016/j.tins.2014.02.002 · Cited by 90
Transient activation of midbrain dopamine neurons by reward risk
Fiorillo, C. D. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.09.037 · Cited by 90
“…firing rate of dopamine neurons gradually increases prior to uncertain reward outcomes (during the “delay period” following a conditioned stimulus ) (Fiorillo et al., 2003), and there is also evidence that the transient response of dopamine neurons may be scaled by prior uncertainty about reward…”
BackgroundResultInfluentialSelf-citationError-Related Functional Connectivity of the Habenula in Humans
Ide, J. S., & Li, C.-S. R. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00025 · Cited by 93
The Utility of Gambling Reconsidered
Diecidue, E., Schmidt, U., & Wakker, P. P. (2004) · Journal of Risk and Uncertainty · doi:10.1023/b:risk.0000046145.25793.37 · Cited by 112
Bailer, U. F., Frank, G. K., Price, J. C., Meltzer, C. C., Becker, C., Mathis, C. A., Wagner, A., Barbarich-Marsteller, N. C., Bloss, C. S., Putnam, K., et al. (2013) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2012.06.010 · Cited by 82
The role of state uncertainty in the dynamics of dopamine
Mikhael, J. G., Kim, H. R., Uchida, N., & Gershman, S. J. (2022) · Current Biology · doi:10.1016/j.cub.2022.01.025 · Cited by 93
LINNET, J., MØLLER, A., PETERSON, E., GJEDDE, A., & DOUDET, D. (2011) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.2010.00837.x · Cited by 88
Disconnection and Cognitive Dysmetria in Schizophrenia
Friston, K. J. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.3.429 · Cited by 89
Dopamine neurons learn relative chosen value from probabilistic rewards
Lak, A., Stauffer, W. R., & Schultz, W. (2016) · eLife · doi:10.7554/elife.18044 · Cited by 91
“Cues that predict reward with high probability evoke larger responses than cues predicting the same reward with lower probability (Fiorillo et al., 2003).”
BackgroundResultInfluentialSelf-citationPhasic dopamine signals: from subjective reward value to formal economic utility
Schultz, W., Carelli, R. M., & Wightman, R. M. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.09.006 · Cited by 88
Self-citationModel‐based learning and the contribution of the orbitofrontal cortex to the model‐free world
McDannald, M. A., Takahashi, Y. K., Lopatina, N., Pietras, B. W., Jones, J. L., & Schoenbaum, G. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07982.x · Cited by 84
Subregions of the ventral striatum show preferential coding of reward magnitude and probability
Yacubian, J., Sommer, T., Schroeder, K., Gläscher, J. P., Braus, D. F., & Büchel, C. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.08.007 · Cited by 101
“Using appetitive classical conditioning, it has been demonstrated that the firing rate of these neurons is correlated with both the probability (Fiorillo et al., 2003) and magnitude (Tobler et al., 2005) of a reward signaled by different conditioned stimuli, which predict the magnitude or…”
BackgroundInfluentialNasrallah, N. A., Clark, J. J., Collins, A. L., Akers, C. A., Phillips, P. E., & Bernstein, I. L. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1017732108 · Cited by 87
“Phasic increases in dopamine transmission are evoked by rewarding outcomes and associated cues (10, 11), both of which have been shown to scale with the magnitude and the probability of reward (12, 13).”
Reward Magnitude Coding in Primate Amygdala Neurons
Bermudez, M. A., & Schultz, W. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00540.2010 · Cited by 90
Self-citationHolland, P. C., & Gallagher, M. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.0390-06.2006 · Cited by 84
Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex
Oya, H., Adolphs, R., Kawasaki, H., Bechara, A., Damasio, A., & Howard, M. A. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0500899102 · Cited by 81
Electrophysiological correlates of anticipating improbable but desired events
Fuentemilla, L., Cucurell, D., Marco‐Pallarés, J., Guitart‐Masip, M., Morís, J., & Rodríguez‐Fornells, A. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.062 · Cited by 91
Social reward monitoring and valuation in the macaque brain
Noritake, A., Ninomiya, T., & Isoda, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0229-7 · Cited by 90
Dopamine and the interdependency of time perception and reward
Fung, B. J., Sutlief, E., & Shuler, M. G. H. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.02.030 · Cited by 91
Parallel Roles for Dopamine in Pathological Gambling and Psychostimulant Addiction
Zack, M., & Poulos, C. (2009) · Current Drug Abuse Reviewse · doi:10.2174/1874473710902010011 · Cited by 86
“To date, no corresponding animal behavioral model of PG has been validated, although models do exist that appear to capture some key elements of the disorder, which we discuss later [24].”
BackgroundInfluentialTonic Dopaminergic Stimulation Impairs Associative Learning in Healthy Subjects
Breitenstein, C., Korsukewitz, C., Flöel, A., Kretzschmar, T., Diederich, K., & Knecht, S. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301167 · Cited by 93
“More sustained phasic release of dopamineF over a period of several hundred millisecondsFoccurs during periods of uncertainty about payoffs and may promote learning by allocating attention to predictors for reward (Fiorillo et al, 2003, 2004, 2005).”
BackgroundThe vulnerability to alcohol and substance abuse in individuals diagnosed with schizophrenia
Krystal, J. H., D’Souza, D. C., Gallinat, J., Driesen, N. R., Abi‐Dargham, A., Petrakis, I. L., Heinz, A., & Pearlson, G. D. (2006) · Neurotoxicity Research · doi:10.1007/bf03033360 · Cited by 93
Groman, S. M., Lee, B., London, E. D., Mandelkern, M. A., James, A. S., Feiler, K., Rivera, R., Dahlbom, M., Sossi, V., Vandervoort, E., et al. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.0363-11.2011 · Cited by 87
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.