Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 101–200 of 1,007
The Roles of the Caudate Nucleus in Human Classification Learning
Seger, C. A., & Cincotta, C. M. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3401-04.2005 · Cited by 407
“dopaminergic cells of the midbrain, particularly in the ventral tegmental area, that are also sensitive to reward (Fiorillo et al., 2003).”
BackgroundProspect theory on the brain? Toward a cognitive neuroscience of decision under risk
Trepel, C., Fox, C. R., & Poldrack, R. A. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.01.016 · Cited by 404
“[40] found that dopaminergic neuronal activity”
BackgroundPlichta, M. M., & Scheres, A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.07.012 · Cited by 424
Yin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416
“When the US is omitted after learning, the DA cells show a brief depression in activity at the expected time of its delivery (Waelti et al., 2001; Fiorillo et al., 2003; Tobler et al., 2003).”
BackgroundInformation processing, dimensionality reduction and reinforcement learning in the basal ganglia
Bar-Gad, I., Morris, G., & Bergman, H. (2003) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2003.12.001 · Cited by 399
“While early studies emphasized the role of dopamine in reward-related (hedonic) behavior (Fibiger and Phillips, 1988; Wise and Rompre, 1989) the recent seminal studies by Schultz and colleagues reveal that the primary function of dopamine neurons is to report the mismatch between the animal’s prediction and reality (H…”
BackgroundResultStress and the brain: Individual variability and the inverted-U
Sapolsky, R. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4109 · Cited by 413
Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry
Clark, L., Lawrence, A. J., Astley-Jones, F., & Gray, N. (2009) · Neuron · doi:10.1016/j.neuron.2008.12.031 · Cited by 410
“This may be analogous to an effect shown in electrophysiological research, that during appetitive Pavlovian conditioning tasks with uncertain rewards, there is a ‘‘ramping up’’ of mesolimbic dopamine cell firing between the CS presentation and the expected time of juice delivery (Fiorillo et al., 2003).”
BackgroundResultKrueger, F., McCabe, K., Moll, J., Kriegeskorte, N., Zahn, R., Strenziok, M., Heinecke, A., & Grafman, J. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0710103104 · Cited by 388
Heinz, A., Siessmeier, T., Wrase, J., Hermann, D., Klein, S., Grüsser-Sinopoli, S. M., Flor, H., Braus, D. F., Buchholz, H. G., Gründer, G., et al. (2004) · American Journal of Psychiatry · doi:10.1176/ajp.161.10.1783 · Cited by 408
“Sustained cue-induced changes in dopamine discharge may code reward uncertainty and elicit a nonselective form of attention or arousal (6).”
BackgroundProcessing of Temporal Unpredictability in Human and Animal Amygdala
Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., Lüthi, A., & Seifritz, E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5218-06.2007 · Cited by 401
“…the context of unexpected or surprising emotionally salient stimulation including punishment (Grillon et al., 2006), reward or reward-predicting stimuli (Holland and Gallagher, 1993; Fiorillo et al., 2003; Paz et al., 2006), or facial expressions of fear (Adolphs et al., 1996; Whalen et al., 2001).”
BackgroundMedication-Related Impulse Control and Repetitive Behaviors in Parkinson Disease
Voon, V., & Fox, S. H. (2007) · Archives of Neurology · doi:10.1001/archneur.64.8.1089 · Cited by 397
Toward a neurobiology of delusions
Corlett, P. R., Taylor, J. R., Wang, X.-J., Fletcher, P. C., & Krystal, J. H. (2010) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2010.06.007 · Cited by 394
“This fits comfortably with the role of dopamine in controlling signal to noise and the numerous proposals that dopamine (at least in terms of its tonic discharge rates) encodes uncertainty or violation of expectations (Fiorillo et al., 2003; Preuschoff et al., 2006).”
BackgroundDissecting the diversity of midbrain dopamine neurons
Roeper, J. (2013) · Trends in Neurosciences · doi:10.1016/j.tins.2013.03.003 · Cited by 373
Ji, H., & Shepard, P. D. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.0958-07.2007 · Cited by 388
“These findings have led to the proposition that DA neurons encode an error signal that reports the fidelity of ongoing events relative to a previous prediction (Schultz et al., 1997; Fiorillo et al., 2003; Satoh et al., 2003; Nakahara et al., 2004; Morris et al., 2006).”
BackgroundRisk-sensitive neurons in macaque posterior cingulate cortex
McCoy, A. N., & Platt, M. L. (2005) · Nature Neuroscience · doi:10.1038/nn1523 · Cited by 379
“Other investigators have shown recently that the responses of a population of midbrain dopamine neurons are selectively enhanced by reward uncertaint”
BackgroundStocco, A., Lebiere, C., & Anderson, J. R. (2010) · Psychological Review · doi:10.1037/a0019077 · Cited by 350
The nucleus accumbens as part of a basal ganglia action selection circuit
Nicola, S. M. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0510-4 · Cited by 362
“The magnitude of the burst depends on the strength of the reward-predictive information: it tends to be larger for cues that predict reward with higher probability or that predict larger rewards (Fiorillo et al. 2003; Satoh et al. 2003; Schultz et al. 1998; Tobler et al. 2003).”
BackgroundDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381
“In particular, it draws on the same notions that link dopamine to the encoding of uncertainty [44] and adaptive responses to changes in neuronal signal to noise levels [45], [46].”
BackgroundInfluentialBeyond the FRN: Broadening the time-course of EEG and ERP components implicated in reward processing
Glazer, J. E., Kelley, N. J., Pornpattananangkul, N., Mittal, V. A., & Nusslock, R. (2018) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2018.02.002 · Cited by 361
“There are two major theories of Cue-N2 function to explain a reduced N2 following reward cues: cognitive-control and template mismatch (Folstein and Van Petten, 2008).”
BackgroundThe prefrontal cortex, pathological anxiety, and anxiety disorders
Kenwood, M. M., Kalin, N. H., & Barbas, H. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-021-01109-z · Cited by 312
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379
“…studies have emphasized the importance of reward prediction errors in associative learning (Montague et al., 1995, 1996; Schultz et al., 1997; Fiorillo et al., 2003; Bayer and Glimcher, 2005; Schultz, 2006; Matsumoto and Hikosaka, 2007, 2009), the absolute value of the reward prediction…”
BackgroundReward Encoding in the Monkey Anterior Cingulate Cortex
Amiez, C., Joseph, J. P., & Procyk, E. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj046 · Cited by 351
“, 2003) and mesencephalic dopaminergic neurons (Fiorillo et al., 2003).”
BackgroundPrediction error as a linear function of reward probability is coded in human nucleus accumbens
Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.001 · Cited by 376
Variation in dopamine genes influences responsivity of the human reward system
Dreher, J.-C., Kohn, P., Kolachana, B., Weinberger, D. R., & Berman, K. F. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0805517106 · Cited by 376
Dialogues on prediction errors
Niv, Y., & Schoenbaum, G. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.03.006 · Cited by 344
Midbrain dopamine neurons control judgment of time
Soares, S., Atallah, B. V., & Paton, J. J. (2016) · Science · doi:10.1126/science.aah5234 · Cited by 360
The origins of altruism in offspring care.
Preston, S. D. (2013) · Psychological Bulletin · doi:10.1037/a0031755 · Cited by 368
Motor Adaptation as a Process of Reoptimization
Izawa, J., Rane, T., Donchin, O., & Shadmehr, R. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.5359-07.2008 · Cited by 347
“…on costs and rewards of the task, and dopamine appears to be a crucial neurotransmitter that responds to reward prediction errors (Schultz et al., 1997) and reward uncertainty (Fiorillo et al., 2003), it is possible that the process of finding an optimal control policy depends on the basal ganglia.”
From drugs to deprivation: A Bayesian framework for understanding models of psychosis
Corlett, P. R., Frith, C. D., & Fletcher, P. C. (2009) · Psychopharmacology · doi:10.1007/s00213-009-1561-0 · Cited by 367
“We believe that a more in-depth consideration of this distinction, in the setting of psychosis, will be very useful in the future.”
BackgroundAdamantidis, A. R., Tsai, H.-C., Boutrel, B., Zhang, F., Stuber, G. D., Budygin, E. A., Tourino, C., Bonci, A., Deisseroth, K., & de Lecea, L. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.2246-11.2011 · Cited by 357
“Those signals update the predictive value of stimuli during associative learning for future reinforcement of behavioral actions (Fiorillo et al., 2003; Bayer and Glimcher, 2005; Tobler et al., 2005; Day et al., 2007; Schultz, 2007).”
BackgroundNeural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356
“…onset reflects a prediction error comparing the value of that stimulus to zero, rather than to the average reward obtainable in the experiment; Fiorillo et al., 2003; Tobler et al., 2005), the prediction error is (t) V(tstimulus), that is, the predicted reward for this trial, which is the…”
BackgroundInfluentialSterling, P. (2004) · Allostasis, Homeostasis, and the Costs of Physiological Adaptation · doi:10.1017/cbo9781316257081.004 · Cited by 353
An fMRI study of reward-related probability learning
Delgado, M. R., Miller, M. M., Inati, S., & Phelps, E. A. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.10.002 · Cited by 346
“This is in accordance with physiology (Fiorillo et al., 2003) and neuroimaging (Aron et al., 2004) data showing that dopamine neurons in the midbrain are sensitive to predictability by varying their firing rates according to uncertainty .”
ResultDopaminergic Modulation of Risk-Based Decision Making
St Onge, J. R., & Floresco, S. B. (2009) · Neuropsychopharmacology · doi:10.1038/npp.2008.121 · Cited by 336
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346
Reinforcement Learning Signals Predict Future Decisions
Cohen, M. X., & Ranganath, C. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4421-06.2007 · Cited by 333
Animal Foraging and the Evolution of Goal‐Directed Cognition
Hills, T. T. (2006) · Cognitive Science · doi:10.1207/s15516709cog0000_50 · Cited by 331
Altered Reward Processing in Women Recovered From Anorexia Nervosa
Wagner, A., Aizenstein, H., Venkatraman, V. K., Fudge, J., May, J. C., Mazurkewicz, L., Frank, G. K., Bailer, U. F., Fischer, L., Nguyen, V., et al. (2007) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2007.07040575 · Cited by 342
Mind the gap: bridging economic and naturalistic risk-taking with cognitive neuroscience
Schonberg, T., Fox, C. R., & Poldrack, R. A. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.10.002 · Cited by 330
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340
“…of divergence or prediction errors (in their nuanced or non-classical sense) and partly resolves the dialectic between dopamine as reporting reward prediction errors (Schultz et al., 1997) and the predictability of rewards (Fiorillo et al., 2003; Redgrave and Gurney, 2006; Schultz et al., 2008).”
BackgroundKnowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339
Artificial Intelligence in Education
Holmes, W. (2019) · Encyclopedia of Education and Information Technologies · doi:10.1007/978-3-319-60013-0_107-1 · Cited by 336
Prospective prevalence of pathologic gambling and medication association in Parkinson disease
Voon, V., Hassan, K., Zurowski, M., Duff-Canning, S., de Souza, M., Fox, S., Lang, A. E., & Miyasaki, J. (2006) · Neurology · doi:10.1212/01.wnl.0000218206.20920.4d · Cited by 329
In vivo imaging identifies temporal signature of D1 and D2 medium spiny neurons in cocaine reward
Calipari, E. S., Bagot, R. C., Purushothaman, I., Davidson, T. J., Yorgason, J. T., Peña, C. J., Walker, D. M., Pirpinias, S. T., Guise, K. G., Ramakrishnan, C., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521238113 · Cited by 335
The mesoaccumbens dopamine in coping with stress
Cabib, S., & Puglisi-Allegra, S. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.012 · Cited by 329
“Moreover, one third of DA neurons show risk-related activations with longer latencies (about 1 s), slower time courses and lower peaks compared with responses to stimuli and reward (Fiorillo et al., 2003; Schultz et al., 2008).”
Effects of Expectation on Placebo-Induced Dopamine Release in Parkinson Disease
Lidstone, S. C., Schulzer, M., Dinelle, K., Mak, E., Sossi, V., Ruth, T. J., de la Fuente-Fernández, R., Phillips, A. G., & Stoessl, A. J. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.88 · Cited by 323
The role of the striatum in aversive learning and aversive prediction errors
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0161 · Cited by 328
“2005) and probability (Fiorillo et al. 2003) of the rewarding outcome.”
BackgroundDynamic modulation of decision biases by brainstem arousal systems
de Gee, J. W., Colizoli, O., Kloosterman, N. A., Knapen, T., Nieuwenhuis, S., & Donner, T. H. (2017) · eLife · doi:10.7554/elife.23232 · Cited by 303
“We computed the combined ‘neuromodulatory brainstem signal’ as the linear combination of responses from LC, VTA, SN, and BF that maximized the correlation to TPR (Materials and methods; correlation coefficient across subjects, 0.146 (±0.014 s.e.m.)).”
BackgroundInfluentialStress and the adolescent brain
Tottenham, N., & Galván, A. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.07.030 · Cited by 324
Baldo, B. A., & Kelley, A. E. (2007) · Psychopharmacology · doi:10.1007/s00213-007-0741-z · Cited by 317
“The importance of novelty vs predictability of reward in determining functional responses of the dopamine system has also been demonstrated in an influential series of ventral tegmental area (VTA) electrophysiological recording studies in primates (Fiorillo et al. 2003; Hollerman and Schultz 1998; Mirenowicz and Schul…”
BackgroundThe Primate Thalamus Is a Key Target for Brain Dopamine
Sánchez-González, M. Á., García-Cabezas, M. Á., Rico, B., & Cavada, C. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0968-05.2005 · Cited by 319
Game theory and neural basis of social decision making
Lee, D. (2008) · Nature Neuroscience · doi:10.1038/nn2065 · Cited by 299
Dysfunctional reward circuitry in obsessive-compulsive disorder
Figee, M., Vink, M., de Geus, F., Vulink, N., Veltman, D. J., Westenberg, H., & Denys, D. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.12.003 · Cited by 315
Increased Occupancy of Dopamine Receptors in Human Striatum during Cue-Elicited Cocaine Craving
Wong, D. F., Kuwabara, H., Schretlen, D. J., Bonson, K. R., Zhou, Y., Nandi, A., Brašić, J. R., Kimes, A. S., Maris, M. A., Kumar, A., et al. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301194 · Cited by 311
“…of cocaine and the maintenance of addiction are known to involve dopaminergic neurotransmission (Amara and Sonders, 1998; Cannon and Bseikri, 2004; Fiorillo et al, 2003; Schultz, 2001; Volkow et al, 2004), previous studies of the effect of cocaine craving on dopaminergic functioning have not…”
BackgroundThe temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310
Self-citationScott, D. J., Heitzeg, M. M., Koeppe, R. A., Stohler, C. S., & Zubieta, J.-K. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2577-06.2006 · Cited by 297
“In this context, mesolimbic DA cells are engaged in the encoding of both reward value and its uncertainty, to appropriately orient the responses of the organism (Fiorillo et al., 2003; Hsu et al., 2005; Tobler et al., 2005).”
BackgroundHart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.2489-13.2014 · Cited by 293
“A large number of studies have now correlated dopaminergic neural activity with the reward prediction error (RPE) term in temporal difference models of learning (Sutton, 1988, Montague et al., 1996, Schultz et al., 1997, Waelti et al., 2001, Flagel et al., 2011, Steinberg et al., 2013).”
BackgroundResultCortico-limbic-striatal circuits subserving different forms of cost-benefit decision making
Floresco, S. B., Onge, J. R. S., Ghods-Sharifi, S., & Winstanley, C. A. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.375 · Cited by 305
Balodis, I. M., Kober, H., Worhunsky, P. D., Stevens, M. C., Pearlson, G. D., & Potenza, M. N. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.01.006 · Cited by 305
“Although the findings were less phase-specific than originally hypothesized, the brain areas in which differences were observed represent predominant projection areas of the mesocorticolimbic dopamine system, which is consistent with dopamine’s role in reward processing (66,67).”
ResultThe role of mesolimbic dopamine in the development and maintenance of ethanol reinforcement
Gonzales, R. A., Job, M. O., & Doyon, W. M. (2004) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2004.06.002 · Cited by 300
Ely, J., Frankel, A., & Kamenica, E. (2015) · Journal of Political Economy · doi:10.1086/677350 · Cited by 291
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300
“…induced attention closely tied to reward-predicting stimuli and not occurring with any other events, although these forms would be difficult to disentangle, and it may be that more sustained dopamine activations serve some attentional functions by coding reward uncertainty (Fiorillo et al., 2003).”
BackgroundSelf-citationCorrelated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298
“Recently, they supported this hypothesis by finding that phasic responses of DA neurons varied monotonically with the change of reward probability (Fiorillo et al., 2003).”
InfluentialFeeding and reward: Perspectives from three rat models of binge eating
Corwin, R. L., Avena, N. M., & Boggiano, M. M. (2011) · Physiology & Behavior · doi:10.1016/j.physbeh.2011.04.041 · Cited by 289
Adolescence: what do transmission, transition and translation have to do with it?
Casey, B. J., Duhoux, S., & Cohen, M. M. (2010) · Neuron · doi:10.1016/j.neuron.2010.08.033 · Cited by 297
“…evidence for this model, framing the review in the context of key questions asked by ethologists to understand animal behavior (Tinbergen, 1974): (1) what may cause or control adolescent behavior; (2) how does the behavior change across development; and (3) how may the behavior be adaptive?”
BackgroundResultInfluentialCaudate Encodes Multiple Computations for Perceptual Decisions
Ding, L., & Gold, J. I. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2894-10.2010 · Cited by 284
“In a positive reinforcement-based paradigm like we used, evaluation likely encodes reward expectation or uncertainty (Sutton and Barto, 1998; Fiorillo et al., 2003; Kepecs et al., 2008; Kiani and Shadlen, 2009).”
A direct projection from superior colliculus to substantia nigra for detecting salient visual events
Comoli, E., Coizet, V., Boyes, J., Bolam, J. P., Canteras, N. S., Quirk, R. H., Overton, P. G., & Redgrave, P. (2003) · Nature Neuroscience · doi:10.1038/nn1113 · Cited by 295
Dorsolateral Prefrontal Cortex Drives Mesolimbic Dopaminergic Regions to Initiate Motivated Behavior
Ballard, I. C., Murty, V. P., Carter, R. M., MacInnes, J. J., Huettel, S. A., & Adcock, R. A. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.0895-11.2011 · Cited by 288
“The VTA, NAcc, and PFC are each strongly implicated in motivation to obtain reward: VTA neurons respond to reward cues and increase their activity before goal-directed behavior (Ljungberg et al., 1992; Schultz, 1998; Fiorillo et al., 2003).”
BackgroundNeural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290
“Early studies found a crucial role for dopamine in reward probability and risk (Fiorillo et al. 2003).”
BackgroundThe anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289
“Similarly, dopamine shows characteristic phasic responses to sensory cues that predict rewards, which return to tonic firing levels that may encode uncertainty or predictability [60,61]; — precision increases whenever a predictable path to a goal is signified by sensory input.”
BackgroundSeger, C. A. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.07.010 · Cited by 274
“Dopamine neurons also code for the degree of uncertainty or risk about the predicted reward during the time period preceding the reward (Fiorillo et al., 2003).”
BackgroundDopamine Neurons Can Represent Context-Dependent Prediction Error
Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., & Hikosaka, O. (2004) · Neuron · doi:10.1016/s0896-6273(03)00869-9 · Cited by 286
“The TD error changes as the value function changes (Equationis consistent with a previous study (Fiorillo et al., 2003).”
InfluentialCompulsive use of dopamine replacement therapy in Parkinson's disease: reward systems gone awry?
Lawrence, A. D., Evans, A. H., & Lees, A. J. (2003) · The Lancet Neurology · doi:10.1016/s1474-4422(03)00529-5 · Cited by 282
Dopamine neurons encode performance error in singing birds
Gadagkar, V., Puzerey, P. A., Chen, R., Baird-Daniel, E., Farhang, A. R., & Goldberg, J. H. (2016) · Science · doi:10.1126/science.aah6837 · Cited by 268
How visual stimuli activate dopaminergic neurons at short latency
Dommett, E., Coizet, V., Blaha, C. D., Martindale, J., Lefebvre, V., Walton, N., Mayhew, J. E. W., Overton, P. G., & Redgrave, P. (2005) · Science · doi:10.1126/science.1107026 · Cited by 285
Individual differences in extraversion and dopamine genetics predict neural reward responses
Cohen, M. X., Young, J., Baek, J.-M., Kessler, C., & Ranganath, C. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.09.018 · Cited by 272
“Indeed, the DRD2 system is heavily involved in reward anticipation and reward prediction [29,70,73], but it is possible that the presence of the DRD2 Taq1A allele only affects experiential or consumatory reward processes, and not all reward processes.”
BackgroundOpposing BOLD responses to reciprocated and unreciprocated altruism in putative reward pathways
Rilling, J. K., Sanfey, A. G., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2004) · NeuroReport · doi:10.1097/00001756-200411150-00022 · Cited by 281
“[14] found that visual stimuli that predicted rewards with a probability between 0 and 1 resulted in both an increased response to reward and a decreased response to its absence, however the magnitude of the increase exceeded that of the decrease when the probability of reward dropped below 0.”
Background(2004) · PubMed · Cited by 279
The Placebo Effect in Sports Performance: A Brief Review
Beedie, C. J., & Foad, A. J. (2009) · Sports Medicine · doi:10.2165/00007256-200939040-00004 · Cited by 261
“For example, Fiorillo and co-workers([50]) demonstrated that placebo-induced dopamine activation is maximal when the probability of experiencing a beneficial outcome is 0.”
BackgroundHuman Midbrain Sensitivity to Cognitive Feedback and Uncertainty During Classification Learning
Aron, A. R., Shohamy, D., Clark, J., Myers, C., Gluck, M. A., & Poldrack, R. A. (2004) · Journal of Neurophysiology · doi:10.1152/jn.01209.2003 · Cited by 269
“One additional subject was excluded due to extensive signal dropout, leaving 15 subjects in the analyzed dataset.”
BackgroundResultInfluentialSteiger, T. K., & Bunzeck, N. (2017) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2017.00001 · Cited by 268
“During Pavlovian conditioning, a cue predicts a subsequent reward and the neuronal response, that is initially observed to the delivery of the reward, shifts to the reward-predicting cue, suggesting a role of the dopaminergic system in reward anticipation (Fiorillo et al., 2003).”
BackgroundJoshua, M., Adler, A., Mitelman, R., Vaadia, E., & Bergman, H. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3839-08.2008 · Cited by 273
“In line with the predictions of reinforcement learning theories, the DAN discharge decreases with omission of predicted rewards (Schultz et al., 1993; Fiorillo et al., 2003; Matsumoto and Hikosaka, 2007).”
ResultExley, R., & Cragg, S. J. (2008) · British Journal of Pharmacology · doi:10.1038/sj.bjp.0707510 · Cited by 250
“…rates (typically 2–5 Hz) to either a phasic increase (burst, 15– 100 Hz) or a decrease (pause) according to the presentation or omission, respectively, of a reward (Hyland et al., 2002; Schultz, 2002; Fiorillo et al., 2003; Tobler et al., 2003, 2005; Morris et al., 2004; Bayer and Glimcher, 2005).”
BackgroundDissociable Effects of Dopamine and Serotonin on Reversal Learning
den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., Franke, B., & Cools, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.030 · Cited by 255
Anticipatory reward processing in addicted populations: a focus on the monetary incentive delay task
Balodis, I. M., & Potenza, M. N. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.08.020 · Cited by 267
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267
Attention-Gated Reinforcement Learning of Internal Representations for Classification
Roelfsema, P. R., & Ooyen, A. V. (2005) · Neural Computation · doi:10.1162/0899766054615699 · Cited by 260
“If the reward is delivered immediately after the animal’s response, then the reward-evoked response of dopamine neurons in the midbrain depends on reward like δ in AGREL (Fiorillo et al., 2003; Morris et al., 2004).”
BackgroundDopamine Neuron-Specific Optogenetic Stimulation in Rhesus Macaques
Stauffer, W. R., Lak, A., Yang, A., Borel, M., Paulsen, O., Boyden, E. S., & Schultz, W. (2016) · Cell · doi:10.1016/j.cell.2016.08.024 · Cited by 257
“…the fast, phasic responses of dopamine neurons code reward prediction errors (Bayer et al., 2007; Bromberg-Martin et al., 2010; Eshel et al., 2015; Fiorillo et al., 2003; Hollerman and Schultz, 1998; Kobayashi and Schultz, 2008; Lak et al., 2014; Ljungberg et al., 1992; Mirenowicz and Schultz,…”
BackgroundSelf-citationReward functions of the basal ganglia
Schultz, W. (2016) · Journal of Neural Transmission · doi:10.1007/s00702-016-1510-0 · Cited by 248
“About one-third of dopamine neurons show also a slower, pre-reward activation that varies with reward risk (mixture of variance and skewness) (Fiorillo et al. 2003), which constitutes the first neuronal risk signal ever observed.”
BackgroundInfluentialSelf-citationPathological gambling in Parkinson's disease improves on chronic subthalamic nucleus stimulation
Ardouin, C., Voon, V., Worbe, Y., Abouazar, N., Czernecki, V., Hosseini, H., Pelissolo, A., Moro, E., Lhommée, E., Lang, A. E., et al. (2006) · Movement Disorders · doi:10.1002/mds.21098 · Cited by 261
Coding of Reward Risk by Orbitofrontal Neurons Is Mostly Distinct from Coding of Reward Value
O'Neill, M., & Schultz, W. (2010) · Neuron · doi:10.1016/j.neuron.2010.09.031 · Cited by 258
“…cortex, and insular cortex have been identified as key areas in processing information about uncertain rewards (Christopoulos et al., 2009; Fiorillo et al., 2003; Huettel et al., 2006; Knoch et al., 2006; Kuhnen and Knutson, 2005; Levy et al., 2010; McCoy and Platt, 2005; Preuschoff et…”
BackgroundInfluentialSelf-citationWen, C., Ogura, Y., & Matsushima, T. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00476 · Cited by 259
What is reinforced by phasic dopamine signals?
Redgrave, P., Gurney, K., & Reynolds, J. (2008) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 261
“This should be surprising because in a later study, when a conditioned stimulus predicted a reward with 50% reliability, a second phasic DA neuronal response was regularly elicited on trials when reward was subsequently delivered (Fiorillo et al., 2003).”
BackgroundInfluentialPredictability and Uncertainty in the Pleasure of Music: A Reward for Learning?
Gold, B. P., Pearce, M. T., Mas-Herrero, E., Dagher, A., & Zatorre, R. J. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.0428-19.2019 · Cited by 250
“In goal-directed learning, dopamine neurons encode both uncertainty leading up to predicted outcomes and “reward prediction errors” (RPEs) afterward, which signal how much better or worse the outcomes were than predicted (Fiorillo et al., 2003).”
BackgroundComputational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
“While midbrain neuron activity has been found to reflect precision-weighting for rewards [52,53], it has been unclear whether such precision-weighting extends beyond rewards and the”
BackgroundDopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257
Self-citationDopamine dysregulation syndrome in Parkinson's disease
Evans, A. H., & Lees, A. J. (2004) · Current Opinion in Neurology · doi:10.1097/01.wco.0000137528.23126.41 · Cited by 257
Valence and salience contribute to nucleus accumbens activation
Cooper, J. C., & Knutson, B. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.08.009 · Cited by 256
“Electrophysiologists studying nonhuman primates responding to juice cues have found a similar pattern in midbrain dopamine neuron firing (Fiorillo et al., 2003).”
BackgroundResultLevodopa: Faster and better word learning in normal humans
Knecht, S., Breitenstein, C., Bushuven, S., Wailke, S., Kamping, S., Flöel, A., Zwitserlood, P., & Ringelstein, E. B. (2004) · Annals of Neurology · doi:10.1002/ana.20125 · Cited by 245
“More sustained release of dopamine occurs during prolonged periods of uncertainty about payoffs and may promote learning by allocating attention to predictors for reward.(3) Future studies comparing the effect of L-dopa and dopamine agonists may help to clarify the role of phasic dopamine release on the overall succes…”
BackgroundAnterior cingulate error‐related activity is modulated by predicted reward
Amiez, C., Joseph, J., & Procyk, E. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04170.x · Cited by 234
“Our results also support the idea of a link between anterior cingulate activity and midbrain dopaminergic activity that encodes ‘reward prediction errors’, and are modulated during trial and error and by expected values (Fiorillo et al., 2003; Satoh et al., 2003; Tobler et al., 2005).”
Result
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.