Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 601–700 of 1,007
Horvath, J. C., & Donoghue, G. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00377 · Cited by 80
Task-evoked pupil responses reflect internal belief states
Colizoli, O., de Gee, J. W., Urai, A. E., & Donner, T. H. (2018) · Scientific Reports · doi:10.1038/s41598-018-31985-3 · Cited by 80
Impulse Control Disorders and Pathological Gambling in Patients With Parkinson Disease
Stamey, W., & Jankovic, J. (2008) · The Neurologist · doi:10.1097/nrl.0b013e31816606a7 · Cited by 70
Motivation and movement: the effect of monetary incentive on performance speed
Mir, P., Trender‐Gerhard, I., Edwards, M. J., Schneider, S. A., Bhatia, K. P., & Jahanshahi, M. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2583-5 · Cited by 75
“Experiments probing reward mechanisms often use a stimulus–response learning paradigm to examine the eVect of the presentation of a reward-predicting stimulus on, for example, neuronal Wring in non-human primates with implanted recording electrodes ( Fiorillo et al. 2003 ) or change in BOLD signal in functional magnet…”
MethodDecision neuroscience: neuroeconomics
Smith, D. V., & Huettel, S. A. (2010) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.73 · Cited by 83
Theoretical and Empirical Studies of Learning
Niv, Y., & Montague, P. R. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00022-1 · Cited by 75
“…have been shown to elicit a phasic dopaminergic response that is indeed proportional to the magnitude and/or probability of the expected reward ( Fiorillo et al ., 2003 ; Morris et al ., 2004 ; Tobler et al ., 2005 ; Figure 22.4a, b ), and firing patterns in tasks involving probabilistic…”
BackgroundMethodInfluentialCognitive Flexibility Deficits Following 6-OHDA Lesions of the Rat Dorsomedial Striatum
Grospe, G. M., Baker, P. M., & Ragozzino, M. E. (2018) · Neuroscience · doi:10.1016/j.neuroscience.2018.01.032 · Cited by 70
Sacheli, M., Murray, D., Vafai, N. M., Cherkasova, M. V., Dinelle, K., Shahinfard, E., Neilson, N., McKenzie, J., Schulzer, M., Appel‐Cresswell, S., et al. (2018) · Movement Disorders · doi:10.1002/mds.27498 · Cited by 74
Encoding of Probabilistic Rewarding and Aversive Events by Pallidal and Nigral Neurons
Joshua, M., Adler, A., Rosin, B., Vaadia, E., & Bergman, H. (2008) · Journal of Neurophysiology · doi:10.1152/jn.90764.2008 · Cited by 68
“Encoding of reward prediction error would predict the opposite trend in the coding of reward probability in the cue and outcome (Fiorillo et al., 2003;Morris et al., 2004).”
BackgroundGarcia, B., Cerrotti, F., & Palminteri, S. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2019.0665 · Cited by 70
“(a) Sample size and number of trials listed in two electrophysiological studies [52,53], two human fMRI studies [25,56] and two human behavioural studies [54,55].”
BackgroundFiorillo, C. D., Tobler, P. N., & Schultz, W. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-7 · Cited by 65
Self-citationIsoda, M., & Tanji, J. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00367.2003 · Cited by 68
Hellberg, S. N., Russell, T. I., & Robinson, M. J. F. (2019) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-018-00662-3 · Cited by 65
Watanabe, N., Sakagami, M., & Haruno, M. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3400-12.2013 · Cited by 65
“Our probabilistic task design allows us to detect the RPE signal in the brain even after the completion of learning (Fiorillo et al., 2003; McClure et al., 2003; Haruno and Kawato, 2006).”
MethodResultComponents and characteristics of the dopamine reward utility signal
Stauffer, W. R., Lak, A., Kobayashi, S., & Schultz, W. (2015) · The Journal of Comparative Neurology · doi:10.1002/cne.23880 · Cited by 72
“Some dopamine neurons show an additional, slower response that codes reward risk (Fiorillo et al., 2003) and may affect the dopamine release induced by the prediction error signal.”
BackgroundSelf-citationMyers, C. E., Sheynin, J., Balsdon, T., Luzardo, A., Beck, K. D., Hogarth, L., Haber, P., & Moustafa, A. A. (2016) · Behavioural Brain Research · doi:10.1016/j.bbr.2015.09.018 · Cited by 64
“…in animals DA neurons respond especially to unexpected rewards (Hollerman & Schultz, 1998; Schultz, 1998), as well as to stimuli that signal upcoming or predicted reward (Fiorillo, Tobler, & Schultz, 2003) and reduce firing in response to omission of expected reward (Hollerman & Schultz, 1998).”
BackgroundCategorization=decision making+generalization
Seger, C. A., & Peterson, E. J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.015 · Cited by 79
“Risk is reflected in the firing patterns of the same dopamine neurons that are involved in reward prediction: risk is typically coded as a gradually increasing firing rate extending from the time of stimulus presentation to reward delivery (Fiorillo et al., 2003).”
BackgroundThe neural dynamics of reward value and risk coding in the human orbitofrontal cortex
Li, Y., Vanni-Mercier, G., Isnard, J., Mauguière, F., & Dreher, J.-C. (2016) · Brain · doi:10.1093/brain/awv409 · Cited by 66
“The medial OFC has been shown to be a core component of a risksensitive processing network comprising the basal ganglia, amygdala, parietal cortex, anterior cingulate cortex and insular cortex (Fiorillo et al., 2003; Hsu et al., 2005; McCoy and Platt, 2005; Preuschoff et al., 2006; Christopoulos et al., 2009; Tobler e…”
BackgroundInfluentialKroemer, N. B., Guevara, A., Ciocanea Teodorescu, I., Wuttig, F., Kobiella, A., & Smolka, M. N. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.07.060 · Cited by 64
“…several studies have found that (subpopulations of) dopaminergic midbrain neurons reflect net utility based on discounting of the value as a function of risk/uncertainty (Fiorillo et al., 2003; Morris et al., 2006), delay (Kobayashi and Schultz, 2008), and effort (Pasquereau and Turner, 2013).”
BackgroundWhat makes the dorsomedial frontal cortex active during reading the mental states of others?
Isoda, M., & Noritake, A. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00232 · Cited by 64
“It has been shown that dopaminecontaining neurons in the midbrain signal uncertainty in the reward prediction (Fiorillo et al., 2003).”
BackgroundDopaminergic signaling of uncertainty and the aetiology of gambling addiction
Zack, M. H., George, R. S., & Clark, L. (2019) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2019.109853 · Cited by 66
The Cambridge Handbook of Substance and Behavioral Addictions
Sussman, S. Y. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108632591 · Cited by 69
Role of ventral striatum in reward-based decision making
Heekeren, H. R., Wartenburger, I., Marschner, A., Mell, T., Villringer, A., & Reischies, F. M. (2007) · NeuroReport · doi:10.1097/wnr.0b013e3281532bd7 · Cited by 65
Yu, R., Zhou, W., & Zhou, X. (2011) · PLoS ONE · doi:10.1371/journal.pone.0029633 · Cited by 60
“Uncertainty was also measured as entropy [4] and similar effects were observed.”
BackgroundMethodInfluentialBehavioral and neural predictors of upcoming decisions
Cohen, M. X., & Ranganath, C. (2005) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.5.2.117 · Cited by 78
Combined effects of peer presence, social cues, and rewards on cognitive control in adolescents
Breiner, K., Li, A., Cohen, A. O., Steinberg, L. D., Bonnie, R. J., Scott, E. S., Taylor‐Thompson, K., Rudolph, M. D., Chein, J. M., Richeson, J. A., et al. (2018) · Developmental Psychobiology · doi:10.1002/dev.21599 · Cited by 68
“…between orbitofrontal and subcortical regions (Haber & Knutson, 2009) have been associated with value updating, reward prediction, and motivated behavior (Fiorillo, Tobler, & Schultz, 2003; Hare, Doherty, Camerer, Schultz, & Rangel, 2008; Rangel & Hare, 2010; Schultz, Dayan, & Montague, 1997).”
Gonzalez, M. Z., Allen, J. P., & Coan, J. A. (2016) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2016.10.003 · Cited by 57
“Moreover, our results are consonant with work suggesting, for example, that midbrain dopaminergic activity associated with reward anticipation increases as the unpredictability of the reward increases (Fiorillo et al., 2003), or that, in rats, exposure to physical stress (e.”
BackgroundResultRisky decision-making predicts dopamine release dynamics in nucleus accumbens shell
Freels, T. G., Gabriel, D. B. K., Lester, D. B., & Simon, N. W. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0527-0 · Cited by 66
Neural substrates of appetitive and aversive prediction error
Iordanova, M. D., Yau, J. O.-Y., McDannald, M. A., & Corbit, L. H. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.10.029 · Cited by 66
Retrograde signaling changes the venue of postsynaptic inhibition in rat substantia nigra
Yanovsky, Y., Mades, S., & Misgeld, U. (2003) · Neuroscience · doi:10.1016/s0306-4522(03)00607-9 · Cited by 64
Behavioral specifications of reward-associated long-term memory enhancement in humans
Wittmann, B. C., Dolan, R. J., & Düzel, E. (2011) · Learning & Memory · doi:10.1101/lm.1996811 · Cited by 62
“However, studies in animals show that that when outcome uncertainty is very high (50% reward probability) there is a ramping up of dopaminergic firing in the time window between the cue signalling the uncertain outcome and the outcome itself (Fiorillo et al., 2003).”
Gu, R., Jiang, Y., Kiser, S., Black, C. L., Broster, L. S., Luo, Y., & Kelly, T. H. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.07.013 · Cited by 66
Woodworth, H. L., Perez‐Bonilla, P., Beekly, B. G., Lewis, T. J., & Leinninger, G. M. (2018) · eNeuro · doi:10.1523/eneuro.0191-17.2018 · Cited by 64
Kim, J. Y., Chung, E. J., Lee, W. Y., Shin, H. Y., Lee, G. H., Choe, Y. S., Choi, Y. S., & Kim, B. J. (2007) · Movement Disorders · doi:10.1002/mds.21787 · Cited by 62
Momentary reward induce changes in excitability of primary motor cortex
Thabit, M. N., Nakatsuka, M., Koganemaru, S., Fawi, G., Fukuyama, H., & Mima, T. (2011) · Clinical Neurophysiology · doi:10.1016/j.clinph.2011.02.021 · Cited by 68
“Since the reward probability in our studywas low, the activation of dopamine neuronsmight be substantial (Fiorillo et al., 2003).”
BackgroundAging and the neuroeconomics of decision making: A review
Brown, S. B. R. E., & Ridderinkhof, K. R. (2009) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.9.4.365 · Cited by 71
Reinforcement learning models of the dopamine system and their behavioral implications
Daw, N. D., & Touretzky, D. S. (2003) · Cited by 69
Eisinger, R. S., Urdaneta, M. E., Foote, K. D., Okun, M. S., & Gunduz, A. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00385 · Cited by 64
“, low probability) corresponds to increased dopamine and the phasic activity of dopamine corresponds to the reward prediction error (Fiorillo et al., 2003).”
BackgroundTransdiagnostic neuroimaging of reward system phenotypes in ADHD and comorbid disorders
Grimm, O., van Rooij, D., Hoogman, M., Klein, M., Buitelaar, J. K., Franke, B., Reif, A., & Plichta, M. M. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.06.025 · Cited by 68
“The variation of this reward contingency is a decisive parameter and significantly influences the neuronal response behaviour: Fiorillo et al. (2003) showed that sustained dopamine signals between cue and reward are highest when reward results occur under conditions of uncertainty.”
BackgroundEnvironmental morale and motivation
Frey, B. S., & Stutzer, A. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511490118.017 · Cited by 101
Formal Learning Theory Dissociates Brain Regions with Different Temporal Integration
Gläscher, J. P., & Büchel, C. (2005) · Neuron · doi:10.1016/j.neuron.2005.06.008 · Cited by 66
“Our pat- Clure et al., 2003; O'Doherty et al., 2003; Seymour et tern of prediction modulation in the FFA and PPA supal ., 2004 ).”
InfluentialFouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G. (2017) · Scientific Reports · doi:10.1038/s41598-017-04507-w · Cited by 61
“σ β φ = − − P i v i v i ( ) ( ( ( ) ( )) ) (3) A A B”
Nicotinic receptors in the ventral tegmental area promote uncertainty-seeking
Naudé, J., Tolu, S., Dongelmans, M., Torquet, N., Valverde, S., Rodriguez, G., Pons, S., Maskos, U., Mourot, A., Marti, F., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4223 · Cited by 61
Seeking motivation and reward: Roles of dopamine, hippocampus, and supramammillo-septal pathway
Kesner, A. J., Calva, C. B., & Ikemoto, S. (2022) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2022.102252 · Cited by 63
“activity and the occurrences of salient stimuli and uncertainty, although evidence for the latter is thin at this time (Fiorillo et al., 2003; Stauffer et al., 2014).”
BackgroundHitsman, B. L., MacKillop, J. M., Lingford-Hughes, A. R., Williams, T. A., Ahmad, F., Adams, S. H., Nutt, D. J., & Munafo, M. R. (2007) · Psychopharmacology · doi:10.1007/s00213-007-0995-5 · Cited by 63
“Reward probability theories (Fiorillo et al. 2003), in contrast, argue that dopamine encodes prediction error (i.”
BackgroundResultNeurobiological underpinnings of reward anticipation and outcome evaluation in gambling disorder
Linnet, J. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00100 · Cited by 53
Understanding the Placebo Effect: Contributions from Neuroimaging
Lidstone, S. C., & Stoessl, A. J. (2007) · Molecular Imaging and Biology · doi:10.1007/s11307-007-0098-z · Cited by 61
“Dopamine neurons also demonstrate a more sustained, tonic firing pattern during the reward expectation period [57], the magnitude of which varies with the probability that the reward will occur: neuron activity is maximal at p = 0.”
BackgroundRamping activity in midbrain dopamine neurons signifies the use of a cognitive mapPreprint
Guru, A., Seo, C., Post, R. J., Kullakanda, D. S., Schaffer, J. A., & Warden, M. R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.05.21.108886 · Cited by 62
P300 amplitude variations, prior probabilities, and likelihoods: a Bayesian ERP study
Kopp, B., Seer, C., Lange, F., Kluytmans, A., Kolossa, A., Fingscheidt, T., & Hoijtink, H. (2016) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-016-0442-3 · Cited by 60
Neuroscience: Waiting for Serotonin
Ranade, S., Pi, H.-J., & Kepecs, Á. (2014) · Current Biology · doi:10.1016/j.cub.2014.07.024 · Cited by 61
Boraud, T., Leblois, A., & Rougier, N. P. (2018) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2018.08.003 · Cited by 64
“Since the seminal works of Wolfram Schultz and his colleagues, it is known that dopamine (DA) is a neural substrate of the reward prediction error signal in the reinforcement learning (RL) process (Apicella et al. , 1992; Fiorillo et al. , 2003; Schultz, 1992).”
BackgroundAging, uncertainty, and decision making—A review
Frank, C. C., & Seaman, K. L. (2023) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-023-01064-w · Cited by 69
Impact of aging on frontostriatal reward processing
Vink, M., Kleerekooper, I., van den Wildenberg, W. P. M., & Kahn, R. S. (2015) · Human Brain Mapping · doi:10.1002/hbm.22771 · Cited by 69
“With decreased probability of reward receipt, the stimulus–reward association will be less profound [Fiorillo et al., 2003].”
Météreau, É., & Dreher, J. (2014) · Cortex · doi:10.1016/j.cortex.2014.08.012 · Cited by 64
Krawczyk, M., Georges, F., Sharma, R., Mason, X., Berthet, A., Bézard, E., & Dumont, É. C. (2011) · Journal of Neurophysiology · doi:10.1152/jn.00710.2010 · Cited by 57
Multiple functions of dopamine neurons
Schultz, W. (2010) · F1000 Biology Reports · doi:10.3410/b2-2 · Cited by 59
“Slightly slower activations code the predicted risk of future rewards in a fraction of dopamine neurons (29%) [9].”
BackgroundSelf-citationClark, L., & Zack, M. (2023) · Addictive Behaviors · doi:10.1016/j.addbeh.2023.107626 · Cited by 59
“In the classic electrophysiological study by Fiorillo et al. (2003), the effect of unpredictable CSs on DA neuron firing (i.e. during the CS-US interval) was notably greatest when the size of the US (i.e. fruit juice volume) was also variable.”
BackgroundEpistasis effects of dopamine genes on interval timing and reward magnitude in humans
Balcı, F., Wiener, M., Çavdaroğlu, B., & Coslett, H. B. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2012.08.002 · Cited by 62
“…bursts occur at the onset of a conditioned stimulus that predicts the reward (e.g., Schultz, Dayan, & Montague, 1997) and stimuli signaling larger or more probable rewards lead to larger phasic responses of dopamine neurons (Fiorillo et al., 2003; Schultz, 2010; Tobler, Fiorillo, & Schultz, 2005).”
Harnessing behavioral diversity to understand neural computations for cognition
Musall, S., Urai, A. E., Sussillo, D., & Churchland, A. K. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.09.011 · Cited by 69
“Latent behavioral variables extracted from these models, such as the inferred accumulation of sensory evidence [17], have been successfully linked to brain function at different levels, from single cells [23,24], microcircuits [25,26], to macro- scale neural mass signals [27,28] and high- dimensional state representat…”
MethodDimensionality, information and learning in prefrontal cortex
Bartolo, R., Saunders, R. C., Mitz, A. R., & Averbeck, B. B. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007514 · Cited by 60
Wang, Y., Toyoshima, O., Kunimatsu, J., Yamada, H., & Matsumoto, M. (2021) · eLife · doi:10.7554/elife.63166 · Cited by 63
“A previous study reported that dopamine neurons exhibited a tonic increase in activity when future rewards were uncertain (Fiorillo et al., 2003).”
BackgroundSupplementary Eye Field Encodes Reward Prediction Error
So, N., & Stuphorn, V. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4419-11.2012 · Cited by 56
“Uncertainty signals observed in midbrain dopamine neurons, anterior insular cortex, and striatum (Fiorillo et al., 2003; Preuschoff et al., 2006) show a similar dynamics as the one in SEF, appearing later than value-encoding signals.”
BackgroundStimulus-response and response-outcome learning mechanisms in the striatum
Horvitz, J. C. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.12.014 · Cited by 58
Dopamine modulates subcortical responses to surprising sounds
Valdés‐Baizabal, C., Carbajal, G. V., Pérez‐González, D., & Malmierca, M. S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000744 · Cited by 60
Pharmacological evidence for the implication of noradrenaline in effort
Borderies, N., Mattioni, J., Bornert, P., Gilardeau, S., & Bouret, S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000793 · Cited by 61
“This provides neurobiological support to the idea that effort differs from other types of costs such as delay or risk, which both relate to the distribution of reward in time and rely more critically upon dopamine [7,20,22,33,72,89,90].”
BackgroundElectrophysiological correlates of decision making under varying levels of uncertainty
Bland, A. R., & Schaefer, A. (2011) · Brain Research · doi:10.1016/j.brainres.2011.08.031 · Cited by 63
Howard‐Jones, P. A., & Jay, T. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.015 · Cited by 68
“Primate studies suggest the variance (or uncertainty) in this probability may influence tonic levels of dopamine, producing a sustained ramping between a cue that a reward may be arriving and delivery of the reward [22].”
The role of reward and reward uncertainty in episodic memory
Mason, A., Farrell, S., Howard‐Jones, P. A., & Ludwig, C. J. H. (2017) · Journal of Memory and Language · doi:10.1016/j.jml.2017.05.003 · Cited by 61
“…areas such as the ventral tegmental area (VTA), the ventral striatum, the frontal cortex and amyg-dala—show several changes in activity in response to rewards and reward-predicting cues (Cromwell & Schultz, 2003; Fiorillo et al., 2003; Paton, Belova, Morrison, & Salzman, 2006; Schultz, 1998, 2002).”
BackgroundInfluentialBallou, S., Kaptchuk, T. J., Hirsch, W., Nee, J., Iturrino, J., Hall, K. T., Kelley, J. M., Cheng, V., Kirsch, I., Jacobson, E., et al. (2017) · Trials · doi:10.1186/s13063-017-1964-x · Cited by 57
Wideman, C. E., Jardine, K. H., & Winters, B. D. (2018) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2018.11.001 · Cited by 53
Incentive salience attribution under reward uncertainty: A Pavlovian model
Anselme, P. (2014) · Behavioural Processes · doi:10.1016/j.beproc.2014.10.016 · Cited by 60
Stallen, M., & Sanfey, A. G. (2013) · The Neuroscientist · doi:10.1177/1073858412469728 · Cited by 79
“In these experiments, punishments and rewards are typically dispensed anonymously, and, importantly, are costly to the participant, as well as having a real effect on the target player.”
Backgroundvan den Bos, R., Davies, W., Dellu-Hagedorn, F., Goudriaan, A. E., Granon, S., Homberg, J. R., Rivalan, M., Swendsen, J. D., & Adriani, W. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.07.005 · Cited by 58
Wadsley, M. G., Covey, J. A., & Ihssen, N. (2021) · Psychological Reports · doi:10.1177/00332941211025271 · Cited by 65
“Furthermore, random-ratio schedules have been shown to maximize the release of dopamine in the midbrain and parts of the basal ganglia known to be involved in reinforcing reward seeking behaviour (Fiorillo et al., 2003; Zald et al., 2004).”
BackgroundMedial prefrontal brain activation to anticipated reward and loss in obsessive–compulsive disorder
Kaufmann, C., Beucke, J. C., Preuße, F., Endrass, T., Schlagenhauf, F., Heinz, A., Juckel, G., & Kathmann, N. (2013) · NeuroImage: Clinical · doi:10.1016/j.nicl.2013.01.005 · Cited by 56
“'s findings trace back to a nonselective formof attention or arousal designed to aid the learning of predictive stimuli and actions (Fiorillo et al., 2003), which was triggered in Figee et al.”
InfluentialStudying the neural representations of uncertainty
Walker, E. Y., Pohl, S., Denison, R. N., Barack, D. L., Lee, J., Block, N., Ma, W. J., & Meyniel, F. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01444-y · Cited by 56
“Previous studies on uncertainty have used correlation 29,51,69 , differences between conditions 31,32,50 , linear regression 46,47,49,55 or decoding 55,103,107,137,138 to establish representations of uncertainty.”
BackgroundAdaptive and aberrant reward prediction signals in the human brain
Roiser, J. P., Stephan, K. E., den Ouden, H. E. M., Friston, K. J., & Joyce, E. M. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.075 · Cited by 52
An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning
Potjans, W., Diesmann, M., & Morrison, A. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001133 · Cited by 61
“Dopaminergic activity is typically recorded in classical conditioning [15,16], instructed-choice instrumental conditioning [17] or simple decision trials with only a few number of possible actions [13].”
Rausch, F., Mier, D. U., Eifler, S., Esslinger, C., Schilling, C., Schirmbeck, F. N., Englisch, S., Meyer‐Lindenberg, A., Kirsch, P., & Zink, M. (2014) · Schizophrenia Research · doi:10.1016/j.schres.2014.04.020 · Cited by 61
“These findings might be interpreted within theories about decision-making in schizophrenia (Fletcher and Frith, 2009) and the functional representation of reward uncertainty by dopaminergic neurotransmission (Fiorillo et al., 2003; Tobler et al., 2005; Schultz et al., 2008; Schultz, 2013).”
D1/D5Modulation of Synaptic NMDA Receptor Currents
Varela, J. A., Hirsch, S. J., Chapman, D. E., Leverich, L. S., & Greene, R. W. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.4746-08.2009 · Cited by 62
“Salience of environmental cues has been associated with the release of a neuromodulator, dopamine (DA), after activation of DA neurons in the ventral tegmental area (VTA) (Fiorillo et al., 2003).”
BackgroundInfluentialExtrastriatal dopaminergic changes in Parkinson's disease patients with impulse control disorders
Lee, J., Seo, S. H., Kim, Y. K., Yoo, H. B., Kim, Y. E., Song, I. C., Lee, J., & Jeon, B. S. (2013) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2013-305549 · Cited by 61
Pickel, V. M., Colago, E. E. O., Mania, I., Molosh, A. I., & Rainnie, D. G. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2006.06.059 · Cited by 61
Serotonergic modulation of reward and punishment: evidence from pharmacological fMRI studies
Macoveanu, J. (2014) · Brain Research · doi:10.1016/j.brainres.2014.02.003 · Cited by 55
Schultz, W. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00021-x · Cited by 57
Self-citationNon-Conscious Prediction and a Role for Consciousness in Correcting Prediction Errors
Pally, R. (2005) · Cortex · doi:10.1016/s0010-9452(08)70282-x · Cited by 58
Farrell, K., Lak, A., & Saleem, A. B. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.111470 · Cited by 59
Morita, K., & Kato, A. (2014) · Frontiers in Neural Circuits · doi:10.3389/fncir.2014.00036 · Cited by 59
“In particular, DA is known to modulate the activity of the two types of striatal projection neurons to the opposite directions (Gerfen and Surmeier, 2011).”
Yoshimi, K., Naya, Y., Mitani, N., Kato, T., Inoue, M., Natori, S., Takahashi, T., Weitemier, A. Z., Nishikawa, N., McHugh, T. J., et al. (2011) · Neuroscience Research · doi:10.1016/j.neures.2011.05.013 · Cited by 56
“4 and 5 seem to satisfy criterion (7) and are in general agreement with the known phasic responses of the temporal characteristics, which are consistent with those of midbrain dopamine neuron excitation (Joshua et al., 2008; Fiorillo et al., 2003; Matsumoto and Hikosaka, 2009; Schultz et al., 1993, 1997; Schultz, 2007…”
BackgroundResultInfluentialBeyene, M., Carelli, R. M., & Wightman, R. M. (2010) · Neuroscience · doi:10.1016/j.neuroscience.2010.06.047 · Cited by 68
“The degree of dopamine activation in response to reward-predictive cues has been found to reflect salient information about predicted rewards, including the probability and timing of their occurrence (Fiorillo et al., 2003, Kobayashi and Schultz, 2008).”
BackgroundDopamine-associated cached values are not sufficient as the basis for action selection
Hollon, N. G., Arnold, M. M., Gan, J. O., Walton, M. E., & Phillips, P. E. M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1419770111 · Cited by 52
Establishing causality for dopamine in neural function and behavior with optogenetics
Steinberg, E. E., & Janak, P. H. (2013) · Brain Research · doi:10.1016/j.brainres.2012.09.036 · Cited by 51
“%) from baseline activity levels, much lower than the peak rate during phasic activations (Fiorillo et al., 2003, 2005; Schultz, 2010).”
Cognition and medial frontal cortex in health and disease
Nachev, P. (2006) · Current Opinion in Neurology · doi:10.1097/01.wco.0000247609.36482.ae · Cited by 60
Over a Decade of Neuroeconomics: What Have We Learned?
Konovalov, A., & Krajbich, I. M. (2016) · Organizational Research Methods · doi:10.1177/1094428116644502 · Cited by 62
Context-Dependent Multiplexing by Individual VTA Dopamine Neurons
Kremer, Y., Flakowski, J., Rohner, C., & Lüscher, C. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.0502-20.2020 · Cited by 59
Llewellyn, S., & Hobson, J. A. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2015.04.005 · Cited by 60
Primate anterior insular cortex represents economic decision variables proposed by prospect theory
Yang, Y.-P., Li, X., & Stuphorn, V. (2022) · Nature Communications · doi:10.1038/s41467-022-28278-9 · Cited by 56
Dopamine neurons encode errors in predicting movement trigger occurrence
Pasquereau, B., & Turner, R. S. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00401.2014 · Cited by 52
Parker, J. G., Wanat, M. J., Soden, M. E., Ahmad, K., Zweifel, L. S., Bamford, N. S., & Palmiter, R. D. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1715-11.2011 · Cited by 53
“Furthermore the phasic activation of DA neurons in response to reward-predictive stimuli increases with the probability with which a stimulus predicts reward (Fiorillo et al., 2003).”
Direct Evidence for Prediction Signals in Frontal Cortex Independent of Prediction Error
Dürschmid, S., Reichert, C., Hinrichs, H., Heinze, H., Kirsch, H. E., Knight, R. T., & Deouell, L. Y. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy331 · Cited by 56
Long‐ and short‐range reward expectancy in the primate orbitofrontal cortex
Hikosaka, K., & Watanabe, M. (2004) · European Journal of Neuroscience · doi:10.1111/j.0953-816x.2004.03120.x · Cited by 53
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.