Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 301–400 of 1,007
Neuroimaging of reward mechanisms in Gambling disorder: an integrative review
Clark, L., Boileau, I., & Zack, M. H. (2018) · Molecular Psychiatry · doi:10.1038/s41380-018-0230-2 · Cited by 177
Satterthwaite, T. D., Green, L., Myerson, J., Parker, J. B., Ramaratnam, M., & Buckner, R. L. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.04.066 · Cited by 174
“In this account, unexpected negative outcomes produce phasic suppression of firing by DA neurons (Fiorillo et al., 2003), which in turn promote the recruitment of a network of frontal regions involved in cognitive control.”
BackgroundInfluentialThe ability of the mesocortical dopamine system to operate in distinct temporal modes
Lapish, C. C., Kroener, S., Durstewitz, D., Lavín, A., & Seamans, J. K. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0527-8 · Cited by 183
“Again, both groups (Fiorillo et al. 2003; Schultz 2002; Redgrave et al. 1999) have highlighted that even in the context of DA-associated attentional processes the ultimate function of the DA signal is still linked to the formation of associations.”
BackgroundInfluentialNeuronal Activity in Primate Orbitofrontal Cortex Reflects the Value of Time
Roesch, M. R., & Olson, C. R. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00373.2005 · Cited by 154
“The probability of receiving a reward is known to exert an impact on both behavior and neural activity related to the anticipation of reward (Fiorillo et al. 2003; Mobini et al. 2002; Platt and Glimcher 1999).”
BackgroundTemporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli
Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4828-09.2010 · Cited by 158
“Such modulation was reported to be seen in response to the primary reward (Hollerman and Schultz, 1998; Fiorillo et al., 2008).”
BackgroundSelf-citationAssociation of DRD3 and GRIN2B with impulse control and related behaviors in Parkinson's disease
Lee, J., Lee, E. K., Park, S. S., Lim, J., Kim, H. J., Kim, J. S., & Jeon, B. S. (2009) · Movement Disorders · doi:10.1002/mds.22678 · Cited by 156
Neurobiological correlates of internet gaming disorder: Similarities to pathological gambling
Fauth‐Bühler, M., & Mann, K. F. (2015) · Addictive Behaviors · doi:10.1016/j.addbeh.2015.11.004 · Cited by 171
The Incentive-Sensitization Theory of Addiction 30 Years On
Robinson, T. E., & Berridge, K. C. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-011624-024031 · Cited by 176
“…gained after a reward cue can magnify the incentive salience attributed to that cue and magnify the cue-triggered activation of mesolimbic dopamine systems—even though uncertainty reduces the predictive reliability of those cues (Fiorillo et al. 2003, Robinson & Anselme 2019, Singer et al. 2020).”
The placebo response in clinical trials: more questions than answers
Enck, P., Klosterhalfen, S., Weimer, K., Horing, B., & Zipfel, S. (2011) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2010.0384 · Cited by 156
Shaping of Motor Responses by Incentive Values through the Basal Ganglia
Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., Gross, C. E., & Boraud, T. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3745-06.2007 · Cited by 146
“…stage of the BG, the striatum, suggest that it is modulated by reward-related activity in midbrain dopaminergic neurons (Hollerman et al., 1998; Fiorillo et al., 2003; Satoh et al., 2003; Morris et al., 2004) and is thus able to determine the potential reward probability and quantity…”
BackgroundMethodInfluentialAlcohol effects on human risk taking
Lane, S. D., Cherek, D. R., Pietras, C. J., & Tcheremissine, O. V. (2004) · Psychopharmacology · doi:10.1007/s00213-003-1628-2 · Cited by 171
“…to probabilistic outcomes (e.g. trial outcomes with gain-loss probability of 0.5) is modulated by dopaminergic neurons in midbrain (Breiter et al. 2001; Fiorillo et al. 2003), and these same dopamine circuits are activated by alcohol administration (Brodie et al. 1990; Engleman et al. 2003).”
BackgroundInfluentialProbabilistic Representation in Human Visual Cortex Reflects Uncertainty in Serial Decisions
van Bergen, R. S., & Jehee, J. F. M. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.3212-18.2019 · Cited by 137
“Specifically, while uncertainty may be represented by dedicated neural populations in e.g. the dopaminergic system (Fiorillo et al., 2003), our findings make it unlikely that specific visual cortical neurons represent summary statistics or are tuned to sensory uncertainty (Vilares and Körding,…”
BackgroundOpen-Label Placebo: Reflections on a Research Agenda
Kaptchuk, T. J. (2018) · Perspectives in biology and medicine · doi:10.1353/pbm.2018.0045 · Cited by 164
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153
Keeler, J. F., Pretsell, D. O., & Robbins, T. W. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.07.021 · Cited by 174
Risk and risk prediction error signals in anterior insula
Bossaerts, P. (2010) · Brain Structure and Function · doi:10.1007/s00429-010-0253-1 · Cited by 146
“Traces of direct risk encoding in dopaminergic neurons has been found in the nonhuman primate brain ( Fiorillo et al. 2003 ) and in sub-cortical dopaminoceptive regions of the human brain (Preuschoff et al. 2006).”
BackgroundHeterogeneity of dopamine neuron activity across traits and states
Marinelli, M., & McCutcheon, J. E. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.07.034 · Cited by 143
“Lastly, the magnitude of dopamine neuron responses to rewards is strongly modulated by multiple factors including reward magnitude, probability, delay, and ambiguity (Fiorillo et al., 2003; Tobler et al., 2005; Roesch et al., 2007; Bromberg-Martin and Hikosaka, 2009).”
BackgroundFriedman, A., Lax, E., Dikshtein, Y., Abraham, L., Flaumenhaft, Y., Sudai, E., Ben-Tzion, M., Ami-Ad, L., Yaka, R., & Yadid, G. (2010) · Neuropharmacology · doi:10.1016/j.neuropharm.2010.06.008 · Cited by 146
“Their encoding frequency is approximately 8–12 Hz (Fiorillo et al., 2003; Nakamura et al., 2008).”
BackgroundThe role of beta-gamma oscillations in unexpected rewards processing
HajiHosseini, A., Rodríguez‐Fornells, A., & Marco‐Pallarés, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.125 · Cited by 156
“Direct recordings from the Ventral Tegmental Area of awaken monkeys have shown an increase in the dopaminergic activity after improbable rewards (Fiorillo et al., 2003; Schultz et al., 1997; Waelti et al., 2001) or after larger than expected rewards (Tobler et al.”
Kim, K. M., Baratta, M. V., Yang, A., Lee, D., Boyden, E. S., & Fiorillo, C. D. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033612 · Cited by 145
Self-citationWhat Is the Relationship between Dopamine and Effort?
Walton, M. E., & Bouret, S. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2018.10.001 · Cited by 147
Reinforcement learning: Computational theory and biological mechanisms
Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246 · Cited by 150
“Schultz and colleagues showed that tonic firing of dopamine neurons during the delay period before stochastic reward delivery is highest when the forthcoming reward is most uncertain: at 50% (Fiorillo et al., 2003).”
Namboodiri, V. M. K., Otis, J. M., van Heeswijk, K., Voets, E. S., Alghorazi, R. A., Rodriguez-Romaguera, J., Mihalas, S., & Stuber, G. D. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0408-1 · Cited by 150
Goldberg, J. A., & Reynolds, J. N. J. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.08.067 · Cited by 163
Encoding uncertainty in the hippocampus
Harrison, L. M., Duggins, A., & Friston, K. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2005.11.002 · Cited by 155
Stimulus representation and the timing of reward-prediction errors in models of the dopamine system
Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-654 · Cited by 148
“The left column shows data from a representative dopamine neuron (Fiorillo et al., 2003); as reward probability increased,”
BackgroundResultInfluentialSatterthwaite, T. D., Ruparel, K., Loughead, J., Elliott, M. A., Gerraty, R. T., Calkins, M. E., Hakonarson, H., Gur, R. C., Gur, R. E., & Wolf, D. H. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.03.060 · Cited by 148
“These results accord with a substantial body of literature demonstrating reward prediction error responses (Dreher et al., 2005; Fiorillo et al., 2003; Satterthwaite et al., 2007), and suggest that the prediction-error frameworkmay generalize to situationswhere explicit rewards are not present.”
Neurobiology of Obsessive-Compulsive Disorder:Serotonin and Beyond
Westenberg, H. G. M., Fineberg, N. A., & Denys, D. (2007) · CNS Spectrums · doi:10.1017/s1092852900002479 · Cited by 148
“A controlled study of early-onset Huntington's disease reported similar impairments to those reported for OCD, with impaired visuospatial planning (dorsolateral prefrontal circuitry), but normal decision-making (OFC task).(171)”
BackgroundPrefrontal dopamine in associative learning and memory
Puig, M. V., Antzoulatos, E. G., & Miller, E. K. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.09.026 · Cited by 139
“Recent work has shown that DA release in the striatum increases gradually (ramps up) as rats expect distant rewards, perhaps providing motivational drive (Howe et al., 2013) or encoding uncertainty (Fiorillo et al., 2003).”
BackgroundNicola, S. M., Taha, S. A., Kim, S. W., & Fields, H. L. (2005) · Neuroscience · doi:10.1016/j.neuroscience.2005.06.088 · Cited by 167
Coccurello, R., & Moles, A. (2010) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2010.04.008 · Cited by 163
“or if they are conveying a value (salience stimuli) higher than expected (Fiorillo et al., 2003; Volkow et al., 2009).”
BackgroundSosa, M., & Giocomo, L. M. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00479-z · Cited by 150
Different Varieties of Uncertainty in Human Decision-Making
Bland, A. R., & Schaefer, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00085 · Cited by 129
“In other words, when an agent is faced with a two-options choice, uncertainty is often said to be maximal when the probability of obtaining a reward linked to any of the two options is p = 0.5 but absent at the two extremes (probability = 1 and probability = 0; e.g., Fiorillo et al., 2003).”
BackgroundInfluentialThe placebo treatments in neurosciences: New insights from clinical and neuroimaging studies
Diederich, N. J., & Goetz, C. G. (2008) · Neurology · doi:10.1212/01.wnl.0000324635.49971.3d · Cited by 143
Mahler, S. V., Smith, R. J., & Aston-Jones, G. (2013) · Psychopharmacology · doi:10.1007/s00213-012-2681-5 · Cited by 140
“Cues are associated with increased ventral tegmental area (VTA) dopamine cell firing, and concomitant forebrain dopamine release is necessary and sufficient for reward seeking and approach efforts (Berridge and Robinson 1998; Cheer et al. 2007; Fiorillo et al. 2003; Ikemoto and Panksepp 1999; Salamone et al. 2007; Sat…”
Santesso, D. L., Evins, A. E., Frank, M. J., Schetter, E. C., Bogdan, R., & Pizzagalli, D. A. (2009) · Human Brain Mapping · doi:10.1002/hbm.20642 · Cited by 141
“Taken together, results from recent ERP studies indicate that positive prediction errors can affect the FRN (specifically, reduce its negativity); more generally, these findings are consistent with the account that unpredicted rewards, supported by phasic DA increases [Fiorillo et al., 2003; Schultz, 2007], also serve…”
BackgroundResultInfluentialPhasic Nucleus Accumbens Dopamine Encodes Risk-Based Decision-Making Behavior
Sugam, J. A., Day, J. J., Wightman, R. M., & Carelli, R. M. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.09.029 · Cited by 142
“Further, cues which predict larger, immediate, or more probable rewards elicit larger spikes in DA neural activity than cues that predict smaller, delayed or less probable rewards (7, 14, 15).”
BackgroundFunctional connectivity of reward processing in the brain
Camara, E. (2008) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.019.2008 · Cited by 141
“Reward information becomes more relevant as the uncertainty of the reward prediction and the magnitude of the outcome increases (Fiorillo et al., 2003; Schultz et al., 1997).”
BackgroundReward-related processing in the human brain: developmental considerations
Fareri, D. S., Martin, L. N., & Delgado, M. R. (2008) · Development and Psychopathology · doi:10.1017/s0954579408000576 · Cited by 141
Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm
Day, J. J., Wheeler, R. A., Roitman, M. F., & Carelli, R. M. (2006) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2006.04654.x · Cited by 140
Dopamine modulates neural networks involved in effort-based decision-making
Assadi, S. M., Yücel, M., & Pantelis, C. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2008.10.010 · Cited by 148
“Tonic activity appears to encode reward uncertainty in the period between occurrences of conditioned stimuli to the expected time of reward (Fiorillo et al., 2003).”
BackgroundSchevernels, H., Krebs, R. M., Santens, P., Woldorff, M. G., & Boehler, C. N. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.09.039 · Cited by 129
“With higher levels of (reward) uncertainty, slower sustained activations of the dopaminergic system have been shown to increase (Fiorillo et al., 2003; Preuschoff et al., 2006).”
BackgroundPositive reward prediction errors during decision-making strengthen memory encoding
Jang, A. I., Nassar, M. R., Dillon, D. G., & Frank, M. J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0597-3 · Cited by 147
The medial prefrontal cortex shapes dopamine reward prediction errors under state uncertainty
Starkweather, C. K., Gershman, S. J., & Uchida, N. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.036 · Cited by 136
“In general, animals ramp up their lick rates sooner for higher reward probabilities, even in the absence of variability in the timing of reward (Fiorillo et al., 2003; Tian and Uchida, 2015).”
Path integration: how the head direction signal maintains and corrects spatial orientation
Valerio, S., & Taube, J. S. (2012) · Nature Neuroscience · doi:10.1038/nn.3215 · Cited by 136
Moral Learning: Conceptual foundations and normative relevance
Railton, P. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.015 · Cited by 155
Reason, emotion and decision-making: risk and reward computation with feeling
Quartz, S. R. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.02.003 · Cited by 159
Placebo mechanisms and reward circuitry: clues from Parkinson's disease
de la Fuente‐Fernández, R., Schulzer, M., & Stoessl, A. J. (2004) · Biological Psychiatry · doi:10.1016/j.biopsych.2003.11.019 · Cited by 143
Expert Financial Advice Neurobiologically “Offloads” Financial Decision-Making under Risk
Engelmann, J. B., Capra, C. M., Noussair, C., & Berns, G. S. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004957 · Cited by 132
“This would be reflected by a reduced sensitivity to reward and probability magnitudes in regions previously associated with reward-related computations, such as the mesolimbic and mesocortical dopamine pathways [16,17,18,19, 20,21], and in regions implicated in probability processing, such as anterior and posterior ci…”
BackgroundReward-based decision-making and aging
Marschner, A., Mell, T., Wartenburger, I., Villringer, A., Reischies, F. M., & Heekeren, H. R. (2005) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2005.06.010 · Cited by 157
Predictive processing, source monitoring, and psychosis
Griffin, J. D., & Fletcher, P. C. (2017) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-045145 · Cited by 133
“…that some studies have suggested that dopamine, in addition to signaling the magnitude of reward PEs (Schultz et al. 1997), is critical in signaling the precision (Fiorillo et al. 2003) or epistemic value (Schwartenbeck et al. 2016) of incoming information, setting the gain on cortical PE signals.”
BackgroundAdding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133
Moral intuition: Its neural substrates and normative significance
Woodward, J., & Allman, J. M. (2007) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2007.12.003 · Cited by 161
“The VENs are strongly labeled with antibodies to the dopamine D3 receptor (Allman et al., 2005), which may signal the expectation of reward under uncertainty (Fiorillo et al., 2003).”
From Bad to Worse: Striatal Coding of the Relative Value of Painful Decisions
Brooks, A. M., Pammi, V. S. C., Noussair, C., Capra, C. M., Engelmann, J. B., & Berns, G. S. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00176 · Cited by 136
“Interestingly, this risk-seeking behavior cannot explain the changes in striatal activation as others have suggested (Fiorillo et al., 2003; Preuschoff et al., 2006) because the variance of the best and worst lotteries is the same in our task.”
BackgroundLévy, R. (2012) · Revue Neurologique · doi:10.1016/j.neurol.2012.05.003 · Cited by 149
Splitting the Difference: How Does the Brain Code Reward Episodes?
KNUTSON, B., & WIMMER, G. E. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.020 · Cited by 129
Computational and dynamic models in neuroimaging
Friston, K. J., & Dolan, R. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.068 · Cited by 131
“We close this review with an example of how computational fMRI might be used to disambiguate between competing functional models and resolve an outstanding question in computational neuroscience (see Fiorillo et al., 2003; Niv et al., 2005).”
Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops
SAMEJIMA, K., & DOYA, K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.024 · Cited by 124
Robinson, M. J. F., Anselme, P., Fischer, A. M., & Berridge, K. C. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2014.03.004 · Cited by 123
“Several studies have highlighted the role of midbrain dopamine in coding reward uncertainty in primates [37,48], healthy humans [49,50], and pathological gamblers [27].”
BackgroundPetter, E. A., Lusk, N. A., Hesslow, G., & Meck, W. H. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.10.015 · Cited by 138
Eston, R., Faulkner, J., St Clair Gibson, A., Noakes, T., & Parfitt, G. (2007) · Psychophysiology · doi:10.1111/j.1469-8986.2007.00558.x · Cited by 124
Neurofeedback-mediated self-regulation of the dopaminergic midbrain
Sulzer, J. S., Sitaram, R., Blefari, M. L., Kollias, S., Birbaumer, N., Stephan, K. E., Luft, A. R., & Gassert, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.115 · Cited by 134
Neural Basis of Motivational and Cognitive Control
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262016438.001.0001 · Cited by 128
Adolescent-specific patterns of behavior and neural activity during social reinforcement learning
Jones, R. M., Somerville, L. H., Li, J., Ruberry, E. J., Powers, A., Mehta, N., Dyke, J., & Casey, B. J. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0257-z · Cited by 127
Dopamine release in ventral striatum of pathological gamblers losing money
Linnet, J., Peterson, E., Doudet, D. J., Gjedde, A., & Møller, A. (2010) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2010.01591.x · Cited by 127
“(33) found increased sustained activation of midbrain dopamine neurons toward stimuli with uncertain reward probability.”
BackgroundAction controls dopaminergic enhancement of reward representations
Guitart-Masip, M., Chowdhury, R., Sharot, T., Dayan, P., Duzel, E., & Dolan, R. J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1202229109 · Cited by 124
Differential effects of dopaminergic manipulations on risky choice
Onge, J. R. S., Chiu, Y. C., & Floresco, S. (2010) · Psychopharmacology · doi:10.1007/s00213-010-1883-y · Cited by 137
“, phasic bursts or dips) in midbrain DA neuron firing (Schultz et al. 1997; Fiorillo et al. 2003).”
BackgroundInfluentialPlastic Corticostriatal Circuits for Action Learning: What's Dopamine Got to Do with It?
COSTA, R. M. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.015 · Cited by 125
Mesolimbic interaction of emotional valence and reward improves memory formation
Wittmann, B. C., Schiltz, K., Boehler, C. N., & Düzel, E. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2007.11.020 · Cited by 136
Impulsivity and apathy in Parkinson's disease
Sinha, N., Manohar, S., & Husain, M. (2013) · Journal of Neuropsychology · doi:10.1111/jnp.12013 · Cited by 102
Dopamine Genes and Pathological Gambling in Discordant Sib-Pairs
Lobo, D. S. S., Vallada, H., Knight, J., Martins, S. S., Tavares, H., Gentil, V., & Kennedy, J. L. (2007) · Journal of Gambling Studies · doi:10.1007/s10899-007-9060-x · Cited by 130
The neural circuitry of reward and its relevance to psychiatric disorders
Chau, D. T., Roth, R. M., & Green, A. I. (2004) · Current Psychiatry Reports · doi:10.1007/s11920-004-0026-8 · Cited by 123
“Dopaminergic neuron activity is substantially increased when the receipt of reward is uncertain [ 100 ].”
BackgroundInfluentialCue-Evoked Dopamine Promotes Conditioned Responding during Learning
Morrens, J., Aydın, Ç., van Rensburg, A. J., Rabell, J. E., & Haesler, S. (2020) · Neuron · doi:10.1016/j.neuron.2020.01.012 · Cited by 129
Dopaminergic Reward Signals Selectively Decrease fMRI Activity in Primate Visual Cortex
Arsenault, J. T., Nelissen, K., Jarraya, B., & Vanduffel, W. (2013) · Neuron · doi:10.1016/j.neuron.2013.01.008 · Cited by 120
Can the neuroeconomics revolution revolutionize psychiatry?
Hasler, G. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.011 · Cited by 140
Learning and decision making in monkeys during a rock–paper–scissors game
Lee, D., McGreevy, B. P., & Barraclough, D. J. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.07.003 · Cited by 131
Distinct Representations of Cognitive and Motivational Signals in Midbrain Dopamine Neurons
Matsumoto, M., & Takada, M. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.002 · Cited by 139
Neurocognitive mechanisms of error-based motor learning
Seidler, R. D., Kwak, Y., Fling, B. W., & Bernard, J. A. (2013) · Advances in Experimental Medicine and Biology · doi:10.1007/978-1-4614-5465-6_3 · Cited by 120
“This network is in keeping with observations of phasic suppression of dopamine neurons on error trials (DeLong 1983; Fiorillo et al. 2003; Ljungberg et al. 1991), and the hypothesized role of this pathway in modulating error processes (Amiez et al. 2005; Brown and Braver 2005; Takenouchi et al.…”
ResultSaddoris, M. P., Sugam, J. A., Stuber, G. D., Witten, I. B., Deisseroth, K., & Carelli, R. M. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.10.024 · Cited by 116
Neural correlates of error detection and correction in a semantic retrieval task
Butterfield, B., & Mangels, J. A. (2003) · Cognitive Brain Research · doi:10.1016/s0926-6410(03)00203-9 · Cited by 133
Neuroeconomics and aging: Neuromodulation of economic decision making in old age
Mohr, P. N. C., Li, S., & Heekeren, H. R. (2009) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.05.010 · Cited by 131
“also a direct link between risk and dopamine release (Fiorillo et al., 2003; St Onge and Floresco, 2009).”
BackgroundInfluentialBüchel, C., Peters, J., Banaschewski, T., Bokde, A. L. W., Bromberg, U., Conrod, P. J., Flor, H., Papadopoulos, D., Garavan, H., Gowland, P., et al. (2017) · Nature Communications · doi:10.1038/ncomms14140 · Cited by 118
Expectation of caffeine induces dopaminergic responses in humans
Kaasinen, V., Aalto, S., Någren, K., & Rinne, J. O. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03310.x · Cited by 118
Rapid dopamine dynamics in the accumbens core and shell Learning and action
Saddoris, M. P. (2013) · Frontiers in Bioscience · doi:10.2741/e615 · Cited by 117
“For example, dopaminergic midbrain neurons display differential activity based on several factors that reflect reward value such as the cost of performing tasks, the delay to reinforcement, and the probability and magnitude of reinforcement (115-117).”
BackgroundNeural connectivity as an intermediate phenotype: Brain networks under genetic control
Meyer‐Lindenberg, A. (2009) · Human Brain Mapping · doi:10.1002/hbm.20639 · Cited by 116
“Because dopaminergic neurotransmission is critical for motivated behavior, working memory [Goldman-Rakic, 1998], and reward-related learning [Schultz, 1998], and is implicated in, among others, schizophrenia [Weinberger, 1987], alcoholism [Grace, 2000], Parkinson’s disease, and pathological gambling [Fiorillo et al.,…”
BackgroundThe dynamics of dopamine in control of motor behavior
Joshua, M., Adler, A., & Bergman, H. (2009) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2009.10.001 · Cited by 126
Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions
de Lafuente, V., & Romo, R. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1117636108 · Cited by 117
“Schultz and colleagues (4) found that a third of the population of DA neurons code reward uncertainty by gradually increasing their firing rate from the onset of a visual stimulus to the time of reward delivery.”
BackgroundInfluentialUnpredictable saccharin reinforcement enhances locomotor responding to amphetamine
Singer, B. F., Scott-Railton, J., & Vezina, P. (2012) · Behavioural Brain Research · doi:10.1016/j.bbr.2011.09.003 · Cited by 117
Presynaptic mechanisms of motor fluctuations in Parkinson’s disease: a probabilistic model
de la Fuente‐Fernández, R., Schulzer, M., Mak, E., Calne, D. B., & Stoessl, A. J. (2004) · Brain · doi:10.1093/brain/awh102 · Cited by 112
“Classical dopamine cells have a ®ring rate ranging from 0.1 to 8 Hz (Fiorillo et al., 2003).”
BackgroundInfluentialThe modulation of savouring by prediction error and its effects on choice
Iigaya, K., Story, G. W., Kurth-Nelson, Z., Dolan, R. J., & Dayan, P. (2016) · eLife · doi:10.7554/elife.13747 · Cited by 115
Casey, B. J., Yeung, N., & Fossella, J. (2002) · Elsevier eBooks · doi:10.1016/b0-12-227210-2/00376-9 · Cited by 117
Paulus, M. P., Feinstein, J. S., Simmons, A., & Stein, M. B. (2004) · Biological Psychiatry · doi:10.1016/j.biopsych.2004.02.023 · Cited by 115
Levodopa ameliorates learning and memory deficits in a murine model of Alzheimer’s disease
Ambrée, O., Richter, H., Sachser, N., Lewejohann, L., Dere, E., de Souza Silva, M. A., Herring, A., Keyvani, K., Paulus, W., & Schäbitz, W.-R. (2009) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2007.11.010 · Cited by 115
The perception of probability.
Gallistel, C. R., Krishan, M., Liu, Y., Miller, R., & Latham, P. E. (2014) · Psychological Review · doi:10.1037/a0035232 · Cited by 105
Bayesian modeling of flexible cognitive control
Jiang, J., Heller, K., & Egner, T. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.06.001 · Cited by 124
“…of reward probability and expected reward (for reviews, see Platt & Huettel 2008; and Rushworth & Behrens 2008) has been found in the midbrain (Fiorillo et al., 2003, 2005), the ventral striatum (Knutson et al., 2005; Preuschoff et al., 2006), the ACC (Amiez et al., 2006), and the medial…”
BackgroundStriatal cholinergic interneurons and cortico‐striatal synaptic plasticity in health and disease
Deffains, M., & Bergman, H. (2015) · Movement Disorders · doi:10.1002/mds.26300 · Cited by 121
Robinson, M. J. F., Anselme, P., Suchomel, K., & Berridge, K. C. (2015) · Behavioral Neuroscience · doi:10.1037/bne0000064 · Cited by 113
Value Learning through Reinforcement
Daw, N. D., & Tobler, P. N. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00015-2 · Cited by 138
“This sort of reasoning explains the response pattern emitted by dopamine neurons to stimuli predicting reward with different probabilities even though at the time those stimuli are delivered no actual rewards are obtained (Fiorillo et al., 2003).”
BackgroundInfluentialSelf-citationVentral Tegmental Dopamine Neurons Control the Impulse Vector during Motivated Behavior
Hughes, R. N., Bakhurin, K. I., Petter, E. A., Watson, G. D. R., Kim, N., Friedman, A. D., & Yin, H. H. (2020) · Current Biology · doi:10.1016/j.cub.2020.05.003 · Cited by 123
Feedback signals in the caudate reflect goal achievement on a declarative memory task
Tricomi, E. M., & Fiez, J. A. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.066 · Cited by 122
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.