Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 201–300 of 1,007
Rolls, E. T., McCabe, C., & Redoute, J. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm097 · Cited by 236
“For brain regions where there was a prior hypothesis as described in the Introduction and under Methods, namely, in the parts of the orbitofrontal cortex, pregenual cingulate cortex, ventral striatum, and midbrain shown to be of interest because of activations found in prior studies in reward paradigms or because neur…”
The Nucleus Accumbens and Pavlovian Reward Learning
Day, J. J., & Carelli, R. M. (2007) · The Neuroscientist · doi:10.1177/1073858406295854 · Cited by 238
Neural systems implicated in delayed and probabilistic reinforcement
Cardinal, R. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.004 · Cited by 248
“DA neurons that innervate the Acb may fire in a manner related to reward probability (Fiorillo et al., 2003, 2005; Niv et al., 2005; Tobler et al., 2005) and the midbrain, the site of the cell bodies of these neurons, responds to stimulus uncertainty in humans (Aron et al., 2004).”
BackgroundCardinal, R. N., & Howes, N. J. (2005) · BMC Neuroscience · doi:10.1186/1471-2202-6-37 · Cited by 232
“Although systemic D2-type DA receptor antagonists can induce impulsive choice involving delayed reinforcement [129], this effect may not occur in the Acb [130], the response of DA neurons specifically to reward uncertainty is debated [32-34], and little is known of the role of DA in choice involving uncertain rewards.”
Animal models of attention-deficit hyperactivity disorder
Russell, V. A., Sagvolden, T., & Johansen, E. B. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-9 · Cited by 234
Roles of "wanting" and "liking" in motivating behavior: gambling, food, and drug addictions
Robinson, M. J. F., Fischer, A. M., Ahuja, A., Lesser, E. N., & Maniates, H. (2015) · Current topics in behavioral neurosciences · doi:10.1007/7854_2015_387 · Cited by 247
“This may not come as a surprise considering that cues that predict an uncertain reward (50 % probability) produce a greater dopamine signal, originating from the ventral midbrain, during the anticipation of the uncertain outcome (Fiorillo et al. 2003), and that this dopaminergic signal appears to promote risk-seeking…”
BackgroundMotivation-dependent responses in the human caudate nucleus.
Delgado, M. R. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh062 · Cited by 246
“Indeed, a recent study found that dopamine neurons, which project to the striatum, respond to uncertainty in accordance with the probability of an eventual outcome (Fiorillo et al., 2003).”
BackgroundThe Social Modulation of Pain: Others as Predictive Signals of Salience – a Systematic Review
Krahé, C., Springer, A., Weinman, J. A., & Fotopoulou, A. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00386 · Cited by 245
“…kind of top-down prediction in sensory cortices is thought to be mediated by cholinergic neuromodulatory mechanisms that optimize the attentional gain of populations encoding prediction errors (Feldman and Friston, 2010), as well as by dopamine in frontostriatal circuits (Fiorillo et al., 2003).”
BackgroundMeaningful silences: how dopamine listens to the ACh pause
Cragg, S. J. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.01.003 · Cited by 245
Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine
Gan, J. O., Walton, M. E., & Phillips, P. E. M. (2010) · Nature Neuroscience · doi:10.1038/nn.2460 · Cited by 233
Gene–gene interaction associated with neural reward sensitivity
Yacubian, J., Sommer, T., Schroeder, K., Gläscher, J., Kalisch, R., Leuenberger, B., Braus, D. F., & Büchel, C. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0702029104 · Cited by 241
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241
“Neural responses to uncertainty or surprise have been reported in multiple structures (den Ouden et al., 2010; Fiorillo et al., 2003; Kepecs et al., 2008; McCoy and Platt, 2005; O’Neill and Schultz, 2010; Preuschoff et al., 2006, 2008; Schultz et al., 2008; So and Stuphorn, 2012; Tobler et al.,…”
BackgroundExplicit neural signals reflecting reward uncertainty
Schultz, W., Preuschoff, K., Camerer, C., Hsu, M., Fiorillo, C. D., Tobler, P. N., & Bossaerts, P. (2008) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2008.0152 · Cited by 239
“Electrophysiological studies have identified the brain’s reward system as a restricted network of structures, which include the dopamine neurons of the pars compacta of substantia nigra and ventral tegmental area, the striatum, orbitofrontal cortex (OFC) and amygdala (Cromwell & Schultz 2003; Fiorillo et al. 2003; Tob…”
InfluentialSelf-citationPhillips, A. G., Ahn, S., & Floresco, S. B. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4653-03.2004 · Cited by 231
“Reconciliation of microdialysis data from the rat mPFC with electrophysiological data from the primate mesencephalon may come from the recent observation of a sustained increase in DA neuron activity related to uncertainty about motivationally relevant stimuli (Fiorillo et al., 2003).”
Calculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine
Phillips, P. E. M., Walton, M. E., & Jhou, T. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0626-6 · Cited by 237
“2005) and also account for the likelihood of reward by scaling the response proportionally to the probability of reward delivery when weaker cues are presented that predict reward only a fraction of the time (Fiorillo et al. 2003).”
BackgroundInfluentialBayesian models: the structure of the world, uncertainty, behavior, and the brain
Vilares, I., & Kording, K. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.05965.x · Cited by 214
Neurons in anterior cingulate cortex multiplex information about reward and action
Hayden, B. Y., & Platt, M. L. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4874-09.2010 · Cited by 236
Neuromodulation of reward‐based learning and decision making in human aging
Eppinger, B., Hämmerer, D., & Li, S. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06230.x · Cited by 234
Rodeberg, N. T., Sandberg, S. G., Johnson, J. A., Phillips, P. E. M., & Wightman, R. M. (2017) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.6b00393 · Cited by 226
The role of the basal ganglia in learning and memory: Insight from Parkinson’s disease
Foerde, K., & Shohamy, D. (2011) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2011.08.006 · Cited by 208
Fervaha, G., Graff-Guerrero, A., Zakzanis, K. K., Foussias, G., Agid, O., & Remington, G. (2013) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2013.08.003 · Cited by 211
Dopamine Neurons Mediate a Fast Excitatory Signal via Their Glutamatergic Synapses
Chuhma, N., Zhang, H., Masson, J., Zhuang, X., Sulzer, D., Hen, R., & Rayport, S. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4317-03.2004 · Cited by 228
“Errors in the prediction of reward are encoded by a transient increase in DA neuron activity after unpredicted reward or by a decrease after failure of a predicted reward (Schultz, 1998), whereas reward uncertainty appears to be encoded by a crescendo of DA neuron activity between the conditioned stimulus and reward (…”
The Teenage Brain: Sensitivity to Rewards
Galván, A. (2013) · Current Directions in Psychological Science · doi:10.1177/0963721413480859 · Cited by 231
“By the end of the experiment, the dopamine neurons are minimally responsive to the actual reward delivery and most responsive to the cue (Fiorillo et al., 2003).”
BackgroundAge-related changes in midbrain dopaminergic regulation of the human reward system
Dreher, J.-C., Meyer-Lindenberg, A., Kohn, P., & Berman, K. F. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0802127105 · Cited by 230
Smolka, M. N., Bühler, M., Klein, S., Zimmermann, U., Mann, K., Heinz, A., & Braus, D. F. (2006) · Psychopharmacology · doi:10.1007/s00213-005-0080-x · Cited by 230
Placebo effects in psychiatry: mediators and moderators
Weimer, K., Colloca, L., & Enck, P. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)00092-3 · Cited by 219
Glutamatergic model psychoses: prediction error, learning, and inference
Corlett, P. R., Honey, G. D., Krystal, J. H., & Fletcher, P. C. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.163 · Cited by 229
From prediction error to psychosis: Ketamine as a pharmacological model of delusions
Corlett, P. R., Honey, G. D., & Fletcher, P. C. (2007) · Journal of Psychopharmacology · doi:10.1177/0269881107077716 · Cited by 227
“The more attention paid to a stimulus, the more readily the organism will associate it with a particular outcome in the environment (Fiorillo et al., 2003, 2005; Grossberg, 1982; Pearce and Hall, 1980).”
BackgroundInfluentialIs a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making
Kalenscher, T., & Pennartz, C. M. A. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2007.11.004 · Cited by 222
“For a more detailed overview about the psychology and neurobiology of decisions under risk, refer to the existing literature (for example, Von Neumann and Morgenstern, 1944; Friedman and Savage, 1948, 1952; Kahneman and Tversky, 1979; Platt and Glimcher, 1999; Montague and Berns, 2002; Fiorillo et al., 2003; Barraclou…”
BackgroundBasal ganglia and dopamine contributions to probabilistic category learning
Shohamy, D., Myers, C. E., Kalanithi, J., & Gluck, M. A. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.07.008 · Cited by 218
“This idea is based on a series of seminal studies demonstrating that midbrain dopamine neurons in animals implement a reward-related ‘‘prediction error’’ (Fiorillo et al., 2003; Hollerman and Schultz, 1998; Schultz, 1998; Schultz et al., 1997).”
BackgroundResultInfluentialGenetic evidence implicating DARPP-32 in human frontostriatal structure, function, and cognition
Meyer-Lindenberg, A., Straub, R. E., Lipska, B. K., Verchinski, B. A., Goldberg, T., Callicott, J. H., Egan, M. F., Huffaker, S. S., Mattay, V. S., Kolachana, B., et al. (2007) · Journal of Clinical Investigation · doi:10.1172/jci30413 · Cited by 226
“Since dopaminergic neurotransmission is critical for motivated behavior, working memory (4), and reward-related learning (5) and is implicated in, among other conditions, schizophrenia (6), alcoholism (7), Parkinson disease, and pathological gambling (8), DARPP-32 has received considerable attention not only in basic…”
BackgroundBasal ganglia circuit loops, dopamine and motivation: a review and enquiry
Ikemoto, S., Yang, C., & Tan, A. (2015) · Behavioural Brain Research · doi:10.1016/j.bbr.2015.04.018 · Cited by 226
Neural plasticity of development and learning
Galván, A. (2010) · Human Brain Mapping · doi:10.1002/hbm.21029 · Cited by 198
“Eventually, the dopamine signal decreases as the organism learns to expect the event [Fiorillo et al., 2003; 2008].”
BackgroundStriatal mechanisms underlying movement, reinforcement, and punishment
Kravitz, A. V., & Kreitzer, A. C. (2012) · Physiology · doi:10.1152/physiol.00004.2012 · Cited by 224
Potential Vulnerabilities of Neuronal Reward, Risk, and Decision Mechanisms to Addictive Drugs
Schultz, W. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.014 · Cited by 220
“From Fiorillo et al. (2003), with permission by American Association for the Advancement of Science.”
BackgroundSelf-citationWhat do the basal ganglia do? A modeling perspective
Chakravarthy, V. S., Joseph, D., & Bapi, R. S. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0401-y · Cited by 215
“The other proposal is that the phasic DA firing relates to prediction error and the tonic DA activity component relates to the uncertainty of the prediction ( Fiorillo et al. 2003 ).”
BackgroundUpdating memories—The role of prediction errors in memory reconsolidation
Exton-McGuinness, M. T. J., Lee, J. L. C., & Reichelt, A. C. (2015) · Behavioural Brain Research · doi:10.1016/j.bbr.2014.10.011 · Cited by 203
Kishida, K. T., Saez, I., Lohrenz, T., Witcher, M. R., Laxton, A. W., Tatter, S. B., White, J. P., Ellis, T. L., Phillips, P. E. M., & Montague, P. R. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1513619112 · Cited by 212
Monetary Reward Processing in Obese Individuals With and Without Binge Eating Disorder
Balodis, I. M., Kober, H., Worhunsky, P. D., White, M. A., Stevens, M. C., Pearlson, G. D., Sinha, R., Grilo, C. M., & Potenza, M. N. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.01.014 · Cited by 210
“Indeed, the VS, hypothalamus, thalamus, and prefrontal cortex represent predominant projection areas of the mesocorticolimbic dopamine system, consistent with the role of this neurotransmitter in reward processing (52,53).”
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
“…fits comfortably with formulations in terms of signal to noise (Cohen and Servan-Schreiber 1992; Goldman-Rakic 1998), uncertainty and precision (Fiorillo et al. 2003; Fiorillo et al. 2008), and the key role that dopamine plays in selecting among alternative actions (Braver et al. 1999; Frank…”
BackgroundPizzagalli, D. A., Evins, A. E., Schetter, E. C., Frank, M. J., Pajtas, P. E., Santesso, D. L., & Culhane, M. (2008) · Psychopharmacology · doi:10.1007/s00213-007-0957-y · Cited by 219
“Omission of a predicted reward, on the other hand, elicits phasic DA decreases (i.e., negative-prediction error; Fiorillo et al. 2003; Hollerman and Schultz 1998), and the behavior that led to lack of reward is extinguished.”
Pan, W.-X., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0277-05.2005 · Cited by 210
“…latency to conditioned stimuli in classical and operant appetitive conditioning tasks (Miller et al., 1981; Schultz, 1998; Hyland et al., 2002; Fiorillo et al., 2003) and show several classical phenomena of conditioned behavior such as generalization and blocking (Mirenowicz and Schultz,…”
BackgroundInfluentialUncertainty of feedback and state estimation determines the speed of motor adaptation
Wei, K. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00011 · Cited by 205
“In recent years researchers have made significant progresses at investigating neural correlates of uncertainty and its computation in the brain (Zemel et al., 1998; Fiorillo et al., 2003; Ma et al., 2006; Preuschoff et al., 2006; Behrens et al., 2007; Deneve, 2008; Rushworth and Behrens, 2008).”
BackgroundThe Neuroscience of Learning: Beyond the Hebbian Synapse
Gallistel, C. R., & Matzel, L. D. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143807 · Cited by 199
“…learning is shown by the finding that signals in the mesolimbic dopaminergic neurons encode the probability and uncertainty of reinforcement (Fiorillo et al. 2003) and by the more recent discovery that the response of neurons in the amygdala to reward-predicting stimuli depends on the…”
BackgroundExercise increases caudate dopamine release and ventral striatal activation in Parkinson's disease
Sacheli, M. A., Neva, J. L., Lakhani, B., Murray, D. K., Vafai, N., Shahinfard, E., English, C., McCormick, S., Dinelle, K., Neilson, N., et al. (2019) · Movement Disorders · doi:10.1002/mds.27865 · Cited by 196
Multiple roles for orexin/hypocretin in addiction
Mahler, S. V., Smith, R. J., Moorman, D. E., Sartor, G. C., & Aston-Jones, G. (2012) · Progress in Brain Research · doi:10.1016/b978-0-444-59489-1.00007-0 · Cited by 215
“One of the key ways in which VTA orexin may act to increase conditioned reward seeking is by modulating forebrain dopamine neurotransmission, which has been tied to reward seeking and approach efforts (Berridge and Robinson, 1998; Cheer et al., 2007; Fiorillo et al., 2003; Ikemoto and Panksepp, 1999; Salamone et al.,…”
Linnet, J., Møller, A., Peterson, E., Gjedde, A., & Doudet, D. (2011) · Addiction · doi:10.1111/j.1360-0443.2010.03126.x · Cited by 214
“This is consistent with reports of increased dopamine response in relation to specific stimuli such as uncertain outcomes [24], reward prediction error [25] and gains [27].”
ResultAdaptation of Reward Sensitivity in Orbitofrontal Neurons
Kobayashi, S., Pinto de Carvalho, O., & Schultz, W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4009-09.2010 · Cited by 206
“Reward value is often subjective, as shown by temporal discounting (Roesch et al., 2007; Kobayashi and Schultz, 2008), and includes probability that reflects the frequency of occurrence rather than direct sensory stimulation (Fiorillo et al., 2003).”
BackgroundSelf-citationKennerley, S. W., & Wallis, J. D. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06743.x · Cited by 210
Dopamine agonists and risk: impulse control disorders in Parkinson's; disease
Voon, V., Gao, J., Brezing, C., Symmonds, M., Ekanayake, V., Fernandez, H., Dolan, R. J., & Hallett, M. (2011) · Brain · doi:10.1093/brain/awr080 · Cited by 209
“Tonic dopamine has also been implicated in representation of risk as measured using variance (Fiorillo et al., 2003).”
MethodMesolimbic dopamine release is linked to symptom severity in pathological gambling
Joutsa, J., Johansson, J., Niemelä, S., Ollikainen, A., Hirvonen, M. M., Piepponen, P., Arponen, E., Alho, H., Voon, V., Rinne, J. O., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.006 · Cited by 209
“Midbrain dopamine neurons have been shown to encode both the reward risk and the reward value in monkeys (Fiorillo et al., 2003), and the human brain has been reported to respond hemodynamically to risk and reward value in a card task (Preuschoff et al.”
BackgroundFunctional Dissociations of Risk and Reward Processing in the Medial Prefrontal Cortex
Xue, G., Lu, Z., Levin, I. P., Weller, J. A., Li, X., & Bechara, A. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn147 · Cited by 207
Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain
Roesch, M. R., Esber, G. R., Li, J., Daw, N. D., & Schoenbaum, G. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07986.x · Cited by 191
The role of ventral frontostriatal circuitry in reward-based learning in humans.
Galvan, A., Hare, T. A., Davidson, M., Spicer, J., Glover, G., & Casey, B. J. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.2431-05.2005 · Cited by 207
“Lack of a sufficient reward at an expected time is similar to the lack of an expected event at a given time, which both result in a dip below baseline (decreased striatal activity) in human imaging (Davidson et al., 2004) and nonhuman studies (Fiorillo et al., 2003).”
BackgroundDopamine Reward Prediction Error Responses Reflect Marginal Utility
Stauffer, W. R., Lak, A., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.064 · Cited by 204
Self-citationAttention and awareness in stage magic: turning tricks into research
Macknik, S. L., King, M., Randi, J., Robbins, A., Teller, Thompson, J., & Martinez-Conde, S. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2473 · Cited by 184
Winn, P. (2006) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2006.05.036 · Cited by 189
“[131] Fiorillo CD, Tobler PN, Schultz W.”
BackgroundResultSaccade Reward Signals in Posterior Cingulate Cortex
McCoy, A. N., Crowley, J. C., Haghighian, G., Dean, H. L., & Platt, M. L. (2003) · Neuron · doi:10.1016/s0896-6273(03)00719-0 · Cited by 195
Esber, G. R., & Haselgrove, M. (2011) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2011.0836 · Cited by 178
Ungless, M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.001 · Cited by 192
“On the basis of these and other studies [12, 13 ], it has been suggested that dopamine neurons encode a reward prediction-error signal [6–9].”
ResultDopamine, Updated: Reward Prediction Error and Beyond
Lerner, T. N., Holloway, A. L., & Seiler, J. L. (2021) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.10.012 · Cited by 167
Redgrave, P., Vautrelle, N., & Reynolds, J. N. J. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.07.060 · Cited by 183
Kennerley, S. W., & Walton, M. E. (2011) · Behavioral Neuroscience · doi:10.1037/a0023575 · Cited by 176
“Moreover, the negative prediction error activity rarely took the form of a depression of activity from baseline (e.g., Figure 3B), as has been reported in dopamine neurons (Fiorillo et al., 2003; Schultz et al., 1997).”
ResultBehavioral and Neural Properties of Social Reinforcement Learning
Jones, R. M., Somerville, L. H., Li, J., Ruberry, E. J., Libby, V., Glover, G., Voss, H. U., Ballon, D. J., & Casey, B. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.2972-11.2011 · Cited by 191
Reinforcement learning, conditioning, and the brain: Successes and challenges
Maia, T. V. (2009) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.9.4.343 · Cited by 187
“The aforementioned sustained response of dopamine neurons during the anticipation of rewards has been interpreted as a risk (i.e., variance) signal (Fiorillo et al., 2003; Schultz et al., 2008).13 When different CSs predict a reward with different probabilities, the greater the risk (i.e., the…”
BackgroundInfluentialReward Processing, Neuroeconomics, and Psychopathology
Zald, D. H., & Treadway, M. T. (2017) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-044957 · Cited by 176
“…options (Schultz et al 2015) and are sensitive to effort (Varazzani et al 2015), temporal (Kobayashi & Schultz 2008) and probability discounting (Fiorillo et al 2003), and exhibit the type of adaptive scaling necessary to represent a wide range of reward values under different contexts (Tobler…”
Why people gamble: a model with five motivational dimensions
Binde, P. (2013) · International Gambling Studies · doi:10.1080/14459795.2012.712150 · Cited by 158
“It has been argued that ‘a behavior as prevalent as gambling must be explained in terms that are consistent with natural selection’ (Fiorillo, Tobler, & Schultz, 2003 , p.”
BackgroundCost, Benefit, Tonic, Phasic: What Do Response Rates Tell Us about Dopamine and Motivation?
NIV, Y. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.018 · Cited by 183
Cerebral cortex : principles of operation
Rolls, E. T. (2016) · doi:10.82444/warw.33712699 · Cited by 175
Cholinergic modulation of midbrain dopaminergic systems
Mena-Segovia, J., Winn, P., & Bolam, J. P. (2008) · Brain Research Reviews · doi:10.1016/j.brainresrev.2008.02.003 · Cited by 172
“…phasic activation of the firing of midbrain dopamine neurons is maximal in conditions of high uncertainty (when the probability that a stimulus signals reward is 0.5) and lowest in conditions of certainty (when the probability that a stimulus signals reward is either 0 or 1; Fiorillo et al., 2003).”
InfluentialNeural coding of distinct statistical properties of reward information in humans
Dreher, J.-C., Kohn, P., & Berman, K. F. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj004 · Cited by 180
Arnon, I., & Ramscar, M. (2012) · Cognition · doi:10.1016/j.cognition.2011.10.009 · Cited by 182
Artificial Intelligence in Education
Holmes, W. (2020) · Springer eBooks · doi:10.1007/978-3-030-10576-1_107 · Cited by 239
Phasic dopamine release in appetitive behaviors and drug addiction
Wanat, M., Willuhn, I., Clark, J., & Phillips, P. (2009) · Current Drug Abuse Reviewse · doi:10.2174/1874473710902020195 · Cited by 177
“In support, dopamine neuron firing correlates with the probability of reward availability [96], the uncertainty of reward delivery [97], as well as the magnitude of the reward [98].”
Strafella, A. P., Ko, J. H., & Monchi, O. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.02.005 · Cited by 177
“Indeed, dopamine neurons have long been implicated with reward mechanisms, involving either expectation of reward (tonic activation) or reward itself (phasic activation) (Wise and Rompre, 1989; Schultz, 1998; Fiorillo et al., 2003).”
Comparison of learning‐related neuronal activity in the dorsal premotor cortex and striatum
Brasted, P. J., & Wise, S. P. (2004) · European Journal of Neuroscience · doi:10.1111/j.0953-816x.2003.03181.x · Cited by 173
Fouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
“…in the framework of learning, some authors argue that surprise can also be considered as a signal predicting the level of risk associated with a future decision outcome, and thus reflect a risk RPE (Fiorillo et al., 2003; Preuschoff, Quartz, & Bossaerts, 2008; Rudorf, Preuschoff, & Weber, 2012).”
BackgroundA perceptual inference mechanism for hallucinations linked to striatal dopamine
Cassidy, C. M., Balsam, P. D., Weinstein, J. J., Rosengard, R. J., Slifstein, M., Daw, N. D., Abi-Dargham, A., & Horga, G. (2018) · Current Biology · doi:10.1016/j.cub.2017.12.059 · Cited by 176
“Dopamine has been increasingly implicated in signaling confidence, certainty, or precision in reward [45, 46] and perceptual tasks [43, 44].”
BackgroundResultA Decade of Orexin/Hypocretin and Addiction: Where Are We Now?
James, M. H., Mahler, S. V., Moorman, D. E., & Aston-Jones, G. (2016) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2016_57 · Cited by 171
(2008) · Monographs of the Society for Research in Child Development · doi:10.1111/j.1540-5834.2007.00436.x · Cited by 171
Uncertainty and Cognitive Control
Mushtaq, F., Bland, A. R., & Schaefer, A. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00249 · Cited by 149
““Jumping to conclusions” (JTC), a frequent symptom in schizophrenia, has also been found to be associated with the construct of IOU (Freeman et al., 2008).”
BackgroundRepresentation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173
“One of these studies (Fiorillo et al., 2003) revealed an additional response phenomenon: a tonic “ramp” of responding between stimulus and reward, maximal for 50% reinforcement.”
BackgroundInfluentialSchultz, W., Stauffer, W. R., & Lak, A. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.03.013 · Cited by 162
Self-citationReward uncertainty enhances incentive salience attribution as sign-tracking
Anselme, P., Robinson, M. J. F., & Berridge, K. C. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2012.10.006 · Cited by 170
Translational perspectives for computational neuroimaging
Stephan, K. E., Iglesias, S., Heinzle, J., & Diaconescu, A. O. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.008 · Cited by 169
Phasic Nucleus Accumbens Dopamine Release Encodes Effort- and Delay-Related Costs
Day, J. J., Jones, J. L., Wightman, R. M., & Carelli, R. M. (2010) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.03.026 · Cited by 160
Medication-related impulse control and repetitive behaviors in Parkinson??s disease
Voon, V., Potenza, M. N., & Thomsen, T. (2007) · Current Opinion in Neurology · doi:10.1097/wco.0b013e32826fbc8f · Cited by 163
Stuber, G. D., Wightman, R. M., & Carelli, R. M. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.036 · Cited by 168
From recurrent choice to skill learning: A reinforcement-learning model.
Fu, W.-T., & Anderson, J. R. (2006) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.135.2.184 · Cited by 167
Generating bursts (and pauses) in the dopamine midbrain neurons
Paladini, C. A., & Roeper, J. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.07.032 · Cited by 160
“The cells normally fire constantly at a low rate, and speed up, firing a phasic burst when reward exceeds prediction (for example an unexpected reward) or pause when an expected reward is withheld (Fiorillo et al., 2003).”
The role of high-frequency oscillatory activity in reward processing and learning
Marco-Pallarés, J., Münte, T. F., & Rodríguez-Fornells, A. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.11.014 · Cited by 164
Brain systems underlying the affective and social monitoring of actions: An integrative review
Koban, L., & Pourtois, G. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.02.014 · Cited by 161
“In this view, positive vs. negative affective value of actions are determined quickly and automatically by the dACC/dMFC in conjunction with specific striatal and limbic structures (Fiorillo et al., 2003; Holroyd and Coles, 2002a).”
Risk prediction and aversion by anterior cingulate cortex
BROWN, J. W., & BRAVER, T. S. (2007) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.7.4.266 · Cited by 160
Fauth‐Bühler, M., Mann, K., & Potenza, M. N. (2017) · Addiction Biology · doi:10.1111/adb.12378 · Cited by 151
WICKENS, J. R., BUDD, C. S., HYLAND, B. I., & ARBUTHNOTT, G. W. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.016 · Cited by 165
Striatal ups and downs: Their roles in vulnerability to addictions in humans
Leyton, M., & Vezina, P. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.01.018 · Cited by 165
“For example, in nonhuman primates, the largest increases in reward cue-induced DA cell firing are seen under conditions of maximal uncertainty (Fiorillo et al., 2003).”
Expectations and associations that heal: Immunomodulatory placebo effects and its neurobiology
Pacheco-López, G., Engler, H., Niemi, M.-B., & Schedlowski, M. (2006) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2006.05.003 · Cited by 164
Distinct Tonic and Phasic Anticipatory Activity in Lateral Habenula and Dopamine Neurons
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.016 · Cited by 148
“Early reports suggested that some dopamine neurons have small changes in tonic activity in anticipation of arm movements to obtain food rewards (Romo and Schultz, 1990) or delivery of probabilistic rewards (Fiorillo et al., 2003), but these changes have not always been reported in later studies including our present d…”
BackgroundResultInfluentialSerotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164
Wichmann, T., & DeLong, M. R. (2016) · Neurotherapeutics · doi:10.1007/s13311-016-0426-6 · Cited by 181
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.