Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120
“Conversely, it may provide a framework in which current debates could be resolved, for example whether dopamine encodes reward prediction error or surpris”
BackgroundNeural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
“but may also be active, at least in their tonic mode, under other circumstances in response to such factors as unpredictabilit”
BackgroundBardo, M. T., & Hammer, R. P. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90226-e · Cited by 3,219
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
“…respond to the first or last event in a sequence (Ravel and Richmond, 2006; Jin and Costa, 2010); that have sustained activation by risky rewards (Fiorillo et al., 2003); or that are activated during body movements (Schultz, 1986; Kiyatkin, 1988a; Puryear et al., 2010; Jin and Costa, 2010; see…”
BackgroundWhat makes us tick? Functional and neural mechanisms of interval timing
Buhusi, C. V., & Meck, W. H. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1764 · Cited by 2,062
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Bayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004
“A bayesian view of sensorimotor learning is consistent with neuro-physiological studies showing that the brain represents the degree of uncertainty when estimating rewards 25–27 and with psychophysical studies addressing the timing of movements 28,29 .”
ResultBerridge, K. C., & Robinson, T. E. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00233-9 · Cited by 1,869
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697
“…et al. 2003, Tobler et al. 2005, Waelti et al. 2001), that they are specialized to respond in relation to positive but not aversive reinforcements, and that they may code uncertainty (Fiorillo et al. 2003, Hsu et al. 2005, Niv et al. 2005, Ungless et al. 2004) or salience (Redgrave & Gurney 2006).”
BackgroundBehavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637
Self-citationMultiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415
“It covaries with reward probability (Fiorillo et al. 2003, Morris et al. 2004) and the expected value of the reward distribution (Tobler et al. 2005) and is modulated by the motivation of the animal (Satoh et al. 2003), the time course of predictions (Nakahara et al. 2004), and the animal’s choice…”
BackgroundMethodInfluentialSelf-citationMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Apathy and the Functional Anatomy of the Prefrontal Cortex–Basal Ganglia Circuits
Levy, R., & Dubois, B. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj043 · Cited by 1,285
“In addition, dopamine may also code uncertainty of reward delivery (Fiorillo et al., 2003).”
BackgroundAdaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Self-citationActive inference: a process theory
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017) · Neural Computation · doi:10.1162/neco_a_00912 · Cited by 1,156
“This can be related to dopamine discharges that have been interpreted in terms of changes in expected reward (Schultz & Dickinson, 2000; Fiorillo et al., 2003) and marginal utility (Stauffer, Lak, & Schultz, 2014).”
BackgroundInfluentialTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
“Larger phasic prediction-error signals for stimuli previously associated with higher rewards ( Fiorillo et al. 2003; Tobler et al. 2005) would then result in faster responding to these cues.”
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
“Dopamine neurons code reward risk during the stimulusreward interval after the value prediction error response (FIGURE 31C) (161).”
BackgroundInfluentialSelf-citationDopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036
Self-citationSchultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028
Self-citationReward-motivated learning: mesolimbic activation precedes memory formation
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. E. (2006) · Neuron · doi:10.1016/j.neuron.2006.03.036 · Cited by 977
“Electrophysiological research indicates that, after reward-based classical conditioning has occurred, VTA dopamine neurons fire in response to cues that predict reward (Schultz, 1998) and during anticipation of reward (Fiorillo et al., 2003).”
BackgroundUnique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976
“Schultz and colleagues have successfully defined the computational roles of tonic and phasic discharge patterns of conventional DA midbrain neurons in reward-prediction error and uncertainty signaling (Fiorillo et al., 2003; Schultz, 2007; Tobler et al., 2005).”
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
“(38) showed that dopamine neurons in classical conditioning tasks seem to show a ramp of activity between cue and reward whenever the rewards are delivered probabilistically, as shown in Fig.”
BackgroundDistributed Neural Representation of Expected Value
Knutson, B., Taylor, J., Kaufman, M., Peterson, R., & Glover, G. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0642-05.2005 · Cited by 955
“, 50 vs 80 and 20%, a quadratic trend), as suggested by electrophysiological recordings of midbrain dopamine neurons in monkeys (Fiorillo et al., 2003).”
BackgroundAn Interoceptive Predictive Coding Model of Conscious Presence
Seth, A. K., Suzuki, K., & Critchley, H. D. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00395 · Cited by 950
“The precision of prediction error signals has been associated specifically with dopaminergic activity (Fiorillo et al., 2003), suggesting a proximate neuronal origin of schizophrenic symptomatology in terms of abnormal dopaminergic neurotransmission (Fletcher and Frith, 2009).”
BackgroundInfluentialOrbitofrontal Cortex and Its Contribution to Decision-Making
Wallis, J. D. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094334 · Cited by 900
“…show that neurons in a variety of regions, including MPFC (Amiez et al. 2006), parietal cortex (Platt & Glimcher 1999), and dopamine neurons (Fiorillo et al. 2003), combine information about the size of a payoff and its probability of occurrence to derive an expected value for a given…”
BackgroundThe Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
Learned predictions of error likelihood in the anterior cingulate cortex
Brown, J. W., & Braver, T. S. (2005) · Science · doi:10.1126/science.1105783 · Cited by 928
Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes
Matell, M. S., & Meck, W. H. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.06.012 · Cited by 877
“[52] showing that tonic firing rates of dopamine cells are directly related to the amount of uncertainty a subject has about the outcome of a future event.”
BackgroundChoice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
High impact running improves learning
Winter, B., Breitenstein, C., Mooren, F. C., Voelker, K., Fobker, M., Lechtermann, A., Krueger, K., Fromme, A., Korsukewitz, C., Floel, A., et al. (2007) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2006.11.003 · Cited by 794
Neuroanatomical and Neurochemical Substrates of Timing
Coull, J. T., Cheng, R.-K., & Meck, W. H. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.113 · Cited by 808
“…for interval timing within the framework of SBF is supported by findings that midbrain dopamine neurons are very sensitive to the durations being timed at trial onset and to the expected time of reward (Fiorillo et al, 2003, 2008), both presumably providing 'timestamps' in the neural circuits.”
ResultMesolimbic dopamine signals the value of work
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 824
“Firing rates of presumed dopamine cells have been previously reported to escalate within trials under some conditions 47 , but this has not been typically reported with reward anticipation.”
States of Curiosity Modulate Hippocampus-Dependent Learning via the Dopaminergic Circuit
Gruber, M. J., Gelman, B. D., & Ranganath, C. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.060 · Cited by 734
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Mendl, M., Burman, O. H. P., Parker, R. M. A., & Paul, E. S. (2009) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2009.02.023 · Cited by 755
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 717
“…‘incentive salience’ (e.g. Horvitz (2000), Redgrave and Gurney (2006) and Redgrave, Prescott, and Gurney (1999)), Berridge and Robbinson’s ‘wanting’ versus ‘liking’ (e.g. Berridge (2007) and Berridge and Robinson (1998)), and ideas about dopamine signaling uncertainty (Fiorillo et al., 2003).”
BackgroundTreadway, M. T., Buckholtz, J. W., Schwartzman, A. N., Lambert, W. E., & Zald, D. H. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006598 · Cited by 769
“Preclinical models suggest that Nacc DA release is greatest for trials with high uncertainty and high reward magnitude [37].”
BackgroundBilder, R. M., Volavka, J., Lachman, H. M., & Grace, A. A. (2004) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300542 · Cited by 769
“Neuropsychopharmacology learning, by updating the links between stimulus and response when an expected reward does not occur (Fiorillo et al, 2003; Schultz 2002, 2003).”
BackgroundThe short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
Dissociable dopamine dynamics for learning and motivation
Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019) · Nature · doi:10.1038/s41586-019-1235-y · Cited by 735
Beyond simple models of self-control to circuit-based accounts of adolescent behavior
Casey, B. J. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015156 · Cited by 742
“Building on prior nonhuman primate studies showing the importance of reinforcement learning and prediction error in dopamine-rich circuitry (Cromwell & Schultz 2003, Fiorillo et al. 2003), investigators varied the magnitudes of monetary reward to discrete stimuli.”
BackgroundShohamy, D., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 707
Neural Signatures of Economic Preferences for Risk and Ambiguity
Huettel, S. A., Stowe, C. J., Gordon, E. M., Warner, B. T., & Platt, M. L. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.024 · Cited by 712
“edu risk, and components of risk like probability and reward variance influence the activity of midbrain dopamine neurons as well as the activation of ventral prefrontal, insular and cingulate cortices (Bechara et al., 1999; Critchley et al., 2001; Fiorillo et al., 2003; Smith et al., 2002).”
BackgroundActive inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
“The composition of extrinsic and epistemic value implicit in expected free energy can also be used to reproduce the empirical responses of dopaminergic cells to CS under different levels of reward and uncertainty (Fiorillo et al., 2003).”
BackgroundResultInfluentialAssociative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710
“Moreover, the events that serve as predictors come to elicit brief dopamine bursts even though they often possess no inherent biological value, and the magnitude of these conditioned neuronal responses is correlated with the certainty of the reward being predicte”
BackgroundNeural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692
“…and reward variance in human dopaminoceptive structures follow a pattern consistent with that found in nonhuman primate ventral midbrain using electrophysiology, which report an immediate , phasic encoding of expected reward and a late-onset, sustained encoding of risk (Fiorillo et al., 2003).”
BackgroundInfluentialThe basal ganglia: learning new tricks and loving it
Graybiel, A. M. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.10.006 · Cited by 649
“The dopamine-containing neurons code a reward-prediction error in their phasic firing and appear also to code the uncertainty of the prediction in their maintained firing levels [19] (but see Niv et al.”
BackgroundDecision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679
“Consistent with a prediction error signal, spiking activity of dopamine neurons increases with both reward magnitude and probability (Fiorillo et al., 2003; Tobler et al., 2005; Roesch et al., 2007).”
BackgroundFriston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., ODoherty, J., & Pezzulo, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.022 · Cited by 674
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Alcaro, A., Huber, R., & Panksepp, J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.07.014 · Cited by 616
“The persistent increase of DA cell firing in such unpredictable conditions (Fiorillo et al., 2003) may promote the emergence of an unstable state, characterized by the release of instinctual behavioral arousal patterns, which drive organisms to explore external stimuli and to cope with…”
BackgroundDopamine signals for reward value and risk: basic and recent data
Schultz, W. (2010) · Behavioral and Brain Functions · doi:10.1186/1744-9081-6-24 · Cited by 653
“The activations increase monotonically with reward probability [18] and reward magnitude, such as liquid volume [23].”
InfluentialSelf-citationUpdating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651
Self-citationErnst, M., Nelson, E. E., Jazbec, S., McClure, E. B., Monk, C. S., Leibenluft, E., Blair, J., & Pine, D. S. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.038 · Cited by 635
Ventral Tegmental Area Neurons in Learned Appetitive Behavior and Positive Reinforcement
Fields, H. L., Hjelmstad, G. O., Margolis, E. B., & Nicola, S. M. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094341 · Cited by 621
“In one test of this idea, Fiorillo et al. (2003) trained monkeys with different visual cues indicating different probabilities of juice delivery.”
BackgroundReward expectation modulates feedback-related negativity and EEG spectra
Cohen, M. X., Elger, C. E., & Ranganath, C. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.11.056 · Cited by 586
Mapping the human brain's cortical-subcortical functional network organization
Ji, J. L., Spronk, M., Kulkarni, K., Repovš, G., Anticevic, A., & Cole, M. W. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.10.006 · Cited by 585
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making
Preuschoff, K. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00115 · Cited by 519
“, 2007) as well as for the tonic dopamine response in non-human primates (Fiorillo et al., 2003).”
BackgroundStriatal circuits for reward learning and decision-making
Cox, J., & Witten, I. B. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0189-2 · Cited by 568
Nobre, A., Correa, A., & Coull, J. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.07.006 · Cited by 562
“In addition, activity of these cells rises monotonically in anticipation of the predicted time of an uncertain reward occurrence or magnitude [66].”
BackgroundCoincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
“(p Ͼ 0.8), indicating that uncertainty level (Fiorillo et al., 2003) did not”
BackgroundInfluentialMenstrual cycle phase modulates reward-related neural function in women
Dreher, J.-C., Schmidt, P. J., Kohn, P., Furman, D., Rubinow, D., & Berman, K. F. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0605569104 · Cited by 567
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565
Risky business: the neuroeconomics of decision making under uncertainty
Platt, M. L., & Huettel, S. A. (2008) · Nature Neuroscience · doi:10.1038/nn2062 · Cited by 556
“(a) Midbrain dopamine neurons in monkeys increase firing in anticipation of probabilistically delivered juice rewards (after C.D. Fiorillo et al., 2003 ) 69 .”
BackgroundMore attention must be paid: The neurobiology of attentional effort
Sarter, M., Gehring, W. J., & Kozak, R. (2006) · Brain Research Reviews · doi:10.1016/j.brainresrev.2005.11.002 · Cited by 552
“Midbrain dopaminergic neurons code such prediction errors; for example, omitted rewards are associated with reduced dopaminergic activity (Fiorillo et al., 2003; Schultz, 1998; Tobler et al., 2003, 2005).”
BackgroundHyperdopaminergic mutant mice have higher "wanting" but not "liking" for sweet rewards
Peciña, S., Cagniard, B., Berridge, K. C., Aldridge, J. W., & Zhuang, X. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-28-09395.2003 · Cited by 551
“…of dopamine function known so far to be elevated by DAT knockdown mutation (Zhuang et al., 2001), to enhance relevant phasic neuronal computations of prediction (Montague and Berns, 2002; Pagnoni et al., 2002; Schultz, 2002; Suri, 2002; Fiorillo et al., 2003; Shizgal and Arvanitogiannis, 2003).”
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
New Insights into the Placebo and Nocebo Responses
Enck, P., Benedetti, F., & Schedlowski, M. (2008) · Neuron · doi:10.1016/j.neuron.2008.06.030 · Cited by 531
“It is known that with a 0.5 probability of reward, 29% of dopaminergic cells are tonically activated (Fiorillo et al., 2003).”
InfluentialYacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
“In nonhuman primates, mesolimbic dopaminergic neurons are involved in the representation of reward probability and reward magnitude (Schultz et al., 1997; Fiorillo et al., 2003; Tobler et al., 2005).”
BackgroundPrefrontal cortical regulation of brainwide circuit dynamics and reward-related behavior
Ferenczi, E. A., Zalocusky, K. A., Liston, C., Grosenick, L., Warden, M. R., Amatya, D., Katovich, K., Mehta, H., Patenaude, B., Ramakrishnan, C., et al. (2016) · Science · doi:10.1126/science.aac9698 · Cited by 535
Hallucinations and strong priors
Corlett, P. R., Horga, G., Fletcher, P. C., Alderson-Day, B., Schmack, K., & Powers, A. R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.001 · Cited by 534
Computational psychiatry: the brain as a phantastic organ
Friston, K. J., Stephan, K. E., Montague, R., & Dolan, R. J. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70275-5 · Cited by 531
Functional diversity and specificity of neostriatal interneurons
Tepper, J. M., & Bolam, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.003 · Cited by 526
“neurons that respond to similar stimuli with a co-incident short increase in activity (Figure 3b) [45], have been proposed to participate in the modulation of the activity of MSNs and hence, the functioning of the neostriatal circuits that underlie reward-based learning and/or motivated behavior [44,46 ].”
BackgroundSchultz, W. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.03.017 · Cited by 517
“The activation with positive prediction errors follows a different slope than the depression with negative prediction errors, which could be due to the low background activity of dopamine neurons onto which the depression needs to act [50].”
BackgroundInfluentialSelf-citationTheory of Mind: A Neural Prediction Problem
Koster-Hale, J., & Saxe, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.020 · Cited by 513
“The central idea of “predictive coding” is that (some) neural responses contain information not about the value of a currently perceived stimulus, but about the difference between the stimulus value and the expected value (Fiorillo et al., 2003; Schultz et al., 1997; Schultz, 2010).”
How prediction errors shape perception, attention, and motivation
den Ouden, H. E. M., Kok, P., & de Lange, F. P. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00548 · Cited by 512
“In line with this account, dopamine neurons have been reported in the primate midbrain that report uncertainty about the predictions (Fiorillo et al., 2003).”
BackgroundAction, Outcome, and Value: A Dual-System Framework for Morality
Cushman, F. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313495594 · Cited by 473
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Money as tool, money as drug: The biological psychology of a strong incentive
Lea, S. E. G., & Webley, P. (2006) · Behavioral and Brain Sciences · doi:10.1017/s0140525x06009046 · Cited by 478
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 465
“Current Opinion in Neurobiology 2006, 16:199–204 linearly with the reward probability [35–37], as predicted by the model.”
BackgroundInfluentialSense of agency in health and disease: A review of cue integration approaches
Moore, J. W., & Fletcher, P. C. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2011.08.010 · Cited by 448
Double dissociation of value computations in orbitofrontal and anterior cingulate neurons
Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2961 · Cited by 454
“This feature of PFC neurons is in stark contrast to activity measured, for example, in dopamine neurons, which encode reward and reward prediction error at predictable times, using a unified coding schem”
MethodIntuition and autism: a possible role for Von Economo neurons
Allman, J. M., Watson, K. K., Tetreault, N. A., & Hakeem, A. Y. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.06.008 · Cited by 484
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 462
“This could reflect an asymmetry reported at the spiking level for dopaminergic neurons (Fiorillo et al., 2003; Bayer and Glimcher, 2005; Niv et al., 2005; Morris et al., 2006), where positive errors are coded more strongly than negative ones.”
BackgroundHuettel, S. A., Song, A. W., & McCarthy, G. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.5070-04.2005 · Cited by 454
Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472
Self-citationA distributional code for value in dopamine-based reinforcement learning
Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020) · Nature · doi:10.1038/s41586-019-1924-6 · Cited by 443
Beyond stereotypes of adolescent risk taking: Placing the adolescent brain in developmental context
Romer, D., Reyna, V. F., & Satterthwaite, T. D. (2017) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.07.007 · Cited by 407
“In particular, abundant evidence from studies in animals suggests that dopamine release from neurons in the midbrain plays an important role in the coding of reward prediction errors and uncertainty (Fiorillo et al., 2003; Tobler et al., 2005).”
Reward processing by the lateral habenula in normal and depressive behaviors
Proulx, C. D., Hikosaka, O., & Malinow, R. (2014) · Nature Neuroscience · doi:10.1038/nn.3779 · Cited by 461
Medial prefrontal cell activity signaling prediction errors of action values
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007) · Nature Neuroscience · doi:10.1038/nn1890 · Cited by 437
“However, although there are both cells activated by the positive feedback stimuli and those activated by the negative feedback stimuli in the medial PFC, dopamine cells uniformly increase their firing to positive prediction errors of rewar”
BackgroundOldham, S., Murawski, C., Fornito, A., Youssef, G., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 425
Role of brain dopamine in food reward and reinforcement
Wise, R. A. (2006) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2006.1854 · Cited by 437
“When visual or auditory cues, or both, predicted reward with 100% certainty, the responses of dopamine neurons shifted from the reward itself (as defined above) to the reward-predictive environmental stimuli (Schultz 1986; Romo & Schultz 1990; Ljungberg et al. 1991, 1992; Schultz et al. 1993; Fiorillo et al. 2003).”
BackgroundMechanisms of motivation–cognition interaction: challenges and opportunities
Braver, T. S., Krug, M. K., Chiew, K. S., Kool, W., Westbrook, J. A., Clement, N. J., Adcock, R. A., Barch, D. M., Botvinick, M. M., Carver, C. S., et al. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0300-0 · Cited by 391
Wise, R. A., & Robble, M. A. (2020) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-103337 · Cited by 404
Influence of Reward Delays on Responses of Dopamine Neurons
Kobayashi, S., & Schultz, W. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 430
“Dopamine responses to conditioned stimuli Previous studies showed that dopamine neurons change their response to conditioned stimuli proportional to magnitude and probability of the associated reward (Fiorillo et al., 2003; Tobler et al., 2005).”
BackgroundResultSelf-citationNeural Circuitry of Reward Prediction Error
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437
“Also see Fiorillo et al. (2003) and Matsumoto & Hikosaka (2009a,b).”
Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression
Stephan, K. E., Manjaly, Z. M., Mathys, C. D., Weber, L. A. E., Paliwal, S., Gard, T., Tittgemeyer, M., Fleming, S. M., Haker, H., Seth, A. K., et al. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00550 · Cited by 437
“…studies in humans and animals indicate that one of the computational quantities encoded by variations in dopamine release is uncertainty (inverse precision) (Fiorillo et al., 2003; de Lafuente and Romo, 2011; Hart et al., 2015; Schwartenbeck et al., 2015a; Tomassini et al., 2016).”
BackgroundREVIEW: Acute withdrawal, protracted abstinence and negative affect in alcoholism: are they linked?
Heilig, M., Egli, M., Crabbe, J. C., & Becker, H. C. (2010) · Addiction Biology · doi:10.1111/j.1369-1600.2009.00194.x · Cited by 435
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
“The amplitude of inhibitions was small, consistent with the fact that inhibitory responses can be difficult to detect and quantify in neurons that already have a low baseline rate of activity, as noted previously in primate studies (Fiorillo et al., 2003; Morris et al., 2004).”
BackgroundResult
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.