Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P , being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“for exteroception this may involve cholingeric neurotransmission via attention (Yu and Dayan, 2005); for interoception, proprioception, and value-learning, prediction error precision is suggested to be encoded by dopamine”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“it is important to distinguish the phasic diachronic role of dopamine in signaling reward prediction error (Schultz and Dickinson, 2000) from its synchronic role in modulating (or optimizing) the precision of prediction errors by modulating signal-to-noise response properties in neuronal signaling (Fiorillo et al., 2003; Friston, 2009, 2010).”
From An Interoceptive Predictive Coding Model of Conscious Presence (2012) · cites this paper as Fiorillo et al., 2003
In the citing paper
“a cue triggering a behavior that yields a reward 50% of the time is by far the most anxiety-inducing of delivery schedules”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“reward anticipation is a much more powerful mediator of strong addictions than outcome evaluations of the stimulus itself”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
In the citing paper
“dopaminergic surges typically occur before the reward, or more precisely when a cue (e.g., a beep indicating that one can press a lever) signals the reliable delivery of a reward”
From Hypernatural Monitoring: A Social Rehearsal Account of Smartphone Addiction (2018) · cites this paper as Fiorillo et al., 2003
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationDynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Gallistel, C. R. (1990) · Cited by 2,571
Research works printed but not linked: 6
- R. P. N. Rao B. A. Olshausen M. S. Lewicki Eds. Probabilistic Models of the Brain (MIT Press Cambridge MA 2002).
- A. Dickinson Contemporary Animal Learning Theory (Cambridge Univ. Press Cambridge 1980).
- R. A. Rescorla A. R. Wagner in Classical Conditioning II: Current Research and Theory A. H. Black W. S. Prokasy Eds. (Appleton-Century-Crofts New York 1972) pp. 64–69.
- 10.1037/0097-7403.10.1.90 · doi:10.1037/0097-7403.10.1.90 · Paper page
- 10.1037/0033-295x.107.2.289 · doi:10.1037/0033-295x.107.2.289 · Paper page
- 10.1037/0033-295x.108.1.96 · doi:10.1037/0033-295x.108.1.96 · Paper page
Cited by
Items 501–600 of 1,007
Camara, E., Kramer, U. M., Cunillera, T., Marco-Pallares, J., Cucurell, D., Nager, W., Mestres-Misse, A., Bauer, P., Schule, R., Schols, L., et al. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhp263 · Cited by 84
“For example, reward anticipation has been shown to rely more on tonic dopaminergic activity (Fiorillo et al. 2003), whereas the processing of unexpected rewards is thought to be related to phasic dopaminergic activity (Schultz 2002).”
Al–Massadi, O., Nogueiras, R., Diéguez, C., & Girault, J. (2019) · Neuropharmacology · doi:10.1016/j.neuropharm.2019.01.001 · Cited by 91
Functional significance of the dorsolateral prefrontal cortex during exhaustive exercise
Bigliassi, M., & Filho, E. (2022) · Biological Psychology · doi:10.1016/j.biopsycho.2022.108442 · Cited by 93
Different representations of relative and absolute subjective value in the human brain
Grabenhorst, F., & Rolls, E. T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.06.045 · Cited by 99
“In addition, macaque dopamine neurons fire much more when large vs small rewards are given (Fiorillo et al., 2003).”
BackgroundProbabilistic reversal learning is impaired in Parkinson's disease
Peterson, D. A., Elliott, C., Song, D. D., Makeig, S., Sejnowski, T. J., & Poizner, H. (2009) · Neuroscience · doi:10.1016/j.neuroscience.2009.07.033 · Cited by 83
Zeeb, F. D., Li, Z., Fisher, D. C., Zack, M. H., & Fletcher, P. J. (2017) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.170003 · Cited by 85
Motivational control of sign-tracking behaviour: A theoretical framework
Anselme, P. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.03.014 · Cited by 86
Atonal Music: Can Uncertainty Lead to Pleasure?
Mencke, I., Omigie, D., Wald‐Fuhrmann, M., & Brattico, E. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00979 · Cited by 94
Hamilos, A. E., Spedicato, G., Hong, Y., Sun, F., Li, Y., & Assad, J. A. (2021) · eLife · doi:10.7554/elife.62583 · Cited by 79
Zachry, J. E., Kutlu, M. G., Yoon, H. J., Leonard, M. Z., Chevée, M., Patel, D. D., Gaidici, A., Kondev, V., Thibeault, K. C., Bethi, R., et al. (2023) · Neuron · doi:10.1016/j.neuron.2023.11.023 · Cited by 88
Adjusting behavior to changing environmental demands with development
Lourenco, F., & Casey, B. J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.03.003 · Cited by 67
“In an experiment modeled upon previously described nonhuman primate studies examining error-driven reinforcement learning (Cromwell and Schultz, 2003; Fiorillo et al., 2003), Galvan and colleagues manipulated the magnitude of reward outcome and examined the effects across development in dopamine-rich subcortical and p…”
MethodModulation of cue-induced firing of ventral tegmental area dopamine neurons by leptin and ghrelin
van der Plasse, G., van Zessen, R., Luijendijk, M. C. M., Erkan, H., Stuber, G. D., Ramakers, G. M. J., & Adan, R. A. H. (2015) · International Journal of Obesity · doi:10.1038/ijo.2015.131 · Cited by 83
“The recorded DA neurons show differential fi ring between a preferred and non-preferred reward, in line with earlier reports, 9 – 12 and show burst patterns similar to those of DA neurons reported before.”
Learning a new behavioral strategy in the shuttle-box increases prefrontal dopamine
Stark, H., Rothe, T., Wagner, T., & Scheich, H. (2004) · Neuroscience · doi:10.1016/j.neuroscience.2004.02.026 · Cited by 93
Neural signals encoding shifts in beliefs
Schwartenbeck, P., FitzGerald, T. H. B., & Dolan, R. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.10.067 · Cited by 82
Embodied inference: or "I think therefore I am, if I am what I think"
Friston, K. J. (2011) · Cited by 172
Dopamine neurons learn to encode the long-term value of multiple future rewards
Enomoto, K., Matsumoto, N., Nakai, S., Satoh, T., Sato, T. K., Ueda, Y., Inokawa, H., Haruno, M., & Kimura, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014457108 · Cited by 83
The Brain Network of Expectancy and Uncertainty Processing
Catena, A., Perales, J. C., Megías‐Robles, A., Cândido, A., Jara, E., & Maldonado, A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040252 · Cited by 85
“[53] registered the activity of midbrain neurons in monkeys, from the onset of the cue to the expected time of the outcome.”
BackgroundPromises and Perils of Experimentation: The Mutual-Internal-Validity Problem
Lin, H., Werner, K. M., & Inzlicht, M. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620974773 · Cited by 91
Temporal dynamics of reward anticipation in the human brain
Zhang, Y., Li, Q., Wang, Z., Liu, X., & Zheng, Y. (2017) · Biological Psychology · doi:10.1016/j.biopsycho.2017.07.011 · Cited by 85
“Primate electrophysiological studies have revealed that dopamine neurons exhibit a sustain mode of activity which is associated with reward uncertainty (maximal when reward probability = 0.5) and builds its size as the arrival of reward delivery (Fiorillo, Tobler, & Schultz, 2003).”
InfluentialStriatal dopamine release codes uncertainty in pathological gambling
Linnet, J., Mouridsen, K., Peterson, E., Møller, A., Doudet, D. J., & Gjedde, A. (2012) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2012.04.012 · Cited by 82
“…dopamine system distinctly codes uncertainty and expected value of reward (Fiorillo et al., 2003; Preuschoff et al., 2006; Schultz, 2006), and dopaminergic sensitivity toward uncertainty may contribute to the reward properties of gambling independent of monetary gains (Fiorillo et al., 2003).”
BackgroundDynamic shaping of dopamine signals during probabilistic Pavlovian conditioning
Hart, A. S., Clark, J. J., & Phillips, P. E. M. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.07.010 · Cited by 82
“Subsequently, at the time of expected reward delivery phasic dopamine activity correlates with the discrepancy between the expected value and the actual reward delivered (Bayer & Glimcher, 2005; Bayer, Lau, & Glimcher, 2007; Fiorillo, 2011; Fiorillo et al., 2003; Nakahara, Itoh, Kawagoe, Takikawa, & Hikosaka, 2004; Pa…”
Influential“Wanting,” “liking,” and their relation to consciousness.
Anselme, P., & Robinson, M. J. F. (2016) · Journal of Experimental Psychology: Animal Learning and Cognition · doi:10.1037/xan0000090 · Cited by 74
The Neuroscience of Adolescence
Galván, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316106143 · Cited by 90
A quantitative reward prediction error signal in the ventral pallidum
Ottenheimer, D. J., Bari, B. A., Sutlief, E., Fraser, K. M., Kim, T. H., Richard, J. M., Cohen, J. Y., & Janak, P. H. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0688-5 · Cited by 92
Efficient reinforcement learning: computational theories, neuroscience and robotics
Kawato, M., & Samejima, K. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.03.004 · Cited by 94
Monosov, I. E., & Hikosaka, O. (2013) · Nature Neuroscience · doi:10.1038/nn.3398 · Cited by 77
Some Thoughts on the Neurobiology of Stalking
Meloy, Jr., & Fisher, H. (2005) · Journal of Forensic Sciences · doi:10.1520/jfs2004508 · Cited by 82
An Analysis of Decision under Risk in Rats
Constantinople, C. M., Piet, A. T., & Brody, C. D. (2019) · Current Biology · doi:10.1016/j.cub.2019.05.013 · Cited by 87
Topical review: placebo responses and therapeutic responses. How are they related?
Greene, C. S., Goddard, G., Macaluso, G. M., & Mauro, G. (2009) · PubMed · Cited by 92
Di Ciano, P., & Everitt, B. J. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03232.x · Cited by 79
Neural systems analysis of decision making during goal-directed navigation
Penner, M. R., & Mizumori, S. J. Y. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.08.010 · Cited by 95
“neurons in a trained animal respond to rewards only when they are unexpected (Fiorillo et al., 2003; Schultz et al., 1997), as would be the case when the relative cost of a reward changes.”
BackgroundBlanchard, T. C., Strait, C. E., & Hayden, B. Y. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00711.2015 · Cited by 86
van den Bos, R., Houx, B. B., & Spruijt, B. M. (2005) · Biological Psychology · doi:10.1016/j.biopsycho.2005.05.003 · Cited by 92
“to help the individual to engage in behaviour (‘risk-taking’/ ‘attention’) that promotes learning about relevant environmental events (Fiorillo et al., 2003; Schultz, 2004; Shizgal and Arvanitogiannis, 2003; Spruijt et al., 2001).”
InfluentialNeural circuitry of judgment and decision mechanisms
Opriș, I. C., & Bruce, C. J. (2005) · Brain Research Reviews · doi:10.1016/j.brainresrev.2004.11.001 · Cited by 92
“These dopaminergic cells have phasic responses that depend on reward probability and tonic responses signaling reward uncertainty [37].”
BackgroundCorwin, R. L., & Wojnicki, F. H. E. (2009) · Behavioural Pharmacology · doi:10.1097/fbp.0b013e3283313168 · Cited by 85
“Work from others has shown that DA neurons in the ventral tegmental area fire differentially in response to reward probability and uncertainty (Fiorillo et al., 2003).”
Backgroundvan Duuren, E., van der Plasse, G., Lankelma, J., Joosten, R. N. J. M. A., Feenstra, M. G. P., & Pennartz, C. M. A. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0005-09.2009 · Cited by 78
“…output also reaches mesencephalic dopamine cells (Phillipson, 1979; Uylings et al., 2003; Van De Werd and Uylings, 2008), potentially supporting the generation of phasic reward-prediction errors and of more tonic signals representing reward uncertainty (Schultz et al., 1997; Fiorillo et al., 2003).”
BackgroundPathological gambling and the loss of willpower: a neurocognitive perspective
Brevers, D., & Noël, X. (2013) · Socioaffective Neuroscience & Psychology · doi:10.3402/snp.v3i0.21592 · Cited by 84
Frontal theta and beta synchronizations for monetary reward increase visual working memory capacity
Kawasaki, M., & Yamaguchi, Y. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nss027 · Cited by 85
Neurobiological features of binge eating disorder
Balodis, I. M., Grilo, C. M., & Potenza, M. N. (2015) · CNS Spectrums · doi:10.1017/s1092852915000814 · Cited by 79
The Science of Animal Welfare: Understanding What Animals Want
Dawkins, M. S. (2021) · doi:10.1093/oso/9780198848981.001.0001 · Cited by 129
The Cambridge Handbook of Psychology and Economic Behaviour
Lewis, A. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511490118 · Cited by 103
Temporal-Difference Reinforcement Learning with Distributed Representations
Kurth-Nelson, Z. L., & Redish, A. D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007362 · Cited by 92
“[68] report that the magnitude of the dopamine burst at the CS is proportional to the probability of reward-delivery.”
BackgroundReward Prediction Error Computation in the Pedunculopontine Tegmental Nucleus Neurons
KOBAYASHI, Y., & OKADA, K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.003 · Cited by 75
“Recent neurophysiological studies have shown that midbrain dopamine neurons encode the reward prediction error signal (Schultz et al., 1997; Hollerman & Schultz, 1998; Waelti et al., 2001; Fiorillo et al., 2003; Nakahara et al., 2004; Morris et al., 2006) and that the striatum (Hollerman & Schultz, 1998; Hikosaka et a…”
ResultDopamine D2receptors mediate two-odor discrimination and reversal learning in C57BL/6 mice
Kruzich, P. J., & Grandy, D. K. (2004) · BMC Neuroscience · doi:10.1186/1471-2202-5-12 · Cited by 79
“Schultz and colleagues have demonstrated that dopamine cells display consistent tonic firing patterns during maintenance of associative learning tasks [33].”
BackgroundTanaka, S. (2005) · Neuroscience · doi:10.1016/j.neuroscience.2005.08.070 · Cited by 90
Dopamine Modulates Egalitarian Behavior in Humans
Sáez, I., Zhu, L., Set, E., Kayser, A. S., & Hsu, M. (2015) · Current Biology · doi:10.1016/j.cub.2015.01.071 · Cited by 86
“For example, in conditioning tasks, substantial neurophysiological evidence indicates that, in addition to fast phasic dopamine response to expected reward on the order of tens of milliseconds, there exists a slower tonic response to reward risk, defined as the expected variance of reward, that can last up to several…”
BackgroundO’Callaghan, C., Naismith, S. L., Hodges, J. R., Lewis, S. J. G., & Hornberger, M. (2012) · Cortex · doi:10.1016/j.cortex.2012.12.003 · Cited by 79
“…between the expected and actual outcomes of a reinforcer, has been directly shown to covary with phasic activation of dopamine neurons in monkey neuronal-recording studies (Fiorillo et al., 2003), and disruption to this learning mechanism is thought to contribute to impulsive behaviours.”
Dissecting motivational circuitry to understand substance abuse
Wheeler, R. A., & Carelli, R. M. (2008) · Neuropharmacology · doi:10.1016/j.neuropharm.2008.06.028 · Cited by 85
Choice-correlated activity fluctuations underlie learning of neuronal category representation
Engel, T. A., Chaisangmongkon, W., Freedman, D. J., & Wang, X.-J. (2015) · Nature Communications · doi:10.1038/ncomms7454 · Cited by 81
Effects of Prefrontal Cortical Inactivation on Neural Activity in the Ventral Tegmental Area
Jo, Y. S., Lee, J., & Mizumori, S. J. Y. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.0118-13.2013 · Cited by 75
“…conditioning paradigms to understand the functional roles of DAergic error signals in reinforcement learning (Montague et al., 1996; Schultz et al., 1997; Fiorillo et al., 2003; Pan et al., 2005; Day et al., 2007; Roesch et al., 2007; Clark et al., 2010; Takahashi et al., 2011; Cohen et al., 2012).”
BackgroundThe uncertainty processing theory of motivation
Anselme, P. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2009.12.020 · Cited by 92
“Uncertainty would then play a role in reinforcement [81], though indirectly.”
Subjective neuronal coding of reward: temporal value discounting and risk
Schultz, W. (2010) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07282.x · Cited by 74
“These latter risk signals have longer latencies than the orbitofrontal risk signal and occur in brain structures that receive dopamine afferents, possibly reflecting input from the similarly slow dopamine risk signal (Fiorillo et al., 2003.”
BackgroundInfluentialSelf-citationBehavioral models of managerial decision-making
Goldfarb, A., Ho, T., Amaldoss, W., Brown, A. L., Chen, Y., Cui, T. H., Galasso, A., Hossain, T., Hsu, M., Lim, N., et al. (2012) · Marketing Letters · doi:10.1007/s11002-012-9183-4 · Cited by 83
“This includes a number of recent papers implicating such dopaminergic regions in decisions under risk and uncertainty (e.g., Fiorillo et al. 2003).”
Perceptual Classification in a Rapidly Changing Environment
Summerfield, C., Behrens, T. E., & Koechlin, E. (2011) · Neuron · doi:10.1016/j.neuron.2011.06.022 · Cited by 75
“Indeed, dopaminoceptive neurons of the striatum are known to encode uncertainty associated with an economic decision, in addition to its value (Bunzeck et al., 2010; Fiorillo et al., 2003; Tobler et al., 2007); our results imply that this may extend to situations where the uncertainty pertains to the category of the s…”
Neuroelectric Signatures of Reward Learning and Decision-Making in the Human Nucleus Accumbens
Cohen, M. X., Axmacher, N., Lenartz, D., Elger, C. E., Sturm, V., & Schlaepfer, T. E. (2008) · Neuropsychopharmacology · doi:10.1038/npp.2008.222 · Cited by 78
“First, cue-related spiking activity in midbrain dopamine neurons (Fiorillo et al, 2003), and the BOLD response in the striatum (Preuschoff et al, 2006), increases for increasingly predictive cues, which is the exact opposite pattern of results to what we found.”
Resultvan der Meulen, J. A. J., Joosten, R. N. J. M. A., de Bruin, J. P. C., & Feenstra, M. G. P. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl057 · Cited by 75
Hämmerer, D., & Eppinger, B. (2012) · Developmental Psychology · doi:10.1037/a0027342 · Cited by 87
Distributional Reinforcement Learning in the Brain
Lowet, A. S., Qiao, Z., Matias, S., Drugowitsch, J., & Uchida, N. (2020) · Trends in Neurosciences · doi:10.1016/j.tins.2020.09.004 · Cited by 95
Adriani, W., & Laviola, G. (2006) · BMC Neuroscience · doi:10.1186/1471-2202-7-52 · Cited by 79
“Such kind of contrasts have been shown to increase dopaminergic activity in the midbrain (Fiorillo et al., 2003), and may support risk-taking behaviour (Van den Bos et al., 2005).”
BackgroundReward‐related activity in the human motor cortex
Kapogiannis, D., Campion, P., Grafman, J., & Wassermann, E. M. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06147.x · Cited by 70
“VTA cells are thought to fire en masse in response to reward-related stimuli (Schultz, 2001; Fiorillo et al., 2003) and VTA projections to M1 and PFC are thought to act in parallel (Berger et al., 1988; Gaspar et al., 1992).”
BackgroundResultInfluentialDopaminergic and non-dopaminergic value systems in conditioning and outcome-specific revaluation
Dranias, M. R., Grossberg, S., & Bullock, D. H. (2008) · Brain Research · doi:10.1016/j.brainres.2008.07.013 · Cited by 79
“…of its prediction of the appearance of bursts at a full range of intermediate times between CS and US during learning, they (Niv et al., 2005) recently used the same TD property to argue that the dopaminergic “uncertainty response” reported by Fiorillo et al. (2003) must be an averaging artifact.”
MethodLong, N. M., Lee, H., & Kuhl, B. A. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1850-16.2016 · Cited by 77
Understanding the Placebo Effect: Contributions from Neuroimaging
Lidstone, S. C., & Stoessl, A. J. (2007) · Molecular Imaging and Biology · doi:10.1007/s11307-007-0086-3 · Cited by 82
“Dopamine neurons also demonstrate a more sustained, tonic firing pattern during the reward expectation period [57], the magnitude of which varies with the probability that the reward will occur: neuron activity is maximal at p = 0.”
BackgroundErroneous and correct actions have a different affective valence: Evidence from ERPs.
Aarts, K., De Houwer, J., & Pourtois, G. (2013) · Emotion · doi:10.1037/a0032808 · Cited by 67
Dopamine, prediction error and associative learning: A model-based account
Smith, A., Li, M., Becker, S., & Kapur, S. (2006) · Network: Computation in Neural Systems · doi:10.1080/09548980500361624 · Cited by 73
“In general, amphetamine disrupts LI (Weiner et al. 1988), while haloperidol (a dopamine antagonist) facilitates LI (Weiner & Feldon 1987; Weiner et al. 1987).”
BackgroundEconomic Choices Reveal Probability Distortion in Macaque Monkeys
Stauffer, W. R., Lak, A., Bossaerts, P. L., & Schultz, W. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3653-14.2015 · Cited by 80
Self-citationSocial modulation of decision-making: a cross-species review
van den Bos, R., Jolles, J. W., & Homberg, J. R. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00301 · Cited by 87
“…observed effects of stress and cortisol on neural structures, their brain seems to be in a “exploratory mode,” i.e., a decreased prefrontal activity and an increased ventral striatal (dopaminergic) activity in which risk-taking, as part of exploratory behavior, is included (Fiorillo et al., 2003).”
BackgroundDopamine Modulates Risk-Taking as a Function of Baseline Sensation-Seeking Trait
Norbury, A., Manohar, S., Rogers, R. D., & Husain, M. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5587-12.2013 · Cited by 68
“While presynaptic D2 autoreceptors negatively regulate phasic DA responses, postsynaptic D2Rs regulate the tonic DA signaling implicated in the representation of risk (Grace, 1991; Fiorillo et al., 2003; Schmitz et al., 2003; Schultz, 2010).”
BackgroundInstrumental vigour in punishment and reward
Dayan, P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08026.x · Cited by 72
“…axis) shows the initial value V*({1,0}) [as it is commonly found, at least in the phasic activity of dopamine cells, that prediction errors at the start of trials are reported relative to a 0 baseline; e.g. (Roesch et al., 2007; Morris et al., 2006; Fiorillo et al., 2003; Tobler et al., 2005)].”
BackgroundSeverity of gambling is associated with severity of depressive symptoms in pathological gamblers
Rømer Thomsen, K., Callesen, M. B., Linnet, J., Kringelbach, M. L., & Møller, A. (2009) · Behavioural Pharmacology · doi:10.1097/fbp.0b013e3283305e7a · Cited by 70
“In addition to the well-established influence of dopamine in substance use disorders, recent findings support a similar dopaminergic involvement in pathological gambling (Laruelle et al., 1995; Volkow et al., 1999, 2005; Betz et al., 2000; Drevets et al., 2001; Fiorillo et al., 2003; Shaffer et al., 2004; Kalivas and…”
ResultMorita, K., Morishima, M., Sakai, K., & Kawaguchi, Y. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.04.009 · Cited by 78
Puryear, C. B., Kim, M. J., & Mizumori, S. J. Y. (2010) · Behavioral Neuroscience · doi:10.1037/a0018865 · Cited by 71
From controlled to compulsive drug-taking: The role of the habenula in addiction
Mathis, V., & Kenny, P. J. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.06.018 · Cited by 75
Khan, M., Takasaki, H., & Komatsu, S. (2005) · Journal of Proteome Research · doi:10.1021/pr0501160 · Cited by 74
Behavioral, but not reward, risk modulates activation of prefrontal, parietal, and insular cortices
Huettel, S. A. (2006) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.6.2.141 · Cited by 83
“This finding was extended by Fiorillo and colleagues, who demonstrated that the activity of such neurons scales directly with the probability that a stimulus predicts future rewards ( Fiorillo, Tobler, & Schultz, 2003 ).”
BackgroundMurray, D., & Stoessl, A. J. (2013) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2013.07.009 · Cited by 84
van Zessen, R., van der Plasse, G., & Adan, R. A. H. (2012) · Proceedings of the Nutrition Society · doi:10.1017/s0029665112000614 · Cited by 70
“When a cue is coupled to a reward over multiple sessions, animals learn that the cue predicts the reward and DA burst firing shifts to the cue onset (11,17,18) .”
MacDonald, C. J., Cheng, R., & Meck, W. H. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00010 · Cited by 74
“…and diffuse input from many brain regions, especially the midbrain dopaminergic area, which is sensitive to both the expected time of reward and changes in reward probability—two conditions that characterize each of our experimental contexts (Hollerman and Schultz, 1998; Fiorillo et al., 2003).”
BackgroundFeedback-related negativity observed in rodent anterior cingulate cortex
Warren, C. M., Hyman, J. M., Seamans, J. K., & Holroyd, C. B. (2014) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2014.08.008 · Cited by 76
“…prediction that FRN amplitude should be sensitive to the interaction of outcome probability with valence, comparable to how phasic increases and decreases in firing of dopaminergic neurons in the ventral tegmental area scale to outcome probability (Schultz et al., 1997; Fiorillo et al., 2003).”
At-risk for pathological gambling: imaging neural reward processing under chronic dopamine agonists
Abler, B., Hahlbrock, R., Unrath, A., Grön, G., & Kassubek, J. (2009) · Brain · doi:10.1093/brain/awp170 · Cited by 77
“…and orbitofrontal cortex. fMRI activation of these areas can be described as a function of reward probability and prediction error as shown for firing rates of dopamine neurons in animal experiments (Fiorillo et al., 2003; Schultz, 2007) and for fMRI signals in human subjects (Abler et al., 2006).”
BackgroundInfluentialFeature-specific prediction errors and surprise across macaque fronto-striatal circuits
Oemisch, M., Westendorff, S., Azimi, M., Hassani, S. A., Ardid, S., Tiesinga, P. H. E., & Womelsdorf, T. (2019) · Nature Communications · doi:10.1038/s41467-018-08184-9 · Cited by 118
Hormonal and Genetic Influences on Processing Reward and Social Information
CALDÚ, X., & DREHER, J. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1412.007 · Cited by 68
Zellner, M. R., & Ranaldi, R. (2009) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.11.011 · Cited by 79
“In addition, cue-evoked firing increases as the probability of reward signified by the cue increases (Fiorillo et al., 2003).”
Toward a Science of Learning Games
Howard‐Jones, P. A., Demetriou, S., Bogacz, R., Yoo, J. H., & Leonards, U. (2011) · Mind Brain and Education · doi:10.1111/j.1751-228x.2011.01108.x · Cited by 89
“In a study of the primate brain (Fiorillo, Tobler, & Schultz, 2003), researchers created a certain reward by frequently presenting a stimulus with a subsequent reward on 100% of occasions.”
BackgroundHalladay, L. R., & Blair, H. T. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00656.2014 · Cited by 79
Opponent Brain Systems for Reward and Punishment Learning
Palminteri, S., & Pessiglione, M. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805308-9.00023-3 · Cited by 81
Alderson, H. L., Latimer, M. P., Blaha, C. D., Phillips, A. G., & Winn, P. (2004) · Neuroscience · doi:10.1016/j.neuroscience.2004.02.015 · Cited by 75
Dopamine, motivation, and the evolutionary significance of gambling-like behaviour
Anselme, P. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2013.07.039 · Cited by 74
Reinforcement learning for a CPG-driven biped robot
Mori, T., Nakamura, Y., Sato, M.-A., & Ishii, S. (2004) · National Conference on Artificial Intelligence · Cited by 70
May, P. J., McHaffie, J. G., Stanford, T. R., Jiang, H., Costello, M. G., Coizet, V., Hayes, L. M., Haber, S. N., & Redgrave, P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06596.x · Cited by 67
Cholinergic modulation of dopamine pathways through nicotinic acetylcholine receptors
de Kloet, S. F., Mansvelder, H. D., & De Vries, T. J. (2015) · Biochemical Pharmacology · doi:10.1016/j.bcp.2015.07.014 · Cited by 69
A history of bingeing on fat enhances cocaine seeking and taking.
Puhl, M. D., Cason, A. M., Wojnicki, F. H. E., Corwin, R. L., & Grigson, P. S. (2011) · Behavioral Neuroscience · doi:10.1037/a0025759 · Cited by 66
“Midbrain dopamine neurons respond differentially to certain and uncertain cues associated with food (Fiorillo et al. 2003).”
BackgroundObsessive-Compulsive Spectrum Disorders: a Multidimensional Approach
Stein, D. J., & Löchner, C. (2006) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2006.02.015 · Cited by 81
Cacciapaglia, F., Saddoris, M. P., Wightman, R. M., & Carelli, R. M. (2012) · Neuropharmacology · doi:10.1016/j.neuropharm.2011.12.027 · Cited by 78
“, reward predictive cues) come to elicit brief bursts of dopaminergic neurons, and the magnitude of these conditioned neuronal responses is correlated with the certainty of the reward being predicted (Fiorillo et al., 2003).”
All That Glitters … Dissociating Attention and Outcome Expectancy From Prediction Errors Signals
Roesch, M. R., Calu, D. J., Esber, G. R., & Schoenbaum, G. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00173.2010 · Cited by 68
Independent Controls of Attentional Influences in Primary and Secondary Somatosensory Cortex
Chapman, C. E., & Meftah, E.-M. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00303.2005 · Cited by 67
Eagle, D. M., Noschang, C., d’Angelo, L.-S. C., Noble, C. A., Day, J. O., Dongelmans, M. L., Theobald, D. E., Mar, A. C., Urcelay, G. P., Morein-Zamir, S., et al. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2013.12.040 · Cited by 61
Nair, A. G., Gutierrez‐Arenas, O., Eriksson, O., Vincent, P., & Kotaleski, J. H. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0730-15.2015 · Cited by 71
Genetic influences on the dynamics of pain and affect in fibromyalgia.
Finan, P. H., Zautra, A., Davis, M. C., Lemery‐Chalfant, K., Covault, J., & Tennen, H. A. (2010) · Health Psychology · doi:10.1037/a0018647 · Cited by 74
Gamification of Learning Deactivates the Default Mode Network
Howard‐Jones, P. A., Jay, T., Mason, A., & Jones, H. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01891 · Cited by 78
Metadata
- Authors
- Fiorillo, C. D., Tobler, P. N., & Schultz, W.
- Affiliations
- University of Fribourg (CH)
- Corresponding author
- Fiorillo, Christopher D.
- Venue
- Science
- Published
- 2003-03-21
- Volume
- 299
- Issue
- 5614
- Pages
- 1898-1902
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2158420012
- Cited by
- 2,085
- References
- 21
- Citations by year
- 2026: 41 · 2025: 41 · 2024: 52 · 2023: 75 · 2022: 72 · 2021: 89 · 2020: 187 · 2019: 88 · 2018: 92 · 2017: 81 · 2016: 95 · 2015: 85 · 2014: 113 · 2013: 111 · 2012: 138
- FWCI
- 27.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Funders
- Wellcome Trust (095495)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- dopaminergic neurons, reward probability, prediction error, risk-taking
- MeSH terms
- Animals, Attention, Conditioning, Classical, Discrimination, Psychological, Dopamine, Electrophysiology, Female, Learning, Macaca fascicularis, Mesencephalon, Motivation, Neurons, Photic Stimulation, Probability, Reinforcement, Psychology, Reward, Risk-Taking, Linear Models, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.