The temporal precision of reward prediction in dopamine neurons
Abstract
Midbrain dopamine neurons are activated when reward is greater than predicted, and this error signal could teach target neurons both the value of reward and when it will occur. We used the dopamine error signal to measure how the expectation of reward was distributed over time. Animals were trained with fixed-duration intervals of 1-16 s between conditioned stimulus onset and reward. In contrast to the weak responses that have been observed after short intervals (1-2 s), activations to reward increased steeply and linearly with the logarithm of the interval. Results with varied stimulus-reward intervals suggest that the neural expectation was substantial after just half an interval had elapsed. Thus, the neural expectation of reward in these experiments was not highly precise and the precision declined sharply with interval duration. The neural precision of expectation appeared to be at least qualitatively similar to the precision of anticipatory licking behavior.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Precision is thought to be signaled by neuromodulators such as dopamine and acetylcholine, depending on the particular inferential hierarchy.
“Precision is thought to be signaled by neuromodulators such as dopamine and acetylcholine, depending on the particular inferential hierarchy (21–23).”
From The Predictive Coding Account of Psychosis (2018) · cites this paper as [21]
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationPredictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
Self-citationDiscrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
Self-citationWhat makes us tick? Functional and neural mechanisms of interval timing
Buhusi, C. V., & Meck, W. H. (2005) · Nature reviews. Neuroscience · doi:10.1038/nrn1764 · Cited by 2,074 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291 · see citing works
Self-citationAdaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · see citing works
Self-citationDopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,103 · see citing works
Self-citationThe neural basis of temporal processing
Mauk, M. D., & Buonomano, D. V. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144247 · Cited by 976 · see citing works
Neuropharmacology of timing and time perception
Meck, W. H. (1996) · Cognitive Brain Research · doi:10.1016/0926-6410(96)00009-2 · Cited by 952 · see citing works
Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes
Matell, M. S., & Meck, W. H. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.06.012 · Cited by 877 · see citing works
The evolution of brain activation during temporal processing
Rao, S. M., Mayer, A. R., & Harrington, D. L. (2001) · Nature Neuroscience · doi:10.1038/85191 · Cited by 827 · see citing works
Toward a neurobiology of temporal cognition: advances and challenges
Gibbon, J., Malapani, C., Dale, C. L., & Gallistel, C. R. (1997) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(97)80005-0 · Cited by 766 · see citing works
Representation of Time by Neurons in the Posterior Parietal Cortex of the Macaque
Leon, M. I., & Shadlen, M. N. (2003) · Neuron · doi:10.1016/s0896-6273(03)00185-5 · Cited by 601 · see citing works
A representation of the hazard rate of elapsed time in macaque area LIP
Janssen, P., & Shadlen, M. N. (2005) · Nature Neuroscience · doi:10.1038/nn1386 · Cited by 601 · see citing works
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574 · see citing works
A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0020 · Cited by 555 · see citing works
Nieder, A., & Miller, E. K. (2003) · Neuron · doi:10.1016/s0896-6273(02)01144-3 · Cited by 514 · see citing works
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429 · see citing works
Neural dynamics of adaptive timing and temporal discrimination during associative learning
Grossberg, S., & Schmajuk, N. A. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90026-9 · Cited by 367 · see citing works
Timing and Neural Encoding of Somatosensory Parametric Working Memory in Macaque Prefrontal Cortex
Brody, C. D. (2003) · Cerebral Cortex · doi:10.1093/cercor/bhg100 · Cited by 350 · see citing works
Scalar expectancy theory and peak-interval timing in humans.
Rakitin, B. C., Gibbon, J., Penney, T. B., Malapani, C., Hinton, S. C., & Meck, W. H. (1998) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.24.1.15 · Cited by 336 · see citing works
Brown, J. W., Bullock, D. H., & Grossberg, S. (1999) · Journal of Neuroscience · doi:10.1523/jneurosci.19-23-10502.1999 · Cited by 302 · see citing works
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298 · see citing works
Attentional modulation in visual cortex depends on task timing
Ghose, G. M., & Maunsell, J. H. R. (2002) · Nature · doi:10.1038/nature01057 · Cited by 297 · see citing works
Suri, R. E., & Schultz, W. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00697-6 · Cited by 294 · see citing works
Self-citationDopamine Neurons Can Represent Context-Dependent Prediction Error
Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., & Hikosaka, O. (2004) · Neuron · doi:10.1016/s0896-6273(03)00869-9 · Cited by 276 · see citing works
Retrospective and prospective coding for predicted reward in the sensory thalamus
Komura, Y., Tamura, R., Uwano, T., Nishijo, H., Kaga, K., & Ono, T. (2001) · Nature · doi:10.1038/35087595 · Cited by 273 · see citing works
Inhibition of climbing fibres is a signal for the extinction of conditioned eyelid responses
Medina, J. F., Nores, W. L., & Mauk, M. D. (2002) · Nature · doi:10.1038/416330a · Cited by 252 · see citing works
A Computational Model of How the Basal Ganglia Produce Sequences
Berns, G. S., & B, T. J. S. A. (1998) · Journal of Cognitive Neuroscience · doi:10.1162/089892998563815 · Cited by 231 · see citing works
Schultz, W., & Romo, R. (1987) · Journal of Neurophysiology · doi:10.1152/jn.1987.57.1.201 · Cited by 184 · see citing works
Self-citationRepresentation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173 · see citing works
PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.
O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007) · Behavioral Neuroscience · doi:10.1037/0735-7044.121.1.31 · Cited by 144 · see citing works
Kacelnik, A., & Abreu, F. B. E. (1998) · Journal of Theoretical Biology · doi:10.1006/jtbi.1998.0763 · Cited by 108 · see citing works
A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior
Contreras-Vidal, J. L., & Schultz, W. (1999) · Journal of Computational Neuroscience · doi:10.1023/a:1008862904946 · Cited by 102 · see citing works
Self-citationTime is a rubberband: neuronal activity in monkey motor cortex in relation to time estimation
Renoult, L., Roux, S., & Alexa Riehle, C. (2006) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2006.04824.x · Cited by 68 · see citing works
Models of Information processing in the Basal Ganglia
Mitchell, I. (1996) · Journal of Chemical Neuroanatomy · doi:10.1016/0891-0618(96)00155-x · Cited by 63 · see citing works
Accuracy of memory for amount in the foraging starling,Sturnus vulgaris
Bateson, M., & Kacelnik, A. (1995) · Animal Behaviour · doi:10.1006/anbe.1995.0257 · Cited by 51 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · see citing works
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · see citing works
Wulf, G., & Lewthwaite, R. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0999-9 · Cited by 1,238 · see citing works
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126 · see citing works
Self-citationDopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036 · see citing works
Self-citationThe Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928 · see citing works
The Predictive Coding Account of Psychosis
Sterzer, P., Adams, R. A., Fletcher, P., Frith, C., Lawrie, S. M., Muckli, L., Petrovic, P., Uhlhaas, P., Voss, M., & Corlett, P. R. (2018) · Biological Psychiatry · doi:10.1016/j.biopsych.2018.05.015 · Cited by 941 · see citing works
The cerebellum: adaptive prediction for movement and cognition
Sokolov, A. A., Miall, R. C., & Ivry, R. B. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.02.005 · Cited by 691 · see citing works
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651 · see citing works
Self-citationDopamine signals for reward value and risk: basic and recent data
Schultz, W. (2010) · Behavioral and Brain Functions · doi:10.1186/1744-9081-6-24 · Cited by 653 · see citing works
Self-citationMidbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548 · see citing works
Dehaene, S., Meyniel, F., Wacongne, C., Wang, L., & Pallier, C. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.019 · Cited by 478 · see citing works
Neural Circuitry of Reward Prediction Error
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437 · see citing works
Rewarded Outcomes Enhance Reactivation of Experience in the Hippocampus
Singer, A. C., & Frank, L. M. (2009) · Neuron · doi:10.1016/j.neuron.2009.11.016 · Cited by 385 · see citing works
Serotonergic neurons signal reward and punishment on multiple timescales
Cohen, J. Y., Amoroso, M. W., & Uchida, N. (2015) · eLife · doi:10.7554/elife.06346 · Cited by 388 · see citing works
The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?
Lazic, S. E. (2010) · BMC Neuroscience · doi:10.1186/1471-2202-11-5 · Cited by 381 · see citing works
The anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340 · see citing works
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289 · see citing works
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236 · see citing works
Neural representation of time in cortico-basal ganglia circuits
Jin, D. Z., Fujii, N., & Graybiel, A. M. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0909881106 · Cited by 246 · see citing works
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219 · see citing works
Dopamine reward prediction errors reflect hidden-state inference across time
Starkweather, C. K., Babayan, B. M., Uchida, N., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4520 · Cited by 206 · see citing works
Dopamine release in the nucleus accumbens core signals perceived saliency
Kutlu, M. G., Zachry, J. E., Melugin, P. R., Cajigas, S. A., Chevee, M. F., Kelly, S. J., Kutlu, B., Tian, L., Siciliano, C. A., & Calipari, E. S. (2021) · Current Biology · doi:10.1016/j.cub.2021.08.052 · Cited by 238 · see citing works
A neural mechanism for sensing and reproducing a time interval
Jazayeri, M., & Shadlen, M. N. (2015) · Current Biology · doi:10.1016/j.cub.2015.08.038 · Cited by 210 · see citing works
Two Dimensions of Value: Dopamine Neurons Represent Reward But Not Aversiveness
Fiorillo, C. D. (2013) · Science · doi:10.1126/science.1238699 · Cited by 203 · see citing works
Self-citationDopamine neurons respond to errors in the prediction of sensory features of expected rewards
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.025 · Cited by 200 · see citing works
Model-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161 · see citing works
Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli
Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4828-09.2010 · Cited by 158 · see citing works
Self-citationThe medial prefrontal cortex shapes dopamine reward prediction errors under state uncertainty
Starkweather, C. K., Gershman, S. J., & Uchida, N. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.036 · Cited by 136 · see citing works
Belief state representation in the dopamine system
Babayan, B. M., Uchida, N., & Gershman, S. J. (2018) · Nature Communications · doi:10.1038/s41467-018-04397-0 · Cited by 101 · see citing works
Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum
Gregorios-Pippas, L., Tobler, P. N., & Schultz, W. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90730.2008 · Cited by 101 · see citing works
Self-citationFiorillo, C. D., Song, M. R., & Yun, S. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3883-12.2013 · Cited by 98 · see citing works
Self-citationDiversity and Homogeneity in Responses of Midbrain Dopamine Neurons
Fiorillo, C. D., Yun, S. R., & Song, M. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3886-12.2013 · Cited by 99 · see citing works
Self-citationTransient activation of midbrain dopamine neurons by reward risk
Fiorillo, C. D. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.09.037 · Cited by 90 · see citing works
Self-citationPhasic dopamine signals: from subjective reward value to formal economic utility
Schultz, W., Carelli, R. M., & Wightman, R. M. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.09.006 · Cited by 88 · see citing works
Dopamine neurons learn to encode the long-term value of multiple future rewards
Enomoto, K., Matsumoto, N., Nakai, S., Satoh, T., Sato, T. K., Ueda, Y., Inokawa, H., Haruno, M., & Kimura, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014457108 · Cited by 83 · see citing works
Time representation in reinforcement learning models of the basal ganglia
Gershman, S. J., Moustafa, A. A., & Ludvig, E. A. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2013.00194 · Cited by 81 · see citing works
Dopamine neuron ensembles signal the content of sensory prediction errors
Stalnaker, T. A., Howard, J. D., Takahashi, Y. K., Gershman, S. J., Kahnt, T., & Schoenbaum, G. (2019) · eLife · doi:10.7554/elife.49315 · Cited by 74 · see citing works
Dissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75 · see citing works
Neural control of dopamine neurotransmission: implications for reinforcement learning
Aggarwal, M., Hyland, B. I., & Wickens, J. R. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08055.x · Cited by 32 · see citing works
Trempler, I., Bürkner, P.-C., El-Sourani, N., Binder, E., Reker, P., Fink, G. R., & Schubotz, R. I. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116674 · Cited by 6 · see citing works
- Authors
- Fiorillo, C. D., Newsome, W. T., & Schultz, W.
- Affiliations
- Stanford University (US)
- University of Cambridge (GB)
- Corresponding author
- Fiorillo, Christopher D
- Venue
- Nature Neuroscience
- Published
- 2008-07-27
- Volume
- 11
- Issue
- 8
- Pages
- 966-973
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2099119340
- Cited by
- 310
- References
- 39
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 18 · 2023: 10 · 2022: 16 · 2021: 14 · 2020: 22 · 2019: 23 · 2018: 19 · 2017: 17 · 2016: 15 · 2015: 18 · 2014: 18 · 2013: 22 · 2012: 17
- FWCI
- 9.66
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Howard Hughes Medical Institute
- Wellcome Trust (095495)
- National Eye Institute (EY 05603)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- dopaminergic neurons, reward prediction error, temporal precision, reward anticipation, midbrain dopamine, fixed-interval schedules
- MeSH terms
- Animals, Conditioning, Psychological, Female, Forecasting, Macaca fascicularis, Macaca mulatta, Mesencephalon, Photic Stimulation, Random Allocation, Reward, Anticipation, Psychological, Dopaminergic Neurons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
