The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Phasic Dopamine Release in the Rat Nucleus Accumbens Symmetrically Encodes a Reward Prediction Error Term
Open accessAbstract
Making predictions about the rewards associated with environmental stimuli and updating those predictions through feedback is an essential aspect of adaptive behavior. Theorists have argued that dopamine encodes a reward prediction error (RPE) signal that is used in such a reinforcement learning process. Recent work with fMRI has demonstrated that the BOLD signal in dopaminergic target areas meets both necessary and sufficient conditions of an axiomatic model of the RPE hypothesis. However, there has been no direct evidence that dopamine release itself also meets necessary and sufficient criteria for encoding an RPE signal. Further, the fact that dopamine neurons have low tonic firing rates that yield a limited dynamic range for encoding negative RPEs has led to significant debate about whether positive and negative prediction errors are encoded on a similar scale. To address both of these issues, we used fast-scan cyclic voltammetry to measure reward-evoked dopamine release at carbon fiber electrodes chronically implanted in the nucleus accumbens core of rats trained on a probabilistic decision-making task. We demonstrate that dopamine concentrations transmit a bidirectional RPE signal with symmetrical encoding of positive and negative RPEs. Our findings strengthen the case that changes in dopamine concentration alone are sufficient to encode the full range of RPEs necessary for reinforcement learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Rat Brain in Stereotaxic Coordinates
(1982) · Elsevier eBooks · doi:10.1016/c2009-0-63235-9 · Cited by 51,963
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · cited 1×
“A large number of studies have now correlated dopaminergic neural activity with the reward prediction error (RPE) term in temporal difference models of learning (Sutton, 1988, Montague et al., 1996, Schultz et al., 1997, Waelti et al., 2001, Flagel et al., 2011, Steinberg et al., 2013).”
BackgroundDiscrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · cited 2×
“A large number of studies have now correlated dopaminergic neural activity with the reward prediction error (RPE) term in temporal difference models of learning (Sutton, 1988, Montague et al., 1996, Schultz et al., 1997, Waelti et al., 2001, Flagel et al., 2011, Steinberg et al., 2013).”
BackgroundResultA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“…dopamine concentration conferred by uptake (Wightman et al., 1988), diffusion (Venton et al., 2003) and short-term elastic modulation of release probability (Montague et al., 2004), as well as the impact of dopamine release and uptake on dopamine receptor occupancy (Dreyer et al., 2010).”
BackgroundMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Self-citationA selective role for dopamine in stimulus-reward learning
Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., Akers, C. A., Clinton, S. M., Phillips, P. E. M., & Akil, H. (2011) · Nature · doi:10.1038/nature09588 · Cited by 1,026 · cited 2×
“A large number of studies have now correlated dopaminergic neural activity with the reward prediction error (RPE) term in temporal difference models of learning (Sutton, 1988, Montague et al., 1996, Schultz et al., 1997, Waelti et al., 2001, Flagel et al., 2011, Steinberg et al., 2013).”
BackgroundResultSelf-citationDopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
Self-citationA causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 869
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565
Self-citationMidbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · cited 1×
“We then Nissl stained mounted sections and located lesion sites using an adult rat brain atlas (Paxinos and Watson, 2005).”
MethodEfflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain
Garris, P., Ciolkowski, E., Pastore, P., & Wightman, R. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-10-06084.1994 · Cited by 475
Influence of Phasic and Tonic Dopamine Release on Receptor Activation
Dreyer, J. K., Herrik, K. F., Berg, R. W., & Hounsgaard, J. D. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1894-10.2010 · Cited by 423
Real-time characterization of dopamine overflow and uptake in the rat striatum
Wightman, R. M., Amatorh, C., Engstrom, R. C., Hale, P. D., Kristensen, E. W., Kuhr, W. G., & May, L. J. (1988) · Neuroscience · doi:10.1016/0306-4522(88)90255-2 · Cited by 418
Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals
Clark, J. J., Sandberg, S. G., Wanat, M. J., Gan, J. O., Horne, E. A., Hart, A. S., Akers, C. A., Parker, J. G., Willuhn, I., Martinez, V., et al. (2010) · Nature Methods · doi:10.1038/nmeth.1412 · Cited by 378 · cited 1×
“…dopamine concentration conferred by uptake (Wightman et al., 1988), diffusion (Venton et al., 2003) and short-term elastic modulation of release probability (Montague et al., 2004), as well as the impact of dopamine release and uptake on dopamine receptor occupancy (Dreyer et al., 2010).”
BackgroundSelf-citationVenton, B. J., Zhang, H., Garris, P. A., Phillips, P. E. M., Sulzer, D., & Wightman, R. M. (2003) · Journal of Neurochemistry · doi:10.1046/j.1471-4159.2003.02109.x · Cited by 283
Self-citationTesting the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244
Self-citationCalculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine
Phillips, P. E. M., Walton, M. E., & Jhou, T. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0626-6 · Cited by 237
Self-citationDissociable cost and benefit encoding of future rewards by mesolimbic dopamine
Gan, J. O., Walton, M. E., & Phillips, P. E. M. (2010) · Nature Neuroscience · doi:10.1038/nn.2460 · Cited by 233
Self-citationMultivariate concentration determination using principal component regression with residual analysis
Keithley, R. B., Mark Wightman, R., & Heien, M. L. (2009) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2009.07.002 · Cited by 204
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 187
Self-citationEsber, G. R., & Haselgrove, M. (2011) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2011.0836 · Cited by 178
Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals
Montague, P. R., McClure, S. M., Baldwin, P. R., Phillips, P. E. M., Budygin, E. A., Stuber, G. D., Kilpatrick, M. R., & Wightman, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4279-03.2004 · Cited by 176
Self-citationSub-Second Dopamine Detection in Human Striatum
Kishida, K. T., Sandberg, S. G., Lohrenz, T. M., Comair, Y. G., Sáez, I., Phillips, P. E. M., & Montague, P. R. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023291 · Cited by 137
Self-citationDopamine, Reward Prediction Error, and Economics*
Caplin, A., & Dean, M. R. W. (2008) · The Quarterly Journal of Economics · doi:10.1162/qjec.2008.123.2.663 · Cited by 132
Measuring Beliefs and Rewards: A Neuroeconomic Approach*
Caplin, A., Dean, M. R. W., Glimcher, P. W., & Rutledge, R. B. (2010) · The Quarterly Journal of Economics · doi:10.1162/qjec.2010.125.3.923 · Cited by 76
Self-citation
Cited by
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Berke, J. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0152-y · Cited by 904
Mesolimbic dopamine signals the value of work
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 824
Reinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446
Neural Circuitry of Reward Prediction Error
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437
Metadata
- Authors
- Hart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2014-01-15
- Volume
- 34
- Issue
- 3
- Pages
- 698-704
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2073602092
- Cited by
- 293
- References
- 28
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.