Dopamine and performance in a reinforcement learning task: evidence from Parkinson’s disease
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ashburner, J., & Friston, K. J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.02.018 · Cited by 7,507 · cited 3×
“Another subject was excluded from the imaging analysis due to an incidental finding of abnormally large ventricles, compromising normalization of this data set to a standard coordinate space.”
InfluentialPredictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 1×
“Dopamine is strongly implicated in reward signalling, playing a central role in reward learning in animals (Wise and Rompre, 1989; Schultz et al., 1997; Schultz, 1998; Wise, 2004; Bayer and Glimcher, 2005) and humans (Pessiglione et al., 2006).”
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Inverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673 · cited 2×
“…Fuster, 2001; Stuss and Knight, 2002) and enhancing dopamine signalling will have had an effect on these processes (Sawaguchi, 2001; Wang et al., 2004; Gibbs and D’Esposito, 2005; Cools and D’Esposito, 2011), one could argue that the poor generalization in the OFF group could stem from the delay.”
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 2×
“After every trial t4 0 the value of the chosen stimulus (e.g. ‘A’) was updated according to the rule QA(t + 1) = QA(t) + (t).”
Self-citationIkemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Bayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423
D1 Dopamine Receptors in Prefrontal Cortex: Involvement in Working Memory
Sawaguchi, T., & Goldman-Rakic, P. S. (1991) · Science · doi:10.1126/science.1825731 · Cited by 1,364
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 3×
“Dopamine is strongly implicated in reward signalling, playing a central role in reward learning in animals (Wise and Rompre, 1989; Schultz et al., 1997; Schultz, 1998; Wise, 2004; Bayer and Glimcher, 2005) and humans (Pessiglione et al., 2006).”
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165
Self-citationTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 3×
“…Parkinson et al., 2002; Day and Carelli, 2007) as well as in motivational engagement and vigour (Ahlenius et al., 1977; Beninger and Phillips, 1981; Berridge and Robinson, 1998; McClure et al., 2003; Niv, 2007; Niv et al., 2007; Bardgett et al., 2009; Lex and Hauber, 2010; Boureau and Dayan, 2011).”
Self-citationThe Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 6×
“Importantly, these other roles remain consistent with the fact that phasic activity of dopamine neurons codes for an appetitive prediction error (McClure et al., 2003).”
InfluentialThe role of dopamine in locomotor activity and learning
Beninger, R. J. (1983) · Brain Research Reviews · doi:10.1016/0165-0173(83)90038-3 · Cited by 848 · cited 1×
“We then performed a small volume correction on the striatal activations in the putamen using an anatomical region of interest defined according to the Talairach Daemon atlas (Lancaster et al., 1997) using the SPM WFU PickAtlas tool (Maldjian et al., 2003).”
Salamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817
Wyvell, C. L., & Berridge, K. C. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-21-08122.2000 · Cited by 764 · cited 1×
“We then performed a small volume correction on the striatal activations in the putamen using an anatomical region of interest defined according to the Talairach Daemon atlas (Lancaster et al., 1997) using the SPM WFU PickAtlas tool (Maldjian et al., 2003).”
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665 · cited 1×
“This region is strongly implicated in valuation (Gottfried et al., 2003; Seymour and McClure, 2008; Boorman et al., 2009; Kable and Glimcher, 2009; Fitzgerald et al., 2010; Plassmann et al., 2010) across a range of experimental manipulations, with mounting evidence pointing to a specific role when subjects have to cho…”
Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation
Mazzoni, P., Hristova, A., & Krakauer, J. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0264-07.2007 · Cited by 557
Steeves, T. D. L., Miyasaki, J., Zurowski, M., Lang, A. E., Pellecchia, G., Van Eimeren, T., Rusjan, P., Houle, S., & Strafella, A. P. (2009) · Brain · doi:10.1093/brain/awp054 · Cited by 502 · cited 1×
“Several studies have also found differences in striatal activations in response to wins and losses when comparing patients with Parkinson’s disease with compulsive behaviour ON medication to patients with idiopathic Parkinson’s disease (Steeves et al., 2009; Voon et al., 2010) or in healthy subjects who were given dop…”
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Self-citationYin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416
Neuronal correlates of goal-based motor selection in the prefrontal cortex.
Matsumoto, K., Suzuki, W., & Tanaka, K. (2003) · Science · doi:10.1126/science.1084204 · Cited by 384
Palmiter, R. D. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1417.003 · Cited by 375 · cited 3×
“Secondly, many aspects of appetitive learning can progress normally in the absence of dopamine, most dramatically in the case of genetically engineered dopamine-deficient mice (Palmiter, 2008).”
Plassmann, H., O'Doherty, J. P., & Rangel, A. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0788-10.2010 · Cited by 356 · cited 1×
“This region is strongly implicated in valuation (Gottfried et al., 2003; Seymour and McClure, 2008; Boorman et al., 2009; Kable and Glimcher, 2009; Fitzgerald et al., 2010; Plassmann et al., 2010) across a range of experimental manipulations, with mounting evidence pointing to a specific role when subjects have to cho…”
Morphine reward in dopamine-deficient mice
Hnasko, T. S., Sotak, B. N., & Palmiter, R. D. (2005) · Nature · doi:10.1038/nature04172 · Cited by 338
Selective D2 receptor actions on the functional circuitry of working memory
Wang, M., Vijayraghavan, S., & Goldman-Rakic, P. S. (2004) · Science · doi:10.1126/science.1091162 · Cited by 319 · cited 2×
“…et al., 1997; Fuster, 2001; Stuss and Knight, 2002) and enhancing dopamine signalling will have had an effect on these processes (Sawaguchi, 2001; Wang et al., 2004; Gibbs and D’Esposito, 2005; Cools and D’Esposito, 2011), one could argue that the poor generalization in the OFF group could stem…”
Neural computations underlying action-based decision making in the human brain
Wunderlich, K., Rangel, A., & O'Doherty, J. P. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0901077106 · Cited by 301 · cited 1×
“We were unable to dissociate whether the neural value correlates were precursors to choice (stimulus values) or the output of the choice process (chosen values) (Wunderlich et al., 2009).”
Self-citationCannon, C. M., & Palmiter, R. D. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-34-10827.2003 · Cited by 280 · cited 1×
“We then performed a small volume correction on the striatal activations in the putamen using an anatomical region of interest defined according to the Talairach Daemon atlas (Lancaster et al., 1997) using the SPM WFU PickAtlas tool (Maldjian et al., 2003).”
Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors
Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H. H., Dolan, R. J., & Hallett, M. (2010) · Neuron · doi:10.1016/j.neuron.2009.12.027 · Cited by 280
Self-citationThe Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects
FitzGerald, T. H. B., Seymour, B., & Dolan, R. J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0717-09.2009 · Cited by 279
Self-citationJocham, G., Klein, T. A., & Ullsperger, M. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3904-10.2011 · Cited by 278
Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards
Robinson, S., Sandstrom, S. M., Denenberg, V. H., & Palmiter, R. D. (2005) · Behavioral Neuroscience · doi:10.1037/0735-7044.119.1.5 · Cited by 247
Dissociating Hippocampal versus Basal Ganglia Contributions to Learning and Transfer
Myers, C. E., Shohamy, D., Gluck, M. A., Grossman, S., Kluger, A., Ferris, S., Golomb, J., Schnirman, G., & Schwartz, R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321208123 · Cited by 243
GIBBS, S. E. B., & D'ESPOSITO, M. (2005) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.5.2.212 · Cited by 145 · cited 2×
“…Fuster, 2001; Stuss and Knight, 2002) and enhancing dopamine signalling will have had an effect on these processes (Sawaguchi, 2001; Wang et al., 2004; Gibbs and D’Esposito, 2005; Cools and D’Esposito, 2011), one could argue that the poor generalization in the OFF group could stem from the delay.”
Habitual versus goal-directed action control in Parkinson disease
de Wit, S., Barker, R. A., Dickinson, A. D., & Cools, R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21514 · Cited by 116
Lex, B., & Hauber, W. (2010) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2009.11.002 · Cited by 61 · cited 3×
“…Parkinson et al., 2002; Day and Carelli, 2007) as well as in motivational engagement and vigour (Ahlenius et al., 1977; Beninger and Phillips, 1981; Berridge and Robinson, 1998; McClure et al., 2003; Niv, 2007; Niv et al., 2007; Bardgett et al., 2009; Lex and Hauber, 2010; Boureau and Dayan, 2011).”
Cited by
Berridge, K. C., & Kringelbach, M. L. (2015) · Neuron · doi:10.1016/j.neuron.2015.02.018 · Cited by 1,730
Berke, J. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0152-y · Cited by 904
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Neuroscience of affect: brain mechanisms of pleasure and displeasure
Berridge, K. C., & Kringelbach, M. L. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.01.017 · Cited by 523
Decision Making and Sequential Sampling from Memory
Shadlen, M. N., & Shohamy, D. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.036 · Cited by 418
Metadata
- Authors
- Shiner, T., Seymour, B., Wunderlich, K., Hill, C., Bhatia, K. P., Dayan, P., & Dolan, R. J.
- Venue
- Brain
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-04-14
- Volume
- 135
- Issue
- 6
- Pages
- 1871-1883
- ISSN
- 0006-8950 (ISSN-L)
- 1460-2156
- OpenAlex
- W2170143116
- Cited by
- 149
- References
- 44
- FWCI
- 7.21
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.