Differential, but not opponent, effects of l-DOPA and citalopram on action learning with reward and punishment
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
RATIONALE: Decision-making involves two fundamental axes of control namely valence, spanning reward and punishment, and action, spanning invigoration and inhibition. We recently exploited a go/no-go task whose contingencies explicitly decouple valence and action to show that these axes are inextricably coupled during learning. This results in a disadvantage in learning to go to avoid punishment and in learning to no-go to obtain a reward. The neuromodulators dopamine and serotonin are likely to play a role in these asymmetries: Dopamine signals anticipation of future rewards and is also involved in an invigoration of motor responses leading to reward, but it also arbitrates between different forms of control. Conversely, serotonin is implicated in motor inhibition and punishment processing. OBJECTIVE: To investigate the role of dopamine and serotonin in the interaction between action and valence during learning.Methods We combined computational modeling with pharmacological manipulation in 90 healthy human volunteers, using levodopa and citalopram to affect dopamine and serotonin, respectively. RESULTS: We found that, after administration of levodopa,action learning was less affected by outcome valence when compared with the placebo and citalopram groups. This highlights in this context a predominant effect of levodopa in controlling the balance between different forms of control.Citalopram had distinct effects, increasing participants'tendency to perform active responses independent of outcome valence, consistent with a role in decreasing motor inhibition. CONCLUSIONS: Our findings highlight the rich complexities of the roles played by dopamine and serotonin during instrumental learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gray, J. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00013066 · Cited by 4,477
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 1×
“…the operation of prefrontal processes such as working memory or rule learning (Clatworthy et al. 2009; Cools and D’Esposito 2011; Mehta et al. 2005; Williams and Goldman-Rakic 1995), perhaps reducing the error in their outputs and thereby increasing their influence on behavior (Daw et al. 2005).”
MethodSelf-citationDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“…Montague et al. 1996; Schultz et al. 1997) in the form of phasic bursts for positive prediction errors and dips below baseline for negative prediction errors (Bayer et al. 2007), to target structures including the striatum (McClure et al. 2003; O’Doherty et al. 2003, 2004; Pessiglione et al. 2006).”
BackgroundSelf-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 4×
“…to a prevalent view in reinforcement learning and decision making, dopamine neurons signal reward prediction error signals (Bayer and Glimcher 2005; Montague et al. 1996; Schultz et al. 1997) in the form of phasic bursts for positive prediction errors and dips below baseline for negative…”
BackgroundInfluentialSelf-citationBy Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationInverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673
Aron, A. R., & Poldrack, R. A. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4682-05.2006 · Cited by 1,630 · cited 2×
“…mechanism by which dopamine may arbitrate between different sorts of control is through recruitment of the prefrontal–subthalamic nucleus pathway (Aron and Poldrack 2006; Fleming et al. 2010) to raise a decision threshold within the basal ganglia and thereby prevent execution of a biased…”
BackgroundModulation of memory fields by dopamine Dl receptors in prefrontal cortex
Williams, G. V., & Goldman-Rakic, P. S. (1995) · Nature · doi:10.1038/376572a0 · Cited by 1,593 · cited 2×
“This function can be achieved, for instance, by facilitating the operation of prefrontal processes such as working memory or rule learning (Clatworthy et al. 2009; Cools and D’Esposito 2011; Mehta et al. 2005; Williams and Goldman-Rakic 1995), perhaps reducing the error in their outputs and thereby increasing their in…”
MethodDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Self-citationTemporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 1×
“…Montague et al. 1996; Schultz et al. 1997) in the form of phasic bursts for positive prediction errors and dips below baseline for negative prediction errors (Bayer et al. 2007), to target structures including the striatum (McClure et al. 2003; O’Doherty et al. 2003, 2004; Pessiglione et al. 2006).”
BackgroundSelf-citationMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 6×
“…motivated behavior (Berridge and Robinson 1998; Niv et al. 2007; Salamone et al. 2007) and to support instrumental learning (Daw and Doya 2006; Frank et al. 2004; Wickens et al. 2007) through model-free reward prediction errors (Bayer and Glimcher 2005; Morris et al. 2006; Schultz et al. 1997).”
BackgroundInfluentialTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
Self-citationA selective role for dopamine in stimulus-reward learning
Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., Akers, C. A., Clinton, S. M., Phillips, P. E. M., & Akil, H. (2011) · Nature · doi:10.1038/nature09588 · Cited by 1,026
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 2×
“2007), to target structures including the striatum (McClure et al. 2003; O’Doherty et al. 2003, 2004; Pessiglione et al. 2006).”
BackgroundHold your horses: a dynamic computational role for the subthalamic nucleus in decision making
Frank, M. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.006 · Cited by 706
Self-citationHikida, T., Kimura, K., Wada, N., Funabiki, K., & Nakanishi, S. (2010) · Neuron · doi:10.1016/j.neuron.2010.05.011 · Cited by 592 · cited 2×
“In the striatum, increases of dopaminewhen an unexpected reward is obtained reinforce the direct pathway and generate go choices, while dips in dopamine levels when an unexpected punishment is obtained reinforce the indirect pathway and generate no-go choices (Frank et al. 2007; Frank et al. 2004; Hikida et al. 2010;…”
BackgroundParkinson, J. A., Olmstead, M. C., Burns, L. H., Robbins, T. W., & Everitt, B. J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-06-02401.1999 · Cited by 555
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · cited 4×
“…motivated behavior (Berridge and Robinson 1998; Niv et al. 2007; Salamone et al. 2007) and to support instrumental learning (Daw and Doya 2006; Frank et al. 2004; Wickens et al. 2007) through model-free reward prediction errors (Bayer and Glimcher 2005; Morris et al. 2006; Schultz et al. 1997).”
BackgroundInfluentialDYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Self-citationSchönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381
Self-citationPalmiter, R. D. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1417.003 · Cited by 375 · cited 2×
“It is known that dopamine depletion results in decreased motor activity and decreased motivated behavior (Palmiter 2008; Ungerstedt 1971), along with decreased vigor or motivation to work for rewards in demanding reinforcement schedules (Niv et al.”
BackgroundOtto (2013) · doi:10.1177/0956797612463080 · Cited by 352
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298
Dopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294 · cited 2×
“A related finding is the observation that levodopa increases the degree to which young healthy participants employ model-based, as opposed to model-free, control in a two-step choice task (Wunderlich et al. 2012), a task sensitive to manipulations of working memory load (Otto et al.”
BackgroundSelf-citationSerotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252 · cited 3×
“…because it is often found that subjects have a tendency either to repeat or avoid doing the same action twice (Lau and Glimcher 2005; Schoenberg et al. 2007; Rutledge et al. 2010) and dietary tryptophan depletion results in increased value independent of choice perseveration (Seymour et al. 2012).”
BackgroundInfluentialSelf-citationClatworthy, P. L., Lewis, S. J. G., Brichard, L., Hong, Y. T., Izquierdo, D., Clark, L., Cools, R., Aigbirhio, F. I., Baron, J.-C., Fryer, T. D., et al. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3266-08.2009 · Cited by 229
Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216
Self-citationTwenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
Self-citationWICKENS, J. R., BUDD, C. S., HYLAND, B. I., & ARBUTHNOTT, G. W. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.016 · Cited by 165 · cited 3×
“…motivated behavior (Berridge and Robinson 1998; Niv et al. 2007; Salamone et al. 2007) and to support instrumental learning (Daw and Doya 2006; Frank et al. 2004; Wickens et al. 2007) through model-free reward prediction errors (Bayer and Glimcher 2005; Morris et al. 2006; Schultz et al. 1997).”
BackgroundBidirectional modulation of goal-directed actions by prefrontal cortical dopamine
Hitchcott, P. K., Quinn, J. J., & Taylor, J. R. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm010 · Cited by 143 · cited 3×
“However, if the predominant effects of the drugs lay elsewhere, for instance, in the modulation of the contribution of prefrontal cortex to control (Hitchcott et al. 2007), then other effects might arise, such as a decrease in the extent of suboptimal behavior.”
BackgroundInfluentialActivation of dorsal raphe serotonin neurons is necessary for waiting for delayed rewards
Miyazaki, K. W., Miyazaki, K., & Doya, K. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0915-12.2012 · Cited by 106
Reliance on habits at the expense of goal-directed control following dopamine precursor depletion
de Wit, S., Standing, H. R., DeVito, E. E., Robinson, O. J., Ridderinkhof, K. R., Robbins, T. W., & Sahakian, B. J. (2012) · Psychopharmacology · doi:10.1007/s00213-011-2563-2 · Cited by 105
Cited by
Action versus valence in decision making
Guitart-Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.01.003 · Cited by 270
Self-citationModeling Trait Anxiety: From Computational Processes to Personality
Raymond, J. G., Steele, J. D., & Seriès, P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00001 · Cited by 203
“(196)], this approach could help elucidate the relative contributions of these neuro-transmitter systems to expected alterations in avoidance learning associated with trait anxiety.”
BackgroundModeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning
Mkrtchian, A., Aylward, J., Dayan, P., Roiser, J. P., & Robinson, O. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.01.017 · Cited by 151
“The most parsimonious model (standard 1 2 approach-avoid 1 2 learning rates; Table 1; Figure 3E; Methods and Materials) is an adapted Rescorla-Wagner model (21) identical to the winning model in prior studies of healthy individuals (8,10), with the exception that there are separate Pavlovian approach, avoid, and learn…”
Self-citationKühn, S., Schubert, F., Mekle, R., Wenger, E., Ittermann, B., Lindenberger, U., & Gallinat, J. (2016) · Brain Structure and Function · doi:10.1007/s00429-015-1057-0 · Cited by 59
Scholl, J., Kolling, N., Nelissen, N., Browning, M., Rushworth, M. F. S., & Harmer, C. J. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000756 · Cited by 58
Metadata
- Authors
- Guitart-Masip, M., Economides, M., Huys, Q. J. M., Frank, M. J., Chowdhury, R., Duzel, E., Dayan, P., & Dolan, R. J.
- Venue
- Psychopharmacology
- Published
- 2013-11-15
- Volume
- 231
- Issue
- 5
- Pages
- 955-966
- ISSN
- 0033-3158 (ISSN-L)
- 1432-2072
- OpenAlex
- W2129123617
- Cited by
- 107
- References
- 36
- FWCI
- 2.91
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.