Phasic Mesolimbic Dopamine Signaling Precedes and Predicts Performance of a Self-Initiated Action Sequence Task
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Background Sequential reward-seeking actions are readily learned despite the temporal gap between the earliest (distal) action in the sequence and the reward delivery. Fast dopamine signaling is hypothesized to mediate this form of learning by reporting errors in reward prediction. However, such a role for dopamine release in voluntarily initiated action sequences remains to be demonstrated. Methods Using fast-scan cyclic voltammetry we monitored phasic mesolimbic dopamine release, in real-time, as rats performed a self-initiated sequence of lever presses to earn sucrose rewards. Prior to testing rats received either 0 (n=11), 5 (n=11) or 10 (n=8) days of action sequence training. Results For rats acquiring the action sequence task at test dopamine release was strongly elicited by response-contingent (but unexpected) rewards. With learning, a significant elevation in dopamine release preceded performance of the proximal action, and subsequently came to precede the distal action. This pre-distal dopamine release response was also observed in rats previously trained on the action sequence task, and the amplitude of this signal predicted the latency with which rats completed the action sequence. Importantly, the dopamine response to contingent reward delivery was not observed in rats given extensive pre-training. Pharmacological analysis confirmed that task performance was dopamine dependent. Conclusions These data suggest that phasic mesolimbic dopamine release mediates the influence that rewards exert over the performance of self-paced, sequentially-organized behavior and shed light on how dopamine signaling abnormalities may contribute to disorders of behavioral control.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Parkinson's disease: clinical features and diagnosis
Jankovic, J. (2008) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.2007.131045 · Cited by 5,720
Koob, G. F., & Volkow, N. D. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.110 · Cited by 5,340
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · cited 2×
“Temporal difference models of reinforcement learning assume that learning is regulated by a reward prediction error signal (21, 22, 44-46), and there is now considerable evidence that this reward prediction error is mediated by phasic dopamine (7, 19).”
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · cited 1×
“Studies recording dopamine cell activity have shown that reward presentation elicits a firing burst that shifts to reward-predictive cues (7-9), or discriminative stimuli (10, 11) with training (12).”
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Dissecting components of reward: ‘liking’, ‘wanting’, and learning
Berridge, K. C., Robinson, T. E., & Aldridge, J. W. (2009) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2008.12.014 · Cited by 1,942
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“There have been several recent attempts to integrate the concept that dopamine mediates incentive motivation into the reinforcement learning framework (50-52).”
Subsecond dopamine release promotes cocaine seeking
Phillips, P. E. M., Stuber, G. D., Heien, M. L. A. V., Wightman, R. M., & Carelli, R. M. (2003) · Nature · doi:10.1038/nature01476 · Cited by 1,113
The Basal Ganglia and Chunking of Action Repertoires
Graybiel, A. M. (1998) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1998.3843 · Cited by 1,042 · cited 1×
“These effects of dopamine receptor antagonism on action sequence performance are consistent with findings from nonhuman primate studies showing that dopamine transmission is preferentially involved during the early stages of action sequence learning (53, 54), when discrete actions are being integrated into sequence-le…”
A selective role for dopamine in stimulus-reward learning
Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., Akers, C. A., Clinton, S. M., Phillips, P. E. M., & Akil, H. (2011) · Nature · doi:10.1038/nature09588 · Cited by 1,026
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · cited 2×
“Temporal difference models of reinforcement learning assume that learning is regulated by a reward prediction error signal (21, 22, 44-46), and there is now considerable evidence that this reward prediction error is mediated by phasic dopamine (7, 19).”
Imaging dopamine's role in drug abuse and addiction
Volkow, N. D., Fowler, J. S., Wang, G. J., Baler, R., & Telang, F. (2009) · Neuropharmacology · doi:10.1016/j.neuropharm.2008.05.022 · Cited by 988
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Dopamine operates as a subsecond modulator of food seeking
Roitman, M. F., Stuber, G. D., Phillips, P. E. M., Wightman, R. M., & Carelli, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3823-03.2004 · Cited by 718 · cited 2×
“These results are generally consistent with the findings of previous studies showing that phasic mesolimbic dopamine signaling transitions from reward delivery to reward-predictive cues during passive Pavlovian learning (7, 13, 14, 39) and discrete–trial, discriminative stimulus-controlled instrumental tasks (10, 11,…”
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710 · cited 2×
“These results are generally consistent with the findings of previous studies showing that phasic mesolimbic dopamine signaling transitions from reward delivery to reward-predictive cues during passive Pavlovian learning (7, 13, 14, 39) and discrete–trial, discriminative stimulus-controlled instrumental tasks (10, 11,…”
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Romo, R., & Schultz, W. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.3.592 · Cited by 353 · cited 1×
“Indeed, both dopamine neuron firing (27, 28) and phasic mesolimbic dopamine release (29, 30) precede well-learned self-initiated actions, consistent with a backpropagating reward prediction error signal.”
A Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299 · cited 1×
“There have been several recent attempts to integrate the concept that dopamine mediates incentive motivation into the reinforcement learning framework (50-52).”
Lynd-Balta, E., & Haber, S. N. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90182-1 · Cited by 233 · cited 1×
“During the test phasic dopamine concentration changes in the ventral portion of the neostriatum/nucleus accumbens, a region previously implicated in reward-motivated behavior (36-38), were monitored with fast-scan cyclic voltammetry (see Supplement) while rats earned a total of 30 sucrose pellets rewards through their…”
Stuber, G. D., Roitman, M. F., Phillips, P. E. M., Carelli, R. M., & Wightman, R. M. (2005) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300619 · Cited by 217
Owesson‐White, C. A., Ariansen, J., Stuber, G. D., Cleaveland, N. A., Cheer, J. F., Mark Wightman, R., & Carelli, R. M. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06916.x · Cited by 126 · cited 1×
“In this respect our results are consistent with the few studies that have examined phasic mesolimbic dopamine release during free-operant behavior for drug reward (29, 30, 41) and importantly show that phasic mesolimbic dopamine signaling comes to precede a self-initiated action under naturalistic conditions in which…”
Ostlund, S. B., Winterbauer, N. E., & Balleine, B. W. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.1176-09.2009 · Cited by 103
Self-citationOstlund, S. B., Wassum, K. M., Murphy, N. P., Balleine, B. W., & Maidment, N. T. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4759-10.2011 · Cited by 94
Self-citation
Cited by
Berke, J. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0152-y · Cited by 904
Dissociable dopamine dynamics for learning and motivation
Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019) · Nature · doi:10.1038/s41586-019-1235-y · Cited by 735
Dopamine neuron activity before action initiation gates and invigorates future movements
da Silva, J. A., Tecuapetla, F., Paixão, V., & Costa, R. M. (2018) · Nature · doi:10.1038/nature25457 · Cited by 623
A Neural Circuit Mechanism for Encoding Aversive Stimuli in the Mesolimbic Dopamine System
de Jong, J. W., Afjei, S. A., Pollak Dorocic, I., Peck, J. R., Liu, C., Kim, C. K., Tian, L., Deisseroth, K., & Lammel, S. (2019) · Neuron · doi:10.1016/j.neuron.2018.11.005 · Cited by 511
Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry
Keiflin, R., & Janak, P. H. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.037 · Cited by 382
Metadata
- Authors
- Wassum, K. M., Ostlund, S. B., & Maidment, N. T.
- Venue
- Biological Psychiatry
- Publisher
- Elsevier BV
- Published
- 2012-05
- Volume
- 71
- Issue
- 10
- Pages
- 846-854
- ISSN
- 0006-3223 (ISSN-L)
- OpenAlex
- W2055197689
- Cited by
- 111
- References
- 30
- FWCI
- 2.88
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.