Mesolimbic dopamine adapts the rate of learning from action
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions 1–3 . Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction 4 ; however, so far there has been little consideration of how direct policy learning might inform our understanding 5 . Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning 6 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 2×
“costP = 1-e-∆t/500 (2) costN = sum( | δπ(t)/δt | ) (3) Robj = costP + wN*costN (4) e.”
MethodReinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Dissecting components of reward: ‘liking’, ‘wanting’, and learning
Berridge, K. C., Robinson, T. E., & Aldridge, J. W. (2009) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2008.12.014 · Cited by 1,942
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266 · cited 1×
“All procedures and animal handling were performed in strict accordance with protocols (11-39) that were approved by the Institutional Animal Care and Use Committee (IACUC) and consistent with the standards set forth by the Association for Assessment and Accreditation of Laboratory Animal Care (AALAC).”
MethodNeuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Cited by
The Formation and Function of the VTA Dopamine System
Hou, G., Hao, M., Duan, J., & Han, M.-H. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25073875 · Cited by 65
“The VTA-NAc circuit has been widely studied, and dopamine release in the NAc is shown to modulate the judgment of importance, reinforcement learning, and processing of aversive stimuli [168–170].”
BackgroundTsuruga, R., Tajika, Y., Minami, M., & Tsutsui-Kimura, I. (2026) · Communications Biology · doi:10.1038/s42003-026-10485-5 · Cited by 2
Reward magnitude determines reinforcement learning efficiency
Gong, S., Martell, A., Dudman, J. T., & Coddington, L. T. (2026) · Science · doi:10.1126/science.aeb0813 · Cited by 2
Self-citation
Metadata
- Authors
- Coddington, L. T., Lindo, S. E., & Dudman, J. T.
- Venue
- Nature
- Published
- 2023-01-18
- Volume
- 614
- Issue
- 7947
- Pages
- 294-302
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W4317390903
- Cited by
- 117
- References
- 25
- FWCI
- 12.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mesolimbic dopamine adapts the rate of learning from action. https://app.cochat.ai/papers/10.1038/s41586-022-05614-z
- The paper
- Coddington, L. T., Lindo, S. E., & Dudman, J. T. (2023). Mesolimbic dopamine adapts the rate of learning from action. Nature. https://doi.org/10.1038/s41586-022-05614-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.