Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 4

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Disentangling prediction error and value in a formal test of dopamine’s role in reinforcement learning

Usypchuk, Alexandra A., Maes, Etienne J.P., Lozzi, Megan, Avramidis, Dimitrios K., Schoenbaum, Geoffrey, Esber, Guillem R., Gardner, Matthew P.H., Iordanova, Mihaela D. · 2025 · Current Biology

Published version of a preprint

Abstract

The discovery that midbrain dopamine (DA) transients can be mapped onto reward prediction errors (RPEs), the critical signal that drives learning, is a landmark in neuroscience. Causal support for the RPE hypothesis comes from studies showing that stimulating DA neurons can drive learning under conditions where it would not otherwise occur.1,2,3 However, such stimulation might also promote learning by adding reward value and indirectly inducing an RPE. This added value could support new learning even when it is insufficient to support instrumental behavior.4,5 Thus, these competing interpretations are challenging to disentangle and require direct comparison under matched conditions. We developed two computational models grounded in temporal difference reinforcement learning (TDRL)6,7,8 that dissociate the role of DA as an RPE versus a value signal. We validated our models by showing that they both predict learning (unblocking) when ventral tegmental area (VTA) DA stimulation occurs during expected reward delivery in a behavioral blocking design and confirmed this behaviorally. We then contrasted the models by delivering constant optogenetic stimulation during reward across both learning phases of blocking. The value model predicted blocking; the RPE model predicted unblocking. Behavioral results aligned with the latter. Moreover, the RPE model uniquely predicted that constant stimulation would unblock learning at higher frequencies (>20 Hz) when the artificial error alone drives learning. This, too, was confirmed experimentally. We demonstrate a principled computational and empirical dissociation between DA as an RPE versus a value signal. Our results advance understanding of how DA neuron stimulation drives learning.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.cub.2025.06.076

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 34

  1. Steinberg, 2013, A causal link between prediction errors, dopamine neurons and learning, Nat. Neurosci., 16, 966, 10.1038/nn.3413 · DOI · Paper page
  2. Keiflin, 2019, Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions, Curr. Biol., 29, 93, 10.1016/j.cub.2018.11.050 · DOI
  3. Sharpe, 2017, Dopamine transients are sufficient and necessary for acquisition of model-based associations, Nat. Neurosci., 20, 735, 10.1038/nn.4538 · DOI · Paper page
  4. Millard, 2024, Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency, Nat. Neurosci., 27, 1253, 10.1038/s41593-024-01643-1 · DOI
  5. Sharpe, 2020, Dopamine transients do not act as model-free prediction errors during associative learning, Nat. Commun., 11, 106, 10.1038/s41467-019-13953-1 · DOI · Paper page
  6. Dickinson, 1976, Surprise and the attenuation of blocking, J. Exp. Psychol. Anim. Behav. Process, 2, 313, 10.1037/0097-7403.2.4.313 · doi:10.1037/0097-7403.2.4.313 · Paper page
  7. Mackintosh, 1977, Locus of the Effect of a Surprising Reinforcer in the Attenuation of Blocking, Q. J. Exp. Psychol., 29, 327, 10.1080/14640747708400608 · DOI
  8. Schultz, 1997, A Neural Substrate of Prediction and Reward, Science, 275, 1593, 10.1126/science.275.5306.1593 · DOI · Paper page
  9. Olds, 1954, Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain, J. Comp. Physiol. Psychol., 47, 419, 10.1037/h0058775 · DOI · Paper page
  10. Phillips, 1973, Dopaminergic and Noradrenergic Substrates of Positive Reinforcement: Differential Effects of d – and l-Amphetamine, Science, 179, 575, 10.1126/science.179.4073.575 · DOI · Paper page
  11. Corbett, 1980, Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: A moveable electrode mapping study, Brain Res., 185, 1, 10.1016/0006-8993(80)90666-6 · DOI · Paper page
  12. Pascoli, 2015, Sufficiency of Mesolimbic Dopamine Neuron Stimulation for the Progression to Addiction, Neuron, 88, 1054, 10.1016/j.neuron.2015.10.017 · DOI · Paper page
  13. Carter, 2024, Does phasic dopamine release cause policy updates?, Eur. J. Neurosci., 59, 1260, 10.1111/ejn.16199 · DOI
  14. Root, 2014, Single rodent mesohabenular axons release glutamate and GABA, Nat. Neurosci., 17, 1543, 10.1038/nn.3823 · DOI · Paper page
  15. Dickinson, 1979, Reinforcer specificity in the enhancement of conditioning by posttrial surprise, J. Exp. Psychol. Anim. Behav. Process, 5, 162, 10.1037/0097-7403.5.2.162 · doi:10.1037/0097-7403.5.2.162
  16. Holland, 1988, Excitation and inhibition in unblocking, J. Exp. Psychol. Anim. Behav. Process., 14, 261, 10.1037/0097-7403.14.3.261 · DOI
  17. Sotoyama, 2021, Resting-state dopaminergic cell firing in the ventral tegmental area negatively regulates affiliative social interactions in a developmental animal model of schizophrenia, Transl. Psychiatry, 11, 236, 10.1038/s41398-021-01346-2 · DOI
  18. Grace, 1984, The control of firing pattern in nigral dopamine neurons: burst firing, J. Neurosci., 4, 2877, 10.1523/JNEUROSCI.04-11-02877.1984 · DOI · Paper page
  19. Marinelli, 2014, Heterogeneity of dopamine neuron activity across traits and states, Neuroscience, 282, 176, 10.1016/j.neuroscience.2014.07.034 · DOI
  20. Adamantidis, 2011, Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior, J. Neurosci., 31, 10829, 10.1523/JNEUROSCI.2246-11.2011 · DOI · Paper page
  21. Iordanova, 2006, Opioid receptors in the nucleus accumbens regulate attentional learning in the blocking paradigm, J. Neurosci., 26, 4036, 10.1523/JNEUROSCI.4679-05.2006 · DOI
  22. Iordanova, 2006, Dopamine activity in the nucleus accumbens modulates blocking in fear conditioning, Eur. J. Neurosci., 24, 3265, 10.1111/j.1460-9568.2006.05195.x · DOI
  23. Iordanova, 2010, Dopamine transmission in the amygdala modulates surprise in an aversive blocking paradigm, Behav. Neurosci., 124, 780, 10.1037/a0021111 · DOI
  24. Kim, 2020, A Unified Framework for Dopamine Signals across Timescales, Cell, 183, 1600, 10.1016/j.cell.2020.11.013 · DOI · Paper page
  25. Hamid, 2016, Mesolimbic dopamine signals the value of work, Nat. Neurosci., 19, 117, 10.1038/nn.4173 · DOI · Paper page
  26. De Jong, 2024, State and rate-of-change encoding in parallel mesoaccumbal dopamine pathways, Nat. Neurosci., 27, 309, 10.1038/s41593-023-01547-6 · DOI
  27. Jeong, 2022, Mesolimbic dopamine release conveys causal associations, Science, 378, eabq6740, 10.1126/science.abq6740 · DOI · Paper page
  28. Fraser, 2023, From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement, J. Neurosci., 43, 3895, 10.1523/JNEUROSCI.0028-23.2023 · DOI
  29. Maes, 2020, Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors, Nat. Neurosci., 23, 176, 10.1038/s41593-019-0574-1 · DOI
  30. Wise, 1978, Major attenuation of food reward with performance-sparing doses of pimozide in the rat, Can. J. Psychol., 32, 77, 10.1037/h0081678 · DOI · Paper page
  31. Howe, 2016, Rapid signalling in distinct dopaminergic axons during locomotion and reward, Nature, 535, 505, 10.1038/nature18942 · DOI · Paper page
  32. Sadacca, 2016, Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework, eLife, 5, 10.7554/eLife.13665 · DOI
  33. Redish, 2004, Addiction as a Computational Process Gone Awry, Science, 306, 1944, 10.1126/science.1102384 · DOI · Paper page
  34. Yap, 2005, Latent inhibition in the developing rat: An examination of context-specific effects, Dev. Psychobiol., 47, 55, 10.1002/dev.20074 · DOI

Metadata

Authors
Usypchuk, A. A., Maes, E. J. P., Lozzi, M., Avramidis, D. K., Schoenbaum, G., Esber, G. R., Gardner, M. P. H., & Iordanova, M. D.
Venue
Current Biology
Publisher
Elsevier BV
Published
2025-08
Volume
35
Issue
16
Pages
4019-4027.e7
ISSN
0960-9822
Cited by
4
References
11
Language
en
Funders
  • McGill University
  • U.S. Department of Health and Human Services
  • Natural Sciences and Engineering Research Council of Canada (RGPIN-2015-03658)
  • National Institutes of Health
  • Nature et technologies
  • Canada Research Chairs Program (950\u2013230456)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.