Dopamine neurons respond to errors in the prediction of sensory features of expected rewards
Open accessAbstract
Midbrain dopamine neurons have been proposed to signal prediction errors as defined in model-free reinforcement learning algorithms. While these algorithms have been extremely powerful in interpreting dopamine activity, these models do not register any error unless there is a difference between the value of what is predicted and what is received. Yet learning often occurs in response to changes in the unique features that characterize what is received, sometimes with no change in its value at all. Here, we show that classic error-signaling dopamine neurons also respond to changes in value-neutral sensory features of an expected reward. This suggests that dopamine neurons have access to a wider variety of information than contemplated by the models currently used to interpret their activity and that, while their firing may conform to predictions of these models in some cases, they are not restricted to signaling errors in the prediction of value.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In rats, VTA dopaminergic neurons fire in response to unexpected changes in the sensory properties of a reinforcer when the relative subjective value is unchanged.
“VTA dopaminergic neurons were shown to fire in response to unexpected changes in the sensory properties of a reinforcer when the relative subjective value was unchanged (e.g. switching the flavor of an equally palatable Kool-Aid reward) (Takahashi et al., 2017; Stalnaker et al., 2019).”
From Ventral striatum dopamine release encodes unique properties of visual stimuli in mice (2023) · cites this paper as Takahashi et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574 · see citing works
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · see citing works
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · see citing works
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · see citing works
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · see citing works
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,103 · see citing works
Dopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036 · see citing works
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · see citing works
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967 · see citing works
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955 · see citing works
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K. A., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 879 · see citing works
Mesolimbic dopamine signals the value of work
Hamid, A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Weele, C. M. V., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2015) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 815 · see citing works
Start/stop signals emerge in nigrostriatal circuits during sequence learning
Jin, X., & Costa, R. M. (2010) · Nature · doi:10.1038/nature09263 · Cited by 611 · see citing works
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597 · see citing works
Self-citationMidbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548 · see citing works
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 544 · see citing works
Margolis, E. B., Lock, H., Hjelmstad, G. O., & Fields, H. L. (2006) · The Journal of Physiology · doi:10.1113/jphysiol.2006.117069 · Cited by 527 · see citing works
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · see citing works
Dopamine: generalization and bonuses
Kakade, S. M., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 477 · see citing works
Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons
Ungless, M. A., & Grace, A. A. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.02.003 · Cited by 431 · see citing works
Influence of Reward Delays on Responses of Dopamine Neurons
Kobayashi, S., & Schultz, W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 431 · see citing works
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429 · see citing works
Dialogues on prediction errors
Niv, Y., & Schoenbaum, G. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.03.006 · Cited by 355 · see citing works
Self-citationHierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342 · see citing works
Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values
Jones, J. L., Esber, G. R., McDannald, M. A., Gruber, A. J., Hernández, A., Mirenzi, A., & Schoenbaum, G. M. (2012) · Science · doi:10.1126/science.1227489 · Cited by 332 · see citing works
Self-citationThe temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310 · see citing works
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300 · see citing works
Hart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.2489-13.2014 · Cited by 294 · see citing works
Dopamine transients are sufficient and necessary for acquisition of model-based associations
Sharpe, M. J., Chang, C. Y., Liu, M. A., Batchelor, H. M., Mueller, L. E., Jones, J. L., Niv, Y., & Schoenbaum, G. M. (2017) · Nature Neuroscience · doi:10.1038/nn.4538 · Cited by 284 · see citing works
Self-citationExpectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · see citing works
Self-citationDopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257 · see citing works
McDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., & Schoenbaum, G. M. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.5499-10.2011 · Cited by 215 · see citing works
Self-citationBrief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors
Chang, C. Y., Esber, G. R., Marrero-Garcia, Y., Yau, H., Bonci, A., & Schoenbaum, G. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4191 · Cited by 211 · see citing works
Self-citationDopamine Reward Prediction Error Responses Reflect Marginal Utility
Stauffer, W. R., Lak, A., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.064 · Cited by 204 · see citing works
Saddoris, M. P., Cacciapaglia, F., Wightman, R. M., & Carelli, R. M. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.2344-15.2015 · Cited by 200 · see citing works
A Pallidus-Habenula-Dopamine Pathway Signals Inferred Stimulus Values
Bromberg-Martin, E. S., Matsumoto, M., Hong, S., & Hikosaka, O. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00158.2010 · Cited by 182 · see citing works
The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards
Burke, K. A., Franz, T. M., Miller, D. N., & Schoenbaum, G. M. (2008) · Nature · doi:10.1038/nature06993 · Cited by 181 · see citing works
Self-citationEsber, G. R., & Haselgrove, M. (2011) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2011.0836 · Cited by 178 · see citing works
Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170 · see citing works
Sensory preconditioning of human subjects.
Brogden, W. J. (1947) · Journal of Experimental Psychology · doi:10.1037/h0058465 · Cited by 124 · see citing works
Pavlovian valuation systems in learning and decision making
Clark, J. J., Hollon, N. G., & Phillips, P. E. M. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.06.004 · Cited by 120 · see citing works
Boorman, E. D., Rajendran, V. G., O’Reilly, J. X., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.014 · Cited by 117 · see citing works
Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation
Collins, A. L., Greenfield, V. Y., Bye, J. K., Linker, K. E., Wang, A. S., & Wassum, K. M. (2016) · Scientific Reports · doi:10.1038/srep20231 · Cited by 116 · see citing works
Takahashi, Y. K., Langdon, A., Niv, Y., & Schoenbaum, G. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.05.015 · Cited by 116 · see citing works
Self-citationNeural signals encoding shifts in beliefs
Schwartenbeck, P., FitzGerald, T. H. B., & Dolan, R. J. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.10.067 · Cited by 82 · see citing works
Mesolimbic Dopamine Encodes Prediction Errors in a State-Dependent Manner
Papageorgiou, G., Baudonnat, M., Cucca, F., & Walton, M. E. (2016) · Cell Reports · doi:10.1016/j.celrep.2016.03.031 · Cited by 76 · see citing works
Effects of Prefrontal Cortical Inactivation on Neural Activity in the Ventral Tegmental Area
Jo, Y. S., Lee, J. J.-H., & Mizumori, S. J. Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.0118-13.2013 · Cited by 74 · see citing works
Learning about qualitatively different outcomes during a blocking procedure
Rescorla, R. A. (1999) · Animal Learning & Behavior · doi:10.3758/bf03199671 · Cited by 53 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hallucinations and strong priors
Corlett, P. R., Horga, G., Fletcher, P. C., Alderson-Day, B., Schmack, K., & Powers, A. R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.001 · Cited by 534 · see citing works
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280 · see citing works
The successor representation: Its computational logic and neural substrates
Gershman, S. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0151-18.2018 · Cited by 191 · see citing works
Solié, C., Girard, B., Righetti, B., Tapparel, M., & Bellone, C. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-00972-9 · Cited by 186 · see citing works
TAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry
Frässle, S., Aponte, E. A., Bollmann, S., Brodersen, K. H., Do, C. T., Harrison, O. K., Harrison, S. J., Heinzle, J., Iglesias, S., Kasper, L., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.680811 · Cited by 169 · see citing works
Model-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161 · see citing works
Self-citationBelief state representation in the dopamine system
Babayan, B. M., Uchida, N., & Gershman, S. J. (2018) · Nature Communications · doi:10.1038/s41467-018-04397-0 · Cited by 101 · see citing works
McCutcheon, R. A., Nour, M. M., Dahoun, T., Jauhar, S., Pepper, F., Expert, P., Veronese, M., Adams, R. A., Turkheimer, F., Mehta, M. A., et al. (2019) · Biological Psychiatry · doi:10.1016/j.biopsych.2018.09.010 · Cited by 104 · see citing works
Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia
Nour, M. M., Dahoun, T., Schwartenbeck, P., Adams, R. A., FitzGerald, T. H. B., Coello, C., Wall, M. B., Dolan, R. J., & Howes, O. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1809298115 · Cited by 101 · see citing works
Neural circuitry of information seeking
Bromberg-Martin, E. S., & Monosov, I. E. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.07.006 · Cited by 89 · see citing works
Dopamine neuron ensembles signal the content of sensory prediction errors
Stalnaker, T. A., Howard, J. D., Takahashi, Y. K., Gershman, S. J., Kahnt, T., & Schoenbaum, G. (2019) · eLife · doi:10.7554/elife.49315 · Cited by 74 · see citing works
Self-citationTo be specific: The role of orbitofrontal cortex in signaling reward identity.
Howard, J. D., & Kahnt, T. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000455 · Cited by 57 · see citing works
Liraglutide restores impaired associative learning in individuals with obesity
Hanssen, R., Rigoux, L., Kuzmanovic, B., Iglesias, S., Kretschmer, A. C., Schlamann, M., Albus, K., Edwin Thanarajah, S., Sitnikow, T., Melzer, C., et al. (2023) · Nature Metabolism · doi:10.1038/s42255-023-00859-y · Cited by 52 · see citing works
McCutcheon, R. A., Bloomfield, M. A. P., Dahoun, T., Mehta, M., & Howes, O. D. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2018.12.011 · Cited by 35 · see citing works
Ventral striatum dopamine release encodes unique properties of visual stimuli in mice
Gonzalez, L. S., Fisher, A. A., D'Souza, S. P., Cotella, E. M., Lang, R. A., & Robinson, J. E. (2023) · eLife · doi:10.7554/elife.85064 · Cited by 18 · see citing works
Predictive Representations: Building Blocks of Intelligence
Carvalho, W., Tomov, M. S., de Cothi, W., Barry, C., & Gershman, S. J. (2024) · Neural Computation · doi:10.1162/neco_a_01705 · Cited by 15 · see citing works
- Authors
- Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G.
- Affiliations
- National Institutes of Health (US)
- University of Maryland, Baltimore (US)
- Corresponding author
- Takahashi, Yuji K.
- Schoenbaum, Geoffrey
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2017-09
- Volume
- 95
- Issue
- 6
- Pages
- 1395-1405.e3
- ISSN
- 0896-6273 (ISSN-L)
- PMC
- PMC5658021
- OpenAlex
- W2754726085
- Cited by
- 200
- References
- 51
- Citations by year
- 2026: 10 · 2025: 21 · 2024: 21 · 2023: 26 · 2022: 21 · 2021: 24 · 2020: 24 · 2019: 27 · 2018: 20 · 2017: 6
- FWCI
- 10.30
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuron (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Drug Abuse (ZIA-DA000587)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- dopaminergic neurons, prediction error, model-free reinforcement learning, sensory features, reward prediction, midbrain dopamine, reward learning, neural coding
- MeSH terms
- Animals, Conditioning, Operant, Male, Models, Neurological, Reward, Sensation, Ventral Tegmental Area, Animals, Genetically Modified, Rats, Dopaminergic Neurons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
