The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Arithmetic and local circuitry underlying dopamine prediction errors
Open accessAbstract
Dopamine neurons are thought to facilitate learning by comparing actual and expected reward. Despite two decades of investigation, little is known about how this comparison is made. To determine how dopamine neurons calculate prediction error, we combined optogenetic manipulations with extracellular recordings in the ventral tegmental area while mice engaged in classical conditioning. Here we demonstrate, by manipulating the temporal expectation of reward, that dopamine neurons perform subtraction, a computation that is ideal for reinforcement learning but rarely observed in the brain. Furthermore, selectively exciting and inhibiting neighbouring GABA (γ-aminobutyric acid) neurons in the ventral tegmental area reveals that these neurons are a source of subtraction: they inhibit dopamine neurons when reward is expected, causally contributing to prediction-error calculations. Finally, bilaterally stimulating ventral tegmental area GABA neurons dramatically reduces anticipatory licking to conditioned odours, consistent with an important role for these neurons in reinforcement learning. Together, our results uncover the arithmetic and local circuitry underlying dopamine prediction errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Millisecond-timescale, genetically targeted optical control of neural activity
Boyden, E. S., Zhang, F., Bamberg, E., Nagel, G., & Deisseroth, K. (2005) · Nature Neuroscience · doi:10.1038/nn1525 · Cited by 4,861
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Self-citationMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Leptin Action on GABAergic Neurons Prevents Obesity and Reduces Inhibitory Tone to POMC Neurons
Vong, L., Ye, C., Yang, Z., Choi, B., Chua, S., & Lowell, B. B. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.028 · Cited by 1,235
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Cortical interneurons that specialize in disinhibitory control
Pi, H.-J., Hangya, B., Kvitsiani, D., Sanders, J. I., Huang, Z. J., & Kepecs, A. (2013) · Nature · doi:10.1038/nature12676 · Cited by 1,136
Speed and accuracy of olfactory discrimination in the rat
Uchida, N., & Mainen, Z. F. (2003) · Nature Neuroscience · doi:10.1038/nn1142 · Cited by 682
Self-citationGABA Neurons of the VTA Drive Conditioned Place Aversion
Tan, K. R., Yvon, C., Turiault, M., Mirzabekov, J. J., Doehner, J., Labouèbe, G., Deisseroth, K., Tye, K. M., & Lüscher, C. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.015 · Cited by 651
Activation of VTA GABA Neurons Disrupts Reward Consumption
van Zessen, R., Phillips, J. L., Budygin, E. A., & Stuber, G. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.016 · Cited by 620
Atasoy, D., Aponte, Y., Su, H. H., & Sternson, S. M. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1954-08.2008 · Cited by 619
Parvalbumin-expressing interneurons linearly transform cortical responses to visual stimuli
Atallah, B. V., Bruns, W., Carandini, M., & Scanziani, M. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.013 · Cited by 612
A mathematical model for simple learning.
Bush, R. R., & Mosteller, F. (1951) · Psychological Review · doi:10.1037/h0054388 · Cited by 586
Division and subtraction by distinct cortical inhibitory networks in vivo
Wilson, N. R., Runyan, C. A., Wang, F. L., & Sur, M. (2012) · Nature · doi:10.1038/nature11347 · Cited by 568
Distinct behavioural and network correlates of two interneuron types in prefrontal cortex
Kvitsiani, D., Ranade, S., Hangya, B., Taniguchi, H., Huang, J. Z., & Kepecs, A. (2013) · Nature · doi:10.1038/nature12176 · Cited by 516
Divisive Normalization in Olfactory Population Codes
Olsen, S. R., Bhandawat, V., & Wilson, R. I. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.009 · Cited by 500
Han, X., Chow, B. Y., Zhou, H., Klapoetke, N. C., Chuong, A., Rajimehr, R., Yang, A., Baratta, M. V., Winkle, J., Desimone, R., et al. (2011) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2011.00018 · Cited by 491
Silver, R. A. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2864 · Cited by 481
Bäckman, C. M., Malik, N., Zhang, Y., Shan, L., Grinberg, A., Hoffer, B. J., Westphal, H., & Tomac, A. C. (2006) · genesis · doi:10.1002/dvg.20228 · Cited by 460
Spatiotemporal Elements of Macaque V1 Receptive Fields
Rust, N. C., Schwartz, O., Movshon, J. A., & Simoncelli, E. P. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.021 · Cited by 457
Hong, S., Jhou, T. C., Smith, M., Saleem, K. S., & Hikosaka, O. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1384-11.2011 · Cited by 390
Lima, S. Q., Hromádka, T., Znamenskiy, P., & Zador, A. M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006099 · Cited by 360
Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates
Holt, G. R., & Koch, C. (1997) · Neural Computation · doi:10.1162/neco.1997.9.5.1001 · Cited by 250
Roy, J. E., & Cullen, K. E. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3988-03.2004 · Cited by 244
Omelchenko, N., & Sesack, S. R. (2009) · Synapse · doi:10.1002/syn.20668 · Cited by 193
Schmitzer-Torbert, N., & Redish, A. D. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00687.2003 · Cited by 175
Stimulus representation and the timing of reward-prediction errors in models of the dopamine system
Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-654 · Cited by 148
Multiplicative Gain Changes Are Induced by Excitation or Inhibition Alone
Murphy, B. K., & Miller, K. D. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-31-10040.2003 · Cited by 146
Neural mechanisms of acquired phasic dopamine responses in learning
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2009) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.11.019 · Cited by 115
Fiorillo, C. D., Song, M. R., & Yun, S. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3883-12.2013 · Cited by 98
Gain Modulation of Neuronal Responses by Subtractive and Divisive Mechanisms of Inhibition
Ayaz, A., & Chance, F. S. (2008) · Journal of Neurophysiology · doi:10.1152/jn.90547.2008 · Cited by 85
Efficient reinforcement learning: computational theories, neuroscience and robotics
Kawato, M., & Samejima, K. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.03.004 · Cited by 84
Tan, C. O., & Bullock, D. H. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.0259-08.2008 · Cited by 43
Vitay, J., & Hamker, F. H. (2014) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2014.00004 · Cited by 30
Conditioning and time representation in long short-term memory networks
Rivest, F., Kalaska, J. F., & Bengio, Y. (2013) · Biological Cybernetics · doi:10.1007/s00422-013-0575-1 · Cited by 13
Cited by
Ultrafast neuronal imaging of dopamine dynamics with designed genetically encoded sensors
Patriarchi, T., Cho, J. R., Merten, K., Howe, M. W., Marley, A., Xiong, W.-H., Folk, R. W., Broussard, G. J., Liang, R., Jang, M. J., et al. (2018) · Science · doi:10.1126/science.aat4422 · Cited by 1,074
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
Striatal circuits for reward learning and decision-making
Cox, J., & Witten, I. B. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0189-2 · Cited by 568
Dopamine and Cognitive Control in Prefrontal Cortex
Ott, T., & Nieder, A. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.006 · Cited by 530
A high-resolution probabilistic in vivo atlas of human subcortical brain nuclei
Pauli, W. M., Nili, A. N., & Tyszka, J. M. (2018) · Scientific Data · doi:10.1038/sdata.2018.63 · Cited by 517
Metadata
- Authors
- Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N.
- Venue
- Nature
- Published
- 2015-08-31
- Volume
- 525
- Issue
- 7568
- Pages
- 243-246
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2152681028
- Cited by
- 392
- References
- 39
- FWCI
- 16.08
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.