Optogenetic Mimicry of the Transient Activation of Dopamine Neurons by Natural Reward Is Sufficient for Operant Reinforcement
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Activation of dopamine receptors in forebrain regions, for minutes or longer, is known to be sufficient for positive reinforcement of stimuli and actions. However, the firing rate of dopamine neurons is increased for only about 200 milliseconds following natural reward events that are better than expected, a response which has been described as a "reward prediction error" (RPE). Although RPE drives reinforcement learning (RL) in computational models, it has not been possible to directly test whether the transient dopamine signal actually drives RL. Here we have performed optical stimulation of genetically targeted ventral tegmental area (VTA) dopamine neurons expressing Channelrhodopsin-2 (ChR2) in mice. We mimicked the transient activation of dopamine neurons that occurs in response to natural reward by applying a light pulse of 200 ms in VTA. When a single light pulse followed each self-initiated nose poke, it was sufficient in itself to cause operant reinforcement. Furthermore, when optical stimulation was delivered in separate sessions according to a predetermined pattern, it increased locomotion and contralateral rotations, behaviors that are known to result from activation of dopamine neurons. All three of the optically induced operant and locomotor behaviors were tightly correlated with the number of VTA dopamine neurons that expressed ChR2, providing additional evidence that the behavioral responses were caused by activation of dopamine neurons. These results provide strong evidence that the transient activation of dopamine neurons provides a functional reward signal that drives learning, in support of RL theories of dopamine function.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 3×
“There is extensive evidence, particularly from pharmacological studies, that dopamine can cause positive reinforcement of stimuli and actions [1,2].”
Olds, J., & Milner, P. (1954) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0058775 · Cited by 3,176 · cited 1×
“The optically driven reinforcement of operant responding observed here is comparable to the electrically driven ‘‘brain stimulation reward’’ (BSR) described first by Olds and Milner [37].”
A psychomotor stimulant theory of addiction.
Wise, R. A., & Bozarth, M. A. (1987) · Psychological Review · doi:10.1037/0033-295x.94.4.469 · Cited by 2,747
Parvalbumin neurons and gamma rhythms enhance cortical circuit performance
Sohal, V. S., Zhang, F., Yizhar, O., & Deisseroth, K. (2009) · Nature · doi:10.1038/nature07991 · Cited by 2,606
Driving fast-spiking cells induces gamma rhythm and controls sensory responses
Cardin, J. A., Carlén, M., Meletis, K., Knoblich, U., Zhang, F., Deisseroth, K., Tsai, L.-H., & Moore, C. I. (2009) · Nature · doi:10.1038/nature08002 · Cited by 2,603
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“compelling evidence that the brief, naturally occurring activation of dopamine neurons is sufficient for positive reinforcement of preceding actions, as hypothesized by RL models of dopamine function [13,14].”
Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry
Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., & Kreitzer, A. C. (2010) · Nature · doi:10.1038/nature09159 · Cited by 1,885
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · cited 1×
“In addition, there is evidence that under some conditions dopamine neurons can be activated by physically salient events that may not have reward value [18,21,22].”
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Self-citationPhasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266 · cited 5×
“Tsai and colleagues first demonstrated the feasibility of optogenetic stimulation of dopamine neurons in mice [16].”
InfluentialTuning arousal with optogenetic modulation of locus coeruleus neurons
Carter, M. E., Yizhar, O., Chikahisa, S., Nguyen, H., Adamantidis, A., Nishino, S., Deisseroth, K., & de Lecea, L. (2010) · Nature Neuroscience · doi:10.1038/nn.2682 · Cited by 993
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
Encoding of conditioned fear in central amygdala inhibitory circuits
Ciocchi, S., Herry, C., Grenier, F., Wolff, S. B. E., Letzkus, J. J., Vlachos, I., Ehrlich, I., Sprengel, R., Deisseroth, K., Stadler, M. B., et al. (2010) · Nature · doi:10.1038/nature09559 · Cited by 924
Cell type-specific loss of BDNF signaling mimics optogenetic control of cocaine reward
Lobo, M. K., Covington, H. E., Chaudhury, D., Friedman, A. K., Sun, H., Damez-Werno, D., Dietz, D. M., Zaman, S., Koo, J. W., Kennedy, P. J., et al. (2010) · Science · doi:10.1126/science.1188472 · Cited by 880
Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking
Stuber, G. D., Sparta, D. R., Stamatakis, A. M., van Leeuwen, W. A., Hardjoprajitno, J. E., Cho, S., Tye, K. M., Kempadoo, K. A., Zhang, F., Deisseroth, K., et al. (2011) · Nature · doi:10.1038/nature10194 · Cited by 878
Genetic dissection of an amygdala microcircuit that gates conditioned fear
Haubensak, W., Kunwar, P. S., Cai, H., Ciocchi, S., Wall, N. R., Ponnusamy, R., Biag, J., Dong, H.-W., Deisseroth, K., Callaway, E. M., et al. (2010) · Nature · doi:10.1038/nature09553 · Cited by 840
Dopamine operates as a subsecond modulator of food seeking
Roitman, M. F., Stuber, G. D., Phillips, P. E. M., Wightman, R. M., & Carelli, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3823-03.2004 · Cited by 718 · cited 2×
“2 s, causing a transient increase in dopamine concentration lasting 1–2 s [17].”
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
Dopaminergic terminals in the nucleus accumbens but not the dorsal striatum corelease glutamate
Stuber, G. D., Hnasko, T. S., Britt, J. P., Edwards, R. H., & Bonci, A. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.1754-10.2010 · Cited by 529 · cited 1×
“Optogenetics has previously been used to selectively target dopamine neurons in mice [16,28–32].”
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · cited 3×
“Second, some have proposed that the transient dopamine signal does not actually represent reward, and thus may not be appropriate for a positive reinforcement signal [18,19], but see [20].”
Cholinergic Interneurons Control Local Circuit Activity and Cocaine Conditioning
Witten, I. B., Lin, S.-C., Brodsky, M., Prakash, R., Diester, I., Anikeeva, P., Gradinaru, V., Ramakrishnan, C., & Deisseroth, K. (2010) · Science · doi:10.1126/science.1193771 · Cited by 467
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Zweifel, L. S., Parker, J. G., Lobb, C. J., Rainwater, A., Wall, V. Z., Fadok, J. P., Darvas, M., Kim, M. J., Mizumori, S. J. Y., Paladini, C. A., et al. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0813415106 · Cited by 426 · cited 1×
“Our results are complemented by those of Zweifel and colleagues [48], who selectively disrupted NMDA-type glutamate receptors in dopamine neurons and found that mice were impaired in their acquisition of a variety of reinforced behaviors.”
Adamantidis, A. R., Tsai, H.-C., Boutrel, B., Zhang, F., Stuber, G. D., Budygin, E. A., Tourino, C., Bonci, A., Deisseroth, K., & de Lecea, L. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.2246-11.2011 · Cited by 357
Glutamatergic signaling by mesolimbic dopamine neurons in the nucleus accumbens
Tecuapetla, F., Patel, J. C., Xenias, H., English, D., Tadros, I., Shah, F., Berlin, J., Deisseroth, K., Rice, M. E., Tepper, J. M., et al. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0265-10.2010 · Cited by 310 · cited 1×
“Optogenetics has previously been used to selectively target dopamine neurons in mice [16,28–32].”
The temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310
Self-citationOptical activation of lateral amygdala pyramidal cells instructs associative fear learning
Johansen, J. P., Hamanaka, H., Monfils, M. H., Behnia, R., Deisseroth, K., Blair, H. T., & LeDoux, J. E. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1002418107 · Cited by 308
Cannon, C. M., & Palmiter, R. D. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-34-10827.2003 · Cited by 280 · cited 2×
“Although the present results demonstrate that the transient activation of dopamine neurons is sufficient for positive reinforcement, it does not appear to be necessary for reinforcement [24–26].”
Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards
Robinson, S., Sandstrom, S. M., Denenberg, V. H., & Palmiter, R. D. (2005) · Behavioral Neuroscience · doi:10.1037/0735-7044.119.1.5 · Cited by 247
Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli
Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4828-09.2010 · Cited by 158 · cited 1×
“nature of the reward-predictive stimulus [11,35]; 200 ms is typical”
Cited by
Injectable, Cellular-Scale Optoelectronics with Applications for Wireless Optogenetics
Kim, T.-I., McCall, J. G., Jung, Y. H., Huang, X., Siuda, E. R., Li, Y., Song, J., Song, Y. M., Pao, H. A., Kim, R.-H., et al. (2013) · Science · doi:10.1126/science.1232437 · Cited by 1,174
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Ventral tegmental area: cellular heterogeneity, connectivity and behaviour
Morales, M., & Margolis, E. B. (2017) · Nature reviews. Neuroscience · doi:10.1038/nrn.2016.165 · Cited by 1,108
Dopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036
Mesolimbic dopamine signals the value of work
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 824
Metadata
- Authors
- Kim, K. M., Baratta, M. V., Yang, A., Lee, D., Boyden, E. S., & Fiorillo, C. D.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-04-10
- Volume
- 7
- Issue
- 4
- Pages
- e33612
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2023916971
- Cited by
- 145
- References
- 35
- FWCI
- 5.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.