Optogenetic Mimicry of the Transient Activation of Dopamine Neurons by Natural Reward Is Sufficient for Operant Reinforcement
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
Activation of dopamine receptors in forebrain regions, for minutes or longer, is known to be sufficient for positive reinforcement of stimuli and actions. However, the firing rate of dopamine neurons is increased for only about 200 milliseconds following natural reward events that are better than expected, a response which has been described as a "reward prediction error" (RPE). Although RPE drives reinforcement learning (RL) in computational models, it has not been possible to directly test whether the transient dopamine signal actually drives RL. Here we have performed optical stimulation of genetically targeted ventral tegmental area (VTA) dopamine neurons expressing Channelrhodopsin-2 (ChR2) in mice. We mimicked the transient activation of dopamine neurons that occurs in response to natural reward by applying a light pulse of 200 ms in VTA. When a single light pulse followed each self-initiated nose poke, it was sufficient in itself to cause operant reinforcement. Furthermore, when optical stimulation was delivered in separate sessions according to a predetermined pattern, it increased locomotion and contralateral rotations, behaviors that are known to result from activation of dopamine neurons. All three of the optically induced operant and locomotor behaviors were tightly correlated with the number of VTA dopamine neurons that expressed ChR2, providing additional evidence that the behavioral responses were caused by activation of dopamine neurons. These results provide strong evidence that the transient activation of dopamine neurons provides a functional reward signal that drives learning, in support of RL theories of dopamine function.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 3×
“There is extensive evidence, particularly from pharmacological studies, that dopamine can cause positive reinforcement of stimuli and actions [1,2].”
Olds, J., & Milner, P. (1954) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0058775 · Cited by 3,176 · cited 1×
“The optically driven reinforcement of operant responding observed here is comparable to the electrically driven ‘‘brain stimulation reward’’ (BSR) described first by Olds and Milner [37].”
A psychomotor stimulant theory of addiction.
Wise, R. A., & Bozarth, M. A. (1987) · Psychological Review · doi:10.1037/0033-295x.94.4.469 · Cited by 2,747
Parvalbumin neurons and gamma rhythms enhance cortical circuit performance
Sohal, V. S., Zhang, F., Yizhar, O., & Deisseroth, K. (2009) · Nature · doi:10.1038/nature07991 · Cited by 2,606
Driving fast-spiking cells induces gamma rhythm and controls sensory responses
Cardin, J. A., Carlén, M., Meletis, K., Knoblich, U., Zhang, F., Deisseroth, K., Tsai, L.-H., & Moore, C. I. (2009) · Nature · doi:10.1038/nature08002 · Cited by 2,603
The debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“compelling evidence that the brief, naturally occurring activation of dopamine neurons is sufficient for positive reinforcement of preceding actions, as hypothesized by RL models of dopamine function [13,14].”
Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry
Kravitz, A. V., Freeze, B. S., Parker, P. R. L., Kay, K., Thwin, M. T., Deisseroth, K., & Kreitzer, A. C. (2010) · Nature · doi:10.1038/nature09159 · Cited by 1,885
Cited by
Injectable, Cellular-Scale Optoelectronics with Applications for Wireless Optogenetics
Kim, T.-I., McCall, J. G., Jung, Y. H., Huang, X., Siuda, E. R., Li, Y., Song, J., Song, Y. M., Pao, H. A., Kim, R.-H., et al. (2013) · Science · doi:10.1126/science.1232437 · Cited by 1,174
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Ventral tegmental area: cellular heterogeneity, connectivity and behaviour
Morales, M., & Margolis, E. B. (2017) · Nature reviews. Neuroscience · doi:10.1038/nrn.2016.165 · Cited by 1,108
Dopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036
Mesolimbic dopamine signals the value of work
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 824
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
Striatal circuits for reward learning and decision-making
Cox, J., & Witten, I. B. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0189-2 · Cited by 568
Wireless Optofluidic Systems for Programmable In Vivo Pharmacology and Optogenetics
Jeong, J.-W., McCall, J. G., Shin, G., Zhang, Y., Al-Hasani, R., Kim, M., Li, S., Sim, J. Y., Jang, K.-I., Shi, Y., et al. (2015) · Cell · doi:10.1016/j.cell.2015.06.058 · Cited by 498
Dorsal raphe neurons signal reward through 5-HT and glutamate
Liu, Z., Zhou, J., Li, Y., Hu, F., Lu, Y., Ma, M., Feng, Q., Zhang, J.-E., Wang, D., Zeng, J., et al. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.010 · Cited by 469
Fields, H. L., & Margolis, E. B. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.01.002 · Cited by 432
Sufficiency of Mesolimbic Dopamine Neuron Stimulation for the Progression to Addiction
Pascoli, V., Terrier, J., Hiver, A., & Lüscher, C. (2015) · Neuron · doi:10.1016/j.neuron.2015.10.017 · Cited by 304
What, If, and When to Move: Basal Ganglia Circuits and Self-Paced Action Initiation
Klaus, A., Alves da Silva, J., & Costa, R. M. (2019) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031033 · Cited by 284
Dopamine Neuron-Specific Optogenetic Stimulation in Rhesus Macaques
Stauffer, W. R., Lak, A., Yang, A., Borel, M., Paulsen, O., Boyden, E. S., & Schultz, W. (2016) · Cell · doi:10.1016/j.cell.2016.08.024 · Cited by 257
Self-citationSimilar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion
Ilango, A., Kesner, A. J., Keller, K. L., Stuber, G. D., Bonci, A., & Ikemoto, S. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1703-13.2014 · Cited by 255
Reward functions of the basal ganglia
Schultz, W. (2016) · Journal of Neural Transmission · doi:10.1007/s00702-016-1510-0 · Cited by 248
Lobo, M. K., Zaman, S., Damez-Werno, D. M., Koo, J. W., Bagot, R. C., DiNieri, J. A., Nugent, A., Finkel, E., Chaudhury, D., Chandra, R., et al. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.1875-13.2013 · Cited by 245
Activity patterns of serotonin neurons underlying cognitive flexibility
Matias, S., Lottem, E., Dugué, G. P., & Mainen, Z. F. (2017) · eLife · doi:10.7554/elife.20552 · Cited by 239
Dopamine release in the nucleus accumbens core signals perceived saliency
Kutlu, M. G., Zachry, J. E., Melugin, P. R., Cajigas, S. A., Chevee, M. F., Kelly, S. J., Kutlu, B., Tian, L., Siciliano, C. A., & Calipari, E. S. (2021) · Current Biology · doi:10.1016/j.cub.2021.08.052 · Cited by 238
Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making
Stopper, C. M., Tse, M. T. L., Montes, D. R., Wiedman, C. R., & Floresco, S. B. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.033 · Cited by 136
Operant self-stimulation of dopamine neurons in the substantia nigra
Rossi, M. A., Sukharnikova, T., Hayrapetyan, V. Y., Yang, L., & Yin, H. H. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065799 · Cited by 119
Dopamine, Prediction Error and Beyond
Diederen, K. M. J., & Fletcher, P. C. (2021) · The Neuroscientist · doi:10.1177/1073858420907591 · Cited by 106
Klanker, M., Sandberg, T., Joosten, R., Willuhn, I., Feenstra, M., & Denys, D. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2015.08.011 · Cited by 37
Bimpisidis, Z., König, N., Stagkourakis, S., Zell, V., Vlcek, B., Dumas, S., Giros, B., Broberger, C., Hnasko, T. S., & Wallén-Mackenzie, Å. (2019) · eNeuro · doi:10.1523/eneuro.0066-19.2019 · Cited by 36
The Functions of Dopamine in Operant Conditioned Reflexes
Maiorov, V. I. (2019) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-019-00815-y · Cited by 10
“Stimulation of optogenetically identifi ed dopamine neurons reinforces the movements induced by stimulation – the animal starts to perform the movement voluntarily on expectation of receipt of stimulation [Tsai et al., 2009; Kim et al., 2012; Rossi et al., 2013; Beier et al., 2015; Pascoli et al., 2015].”
Metadata
- Authors
- Kim, K. M., Baratta, M. V., Yang, A., Lee, D., Boyden, E. S., & Fiorillo, C. D.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-04-10
- Volume
- 7
- Issue
- 4
- Pages
- e33612
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2023916971
- Cited by
- 145
- References
- 37
- FWCI
- 5.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.