Neurons in the Ventral Striatum Exhibit Cell-Type-Specific Representations of Outcome during Learning
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
SUMMARY The ventromedial striatum (VMS) is a node in circuits underpinning both affect and reinforcement learning. The cellular bases of these functions and especially their potential linkages have been unclear. VMS cholinergic interneurons, however, have been singled out as being related both to affect and to reinforcement-based conditioning, raising the possibility that unique aspects of their signaling could account for these functions. Here we show that VMS tonically active neurons (TANs), including putative cholinergic interneurons, generate unique bidirectional outcome responses during reward-based learning, reporting both positive (reward) and negative (reward omission) outcomes when behavioral change is prompted by switches in reinforcement contingencies. VMS output neurons (SPNs), by contrast, are nearly insensitive to switches in reinforcement contingencies, gradually losing outcome signaling while maintaining responses at trial initiation and goal approach. Thus TANs and SPNs in the VMS provide distinct signals optimized for different aspects of the learning process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
NEURAL MECHANISMS OF ADDICTION: The Role of Reward-Related Learning and Memory
Hyman, S. E., Malenka, R. C., & Nestler, E. J. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113009 · Cited by 2,647
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
The role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · cited 2×
“A remarkable set of findings now suggests that these limbic circuits can be divided into pathways that are specialized for processing either rewarding or aversive outcomes, and that they may be implicated differentially in affective disorders (Bromberg-Martin et al., 2010; Brown et al., 2012; Cohen et al., 2012; Kreit…”
The mesolimbic dopamine reward circuit in depression
Nestler, E. J., & Carlezon, W. A. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.09.018 · Cited by 2,059
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 4×
“This view is consistent with the position of the striatum as a principal recipient of signals from midbrain dopamine-containing neurons that could mediate reinforcement-based learning (Dezfouli and Balleine, 2012; Glascher et al., 2010; Graybiel, 2008; Hyman et al., © 2014 Elsevier Inc.”
InfluentialSelf-citationNeurocircuitry of Mood Disorders
Price, J. L., & Drevets, W. C. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.104 · Cited by 1,606 · cited 2×
“…(Berridge et al., 2010; Brown et al., 2012; Der-Avakian and Markou, 2012; Greening et al., 2013; Hyman et al., 2006; Keedwell et al., 2005; Kreitzer and Malenka, 2008; Nestler and Carlezon, 2006; Price and Drevets, 2010; Tan et al., 2012; Wacker et al., 2009; Warner-Schmidt et al., 2012).”
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · cited 3×
“Behavioral over-training, by this view, leads to a shift towards DLS-controlled habitual behavior that can be executed without regard to changes in the identity or value of the reward that originally motivated the learning (Balleine and Dickinson, 1998; Everitt and Robbins, 2005; Thorn et al., 2010; Yin et al., 2008;…”
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · cited 2×
“P A A uthor M anuscript Matsumoto and Hikosaka, 2009; Morris et al., 2004; Nakahara et al., 2004; Pan et al., 2005; Roesch et al., 2007; Schultz, 2002; Schultz et al., 1993; Schultz et al., 1997).”
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · cited 3×
“…Error Signals The increased firing to reward presentation and pause in firing for reward omission observed in TANs are strikingly reminiscent of the patterns of learning-related modulation classically described for the dopamine-containing neurons of the midbrain (Cohen et al., 2012; Neuron.”
Putting a spin on the dorsal-ventral divide of the striatum
Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.06.006 · Cited by 1,188
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
The neurobiology of anhedonia and other reward-related deficits
Der-Avakian, A., & Markou, A. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2011.11.005 · Cited by 1,130
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099
Dopamine neurons modulate neural encoding and expression of depression-related behaviour
Tye, K. M., Mirzabekov, J. J., Warden, M. R., Ferenczi, E. A., Tsai, H.-C., Finkelstein, J., Kim, S.-Y., Adhikari, A., Thompson, K. R., Andalman, A. S., et al. (2013) · Nature · doi:10.1038/nature11740 · Cited by 1,073 · cited 1×
“P A A uthor M anuscript 2006; Liljeholm and O’Doherty, 2012; Tobler et al., 2003; Wickens et al., 2007; Yin and Knowlton, 2006).”
BackgroundGong, S., Doughty, M., Harbaugh, C. R., Cummins, A., Hatten, M. E., Heintz, N., & Gerfen, C. R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2707-07.2007 · Cited by 977 · cited 2×
“P A A uthor M anuscript (Gong et al., 2007) in which a Cre-dependent AAV5 virus had been injected into the striatum (Figure 1G; Supplemental Experimental Procedures).”
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 2×
“, 2010), and neural signals in the VMS are sensitive both to the actions resulting in rewarded outcomes as well as to the outcomes themselves (Li and Daw, 2011; McClure et al., 2003).”
The tempted brain eats: Pleasure and desire circuits in obesity and eating disorders
Berridge, K. C., Ho, C.-Y., Richard, J. M., & DiFeliceantonio, A. G. (2010) · Brain Research · doi:10.1016/j.brainres.2010.04.003 · Cited by 873
Building neural representations of habits.
Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., & Graybiel, A. M. (1999) · Science · doi:10.1126/science.286.5445.1745 · Cited by 834
Self-citationBelin, D., & Everitt, B. J. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.019 · Cited by 697
The Neural Correlates of Anhedonia in Major Depressive Disorder
Keedwell, P. A., Andrew, C., Williams, S. C. R., Brammer, M. J., & Phillips, M. L. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.05.019 · Cited by 650 · cited 2×
“…of neuropsychiatric conditions (Berridge et al., 2010; Brown et al., 2012; Der-Avakian and Markou, 2012; Greening et al., 2013; Hyman et al., 2006; Keedwell et al., 2005; Kreitzer and Malenka, 2008; Nestler and Carlezon, 2006; Price and Drevets, 2010; Tan et al., 2012; Wacker et al., 2009;…”
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories
Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., & Graybiel, A. M. (2005) · Nature · doi:10.1038/nature04053 · Cited by 595 · cited 3×
“Thus the ensemble activities of all three classes of VMS neurons emphasized mainly the early and late periods of the maze runs, a characteristic previously found for dorsal striatal neurons, especially in the dorsolateral striatum (DLS) (Barnes et al., 2005; Jog et al., 1999; Thorn et al., 2010).”
Self-citationCoincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574 · cited 4×
“P A A uthor M anuscript Matsumoto and Hikosaka, 2009; Morris et al., 2004; Nakahara et al., 2004; Pan et al., 2005; Roesch et al., 2007; Schultz, 2002; Schultz et al., 1993; Schultz et al., 1997).”
BackgroundInfluentialAlterations in 5-HT 1B Receptor Function by p11 in Depression-Like States
Svenningsson, P., Chergui, K., Rachleff, I., Flajolet, M., Zhang, X., Yacoubi, M. E., Vaugeois, J.-M., Nomikos, G. G., & Greengard, P. (2006) · Science · doi:10.1126/science.1117571 · Cited by 572 · cited 1×
“Deletion of p11 or silencing of the cholinergic interneurons precipitates anhedonia-like states in mouse models, and p11 expression is diminished in human depression (Alexander et al., 2010; Svenningsson et al., 2006; Warner-Schmidt et al., 2012).”
Activation of lateral habenula inputs to the ventral midbrain promotes behavioral avoidance
Stamatakis, A. M., & Stuber, G. D. (2012) · Nature Neuroscience · doi:10.1038/nn.3145 · Cited by 564
Physiology and pharmacology of striatal neurons
Kreitzer, A. C. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135422 · Cited by 530 · cited 3×
“(Berke, 2008; Gittis and Kreitzer, 2012; Kawaguchi et al., 1995; Kreitzer, 2009).”
Functional diversity and specificity of neostriatal interneurons
Tepper, J. M., & Bolam, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.003 · Cited by 526
Cholinergic Interneurons Control Local Circuit Activity and Cocaine Conditioning
Witten, I. B., Lin, S.-C., Brodsky, M., Prakash, R., Diester, I., Anikeeva, P., Gradinaru, V., Ramakrishnan, C., & Deisseroth, K. (2010) · Science · doi:10.1126/science.1193771 · Cited by 467 · cited 1×
“Nevertheless, our comparisons of the firing properties of the TANS to the optogenetically confirmed TANs in ChAT-Cre mice suggest that the population of neurons identified as TANs is likely to have a large component of cholinergic interneurons (English et al., 2012; Witten et al., 2010).”
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429 · cited 3×
“P A A uthor M anuscript Matsumoto and Hikosaka, 2009; Morris et al., 2004; Nakahara et al., 2004; Pan et al., 2005; Roesch et al., 2007; Schultz, 2002; Schultz et al., 1993; Schultz et al., 1997).”
Aosaki, T., Graybiel, A. M., & Kimura, M. (1994) · Science · doi:10.1126/science.8023166 · Cited by 427
Self-citationYin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416 · cited 2×
“Behavioral over-training, by this view, leads to a shift towards DLS-controlled habitual behavior that can be executed without regard to changes in the identity or value of the reward that originally motivated the learning (Balleine and Dickinson, 1998; Everitt and Robbins, 2005; Thorn et al., 2010; Yin et al., 2008;…”
Wacker, J., Dillon, D. G., & Pizzagalli, D. A. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.01.058 · Cited by 405 · cited 2×
“In parallel with its key function in the early stages of reinforcement learning, the VMS is centrally implicated in regulating mood and affect and in the etiology of neuropsychiatric conditions (Berridge et al., 2010; Brown et al., 2012; Der-Avakian and Markou, 2012; Greening et al., 2013; Hyman et al., 2006; Keedwell…”
Responses to reward in monkey dorsal and ventral striatum
Apicella, P., Ljungberg, T., Scarnati, E., & Schultz, W. (1991) · Experimental Brain Research · doi:10.1007/bf00231732 · Cited by 369 · cited 2×
“Instead, we found that the SPNs that we identified as exhibiting a selective reward-related response (Apicella et al., 1991; Carelli and Deadwyler, 1994; Taha and Fields, 2005) had a strong reward response during initial acquisition that faded during over-training and did not react abruptly to the cue switch with a re…”
Shabel, S. J., Proulx, C. D., Trias, A., Murphy, R. T., & Malinow, R. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.037 · Cited by 334 · cited 1×
“…outcomes, and that they may be implicated differentially in affective disorders (Bromberg-Martin et al., 2010; Brown et al., 2012; Cohen et al., 2012; Kreitzer and Malenka, 2008; Setlow et al., 2003; Shabel et al., 2012; Stamatakis and Stuber, 2012; Tan et al., 2012; Warner-Schmidt et al., 2012).”
BackgroundContributions of the striatum to learning, motivation, and performance: an associative account
Liljeholm, M., & O’Doherty, J. P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.007 · Cited by 317
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300 · cited 2×
“P A A uthor M anuscript 2006; Liljeholm and O’Doherty, 2012; Tobler et al., 2003; Wickens et al., 2007; Yin and Knowlton, 2006).”
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297 · cited 2×
“This view is consistent with the position of the striatum as a principal recipient of signals from midbrain dopamine-containing neurons that could mediate reinforcement-based learning (Dezfouli and Balleine, 2012; Glascher et al., 2010; Graybiel, 2008; Hyman et al., © 2014 Elsevier Inc.”
Dopaminergic Mechanisms in Actions and Habits: Figure 1.
Wickens, J. R., Horvitz, J. C., Costa, R. M., & Killcross, S. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1671-07.2007 · Cited by 271
Meaningful silences: how dopamine listens to the ACh pause
Cragg, S. J. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.01.003 · Cited by 245
Separate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241 · cited 1×
“Instead, we found that the SPNs that we identified as exhibiting a selective reward-related response (Apicella et al., 1991; Carelli and Deadwyler, 1994; Taha and Fields, 2005) had a strong reward response during initial acquisition that faded during over-training and did not react abruptly to the…”
BackgroundSelf-citationEarly consolidation of instrumental learning requires protein synthesis in the nucleus accumbens
Hernandez, P. J., Sadeghian, K., & Kelley, A. E. (2002) · Nature Neuroscience · doi:10.1038/nn973 · Cited by 186 · cited 1×
“In rodents, lesions and inactivations of the VMS impair the capacity of the animals to learn how to obtain rewards, especially early in training (Atallah et al., 2007; Chang et al., 2012; Everitt and Robbins, 2005; Gill et al., 2010; Hernandez et al., 2002).”
Smith, K. S., Virkud, A., Deisseroth, K., & Graybiel, A. M. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1216264109 · Cited by 173
Self-citationCholinergic interneurons in the nucleus accumbens regulate depression-like behavior
Warner-Schmidt, J. L., Schmidt, E. F., Marshall, J. J., Rubin, A. J., Arango-Lievano, M., Kaplitt, M. G., Ibañez-Tallon, I., Heintz, N., & Greengard, P. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1209293109 · Cited by 163 · cited 9×
“The disruption of VMS cholinergic interneurons has been linked to symptoms of depression, including anhedonia and despair, reflecting a lack of responsiveness to positive or negative outcomes (Warner-Schmidt et al., 2012).”
InfluentialReversal of Depressed Behaviors in Mice by p11 Gene Therapy in the Nucleus Accumbens
Alexander, B., Warner-Schmidt, J., Eriksson, T. M., Tamminga, C., Arango-Lievano, M., Ghose, S., Vernov, M., Stavarache, M., Musatov, S., Flajolet, M., et al. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3001079 · Cited by 140 · cited 3×
“The brain circuits that process reinforcement exert powerful influences over behavior: they enable normal responsiveness to rewarding and aversive stimuli, mediate reinforcementbased learning, and have been implicated as dysfunctional in neuropsychiatric disorders ranging from depression to addictive states (Alexander…”
Greening, S. G., Osuch, E. A., Williamson, P. C., & Mitchell, D. G. V. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst027 · Cited by 99 · cited 4×
“In parallel with its key function in the early stages of reinforcement learning, the VMS is centrally implicated in regulating mood and affect and in the etiology of neuropsychiatric conditions (Berridge et al., 2010; Brown et al., 2012; Der-Avakian and Markou, 2012; Greening et al., 2013; Hyman et al., 2006; Keedwell…”
Influential
Cited by
The striatum: where skills and habits meet
Graybiel, A. M., & Grafton, S. T. (2015) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a021691 · Cited by 463
A Dedicated Population for Reward Coding in the Hippocampus
Gauthier, J. L., & Tank, D. W. (2018) · Neuron · doi:10.1016/j.neuron.2018.06.008 · Cited by 394
“If reward-associated cell axons did reach the ventral striatum, their sequential activation could conceivably (Goldman, 2009) underlie the ramping spike rate that precedes reward (Atallah et al., 2014) and might even contribute to reward prediction error signals of dopaminergic neurons (Schultz, 1998).”
Castro, D. C., & Bruchas, M. R. (2019) · Neuron · doi:10.1016/j.neuron.2019.03.003 · Cited by 290
Gonzales, K. K., & Smith, Y. (2015) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12762 · Cited by 166
New Insights Into Cholinergic Neuron Diversity
Ahmed, N. Y., Knowles, R., & Dehorter, N. (2019) · Frontiers in Molecular Neuroscience · doi:10.3389/fnmol.2019.00204 · Cited by 108
“ACh also attunes motivation (Marche et al., 2017), behavioral flexibility (Aoki et al., 2015, 2018; Okada et al., 2018), and associative learning (Atallah et al., 2014; for review, see Yamanaka et al., 2018).”
Background
Metadata
- Authors
- Atallah, H. E., McCool, A. D., Howe, M. W., & Graybiel, A. M.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2014-06
- Volume
- 82
- Issue
- 5
- Pages
- 1145-1156
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W1976864358
- Cited by
- 100
- References
- 47
- FWCI
- 3.62
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.