Dissociable responses to punishment in distinct striatal regions during reversal learning
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Adaptive behavior depends on the ability to flexibly alter our choices in response to changes in reward and punishment contingencies. One brain region frequently implicated in such behavior is the striatum. However, this region is functionally diverse and there are a number of apparent inconsistencies across previous studies. For instance, how can significant BOLD responses in the ventral striatum during punishment-based reversal learning be reconciled with the frequently demonstrated role of the ventral striatum in reward processing? Here we attempt to address this question by separately examining BOLD responses during reversal learning driven by reward and during reversal learning driven by punishment. We demonstrate simultaneous valence-specific and valence-nonspecific signals in the striatum, with the posterior dorsal striatum responding only to unexpected reward, and the anterior ventral striatum responding to both unexpected punishment as well as unexpected reward. These data help to reconcile conflicting findings from previous studies by showing that distinct regions of the striatum exhibit dissociable responses to punishment during reversal learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 4×
“Whereas activity in theventral striatumin the study by O'Doherty et al. (2004)was reward-signed, our study revealed that this anterior ventral signal was mostly valence-nonspecific (Fig.”
BackgroundResultInfluentialA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“…the ventral striatum has been implicated in the prediction of biologically relevant events, such as rewards and punishments, in Pavlovian paradigms, the dorsal striatum has been implicated in the instrumental control of actions based on these predictions (Montague et al., 1996) (Yin et al., 2006).”
BackgroundAbstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationThe neural basis of loss aversion in decision-making under risk
Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007) · Science · doi:10.1126/science.1134239 · Cited by 1,733 · cited 2×
“fMRI studies have reported greater-than-average activity in the striatum for reward prediction errors alongside less-than-average activity for punishment prediction errors (Kim et al., 2006; Pessiglione et al., 2006; Yacubian et al., 2006; Tom et al., 2007).”
BackgroundHaber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610 · cited 2×
“The regional distribution of activity in the anterior ventral and posterior dorsal/lateral parts of the striatum is reminiscent of known anatomical subdivisions of the striatum (Haber et al., 2000; Voorn et al., 2004), which have been associated withmotivational versus cognitive/ motor aspects of behavior respectively.”
BackgroundDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · cited 3×
“Many midbrain dopamine neurons exclusively exhibit reward-signed coding (Schultz, 2002; Ungless, 2004 but see Brischoux et al., 2009; Matsumoto and Hikosaka, 2009) and baseline firing rates of dopamine neurons are relatively low, leaving little opportunity for decreases in firing to code…”
BackgroundMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 1×
“…but see Brischoux et al., 2009; Matsumoto and Hikosaka, 2009) and baseline firing rates of dopamine neurons are relatively low, leaving little opportunity for decreases in firing to code negatively signed prediction errors (Daw et al., 2002; Bayer and Glimcher, 2005; but see Bayer et al., 2007).”
BackgroundDichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190 · cited 2×
“Frank (2005) has proposed that dopamine-induced impairment in punishment learning reflects disruption of striatal processing associated with punishment and subsequent suppression (NOGO) of actions associated with aversive outcomes (Frank et al., 2004; Shen et al., 2008; Cohen and Frank, 2009).”
BackgroundPutting a spin on the dorsal-ventral divide of the striatum
Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.06.006 · Cited by 1,188
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967 · cited 2×
“The striatum has been linked to aversive processing in animals (Ikemoto and Panksepp, 1999; Horvitz, 2000; Schoenbaum et al., 2003), fMRI studies have demonstrated aversive prediction errors within the striatum (Jensen et al., 2003, 2007; Seymour et al., 2005, 2007a; Delgado et al., 2008) and the…”
BackgroundFrank, M. J. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929052880093 · Cited by 925
Self-citationOpponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
O'Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-21-07931.2003 · Cited by 611
Yacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · cited 1×
“The alternative hypothesis is that it reflects a single valenceindependent signal, for example reflecting a mismatch between expected and actual outcomes (Redgrave et al., 1999; Zink et al., 2003; Jensen et al., 2007; Matsumoto and Hikosaka, 2009).”
BackgroundDirect Activation of the Ventral Striatum in Anticipation of Aversive Stimuli
Jensen, J., McIntosh, A. R., Crawley, A. P., Mikulis, D. J., Remington, G., & Kapur, S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00724-4 · Cited by 455 · cited 1×
“…(Ikemoto and Panksepp, 1999; Horvitz, 2000; Schoenbaum et al., 2003), fMRI studies have demonstrated aversive prediction errors within the striatum (Jensen et al., 2003, 2007; Seymour et al., 2005, 2007a; Delgado et al., 2008) and the effect of dopaminergicmedication in PD on punishment learning…”
BackgroundOpponent appetitive-aversive neural processes underlie predictive learning of pain relief
Seymour, B., O'Doherty, J. P., Koltzenburg, M., Wiech, K., Frackowiak, R., Friston, K., & Dolan, R. (2005) · Nature Neuroscience · doi:10.1038/nn1527 · Cited by 439 · cited 5×
“…Horvitz, 2000; Schoenbaum et al., 2003), fMRI studies have demonstrated aversive prediction errors within the striatum (Jensen et al., 2003, 2007; Seymour et al., 2005, 2007a; Delgado et al., 2008) and the effect of dopaminergicmedication in PD on punishment learning is comparable to the effect…”
BackgroundResultInfluentialHuman striatal response to salient nonrewarding stimuli
Zink, C. F., Pagnoni, G., Martin, M. E., Dhamala, M., & Berns, G. S. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-22-08092.2003 · Cited by 312 · cited 1×
“The alternative hypothesis is that it reflects a single valenceindependent signal, for example reflecting a mismatch between expected and actual outcomes (Redgrave et al., 1999; Zink et al., 2003; Jensen et al., 2007; Matsumoto and Hikosaka, 2009).”
BackgroundSerotonin, inhibition, and negative mood
Dayan, P., & Huys, Q. J. M. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040004 · Cited by 263 · cited 1×
“Although subjects had no control over the outcomes, it is possible that they nevertheless exhibited valence-dependent response biases to aversive or appetitive predictions and outcomes that are immaterial to the delivery of the outcomes (Dayan and Huys, 2008; Dayan and Seymour, 2008).”
BackgroundClarke, H. F., Robbins, T. W., & Roberts, A. C. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1521-08.2008 · Cited by 263 · cited 2×
“Lesions of the striatum impair reversal learning in animals (Divac et al., 1967; Taghzouti et al., 1985; Annett et al., 1989; Stern and Passingham, 1996; Clarke et al., 2008) and increased striatal BOLD signal is seen in humans during the punishment events that lead to reversal (Cools et al.”
BackgroundCools, R., Robinson, O. J., & Sahakian, B. (2008) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301598 · Cited by 172 · cited 1×
“Although other neurotransmitters, like serotonin, should be considered as plausible candidates ((Daw et al., 2002; Cools et al., 2008), it is possible that punishment-related activity in the striatum during reversal learning reflects dopaminergic influences from the midbrain.”
BackgroundSelf-citationREMIJNSE, P., NIELEN, M., UYLINGS, H., & VELTMAN, D. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.02.009 · Cited by 163 · cited 2×
“The network closely resembled that observed previously during instrumental reversal learning (Cools et al., 2002) (O'Doherty et al., 2001; Cools et al., 2002; O'Doherty et al., 2003; O'Doherty et al., 2004; Remijnse et al., 2005; Hampton and O'Doherty, 2007).”
ResultDodds, C. M., Muller, U., Clark, L., van Loon, A., Cools, R., & Robbins, T. W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1153-08.2008 · Cited by 113 · cited 5×
“…impair reversal learning in animals (Divac et al., 1967; Taghzouti et al., 1985; Annett et al., 1989; Stern and Passingham, 1996; Clarke et al., 2008) and increased striatal BOLD signal is seen in humans during the punishment events that lead to reversal (Cools et al., 2002; Dodds et al., 2008).”
BackgroundSelf-citation
Cited by
The integration of negative affect, pain and cognitive control in the cingulate cortex
Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn2994 · Cited by 2,082
Walsh, M. M., & Anderson, J. R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.008 · Cited by 544
The dissociable effects of punishment and reward on motor learning
Galea, J. M., Mallia, E., Rothwell, J., & Diedrichsen, J. (2015) · Nature Neuroscience · doi:10.1038/nn.3956 · Cited by 363
O’Hara, C. B., Campbell, I. C., & Schmidt, U. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.02.012 · Cited by 274
Robinson, O. J., Cools, R., Carlisi, C. O., Sahakian, B. J., & Drevets, W. C. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010137 · Cited by 232
“Consistent with previous versions of this task (8, 11), the same stimulus was highlighted after the unexpected outcome and was presented until participants correctly reversed their predictions.”
BackgroundMethodResultSelf-citation
Metadata
- Authors
- Robinson, O. J., Frank, M. J., Sahakian, B. J., & Cools, R.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2010-07
- Volume
- 51
- Issue
- 4
- Pages
- 1459-1467
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2097509759
- Cited by
- 74
- References
- 27
- FWCI
- 4.53
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.