Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
A critical problem in daily decision making is how to choose actions now in order to bring about rewards later. Indeed, many of our actions have long-term consequences, and it is important to not be myopic in balancing the pros and cons of different options, but rather to take into account both immediate and delayed consequences of actions. Failures to do so may be manifest as persistent, maladaptive decision-making, one example of which is addiction where behavior seems to be driven by the immediate positive experiences with drugs, despite the delayed adverse consequences. A recent study by Takahashi et al. (2007) investigated the effects of cocaine sensitization on decision making in rats and showed that drug use resulted in altered representations in the ventral striatum and the dorsolateral striatum, areas that have been implicated in the neural instantiation of a computational solution to optimal long-term actions selection called the Actor/Critic framework. In this Focus article we discuss their results and offer a computational interpretation in terms of drug-induced impairments in the Critic. We first survey the different lines of evidence linking the subparts of the striatum to the Actor/Critic framework, and then suggest two possible scenarios of breakdown that are suggested by Takahashi et al.'s (2007) data. As both are compatible with the current data, we discuss their different predictions and how these could be empirically tested in order to further elucidate (and hopefully inch towards curing) the neural basis of drug addiction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The functional anatomy of basal ganglia disorders
Albin, R. L., Young, A. B., & Penney, J. B. (1989) · Trends in Neurosciences · doi:10.1016/0166-2236(89)90074-x · Cited by 5,353
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · cited 1×
“The crux of this model is a simple learning rule based on a Temporal Difference (TD) prediction error (Barto et al., 1989; Sutton, 1988; Sutton and Barto, 1990): δ t = r(S t ) + V(S t ) − V(S t−1 ), by which the Critic learns what feedback to give to each action, and the Actor learns an improved…”
MethodDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 2×
“Indeed, within a broader scheme, one might view the basal-ganglia/dopamine system as a system of multiple Actors and Critics operating in parallel, which can collaborate or compete to control behavior (Daw et al., 2005).”
BackgroundFrom motivation to action: Functional interface between the limbic system and the motor system
MOGENSON, G., JONES, D., & YIM, C. (1980) · Progress in Neurobiology · doi:10.1016/0301-0082(80)90018-0 · Cited by 2,331 · cited 1×
“Due to this strategic positioning, the ventral striatum has been likened to a “limbicmotor interface” (Mogenson et al., 1980).”
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“Dopaminergic projections to the striatum are thought to play a critical role in this process by signaling reward prediction errors (Montague et al., 1996; Schultz et al., 1997) and modulating plasticity in cortico-striatal synapses (Wickens, 1990; Wickens et al., 2003).”
BackgroundA neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610 · cited 1×
“…to those dopaminergic neurons that project back to the ventral striatum and to those that project to the more motor-related dorsal striatum (Haber et al., 1990, 2000; Joel and Weiner, 1999, 2000; Lynd-Balta and Haber, 1994; 90 | July 2008 | Volume 2 | Issue 1 www.frontiersin.org Nauta et…”
BackgroundDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292 · cited 1×
“…the two midbrain dopaminergic nuclei – based on state values from ventral striatal afferents, and outcome information from sources such as the pedunculopontine nucleus (PPTN), the habenula etc. (Christoph et al., 1986; Ji and Shepard, 2007; Kobayashi and Okada, 2007; Matsumoto and Hikosaka, 2007).”
BackgroundYin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · cited 1×
“…of habitual responding (Atallah et al., 2007; Bailey and Mair, 2006; Balleine, 2005; Knowlton et al., 1996; Racht-Delatour and El Massioui, 2000; Yin et al., 2004, 2006), and is also the major site of dopamine loss in idiopathic Parkinson’s disease, with its predominantly motorrelated symptoms…”
BackgroundPutting a spin on the dorsal-ventral divide of the striatum
Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.06.006 · Cited by 1,188
Reward processing in primate orbitofrontal cortex and basal ganglia
Schultz, W. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.272 · Cited by 942
Building neural representations of habits.
Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., & Graybiel, A. M. (1999) · Science · doi:10.1126/science.286.5445.1745 · Cited by 834
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748
Drug Seeking Becomes Compulsive After Prolonged Cocaine Self-Administration
Vanderschuren, L. J. M. J., & Everitt, B. J. (2004) · Science · doi:10.1126/science.1098975 · Cited by 733 · cited 1×
“…drugs has been shown to cause maladaptive decision-making in rats performing such tasks (Bechara and Damasio, 2002; Calu et al., 2007; Jentsch et al., 2002; Nelson and Killcross, 2006; Schoenbaum and Setlow, 2005; Schoenbaum et al., 2004; Simon et al., 2007; Vanderschuren and Everitt, 2004).”
BackgroundBelin, D., & Everitt, B. J. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.019 · Cited by 697 · cited 1×
“Finally, recent results show that disconnecting the dopaminergic infl uence of the accumbens core on the dorsolateral striatum impairs the execution of habitual responding for a second order reinforcer predictive of cocaine (Belin and Everitt, 2008).”
ResultImpulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals.
Coffey, S. F., Gudleski, G. D., Saladin, M. E., & Brady, K. T. (2003) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.11.1.18 · Cited by 625 · cited 1×
“For instance, addicts and drug-treated animals show increased sensitivity to delay of reward, behaving more “impulsively”, and also show increased sensitivity to reward magnitude (Coffey et al., 2003; Roesch et al., 2007; Simon et al., 2007).”
BackgroundParallel incentive processing: an integrated view of amygdala function
Balleine, B. W., & Killcross, S. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.03.002 · Cited by 622 · cited 4×
“, 2006), thus placing the Pavlovian “Actor” in value learning areas such as the ventral striatum and the amygdala (Balleine and Killcross, 2006).”
BackgroundInfluentialJoel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614 · cited 1×
“…those dopaminergic neurons that project back to the ventral striatum and to those that project to the more motor-related dorsal striatum (Haber et al., 1990, 2000; Joel and Weiner, 1999, 2000; Lynd-Balta and Haber, 1994; 90 | July 2008 | Volume 2 | Issue 1 www.frontiersin.org Nauta et al., 1978).”
BackgroundOrbitofrontal Cortex and Representation of Incentive Value in Associative Learning
Gallagher, M., McMahan, R. W., & Schoenbaum, G. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-15-06610.1999 · Cited by 593 · cited 4×
“…has been widely used to record in other areas, and robust behavioral and neural effects have been shown with previous pharmacological or lesion manipulations (Gallagher et al., 1999; Saddoris et al., 2005; Schoenbaum et al., 1998, 1999a, 2003; Setlow et al., 2003; Stalnaker et al., 2006, 2007).”
BackgroundInfluentialSchoenbaum, G., Chiba, A. A., & Gallagher, M. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01876.1999 · Cited by 579
Self-citationCorbit, L. H., Muir, J. L., & Balleine, B. W. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-09-03251.2001 · Cited by 564 · cited 2×
“…nucleus accumbens core and shell, putatively involved in general value predictions (as in the Critic) and specifi c outcome values (as is necessary for correct goal-directed responding), respectively, is also in line with current data (Cardinal et al., 2002; Corbit et al., 2001; Hall et al., 2001).”
ResultEveritt, B. J., Morris, K. A., O'Brien, A., & Robbins, T. W. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90145-e · Cited by 507 · cited 2×
“More specifi cally, the ventral striatum has been shown to be involved in reward anticipation, in attributing value to Pavlovian stimuli, and in mediating the ability of the affective value of anticipated outcomes to affect instrumental performance, as in paradigms such as Pavlovian-instrumental transfer and condition…”
BackgroundActor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Self-citationLimbic-striatal interactions in reward-related processes
Robbins, T. W., Cador, M., Taylor, J. R., & Everitt, B. J. (1989) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(89)80025-9 · Cited by 458
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04219.x · Cited by 447 · cited 2×
“…more recent data advocate a refi nement of the Actor to include only the dorsolateral striatum (with the dorsomedial striatum implicated in a different form of goal-directed action selection that cannot be supported computationally by an Actor/Critic architecture; Balleine, 2005; Yin et al., 2005).”
BackgroundResultAmphetamine Exposure Enhances Habit Formation
Nelson, A., & Killcross, S. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4305-05.2006 · Cited by 445
Joel, D., & Weiner, I. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90536-3 · Cited by 430
The role of prelimbic cortex in instrumental conditioning
Corbit, L. H., & Balleine, B. W. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.023 · Cited by 423
Christoph, G., Leonzio, R., & Wilcox, K. (1986) · The Journal of Neuroscience · doi:10.1523/jneurosci.06-03-00613.1986 · Cited by 418
Ji, H., & Shepard, P. D. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.0958-07.2007 · Cited by 388 · cited 2×
“(Christoph et al., 1986; Ji and Shepard, 2007; Kobayashi and Okada, 2007; Matsumoto and Hikosaka, 2007).”
BackgroundHall, J., Parkinson, J. A., Connor, T. M., Dickinson, A., & Everitt, B. J. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01577.x · Cited by 355
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326
Neural mechanisms of reward-related motor learning
Wickens, J. R., Reynolds, J. N., & Hyland, B. I. (2003) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2003.10.013 · Cited by 284 · cited 2×
“Dopaminergic projections to the striatum are thought to play a critical role in this process by signaling reward prediction errors (Montague et al., 1996; Schultz et al., 1997) and modulating plasticity in cortico-striatal synapses (Wickens, 1990; Wickens et al., 2003).”
BackgroundSeparate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241 · cited 3×
“…which the ventral subdivision of the striatum (including the nucleus accumbens) embodies the value learning part of the Critic that infl uences dopaminergic prediction errors, and the dorsal striatum is the Actor (Joel et al., 2002; but see Atallah et al., 2007 for a slightly different suggestion).”
BackgroundNeuronal responses in the ventral striatum of the behaving macaque
WILLIAMS, G., ROLLS, E., LEONARD, C., & STERN, C. (1993) · Behavioural Brain Research · doi:10.1016/0166-4328(93)90120-f · Cited by 146 · cited 2×
“Still, prominent in these are representations of outcome anticipation in ventral striatal areas (Schultz et al., 1992; Tremblay et al., 1998; Williams et al., 1993) and those of actions and of action “chunks” in the dorsal striatum (Kimura, 1986, 1995; Jog et al.”
Background
Cited by
Inside the brain of an elite athlete: the neural processes that support high achievement in sports
Yarrow, K., Brown, P., & Krakauer, J. W. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2672 · Cited by 637
Flexible Cognitive Strategies during Motor Learning
Taylor, J. A., & Ivry, R. B. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001096 · Cited by 352
“An alternative approach to model strategy change could be derived from models of reinforcement learning [34–37].”
MethodToward a neurobiology of delusions
Corlett, P. R., Taylor, J. R., Wang, X.-J., Fletcher, P. C., & Krystal, J. H. (2010) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2010.06.007 · Cited by 394
Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry
Keiflin, R., & Janak, P. H. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.037 · Cited by 382
From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making
Burton, A. C., Nakamura, K., & Roesch, M. R. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.05.003 · Cited by 238
Metadata
- Authors
- Takahashi, Y.
- Venue
- Frontiers in Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2008-07-15
- Volume
- 2
- Issue
- 1
- Pages
- 86-99
- ISSN
- 1662-453X (ISSN-L)
- 1662-4548
- OpenAlex
- W2013790022
- Cited by
- 117
- References
- 38
- FWCI
- 1.81
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.