Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Background: Delays between actions and their outcomes severely hinder reinforcement learning systems, but little is known of the neural mechanism by which animals overcome this problem and bridge such delays. The nucleus accumbens core (AcbC), part of the ventral striatum, is required for normal preference for a large, delayed reward over a small, immediate reward (self-controlled choice) in rats, but the reason for this is unclear. We investigated the role of the AcbC in learning a free-operant instrumental response using delayed reinforcement, performance of a previously-learned response for delayed reinforcement, and assessment of the relative magnitudes of two different rewards. Results: Groups of rats with excitotoxic or sham lesions of the AcbC acquired an instrumental response with different delays (0, 10, or 20 s) between the lever-press response and reinforcer delivery. A second (inactive) lever was also present, but responding on it was never reinforced. As expected, the delays retarded learning in normal rats. AcbC lesions did not hinder learning in the absence of delays, but AcbC-lesioned rats were impaired in learning when there was a delay, relative to sham-operated controls. All groups eventually acquired the response and discriminated the active lever from the inactive lever to some degree. Rats were subsequently trained to discriminate reinforcers of different magnitudes. AcbC-lesioned rats were more sensitive to differences in reinforcer magnitude than sham-operated controls, suggesting that the deficit in self-controlled choice previously observed in such rats was a consequence of reduced preference for delayed rewards relative to immediate rewards, not of reduced preference for large rewards relative to small rewards. AcbC lesions also impaired the performance of a previously-learned instrumental response in a delay-dependent fashion. Conclusions: These results demonstrate that the AcbC contributes to instrumental learning and performance by bridging delays between subjects' actions and the ensuing outcomes that reinforce behaviour.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Robust Tests for the Equality of Variances
Brown, M. B., & Forsythe, A. B. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10482955 · Cited by 3,216
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708 · cited 1×
“Impulsive choice, one aspect of impulsivity [4], is exemplified by the tendency to choose small rewards that are available immediately instead of larger rewards that are only available after a delay [5,6], and may reflect dysfunction of reinforcement learning systems mediating the effects of delayed rewards [5,7].”
BackgroundHerrnstein, R. J. (1970) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1970.13-243 · Cited by 2,611
RELATIVE AND ABSOLUTE STRENGTH OF RESPONSE AS A FUNCTION OF FREQUENCY OF REINFORCEMENT 1, 2
Herrnstein, R. J. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-267 · Cited by 2,281
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244
Evenden, J. L. (1999) · Psychopharmacology · doi:10.1007/pl00005481 · Cited by 2,044 · cited 1×
“Impulsive choice, one aspect of impulsivity [4], is exemplified by the tendency to choose small rewards that are available immediately instead of larger rewards that are only available after a delay [5,6], and may reflect dysfunction of reinforcement learning systems mediating the effects of delayed rewards [5,7].”
BackgroundGoal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · cited 1×
“The prelimbic area of rat prefrontal cortex is important for the detection of instrumental contingencies and contributes to goal-directed, rather than habitual, action [78,79].”
BackgroundIkemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Neuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427 · cited 1×
“The nucleus accumbens (Acb) responds to anticipated rewards in humans, other primates, and rats [8-15], and is innervated by dopamine (DA) neurons that respond to errors in reward prediction in a manner appropriate for a teaching signal [16-19].”
BackgroundFunctional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · cited 1×
“The dorsal striatum is another such structure [10,15,72,73]; expression of stimulus-response habits requires the dorsal striatum [74,75], and the rate at which rats learn an arbitrary response that delivers electrical stimulation to the substantia nigra is correlated with the degree of potentiation of synapses made by…”
BackgroundModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Motivational control of goal-directed action
Dickinson, A., & Balleine, B. (1994) · Animal Learning & Behavior · doi:10.3758/bf03199951 · Cited by 995 · cited 2×
“It is not presently known to what degree responses acquired with a response-reinforcer delay are governed by declarative processes (the action-outcome contingency plus a representation of the instrumental incentive value of the outcome) or procedural mechanisms (stimulus-response habits), both of which are known to in…”
BackgroundMethodHuynh, H., & Feldt, L. S. (1970) · Journal of the American Statistical Association · doi:10.2307/2284340 · Cited by 945
Reward processing in primate orbitofrontal cortex and basal ganglia
Schultz, W. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.272 · Cited by 942
Impulsive choice induced in rats by lesions of the nucleus accumbens core
Cardinal, R. N., Pennicott, D. R., Lakmali, C., Sugathapala, Robbins, T. W., & Everitt, B. J. (2001) · Science · doi:10.1126/science.1060818 · Cited by 923 · cited 7×
“Finally, the lesions did not impair the perception of relative reward magnitude as assessed by responding on identical concurrent interval schedules for reinforcers of different magnitude, suggesting that the impulsive choice previously exhibited by AcbC-lesioned rats [22] is attributable to deficits in dealing with d…”
BackgroundMethodResultInfluentialSelf-citationA cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879 · cited 1×
“The dorsal striatum is another such structure [10,15,72,73]; expression of stimulus-response habits requires the dorsal striatum [74,75], and the rate at which rats learn an arbitrary response that delivers electrical stimulation to the substantia nigra is correlated with the degree of potentiation of synapses made by…”
BackgroundSalamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817
Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice
Winstanley, C. A., Theobald, D. E. H., Cardinal, R. N., & Robbins, T. W. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.5606-03.2004 · Cited by 675 · cited 1×
“Similarly, the orbitofrontal cortex and basolateral amygdala encode reinforcement information and project to the AcbC, and lesions of these structures can produce impulsive choice see [24,80-82].”
BackgroundSelf-citationIncentive-elicited brain activation in adolescents: Similarities and differences from young adults
Bjork, J. M., Knutson, B., Fong, G. W., Caggiano, D. M., Bennett, S. M., & Hommer, D. W. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4862-03.2004 · Cited by 567
Corbit, L. H., Muir, J. L., & Balleine, B. W. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-09-03251.2001 · Cited by 564 · cited 3×
“It is therefore of interest that when differences between AcbC-lesioned rats and shams have been observed, AcbC-lesioned animals have been found to respond somewhat less than shams on such schedules late in training, when the ratio requirement is high [44,45], consistent with our present results.”
BackgroundResultParkinson, J. A., Olmstead, M. C., Burns, L. H., Robbins, T. W., & Everitt, B. J. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-06-02401.1999 · Cited by 555
Everitt, B. J., Morris, K. A., O'Brien, A., & Robbins, T. W. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90145-e · Cited by 507 · cited 2×
“[23,29,49-52], there was no Pavlovian stimulus that was differentially associated with”
BackgroundThe role of prelimbic cortex in instrumental conditioning
Corbit, L. H., & Balleine, B. W. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.023 · Cited by 423
S., M., S., B., M.-Y., H., C., B., E., S., J., D., & I., A. (2002) · Psychopharmacology · doi:10.1007/s00213-001-0983-0 · Cited by 409
Hall, J., Parkinson, J. A., Connor, T. M., Dickinson, A., & Everitt, B. J. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01577.x · Cited by 355
Altered reinforcement mechanisms in attention-deficit/hyperactivity disorder
Sagvolden, T., Aase, H., Zeiner, P., & Berger, D. (1998) · Behavioural Brain Research · doi:10.1016/s0166-4328(97)00170-8 · Cited by 345
Reward prediction in primate basal ganglia and frontal cortex
Schultz, W., Tremblay, L., & Hollerman, J. R. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00071-9 · Cited by 318
Parkinson, J. A., Willoughby, P. J., Robbins, T. W., & Everitt, B. J. (2000) · Behavioral Neuroscience · doi:10.1037/0735-7044.114.1.42 · Cited by 314
Cousins, M. S., Atherton, A., Turner, L., & Salamone, J. D. (1996) · Behavioural Brain Research · doi:10.1016/0166-4328(95)00151-4 · Cited by 236 · cited 1×
“Similarly, DA depletion of the Acb does not affect the ability to discriminate large from small reinforcers [59-61], and systemic DA antagonists do not affect the perceived”
BackgroundCardinal, R. N., Parkinson, J. A., Lachenal, G., Halkerston, K. M., Rudarakanchana, N., Hall, J., Morrison, C. H., Howes, S. R., Robbins, T. W., & Everitt, B. J. (2002) · Behavioral Neuroscience · doi:10.1037/0735-7044.116.4.553 · Cited by 201 · cited 2×
“[23,29,49-52], there was no Pavlovian stimulus that was differentially associated with”
BackgroundSelf-citationDickinson, A. (1994) · Animal Learning and Cognition · doi:10.1016/b978-0-08-057169-0.50009-7 · Cited by 200
Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens
Hernandez, P. J., Sadeghian, K., & Kelley, A. E. (2002) · Nature Neuroscience · doi:10.1038/nn973 · Cited by 186 · cited 1×
“Furthermore, infusion of a PKA inhibitor [41] or a protein synthesis inhibitor [43] into the AcbC after instrumental training sessions impairs subsequent performance, implying that PKA activity and protein synthesis in the AcbC contribute to the consolidation of instrumental behaviour.”
Background
Cited by
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
“The core is also necessary for instrumental learning when there is a delay between the response and the reinforce”
BackgroundBardo, M. T., & Hammer, R. P. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90226-e · Cited by 3,219
The structural basis of inter-individual differences in human behaviour and cognition
Kanai, R., & Rees, G. (2011) · Nature reviews. Neuroscience · doi:10.1038/nrn3000 · Cited by 1,055
“is consistent with studies showing that rats with hippocampal damage choose immediate rewards in delay-discounting task”
ResultThe Nucleus Accumbens: An Interface Between Cognition, Emotion, and Action
Floresco, S. B. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010213-115159 · Cited by 938
The nucleus accumbens: a comprehensive review
Salgado, S., & Kaplitt, M. G. (2015) · Stereotactic and Functional Neurosurgery · doi:10.1159/000368279 · Cited by 489
Metadata
- Authors
- Cardinal, R. N., & Cheung, T. H.
- Venue
- BMC Neuroscience
- Published
- 2005-02-03
- Volume
- 6
- Issue
- 1
- Pages
- 9-9
- ISSN
- 1471-2202 (ISSN-L)
- OpenAlex
- W1884351292
- Cited by
- 100
- References
- 34
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.