Limbic Corticostriatal Systems and Delayed Reinforcement
The DOI matches its Crossref record; 131 of the 131 references it cites were found in a registry or an index.
Abstract
A bstract: Impulsive choice, one aspect of impulsivity, is characterized by an abnormally high preference for small, immediate rewards over larger delayed rewards, and can be a feature of adolescence, but also attention‐deficit/hyperactivity disorder (ADHD), addiction, and other neuropsychiatric disorders. Both the serotonin and dopamine neuromodulator systems are implicated in impulsivity; manipulations of these systems affect animal models of impulsive choice, though these effects may depend on the receptor subtype and whether or not the reward is signaled. These systems project to limbic cortical and striatal structures shown to be abnormal in animal models of ADHD. Damage to the nucleus accumbens core (AcbC) causes rats to exhibit impulsive choice. These rats are also hyperactive, but are unimpaired in tests of visuospatial attention; they may therefore represent an animal model of the hyperactive‐impulsive subtype of ADHD. Lesions to the anterior cingulate or medial prefrontal cortex, two afferents to the AcbC, do not induce impulsive choice, but lesions of the basolateral amygdala do, while lesions to the orbitofrontal cortex have had opposite effects in different tasks measuring impulsive choice. In theory, impulsive choice may emerge as a result of abnormal processing of the magnitude of rewards, or as a result of a deficit in the effects of delayed reinforcement. Recent evidence suggests that AcbC‐lesioned rats perceive reward magnitude normally, but exhibit a selective deficit in learning instrumental responses using delayed reinforcement, suggesting that the AcbC is a reinforcement learning system that mediates the effects of delayed rewards.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708
The Psychology of Animal Learning
Hill, W. F., & Mackintosh, N. J. (1976) · The American Journal of Psychology · doi:10.2307/1421765 · Cited by 2,388
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129
Self-citationEvenden, J. L. (1999) · Psychopharmacology · doi:10.1007/pl00005481 · Cited by 2,044
Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making
Bechara, A., Damasio, H., Damasio, A. R., & Lee, G. P. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-13-05473.1999 · Cited by 2,001
An adjusting procedure for studying delayed reinforcement.
Commons, M. L., Mazur, J., Nevin, J. A., & Rachlin, H. C. (1987) · doi:10.4324/9781315825502-11 · Cited by 1,861
Mowrer, O. H. (1960) · doi:10.1037/10802-000 · Cited by 1,670
Adolescent Brain Development: A Period of Vulnerabilities and Opportunities. Keynote Address
DAHL, R. E. (2004) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1308.001 · Cited by 1,667
Gonzalez, R. C., & Mowrer, O. H. (1962) · The American Journal of Psychology · doi:10.2307/1419634 · Cited by 1,583
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Cited by
Inhibition and impulsivity: behavioral and neural basis of response control
Bari, A., & Robbins, T. W. (2013) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2013.06.005 · Cited by 1,921
Self-citationCortico-Basal Ganglia Reward Network: Microcircuitry
Sesack, S. R., & Grace, A. A. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.93 · Cited by 985
“Deficits in reward-related activity are central to the anhedonia of depression (Hyman et al, 2006), and altered stimulus valuation is also a known component of attention-deficit hyperactivity disorder and obsessive-compulsive disorder (Cardinal et al, 2004; Everitt et al, 2008; Huey et al, 2008).”
Impulsivity, frontal lobes and risk for addiction
Crews, F. T., & Boettiger, C. A. (2009) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2009.04.018 · Cited by 679
“Impulsive choice is one aspect of impulsivity that involves the choice of small, sooner rewards over larger, delayed rewards (Cardinal et al., 2004).”
BackgroundREVIEW: Behavioural assessment of drug reinforcement and addictive features in rodents: an overview
Sanchis‐Segura, C., & Spanagel, R. (2006) · Addiction Biology · doi:10.1111/j.1369-1600.2006.00012.x · Cited by 629
The IMAGEN study: reinforcement-related behaviour in normal brain function and psychopathology
Schumann, G., Loth, E., Banaschewski, T., Barbot, A., Barker, G., Büchel, C., Conrod, P. J., Dalley, J. W., Flor, H., Gallinat, J., et al. (2010) · Molecular Psychiatry · doi:10.1038/mp.2010.4 · Cited by 626
Self-citationAlcaro, A., Huber, R., & Panksepp, J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.07.014 · Cited by 616
Decision making, impulsivity and time perception
Wittmann, M., & Paulus, M. P. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.10.004 · Cited by 602
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Eagle, D. M., Bari, A., & Robbins, T. W. (2008) · Psychopharmacology · doi:10.1007/s00213-008-1127-6 · Cited by 548
Self-citationThe neurobiology and genetics of impulse control disorders: Relationships to drug addictions
Brewer, J. A., & Potenza, M. N. (2008) · Biochemical Pharmacology · doi:10.1016/j.bcp.2007.06.043 · Cited by 511
Neurobehavioral mechanisms of impulsivity: fronto-striatal systems and functional neurochemistry
Dalley, J. W., Mar, A. C., Economidou, D., & Robbins, T. W. (2008) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2007.12.021 · Cited by 467
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464
Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine
Salamone, J., Correa, M., Mingote, S., & Weber, S. (2005) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2004.09.004 · Cited by 458
Dissociable neural representations of future reward magnitude and delay during temporal discounting
Ballard, K., & Knutson, B. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.11.004 · Cited by 453
Rodent Models of Attention-Deficit/Hyperactivity Disorder
Sagvolden, T., Russell, V. A., Aase, H., Johansen, E. B., & Farshbaf, M. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.02.002 · Cited by 448
Forbes, E. E., Brown, S. M., Kimak, M., Ferrell, R. E., Manuck, S. B., & Hariri, A. R. (2009) · Molecular Psychiatry · doi:10.1038/sj.mp.4002086 · Cited by 408
Stop-Signal Reaction-Time Task Performance: Role of Prefrontal Cortex and Subthalamic Nucleus
Eagle, D. M., Baunez, C., Hutcheson, D. M., Lehmann, O., Shah, A. P., & Robbins, T. W. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm044 · Cited by 404
“Although IL lesions were not tested on this task per se, a combined IL/PL (medial prefrontal cortex) lesion had very different effects on performance of this task, rendering the rats generally insensitive to delay (Cardinal et al. 2004).”
BackgroundSelf-citationProspect theory on the brain? Toward a cognitive neuroscience of decision under risk
Trepel, C., Fox, C. R., & Poldrack, R. A. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.01.016 · Cited by 404
Seidman, L. J., Valera, E. M., Makris, N., Monuteaux, M. C., Boriel, D. L., Kelkar, K., Kennedy, D. N., Caviness, V. S., Bush, G., Aleardi, M., et al. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.04.031 · Cited by 382
Role of dopamine in the motivational and cognitive control of behavior
Cools, R. (2008) · The Neuroscientist · doi:10.1177/1073858408317009 · Cited by 352
Clarke, H., Walker, S., Dalley, J., Robbins, T., & Roberts, A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj120 · Cited by 351
Self-citationSonuga-Barke, E. J. S., Sergeant, J. A., Nigg, J., & Willcutt, E. (2008) · Child and Adolescent Psychiatric Clinics of North America · doi:10.1016/j.chc.2007.11.008 · Cited by 348
Tripp, G., & Wickens, J. R. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01851.x · Cited by 368
Reward, Addiction, and Emotion Regulation Systems Associated With Rejection in Love
Fisher, H. E., Brown, L. L., Aron, A., Strong, G., & Mashek, D. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00784.2009 · Cited by 328
Risk Taking and Novelty Seeking in Adolescence: Introduction to Part I
KELLEY, A. E., SCHOCHET, T., & LANDRY, C. F. (2004) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1308.003 · Cited by 325
Everitt, B. J. (2014) · European Journal of Neuroscience · doi:10.1111/ejn.12644 · Cited by 318
“Perhaps less widely appreciated is the importance of both the BLA and the NAcb core in mediating delays to reinforcement, a process that is markedly facilitated by the presentation of conditioned reinforcers (Cardinal et al., 2004; Winstanley et al., 2004).”
BackgroundSelf-citationEscalated aggressive behavior: Dopamine, serotonin and GABA
de Almeida, R. M. M., Ferrari, P. F., Parmigiani, S., & Miczek, K. A. (2005) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2005.10.004 · Cited by 305
Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation
Roesch, M. R., Taylor, A. R., & Schoenbaum, G. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.027 · Cited by 297
“…regions disrupts the breakpoint at which subjects choose the small reward over the larger delayed reward, typically causing animals to behave more impulsively (i.e., choosing the small immediate reward at shorter delays) (Cardinal et al., 2001, 2004; Mobini et al., 2002; Winstanley et al., 2004).”
InfluentialThe neurobiology of individual differences in complex behavioral traits
Hariri, A. R. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135335 · Cited by 273
Eagle, D. M., & Baunez, C. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.07.003 · Cited by 280
Heidbreder, C. A., & Newman, A. H. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.05149.x · Cited by 273
Neural antecedents of self-initiated actions in secondary motor cortex
Murakami, M., Vicente, M. I., Costa, G. M., & Mainen, Z. F. (2014) · Nature Neuroscience · doi:10.1038/nn.3826 · Cited by 270
Christakou, A., Brammer, M., & Rubia, K. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.08.067 · Cited by 266
“…implicate limbic frontostriatal circuitry in the regulation of reward discounting, with dopaminergic and serotonergic mechanisms playing a key role (Cardinal et al., 2001, 2004; Kheramin et al., 2004; Kobayashi and Schultz, 2008; Mobini et al., 2002; Schweighofer et al., 2008; Winstanley et al.,…”
Dopamine, Time, and Impulsivity in Humans
Pine, A., Shiner, T., Seymour, B., & Dolan, R. J. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6028-09.2010 · Cited by 300
“…preference for smaller–sooner over larger–later rewards, due to excessive discounting of future rewards (Ainslie, 1975; Evenden, 1999; Ho et al., 1999; Cardinal et al., 2004)—is much harder to account for in terms of learning, although it remains an important feature of putative dopaminergic…”
Posner, J., Park, C., & Wang, Z. (2014) · Neuropsychology Review · doi:10.1007/s11065-014-9251-z · Cited by 257
“…lines of evidence, including human neuroimaging studies and studies of animal models, suggest that impulsivity involves functional and anatomical abnormalities within the CSTC loops, particularly the cognitive and limbic CSTC loops (Casey et al. 1997a, b; Cardinal et al. 2004; Marsh et al. 2009).”
BackgroundWittmann, M., Leland, D. S., & Paulus, M. P. (2007) · Experimental Brain Research · doi:10.1007/s00221-006-0822-y · Cited by 255
Field, M., Santarcangelo, M., Sumnall, H., Goudie, A., & Cole, J. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0385-4 · Cited by 250
“2005) and regions of the brain (e.g. the orbitofrontal cortex and basolateral amygdala; Cardinal et al. 2004) have also been implicated in delay discounting, but the present behavioural study does not permit an evaluation of the neuronal mechanisms that may have mediated the observed effects of nicotine deprivation.”
BackgroundDelay discounting in schizophrenia
Heerey, E. A., Robinson, B. M., McMahon, R. P., & Gold, J. M. (2007) · Cognitive Neuropsychiatry · doi:10.1080/13546800601005900 · Cited by 248
“Indeed, lesions of the nucleus accumbens core have been found to increase myopic choice (Cardinal, Winstanley, Robbins, & Everitt, 2004 ).”
BackgroundNeural systems implicated in delayed and probabilistic reinforcement
Cardinal, R. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.004 · Cited by 248
Self-citationAnimal models of attention-deficit hyperactivity disorder
Russell, V. A., Sagvolden, T., & Johansen, E. B. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-9 · Cited by 234
Myopic Discounting of Future Rewards after Medial Orbitofrontal Damage in Humans
Sellitto, M., Ciaramelli, E., & di Pellegrino, G. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2516-10.2010 · Cited by 232
“Several nonhuman animal studies have shown abnormal TD following lesion to the OFC (Cardinal et al., 2004; Winstanley et al., 2004; Rudebeck et al., 2006), providing evidence for a causal involvement in TD.”
BackgroundInfluentialCao, X., Cao, Q., Long, X., Sun, L., Sui, M., Zhu, C., Zuo, X., Zang, Y., & Wang, Y. (2009) · Brain Research · doi:10.1016/j.brainres.2009.08.029 · Cited by 200
Impaired time perception and motor timing in stimulant-dependent subjects
Wittmann, M., Leland, D. S., Churan, J., & Paulus, M. P. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2007.03.005 · Cited by 199
“Dopaminergic systems in the striatum and prefrontal cortex are also thought to be involved in attention, working memory and impulse control (Arnsten and Li, 2005; Cardinal et al., 2004; Evenden, 1999; Paulus et al., 2002).”
BackgroundRoesch, M. R., Singh, T., Brown, P. L., Mullins, S. E., & Schoenbaum, G. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2572-09.2009 · Cited by 190
“Lesions or other manipulations of VS make animals more likely to abandon a larger, delayed or higher cost reward in favor of a smaller, more immediate or lower cost reward (Cousins et al., 1996; Cardinal et al., 2001; Cardinal et al., 2004; Winstanley et al., 2004; Bezzina et al., 2007; Floresco et al., 2008; Kalensch…”
BackgroundAndrews, M. M., Meda, S. A., Thomas, A. D., Potenza, M. N., Krystal, J. H., Worhunsky, P., Stevens, M. C., O'Malley, S., Book, G. A., Reynolds, B., et al. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.09.049 · Cited by 181
The neural basis of delay discounting: A review and preliminary model
Frost, R., & McNaughton, N. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.04.022 · Cited by 172
“…delayed rewards (Cardinal et al., 2004) Amygdala, basolateral Rat (+) k (Winstanley et al., 2004) Rat (+) Impairment in choosing delayed rewards (Cardinal et al., 2004; Winstanley et al., 2004) Prefrontal Cortex, medial Rat (+) Preference for smaller reward when both rewards have zero delay;…”
BackgroundInfluentialEncoding of Marginal Utility across Time in the Human Brain
Pine, A., Seymour, B., Roiser, J. P., Bossaerts, P., Friston, K. J., Curran, H. V., & Dolan, R. J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1126-09.2009 · Cited by 169
TOWARDS A GENERAL MODEL OF TEMPORAL DISCOUNTING
van den Bos, W., & McClure, S. M. (2013) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.6 · Cited by 149
The bivalent side of the nucleus accumbens
Levita, L., Hare, T. A., Voss, H. U., Glover, G., Ballon, D. J., & Casey, B. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.09.039 · Cited by 117
Cho, S. S., Koshimori, Y., Aminian, K., Obeso, I., Rusjan, P., Lang, A. E., Daskalakis, Z. J., Houle, S., & Strafella, A. P. (2015) · Neuropsychopharmacology · doi:10.1038/npp.2014.211 · Cited by 112
Ziegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
Impulsivity and chronic stress are associated with amphetamine-induced striatal dopamine release
Oswald, L. M., Wong, D. F., Zhou, Y., Kumar, A., Brasic, J., Alexander, M., Ye, W., Kuwabara, H., Hilton, J., & Wand, G. S. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.01.055 · Cited by 108
Adriani, W., Boyer, F., Gioiosa, L., Macrì, S., Dreyer, J.-L., & Laviola, G. (2009) · Neuroscience · doi:10.1016/j.neuroscience.2008.11.042 · Cited by 97
Andrzejewski, M. E., Schochet, T. L., Feit, E. C., Harris, R., Mckee, B. L., & Kelley, A. E. (2011) · Behavioral Neuroscience · doi:10.1037/a0022038 · Cited by 96
Dopamine neurons learn to encode the long-term value of multiple future rewards
Enomoto, K., Matsumoto, N., Nakai, S., Satoh, T., Sato, T. K., Ueda, Y., Inokawa, H., Haruno, M., & Kimura, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014457108 · Cited by 83
Mindfulness impairments in individuals seeking treatment for substance use disorders
Dakwar, E., Mariani, J. P., & Levin, F. R. (2011) · The American Journal of Drug and Alcohol Abuse · doi:10.3109/00952990.2011.553978 · Cited by 55
Metadata
- Authors
- CARDINAL, R. N., WINSTANLEY, C. A., ROBBINS, T. W., & EVERITT, B. J.
- Venue
- Annals of the New York Academy of Sciences
- Publisher
- Wiley
- Published
- 2004-06
- Volume
- 1021
- Issue
- 1
- Pages
- 33-50
- ISSN
- 0077-8923 (ISSN-L)
- 1749-6632
- OpenAlex
- W1990385430
- Cited by
- 265
- References
- 131
- FWCI
- 10.72
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.