CoChat
Sign Up
Cited by 248

The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.

Neural systems implicated in delayed and probabilistic reinforcement

Cardinal, Rudolf N.·2006·Neural Networks
DOI 10.1016/j.neunet.2006.03.004
Open this paper in CoChat — free account

Abstract

This review considers the theoretical problems facing agents that must learn and choose on the basis of reward or reinforcement that is uncertain or delayed, in implicit or procedural (stimulus-response) representational systems and in explicit or declarative (action-outcome-value) representational systems. Individual differences in sensitivity to delays and uncertainty may contribute to impulsivity and risk taking. Learning and choice with delayed and uncertain reinforcement are related but in some cases dissociable processes. The contributions to delay and uncertainty discounting of neuromodulators including serotonin, dopamine, and noradrenaline, and of specific neural structures including the nucleus accumbens core, nucleus accumbens shell, orbitofrontal cortex, basolateral amygdala, anterior cingulate cortex, medial prefrontal (prelimbic/infralimbic) cortex, insula, subthalamic nucleus, and hippocampus are examined.

Overview

Integrity signals

Registry

Crossref · 10.1016/j.neunet.2006.03.004

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 46 references

Research works printed but not linked: 288

  1. Amiez, 2005, Anterior cingulate error-related activity is modulated by predicted reward, European Journal of Neuroscience, 21, 3447, 10.1111/j.1460-9568.2005.04170.x · DOI
  2. Amiez, C., Joseph, J. P., & Procyk, E. (2005b). Reward encoding in the monkey anterior cingulate cortex [Advance Access ahead of print; doi:10.1093/cercor/bhj046]. Cerebral Cortex · DOI
  3. Anagnostaras, 1999, Temporally graded retrograde amnesia of contextual fear after hippocampal damage in rats: Within-subjects examination, Journal of Neuroscience, 19, 1106, 10.1523/JNEUROSCI.19-03-01106.1999 · DOI
  4. Anderson, 2003, The effect of acute tryptophan depletion on probabilistic choice, Journal of Psychopharmacology, 17, 3, 10.1177/0269881103017001687 · DOI
  5. Apter, 1993, Anxiety, impulsivity and depressed mood in relation to suicidal and violent behavior, Acta Psychiatrica Scandinavica, 87, 1, 10.1111/j.1600-0447.1993.tb03321.x · DOI
  6. Aron, 2004, Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning, Journal of Neurophysiology, 92, 1144, 10.1152/jn.01209.2003 · DOI
  7. Åsberg, 1997, Neurotransmitters and suicidal behavior. The evidence from cerebrospinal fluid studies, Annals of the New York Academy of Sciences, 836, 158, 10.1111/j.1749-6632.1997.tb52359.x · DOI
  8. Åsberg, 1976, 5-HIAA in the cerebrospinal fluid: A biochemical suicide predictor, Archives of General Psychiatry, 33, 1193, 10.1001/archpsyc.1976.01770100055005 · DOI
  9. Bäckman, 2000, Changes of cerebrospinal fluid monoamine metabolites during long-term antidepressant treatment, European Neuropsychopharmacology, 10, 341, 10.1016/S0924-977X(00)00089-4 · DOI
  10. Balleine, 1994, Effects of ibotenic acid lesions of the nucleus accumbens on instrumental action, Behavioural Brain Research, 65, 181, 10.1016/0166-4328(94)90104-X · DOI
  11. Balleine, 1998, Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates, Neuropharmacology, 37, 407, 10.1016/S0028-3908(98)00033-1 · DOI · Paper page
  12. Bassareo, 1999, Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments, Neuroscience, 89, 637, 10.1016/S0306-4522(98)00583-1 · DOI
  13. Basso, 1999, Feeding induced by GABA(A) receptor stimulation within the nucleus accumbens shell: Regional mapping and characterization of macronutrient and taste preference, Behavioral Neuroscience, 113, 324, 10.1037/0735-7044.113.2.324 · DOI
  14. Baunez, 2001, Effects of STN lesions on simple vs choice reaction time tasks in the rat: Preserved motor readiness, but impaired response selection, European Journal of Neuroscience, 13, 1609, 10.1046/j.0953-816x.2001.01521.x · DOI
  15. Baunez, 1997, Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats, European Journal of Neuroscience, 9, 2086, 10.1111/j.1460-9568.1997.tb01376.x · DOI
  16. Bechara, 1997, Deciding advantageously before knowing the advantageous strategy, Science, 275, 1293, 10.1126/science.275.5304.1293 · DOI · Paper page
  17. Bechara, 1996, Failure to respond autonomically to anticipated future outcomes following damage to prefrontal cortex, Cerebral Cortex, 6, 215, 10.1093/cercor/6.2.215 · DOI · Paper page
  18. Biggio, 1974, Rapid depletion of serum tryptophan, brain tryptophan, serotonin and 5-hydroxyindoleacetic acid by a trytophan-free diet, Life Sciences, 14, 1321, 10.1016/0024-3205(74)90440-8 · DOI
  19. Bizot, 1999, Serotonin and tolerance to delay of reward in rats, Psychopharmacology, 146, 400, 10.1007/PL00005485 · DOI
  20. Bradshaw, 1992, Choice between delayed reinforcers in a discrete-trials schedule—the effect of deprivation level, Quarterly Journal of Experimental Psychology, Section B—Comparative and Physiological Psychology, 44B, 1
  21. Breiter, 2001, Functional imaging of neural responses to expectancy and experience of monetary gains and losses, Neuron, 30, 619, 10.1016/S0896-6273(01)00303-8 · DOI · Paper page
  22. Brog, 1993, The patterns of afferent innervation of the core and shell in the “accumbens” part of the rat ventral striatum: Immunohistochemical detection of retrogradely transported fluoro-gold, Journal of Comparative Neurology, 338, 255, 10.1002/cne.903380209 · DOI · Paper page
  23. Brown, 1990, CSF serotonin metabolite (5HIAA) studies in depression, impulsivity and violence, Journal of Clinical Psychiatry, 51, 31
  24. Brown, 1995, Discriminative cues indicating reward magnitude continue to determine reaction time of rats following lesions of the nucleus accumbens, European Journal of Neuroscience, 7, 2479, 10.1111/j.1460-9568.1995.tb01046.x · DOI
  25. Burk, 2001, Effects of dorsal and ventral striatal lesions on delayed matching trained with retractable levers, Behavioural Brain Research, 122, 67, 10.1016/S0166-4328(01)00169-3 · DOI
  26. Bush, 1999, Anterior cingulate cortex dysfunction in attention-deficit/hyperactivity disorder revealed by fMRI and the Counting Stroop, Biological Psychiatry, 45, 1542, 10.1016/S0006-3223(99)00083-9 · DOI · Paper page
  27. Bussey, 1997, Dissociable effects of cingulate and medial frontal cortex lesions on stimulus–reward learning using a novel Pavlovian autoshaping procedure for the rat: Implications for the neurobiology of emotion, Behavioral Neuroscience, 111, 908, 10.1037/0735-7044.111.5.908 · DOI
  28. Bussey, 1997, Triple dissociation of anterior cingulate, posterior cingulate, and medial frontal cortices on visual discrimination tasks using a touchscreen testing procedure for the rat, Behavioral Neuroscience, 111, 920, 10.1037/0735-7044.111.5.920 · DOI
  29. Cardinal, 2005, Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat, BMC Neuroscience, 6, 9, 10.1186/1471-2202-6-9 · DOI
  30. Cardinal, 2005, Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats, BMC Neuroscience, 6, 37, 10.1186/1471-2202-6-37 · DOI
  31. Cardinal, 2003, Role of the anterior cingulate cortex in the control over behaviour by Pavlovian conditioned stimuli in rats, Behavioral Neuroscience, 117, 566, 10.1037/0735-7044.117.3.566 · DOI
  32. Cardinal, 2002, Emotion and motivation: The role of the amygdala, ventral striatum, and prefrontal cortex, Neuroscience and Biobehavioral Reviews, 26, 321, 10.1016/S0149-7634(02)00007-6 · DOI · Paper page
  33. Cardinal, 2001, Impulsive choice induced in rats by lesions of the nucleus accumbens core, Science, 292, 2499, 10.1126/science.1060818 · DOI · Paper page
  34. Cardinal, 2000, The effects of d-amphetamine, chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats, Psychopharmacology, 152, 362, 10.1007/s002130000536 · DOI
  35. Cardinal, 2004, Limbic corticostriatal systems and delayed reinforcement, Annals of the New York Academy of Sciences, 1021, 33, 10.1196/annals.1308.004 · DOI
  36. Carey, 1998, Differential distribution, affinity and plasticity of dopamine D-1 and D-2 receptors in the target sites of the mesolimbic system in an animal model of ADHD, Behavioural Brain Research, 94, 173, 10.1016/S0166-4328(97)00178-2 · DOI
  37. Carpenter, 1998, Tryptophan depletion during continuous CSF sampling in healthy human subjects, Neuropsychopharmacology, 19, 26, 10.1016/S0893-133X(97)00198-X · DOI
  38. Charrier, 1996, Effects of psychotropic drugs on rat responding in an operant paradigm involving choice between delayed reinforcers, Pharmacology, Biochemistry and Behavior, 54, 149, 10.1016/0091-3057(95)02114-0 · DOI
  39. Chen, 1996, Hippocampal lesions impair contextual fear conditioning in two strains of mice, Behavioral Neuroscience, 110, 1177, 10.1037/0735-7044.110.5.1177 · DOI · Paper page
  40. Cheung, 2005, Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats, BMC Neuroscience, 6, 36, 10.1186/1471-2202-6-36 · DOI
  41. Christakou, 2004, Prefrontal cortical–ventral striatal interactions involved in affective modulation of attentional performance: Implications for corticostriatal circuit function, Journal of Neuroscience, 24, 773, 10.1523/JNEUROSCI.0949-03.2004 · DOI
  42. Clark, 2003, The contributions of lesion laterality and lesion volume to decision-making impairment following frontal lobe damage, Neuropsychologia, 41, 1474, 10.1016/S0028-3932(03)00081-2 · DOI
  43. Clark, 2005, Stop signal response inhibition is not modulated by tryptophan depletion or the serotonin transporter polymorphism in healthy volunteers: Implications for the 5-HT theory of impulsivity, Psychopharmacology, 182, 570, 10.1007/s00213-005-0104-6 · DOI · Paper page
  44. Clemens, 1980, The effect of a tryptophan-free diet on prolactin and corticosterone release by serotonergic stimuli, Hormone and Metabolic Research, 12, 35, 10.1055/s-2007-996191 · DOI
  45. Coccaro, 1995, The neuropsychopharmacology of personality disorders, vol. 1567–1579
  46. Coffey, 2003, Impulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals, Experimental and Clinical Psychopharmacology, 11, 18, 10.1037/1064-1297.11.1.18 · DOI · Paper page
  47. Cole, 1989, Effects of 6-hydroxydopamine lesions of the nucleus accumbens septi on performance of a 5-choice serial reaction time task in rats: Implications for theories of selective attention and arousal, Behavioural Brain Research, 33, 165, 10.1016/S0166-4328(89)80048-8 · DOI
  48. Coleman-Mesches, 1996, Unilateral amygdala inactivation after training attenuates memory for reduced reward, Behavioural Brain Research, 77, 175, 10.1016/0166-4328(95)00231-6 · DOI
  49. Cools, 2005, Tryptophan depletion disrupts the motivational guidance of goal-directed behavior as a function of trait impulsivity, Neuropsychopharmacology, 30, 1362, 10.1038/sj.npp.1300704 · DOI
  50. Cooper, 1992, 5-Hydroxyindoleacetic acid in cerebrospinal fluid and prediction of suicidal behaviour in schizophrenia, Lancet, 340, 940, 10.1016/0140-6736(92)92819-2 · DOI
  51. Corruble, 2003, Understanding impulsivity in severe depression? A psychometrical contribution, Progress in Neuro-Psychopharmacology and Biological Psychiatry, 27, 829, 10.1016/S0278-5846(03)00115-5 · DOI
  52. Cousins, 1996, Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task, Behavioural Brain Research, 74, 189, 10.1016/0166-4328(95)00151-4 · DOI · Paper page
  53. Crean, 2002, Effect of tryptophan depletion on impulsive behavior in men with or without a family history of alcoholism, Behavioural Brain Research, 136, 349, 10.1016/S0166-4328(02)00132-8 · DOI
  54. Cremniter, 1999, CSF 5-HIAA levels are lower in impulsive as compared to nonimpulsive violent suicide attempters and control subjects, Biological Psychiatry, 45, 1572, 10.1016/S0006-3223(98)00382-5 · DOI
  55. Cromwell, 2003, Effects of expectations for different reward magnitudes on neuronal activity in primate striatum, Journal of Neurophysiology, 89, 2823, 10.1152/jn.01014.2002 · DOI
  56. Dalley, 2002, Deficits in impulse control associated with tonically-elevated serotonergic function in rat prefrontal cortex, Neuropsychopharmacology, 26, 716, 10.1016/S0893-133X(01)00412-2 · DOI
  57. Daw, 2002, Long-term reward prediction in TD models of the dopamine system, Neural Computation, 14, 2567, 10.1162/089976602760407973 · DOI
  58. de Borchgrave, 2002, Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats, Experimental Brain Research, 144, 50, 10.1007/s00221-002-1031-y · DOI
  59. de Villiers, 1995, Alpha 2-adrenoceptor mediated inhibition of [3H]dopamine release from nucleus accumbens slices and monoamine levels in a rat model for attention-deficit hyperactivity disorder, Neurochemical Research, 20, 427, 10.1007/BF00973098 · DOI
  60. Deakin, 2004, Risk taking during decision-making in normal volunteers changes with age, Journal of the International Neuropsychological Society, 10, 590, 10.1017/S1355617704104104 · DOI
  61. Delgado, 1990, Serotonin function and the mechanism of antidepressant action. Reversal of antidepressant-induced remission by rapid depletion of plasma tryptophan, Archives of General Psychiatry, 47, 411, 10.1001/archpsyc.1990.01810170011002 · DOI
  62. Delgado, 1989, Neuroendocrine and behavioral effects of dietary tryptophan restriction in healthy subjects, Life Sciences, 45, 2323, 10.1016/0024-3205(89)90114-8 · DOI
  63. Dhaenen, 2001, Imaging the serotonergic system in depression, European Archives of Psychiatry and Clinical Neuroscience, 251, II76, 10.1007/BF03035133 · DOI
  64. Dickinson, 1992, Free-operant acquisition with delayed reinforcement, Quarterly Journal of Experimental Psychology, Section B—Comparative and Physiological Psychology, 45, 241
  65. Dietrich, 1998, Functional dissociation of the prefrontal cortex and the hippocampus in timing behavior, Behavioral Neuroscience, 112, 1043, 10.1037/0735-7044.112.5.1043 · DOI
  66. Dietrich, 1997, Is the hippocampus involved in temporal discrimination and the memory of short intervals?, International Journal of Neuroscience, 90, 255, 10.3109/00207459709000642 · DOI
  67. Dunnett, 1990, Role of prefrontal cortex and striatal output systems in short-term memory deficits associated with ageing, basal forebrain lesions, and cholinergic-rich grafts, Canadian Journal of Psychology, 44, 210, 10.1037/h0084240 · DOI
  68. Eagle, 2003, Lesions of the medial prefrontal cortex or nucleus accumbens core do not impair inhibitory control in rats performing a stop-signal reaction time task, Behavioural Brain Research, 146, 131, 10.1016/j.bbr.2003.09.022 · DOI
  69. Ernst, 2004, Choice selection and reward anticipation: An fMRI study, Neuropsychologia, 42, 1585, 10.1016/j.neuropsychologia.2004.05.011 · DOI · Paper page
  70. Ernst, 1998, DOPA decarboxylase activity in attention deficit hyperactivity disorder adults. A [fluorine-18]fluorodopa positron emission tomographic study, Journal of Neuroscience, 18, 5901, 10.1523/JNEUROSCI.18-15-05901.1998 · DOI · Paper page
  71. Ersche, 2005, Punishment induces risky decision-making in methadone-maintained opiate users but not in heroin users or healthy volunteers, Neuropsychopharmacology, 30, 2115, 10.1038/sj.npp.1300812 · DOI
  72. Evenden, J. L. 1998. Serotonergic and steroidal influences on impulsive behaviour in rats. In: Comprehensive summaries of Uppsala dissertations from the faculty of medicine, vol. 764
  73. Evenden, 1999, Impulsivity: A discussion of clinical and experimental findings, Journal of Psychopharmacology, 13, 180, 10.1177/026988119901300211 · DOI
  74. Evenden, 1999, Varieties of impulsivity, Psychopharmacology, 146, 348, 10.1007/PL00005481 · DOI · Paper page
  75. Evenden, 1999, The behavior of spontaneously hypertensive and Wistar Kyoto rats under a paced fixed consecutive number schedule of reinforcement, Pharmacology, Biochemistry and Behavior, 63, 71, 10.1016/S0091-3057(98)00222-6 · DOI
  76. Evenden, 1996, The pharmacology of impulsive behaviour in rats: The effects of drugs on response choice with varying delays of reinforcement, Psychopharmacology, 128, 161, 10.1007/s002130050121 · DOI
  77. Evenden, 1999, The pharmacology of impulsive behaviour in rats VI: The effects of ethanol and selective serotonergic drugs on response choice with varying delays of reinforcement, Psychopharmacology, 146, 413, 10.1007/PL00005486 · DOI
  78. Fellows, 2005, Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans, Cerebral Cortex, 15, 58, 10.1093/cercor/bhh108 · DOI
  79. Fiorillo, 2003, Discrete coding of reward probability and uncertainty by dopamine neurons, Science, 299, 1898, 10.1126/science.1077349 · DOI · Paper page
  80. Fiorillo, 2005, Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than backpropagating TD errors, Behavioral and Brain Functions, 1, 7, 10.1186/1744-9081-1-7 · DOI
  81. Fone, 2005, Stimulants: Use and abuse in the treatment of attention deficit hyperactivity disorder, Current Opinion in Pharmacology, 5, 87, 10.1016/j.coph.2004.10.001 · DOI
  82. Gibbon, 1997, Toward a neurobiology of temporal cognition: Advances and challenges, Current Opinion in Neurobiology, 7, 170, 10.1016/S0959-4388(97)80005-0 · DOI
  83. Giertler, 2003, The rat nucleus accumbens is involved in guiding of instrumental responses by stimuli predicting reward magnitude, European Journal of Neuroscience, 18, 1993, 10.1046/j.1460-9568.2003.02904.x · DOI
  84. Giertler, 2004, Transient inactivation of the rat nucleus accumbens does not impair guidance of instrumental behaviour by stimuli predicting reward magnitude, Behavioural Pharmacology, 15, 55, 10.1097/00008877-200402000-00007 · DOI
  85. Gilbert, 2002, The amygdala but not the hippocampus is involved in pattern separation based on reward value, Neurobiology of Learning and Memory, 77, 338, 10.1006/nlme.2001.4033 · DOI
  86. Good, 2002, Spatial memory and hippocampal function: Where are we now?, Psicológica, 23, 109
  87. Good, 1991, Conditioning and contextual retrieval in hippocampal rats, Behavioral Neuroscience, 105, 499, 10.1037/0735-7044.105.4.499 · DOI
  88. Grace, 1996, Choice between fixed and variable delays to reinforcement in the adjusting-delay procedure and concurrent chains, Journal of Experimental Psychology: Animal Behavior Processes, 22, 362, 10.1037/0097-7403.22.3.362 · DOI
  89. Green, 1981, Preference reversal and self control: Choice as a function of reward amount and delay, Behaviour Analysis Letters, 1, 43
  90. Green, 1996, Exponential versus hyperbolic discounting of delayed outcomes: Risk and waiting time, American Zoologist, 36, 496, 10.1093/icb/36.4.496 · DOI · Paper page
  91. Green, 2004, A discounting framework for choice with delayed and probabilistic rewards, Psychological Bulletin, 130, 769, 10.1037/0033-2909.130.5.769 · DOI · Paper page
  92. Green, 1999, Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes, Journal of Experimental Psychology. Learning, Memory, and Cognition, 25, 418, 10.1037/0278-7393.25.2.418 · DOI
  93. Grice, 1948, The relation of secondary reinforcement to delayed reward in visual discrimination learning, Journal of Experimental Psychology, 38, 1, 10.1037/h0061016 · DOI
  94. Grindley, 1932, The formation of a simple habit in guinea pigs, British Journal of Psychology, 23, 127
  95. Hall, 2001, Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour, European Journal of Neuroscience, 13, 1984, 10.1046/j.0953-816x.2001.01577.x · DOI · Paper page
  96. Hamani, 2004, The subthalamic nucleus in the context of movement disorders, Brain, 127, 4, 10.1093/brain/awh029 · DOI
  97. Harker, 1956, Delay of reward and performance of an instrumental response, Journal of Experimental Psychology, 51, 303, 10.1037/h0042073 · DOI
  98. Hata, 1998, Effects of fimbria-fornix lesion on the temporal discrimination revealed by peak interval procedure in rats, Shinrigaku Kenkyu (Japanese Journal of Psychology), 69, 304, 10.4992/jjpsy.69.304 · DOI
  99. Hauber, 2000, NMDA, but not dopamine D(2), receptors in the rat nucleus accumbens are involved in guidance of instrumental behavior by stimuli predicting reward magnitude, Journal of Neuroscience, 20, 6282, 10.1523/JNEUROSCI.20-16-06282.2000 · DOI
  100. Heyman, 1996, Resolving the contradictions of addiction, Behavioral and Brain Sciences, 19, 561, 10.1017/S0140525X00042990 · DOI
  101. Hikosaka, 2000, Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards, Cerebral Cortex, 10, 263, 10.1093/cercor/10.3.263 · DOI
  102. Hirsh, 1974, The hippocampus and contextual retrieval of information from memory: A theory, Behavioral Biology, 12, 421, 10.1016/S0091-6773(74)92231-7 · DOI · Paper page
  103. Ho, 1999, Theory and method in the quantitative analysis of “impulsive choice” behaviour: Implications for psychopharmacology, Psychopharmacology, 146, 362, 10.1007/PL00005482 · DOI
  104. Holland, 1999, Hippocampus and context in classical conditioning, Current Opinion in Neurobiology, 9, 195, 10.1016/S0959-4388(99)80027-0 · DOI
  105. Holt, 2003, Is discounting impulsive? Evidence from temporal and probability discounting in gambling and non-gambling college students, Behavioural Processes, 64, 355, 10.1016/S0376-6357(03)00141-4 · DOI
  106. Holyoak, 1993, Thinking, Annual Review of Psychology, 44, 265, 10.1146/annurev.ps.44.020193.001405 · DOI
  107. Honey, 1993, Selective hippocampal lesions abolish the contextual specificity of latent inhibition and conditioning, Behavioral Neuroscience, 107, 23, 10.1037/0735-7044.107.1.23 · DOI
  108. Ikemoto, 1999, The role of nucleus accumbens dopamine in motivated behavior: A unifying interpretation with special reference to reward-seeking, Brain Research Reviews, 31, 6, 10.1016/S0165-0173(99)00023-5 · DOI · Paper page
  109. Ito, 1985, Choice and amount of reinforcement in rats, Learning & Motivation, 16, 95, 10.1016/0023-9690(85)90006-2 · DOI
  110. Ito, 2000, Dissociation in conditioned dopamine release in the nucleus accumbens core and shell in response to cocaine cues and during cocaine-seeking behavior in rats, Journal of Neuroscience, 20, 7489, 10.1523/JNEUROSCI.20-19-07489.2000 · DOI
  111. Ito, 2005, The hippocampus and appetitive Pavlovian conditioning: Effects of excitotoxic hippocampal lesions on conditioned locomotor activity and autoshaping, Hippocampus, 15, 713, 10.1002/hipo.20094 · DOI
  112. Izawa, 2005, Neural correlates of the proximity and quantity of anticipated food rewards in the ventral striatum of domestic chicks, European Journal of Neuroscience, 22, 1502, 10.1111/j.1460-9568.2005.04311.x · DOI
  113. Izawa, 2003, Localized lesion of caudal part of lobus parolfactorius caused impulsive choice in the domestic chick: Evolutionarily conserved function of ventral striatum, Journal of Neuroscience, 23, 1894, 10.1523/JNEUROSCI.23-05-01894.2003 · DOI
  114. Izquierdo, 2005, Opposing effects of amygdala and orbital prefrontal cortex lesions on the extinction of instrumental responding in macaque monkeys, European Journal of Neuroscience, 22, 2341, 10.1111/j.1460-9568.2005.04434.x · DOI
  115. Jarrard, 1993, On the role of the hippocampus in learning and memory in the rat, Behavioral and Neural Biology, 60, 9, 10.1016/0163-1047(93)90664-4 · DOI · Paper page
  116. Jongen-Relo, 2003, A differential involvement of the shell and core subterritories of the nucleus accumbens of rats in memory processes, Behavioral Neuroscience, 117, 150, 10.1037/0735-7044.117.1.150 · DOI
  117. Kacelnik, A. (1997b). Normative and descriptive models of decision making: Time discounting and risk sensitivity. In CIBA foundation symposium 208 (pp. 51–67). discussion 67–70 · DOI
  118. Kelley, 1999, Neural integrative activities of nucleus accumbens subregions in relation to learning and motivation, Psychobiology, 27, 198, 10.3758/BF03332114 · DOI
  119. Kelley, 2002, The neuroscience of natural rewards: Relevance to addictive drugs, Journal of Neuroscience, 22, 3306, 10.1523/JNEUROSCI.22-09-03306.2002 · DOI
  120. Kelley, 1997, Feeding induced by blockade of AMPA and kainate receptors within the ventral striatum: A microinfusion mapping study, Behavioural Brain Research, 89, 107, 10.1016/S0166-4328(97)00054-5 · DOI
  121. Kesner, 1995, Memory for magnitude of reinforcement: Dissociation between the amygdala and hippocampus, Neurobiology of Learning and Memory, 64, 237, 10.1006/nlme.1995.0006 · DOI
  122. Kheramin, 2005, The effect of orbital prefrontal cortex lesions on performance on a progressive ratio schedule: Implications for models of inter-temporal choice, Behavioural Brain Research, 156, 145, 10.1016/j.bbr.2004.05.017 · DOI
  123. Kheramin, 2003, Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: A quantitative analysis, Behavioural Processes, 64, 239, 10.1016/S0376-6357(03)00142-6 · DOI
  124. Kheramin, 2004, Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: A quantitative analysis, Psychopharmacology, 175, 206, 10.1007/s00213-004-1813-y · DOI · Paper page
  125. Kheramin, 2002, Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: A quantitative analysis, Psychopharmacology, 165, 9, 10.1007/s00213-002-1228-6 · DOI
  126. Kieres, 2004, Effects of morphine and naltrexone on impulsive decision making in rats, Psychopharmacology, 173, 167, 10.1007/s00213-003-1697-2 · DOI
  127. Killeen, 1988, A behavioral theory of timing, Psychological Review, 95, 274, 10.1037/0033-295X.95.2.274 · DOI
  128. Kim, 1993, Effects of amygdala, hippocampus, and periaqueductal gray lesions on short- and long-term contextual fear, Behavioral Neuroscience, 107, 1093, 10.1037/0735-7044.107.6.1093 · DOI · Paper page
  129. Kirby, 2004, Heroin and cocaine abusers have higher discount rates for delayed rewards than alcoholics or non-drug-using controls, Addiction, 99, 461, 10.1111/j.1360-0443.2003.00669.x · DOI · Paper page
  130. Knutson, 2001, Anticipation of increasing monetary reward selectively recruits nucleus accumbens, Journal of Neuroscience, 21, RC159, 10.1523/JNEUROSCI.21-16-j0002.2001 · DOI · Paper page
  131. Kuhnen, 2005, The neural basis of financial risk taking, Neuron, 47, 763, 10.1016/j.neuron.2005.08.008 · DOI · Paper page
  132. Kuntsi, 2001, Psychological mechanisms in hyperactivity: I. Response inhibition deficit, working memory impairment, delay aversion, or something else?, Journal of Child Psychology and Psychiatry and Allied Disciplines, 42, 199, 10.1111/1469-7610.00711 · DOI · Paper page
  133. Lagorio, 2005, Delay discounting of real and hypothetical rewards III: Steady-state assessments, forced-choice trials, and all real rewards, Behavioural Processes, 69, 173, 10.1016/j.beproc.2005.02.003 · DOI · Paper page
  134. Lane, 2004, Alcohol effects on human risk taking, Psychopharmacology, 172, 68, 10.1007/s00213-003-1628-2 · DOI
  135. Lattal, 1990, Response acquisition with delayed reinforcement, Journal of Experimental Psychology: Animal Behavior Processes, 16, 27, 10.1037/0097-7403.16.1.27 · DOI
  136. Leland, 2005, Increased risk-taking decision-making but not altered response to punishment in stimulant-using young adults, Drug and Alcohol Dependence, 78, 83, 10.1016/j.drugalcdep.2004.10.001 · DOI
  137. Levita, 2002, Disruption of Pavlovian contextual conditioning by excitotoxic lesions of the nucleus accumbens core, Behavioral Neuroscience, 116, 539, 10.1037/0735-7044.116.4.539 · DOI
  138. Liao, 2003, Differential effects of diazepam infused into the amygdala and hippocampus on negative contrast, Pharmacology, Biochemistry and Behavior, 74, 953, 10.1016/S0091-3057(03)00023-6 · DOI
  139. Linnoila, 1993, Impulse control disorders, International Clinical Psychopharmacology, 8, 53, 10.1097/00004850-199309001-00008 · DOI
  140. Linnoila, 1983, Low cerebrospinal fluid 5-hydroxyindoleacetic acid concentration differentiates impulsive from nonimpulsive violent behavior, Life Sciences, 33, 2609, 10.1016/0024-3205(83)90344-2 · DOI · Paper page
  141. Liu, 2004, Effects of acute and chronic buspirone on impulsive choice and efflux of 5-HT and dopamine in hippocampus, nucleus accumbens and prefrontal cortex, Psychopharmacology, 173, 175, 10.1007/s00213-003-1726-1 · DOI
  142. Loewenstein, G. F., & O’Donoghue, T. (2004). Animal spirits: Affective and deliberative processes in economic behavior. http://ssrn.com/abstract=539843 · DOI
  143. Logue, 1992, Cocaine decreases self-control in rats: A preliminary report, Psychopharmacology, 109, 245, 10.1007/BF02245509 · DOI
  144. Madden, 1999, Discounting of delayed rewards in opioid-dependent outpatients: Exponential or hyperbolic discounting functions?, Experimental and Clinical Psychopharmacology, 7, 284, 10.1037/1064-1297.7.3.284 · DOI
  145. Maldonado-Irizarry, 1995, Excitotoxic lesions of the core and shell subregions of the nucleus accumbens differentially disrupt body-weight regulation and motor activity in the rat, Brain Research Bulletin, 38, 551, 10.1016/0361-9230(95)02030-2 · DOI
  146. Manes, 2002, Decision-making processes following damage to the prefrontal cortex, Brain, 125, 624, 10.1093/brain/awf049 · DOI · Paper page
  147. Maren, 1997, Electrolytic lesions of the fimbria/fornix, dorsal hippocampus, or entorhinal cortex produce anterograde deficits in contextual fear conditioning in rats, Neurobiology of Learning and Memory, 67, 142, 10.1006/nlme.1996.3752 · DOI
  148. Martin, 2000, Effects of reward anticipation, reward presentation, and spatial parameters on the firing of single neurons recorded in the subiculum and nucleus accumbens of freely moving rats, Behavioural Brain Research, 116, 23, 10.1016/S0166-4328(00)00249-7 · DOI
  149. Martin-Iverson, 1987, Effects of haloperidol and d-amphetamine on perceived quantity of food and tones, Psychopharmacology, 93, 374, 10.1007/BF00187260 · DOI
  150. Matthews, 2004, Selective activation of the nucleus accumbens during risk-taking decision making, Neuroreport, 15, 2123, 10.1097/00001756-200409150-00025 · DOI · Paper page
  151. Maurice, 1998, Relationships between the prefrontal cortex and the basal ganglia in the rat: Physiology of the corticosubthalamic circuits, Journal of Neuroscience, 18, 9539, 10.1523/JNEUROSCI.18-22-09539.1998 · DOI
  152. Mazur, 1989, Theories of probabilistic reinforcement, Journal of the Experimental Analysis of Behavior, 51, 87, 10.1901/jeab.1989.51-87 · DOI
  153. Mazur, 1995, Conditioned reinforcement and choice with delayed and uncertain primary reinforcers, Journal of the Experimental Analysis of Behavior, 63, 139, 10.1901/jeab.1995.63-139 · DOI
  154. Mazur, 1997, Choice, delay, probability, and conditioned reinforcement, Animal Learning & Behavior, 25, 131, 10.3758/BF03199051 · DOI
  155. Mazur, 1987, Self-control choice with electrical stimulation of the brain as a reinforcer, Behavioural Processes, 15, 143, 10.1016/0376-6357(87)90003-9 · DOI
  156. McClure, 2003, A computational substrate for incentive salience, Trends in Neurosciences, 26, 423, 10.1016/S0166-2236(03)00177-2 · DOI
  157. McClure, 2004, Separate neural systems value immediate and delayed monetary rewards, Science, 306, 503, 10.1126/science.1100907 · DOI · Paper page
  158. Meck, 1988, Hippocampal function is required for feedback control of an internal clock’s criterion, Behavioral Neuroscience, 102, 54, 10.1037/0735-7044.102.1.54 · DOI
  159. Meck, 1984, Hippocampus, time, and memory, Behavioral Neuroscience, 98, 3, 10.1037/0735-7044.98.1.3 · DOI
  160. Mehlman, 1994, Low CSF 5-HIAA concentrations and severe aggression and impaired impulse control in nonhuman primates, American Journal of Psychiatry, 151, 1485, 10.1176/ajp.151.10.1485 · DOI
  161. Mingote, 2005, Ratio and time requirements on operant schedules: Effort-related effects of nucleus accumbens dopamine depletions, European Journal of Neuroscience, 21, 1749, 10.1111/j.1460-9568.2005.03972.x · DOI
  162. Mischel, 1966, Theory and research on the antecedents of self-imposed delay of reward, Vol. 3, 85
  163. Mitchell, 2004, Effects of short-term nicotine deprivation on decision-making: Delay, uncertainty and effort discounting, Nicotine & Tobacco Research, 6, 819, 10.1080/14622200412331296002 · DOI
  164. Mitchell, 2004, Measuring impulsivity and modeling its association with cigarette smoking, Behavioral and Cognitive Neuroscience Reviews, 3, 261, 10.1177/1534582305276838 · DOI
  165. Miyazaki, 1998, Reward-quality dependent anticipation in rat nucleus accumbens, Neuroreport, 9, 3943, 10.1097/00001756-199812010-00032 · DOI
  166. Mobini, 2002, Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement, Psychopharmacology, 160, 290, 10.1007/s00213-001-0983-0 · DOI · Paper page
  167. Mobini, 2000, Effect of central 5-hydroxytryptamine depletion on inter-temporal choice: A quantitative analysis, Psychopharmacology, 149, 313, 10.1007/s002130000385 · DOI
  168. Mobini, 2000, Effects of central 5-hydroxytryptamine depletion on sensitivity to delayed and probabilistic reinforcement, Psychopharmacology, 152, 390, 10.1007/s002130000542 · DOI
  169. Muir, 1996, The cerebral cortex of the rat and visual attentional function: Dissociable effects of mediofrontal, cingulate, anterior dorsolateral, and parietal cortex lesions on a five-choice serial reaction time task, Cerebral Cortex, 6, 470, 10.1093/cercor/6.3.470 · DOI
  170. Myerson, 1995, Discounting of delayed rewards: Models of individual choice, Journal of the Experimental Analysis of Behavior, 64, 263, 10.1901/jeab.1995.64-263 · DOI
  171. Myerson, 2003, Discounting delayed and probabilistic rewards: Processes and traits, Journal of Economic Psychology, 24, 619, 10.1016/S0167-4870(03)00005-9 · DOI
  172. O’Carroll, 2003, Decision making in humans: The effect of manipulating the central noradrenergic system, Journal of Neurology, Neurosurgery and Psychiatry, 74, 376, 10.1136/jnnp.74.3.376 · DOI
  173. Ohmura, 2005, Discounting delayed and probabilistic monetary gains and losses by smokers of cigarettes, Psychopharmacology, 182, 508, 10.1007/s00213-005-0110-8 · DOI
  174. Olton, 1987, Separation of hippocampal and amygdaloid involvement in temporal memory dysfunctions, Brain Research, 404, 180, 10.1016/0006-8993(87)91369-2 · DOI
  175. Ortner, 2003, Alcohol intoxication reduces impulsivity in the delay-discounting paradigm, Alcohol and Alcoholism, 38, 151, 10.1093/alcalc/agg041 · DOI
  176. Ostaszewski, 1998, Effects of inflation on the subjective value of delayed and probabilistic rewards, Psychonomic Bulletin & Review, 5, 324, 10.3758/BF03212959 · DOI
  177. Ostaszewski, 2002, Discounting of delayed and probabilistic losses of different amounts, European Psychologist, 7, 295, 10.1027//1016-9040.7.4.295 · DOI
  178. Packard, 1996, Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning, Neurobiology of Learning and Memory, 65, 65, 10.1006/nlme.1996.0007 · DOI
  179. Paine, 2003, Effects of chronic cocaine on impulsivity: Relation to cortical serotonin mechanisms, Behavioural Brain Research, 147, 135, 10.1016/S0166-4328(03)00156-6 · DOI
  180. Papa, 1996, Reduced CaMKII-positive neurones in the accumbens shell of an animal model of attention-deficit hyperactivity disorder, Neuroreport, 7, 3017, 10.1097/00001756-199611250-00044 · DOI
  181. Papa, 1998, Reduced transduction mechanisms in the anterior accumbal interface of an animal model of attention-deficit hyperactivity disorder, Behavioural Brain Research, 94, 187, 10.1016/S0166-4328(97)00179-4 · DOI
  182. Parkinson, 2000, Limbic cortical–ventral striatal systems underlying appetitive conditioning, Progress in Brain Research, 126, 263, 10.1016/S0079-6123(00)26019-6 · DOI · Paper page
  183. Parkinson, 2002, Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: Implications for mesoaccumbens dopamine function, Behavioural Brain Research, 137, 149, 10.1016/S0166-4328(02)00291-7 · DOI · Paper page
  184. Parkinson, 1999, Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive Pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine, Journal of Neuroscience, 19, 2401, 10.1523/JNEUROSCI.19-06-02401.1999 · DOI · Paper page
  185. Parkinson, 1999, Selective excitotoxic lesions of the nucleus accumbens core and shell differentially affect aversive Pavlovian conditioning to discrete and contextual cues, Psychobiology, 27, 256, 10.3758/BF03332119 · DOI
  186. Parkinson, 2000, Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: Further evidence for limbic cortical–ventral striatopallidal systems, Behavioral Neuroscience, 114, 42, 10.1037/0735-7044.114.1.42 · DOI · Paper page
  187. Paulus, 2003, Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism, Neuroimage, 19, 1439, 10.1016/S1053-8119(03)00251-9 · DOI · Paper page
  188. Perin, 1943, A quantitative investigation of the delay-of-reinforcement gradient, Journal of Experimental Psychology, 32, 37, 10.1037/h0056738 · DOI
  189. Phillips, 1994, Lesions of the dorsal hippocampal formation interfere with background but not foreground contextual fear conditioning, Learning & Memory, 1, 34, 10.1101/lm.1.1.34 · DOI · Paper page
  190. Phillips, 1995, Lesions of the fornix but not the entorhinal or perirhinal cortex interfere with contextual fear conditioning, Journal of Neuroscience, 15, 5308, 10.1523/JNEUROSCI.15-07-05308.1995 · DOI
  191. Plutchik, 1989, The measurement of suicidality, aggressivity and impulsivity, Progress in Neuro-Psychopharmacology and Biological Psychiatry, 13, S23, 10.1016/0278-5846(89)90107-3 · DOI
  192. Port, 1986, Retention and acquisition of classical trace conditioned responses by rabbits with hippocampal lesions, Behavioral Neuroscience, 100, 745, 10.1037/0735-7044.100.5.745 · DOI
  193. Pothuizen, 2005, Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats, European Journal of Neuroscience, 22, 2605, 10.1111/j.1460-9568.2005.04388.x · DOI
  194. Pothuizen, 2006, Latent inhibition of conditioned taste aversion is not disrupted, but can be enhanced, by selective nucleus accumbens shell lesions in rats, Neuroscience, 137, 1119, 10.1016/j.neuroscience.2005.10.032 · DOI
  195. Poulos, 1996, Dexfenfluramine and 8-OH-DPAT modulate impulsivity in a delay-of-reward paradigm: Implications for a correspondence with alcohol consumption, Behavioural Pharmacology, 7, 395, 10.1097/00008877-199608000-00011 · DOI
  196. Rachlin, 2000, Discounting in judgments of delay and probability, Journal of Behavioral Decision Making, 13, 145, 10.1002/(SICI)1099-0771(200004/06)13:2<145::AID-BDM320>3.0.CO;2-4 · DOI
  197. Rachlin, 1987, Probability and delay in commitment, Journal of the Experimental Analysis of Behavior, 48, 347, 10.1901/jeab.1987.48-347 · DOI
  198. Rachlin, 1986, Cognition and behavior in studies of choice, Psychological Review, 93, 33, 10.1037/0033-295X.93.1.33 · DOI
  199. Rachlin, 1991, Subjective probability and delay, Journal of the Experimental Analysis of Behavior, 55, 233, 10.1901/jeab.1991.55-233 · DOI · Paper page
  200. Rachlin, 1994, Temporal patterning in probabilistic choice, Organizational Behavior and Human Decision Processes, 59, 161, 10.1006/obhd.1994.1054 · DOI

200 of 288 shown

Cited by

Sorted by citations

See all 248

Metadata

Authors
Cardinal, R. N.
Venue
Neural Networks
Publisher
Elsevier BV
Published
2006-10
Volume
19
Issue
8
Pages
1277-1301
ISSN
0893-6080
DOI
10.1016/j.neunet.2006.03.004
Cited by
248
References
46
Language
en
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Neural systems implicated in delayed and probabilistic reinforcement. https://app.cochat.ai/papers/10.1016/j.neunet.2006.03.004
The paper
Cardinal, R. N. (2006). Neural systems implicated in delayed and probabilistic reinforcement. Neural Networks. https://doi.org/10.1016/j.neunet.2006.03.004