A Computational Model of the Functional Role of the Ventral-Striatal D2 Receptor in the Expression of Previously Acquired Behaviors
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
The functional role of dopamine has attracted a great deal of interest ever since it was empirically discovered that dopamine-blocking drugs could be used to treat psychosis. Specifically, the D2 receptor and its expression in the ventral striatum have emerged as pivotal in our understanding of the complex role of the neuromodulator in schizophrenia, reward, and motivation. Our departure from the ubiquitous temporal difference (TD) model of dopamine neuron firing allows us to account for a range of experimental evidence suggesting that ventral striatal dopamine D2 receptor manipulation selectively modulates motivated behavior for distal versus proximal outcomes. Whether an internal model or the TD approach (or a mixture) is better suited to a comprehensive exposition of tonic and phasic dopamine will have important implications for our understanding of reward, motivation, schizophrenia, and impulsivity. We also use the model to help unite some of the leading cognitive hypotheses of dopamine function under a computational umbrella. We have used the model ourselves to stimulate and focus new rounds of experimental research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“Our approach is different from the TD prediction-error hypothesis (see Houk et al., 1995; Schultz et al., 1997; Montague et al., 1996), which suggests that the phasic dopamine response signals the difference (error) between the future reward predicted by the animal and the actual reward received.”
ResultBerridge, K. C., & Robinson, T. E. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00233-9 · Cited by 1,869
Grace, A. A. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90196-u · Cited by 1,831
Dissociation in prefrontal cortex of affective and attentional shifts
Dias, R., Robbins, T. W., & Roberts, A. C. (1996) · Nature · doi:10.1038/380069a0 · Cited by 1,527
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 1×
“, 2002; Nishijo, Ono, & Nishijo, 1988; Robbins, Cador, Taylor, & Everitt, 1989; Tremblay & Schultz, 2000a, 2000b) and the orbito-frontal cortex (Arana et al., 2003; Bechara, Damasio, Tranel, & Anderson, 1998; Cardinal et al., 2002; Iversen & Mishkin, 1970; Masterman & Cummings, 1997; O’Doherty et al., 2003) in the rep…”
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Subsecond dopamine release promotes cocaine seeking
Phillips, P. E. M., Stuber, G. D., Heien, M. L. A. V., Wightman, R. M., & Carelli, R. M. (2003) · Nature · doi:10.1038/nature01476 · Cited by 1,113
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Neural Economics and the Biological Substrates of Valuation
Montague, P. R., & Berns, G. S. (2002) · Neuron · doi:10.1016/s0896-6273(02)00974-1 · Cited by 994
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
Reward processing in primate orbitofrontal cortex and basal ganglia
Schultz, W. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.272 · Cited by 942
Impulsive choice induced in rats by lesions of the nucleus accumbens core
Cardinal, R. N., Pennicott, D. R., Lakmali, C., Sugathapala, Robbins, T. W., & Everitt, B. J. (2001) · Science · doi:10.1126/science.1060818 · Cited by 923 · cited 1×
“Interestingly, Cardinal et al. (2001) demonstrate that accumbens (core) lesions in rats cause exactly the same kinds of effects, leading them to conclude that the accumbens is involved in the pathogenesis of impulsive choice, a finding they suggest can shed light on ADHD, addiction, and other…”
BackgroundWise, R. A. (2002) · Neuron · doi:10.1016/s0896-6273(02)00965-0 · Cited by 881
Involvement of the amygdala in stimulus-reward associations: interaction with the ventral striatum
Cador, M., Robbins, T. W., & Everitt, B. J. (1989) · Neuroscience · doi:10.1016/0306-4522(89)90354-0 · Cited by 652
Solanto, M. V., Abikoff, H., Sonuga-Barke, E., Schachar, R., Logan, G. D., Wigal, T., Hechtman, L., Hinshaw, S., & Turkel, E. (2001) · Journal of Abnormal Child Psychology · doi:10.1023/a:1010329714819 · Cited by 639 · cited 1×
“Second, it must be acknowledged that ADHD is a complex and multidimensional disorder, and the delay aversion hypothesis (see Solanto et al., 2001) and delay discounting hypothesis (see Catania, in press), represent only one line of argument.”
BackgroundEveritt, B. J., Morris, K. A., O'Brien, A., & Robbins, T. W. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90145-e · Cited by 507
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · cited 1×
“However, a number of criticisms have been discussed (Berridge & Robinson, 1998; Horvitz, 2000, 2002; Redgrave et al., 1999).”
BackgroundLimbic-striatal interactions in reward-related processes
Robbins, T. W., Cador, M., Taylor, J. R., & Everitt, B. J. (1989) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(89)80025-9 · Cited by 458
Direct Activation of the Ventral Striatum in Anticipation of Aversive Stimuli
Jensen, J., McIntosh, A. R., Crawley, A. P., Mikulis, D. J., Remington, G., & Kapur, S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00724-4 · Cited by 455
Self-citationEveritt, B. J., Cador, M., & Robbins, T. W. (1989) · Neuroscience · doi:10.1016/0306-4522(89)90353-9 · Cited by 444
Acute administration of d-amphetamine decreases impulsivity in healthy volunteers
de Wit, H. (2002) · Neuropsychopharmacology · doi:10.1016/s0893-133x(02)00343-3 · Cited by 436 · cited 2×
“If blocking dopamine can disrupt avoidance responding in rats, enhancing dopamine seems to decrease impulsivity as measured by this kind of delay discounting (Wade et al., 2000; Richards et al., 1999; de Wit et al., 2002).”
BackgroundThe effect of the instrumental training contingency on susceptibility to reinforcer devaluation
Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748308400912 · Cited by 338
Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats
de Wit, H., Wade, T. R., & Richards, J. B. (2000) · Psychopharmacology · doi:10.1007/s002130000402 · Cited by 333 · cited 12×
“…consistent account of some apparently diverse results is also required for the expression (''zeroth derivative " ) of previously acquired behaviors (Cousins et al., 1996; Rolls et al., 1974; Maffii, 1959; Berridge & Robinson, 1998; Wade et al., 2000; Richards et al., 1999; Wadenberg et al., 2001).”
BackgroundMethodInfluentialGiordano, L., Bickel, W., Loewenstein, G., Jacobs, E., Marsch, L., & Badger, G. (2002) · Psychopharmacology · doi:10.1007/s00213-002-1159-2 · Cited by 294 · cited 2×
“Some suggestions include (1) constraint of the look-ahead process via its action as an online discount factor; (2) adaptation of the trade-off between proximal and distal rewards in response to environment cues (Wilson & Daly, 2004) or deprivational states (Giordano et al., 2002); or (3) global control of general moti…”
BackgroundAnti-hyperactivity medication: methylphenidate and amphetamine
Seeman, P., & Madras, B. K. (1998) · Molecular Psychiatry · doi:10.1038/sj.mp.4000421 · Cited by 252
Cousins, M. S., Atherton, A., Turner, L., & Salamone, J. D. (1996) · Behavioural Brain Research · doi:10.1016/0166-4328(95)00151-4 · Cited by 236 · cited 5×
“(b) The T-maze experimental setup used in Cousins et al. (1996).”
BackgroundMethodInfluentialEffects of methamphetamine on the adjusting amount procedure, a model of impulsive behavior in rats
Richards, J. B., Sabol, K. E., & de Wit, H. (1999) · Psychopharmacology · doi:10.1007/pl00005488 · Cited by 203
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173
Wolf, D. H. (2006) · Current Psychiatry Reports · doi:10.1007/s11920-006-0069-0 · Cited by 66
Nucleus accumbens, entorhinal cortex and latent inhibition: A neural network model
Schmajuk, N. A., Cox, L., & Gray, J. A. (2001) · Behavioural Brain Research · doi:10.1016/s0166-4328(00)00319-3 · Cited by 52 · cited 1×
“Indeed, they have been used not only within formal reinforcement learning (reviewed in Sutton & Barto, 1998), but also to model animal conditioning (Schmajuk, 1988) and the dopamine system (Schmajuk, Cox, & Gray, 2001; Suri, Bargas, & Arbib, 2001; Suri, 2001).”
Method
Cited by
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
“Most other reinforcement learning theories that contemplate model-based control either completely replace caching with searc”
BackgroundFrom reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 722
Neurobehavioral mechanisms of impulsivity: fronto-striatal systems and functional neurochemistry
Dalley, J. W., Mar, A. C., Economidou, D., & Robbins, T. W. (2008) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2007.12.021 · Cited by 467
Opponent appetitive-aversive neural processes underlie predictive learning of pain relief
Seymour, B., O'Doherty, J. P., Koltzenburg, M., Wiech, K., Frackowiak, R., Friston, K., & Dolan, R. (2005) · Nature Neuroscience · doi:10.1038/nn1527 · Cited by 439
“However, taken alone, it is possible that this ventral striatal activity is modulated by a single prediction-error signal for both relief and exacerbation cue”
BackgroundThe mesoaccumbens dopamine in coping with stress
Cabib, S., & Puglisi-Allegra, S. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.012 · Cited by 329
Metadata
- Authors
- Smith, A. J., Becker, S., & Kapur, S.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2005-02-01
- Volume
- 17
- Issue
- 2
- Pages
- 361-395
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2116128159
- Cited by
- 37
- References
- 35
- FWCI
- 1.70
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.