A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Recently, dopamine (DA) neurons of the substantia nigra pars compacta (SNc) were found to exhibit sustained responses related to reward uncertainty, in addition to the phasic responses related to reward-prediction errors (RPEs). Thus, cue-dependent anticipations of the timing, magnitude, and uncertainty of rewards are learned and reflected in components of DA signals. Here we simulate a local circuit model to show how learned uncertainty responses are generated, along with phasic RPE responses, on single trials. Both types of simulated DA responses exhibit the empirically observed dependencies on conditional probability, expected value of reward, and time since onset of the reward-predicting cue. The model's three major pathways compute expected values of cues, timed predictions of reward magnitudes, and uncertainties associated with these predictions. The first two pathways' computations refine those modeled by Brown et al. (1999). The third, newly modeled, pathway involves medium spiny projection neurons (MSPNs) of the striatal matrix, whose axons corelease GABA and substance P, both at synapses with GABAergic neurons in the substantia nigra pars reticulata (SNr) and with distal dendrites (in SNr) of DA neurons whose somas are located in ventral SNc. Corelease enables efficient computation of uncertainty responses that are a nonmonotonic function of the conditional probability of reward, and variability in striatal cholinergic transmission can explain observed individual differences in the amplitudes of uncertainty responses. The involvement of matricial MSPNs and cholinergic transmission within the striatum implies a relation between uncertainty in cue–reward contingencies and action-selection functions of the basal ganglia.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
The principal features and mechanisms of dopamine modulation in the prefrontal cortex
Seamans, J. K., & Yang, C. R. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.05.006 · Cited by 1,592 · cited 1×
“Predictions vary considerably across conceptualizations of the role of DA in forebrain function (Seamans and Yang, 2004).”
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · cited 12×
“This dual dependence of CS-induced striatal activations is consistent with experimental observations (Tobler et al., 2005).”
InfluentialModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003) · Nature Neuroscience · doi:10.1038/nn1103 · Cited by 1,106
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · cited 1×
“” The predictively redundant CSB is blocked from becoming able to generate DA bursts, in accord with neural data (Waelti et al., 2001).”
The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia
Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · cited 3×
“A large number of reports have accumulated regarding the sensitivity of striatal (Samejima et al., 2005; Kawagoe et al., 1998; Watanabe et al., 2003) and DA neurons to a number of properties of cue-reward contingencies that are important both for learning to predict outcomes and for intelligent…”
Different time courses of learning-related activity in the prefrontal cortex and striatum.
Pasupathy, A., & Miller, E. K. (2005) · Nature · doi:10.1038/nature03287 · Cited by 712 · cited 1×
“Uncertainty responses may also facilitate behavioral switches during reversal learning (Pasupathy and Miller, 2005) (but see Tan and Bullock, 2008a).”
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Joel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614 · cited 4×
“However, GABAergic MSPNs of the striatal matrix and striosome compartments, respectively, can inhibit the firing of the GABAergic pars reticulata (SNr) and dopaminergic pars compacta (SNc) cells, respectively (Ragsdale and Graybiel, 1990; Joel and Weiner, 2000).”
InfluentialSchultz, W. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.03.017 · Cited by 517
Modulatory functions of neurotransmitters in the striatum: ACh/dopamine/NMDA interactions
Di Chiara, G., Morelli, M., & Consolo, S. (1994) · Trends in Neurosciences · doi:10.1016/0166-2236(94)90005-1 · Cited by 467 · cited 3×
“Acetylcholine (ACh) release by model striatal TANs modulates the matriceal MSPNs via a dual control of FS-INs by ACh (Koos and Tepper, 2002; DeRover et al., 2002), and via a direct inhibitory influence of ACh on SP-MSPNs (Bernardi et al., 1993; DiChiara et al., 1994).”
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Romo, R., & Schultz, W. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.3.592 · Cited by 353 · cited 1×
“This “offset hypothesis” is supported by data indicating that DA cells do not show responses that are time locked to the motor initiations gated by matricial MSPN outputs (DeLong et al., 1983; Romo and Schultz, 1990; Schultz, 1998).”
Relations between movement and single cell discharge in the substantia nigra of the behaving monkey
DeLong, M., Crutcher, M., & Georgopoulos, A. (1983) · The Journal of Neuroscience · doi:10.1523/jneurosci.03-08-01599.1983 · Cited by 247 · cited 1×
“This “offset hypothesis” is supported by data indicating that DA cells do not show responses that are time locked to the motor initiations gated by matricial MSPN outputs (DeLong et al., 1983; Romo and Schultz, 1990; Schultz, 1998).”
Wörgötter, F., & Porr, B. (2005) · Neural Computation · doi:10.1162/0899766053011555 · Cited by 203
Neuropeptides as synaptic transmitters
Salio, C., Lossi, L., Ferrini, F., & Merighi, A. (2006) · Cell and Tissue Research · doi:10.1007/s00441-006-0268-3 · Cited by 181
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153
Cited by
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
“In particular, models using a biologically consistent striatal and midbrain circuit replicate well the dopamine risk ramp (581).”
MethodNeural Circuitry of Reward Prediction Error
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031109 · Cited by 437
Arithmetic and local circuitry underlying dopamine prediction errors
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N. (2015) · Nature · doi:10.1038/nature14855 · Cited by 392
Chorley, P., & Seth, A. K. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00021 · Cited by 16
“Our model is consistent with suggestions that competition between excitatory and inhibitory pathways play a significant role in basal ganglia operation (Redgrave and Gurney, 2006), specifically in the generation of DA responses (Brown et al., 1999; Pan et al., 2008; Tan and Bullock, 2008), where their functional signi…”
BackgroundMethodResultInfluential
Metadata
- Authors
- Tan, C. O., & Bullock, D.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2008-10-01
- Volume
- 28
- Issue
- 40
- Pages
- 10062-10074
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2127914429
- Cited by
- 41
- References
- 20
- FWCI
- 1.55
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.