Shifting Responsibly: The Importance of Striatal Modularity to Reinforcement Learning in Uncertain Environments
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
We propose here that the modular organization of the striatum reflects a context-sensitive modular learning architecture in which clustered striosome-matrisome domains participate in modular reinforcement learning (RL). Based on anatomical and physiological evidence, it has been suggested that the modular organization of the striatum could represent a learning architecture. There is not, however, a coherent view of how such a learning architecture could relate to the organization of striatal outputs into the direct and indirect pathways of the basal ganglia, nor a clear formulation of how such a modular architecture relates to the RL functions attributed to the striatum. Here, we hypothesize that striosome-matrisome modules not only learn to bias behavior toward specific actions, as in standard RL, but also learn to assess their own relevance to the environmental context and modulate their own learning and activity on this basis. We further hypothesize that the contextual relevance or "responsibility" of modules is determined by errors in predictions of environmental features and that such responsibility is assigned by striosomes and conveyed to matrisomes via local circuit interneurons. To examine these hypotheses and to identify the general requirements for realizing this architecture in the nervous system, we developed a simple modular RL model. We then constructed a network model of basal ganglia circuitry that includes these modules and the direct and indirect pathways. Based on simple assumptions, this model suggests that while the direct pathway may promote actions based on striatal action values, the indirect pathway may act as a gating network that facilitates or suppresses behavioral modules on the basis of striatal responsibility signals. Our modeling functionally unites the modular compartmental organization of the striatum with the direct-indirect pathway divisions of the basal ganglia, a step that we suggest will have important clinical implications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Functional anatomy of the basal ganglia. I. The cortico-basal ganglia-thalamo-cortical loop
Parent, A., & Hazrati, L.-N. (1995) · Brain Research Reviews · doi:10.1016/0165-0173(94)00007-c · Cited by 2,161 · cited 1×
“This fact could result in a progressive coarsening of the resolution of represented information along these pathways (Bar-Gad et al., 2003), long hypothesized as a “funneling” process (Percheron and Filion, 1991; Bolam et al., 1993; Parent and Hazrati, 1995).”
BackgroundA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 2×
“, 1997; Schultz, 1998), and dopamine-dependent long-term synaptic plasticity at corticostriatal synapses has been proposed as the basis for reinforcement learning (RL; Montague et al., 1996; Reynolds et al., 2001).”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Habits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 2×
“…as movements: the associative striatum, which receives input from prefrontal and parietal association cortices, may more strongly influence the former, whereas the sensorimotor striatum, which receives input from sensorimotor cortices, may more strongly influence the latter (Graybiel, 1997, 2008).”
BackgroundSelf-citationTwo types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · cited 1×
“Matsumoto and Hikosaka (2009) have observed in monkeys that, although some SNc (putatively dopamine-containing) neurons prefer reward-predicting stimuli to aversion-predicting stimuli, a large number of these neurons are excited by either of these types of stimulation.”
BackgroundA unifying computational framework for motor control and social interaction
Wolpert, D. M., Doya, K., & Kawato, M. (2003) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1238 · Cited by 1,228
Surmeier, D. J., Ding, J., Day, M., Wang, Z., & Shen, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.008 · Cited by 1,203 · cited 3×
“(2008) found that activation of dopamine receptors is required for spike timing-dependent longterm potentiation (LTP) at direct-pathway MSNs and long-term depression (LTD) at indirect-pathway MSNs (Surmeier et al., 2007; Shen et al., 2008).”
BackgroundDichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190
The Basal Ganglia and Chunking of Action Repertoires
Graybiel, A. M. (1998) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1998.3843 · Cited by 1,042
Complementary roles of basal ganglia and cerebellum in learning and motor control
Doya, K. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00153-7 · Cited by 964
Cited by
Basal Ganglia Disorders Associated with Imbalances in the Striatal Striosome and Matrix Compartments
Crittenden, J. R., & Graybiel, A. M. (2011) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2011.00059 · Cited by 481
Self-citationThe Basal Ganglia Over 500 Million Years
Grillner, S., & Robertson, B. (2016) · Current Biology · doi:10.1016/j.cub.2016.06.041 · Cited by 357
Deep homology of arthropod central complex and vertebrate basal ganglia.
Strausfeld, N. J., & Hirth, F. (2013) · Science · doi:10.1126/science.1231828 · Cited by 354
Goal-oriented searching mediated by ventral hippocampus early in trial-and-error learning
Ruediger, S., Spirig, D., Donato, F., & Caroni, P. (2012) · Nature Neuroscience · doi:10.1038/nn.3224 · Cited by 135
A Genetically Defined Compartmentalized Striatal Direct Pathway for Negative Reinforcement
Xiao, X., Deng, H., Furlan, A., Yang, T., Zhang, X., Hwang, G.-R., Tucciarone, J., Wu, P., He, M., Palaniswamy, R., et al. (2020) · Cell · doi:10.1016/j.cell.2020.08.032 · Cited by 101
“This correlation likely represents the function of matrix neurons (Amemori et al., 2011; Crittenden and Graybiel, 2016), which can be preferentially imaged as the matrix compartment occupies a much larger fraction of the DS than the striosome (Gerfen, 1992; Graybiel and Ragsdale, 1978).”
BackgroundInfluential
Metadata
- Authors
- Amemori, K.-I., Gibb, L. G., & Graybiel, A. M.
- Venue
- Frontiers in Human Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2011
- Volume
- 5
- Pages
- 47-47
- ISSN
- 1662-5161 (ISSN-L)
- OpenAlex
- W2074239791
- Cited by
- 73
- References
- 25
- FWCI
- 1.51
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Shifting Responsibly: The Importance of Striatal Modularity to Reinforcement Learning in Uncertain Environments. https://app.cochat.ai/papers/10.3389/fnhum.2011.00047
- The paper
- Amemori, K.-I., Gibb, L. G., & Graybiel, A. M. (2011). Shifting Responsibly: The Importance of Striatal Modularity to Reinforcement Learning in Uncertain Environments. Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2011.00047
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.