The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Neurocomputational models of basal ganglia function in learning, memory and choice
Open accessAbstract
The basal ganglia (BG) are critical for the coordination of several motor, cognitive, and emotional functions and become dysfunctional in several pathological states ranging from Parkinson's disease to Schizophrenia. Here we review principles developed within a neurocomputational framework of BG and related circuitry which provide insights into their functional roles in behavior. We focus on two classes of models: those that incorporate aspects of biological realism and constrained by functional principles, and more abstract mathematical models focusing on the higher level computational goals of the BG. While the former are arguably more “realistic”, the latter have a complementary advantage in being able to describe functional principles of how the system works in a relatively simple set of equations, but are less suited to making specific hypotheses about the roles of specific nuclei and neurophysiological processes. We review the basic architecture and assumptions of these models, their relevance to our understanding of the neurobiological and cognitive functions of the BG, and provide an update on the potential roles of biological details not explicitly incorporated in existing models. Empirical studies ranging from those in transgenic mice to dopaminergic manipulation, deep brain stimulation, and genetics in humans largely support model predictions and provide the basis for further refinement. Finally, we discuss possible future directions and possible ways to integrate different types of models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785 · cited 1×
“In contrast, ventromedial regions, including the nucleus accumbens, are more implicated in reinforcement-guided learning and addiction-related processes [28,52,93].”
BackgroundGetting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Drug abuse: hedonic homeostatic dysregulation
Koob, G. F., & Moal, M. L. (1997) · Science · doi:10.1126/science.278.5335.52 · Cited by 2,535 · cited 1×
“In contrast, ventromedial regions, including the nucleus accumbens, are more implicated in reinforcement-guided learning and addiction-related processes [28,52,93].”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 2×
“Nonetheless, by adapting a ’top-down’ functional approach, these models have proven valuable in uncovering the cognitive mechanisms of reward-guided learning and decision-making, and have made several strides in linking these mechanisms to the neurobiology of the basal ganglia, prefrontal cortical, and dopamine system…”
BackgroundResult
Cited by
Modulation of Striatal Projection Systems by Dopamine
Gerfen, C. R., & Surmeier, D. J. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113641 · Cited by 1,718
Haber, S. N. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/shaber · Cited by 922
Corticostriatal connectivity and its role in disease
Shepherd, G. M. G. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3469 · Cited by 824
Motivation of extended behaviors by anterior cingulate cortex
Holroyd, C. B., & Yeung, N. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.008 · Cited by 618
Fine motor skills and early comprehension of the world: two new school readiness indicators.
Grissmer, D., Grimm, K. J., Aiyer, S. M., Murrah, W. M., & Steele, J. S. (2010) · Developmental Psychology · doi:10.1037/a0020104 · Cited by 585
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions
Bogacz, R., & Gurney, K. (2007) · Neural Computation · doi:10.1162/neco.2007.19.2.442 · Cited by 417
The ins and outs of the striatum: Role in drug addiction
Yager, L. M., Garcia, A. F., Wunsch, A. M., & Ferguson, S. M. (2015) · Neuroscience · doi:10.1016/j.neuroscience.2015.06.033 · Cited by 417
Robust Pacemaking in Substantia Nigra Dopaminergic Neurons
Guzman, J. N., Sánchez-Padilla, J., Chan, C. S., & Surmeier, D. J. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2519-09.2009 · Cited by 411
Personality, Addiction, Dopamine: Insights from Parkinson's Disease
Dagher, A., & Robbins, T. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.031 · Cited by 340
Neuromodulation of neurons and synapses
Nadim, F., & Bucher, D. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.05.003 · Cited by 338
Frontal Cortex and the Discovery of Abstract Action Rules
Badre, D., Kayser, A. S., & D'Esposito, M. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.025 · Cited by 329
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 317
Striatal Local Circuitry: A New Framework for Lateral Inhibition
Burke, D. A., Rotstein, H. G., & Alvarez, V. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.019 · Cited by 268
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A., & Glimcher, P. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3524-09.2009 · Cited by 255
Camille, N., Tsuchida, A., & Fellows, L. K. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3164-11.2011 · Cited by 195
“*Significantly worse performance of the patient group compared to the control group, one-tailed unpaired t test, p 0.05. action value in dACC, or in which dACC plays a very general “critic” role in learning about both stimuli and actions (Holroyd and Krigolson, 2007; Cohen and Frank, 2009).”
BackgroundInfluentialTwenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
The research domain criteria framework: The case for anterior cingulate cortex
Holroyd, C. B., & Umemoto, A. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.021 · Cited by 187
A critical review of habit learning and the basal ganglia
Seger, C. A., & Spiering, B. J. (2011) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2011.00066 · Cited by 170
Pan, W. X., Mao, T., & Dudman, J. T. (2010) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2010.00147 · Cited by 150
“…an apparent minority of the labeling, the zones of overlap may be disproportionately important to the function of the dorsolateral striatum in the rat (Brown and Sharp, 1995), as has been proposed in the primate (Graybiel et al., 1994; Haber, 2003; Cohen and Frank, 2009; Haber and Calzavara, 2009).”
BackgroundPutting desire on a budget: dopamine and energy expenditure, reconciling reward and resources
Beeler, J. A., Frazier, C. R. M., & Zhuang, X. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00049 · Cited by 136
“, 1995; Mink, 1996), respectively, sometimes referred to as the GO and NOGO pathways (Cohen and Frank, 2009).”
BackgroundWaltz, J. A., Frank, M. J., Wiecki, T. V., & Gold, J. M. (2011) · Neuropsychology · doi:10.1037/a0020882 · Cited by 132
Self-citationZiegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
Contributions of ERK signaling in the striatum to instrumental learning and performance
Shiflett, M. W., & Balleine, B. W. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.12.010 · Cited by 94
An anterior-posterior gradient of cognitive control within the dorsomedial striatum
Mestres-Missé, A., Turner, R., & Friederici, A. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.05.021 · Cited by 91
“However, in the last years this viewhas dramatically changed, establishing that basal ganglia and thalamus, through their interaction with cortex, are involved in a wide range of other functions, coordinating sensorimotor, limbic, and cognitive systems (Cohen and Frank, 2009; Middleton and Strick, 2000, 2002; Seger, 2…”
Time representation in reinforcement learning models of the basal ganglia
Gershman, S. J., Moustafa, A. A., & Ludvig, E. A. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2013.00194 · Cited by 81
“Computational models of reinforcement learning (RL) have had a profound influence on the contemporary understanding of the basal ganglia (Joel et al., 2002; Cohen and Frank, 2009).”
BackgroundInfluentialThe empirical status of predictive coding and active inference
Hodson, R., Mehta, M., & Smith, R. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105473 · Cited by 69
Dissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75
“Models that consider dopamine as a general teaching signal for cortico-striatal learning (Calabresi et al., 2007; Cohen and Frank, 2009; Reynolds and Wickens, 2002; O’Doherty et al., 2004) should be able to accommodate different responses for rewards that occur at different times, even if the…”
Background
Metadata
- Authors
- Cohen, M. X., & Frank, M. J.
- Venue
- Behavioural Brain Research
- Publisher
- Elsevier BV
- Published
- 2009-04
- Volume
- 199
- Issue
- 1
- Pages
- 141-156
- ISSN
- 0166-4328 (ISSN-L)
- OpenAlex
- W2076759530
- Cited by
- 237
- References
- 56
- FWCI
- 10.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.