The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Neurocomputational models of basal ganglia function in learning, memory and choice
Open accessAbstract
The basal ganglia (BG) are critical for the coordination of several motor, cognitive, and emotional functions and become dysfunctional in several pathological states ranging from Parkinson's disease to Schizophrenia. Here we review principles developed within a neurocomputational framework of BG and related circuitry which provide insights into their functional roles in behavior. We focus on two classes of models: those that incorporate aspects of biological realism and constrained by functional principles, and more abstract mathematical models focusing on the higher level computational goals of the BG. While the former are arguably more “realistic”, the latter have a complementary advantage in being able to describe functional principles of how the system works in a relatively simple set of equations, but are less suited to making specific hypotheses about the roles of specific nuclei and neurophysiological processes. We review the basic architecture and assumptions of these models, their relevance to our understanding of the neurobiological and cognitive functions of the BG, and provide an update on the potential roles of biological details not explicitly incorporated in existing models. Empirical studies ranging from those in transgenic mice to dopaminergic manipulation, deep brain stimulation, and genetics in humans largely support model predictions and provide the basis for further refinement. Finally, we discuss possible future directions and possible ways to integrate different types of models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Drug abuse: hedonic homeostatic dysregulation
Koob, G. F., & Moal, M. L. (1997) · Science · doi:10.1126/science.278.5335.52 · Cited by 2,535 · cited 1×
“In contrast, ventromedial regions, including the nucleus accumbens, are more implicated in reinforcement-guided learning and addiction-related processes [28,52,93].”
BackgroundDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“…functional approach, these models have proven valuable in uncovering the cognitive mechanisms of reward-guided learning and decision-making,andhavemadeseveralstridesinlinkingthesemechanisms to the neurobiology of the basal ganglia, prefrontal cortical, and dopamine systems [34,111,47,122].”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
The neural basis of error detection: Conflict monitoring and the error-related negativity
Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.931 · Cited by 1,924
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“Some have proposed that the amount of uncertaintyplaysaroleindeterminingthelearningrate[14,47,187].”
BackgroundHaber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
The primate basal ganglia: Parallel and integrative networks
Haber, S. N. (2003) · Journal of Chemical Neuroanatomy · doi:10.1016/j.jchemneu.2003.10.003 · Cited by 1,479 · cited 2×
“The basal ganglia is richly anatomically connected to the frontal cortex and the thalamocortical motor system, via several distinct but partly overlapping loops [68,116,72].”
BackgroundNeuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 1×
“This led to the question of whether separate non-dopaminergic mechanisms in the brain are required to code negative prediction errors [46,12].”
BackgroundSurmeier, D. J., Ding, J., Day, M., Wang, Z., & Shen, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.008 · Cited by 1,203
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“…cortex [47], when to explore in a dynamic probabilistic environment [48,107], how much vigor to respond with in variable reward schedules [121], and when to supplement basic dopamine-mediated reinforcement learning with an explicit rule for detecting when the environment has changed [74].”
BackgroundAnimal intelligence; experimental studies
Thorndike, E. L. (1911) · doi:10.5962/bhl.title.55072 · Cited by 1,100
Aron, A. R., Behrens, T. E., Smith, S., Frank, M. J., & Poldrack, R. A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0519-07.2007 · Cited by 982 · cited 2×
“Neuroimaging studies support this conclusion, whereby increased co-activation between dorsomedial frontal cortex and STN is associated with increasingly slowed response times in high but not low conflict conditions (Aron et al., 2007).”
BackgroundResultSelf-citationDopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens
Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.185 · Cited by 931
Learned predictions of error likelihood in the anterior cingulate cortex
Brown, J. W., & Braver, T. S. (2005) · Science · doi:10.1126/science.1105783 · Cited by 928
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
Cools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
A cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879
Interactions between frontal cortex and basal ganglia in working memory: A computational model
FRANK, M. J., LOUGHRY, B., & O'REILLY, R. C. (2001) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.1.2.137 · Cited by 879
Self-citationPrediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004) · Nature Neuroscience · doi:10.1038/nn1279 · Cited by 779
Endocannabinoid-mediated rescue of striatal LTD and motor deficits in Parkinson's disease models
Kreitzer, A. C., & Malenka, R. C. (2007) · Nature · doi:10.1038/nature05506 · Cited by 735
Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making
Frank, M. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.006 · Cited by 706
Self-citationBOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Joel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614 · cited 1×
“The primary value (PV) system, mediated by patch-like striosomal neurons in the ventral striatum, is responsible for learning when unconditioned rewards will occur, and act to cancel out the dopamine burst when these are expected (due to inhibitory projections from striosomes into SNc and VTA [86]).”
BackgroundTemporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
Nieuwenhuis, S., Holroyd, C. B., Mol, N., & Coles, M. G. H. (2004) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2004.05.003 · Cited by 557
Substantia nigra/ventral tegmental reward prediction error disruption in psychosis
Murray, G. K., Corlett, P. R., Clark, L., Pessiglione, M., Blackwell, A. D., Honey, G., Jones, P. B., Bullmore, E. T., Robbins, T. W., & Fletcher, P. C. (2008) · Molecular Psychiatry · doi:10.1038/sj.mp.4002058 · Cited by 532
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
Hernández-López, S., Tkatch, T., Perez-Garci, E., Galarraga, E., Bargas, J., Hamm, H., & Surmeier, D. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-24-08987.2000 · Cited by 443
The Dopamine Transporter and Attention-Deficit/Hyperactivity Disorder
Madras, B. K., Miller, G. M., & Fischman, A. J. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2004.10.011 · Cited by 393
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Gerfen, C., Keefe, K., & Gauda, E. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-12-08167.1995 · Cited by 283
Separate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241 · cited 2×
“, 2004), the story is likely to be more complex (Atallah et al., 2007).”
BackgroundGene–gene interaction associated with neural reward sensitivity
Yacubian, J., Sommer, T., Schroeder, K., Gläscher, J., Kalisch, R., Leuenberger, B., Braus, D. F., & Büchel, C. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0702029104 · Cited by 241
Frank, M. J., Santamaria, A., O'Reilly, R. C., & Willcutt, E. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301278 · Cited by 227 · cited 1×
“ADHD participants, who showed better positive than negative feedback learning while on stimulant medications [62] (Fig.”
BackgroundWörgötter, F., & Porr, B. (2005) · Neural Computation · doi:10.1162/0899766053011555 · Cited by 203
WICKENS, J. R., BUDD, C. S., HYLAND, B. I., & ARBUTHNOTT, G. W. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.016 · Cited by 165
Reinforcement learning and decision making in monkeys during a competitive game
Lee, D., Conroy, M. L., McGreevy, B. P., & Barraclough, D. J. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.07.007 · Cited by 156
Collins, P., Wilkinson, L. S., Everitt, B. J., Robbins, T. W., & Roberts, A. C. (2000) · Behavioral Neuroscience · doi:10.1037/0735-7044.114.1.3 · Cited by 129
Cited by
Modulation of Striatal Projection Systems by Dopamine
Gerfen, C. R., & Surmeier, D. J. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113641 · Cited by 1,718
Haber, S. N. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/shaber · Cited by 922
Corticostriatal connectivity and its role in disease
Shepherd, G. M. G. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3469 · Cited by 824
Motivation of extended behaviors by anterior cingulate cortex
Holroyd, C. B., & Yeung, N. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.008 · Cited by 618
Fine motor skills and early comprehension of the world: two new school readiness indicators.
Grissmer, D., Grimm, K. J., Aiyer, S. M., Murrah, W. M., & Steele, J. S. (2010) · Developmental Psychology · doi:10.1037/a0020104 · Cited by 585
Metadata
- Authors
- Cohen, M. X., & Frank, M. J.
- Venue
- Behavioural Brain Research
- Publisher
- Elsevier BV
- Published
- 2009-04
- Volume
- 199
- Issue
- 1
- Pages
- 141-156
- ISSN
- 0166-4328 (ISSN-L)
- OpenAlex
- W2076759530
- Cited by
- 237
- References
- 52
- FWCI
- 10.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.