The DOI matches its Crossref record; 108 of the 108 references it cites were found in a registry or an index.
From reinforcement learning models to psychiatric and neurological disorders
Open accessAbstract
Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits. We review this approach and its existing and potential applications to Parkinson's disease, Tourette's syndrome, attention-deficit/hyperactivity disorder, addiction, schizophrenia and preclinical animal models used to screen new antipsychotic drugs. The approach's proven explanatory and predictive power bodes well for the continued growth of computational psychiatry and computational neurology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Parallel organization of functionally segregated circuits linking basal ganglia and cortex
Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.09.030186.002041 · Cited by 8,527
The functional anatomy of basal ganglia disorders
Albin, R. L., Young, A. B., & Penney, J. B. (1989) · Trends in Neurosciences · doi:10.1016/0166-2236(89)90074-x · Cited by 5,353
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
Primate models of movement disorders of basal ganglia origin
DeLong, M. R. (1990) · Trends in Neurosciences · doi:10.1016/0166-2236(90)90110-v · Cited by 3,824
Kapur, S. (2003) · American Journal of Psychiatry · doi:10.1176/appi.ajp.160.1.13 · Cited by 2,813
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Cited by
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
Self-citationMontague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809
Cerebellar networks with the cerebral cortex and basal ganglia
Bostan, A. C., Dum, R. P., & Strick, P. L. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.03.003 · Cited by 759
Topographic organization of the human subcortex unveiled with functional connectivity gradients
Tian, Y., Margulies, D. S., Breakspear, M., & Zalesky, A. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-00711-6 · Cited by 784
Dopamine and glutamate in schizophrenia: biology, symptoms and treatment
McCutcheon, R. A., Krystal, J. H., & Howes, O. D. (2020) · World Psychiatry · doi:10.1002/wps.20693 · Cited by 612
Computational psychiatry: the brain as a phantastic organ
Friston, K. J., Stephan, K. E., Montague, R., & Dolan, R. J. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70275-5 · Cited by 531
Cellular Taxonomy of the Mouse Striatum as Revealed by Single-Cell RNA-Seq
Gokce, O., Stanley, G. M., Treutlein, B., Neff, N. F., Camp, J. G., Malenka, R. C., Rothwell, P. E., Fuccillo, M. V., Südhof, T. C., & Quake, S. R. (2016) · Cell Reports · doi:10.1016/j.celrep.2016.06.059 · Cited by 430
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
The Importance of Falsification in Computational Cognitive Modeling
Palminteri, S., Wyart, V., & Koechlin, E. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.03.011 · Cited by 448
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 426
Self-citationDeco, G., & Kringelbach, M. L. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.034 · Cited by 410
Collins, A. G. E., & Frank, M. J. (2014) · Psychological Review · doi:10.1037/a0037015 · Cited by 403
Self-citationFrancis, T. C., Chandra, R., Friend, D. M., Finkel, E., Dayrit, G., Miranda, J., Brooks, J. M., Iñiguez, S. D., O’Donnell, P., Kravitz, A., et al. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.07.021 · Cited by 379
Mundy, P. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.13720 · Cited by 396
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
Dopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381
“Dopamine has been implicated in a bewildering variety of processes and pathologies in the human brain; ranging from cortical excitability to attentional deficits [5], [6]; from motor control to akinesia and set switching deficits in Parkinson’s disease [7], [8], [9]; from working memory to schizophrenia [10], [11]; fr…”
BackgroundHassani, S. A., Oemisch, M., Balcarras, M., Westendorff, S., Ardid, S., van der Meer, M. A., Tiesinga, P., & Womelsdorf, T. (2017) · Scientific Reports · doi:10.1038/srep40606 · Cited by 360
Spontaneous eye blink rate as predictor of dopamine-related cognitive function—A review
Jongkees, B. J., & Colzato, L. S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.08.020 · Cited by 300
Anorexia nervosa and obesity are associated with opposite brain reward response
Frank, G. K. W., Reynolds, J. R., Shott, M. E., Jappe, L., Yang, T. T., Tregellas, J. R., & O'Reilly, R. C. (2012) · Neuropsychopharmacology · doi:10.1038/npp.2012.51 · Cited by 311
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 317
An integrative perspective on the role of dopamine in schizophrenia
Maia, T. V., & Frank, M. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2016.05.021 · Cited by 293
Computational psychiatry: towards a mathematically informed understanding of mental illness
Adams, R. A., Huys, Q. J. M., & Roiser, J. P. (2015) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2015-310737 · Cited by 285
“…which teach the ventral striatum (critic) the values of states, and the dorsal striatum (actor) to associate states with optimal actions (by increasing the excitation or reducing the inhibition of the currently selected action 6 ), with recent evidence for a causal role of dopamine in this regard.”
BackgroundInfluentialWang, X.-J., & Krystal, J. H. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.018 · Cited by 312
Goschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293
Dichter, G. S., Damiano, C. A., & Allen, J. A. (2012) · Journal of Neurodevelopmental Disorders · doi:10.1186/1866-1955-4-19 · Cited by 299
Computational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
The drift diffusion model as the choice rule in reinforcement learning
Pedersen, M. L., Frank, M. J., & Biele, G. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1199-y · Cited by 283
Self-citationOleson, E. B., Gentry, R. N., Chioma, V. C., & Cheer, J. F. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3087-12.2012 · Cited by 272
Dissociable Effects of Dopamine and Serotonin on Reversal Learning
den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., Franke, B., & Cools, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.030 · Cited by 255
Ng, T. H., Alloy, L. B., & Smith, D. V. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0644-x · Cited by 244
Depression: a decision-theoretic analysis
Huys, Q. J. M., Daw, N. D., & Dayan, P. (2015) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071714-033928 · Cited by 221
Reinforcement learning in depression: a review of computational research
Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.05.005 · Cited by 233
NOW vs LATER brain circuits: implications for obesity and addiction
Volkow, N. D., & Baler, R. D. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.04.002 · Cited by 237
Dopamine D2 receptors regulate the anatomical and functional balance of basal ganglia circuitry
Cazorla, M., de Carvalho, F. D., Chohan, M. O., Shegda, M., Chuhma, N., Rayport, S., Ahmari, S. E., Moore, H., & Kellendonk, C. (2014) · Neuron · doi:10.1016/j.neuron.2013.10.041 · Cited by 226
The molecular and cellular mechanisms of depression: a focus on reward circuitry
Fox, M. E., & Lobo, M. K. (2019) · Molecular Psychiatry · doi:10.1038/s41380-019-0415-3 · Cited by 232
Computational Models of Interoception and Body Regulation
Petzschner, F. H., Garfinkel, S. N., Paulus, M. P., Koch, C., & Khalsa, S. S. (2021) · Trends in Neurosciences · doi:10.1016/j.tins.2020.09.012 · Cited by 227
Striatal Dopamine and the Interface between Motivation and Cognition
Aarts, E., van Holstein, M., & Cools, R. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00163 · Cited by 211
Petzschner, F. H., Weber, L. A. E., Gard, T., & Stephan, K. E. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.05.012 · Cited by 181
Twenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
The research domain criteria framework: The case for anterior cingulate cortex
Holroyd, C. B., & Umemoto, A. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.021 · Cited by 187
Translational perspectives for computational neuroimaging
Stephan, K. E., Iglesias, S., Heinzle, J., & Diaconescu, A. O. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.008 · Cited by 169
Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis
Stephan, K. E., Bach, D. R., Fletcher, P. C., Flint, J., Frank, M. J., Friston, K. J., Heinz, A., Huys, Q. J. M., Owen, M. J., Binder, E. B., et al. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00361-2 · Cited by 180
Self-citationTAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry
Frässle, S., Aponte, E. A., Bollmann, S., Brodersen, K. H., Do, C. T., Harrison, O. K., Harrison, S. J., Heinzle, J., Iglesias, S., Kasper, L., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.680811 · Cited by 169
Assessing inter individual differences with task related functional neuroimaging
Lebreton, M., Bavard, S., Daunizeau, J., & Palminteri, S. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0681-8 · Cited by 160
SIRT1 Mediates Depression-Like Behaviors in the Nucleus Accumbens
Kim, H.-D., Hesterman, J., Call, T., Magazu, S., Keeley, E., Armenta, K., Kronman, H., Neve, R. L., Nestler, E. J., & Ferguson, D. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.0212-16.2016 · Cited by 158
FOXP2 and the role of cortico-basal ganglia circuits in speech and language evolution
Enard, W. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.008 · Cited by 149
Liu, Q.-R., Canseco-Alba, A., Zhang, H.-Y., Tagliaferro, P., Chung, M., Dennis, E., Sanabria, B., Schanz, N., Escosteguy-Neto, J. C., Ishiguro, H., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-17796-y · Cited by 147
Advances from neuroimaging studies in eating disorders
Frank, G. K. W. (2015) · CNS Spectrums · doi:10.1017/s1092852915000012 · Cited by 144
van Timmeren, T., Daams, J. G., van Holst, R. J., & Goudriaan, A. E. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.11.022 · Cited by 128
“One way to both improve patient classification and understanding of the mechanisms underlying performance deficits is by using computational modeling, i.e. ‘computational psychiatry’ (Huys et al., 2016; Maia and Frank, 2011).”
Ziegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
Role of Mesolimbic Brain-Derived Neurotrophic Factor in Depression
Koo, J. W., Chaudhury, D., Han, M.-H., & Nestler, E. J. (2019) · Biological Psychiatry · doi:10.1016/j.biopsych.2019.05.020 · Cited by 112
Der-Avakian, A., D'Souza, M. S., Pizzagalli, D. A., & Markou, A. (2013) · Translational Psychiatry · doi:10.1038/tp.2013.74 · Cited by 100
Dendritic remodeling of D1 neurons by RhoA/Rho-kinase mediates depression-like behavior
Fox, M. E., Chandra, R., Menken, M. S., Larkin, E. J., Nam, H., Engeln, M., Francis, T. C., & Lobo, M. K. (2020) · Molecular Psychiatry · doi:10.1038/s41380-018-0211-5 · Cited by 100
Palminteri, S., Lebreton, M., Worbe, Y., Hartmann, A., Lehéricy, S., Vidailhet, M., Grabli, D., & Pessiglione, M. (2011) · Brain · doi:10.1093/brain/awr147 · Cited by 90
Vinckier, F., Gaillard, R., Palminteri, S., Rigoux, L., Salvador, A., Fornito, A., Adapa, R., Krebs, M. O., Pessiglione, M., & Fletcher, P. C. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.73 · Cited by 87
Hauser, T. U., Iannaccone, R., Dolan, R. J., Ball, J., Hättenschwiler, J., Drechsler, R., Rufer, M., Brandeis, D., Walitza, S., & Brem, S. (2017) · Psychological Medicine · doi:10.1017/s0033291716003305 · Cited by 86
van der Schaaf, M. E., Warmerdam, E., Crone, E. A., & Cools, R. (2011) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2011.06.007 · Cited by 70
Charting the landscape of priority problems in psychiatry, part 2: pathogenesis and aetiology
Stephan, K. E., Binder, E. B., Breakspear, M., Dayan, P., Johnstone, E. C., Meyer-Lindenberg, A., Schnyder, U., Wang, X.-J., Bach, D. R., Fletcher, P. C., et al. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00360-0 · Cited by 64
Self-citationThe promises and pitfalls of applying computational models to neurological and psychiatric disorders
Teufel, C., & Fletcher, P. C. (2016) · Brain · doi:10.1093/brain/aww209 · Cited by 64
Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty
Sandhu, T. R., Xiao, B., & Lawson, R. P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105123 · Cited by 51
Sonuga‐Barke, E. J. S. (2011) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2011.02444.x · Cited by 43
The Effect of Dopamine Agonists on Adaptive and Aberrant Salience in Parkinson's Disease
Nagy, H., Levy-Gigi, E., Somlai, Z., Takáts, A., Bereczki, D., & Kéri, S. (2012) · Neuropsychopharmacology · doi:10.1038/npp.2011.278 · Cited by 42
“Therefore, therapies that restore dopamine level in the dorsal striatum result in dopamine ‘overdose’ in the ventral striatum, which may lead to impaired performance on some cognitive tasks (Gotham et al, 1988; Cools et al, 2001, 2003; Shohamy et al, 2006; Jahanshahi et al, 2010; MacDonald et al, 2011) and in some cas…”
BackgroundA neural mechanism for conserved value computations integrating information and rewards
Bromberg-Martin, E. S., Feng, Y.-Y., Ogasawara, T., White, J. K., Zhang, K., & Monosov, I. E. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01511-4 · Cited by 43
The Role of Serotonin in Orbitofrontal Function and Obsessive-Compulsive Disorder
Maia, T. V., & Cano-Colino, M. (2015) · Clinical Psychological Science · doi:10.1177/2167702614566809 · Cited by 42
Self-citationDepressive rumination and the C957T polymorphism of the DRD2 gene
Whitmer, A. J., & Gotlib, I. H. (2012) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-012-0112-z · Cited by 30
Reinforcement learning and bandits for speech and language processing: Tutorial, review and outlook
Lin, B. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122254 · Cited by 28
Heym, N., Kantini, E., Checkley, H. L. R., & Cassaday, H. J. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.01.012 · Cited by 13
Alaghband-rad, J., Dashti, B., Tehranidoost, M., Zargarinejad, G., & FarhadBeigi, P. (2021) · Journal of Nervous & Mental Disease · doi:10.1097/nmd.0000000000001247 · Cited by 10
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Addressing altered anticipation as a transdiagnostic target through computational psychiatry
Sepúlveda, P., Aitsahalia, I., Kumar, K., Atkin, T., & Iigaya, K. (2025) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2025.02.014 · Cited by 3
Metadata
- Authors
- Maia, T. V., & Frank, M. J.
- Venue
- Nature Neuroscience
- Published
- 2011-01-26
- Volume
- 14
- Issue
- 2
- Pages
- 154-162
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2058279623
- Cited by
- 722
- References
- 108
- FWCI
- 17.39
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.