By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here, we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically based computational model of basal ganglia–dopamine interactions in cognition, which has separate pathways for “Go” and “NoGo” responses that are differentially modulated by positive and negative reinforcement.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Neural mechanisms underlying decision making and learning involve fronto-striatal networks generating reward prediction errors which are highly sensitive to dopaminergic modulation (Frank et al. 2004; Kehagia et al. 2010; Schultz and Dickinson 2000).”
From Dopaminergic drug treatment remediates exaggerated cingulate prediction error responses in obsessive-compulsive disorder (2019) · cites this paper as Frank et al. 2004
In the citing paper
“Operant learning is, however, known to be impaired in PD (Avila et al., 2009; Frank et al., 2004; Rushworth et al., 1998; Rutledge et al., 2009; Shohamy et al., 2005).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Frank et al., 2004
In the citing paper
“Substantial work in conditions such as Parkinson’s disease, in which dopamine is reduced, shows that manipulations favoring D1 and D2 dopamine receptors result in effects that are most readily interpretable in a model-free manner (Frank et al., 2004).”
From Goals and Habits in the Brain (2013) · cites this paper as Frank et al., 2004
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968
Uneven Pattern of Dopamine Loss in the Striatum of Patients with Idiopathic Parkinson's Disease
Kish, S. J., Shannak, K., & Hornykiewicz, O. (1988) · New England Journal of Medicine · doi:10.1056/nejm198804073181402 · Cited by 1,953
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
A neuropsychological theory of multiple systems in category learning.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998) · Psychological Review · doi:10.1037//0033-295x.105.3.442 · Cited by 1,223
The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia
Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004
Cools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
Research works printed but not linked: 6
- 10.1037/0033-295x.105.3.442 · doi:10.1037/0033-295x.105.3.442 · Paper page
- M. Frank J. Cognit. Neurosci. in press.
- R. C. O'Reilly M. J. Frank ICS Tech. Rep. University of Colorado Institute of Cognitive Science (2004).
- 10.1088/0954-898x_8_4_001 · doi:10.1088/0954-898x_8_4_001 · Paper page
- M. Frank J. Rudy W. Levy R. O'Reilly Mem. Cognit. in press.
- 10.1037/0097-7403.17.3.334 · doi:10.1037/0097-7403.17.3.334 · Paper page
Cited by
Items 101–200 of 1,007
Dopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294
“Some studies have suggested that dopamine levels might have differential effects on positive and negative updating (Frank et al., 2004; Pessiglione et al., 2006).”
BackgroundInfluentialEye tracking and pupillometry are indicators of dissociable latent decision processes.
Cavanagh, J. F., Wiecki, T. V., Kochar, A., & Frank, M. J. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0035813 · Cited by 289
“…trials into two distinct sets associated with choosing the most rewarding option (defined as the accuracy of choosing A over C, D, E, and F) versus the avoiding the most punishing option (defined as the accuracy of choosing C, D, E, and F over the most punishing symbol, B; Frank et al., 2004).”
InfluentialSelf-citationReversal learning in Parkinson's disease depends on medication status and outcome valence
Cools, R., Altamirano, L., & D’Esposito, M. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.03.030 · Cited by 289
L-DOPA Disrupts Activity in the Nucleus Accumbens during Reversal Learning in Parkinson's Disease
Cools, R., Lewis, S. J. G., Clark, L., Barker, R. A., & Robbins, T. W. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301153 · Cited by 289
A neurobiological theory of automaticity in perceptual categorization
Ashby, F. G., Ennis, J. M., & Spiering, B. J. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.632 · Cited by 286
Contextual modulation of value signals in reward and punishment learning
Palminteri, S., Khamassi, M., Joffily, M., & Coricelli, G. (2015) · Nature Communications · doi:10.1038/ncomms9096 · Cited by 286
The drift diffusion model as the choice rule in reinforcement learning
Pedersen, M. L., Frank, M. J., & Biele, G. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1199-y · Cited by 283
Self-citationMechanisms underlying dopamine-mediated reward bias in compulsive behaviors
Voon, V., Pessiglione, M., Brezing, C., Gallea, C., Fernandez, H. H., Dolan, R. J., & Hallett, M. (2010) · Neuron · doi:10.1016/j.neuron.2009.12.027 · Cited by 280
Jocham, G., Klein, T. A., & Ullsperger, M. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3904-10.2011 · Cited by 278
Dopamine enhances willingness to exert effort for reward in Parkinson's disease
Chong, T. T.-J., Bonnelle, V., Manohar, S., Veromann, K.-R., Muhammed, K., Tofaris, G. K., Hu, M., & Husain, M. (2015) · Cortex · doi:10.1016/j.cortex.2015.04.003 · Cited by 276
van den Bos, W., Cohen, M. X., Kahnt, T., & Crone, E. A. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr198 · Cited by 276
Joshua, M., Adler, A., Mitelman, R., Vaadia, E., & Bergman, H. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3839-08.2008 · Cited by 273
Impulse Control and Related Disorders in Parkinson's Disease
Weintraub, D., & Claassen, D. O. (2017) · International Review of Neurobiology · doi:10.1016/bs.irn.2017.04.006 · Cited by 272
Leow, L.-A., de Rugy, A., Loftus, A. M., & Hammond, G. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00055 · Cited by 271
Dopamine Dysregulation Syndrome: An Overview of its Epidemiology, Mechanisms and Management
OʼSullivan, S. S., Evans, A. H., & Lees, A. J. (2009) · CNS Drugs · doi:10.2165/00023210-200923020-00005 · Cited by 265
Human substantia nigra neurons encode unexpected financial rewards
Zaghloul, K. A., Blanco, J. A., Weidemann, C. T., McGill, K., Jaggi, J. L., Baltuch, G. H., & Kahana, M. J. (2009) · Science · doi:10.1126/science.1167342 · Cited by 264
Curiosity and reward: Valence predicts choice and information prediction errors enhance learning.
Marvin, C. B., & Shohamy, D. (2016) · Journal of Experimental Psychology: General · doi:10.1037/xge0000140 · Cited by 264
Model-Based and Model-Free Mechanisms of Human Motor Learning
Haith, A. M., & Krakauer, J. W. (2013) · Advances in Experimental Medicine and Biology · doi:10.1007/978-1-4614-5465-6_1 · Cited by 263
Clarke, H. F., Robbins, T. W., & Roberts, A. C. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1521-08.2008 · Cited by 263
“Computational models of frontostriatal dynamics postulate that the striatum acts as an integrative gate to the flow of environmental information from other regions of the brain to the frontal lobes (Houk and Wise, 1995; Frank et al., 2001, 2004).”
BackgroundDopamine modulates novelty seeking behavior during decision making.
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2014) · Behavioral Neuroscience · doi:10.1037/a0037128 · Cited by 263
Shine, J. M., Matar, E., Ward, P. B., Frank, M. J., Moustafa, A. A., Pearson, M., Naismith, S. L., & Lewis, S. J. G. (2013) · Brain · doi:10.1093/brain/awt272 · Cited by 261
Self-citationComputational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
The role of the basal ganglia in learning and memory: Neuropsychological studies
Grahn, J. A., Parkinson, J. A., & Owen, A. M. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.11.020 · Cited by 255
Dissociable Effects of Dopamine and Serotonin on Reversal Learning
den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., Franke, B., & Cools, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.030 · Cited by 255
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A., & Glimcher, P. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3524-09.2009 · Cited by 255
Düzel, E., Bunzeck, N., Guitart-Masip, M., & Düzel, S. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.08.006 · Cited by 249
Rogers, R. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.165 · Cited by 249
Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning
Palminteri, S., Justo, D., Jauffret, C., Pavlicek, B., Dauta, A., Delmaire, C., Czernecki, V., Karachi, C., Capelle, L., Durr, A., et al. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.017 · Cited by 242
Carrots and sticks fail to change behavior in cocaine addiction
Ersche, K. D., Gillan, C. M., Jones, P. S., Williams, G. B., Ward, L. H. E., Luijten, M., de Wit, S., Sahakian, B. J., Bullmore, E. T., & Robbins, T. W. (2016) · Science · doi:10.1126/science.aaf3700 · Cited by 239
Neurocomputational models of basal ganglia function in learning, memory and choice
Cohen, M. X., & Frank, M. J. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.029 · Cited by 237
Self-citationLearning the opportunity cost of time in a patch-foraging task
Constantino, S. M., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0350-y · Cited by 236
Cognitive deficits in Parkinson's disease: A cognitive neuroscience perspective
Robbins, T. W., & Cools, R. (2014) · Movement Disorders · doi:10.1002/mds.25853 · Cited by 235
Neuropsychiatric effects of subthalamic neurostimulation in Parkinson disease
Volkmann, J., Daniels, C., & Witt, K. (2010) · Nature Reviews Neurology · doi:10.1038/nrneurol.2010.111 · Cited by 234
Neuromodulation of reward‐based learning and decision making in human aging
Eppinger, B., Hämmerer, D., & Li, S. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06230.x · Cited by 234
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
Impulsive and compulsive behaviors in Parkinson's disease
Evans, A. H., Strafella, A. P., Weintraub, D., & Stacy, M. (2009) · Movement Disorders · doi:10.1002/mds.22505 · Cited by 233
Strauss, G. P., Frank, M. J., Waltz, J. A., Kasanova, Z., Herbener, E. S., & Gold, J. M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.10.015 · Cited by 233
Self-citationNucleus accumbens D2R cells signal prior outcomes and control risky decision-making
Zalocusky, K. A., Ramakrishnan, C., Lerner, T. N., Davidson, T. J., Knutson, B., & Deisseroth, K. (2016) · Nature · doi:10.1038/nature17400 · Cited by 233
Robinson, O. J., Cools, R., Carlisi, C. O., Sahakian, B. J., & Drevets, W. C. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010137 · Cited by 232
“This hypothesis would be compatible with evidence that the functioning of the mesolimbic dopaminergic system, which plays a major modulatory role within the limbic-cortical-striatal-pallidal-thalamic circuitry (30), is reduced in depression (both in general and in response to unpredicted reward) (1, 14, 31, 32) and wi…”
ResultThe Behavioral Activation System and Mania
Johnson, S. L., Edge, M. D., Holmes, M. K., & Carver, C. S. (2012) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032511-143148 · Cited by 229
Frank, M. J., Santamaria, A., O'Reilly, R. C., & Willcutt, E. (2007) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301278 · Cited by 227
Self-citationComputational Models of Interoception and Body Regulation
Petzschner, F. H., Garfinkel, S. N., Paulus, M. P., Koch, C., & Khalsa, S. S. (2021) · Trends in Neurosciences · doi:10.1016/j.tins.2020.09.012 · Cited by 227
“While no model currently links all levels of analysis, there are a few promising examples that have connected the activity of neuronal populations to behavior and symptoms leading to advances in the field of psychiatry [87] and neurology [88,89].”
BackgroundDopamine D2 receptors regulate the anatomical and functional balance of basal ganglia circuitry
Cazorla, M., de Carvalho, F. D., Chohan, M. O., Shegda, M., Chuhma, N., Rayport, S., Ahmari, S. E., Moore, H., & Kellendonk, C. (2014) · Neuron · doi:10.1016/j.neuron.2013.10.041 · Cited by 226
“The opposing function of both pathways is also discussed in the context of motivation and cognition, with the direct pathway being involved in learning from positive outcomes and the indirect pathway in learning from negative outcomes (Durieux et al., 2009; Ferguson et al., 2011; Frank et al., 2004; Graybiel, 2000; Hi…”
Stress and decision making: effects on valuation, learning, and risk-taking
Porcelli, A. J., & Delgado, M. R. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.11.015 · Cited by 226
Pizzagalli, D. A., Evins, A. E., Schetter, E. C., Frank, M. J., Pajtas, P. E., Santesso, D. L., & Culhane, M. (2008) · Psychopharmacology · doi:10.1007/s00213-007-0957-y · Cited by 219
Self-citationBarch, D. M., Pagliaccio, D., & Luking, K. (2015) · Current Topics in Behavioral Neurosciences · doi:10.1007/7854_2015_376 · Cited by 218
Cohen, M. X., Elger, C. E., & Fell, J. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.21020 · Cited by 218
Basal ganglia and dopamine contributions to probabilistic category learning
Shohamy, D., Myers, C. E., Kalanithi, J., & Gluck, M. A. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.07.008 · Cited by 218
Dopaminergic Modulation of Decision Making and Subjective Well-Being
Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0702-15.2015 · Cited by 217
Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216
“This pattern also fits well with the known role of striatum and dopaminergic system in reward-guided action learning (Robbins and Everitt, 2002; Frank et al., 2004).”
A Transdiagnostic Review of Negative Symptom Phenomenology and Etiology
Strauss, G. P., & Cohen, A. S. (2017) · Schizophrenia Bulletin · doi:10.1093/schbul/sbx066 · Cited by 216
How Dopamine Enhances an Optimism Bias in Humans
Sharot, T., Guitart-Masip, M., Korn, C. W., Chowdhury, R., & Dolan, R. J. (2012) · Current Biology · doi:10.1016/j.cub.2012.05.053 · Cited by 215
Instructional control of reinforcement learning: A behavioral and neurocomputational investigation
Doll, B. B., Jacobs, W. J., Sanfey, A. G., & Frank, M. J. (2009) · Brain Research · doi:10.1016/j.brainres.2009.07.007 · Cited by 215
Self-citationAmo, R., Fredes, F., Kinoshita, M., Aoki, R., Aizawa, H., Agetsuma, M., Aoki, T., Shiraki, T., Kakinuma, H., Matsuda, M., et al. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.035 · Cited by 214
Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia
Collins, A. G. E., Brown, J. K., Gold, J. M., Waltz, J. A., & Frank, M. J. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.0989-14.2014 · Cited by 212
Self-citationStriatal Dopamine and the Interface between Motivation and Cognition
Aarts, E., van Holstein, M., & Cools, R. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00163 · Cited by 211
Fervaha, G., Graff-Guerrero, A., Zakzanis, K. K., Foussias, G., Agid, O., & Remington, G. (2013) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2013.08.003 · Cited by 211
Calabresi, P., Castrioto, A., Di Filippo, M., & Picconi, B. (2013) · The Lancet Neurology · doi:10.1016/s1474-4422(13)70118-2 · Cited by 210
The role of the basal ganglia in learning and memory: Insight from Parkinson’s disease
Foerde, K., & Shohamy, D. (2011) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2011.08.006 · Cited by 208
Neurobiological Basis of Language Learning Difficulties
Krishnan, S., Watkins, K. E., & Bishop, D. V. M. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.06.012 · Cited by 207
Ratcliff, R., & Frank, M. J. (2012) · Neural Computation · doi:10.1162/neco_a_00270 · Cited by 207
Self-citationShi, Y., Du, J., Ahn, C. R., & Ragan, E. (2019) · Automation in Construction · doi:10.1016/j.autcon.2019.04.015 · Cited by 206
Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia
Humphries, M. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00009 · Cited by 206
Stimulus-dependent dopamine release in attention-deficit/hyperactivity disorder.
Sikström, S., & Söderlund, G. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.1047 · Cited by 205
Cerebellar and Prefrontal Cortex Contributions to Adaptation, Strategies, and Reinforcement Learning
Taylor, J. A., & Ivry, R. B. (2014) · Progress in Brain Research · doi:10.1016/b978-0-444-63356-9.00009-1 · Cited by 205
Modeling Trait Anxiety: From Computational Processes to Personality
Raymond, J. G., Steele, J. D., & Seriès, P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00001 · Cited by 203
Cools, R., & Arnsten, A. F. T. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-021-01100-8 · Cited by 202
Nonomura, S., Nishizawa, K., Sakai, Y., Kawaguchi, Y., Kato, S., Uchigashima, M., Watanabe, M., Yamanaka, K., Enomoto, K., Chiken, S., et al. (2018) · Neuron · doi:10.1016/j.neuron.2018.08.002 · Cited by 202
The role of dopamine in the brain - lessons learned from Parkinson's disease
Meder, D., Herz, D. M., Rowe, J. B., Lehéricy, S., & Siebner, H. R. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.11.021 · Cited by 201
Frontal Theta Overrides Pavlovian Learning Biases
Cavanagh, J. F., Eisenberg, I., Guitart-Masip, M., Huys, Q., & Frank, M. J. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.5754-12.2013 · Cited by 199
Self-citationSubcortical Structures and Cognition
Koziol, L. F., & Budding, D. E. (2009) · Springer eBooks · doi:10.1007/978-0-387-84868-6 · Cited by 198
Disconnecting force from money: effects of basal ganglia damage on incentive motivation
Schmidt, L., d’Arc, B. F., Lafargue, G., Galanaud, D., Czernecki, V., Grabli, D., Schüpbach, M., Hartmann, A., Lévy, R., Dubois, B., et al. (2008) · Brain · doi:10.1093/brain/awn045 · Cited by 197
Reward-Dependent Modulation of Movement Variability
Pekny, S. E., Izawa, J., & Shadmehr, R. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.3244-14.2015 · Cited by 197
Reappraising striatal D1- and D2-neurons in reward and aversion
Soares-Cunha, C., Coimbra, B., Sousa, N., & Rodrigues, A. J. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.05.021 · Cited by 195
Schott, B. H., Niehaus, L., Wittmann, B. C., Schutze, H., Seidenbecher, C. I., Heinze, H.-J., & Duzel, E. (2007) · Brain · doi:10.1093/brain/awm147 · Cited by 195
Feedback Timing Modulates Brain Systems for Learning in Humans
Foerde, K., & Shohamy, D. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.2701-11.2011 · Cited by 194
Impulse control disorders in Parkinson's disease: recent advances
Voon, V., Mehta, A. R., & Hallett, M. (2011) · Current Opinion in Neurology · doi:10.1097/wco.0b013e3283489687 · Cited by 193
A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism
Moustafa, A. A., Sherman, S. J., & Frank, M. J. (2008) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2008.07.011 · Cited by 192
Self-citationA reinforcement learning diffusion decision model for value-based decisions
Fontanesi, L., Gluth, S., Spektor, M. S., & Rieskamp, J. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1554-2 · Cited by 192
Neurogenetics and Pharmacology of Learning, Motivation, and Cognition
Frank, M. J., & Fossella, J. A. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.96 · Cited by 192
Self-citationTwenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
Differentiable plasticity: training plastic neural networks with backpropagationPreprint
Miconi, T., Clune, J., & Stanley, K. O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02464 · Cited by 189
The research domain criteria framework: The case for anterior cingulate cortex
Holroyd, C. B., & Umemoto, A. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.021 · Cited by 187
Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias
Doll, B. B., Hutchison, K. E., & Frank, M. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6486-10.2011 · Cited by 184
Self-citationDopaminergic modulation of striato-frontal connectivity during motor timing in Parkinson's disease
Jahanshahi, M., Jones, C. R. G., Zijlmans, J., Katzenschlager, R., Lee, L., Quinn, N., Frith, C. D., & Lees, A. J. (2010) · Brain · doi:10.1093/brain/awq012 · Cited by 184
Stocco, A., Yamasaki, B. L., Natalenko, R., & Prat, C. S. (2012) · International Journal of Bilingualism · doi:10.1177/1367006912456617 · Cited by 183
The basal ganglia in human learning
Seger, C. A. (2006) · The Neuroscientist · doi:10.1177/1073858405285632 · Cited by 182
Prefrontal cortex and basal ganglia contributions to visual working memory
Voytek, B., & Knight, R. T. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1007277107 · Cited by 181
Model-Based and Model-Free Decisions in Alcohol Dependence
Sebold, M., Deserno, L., Nebe, S., Schad, D. J., Garbusow, M., Hägele, C., Keller, J., Jünger, E., Kathmann, N., Smolka, M., et al. (2014) · Neuropsychobiology · doi:10.1159/000362840 · Cited by 181
Hämmerer, D., Li, S.-C., Müller, V., & Lindenberger, U. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21475 · Cited by 181
The Computational Development of Reinforcement Learning during Adolescence
Palminteri, S., Kilford, E. J., Coricelli, G., & Blakemore, S. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004953 · Cited by 180
Working memory and reward association learning impairments in obesity
Coppin, G., Nolan-Poupart, S., Jones-Gotman, M., & Small, D. M. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.10.004 · Cited by 179
Skvortsova, V., Palminteri, S., & Pessiglione, M. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1350-14.2014 · Cited by 177
“While the reward prediction errors in the positive valuation system have been linked to dopamine release (Schultz et al., 1997; Frank et al., 2004; Pessiglione et al., 2006), the origin of effort prediction error signals is less clear.”
BackgroundMotivational neural circuits underlying reinforcement learning
Averbeck, B. B., & Costa, V. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4506 · Cited by 177
Fouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
Reward Processing, Neuroeconomics, and Psychopathology
Zald, D. H., & Treadway, M. T. (2017) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-044957 · Cited by 176
Toward a unified framework for interpreting machine-learning models in neuroimaging
Kohoutová, L., Heo, J., Cha, S., Lee, S., Moon, T., Wager, T. D., & Woo, C.-W. (2020) · Nature Protocols · doi:10.1038/s41596-019-0289-5 · Cited by 176
A Framework for Industry 4.0 Readiness and Maturity of Smart Manufacturing Enterprises: A Case Study
Çınar, Z. M., Zeeshan, Q., & Korhan, O. (2021) · Sustainability · doi:10.3390/su13126659 · Cited by 176
Computational models of motivated action selection in corticostriatal circuits
Frank, M. J. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.013 · Cited by 175
Self-citationWhat Are Memories For? The Hippocampus Bridges Past Experience with Future Decisions
Biderman, N., Bakkour, A., & Shohamy, D. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.04.004 · Cited by 175
Metadata
- Authors
- Frank, M. J., Seeberger, L. C., & O'Reilly, R. C.
- Affiliations
- University of Colorado Boulder (US)
- Department of Psychology and Center for Neuroscience, University of Colorado Boulder, Boulder, CO 80309–0345, USA.
- Corresponding author
- Frank, Michael J.
- Venue
- Science
- Published
- 2004-12-10
- Volume
- 306
- Issue
- 5703
- Pages
- 1940-1943
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2163647009
- Cited by
- 1,964
- References
- 33
- Citations by year
- 2026: 57 · 2025: 73 · 2024: 83 · 2023: 79 · 2022: 91 · 2021: 99 · 2020: 134 · 2019: 104 · 2018: 125 · 2017: 111 · 2016: 115 · 2015: 111 · 2014: 134 · 2013: 111 · 2012: 108
- FWCI
- 11.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (R01MH069597, MH 069597-01)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Health Sciences › Medicine › Clinical Neurology › Parkinson's Disease Mechanisms and Treatments
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- Parkinson's disease, parkinsonism, dopamine, basal ganglia, positive reinforcement, negative reinforcement, avoidance learning, procedural learning, computational model, trial-and-error learning
- MeSH terms
- Aged, Antiparkinson Agents, Basal Ganglia, Cognition, Computer Simulation, Dopamine, Female, Frontal Lobe, Humans, Learning, Male, Middle Aged, Models, Neurological, Parkinson Disease, Probability, Reinforcement, Psychology, Matched-Pair Analysis, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.