By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here, we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically based computational model of basal ganglia–dopamine interactions in cognition, which has separate pathways for “Go” and “NoGo” responses that are differentially modulated by positive and negative reinforcement.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Neural mechanisms underlying decision making and learning involve fronto-striatal networks generating reward prediction errors which are highly sensitive to dopaminergic modulation (Frank et al. 2004; Kehagia et al. 2010; Schultz and Dickinson 2000).”
From Dopaminergic drug treatment remediates exaggerated cingulate prediction error responses in obsessive-compulsive disorder (2019) · cites this paper as Frank et al. 2004
In the citing paper
“Operant learning is, however, known to be impaired in PD (Avila et al., 2009; Frank et al., 2004; Rushworth et al., 1998; Rutledge et al., 2009; Shohamy et al., 2005).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Frank et al., 2004
In the citing paper
“Substantial work in conditions such as Parkinson’s disease, in which dopamine is reduced, shows that manipulations favoring D1 and D2 dopamine receptors result in effects that are most readily interpretable in a model-free manner (Frank et al., 2004).”
From Goals and Habits in the Brain (2013) · cites this paper as Frank et al., 2004
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968
Uneven Pattern of Dopamine Loss in the Striatum of Patients with Idiopathic Parkinson's Disease
Kish, S. J., Shannak, K., & Hornykiewicz, O. (1988) · New England Journal of Medicine · doi:10.1056/nejm198804073181402 · Cited by 1,953
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
A neuropsychological theory of multiple systems in category learning.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998) · Psychological Review · doi:10.1037//0033-295x.105.3.442 · Cited by 1,223
The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia
Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004
Cools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
Interactions between frontal cortex and basal ganglia in working memory: A computational model
FRANK, M. J., LOUGHRY, B., & O'REILLY, R. C. (2001) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.1.2.137 · Cited by 879
Self-citationDisinhibition as a basic process in the expression of striatal functions
Chevalier, G., & Deniau, J. M. (1990) · Trends in Neurosciences · doi:10.1016/0166-2236(90)90109-n · Cited by 865
Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli
Ungless, M. A., Magill, P. J., & Bolam, J. P. (2004) · Science · doi:10.1126/science.1093360 · Cited by 782
Hernández-López, S., Bargas, J., Surmeier, D. J., Reyes, A., & Galarraga, E. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-09-03334.1997 · Cited by 495
Swainson, R., Rogers, R. D., Sahakian, B. J., Summers, B. A., Polkey, C. E., & Robbins, T. W. (2000) · Neuropsychologia · doi:10.1016/s0028-3932(99)00103-7 · Cited by 494
Hernández-López, S., Tkatch, T., Perez-Garci, E., Galarraga, E., Bargas, J., Hamm, H., & Surmeier, D. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-24-08987.2000 · Cited by 443
Human Striatal Responses to Monetary Reward Depend On Saliency
Zink, C. F., Pagnoni, G., Martin-Skurski, M. E., Chappelow, J. C., & Berns, G. S. (2004) · Neuron · doi:10.1016/s0896-6273(04)00183-7 · Cited by 434
Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations
DELGADO, M. R., LOCKE, H. M., STENGER, V. A., & FIEZ, J. A. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.1.27 · Cited by 431
Bidirectional Regulation of DARPP-32 Phosphorylation by Dopamine
Nishi, A., Snyder, G. L., & Greengard, P. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-21-08147.1997 · Cited by 424
Transitive inference formation in pigeons.
von Fersen, L., Wynne, C. D., Delius, J. D., & Staddon, J. E. (1991) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037//0097-7403.17.3.334 · Cited by 397
Dopaminergic control of synaptic plasticity in the dorsal striatum
Centonze, D., Picconi, B., Gubellini, P., Bernardi, G., & Calabresi, P. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01485.x · Cited by 364
Shohamy, D. (2004) · Brain · doi:10.1093/brain/awh100 · Cited by 356
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298
Dopamine Transmission in the Human Striatum during Monetary Reward Tasks
Zald, D. H., Boileau, I., El-Dearedy, W., Gunn, R., McGlone, F., Dichter, G. S., & Dagher, A. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4643-03.2004 · Cited by 238
Aubert, I., Ghorayeb, I., Normand, E., & Bloch, B. (2000) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(20000228)418:1<22::aid-cne2>3.0.co;2-q · Cited by 170
Van Elzakker, M., O'Reilly, R. C., & Rudy, J. W. (2003) · Hippocampus · doi:10.1002/hipo.10083 · Cited by 168
Self-citationPersonality traits and brain dopaminergic function in Parkinson's disease
Kaasinen, V., Nurmi, E., Bergman, J., Eskola, O., Solin, O., Sonninen, P., & Rinne, J. O. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.231313198 · Cited by 148
Frank, M. J., Rudy, J. W., & O’Reilly, R. C. (2003) · Hippocampus · doi:10.1002/hipo.10084 · Cited by 140
Self-citationNovelty seeking and harm avoidance in Parkinson's disease: effects of asymmetric dopamine deficiency
Tomer, R. (2004) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp.2003.024885 · Cited by 123
Reinforcement accounts for transitive inference performance
Wynne, C. D. L. (1995) · Animal Learning & Behavior · doi:10.3758/bf03199936 · Cited by 114
Basal ganglia: structure and computations
Wickens, J. R. (1997) · Network Computation in Neural Systems · doi:10.1088/0954-898x_8_4_001 · Cited by 82
Transitive responding in animals and humans: Exaptation rather than adaptation?
Delius, J. D., & Siemann, M. (1998) · Behavioural Processes · doi:10.1016/s0376-6357(97)00072-7 · Cited by 71
Research works printed but not linked: 5
- 10.1037/0033-295X.105.3.442 · doi:10.1037/0033-295x.105.3.442 · Paper page
- M. Frank J. Cognit. Neurosci. in press.
- R. C. O'Reilly M. J. Frank ICS Tech. Rep. University of Colorado Institute of Cognitive Science (2004).
- M. Frank J. Rudy W. Levy R. O'Reilly Mem. Cognit. in press.
- 10.1037/0097-7403.17.3.334 · doi:10.1037/0097-7403.17.3.334 · Paper page
Cited by
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
“Frank has proposed a neural-network model for choice between appetitive and aversive habitual valuation”
BackgroundInverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Multiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415
Metadata
- Authors
- Frank, M. J., Seeberger, L. C., & O'Reilly, R. C.
- Affiliations
- University of Colorado Boulder (US)
- Department of Psychology and Center for Neuroscience, University of Colorado Boulder, Boulder, CO 80309–0345, USA.
- Corresponding author
- Frank, Michael J.
- Venue
- Science
- Published
- 2004-12-10
- Volume
- 306
- Issue
- 5703
- Pages
- 1940-1943
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2163647009
- Cited by
- 1,964
- References
- 32
- Citations by year
- 2026: 57 · 2025: 73 · 2024: 83 · 2023: 79 · 2022: 91 · 2021: 99 · 2020: 134 · 2019: 104 · 2018: 125 · 2017: 111 · 2016: 115 · 2015: 111 · 2014: 134 · 2013: 111 · 2012: 108
- FWCI
- 11.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (R01MH069597, MH 069597-01)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Health Sciences › Medicine › Clinical Neurology › Parkinson's Disease Mechanisms and Treatments
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- Parkinson's disease, parkinsonism, dopamine, basal ganglia, positive reinforcement, negative reinforcement, avoidance learning, procedural learning, computational model, trial-and-error learning
- MeSH terms
- Aged, Antiparkinson Agents, Basal Ganglia, Cognition, Computer Simulation, Dopamine, Female, Frontal Lobe, Humans, Learning, Male, Middle Aged, Models, Neurological, Parkinson Disease, Probability, Reinforcement, Psychology, Matched-Pair Analysis, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.