By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Abstract
To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error. Here, we show, using two cognitive procedural learning tasks, that Parkinson's patients off medication are better at learning to avoid choices that lead to negative outcomes than they are at learning from positive outcomes. Dopamine medication reverses this bias, making patients more sensitive to positive than negative outcomes. This pattern was predicted by our biologically based computational model of basal ganglia–dopamine interactions in cognition, which has separate pathways for “Go” and “NoGo” responses that are differentially modulated by positive and negative reinforcement.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Neural mechanisms underlying decision making and learning involve fronto-striatal networks generating reward prediction errors which are highly sensitive to dopaminergic modulation (Frank et al. 2004; Kehagia et al. 2010; Schultz and Dickinson 2000).”
From Dopaminergic drug treatment remediates exaggerated cingulate prediction error responses in obsessive-compulsive disorder (2019) · cites this paper as Frank et al. 2004
In the citing paper
“Operant learning is, however, known to be impaired in PD (Avila et al., 2009; Frank et al., 2004; Rushworth et al., 1998; Rutledge et al., 2009; Shohamy et al., 2005).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Frank et al., 2004
In the citing paper
“Substantial work in conditions such as Parkinson’s disease, in which dopamine is reduced, shows that manipulations favoring D1 and D2 dopamine receptors result in effects that are most readily interpretable in a model-free manner (Frank et al., 2004).”
From Goals and Habits in the Brain (2013) · cites this paper as Frank et al., 2004
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS
MINK, J. W. (1996) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00042-1 · Cited by 2,486
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968
Uneven Pattern of Dopamine Loss in the Striatum of Patients with Idiopathic Parkinson's Disease
Kish, S. J., Shannak, K., & Hornykiewicz, O. (1988) · New England Journal of Medicine · doi:10.1056/nejm198804073181402 · Cited by 1,953
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
A neuropsychological theory of multiple systems in category learning.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998) · Psychological Review · doi:10.1037//0033-295x.105.3.442 · Cited by 1,223
The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia
Gerfen, C. R. (1992) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.15.030192.001441 · Cited by 1,004
Cools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
Research works printed but not linked: 6
- 10.1037/0033-295x.105.3.442 · doi:10.1037/0033-295x.105.3.442 · Paper page
- M. Frank J. Cognit. Neurosci. in press.
- R. C. O'Reilly M. J. Frank ICS Tech. Rep. University of Colorado Institute of Cognitive Science (2004).
- 10.1088/0954-898x_8_4_001 · doi:10.1088/0954-898x_8_4_001 · Paper page
- M. Frank J. Rudy W. Levy R. O'Reilly Mem. Cognit. in press.
- 10.1037/0097-7403.17.3.334 · doi:10.1037/0097-7403.17.3.334 · Paper page
Cited by
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
“Frank has proposed a neural-network model for choice between appetitive and aversive habitual valuation”
BackgroundInverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Multiple dopamine functions at different time courses
Schultz, W. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135722 · Cited by 1,415
“…1988) and show reductions in emotional responses to affective and approachgenerating stimuli (Canavan et al. 1989, Linden et al. 1990, Vriezen & Moscovitch 1990, Sprengelmeyer et al. 1995, Knowlton et al. 1996), which may be more pronouced for positive than negative reinforcers (Frank et al. 2004).”
BackgroundMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Event perception: a mind-brain perspective
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.2.273 · Cited by 1,148
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
Hold Your Horses: Impulsivity, Deep Brain Stimulation, and Medication in Parkinsonism
Frank, M. J., Samanta, J., Moustafa, A. A., & Sherman, S. J. (2007) · Science · doi:10.1126/science.1146157 · Cited by 1,120
Self-citationO'Reilly, R. C., & Frank, M. J. (2006) · Neural Computation · doi:10.1162/089976606775093909 · Cited by 1,066
Self-citationDolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Distinct roles for direct and indirect pathway striatal neurons in reinforcement
Kravitz, A. V., Tye, L. D., & Kreitzer, A. C. (2012) · Nature Neuroscience · doi:10.1038/nn.3100 · Cited by 972
The cognitive functions of the caudate nucleus
Grahn, J. A., Parkinson, J. A., & Owen, A. M. (2008) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2008.09.004 · Cited by 940
The Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928
Cools, R. (2006) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2005.03.024 · Cited by 903
Kehagia, A. A., Barker, R. A., & Robbins, T. W. (2010) · The Lancet Neurology · doi:10.1016/s1474-4422(10)70212-x · Cited by 881
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809
Reward‐Related Responses in the Human Striatum
DELGADO, M. R. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.002 · Cited by 736
From reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 722
Self-citationTen simple rules for the computational modeling of behavioral data
Wilson, R. C., & Collins, A. G. (2019) · eLife · doi:10.7554/elife.49547 · Cited by 710
Shohamy, D., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 707
Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making
Frank, M. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.006 · Cited by 706
Self-citationReinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706111104 · Cited by 683
Self-citationSerotoninergic regulation of emotional and behavioural control processes
Cools, R., Roberts, A. C., & Robbins, T. W. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.10.011 · Cited by 656
“Such effects would certainly mirror similar interactions between phasic and tonic neurotransmission modes that have been proposed for dopamine and noradrenaline, where tonic levels regulate phasic responses to behaviourally relevant stimuli [72,83,84].”
Theta lingua franca: A common mid‐frontal substrate for action monitoring processes
Cavanagh, J. F., Zambrano‐Vazquez, L., & Allen, J. J. B. (2012) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01293.x · Cited by 651
The basal ganglia: learning new tricks and loving it
Graybiel, A. M. (2005) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2005.10.006 · Cited by 649
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
“The opposite is thought to occur during pauses in DA neuron firing, thus supporting plasticity in the indirect pathway, thereby promoting punishment learning (188).”
BackgroundA rotarod test for evaluation of motor skill learning
Shiotsuki, H., Yoshimi, K., Shimo, Y., Funayama, M., Takamatsu, Y., Ikeda, K., Takahashi, R., Kitazawa, S., & Hattori, N. (2010) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2010.03.026 · Cited by 627
Frontal Cortex and the Hierarchical Control of Behavior
Badre, D., & Nee, D. E. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.005 · Cited by 611
Fractionating impulsivity: neuropsychiatric implications
Dalley, J. W., & Robbins, T. W. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2017.8 · Cited by 607
Reward Changes Salience in Human Vision via the Anterior Cingulate
Hickey, C., Chelazzi, L., & Theeuwes, J. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1026-10.2010 · Cited by 606
“Second, low-magnitude reward should result in a relative devaluation of features that characterize a target such that attention is less likely to be deployed to these features (Berridge and Robinson, 1998; Frank et al., 2004; Della Libera and Chelazzi, 2009).”
BackgroundPrefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
Karbach, J., & Verhaeghen, P. (2014) · Psychological Science · doi:10.1177/0956797614548725 · Cited by 589
Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 585
Self-citationStriatal circuits for reward learning and decision-making
Cox, J., & Witten, I. B. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0189-2 · Cited by 568
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
Integrating Memories in the Human Brain: Hippocampal-Midbrain Encoding of Overlapping Events
Shohamy, D., & Wagner, A. D. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.023 · Cited by 539
Yacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
Jones, M., & Love, B. C. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10003134 · Cited by 533
Reward Processing in Schizophrenia: A Deficit in the Representation of Value
Gold, J. M., Waltz, J. A., Prentice, K. J., Morris, S. E., & Heerey, E. A. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn068 · Cited by 521
A high-resolution probabilistic in vivo atlas of human subcortical brain nuclei
Pauli, W. M., Nili, A. N., & Tyszka, J. M. (2018) · Scientific Data · doi:10.1038/sdata.2018.63 · Cited by 517
How prediction errors shape perception, attention, and motivation
den Ouden, H. E. M., Kok, P., & de Lange, F. P. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00548 · Cited by 512
“Systemic manipulations of dopamine levels in patients and healthy subjects show opposite effects on rewardand punishment-based learning (Frank et al., 2004; Moustafa et al., 2008; Cools et al., 2009; Palminteri et al., 2009).”
BackgroundCortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks
Lo, C.-C., & Wang, X.-J. (2006) · Nature Neuroscience · doi:10.1038/nn1722 · Cited by 505
“The indirect pathway is likely to produce global inhibition of undesired saccade”
BackgroundA unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501
Schott, B. H., Minuzzi, L., Krebs, R. M., Elmenhorst, D., Lang, M., Winz, O. H., Seidenbecher, C. I., Coenen, H. H., Heinze, H.-J., Zilles, K., et al. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2058-08.2008 · Cited by 496
Differential Innervation of Direct- and Indirect-Pathway Striatal Projection Neurons
Wall, N. R., De La Parra, M., Callaway, E. M., & Kreitzer, A. C. (2013) · Neuron · doi:10.1016/j.neuron.2013.05.014 · Cited by 489
Barch, D. M., & Dowd, E. C. (2010) · Schizophrenia Bulletin · doi:10.1093/schbul/sbq068 · Cited by 477
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
Action, Outcome, and Value: A Dual-System Framework for Morality
Cushman, F. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313495594 · Cited by 473
Frank, M. J., & O'Reilly, R. C. (2006) · Behavioral Neuroscience · doi:10.1037/0735-7044.120.3.497 · Cited by 472
Self-citationHuang, V. S., Haith, A., Mazzoni, P., & Krakauer, J. W. (2011) · Neuron · doi:10.1016/j.neuron.2011.04.012 · Cited by 472
“Operant learning is, however, known to be impaired in PD (Avila et al., 2009; Frank et al., 2004; Rushworth et al., 1998; Rutledge et al., 2009; Shohamy et al., 2005).”
Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009) · Nature Neuroscience · doi:10.1038/nn.2342 · Cited by 471
Self-citationFrontal theta links prediction errors to behavioral adaptation in reinforcement learning
Cavanagh, J. F., Frank, M. J., Klein, T. J., & Allen, J. J. B. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.080 · Cited by 469
Self-citationWorking-memory capacity protects model-based learning from stress
Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A., & Daw, N. D. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1312011110 · Cited by 465
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456
Trial-by-trial data analysis using computational models
Daw, N. D. (2011) · Decision Making, Affect, and Learning · doi:10.1093/acprof:oso/9780199600434.003.0001 · Cited by 456
Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2055 · Cited by 452
Self-citationLeeman, R. F., & Potenza, M. N. (2012) · Psychopharmacology · doi:10.1007/s00213-011-2550-7 · Cited by 448
Reinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J., & Daw, N. D. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-122414-033625 · Cited by 446
A distributional code for value in dopamine-based reinforcement learning
Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020) · Nature · doi:10.1038/s41586-019-1924-6 · Cited by 443
Inflammation Effects on Motivation and Motor Activity: Role of Dopamine
Felger, J. C., & Treadway, M. T. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2016.143 · Cited by 442
Habenula: crossroad between the basal ganglia and the limbic system
Hikosaka, O., Sesack, S. R., Lecourtier, L., & Shepard, P. D. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3463-08.2008 · Cited by 432
Genetically Determined Differences in Learning from Errors
Klein, T. A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D. Y., & Ullsperger, M. (2007) · Science · doi:10.1126/science.1145044 · Cited by 426
Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases
Frank, M. J., Woroch, B. S., & Curran, T. (2005) · Neuron · doi:10.1016/j.neuron.2005.06.020 · Cited by 426
Self-citationGo and no-go learning in reward and punishment: Interactions between affect and effect
Guitart-Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 421
Caligiore, D., Pezzulo, G., Baldassarre, G., Bostan, A. C., Strick, P. L., Doya, K., Helmich, R. C., Dirkx, M., Houk, J., Jörntell, H., et al. (2017) · The Cerebellum · doi:10.1007/s12311-016-0763-3 · Cited by 421
The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions
Bogacz, R., & Gurney, K. (2007) · Neural Computation · doi:10.1162/neco.2007.19.2.442 · Cited by 417
Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding
Huys, Q. J. M., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R. J., & Dayan, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002028 · Cited by 416
A computational model of inhibitory control in frontal cortex and basal ganglia.
Wiecki, T. V., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0031542 · Cited by 404
Self-citationCollins, A. G. E., & Frank, M. J. (2014) · Psychological Review · doi:10.1037/a0037015 · Cited by 403
Self-citationMedication-Related Impulse Control and Repetitive Behaviors in Parkinson Disease
Voon, V., & Fox, S. H. (2007) · Archives of Neurology · doi:10.1001/archneur.64.8.1089 · Cited by 397
Banishing the homunculus: Making working memory work
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2005.04.067 · Cited by 385
Self-citationSerotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381
Optimal decision-making theories: linking neurobiology with behaviour
Bogacz, R. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.12.006 · Cited by 379
Chermahini, S. A., & Hommel, B. (2010) · Cognition · doi:10.1016/j.cognition.2010.03.007 · Cited by 374
A Physiologically Plausible Model of Action Selection and Oscillatory Activity in the Basal Ganglia
Humphries, M. D., Stewart, R. D., & Gurney, K. N. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.3486-06.2006 · Cited by 370
Cools, R., Frank, M. J., Gibbs, S. E., Miyakawa, A., Jagust, W., & D'Esposito, M. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4467-08.2009 · Cited by 364
Self-citationBodi, N., Keri, S., Nagy, H., Moustafa, A., Myers, C. E., Daw, N., Dibo, G., Takats, A., Bereczki, D., & Gluck, M. A. (2009) · Brain · doi:10.1093/brain/awp094 · Cited by 363
The dissociable effects of punishment and reward on motor learning
Galea, J. M., Mallia, E., Rothwell, J., & Diedrichsen, J. (2015) · Nature Neuroscience · doi:10.1038/nn.3956 · Cited by 363
Cognition and resting-state functional connectivity in schizophrenia
Sheffield, J. M., & Barch, D. M. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.12.007 · Cited by 362
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
Role of dopamine in the motivational and cognitive control of behavior
Cools, R. (2008) · The Neuroscientist · doi:10.1177/1073858408317009 · Cited by 352
Learning with Reinforcement: the Role of Immediate Feedback and the Internal Model of the Situation
Kozunova, G. L., Воронин, Н. А., Venediktov, V. V., & Stroganova, T. A. (2019) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-019-00852-7 · Cited by 351
Stocco, A., Lebiere, C., & Anderson, J. R. (2010) · Psychological Review · doi:10.1037/a0019077 · Cited by 350
Forming beliefs: Why valence matters
Sharot, T., & Garrett, N. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.11.002 · Cited by 345
Personality, Addiction, Dopamine: Insights from Parkinson's Disease
Dagher, A., & Robbins, T. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.031 · Cited by 340
Calabresi, P., Picconi, B., Parnetti, L., & Di Filippo, M. (2006) · The Lancet Neurology · doi:10.1016/s1474-4422(06)70600-7 · Cited by 338
Frank, M. J., & Badre, D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr114 · Cited by 337
Self-citationDunsmoor, J. E., Niv, Y., Daw, N., & Phelps, E. A. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.028 · Cited by 329
Frontal Cortex and the Discovery of Abstract Action Rules
Badre, D., Kayser, A. S., & D'Esposito, M. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.025 · Cited by 329
Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.09.042 · Cited by 327
Self-citationPredictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J., & Daw, N. D. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 326
Bari, A., Theobald, D. E., Caprioli, D., Mar, A. C., Aidoo-Micah, A., Dalley, J. W., & Robbins, T. W. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.233 · Cited by 325
Effects of Expectation on Placebo-Induced Dopamine Release in Parkinson Disease
Lidstone, S. C., Schulzer, M., Dinelle, K., Mak, E., Sossi, V., Ruth, T. J., de la Fuente-Fernández, R., Phillips, A. G., & Stoessl, A. J. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.88 · Cited by 323
The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers
Botvinick, M. M., & Cohen, J. D. (2014) · Cognitive Science · doi:10.1111/cogs.12126 · Cited by 322
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 317
Self-citationApplication of machine learning in intelligent fish aquaculture: A review
Zhao, S., Zhang, S., Liu, J., Wang, H., Zhu, J., Li, D., & Zhao, R. (2021) · Aquaculture · doi:10.1016/j.aquaculture.2021.736724 · Cited by 314
Spatial and temporal scales of dopamine transmission
Liu, C., Goel, P., & Kaeser, P. S. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00455-7 · Cited by 301
Metadata
- Authors
- Frank, M. J., Seeberger, L. C., & O'Reilly, R. C.
- Affiliations
- University of Colorado Boulder (US)
- Department of Psychology and Center for Neuroscience, University of Colorado Boulder, Boulder, CO 80309–0345, USA.
- Corresponding author
- Frank, Michael J.
- Venue
- Science
- Published
- 2004-12-10
- Volume
- 306
- Issue
- 5703
- Pages
- 1940-1943
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2163647009
- Cited by
- 1,964
- References
- 33
- Citations by year
- 2026: 57 · 2025: 73 · 2024: 83 · 2023: 79 · 2022: 91 · 2021: 99 · 2020: 134 · 2019: 104 · 2018: 125 · 2017: 111 · 2016: 115 · 2015: 111 · 2014: 134 · 2013: 111 · 2012: 108
- FWCI
- 11.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (R01MH069597, MH 069597-01)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Health Sciences › Medicine › Clinical Neurology › Parkinson's Disease Mechanisms and Treatments
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- Parkinson's disease, parkinsonism, dopamine, basal ganglia, positive reinforcement, negative reinforcement, avoidance learning, procedural learning, computational model, trial-and-error learning
- MeSH terms
- Aged, Antiparkinson Agents, Basal Ganglia, Cognition, Computer Simulation, Dopamine, Female, Frontal Lobe, Humans, Learning, Male, Middle Aged, Models, Neurological, Parkinson Disease, Probability, Reinforcement, Psychology, Matched-Pair Analysis, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.