Tonic dopamine: Opportunity costs and the control of response vigor

Niv, Yael; Daw, Nathaniel D.; Joel, Daphna; Dayan, Peter

doi:10.1007/s00213-006-0502-4

Cited by 1,133Neurotransmitter Receptor Influence on BehaviorNeural and Behavioral Psychology StudiesReinforcement Learning in Robotics

Tonic dopamine: Opportunity costs and the control of response vigor

Niv, Yael, Daw, Nathaniel D., Joel, Daphna, Dayan, Peter·2007·Psychopharmacology

DOI 10.1007/s00213-006-0502-4

Open this paper in CoChat — free account

Abstract

Rationale Dopamine neurotransmission has long been known to exert a powerful influence over the vigor, strength, or rate of responding. However, there exists no clear understanding of the computational foundation for this effect; predominant accounts of dopamine's computational function focus on a role for phasic dopamine in controlling the discrete selection between different actions and have nothing to say about response vigor or indeed the free-operant tasks in which it is typically measured.Objectives We seek to accommodate free-operant behavioral tasks within the realm of models of optimal control and thereby capture how dopaminergic and motivational manipulations affect response vigor.Methods We construct an average reward reinforcement learning model in which subjects choose both which action to perform and also the latency with which to perform it. Optimal control balances the costs of acting quickly against the benefits of getting reward earlier and thereby chooses a best response latency.Results In this framework, the long-run average rate of reward plays a key role as an opportunity cost and mediates motivational influences on rates and vigor of responding. We review evidence suggesting that the average reward rate is reported by tonic levels of dopamine putatively in the nucleus accumbens.Conclusions Our extension of reinforcement learning models to free-operant tasks unites psychologically and computationally inspired ideas about the role of tonic dopamine in striatum, explaining from a normative point of view why higher levels of dopamine might be associated with more vigorous responding.

Integrity signals

Registry

Crossref · 10.1007/s00213-006-0502-4

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Tonic DA levels are largely independent of environmental triggers and may index an individual's level of incentive-related drive.
“Tonic DA levels are largely independent of environmental triggers and may index an individual’s level of incentive-related drive (Niv et al, 2007; Willuhn, Wanat, Clark, & Phillips, 2010).”
From Incentive Motivation, Cognitive Control, and the Adolescent Brain: Is It Time for a Paradigm Shift? (2012) · cites this paper as Niv et al, 2007
Supportive evidence for the proposed interplay between tonic and phasic modes of DA activity and effects on behavior relies entirely on animal and computational models.
“Supportive evidence for this proposed interplay between tonic and phasic modes of DA activity and effects on behavior relies entirely on animal and computational models (Niv et al., 2007; Willuhn, 2010).”
From Incentive Motivation, Cognitive Control, and the Adolescent Brain: Is It Time for a Paradigm Shift? (2012) · cites this paper as Niv et al., 2007
Tonic DA activity regulates phasic signaling in an inhibitory fashion and phasic DA enhances tonic activity.
“These systems also interact as tonic DA activity regulates phasic signaling in an inhibitory fashion and phasic DA has been shown to enhance tonic activity (Niv, Daw et al. 2007).”
From Developmental imaging genetics: Linking dopamine function to adolescent behavior (2014) · cites this paper as Niv, Daw et al. 2007

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citation
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127 · see citing works
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601 · see citing works
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 2× · see citing works
“One possible explanation for these effects is that phasic signals transiently affect dopamine tone (Phillips and Wightman 2004; Roitman et al. 2004; Wise 2004), influencing vigor selection.”
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708 · see citing works
ON THE LAW OF EFFECT 1
Herrnstein, R. J. (1970) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1970.13-243 · Cited by 2,611 · see citing works
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Self-citation
RELATIVE AND ABSOLUTE STRENGTH OF RESPONSE AS A FUNCTION OF FREQUENCY OF REINFORCEMENT 1, 2
Herrnstein, R. J. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-267 · Cited by 2,281 · see citing works
Scalar expectancy theory and Weber's law in animal timing.
Gibbon, J. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.279 · Cited by 2,224 · see citing works
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224 · see citing works
Self-citation
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citation
Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia
Grace, A. A. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90196-u · Cited by 1,831 · see citing works
Learning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · see citing works
An opponent-process theory of motivation. I. Temporal dynamics of affect
Solomon, R. L., & Corbit, J. D. (1974) · Psychological Review · doi:10.1037/h0036128 · Cited by 1,704 · see citing works
Neuro-Dynamic Programming
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,486 · see citing works
The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450 · see citing works
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · see citing works
Motivation concepts in behavioral neuroscience
Berridge, K. C. (2004) · Physiology & Behavior · doi:10.1016/j.physbeh.2004.02.004 · Cited by 1,302 · see citing works
Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291 · see citing works
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · see citing works
Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · see citing works
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217 · see citing works
The basal ganglia: a vertebrate solution to the selection problem?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00319-4 · Cited by 1,199 · see citing works
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190 · see citing works
Time, rate, and conditioning.
Gallistel, C. R., & Gibbon, J. (2000) · Psychological Review · doi:10.1037/0033-295x.107.2.289 · Cited by 1,122 · see citing works
Subsecond dopamine release promotes cocaine seeking
Phillips, P. E. M., Stuber, G. D., Heien, M. L. A. V., Wightman, R. M., & Carelli, R. M. (2003) · Nature · doi:10.1038/nature01476 · Cited by 1,113 · see citing works
Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission
Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003) · Nature Neuroscience · doi:10.1038/nn1103 · Cited by 1,106 · see citing works
Mesocorticolimbic dopaminergic network: functional and regulatory roles
Moal, M. L., & Simon, H. (1991) · Physiological Reviews · doi:10.1152/physrev.1991.71.1.155 · Cited by 1,096 · see citing works
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · see citing works
Motivational control of goal-directed action
Dickinson, A., & Balleine, B. (1994) · Animal Learning & Behavior · doi:10.3758/bf03199951 · Cited by 995 · see citing works
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987 · see citing works
The principles of learning and behavior
Domjan, M. P. (1982) · Cited by 926 · see citing works
The role of dopamine in locomotor activity and learning
Beninger, R. J. (1983) · Brain Research Reviews · doi:10.1016/0165-0173(83)90038-3 · Cited by 848 · see citing works
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
Self-citation
Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine
Salamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817 · see citing works
A QUANTITATIVE ANALYSIS OF THE RESPONDING MAINTAINED BY INTERVAL SCHEDULES OF REINFORCEMENT1
Catania, A. C., & Reynolds, G. S. (1968) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1968.11-s327 · Cited by 796 · see citing works
Dopamine operates as a subsecond modulator of food seeking
Roitman, M. F., Stuber, G. D., Phillips, P. E. M., Wightman, R. M., & Carelli, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3823-03.2004 · Cited by 718 · see citing works
Integrative activity of the brain : an interdisciplinary approach
Konorski, J. (1967) · University of Chicago Press eBooks · Cited by 716 · see citing works
Theory of Motivation
Bolles, R. C. (1967) · Medical Entomology and Zoology · Cited by 610 · see citing works
Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior
Goto, Y., & Grace, A. A. (2005) · Nature Neuroscience · doi:10.1038/nn1471 · Cited by 576 · see citing works
STEVENS' HANDBOOK OF EXPERIMENTAL PSYCHOLOGY
Pashler, H. (2002) · Wiley eBooks · doi:10.1002/0471214426 · Cited by 576 · see citing works
A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0020 · Cited by 555 · see citing works
Integrative Activity of the Brain: An Interdisciplinary Approach
Ochs, S. (1968) · JAMA · doi:10.1001/jama.1968.03140050055025 · Cited by 526 · see citing works
Learning and Computational Neuroscience: Foundations of Adaptive Networks
Gabriel, M., & Moore, J. (1990) · Cited by 519 · see citing works
Time-Derivative Models of Pavlovian Reinforcement
Sutton, R. S., & Barto, A. G. (1990) · Cited by 496 · see citing works
Adaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0018 · Cited by 466 · see citing works
An Opponent-Process Theory of Motivation
Solomon, R. L., & Corbit, J. D. (2008) · Cited by 461 · see citing works
Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens
Taylor, J. R., & Robbins, T. W. (1984) · Psychopharmacology · doi:10.1007/bf00555222 · Cited by 440 · see citing works
A computational substrate for incentive salience
McClure, S. M., Daw, N. D., & Montague, P. R. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00177-2 · Cited by 416 · see citing works
Self-citation
A neural correlate of response bias in monkey caudate nucleus
Lauwereyns, J., Watanabe, K., Coe, B. C., & Hikosaka, O. (2002) · Nature · doi:10.1038/nature00892 · Cited by 415 · see citing works
Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation
Faure, A., Haberland, U., Condé, F., & Massioui, N. E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3894-04.2005 · Cited by 402 · see citing works
The Role of Learning in the Operation of Motivational Systems
Dickinson, A., & Balleine, B. W. (2002) · Wiley eBooks · doi:10.1002/0471214426.pas0312 · Cited by 387 · see citing works
Average reward reinforcement learning: Foundations, algorithms, and empirical results
Mahadevan, S. (1996) · Machine Learning · doi:10.1007/bf00114727 · Cited by 386 · see citing works
Neural substrates of latent inhibition: The switching model.
Weiner, I. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.442 · Cited by 380 · see citing works
The laterodorsal tegmentum is essential for burst firing of ventral tegmental area dopamine neurons
Lodge, D. J., & Grace, A. A. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0510715103 · Cited by 349 · see citing works
Functional Studies of the Central Catecholamines
Robbins, T. W., & Everitt, B. J. (1982) · International review of neurobiology · doi:10.1016/s0074-7742(08)60628-5 · Cited by 339 · see citing works
A functional effect of dopamine in the nucleus accumbens and in some other dopamine-rich parts of the rat brain
Jackson, D. M., And�n, N.-E., & Dahlstr�m, A. (1975) · Psychopharmacology · doi:10.1007/bf00429052 · Cited by 322 · see citing works
6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine
Taylor, J. R., & Robbins, T. W. (1986) · Psychopharmacology · doi:10.1007/bf00179197 · Cited by 317 · see citing works
Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals.Preprint
Schoenbaum, G., Setlow, B., Nugent, S. L., Saddoris, M. P., & Gallagher, M. (2003) · Learning & memory (Cold Spring Harbor, N.Y.) · doi:10.1101/lm.55203 · Cited by 317 · see citing works
A Reinforcement Learning Method for Maximizing Undiscounted Rewards
Schwartz, A. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50045-9 · Cited by 315 · see citing works
Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists.
Dickinson, A., Smith, J. W., & Mirenowicz, J. (2000) · Behavioral Neuroscience · doi:10.1037//0735-7044.114.3.468 · Cited by 312 · see citing works
The role of noradrenaline in tuning and dopamine in switching between signals in the CNS
Oades, R. D. (1985) · Neuroscience & Biobehavioral Reviews · doi:10.1016/0149-7634(85)90050-8 · Cited by 306 · see citing works
Weighing up the benefits of work: Behavioral and neural analyses of effort-related decision making
Walton, M. E., Kennerley, S. W., Bannerman, D. M., Phillips, P. E. M., & Rushworth, M. F. S. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.005 · Cited by 305 · see citing works
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298 · see citing works
Dopaminergic correlates of motivated behavior: importance of drive
Wilson, C., Nomikos, G., Collu, M., & Fibiger, H. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-07-05169.1995 · Cited by 294 · see citing works
Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort
Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F. S., & Bannerman, D. M. (2004) · Psychopharmacology · doi:10.1007/s00213-004-2059-4 · Cited by 294 · see citing works
Current Developments in Psychopharmacology
Essman, W. B., & Valzelli, L. (1981) · Springer eBooks · doi:10.1007/978-94-011-8123-5 · Cited by 291 · see citing works
Modulation of saccadic eye movements by predicted reward outcome
Takikawa, Y., Kawagoe, R., Itoh, H., Nakahara, H., & Hikosaka, O. (2002) · Experimental Brain Research · doi:10.1007/s00221-001-0928-1 · Cited by 287 · see citing works
The Hippocampus Modulates Dopamine Neuron Responsivity by Regulating the Intensity of Phasic Neuron Activation
Lodge, D. J., & Grace, A. A. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300963 · Cited by 280 · see citing works
Value-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268 · see citing works
Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement
Aberman, J. E., & Salamone, J. D. (1999) · Neuroscience · doi:10.1016/s0306-4522(99)00004-4 · Cited by 263 · see citing works
The regulation of forebrain dopamine transmission: relevance to the pathophysiology and psychopathology of schizophrenia.
Moore, H., West, A. R., & Grace, A. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(99)00078-5 · Cited by 257 · see citing works
A normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251 · see citing works
Self-citation
Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists.
Dickinson, A., Smith, J. W., & Mirenowicz, J. (2000) · Behavioral Neuroscience · doi:10.1037/0735-7044.114.3.468 · Cited by 249 · see citing works
Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task
Cousins, M. S., Atherton, A., Turner, L., & Salamone, J. D. (1996) · Behavioural Brain Research · doi:10.1016/0166-4328(95)00151-4 · Cited by 236 · see citing works
Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement
Salamone, J. D., Wisniecki, A., Carlson, B. B., & Correa, M. (2001) · Neuroscience · doi:10.1016/s0306-4522(01)00249-4 · Cited by 198 · see citing works
Cellular Models of Reinforcement
Wickens, J. R., & Kötter, R. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0017 · Cited by 166 · see citing works
The Role of Accumbens Dopamine in Lever Pressing and Response Allocation: Effects of 6-OHDA Injected into Core and Dorsomedial Shell
Sokolowski, J. D., & Salamone, J. D. (1998) · Pharmacology Biochemistry and Behavior · doi:10.1016/s0091-3057(97)00544-3 · Cited by 166 · see citing works
Increased response switching, perseveration and perseverative switching following d-amphetamine in the rat
Evenden, J. L., & Robbins, T. W. (1983) · Psychopharmacology · doi:10.1007/bf00427498 · Cited by 151 · see citing works
Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model
Wickens, J. R. (1990) · Journal of Neural Transmission · doi:10.1007/bf01245020 · Cited by 145 · see citing works
Nucleus accumbens dopamine and work requirements on interval schedules
Correa, M., Carlson, B. B., Wisniecki, A., & Salamone, J. D. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00292-9 · Cited by 130 · see citing works
Behavioral reactions reflecting differential reward expectations in monkeys
Watanabe, M., Cromwell, H. C., Tremblay, L., Hollerman, J. R., Hikosaka, K., & Schultz, W. (2001) · Experimental Brain Research · doi:10.1007/s002210100856 · Cited by 116 · see citing works
Normative and Descriptive Models of Decision Making: Time Discounting and Risk Sensitivity
Kacelnik, A. (2007) · Novartis Foundation symposium · doi:10.1002/9780470515372.ch5 · Cited by 115 · see citing works
Parametric analysis of brain stimulation reward in the rat: I. The transient process and the memory-containing process.
Gallistel, C. R., Stellar, J. R., & Bubis, E. A. (1974) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0037220 · Cited by 115 · see citing works
A COMPARISON OF VARIABLE‐RATIO AND VARIABLE‐INTERVAL SCHEDULES OF REINFORCEMENT1
Zuriff, G. E. (1970) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1970.13-369 · Cited by 108 · see citing works
Behavioral study after local injection of 6-hydroxydopamine into the nucleus accumbens in the rat
Taghzouti, K., Simon, H., Louilot, A., Herman, J., & Moal, M. L. (1985) · Brain Research · doi:10.1016/0006-8993(85)91184-9 · Cited by 107 · see citing works
‘Passive stabilization’ of striatal extracellular dopamine across the lesion spectrum encompassing the presymptomatic phase of Parkinson's disease: a voltammetric study in the 6‐OHDA‐lesioned rat
Bergstrom, B. P., & Garris, P. A. (2003) · Journal of Neurochemistry · doi:10.1046/j.1471-4159.2003.02104.x · Cited by 107 · see citing works
Prolonged rewarding stimulation of the rat medial forebrain bundle: Neurochemical and behavioral consequences.
Hernández, G., Hamdani, S., Rajabi, H., Conover, K. L., Stewart, J., Arvanitogiannis, A., & Shizgal, P. (2006) · Behavioral Neuroscience · doi:10.1037/0735-7044.120.4.888 · Cited by 105 · see citing works
How fast to work: Response vigor, motivation and tonic dopamine
Niv, Y., Daw, N. D., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 102 · see citing works
Self-citation
Ratio and time requirements on operant schedules: effort‐related effects of nucleus accumbens dopamine depletions
Mingote, S., Weber, S. M., Ishiwari, K., Correa, M., & Salamone, J. D. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.03972.x · Cited by 101 · see citing works
ECONOMICS, ECOLOGICS, AND MECHANICS: THE DYNAMICS OF RESPONDING UNDER CONDITIONS OF VARYING MOTIVATION
Killeen, P. R. (1995) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1995.64-405 · Cited by 99 · see citing works
Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performancePreprint
Murschall, A., & Hauber, W. (2006) · Learning & Memory · doi:10.1101/lm.127106 · Cited by 97 · see citing works
Adaptive Dynamics
Staddon, J. E. R. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/1092.001.0001 · Cited by 96 · see citing works
STEADY‐STATE PERFORMANCE ON FIXED‐, MIXED‐, AND RANDOM‐RATIO SCHEDULES
Mazur, J. (1983) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1983.39-293 · Cited by 92 · see citing works
Why Choose This Book?: How We Make Decisions
Montague, P. R. (2006) · Medical Entomology and Zoology · Cited by 84 · see citing works
The dynamics of operant conditioning.
Dragoi, V., & Staddon, J. E. R. (1999) · Psychological Review · doi:10.1037/0033-295x.106.1.20 · Cited by 81 · see citing works
Effects of systemic and intracranial amphetamine injections on behavior in the open field: A detailed analysis
Carr, G. D., & White, N. M. (1987) · Pharmacology Biochemistry and Behavior · doi:10.1016/0091-3057(87)90485-0 · Cited by 75 · see citing works
Long-Term Reward Prediction in TD Models of the Dopamine System
Daw, N. D., & Touretzky, D. S. (2002) · Neural Computation · doi:10.1162/089976602760407973 · Cited by 71 · see citing works
Self-citation
Disruptive effects of low doses of d-amphetamine on the ability of rats to organize behaviour into functional sequences
Ljungberg, T., & Enquist, M. (1987) · Psychopharmacology · doi:10.1007/bf00179924 · Cited by 70 · see citing works
Activation of 5-HT 1B receptors in the nucleus accumbens reduces amphetamine-induced enhancement of responding for conditioned reward
Fletcher, P., & Korth, K. M. (1999) · Psychopharmacology · doi:10.1007/s002130050876 · Cited by 68 · see citing works
YOKED VARIABLE‐RATIO AND VARIABLE‐INTERVAL RESPONDING IN PIGEONS1
Catania, A. C., Matthews, T. J., Silverman, P. J., & Yohalem, R. (1977) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1977.28-155 · Cited by 65 · see citing works
Reinforcement learning models of the dopamine system and their behavioral implications
Daw, N. D., & Touretzky, D. S. (2003) · Cited by 60 · see citing works
Self-citation
Evidence that dopamine in the nucleus accumbens is involved in the ability of rats to switch to cue-directed behaviours
van den Bos, R., Ortiz, G. A. C., Bergmans, A. C., & Cools, A. R. (1991) · Behavioural Brain Research · doi:10.1016/s0166-4328(05)80045-2 · Cited by 54 · see citing works
Dopamine in the CNS I
(2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-642-56051-4 · Cited by 39 · see citing works
Dopamine in Schizophrenia Dysfunctional Information Processing in Basal Ganglia — Thalamocortical Split Circuits
Weiner, I., & Joel, D. (2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-06765-9_13 · Cited by 38 · see citing works
Self-citation
Performance on ratio and interval schedules with matched reinforcement rates.
Dawson, G. R., & Dickinson, A. (1990) · PubMed · Cited by 36 · see citing works
EFFECTS OF ETHANOL ON MULTIPLE FIXED‐INTERVAL FIXED‐RATIO SCHEDULE PERFORMANCES: DYNAMIC INTERACTIONS AT DIFFERENT FIXED‐RATIO VALUES
Barrett, J. E., & Stanley, J. A. (1980) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1980.34-185 · Cited by 34 · see citing works
OPEN VERSUS CLOSED ECONOMIES: PERFORMANCE OF DOMESTIC HENS UNDER FIXED‐RATIO SCHEDULES
Foster, T. M., Blackman, K. A., & Temple, W. J. (1997) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1997.67-67 · Cited by 34 · see citing works
Respective contributions of neuronal activity and presynaptic mechanisms in the control of the in vivo release of dopamine
Chéramy, A., Barbeito, L., Godeheu, G., Desce, J. M., Pittaluga, A., Galli, T., Artaud, F., & Glowinski, J. (1990) · Springer eBooks · doi:10.1007/978-3-7091-9050-0_18 · Cited by 30 · see citing works
Extrasynaptic dopamine and phasic neuronal activity
Phillips, P. E. M., & Wightman, R. M. (2004) · Nature Neuroscience · doi:10.1038/nn0304-199a · Cited by 25 · see citing works
Presynaptic Regulation of Dopamine Release Implications for the Functional Organization of the Basal Ganglia
Chesselet, M.-F. (1990) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1990.tb31979.x · Cited by 11 · see citing works
Motivational effects on behavior: Towards a reinforcement learning model of rates of responding
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 3 · see citing works
Self-citation
Presynaptic Regulation of Dopamine Release
Glowinski, J., Chéramy, A., & Kemel, M. L. (2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-06765-9_2 · Cited by 1 · see citing works

Flagged first, then by citations

Sorted by citations

Autoradiographic localization of dopamine D1 and D2 receptors in rat nucleus accumbens: Resistance to differential rearing conditions
Bardo, M. T., & Hammer, R. P. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90226-e · Cited by 3,229 · see citing works
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · see citing works
An opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381 · see citing works
Affective neuroscience of pleasure: reward in humans and animals
Berridge, K. C., & Kringelbach, M. L. (2008) · Psychopharmacology · doi:10.1007/s00213-008-1099-6 · Cited by 1,254 · see citing works
“Liking” and “wanting” food rewards: Brain substrates and roles in eating disorders
Berridge, K. C. (2009) · Physiology & Behavior · doi:10.1016/j.physbeh.2009.02.044 · Cited by 1,147 · see citing works
Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease
Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-Oroz, M. C., Lehericy, S., Bergman, H., Agid, Y., DeLong, M. R., & Obeso, J. A. (2010) · Nature reviews. Neuroscience · doi:10.1038/nrn2915 · Cited by 1,134 · see citing works
Toward a rational and mechanistic account of mental effort
Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T. L., Cohen, J. D., & Botvinick, M. M. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031526 · Cited by 1,077 · see citing works
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955 · see citing works
Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits
Salamone, J. D., Correa, M., Farrar, A., & Mingote, S. M. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0668-9 · Cited by 1,000 · see citing works
Goals and Habits in the Brain
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977 · see citing works
Self-citation
Motivation and cognitive control: from behavior to neural mechanism
Botvinick, M., & Braver, T. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015044 · Cited by 909 · see citing works
What does dopamine mean?
Berke, J. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0152-y · Cited by 904 · see citing works
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 724 · see citing works
Self-citation
From reinforcement learning models to psychiatric and neurological disorders
Maia, T. V., & Frank, M. J. (2011) · Nature Neuroscience · doi:10.1038/nn.2723 · Cited by 715 · see citing works
Dopamine and adaptive memory
Daphna Shohamy, A., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 702 · see citing works
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 690 · see citing works
Self-citation
From prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700 · see citing works
Active Inference, homeostatic regulation and adaptive behavioural control
Pezzulo, G., Rigoli, F., & Friston, K. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.001 · Cited by 650 · see citing works
“In reinforcement learning, phasic and tonic facets of dopaminergic discharges are assumed to encode reward prediction error (Schultz et al., 1997) and average reward expectation, respectively (Niv et al., 2007).”
Modulators of decision making
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642 · see citing works
Neuroscience of apathy and anhedonia: a transdiagnostic approach
Husain, M., & Roiser, J. P. (2018) · Nature reviews. Neuroscience · doi:10.1038/s41583-018-0029-9 · Cited by 642 · see citing works
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634 · see citing works
Cognitive effort: A neuroeconomic approach
Westbrook, A., & Braver, T. S. (2015) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-015-0334-y · Cited by 631 · see citing works
Behavioral functions of the mesolimbic dopaminergic system: An affective neuroethological perspective
Alcaro, A., Huber, R., & Panksepp, J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.07.014 · Cited by 614 · see citing works
Rapid signalling in distinct dopaminergic axons during locomotion and reward
Howe, M. W., & Dombeck, D. A. (2016) · Nature · doi:10.1038/nature18942 · Cited by 597 · see citing works
Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation
Mazzoni, P., Hristova, A., & Krakauer, J. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0264-07.2007 · Cited by 557 · see citing works
Moment-to-moment brain signal variability: A next frontier in human brain mapping?
Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W. S., Lindenberger, U., McIntosh, A. R., & Grady, C. L. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.015 · Cited by 594 · see citing works
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565 · see citing works
What Is the Subjective Cost of Cognitive Effort? Load, Trait, and Aging Effects Revealed by Economic Preference
Westbrook, A., Kester, D., & Braver, T. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068210 · Cited by 537 · see citing works
Separate Valuation Subsystems for Delay and Effort Decision Costs
Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2752-10.2010 · Cited by 499 · see citing works
Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-3-12 · Cited by 480 · see citing works
Self-citation
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464 · see citing works
Self-citation
Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation
Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009) · Nature Neuroscience · doi:10.1038/nn.2342 · Cited by 452 · see citing works
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456 · see citing works
Self-citation
Inflammation Effects on Motivation and Motor Activity: Role of Dopamine
Felger, J. C., & Treadway, M. T. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.143 · Cited by 453 · see citing works
Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour
Rushworth, M. F. S., Behrens, T. E. J., Rudebeck, P. H., & Walton, M. E. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.01.004 · Cited by 507 · see citing works
Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks
Yin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416 · see citing works
The anticipation and outcome phases of reward and loss processing: A neuroimaging meta‐analysis of the monetary incentive delay task
Oldham, S., Murawski, C., Fornito, A., Youssef, G. J., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 425 · see citing works
Go and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 418 · see citing works
Self-citation
Basal ganglia contributions to motor control: a vigorous tutor
Turner, R. S., & Desmurget, M. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.08.022 · Cited by 422 · see citing works
The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward.
Humphries, M. D., & Prescott, T. J. (2009) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2009.11.003 · Cited by 414 · see citing works
Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding
Huys, Q., Cools, R., Gölzer, M., Friedel, E., Heinz, A., Dolan, R., & Dayan, P. (2011) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1002028 · Cited by 470 · see citing works
Self-citation
Inhibitory Input from the Lateral Hypothalamus to the Ventral Tegmental Area Disinhibits Dopamine Neurons and Promotes Behavioral Activation
Nieh, E. H., Vander Weele, C. M., Matthews, G. A., Presbrey, K. N., Wichmann, R., Leppla, C. A., Izadmehr, E. M., & Tye, K. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.035 · Cited by 387 · see citing works
Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.
Collins, A. G. E., & Frank, M. J. (2014) · Psychological Review · doi:10.1037/a0037015 · Cited by 395 · see citing works
The neuromodulator of exploration: A unifying theory of the role of dopamine in personality
DeYoung, C. G. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00762 · Cited by 387 · see citing works
A unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501 · see citing works
Mechanisms of motivation–cognition interaction: challenges and opportunities
Braver, T. S., Krug, M. K., Chiew, K. S., Kool, W., Westbrook, J. A., Clement, N. J., Adcock, R. A., Barch, D. M., Botvinick, M., Carver, C. S., et al. (2014) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-014-0300-0 · Cited by 400 · see citing works
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381 · see citing works
Self-citation
Abnormal temporal difference reward-learning signals in major depression
Kumar, P., Waiter, G., Ahearn, T., Milders, M., Reid, I., & Steele, J. D. (2008) · Brain · doi:10.1093/brain/awn136 · Cited by 374 · see citing works
Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry
Keiflin, R., & Janak, P. H. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.037 · Cited by 370 · see citing works
The anatomy of apathy: a neurocognitive framework for amotivated behaviour
Le Heron, C., Apps., M. A. J., & Husain, M. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.07.003 · Cited by 380 · see citing works
Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340 · see citing works
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357 · see citing works
Self-citation
The nucleus accumbens as part of a basal ganglia action selection circuit
Nicola, S. M. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0510-4 · Cited by 360 · see citing works
The dissociable effects of punishment and reward on motor learning
Galea, J. M., Mallia, E., Rothwell, J. C., & Diedrichsen, J. (2015) · Nature Neuroscience · doi:10.1038/nn.3956 · Cited by 359 · see citing works
Speed/accuracy trade-off between the habitual and the goal-directed processes
Keramati, M., Dezfouli, A., & Piray, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002055 · Cited by 346 · see citing works
Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target
Parker, N. F., Cameron, C. M., Taliaferro, J. P., Lee, J., Choi, J. Y., Davidson, T. J., Daw, N. D., & Witten, I. B. (2016) · Nature Neuroscience · doi:10.1038/nn.4287 · Cited by 340 · see citing works
Self-citation
Prefrontal dopamine and behavioral flexibility: shifting from an “inverted-U” toward a family of functions
Floresco, S. B. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00062 · Cited by 336 · see citing works
Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology
Salamone, J. D., Yohn, S. E., López-Cruz, L., San Miguel, N., & Correa, M. (2016) · Brain · doi:10.1093/brain/aww050 · Cited by 350 · see citing works
The mesoaccumbens dopamine in coping with stress
Cabib, S., & Puglisi-Allegra, S. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.012 · Cited by 329 · see citing works
Personality, Addiction, Dopamine: Insights from Parkinson's Disease
Dagher, A., & Robbins, T. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.031 · Cited by 339 · see citing works
Spontaneous eye blink rate as predictor of dopamine-related cognitive function—A review
Jongkees, B. J., & Colzato, L. S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.08.020 · Cited by 300 · see citing works
Predictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M., Gershman, S. J., & Daw, N. D. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 321 · see citing works
Self-citation
Emotion, decision making, and the amygdala.
Seymour, B., & Dolan, R. (2008) · Neuron · doi:10.1016/j.neuron.2008.05.020 · Cited by 324 · see citing works
Dopamine Is Required for the Neural Representation and Control of Movement Vigor
Panigrahi, B., Martin, K. A., Li, Y., Graves, A. R., Vollmer, A., Olson, L., Mensh, B. D., Karpova, A. Y., & Dudman, J. T. (2015) · Cell · doi:10.1016/j.cell.2015.08.014 · Cited by 315 · see citing works
Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control
Manohar, S., Chong, T. T. ‐., Apps, M. A. J., Batla, A., Stamelou, M., Jarman, P., Bhatia, K. P., & Masud Husain, B. (2015) · Current Biology · doi:10.1016/j.cub.2015.05.038 · Cited by 323 · see citing works
Dopamine Does Double Duty in Motivating Cognitive Effort
Westbrook, A., & Braver, T. S. (2016) · Neuron · doi:10.1016/j.neuron.2015.12.029 · Cited by 313 · see citing works
Process Account of Curiosity and Interest: A Reward-Learning Perspective
Murayama, K., FitzGibbon, L., & Sakaki, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09499-9 · Cited by 306 · see citing works
Dopamine enhances willingness to exert effort for reward in Parkinson's disease
Chong, T. T.-J., Bonnelle, V., Manohar, S., Veromann, K.-R., Muhammed, K., Tofaris, G. K., Hu, M., & Husain, M. (2015) · Cortex · doi:10.1016/j.cortex.2015.04.003 · Cited by 276 · see citing works
A Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299 · see citing works
A Unified Framework for Dopamine Signals across Timescales
Kim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui‐Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe‐Uchida, M., Gershman, S. J., et al. (2020) · Cell · doi:10.1016/j.cell.2020.11.013 · Cited by 297 · see citing works
Can Pornography be Addictive? An fMRI Study of Men Seeking Treatment for Problematic Pornography Use
Gola, M., Wordecha, M., Sescousse, G., Lew-Starowicz, M., Kossowski, B., Wypych, M., Makeig, S., Potenza, M. N., & Marchewka, A. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.78 · Cited by 299 · see citing works
Opponent and bidirectional control of movement velocity in the basal ganglia
Yttri, E. A., & Dudman, J. T. (2016) · Nature · doi:10.1038/nature17639 · Cited by 291 · see citing works
Habits, action sequences and reinforcement learning
Dezfouli, A., & Balleine, B. W. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08050.x · Cited by 297 · see citing works
Mitochondrial function in the brain links anxiety with social subordination
Hollis, F., van der Kooij, M. A., Zanoletti, O., Lozano, L., Cantó, C., & Sandi, C. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1512653112 · Cited by 288 · see citing works
Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective
Story, G. W., Vlaev, I., Seymour, B., Darzi, A. W., & Dolan, R. J. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00076 · Cited by 284 · see citing works
The drift diffusion model as the choice rule in reinforcement learning
Pedersen, M. L., Frank, M. J., & Biele, G. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1199-y · Cited by 272 · see citing works
Dopamine, behavioral economics, and effort
Salamone, J. D. (2009) · Frontiers in Behavioral Neuroscience · doi:10.3389/neuro.08.013.2009 · Cited by 277 · see citing works
The Orbitofrontal Cortex and Ventral Tegmental Area Are Necessary for Learning from Unexpected Outcomes
Takahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., Burke, K. A., & Schoenbaum, G. M. (2009) · Neuron · doi:10.1016/j.neuron.2009.03.005 · Cited by 284 · see citing works
The basal ganglia: from motor commands to the control of vigor
Dudman, J. T., & Krakauer, J. W. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.02.005 · Cited by 282 · see citing works
The Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects
FitzGerald, T. H. B., Seymour, B., & Dolan, R. J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0717-09.2009 · Cited by 279 · see citing works
Action versus valence in decision making
Guitart-Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.01.003 · Cited by 270 · see citing works
Self-citation
Dopamine modulates novelty seeking behavior during decision making.
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2014) · Behavioral Neuroscience · doi:10.1037/a0037128 · Cited by 263 · see citing works
Serotonin, inhibition, and negative mood
Dayan, P., & Huys, Q. J. M. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040004 · Cited by 263 · see citing works
Self-citation
Dopamine neurons encode performance error in singing birds
Gadagkar, V., Puzerey, P. A., Chen, R., Baird-Daniel, E., Farhang, A., & Goldberg, J. H. (2016) · Science · doi:10.1126/science.aah6837 · Cited by 270 · see citing works
NOvelty-related Motivation of Anticipation and exploration by Dopamine (NOMAD): Implications for healthy aging
Düzel, E., Bunzeck, N., Guitart-Masip, M., & Düzel, S. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.08.006 · Cited by 249 · see citing works
A Representation of Effort in Decision-Making and Motor Control
Shadmehr, R., Huang, H. J., & Ahmed, A. A. (2016) · Current Biology · doi:10.1016/j.cub.2016.05.065 · Cited by 267 · see citing works
A role for mesencephalic dopamine in activation: commentary on Berridge (2006)
Robbins, T. W., & Everitt, B. J. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0528-7 · Cited by 265 · see citing works
Thermodynamics as a theory of decision-making with information-processing costs
Ortega, P. A., & Braun, D. A. (2013) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2012.0683 · Cited by 266 · see citing works
Believing in dopamine
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236 · see citing works
Dopaminergic Drugs Modulate Learning Rates and Perseveration in Parkinson's Patients in a Dynamic Foraging Task
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A., & Glimcher, P. W. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.3524-09.2009 · Cited by 255 · see citing works
Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233 · see citing works
Computational models link cellular mechanisms of neuromodulation to large-scale neural dynamics
Shine, J. M., Müller, E. J., Munn, B., Cabral, J., Moran, R., & Breakspear, M. J. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00824-6 · Cited by 236 · see citing works
Activity patterns of serotonin neurons underlying cognitive flexibility
Matias, S., Lottem, E., Dugué, G. P., & Mainen, Z. F. (2017) · eLife · doi:10.7554/elife.20552 · Cited by 239 · see citing works
Depression: a decision-theoretic analysis
Huys, Q. J. M., Daw, N. D., & Dayan, P. (2015) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071714-033928 · Cited by 221 · see citing works
Self-citation
The Flexible Approach Hypothesis: Unification of Effort and Cue-Responding Hypotheses for the Role of Nucleus Accumbens Dopamine in the Activation of Reward-Seeking Behavior
Nicola, S. M. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.3958-10.2010 · Cited by 238 · see citing works
Neurocomputational models of basal ganglia function in learning, memory and choice
Cohen, M. X., & Frank, M. J. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.09.029 · Cited by 238 · see citing works
Learning the opportunity cost of time in a patch-foraging task
Constantino, S. M., & Daw, N. D. (2015) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-015-0350-y · Cited by 238 · see citing works
Self-citation
Positive Affect Versus Reward: Emotional and Motivational Influences on Cognitive Control
Chiew, K. S., & Braver, T. S. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00279 · Cited by 238 · see citing works
NOW vs LATER brain circuits: implications for obesity and addiction
Volkow, N. D., & Baler, R. D. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.04.002 · Cited by 237 · see citing works
Building a neuroscience of pleasure and well-being
Berridge, K. C., & Kringelbach, M. L. (2011) · Psychology of Well-Being: Theory, Research and Practice · doi:10.1186/2211-1522-1-3 · Cited by 229 · see citing works

Authors: Niv, Y., Daw, N. D., Joel, D., & Dayan, P.
Affiliations: Hebrew University of Jerusalem (IL)
Gatsby Computational Neuroscience Unit (GB)
Tel Aviv University (IL)
Corresponding author: Niv, Yael
ORCID: Niv, Yael
Daw, Nathaniel D.
Joel, Daphna
Dayan, Peter
Venue: Psychopharmacology
Publisher: Springer Science and Business Media LLC
Published: 2006-10-10
Volume: 191
Issue: 3
Pages: 507-520
ISSN: 0033-3158 (ISSN-L)
1432-2072
DOI: 10.1007/s00213-006-0502-4
OpenAlex: W2156952192
Semantic Scholar: e1625352c7b6e275ec9a63984a5af25a98032671
Cited by: 1,133
References: 116
Citations by year: 2026: 49 · 2025: 65 · 2024: 75 · 2023: 59 · 2022: 76 · 2021: 80 · 2020: 64 · 2019: 63 · 2018: 69 · 2017: 69 · 2016: 84 · 2015: 52 · 2014: 62 · 2013: 61 · 2012: 44
FWCI: 9.35
Type: journal article
Language: en
Open access: closed
Licence: www.springer.com/tdm (tdm)
Locations: Psychopharmacology (publishedVersion)
PubMed (publishedVersion)
UCL Discovery (University College London) (submittedVersion)
MPG.PuRe (Max Planck Society) (submittedVersion)
Funders: Gatsby Charitable Foundation
Hebrew University of Jerusalem
Topics: Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords: tonic dopamine, response vigor, opportunity cost, optimal control, average-reward reinforcement learning, response latency, nucleus accumbens, striatum, free-operant behavior, reward rate, motivation, phasic dopamine
MeSH terms: Animals, Brain, Choice Behavior, Conditioning, Operant, Dopamine, Models, Psychological, Motivation, Reaction Time, Reinforcement, Psychology, Reinforcement Schedule, Reward, Neurotransmitter Agents, Rats

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Tonic dopamine: Opportunity costs and the control of response vigor

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks