Cited by 1,133Neurotransmitter Receptor Influence on BehaviorNeural and Behavioral Psychology StudiesReinforcement Learning in Robotics

The DOI matches its Crossref record; 116 of the 116 references it cites were found in a registry or an index.

Tonic dopamine: opportunity costs and the control of response vigor

Niv, Yael, Daw, Nathaniel D., Joel, Daphna, Dayan, Peter·2007·Psychopharmacology

DOI 10.1007/s00213-006-0502-4

Open this paper in CoChat — free account

Abstract

Rationale Dopamine neurotransmission has long been known to exert a powerful influence over the vigor, strength, or rate of responding. However, there exists no clear understanding of the computational foundation for this effect; predominant accounts of dopamine's computational function focus on a role for phasic dopamine in controlling the discrete selection between different actions and have nothing to say about response vigor or indeed the free-operant tasks in which it is typically measured.Objectives We seek to accommodate free-operant behavioral tasks within the realm of models of optimal control and thereby capture how dopaminergic and motivational manipulations affect response vigor.Methods We construct an average reward reinforcement learning model in which subjects choose both which action to perform and also the latency with which to perform it. Optimal control balances the costs of acting quickly against the benefits of getting reward earlier and thereby chooses a best response latency.Results In this framework, the long-run average rate of reward plays a key role as an opportunity cost and mediates motivational influences on rates and vigor of responding. We review evidence suggesting that the average reward rate is reported by tonic levels of dopamine putatively in the nucleus accumbens.Conclusions Our extension of reinforcement learning models to free-operant tasks unites psychologically and computationally inspired ideas about the role of tonic dopamine in striatum, explaining from a normative point of view why higher levels of dopamine might be associated with more vigorous responding.

Integrity signals

Registry

Crossref · 10.1007/s00213-006-0502-4

Cites

116 of 116 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citation
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 2×
“One possible explanation for these effects is that phasic signals transiently affect dopamine tone (Phillips and Wightman 2004; Roitman et al. 2004; Wise 2004), influencing vigor selection.”
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708
ON THE LAW OF EFFECT 1
Herrnstein, R. J. (1970) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1970.13-243 · Cited by 2,611
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citation
Scalar expectancy theory and Weber's law in animal timing.
Gibbon, J. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.279 · Cited by 2,310
RELATIVE AND ABSOLUTE STRENGTH OF RESPONSE AS A FUNCTION OF FREQUENCY OF REINFORCEMENT 1, 2
Herrnstein, R. J. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-267 · Cited by 2,281
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Self-citation
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Self-citation
Integrative activity of the brain
Konorski, J. (1967) · University of Chicago Press eBooks · Cited by 2,043
Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia
Grace, A. A. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90196-u · Cited by 1,831
Learning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717
Stevens' Handbook of Experimental Psychology
Pashler, H. (2002) · Wiley eBooks · doi:10.1002/0471214426 · Cited by 1,713
An opponent-process theory of motivation. I. Temporal dynamics of affect
Solomon, R. L., & Corbit, J. D. (1974) · Psychological Review · doi:10.1037/h0036128 · Cited by 1,704
Neuro-Dynamic Programming
Bertsekas, D. P. (2001) · Springer eBooks · doi:10.1007/0-306-48332-7_333 · Cited by 1,519
The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450
Motivation concepts in behavioral neuroscience
Berridge, K. C. (2004) · Physiology & Behavior · doi:10.1016/j.physbeh.2004.02.004 · Cited by 1,302
Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · cited 1×
“Larger phasic prediction-error signals for stimuli previously associated with higher rewards (Fiorillo et al. 2003; Tobler et al. 2005) would then result in faster responding to these cues.”
Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
The basal ganglia: a vertebrate solution to the selection problem?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00319-4 · Cited by 1,199
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Time, rate, and conditioning.
Gallistel, C. R., & Gibbon, J. (2000) · Psychological Review · doi:10.1037/0033-295x.107.2.289 · Cited by 1,134
Subsecond dopamine release promotes cocaine seeking
Phillips, P. E. M., Stuber, G. D., Heien, M. L. A. V., Wightman, R. M., & Carelli, R. M. (2003) · Nature · doi:10.1038/nature01476 · Cited by 1,113
Mesocorticolimbic dopaminergic network: functional and regulatory roles
Moal, M. L., & Simon, H. (1991) · Physiological Reviews · doi:10.1152/physrev.1991.71.1.155 · Cited by 1,108
Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission
Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003) · Nature Neuroscience · doi:10.1038/nn1103 · Cited by 1,106
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Motivational control of goal-directed action
Dickinson, A., & Balleine, B. (1994) · Animal Learning & Behavior · doi:10.3758/bf03199951 · Cited by 995
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
The principles of learning and behavior
Domjan, M. P. (1982) · Cited by 916
The role of dopamine in locomotor activity and learning
Beninger, R. J. (1983) · Brain Research Reviews · doi:10.1016/0165-0173(83)90038-3 · Cited by 848
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Self-citation
A QUANTITATIVE ANALYSIS OF THE RESPONDING MAINTAINED BY INTERVAL SCHEDULES OF REINFORCEMENT1
Catania, A. C., & Reynolds, G. S. (1968) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1968.11-s327 · Cited by 817
Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine
Salamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817
Dopamine operates as a subsecond modulator of food seeking
Roitman, M. F., Stuber, G. D., Phillips, P. E. M., Wightman, R. M., & Carelli, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3823-03.2004 · Cited by 718
Theory of Motivation
Bolles, R. C. (1967) · Medical Entomology and Zoology · Cited by 610
Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior
Goto, Y., & Grace, A. A. (2005) · Nature Neuroscience · doi:10.1038/nn1471 · Cited by 576
A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0020 · Cited by 554
Integrative Activity of the Brain: An Interdisciplinary Approach
Ochs, S. (1968) · JAMA · doi:10.1001/jama.1968.03140050055025 · Cited by 524
Learning and Computational Neuroscience: Foundations of Adaptive Networks
Gabriel, M., & Moore, J. (1990) · Cited by 518
Time-Derivative Models of Pavlovian Reinforcement
Sutton, R. S., & Barto, A. G. (1990) · Cited by 501
Adaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0018 · Cited by 476
An Opponent-Process Theory of Motivation
Solomon, R. L., & Corbit, J. D. (2008) · Cited by 461
Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens
Taylor, J. R., & Robbins, T. W. (1984) · Psychopharmacology · doi:10.1007/bf00555222 · Cited by 440
Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results
Mahadevan, S. (1996) · Machine Learning · doi:10.1007/bf00114727 · Cited by 419
A computational substrate for incentive salience
McClure, S. M., Daw, N. D., & Montague, P. R. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00177-2 · Cited by 416
Self-citation
A neural correlate of response bias in monkey caudate nucleus
Lauwereyns, J., Watanabe, K., Coe, B. C., & Hikosaka, O. (2002) · Nature · doi:10.1038/nature00892 · Cited by 415
Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation
Faure, A., Haberland, U., Condé, F., & Massioui, N. E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3894-04.2005 · Cited by 402
The Role of Learning in the Operation of Motivational Systems
Dickinson, A., & Balleine, B. W. (2002) · Wiley eBooks · doi:10.1002/0471214426.pas0312 · Cited by 387
Neural substrates of latent inhibition: The switching model.
Weiner, I. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.442 · Cited by 380
The laterodorsal tegmentum is essential for burst firing of ventral tegmental area dopamine neurons
Lodge, D. J., & Grace, A. A. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0510715103 · Cited by 349
Functional Studies of the Central Catecholamines
Robbins, T. W., & Everitt, B. J. (1982) · International review of neurobiology · doi:10.1016/s0074-7742(08)60628-5 · Cited by 339
Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists.
Dickinson, A., Smith, J. W., & Mirenowicz, J. (2000) · Behavioral Neuroscience · doi:10.1037//0735-7044.114.3.468 · Cited by 322
A functional effect of dopamine in the nucleus accumbens and in some other dopamine-rich parts of the rat brain
Jackson, D. M., And�n, N.-E., & Dahlstr�m, A. (1975) · Psychopharmacology · doi:10.1007/bf00429052 · Cited by 322
6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine
Taylor, J. R., & Robbins, T. W. (1986) · Psychopharmacology · doi:10.1007/bf00179197 · Cited by 317
Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals.Preprint
Schoenbaum, G., Setlow, B., Nugent, S. L., Saddoris, M. P., & Gallagher, M. (2003) · Learning & memory (Cold Spring Harbor, N.Y.) · doi:10.1101/lm.55203 · Cited by 317
A Reinforcement Learning Method for Maximizing Undiscounted Rewards
Schwartz, A. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50045-9 · Cited by 310
Weighing up the benefits of work: Behavioral and neural analyses of effort-related decision making
Walton, M. E., Kennerley, S. W., Bannerman, D. M., Phillips, P. E. M., & Rushworth, M. F. S. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.005 · Cited by 305
The role of noradrenaline in tuning and dopamine in switching between signals in the CNS
Oades, R. D. (1985) · Neuroscience & Biobehavioral Reviews · doi:10.1016/0149-7634(85)90050-8 · Cited by 304
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298
Dopaminergic correlates of motivated behavior: importance of drive
Wilson, C., Nomikos, G., Collu, M., & Fibiger, H. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-07-05169.1995 · Cited by 294
Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort
Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F. S., & Bannerman, D. M. (2004) · Psychopharmacology · doi:10.1007/s00213-004-2059-4 · Cited by 294
Current Developments in Psychopharmacology
Essman, W. B., & Valzelli, L. (1981) · Springer eBooks · doi:10.1007/978-94-011-8123-5 · Cited by 290
Modulation of saccadic eye movements by predicted reward outcome
Takikawa, Y., Kawagoe, R., Itoh, H., Nakahara, H., & Hikosaka, O. (2002) · Experimental Brain Research · doi:10.1007/s00221-001-0928-1 · Cited by 287
The Hippocampus Modulates Dopamine Neuron Responsivity by Regulating the Intensity of Phasic Neuron Activation
Lodge, D. J., & Grace, A. A. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300963 · Cited by 280
Value-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268
Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement
Aberman, J. E., & Salamone, J. D. (1999) · Neuroscience · doi:10.1016/s0306-4522(99)00004-4 · Cited by 263
The regulation of forebrain dopamine transmission: relevance to the pathophysiology and psychopathology of schizophrenia.
Moore, H., West, A. R., & Grace, A. (1999) · Biological Psychiatry · doi:10.1016/s0006-3223(99)00078-5 · Cited by 257
A normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251
Self-citation
Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task
Cousins, M. S., Atherton, A., Turner, L., & Salamone, J. D. (1996) · Behavioural Brain Research · doi:10.1016/0166-4328(95)00151-4 · Cited by 236
Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement
Salamone, J. D., Wisniecki, A., Carlson, B. B., & Correa, M. (2001) · Neuroscience · doi:10.1016/s0306-4522(01)00249-4 · Cited by 198
Cellular Models of Reinforcement
Wickens, J. R., & Kötter, R. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0017 · Cited by 168
The Role of Accumbens Dopamine in Lever Pressing and Response Allocation: Effects of 6-OHDA Injected into Core and Dorsomedial Shell
Sokolowski, J. D., & Salamone, J. D. (1998) · Pharmacology Biochemistry and Behavior · doi:10.1016/s0091-3057(97)00544-3 · Cited by 166
Increased response switching, perseveration and perseverative switching following d-amphetamine in the rat
Evenden, J. L., & Robbins, T. W. (1983) · Psychopharmacology · doi:10.1007/bf00427498 · Cited by 151
Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model
Wickens, J. R. (1990) · Journal of Neural Transmission · doi:10.1007/bf01245020 · Cited by 145
Nucleus accumbens dopamine and work requirements on interval schedules
Correa, M., Carlson, B. B., Wisniecki, A., & Salamone, J. D. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00292-9 · Cited by 130
Behavioral reactions reflecting differential reward expectations in monkeys
Watanabe, M., Cromwell, H. C., Tremblay, L., Hollerman, J. R., Hikosaka, K., & Schultz, W. (2001) · Experimental Brain Research · doi:10.1007/s002210100856 · Cited by 116
Normative and Descriptive Models of Decision Making: Time Discounting and Risk Sensitivity
Kacelnik, A. (2007) · Novartis Foundation symposium · doi:10.1002/9780470515372.ch5 · Cited by 115
Parametric analysis of brain stimulation reward in the rat: I. The transient process and the memory-containing process.
Gallistel, C. R., Stellar, J. R., & Bubis, E. A. (1974) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0037220 · Cited by 115
Behavioral study after local injection of 6-hydroxydopamine into the nucleus accumbens in the rat
Taghzouti, K., Simon, H., Louilot, A., Herman, J., & Moal, M. L. (1985) · Brain Research · doi:10.1016/0006-8993(85)91184-9 · Cited by 107
Prolonged rewarding stimulation of the rat medial forebrain bundle: Neurochemical and behavioral consequences.
Hernández, G., Hamdani, S., Rajabi, H., Conover, K. L., Stewart, J., Arvanitogiannis, A., & Shizgal, P. (2006) · Behavioral Neuroscience · doi:10.1037/0735-7044.120.4.888 · Cited by 107
‘Passive stabilization’ of striatal extracellular dopamine across the lesion spectrum encompassing the presymptomatic phase of Parkinson's disease: a voltammetric study in the 6‐OHDA‐lesioned rat
Bergstrom, B. P., & Garris, P. A. (2003) · Journal of Neurochemistry · doi:10.1046/j.1471-4159.2003.02104.x · Cited by 107
A COMPARISON OF VARIABLE‐RATIO AND VARIABLE‐INTERVAL SCHEDULES OF REINFORCEMENT1
Zuriff, G. E. (1970) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1970.13-369 · Cited by 106
How fast to work: Response vigor, motivation and tonic dopamine
Niv, Y., Daw, N. D., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 103
Self-citation
Ratio and time requirements on operant schedules: effort‐related effects of nucleus accumbens dopamine depletions
Mingote, S., Weber, S. M., Ishiwari, K., Correa, M., & Salamone, J. D. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.03972.x · Cited by 101
ECONOMICS, ECOLOGICS, AND MECHANICS: THE DYNAMICS OF RESPONDING UNDER CONDITIONS OF VARYING MOTIVATION
Killeen, P. R. (1995) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1995.64-405 · Cited by 99
Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performancePreprint
Murschall, A., & Hauber, W. (2006) · Learning & Memory · doi:10.1101/lm.127106 · Cited by 97
Adaptive Dynamics
Staddon, J. E. R. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/1092.001.0001 · Cited by 96
STEADY‐STATE PERFORMANCE ON FIXED‐, MIXED‐, AND RANDOM‐RATIO SCHEDULES
Mazur, J. (1983) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1983.39-293 · Cited by 95
Why Choose This Book?: How We Make Decisions
Montague, P. R. (2006) · Medical Entomology and Zoology · Cited by 84
The dynamics of operant conditioning.
Dragoi, V., & Staddon, J. E. R. (1999) · Psychological Review · doi:10.1037/0033-295x.106.1.20 · Cited by 83
Effects of systemic and intracranial amphetamine injections on behavior in the open field: A detailed analysis
Carr, G. D., & White, N. M. (1987) · Pharmacology Biochemistry and Behavior · doi:10.1016/0091-3057(87)90485-0 · Cited by 75
Long-Term Reward Prediction in TD Models of the Dopamine System
Daw, N. D., & Touretzky, D. S. (2002) · Neural Computation · doi:10.1162/089976602760407973 · Cited by 71
Self-citation
Disruptive effects of low doses of d-amphetamine on the ability of rats to organize behaviour into functional sequences
Ljungberg, T., & Enquist, M. (1987) · Psychopharmacology · doi:10.1007/bf00179924 · Cited by 70
Activation of 5-HT 1B receptors in the nucleus accumbens reduces amphetamine-induced enhancement of responding for conditioned reward
Fletcher, P. J., & Korth, K. M. (1999) · Psychopharmacology · doi:10.1007/s002130050876 · Cited by 67
YOKED VARIABLE‐RATIO AND VARIABLE‐INTERVAL RESPONDING IN PIGEONS1
Catania, A. C., Matthews, T. J., Silverman, P. J., & Yohalem, R. (1977) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1977.28-155 · Cited by 66
Reinforcement learning models of the dopamine system and their behavioral implications
Daw, N. D., & Touretzky, D. S. (2003) · Cited by 62
Self-citation
Evidence that dopamine in the nucleus accumbens is involved in the ability of rats to switch to cue-directed behaviours
van den Bos, R., Ortiz, G. A. C., Bergmans, A. C., & Cools, A. R. (1991) · Behavioural Brain Research · doi:10.1016/s0166-4328(05)80045-2 · Cited by 54
Dopamine in the CNS I
(2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-642-56051-4 · Cited by 39
Dopamine in Schizophrenia Dysfunctional Information Processing in Basal Ganglia — Thalamocortical Split Circuits
Weiner, I., & Joel, D. (2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-06765-9_13 · Cited by 37
Self-citation
Performance on ratio and interval schedules with matched reinforcement rates.
Dawson, G. R., & Dickinson, A. (1990) · PubMed · Cited by 36
EFFECTS OF ETHANOL ON MULTIPLE FIXED‐INTERVAL FIXED‐RATIO SCHEDULE PERFORMANCES: DYNAMIC INTERACTIONS AT DIFFERENT FIXED‐RATIO VALUES
Barrett, J. E., & Stanley, J. A. (1980) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1980.34-185 · Cited by 34
OPEN VERSUS CLOSED ECONOMIES: PERFORMANCE OF DOMESTIC HENS UNDER FIXED‐RATIO SCHEDULES
Foster, T. M., Blackman, K. A., & Temple, W. J. (1997) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1997.67-67 · Cited by 34
Respective contributions of neuronal activity and presynaptic mechanisms in the control of the in vivo release of dopamine
Chéramy, A., Barbeito, L., Godeheu, G., Desce, J. M., Pittaluga, A., Galli, T., Artaud, F., & Glowinski, J. (1990) · Springer eBooks · doi:10.1007/978-3-7091-9050-0_18 · Cited by 30
Extrasynaptic dopamine and phasic neuronal activity
Phillips, P. E. M., & Wightman, R. M. (2004) · Nature Neuroscience · doi:10.1038/nn0304-199a · Cited by 25
Presynaptic Regulation of Dopamine Release Implications for the Functional Organization of the Basal Ganglia
Chesselet, M.-F. (1990) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1990.tb31979.x · Cited by 11
Motivational effects on behavior: Towards a reinforcement learning model of rates of responding
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 3
Self-citation
Presynaptic Regulation of Dopamine Release
Glowinski, J., Chéramy, A., & Kemel, M. L. (2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-06765-9_2 · Cited by 1

Flagged first, then by citations

Sorted by citations

Autoradiographic localization of dopamine D1 and D2 receptors in rat nucleus accumbens: Resistance to differential rearing conditions
Bardo, M. T., & Hammer, R. P. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90226-e · Cited by 3,229
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
An opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381
Affective neuroscience of pleasure: reward in humans and animals
Berridge, K. C., & Kringelbach, M. L. (2008) · Psychopharmacology · doi:10.1007/s00213-008-1099-6 · Cited by 1,254
“Liking” and “wanting” food rewards: Brain substrates and roles in eating disorders
Berridge, K. C. (2009) · Physiology & Behavior · doi:10.1016/j.physbeh.2009.02.044 · Cited by 1,147

See all 1,133

Authors: Niv, Y., Daw, N. D., Joel, D., & Dayan, P.
Affiliations: Hebrew University of Jerusalem (IL)
Gatsby Computational Neuroscience Unit (GB)
Tel Aviv University (IL)
Corresponding author: Niv, Yael
ORCID: Niv, Yael
Daw, Nathaniel D.
Joel, Daphna
Dayan, Peter
Venue: Psychopharmacology
Publisher: Springer Science and Business Media LLC
Published: 2006-10-10
Volume: 191
Issue: 3
Pages: 507-520
ISSN: 0033-3158 (ISSN-L)
1432-2072
DOI: 10.1007/s00213-006-0502-4
OpenAlex: W2156952192
Semantic Scholar: e1625352c7b6e275ec9a63984a5af25a98032671
Cited by: 1,133
References: 116
Citations by year: 2026: 49 · 2025: 65 · 2024: 75 · 2023: 59 · 2022: 76 · 2021: 80 · 2020: 64 · 2019: 63 · 2018: 69 · 2017: 69 · 2016: 84 · 2015: 52 · 2014: 62 · 2013: 61 · 2012: 44
FWCI: 9.35
Type: journal article
Language: en
Open access: closed
Licence: www.springer.com/tdm (tdm)
Locations: Psychopharmacology (publishedVersion)
PubMed (publishedVersion)
UCL Discovery (University College London) (submittedVersion)
MPG.PuRe (Max Planck Society) (submittedVersion)
Funders: Gatsby Charitable Foundation
Hebrew University of Jerusalem
Topics: Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords: tonic dopamine, response vigor, opportunity cost, optimal control, average-reward reinforcement learning, response latency, nucleus accumbens, striatum, free-operant behavior, reward rate, motivation, phasic dopamine
MeSH terms: Animals, Brain, Choice Behavior, Conditioning, Operant, Dopamine, Models, Psychological, Motivation, Reaction Time, Reinforcement, Psychology, Reinforcement Schedule, Reward, Neurotransmitter Agents, Rats

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Tonic dopamine: opportunity costs and the control of response vigor

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks