Cited by 1,133Neurotransmitter Receptor Influence on BehaviorNeural and Behavioral Psychology StudiesReinforcement Learning in Robotics

The DOI matches its Crossref record; 116 of the 116 references it cites were found in a registry or an index.

Tonic dopamine: opportunity costs and the control of response vigor

Niv, Yael, Daw, Nathaniel D., Joel, Daphna, Dayan, Peter·2007·Psychopharmacology

DOI 10.1007/s00213-006-0502-4

Open this paper in CoChat — free account

Abstract

Rationale Dopamine neurotransmission has long been known to exert a powerful influence over the vigor, strength, or rate of responding. However, there exists no clear understanding of the computational foundation for this effect; predominant accounts of dopamine's computational function focus on a role for phasic dopamine in controlling the discrete selection between different actions and have nothing to say about response vigor or indeed the free-operant tasks in which it is typically measured.Objectives We seek to accommodate free-operant behavioral tasks within the realm of models of optimal control and thereby capture how dopaminergic and motivational manipulations affect response vigor.Methods We construct an average reward reinforcement learning model in which subjects choose both which action to perform and also the latency with which to perform it. Optimal control balances the costs of acting quickly against the benefits of getting reward earlier and thereby chooses a best response latency.Results In this framework, the long-run average rate of reward plays a key role as an opportunity cost and mediates motivational influences on rates and vigor of responding. We review evidence suggesting that the average reward rate is reported by tonic levels of dopamine putatively in the nucleus accumbens.Conclusions Our extension of reinforcement learning models to free-operant tasks unites psychologically and computationally inspired ideas about the role of tonic dopamine in striatum, explaining from a normative point of view why higher levels of dopamine might be associated with more vigorous responding.

Integrity signals

Registry

Crossref · 10.1007/s00213-006-0502-4

Cites

116 of 116 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citation
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
Neuro-dynamic Programming
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 2×
“One possible explanation for these effects is that phasic signals transiently affect dopamine tone (Phillips and Wightman 2004; Roitman et al. 2004; Wise 2004), influencing vigor selection.”
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708
ON THE LAW OF EFFECT 1
Herrnstein, R. J. (1970) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1970.13-243 · Cited by 2,611
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citation

Flagged first, then by citations

Show all 116 references

Sorted by citations

Items 301–400 of 1,000

Dopaminergic involvement during mental fatigue in health and cocaine addiction
Moeller, S. J., Tomasi, D. G., Honorio, J., Volkow, N. D., & Goldstein, R. Z. (2012) · Translational Psychiatry · doi:10.1038/tp.2012.110 · Cited by 77
Neuroscience of inhibition for addiction medicine
Moeller, S. J., Bederson, L., Alia‐Klein, N., & Goldstein, R. Z. (2015) · Progress in brain research · doi:10.1016/bs.pbr.2015.07.007 · Cited by 77
Phasic Dopamine Transmission Reflects Initiation Vigor and Exerted Effort in an Action- and Region-Specific Manner
Ko, D., & Wanat, M. J. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1279-15.2016 · Cited by 74
From Experienced Utility to Decision Utility
Berridge, K. C., & O’Doherty, J. P. (2013) · Elsevier eBooks · doi:10.1016/b978-0-12-416008-8.00018-8 · Cited by 82
Anesthetics Eliminate Somatosensory-Evoked Discharges of Neurons in the Somatotopically Organized Sensorimotor Striatum of the Rat
West, M. O. (1998) · Journal of Neuroscience · doi:10.1523/jneurosci.18-21-09055.1998 · Cited by 72
Exploring the cognitive and motor functions of the basal ganglia: an integrative review of computational cognitive neuroscience models
Hélie, S., Chakravarthy, V. S., & Moustafa, A. A. (2013) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2013.00174 · Cited by 71
Learning and Choice in Mood Disorders: Searching for the Computational Parameters of Anhedonia
Robinson, O. J., & Chase, H. W. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00009 · Cited by 66
Time perception and impulsivity: A proposed relationship in addictive disorders
Paasche, C., Weibel, S., Wittmann, M., & Lalanne, L. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.12.006 · Cited by 81
Motor Skill in Autism Spectrum Disorders
Chukoskie, L., Townsend, J., & Westerfield, M. (2013) · International review of neurobiology · doi:10.1016/b978-0-12-418700-9.00007-1 · Cited by 74
Control without Controllers: Toward a Distributed Neuroscience of Executive Control
Eisenreich, B. R., Akaishi, R., & Hayden, B. Y. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01139 · Cited by 79
Motivational effects on interval timing in dopamine transporter (DAT) knockdown mice
Balcı, F., Ludvig, E. A., Abner, R. C., Zhuang, X., Poon, P., & Brunner, D. (2010) · Brain Research · doi:10.1016/j.brainres.2010.02.034 · Cited by 68
Addiction as vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. C. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x08004986 · Cited by 69
Serotonin and dopamine differentially affect appetitive and aversive general Pavlovian-to-instrumental transfer
Hebart, M. N., & Gläscher, J. P. (2014) · Psychopharmacology · doi:10.1007/s00213-014-3682-3 · Cited by 66
Evaluation of electrochemical methods for tonic dopamine detection in vivo
Rusheen, A. E., Gee, T. A., Jang, D. P., Blaha, C. D., Bennet, K. E., Lee, K. H., Heien, M. L., & Oh, Y. (2020) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2020.116049 · Cited by 67
Oestradiol influences on dopamine release from the nucleus accumbens shell: sex differences and the role of selective oestradiol receptor subtypes
Yoest, K. E., Cummings, J., & Becker, J. B. (2018) · British Journal of Pharmacology · doi:10.1111/bph.14531 · Cited by 73
Mesolimbic dopamine projections mediate cue-motivated reward seeking but not reward retrieval in rats
Halbout, B., Marshall, A. T., Azimi, A., Liljeholm, M., Mahler, S. V., Wassum, K. M., & Ostlund, S. B. (2019) · eLife · doi:10.7554/elife.43551 · Cited by 69
Nucleus Accumbens Mediates Relative Motivation for Rewards in the Absence of Choice
Clithero, J. A., Reeck, C., Carter, R. M., Smith, D. V., & Huettel, S. A. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00087 · Cited by 69
Differentiable Neural Substrates for Learned and Described Value and Risk
FitzGerald, T. H. B., Seymour, B., Bach, D. R., & Dolan, R. J. (2010) · Current Biology · doi:10.1016/j.cub.2010.08.048 · Cited by 65
Dopamine Increases a Value-Independent Gambling Propensity
Rigoli, F., Rutledge, R. B., Chew, B., Ousdal, O. T., Dayan, P., & Dolan, R. J. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.68 · Cited by 63
Self-citation
Precisely timed dopamine signals establish distinct kinematic representations of skilled movements
Bova, A. S., Gaidica, M., Hurst, A., Iwai, Y., Hunter, J., & Leventhal, D. (2020) · eLife · doi:10.7554/elife.61591 · Cited by 63
Valence-dependent influence of serotonin depletion on model-based choice strategy
Worbe, Y., Palminteri, S., Savulich, G., Daw, N. D., Fernandez-Egea, E., Robbins, T. W., & Voon, V (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.46 · Cited by 75
Self-citation
Dopamine Function and the Efficiency of Human Movement
Gepshtein, S., Li, X., Snider, J., Plank, M., Lee, D., & Howard Poizner, A. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00503 · Cited by 63
It's all about timing: An electrophysiological examination of feedback-based learning with immediate and delayed feedback
Arbel, Y., Hong, L., Baker, T. E., & Holroyd, C. B. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.03.003 · Cited by 69
L-DOPA reduces model-free control of behavior by attenuating the transfer of value to action
Kroemer, N. B., Lee, Y., Pooseh, S., Eppinger, B., Goschke, T., & Smolka, M. N. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.10.075 · Cited by 62
Behavioral specifications of reward-associated long-term memory enhancement in humans
Wittmann, B. C., Dolan, R. J., & Düzel, E. (2011) · Learning & Memory · doi:10.1101/lm.1996811 · Cited by 62
Motivation and movement: the effect of monetary incentive on performance speed
Mir, P., Trender‐Gerhard, I., Edwards, M. J., Schneider, S. A., Bhatia, K. P., & Jahanshahi, M. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2583-5 · Cited by 67
Theoretical and Empirical Studies of Learning
Niv, Y., & Montague, P. R. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00022-1 · Cited by 65
Self-citation
Pattern of dopamine signaling during aversive events predicts active avoidance learning
Stelly, C. E., Haug, G. C., Fonzi, K. M., Garcia, M. A., Tritley, S. C., Magnon, A. P., Ramos, M. A. P., & Wanat, M. J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1904249116 · Cited by 63
Brainstem Mechanisms of Pain Modulation: A within-Subjects 7T fMRI Study of Placebo Analgesic and Nocebo Hyperalgesic Responses
Crawford, L. S., Mills, E. P., Hanson, T., Macey, P. M., Glarin, R., Macefield, V. G., Keay, K. A., & Henderson, L. A. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.0806-21.2021 · Cited by 92
On the Evolution and Optimality of Mood States
Trimmer, P. C., Paul, E. S., Mendl, M., McNamara, J. M., & Houston, A. I. (2013) · Behavioral Sciences · doi:10.3390/bs3030501 · Cited by 61
Pleasurable music affects reinforcement learning according to the listener
Gold, B. P., Frank, M. J., Bogert, B., & Brattico, E. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00541 · Cited by 70
Central Ghrelin Resistance Permits the Overconsolidation of Fear Memory
Harmatz, E. S., Stone, L. A., Lim, S. H., Lee, G. L., McGrath, A. G., Gisabella, B., Peng, X., Kosoy, E., Yao, J., Liu, E. E., et al. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2016.11.009 · Cited by 61
Neurocomputational mechanisms underlying motivated seeing
Leong, Y. C., Hughes, B., Wang, Y., & Zaki, J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0637-z · Cited by 68
Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons
Lee, R. S., Mattar, M. G., Parker, N. F., Witten, I. B., & Daw, N. D. (2019) · eLife · doi:10.7554/elife.42992 · Cited by 59
Self-citation
Integrating Models of Interval Timing and Reinforcement Learning
Petter, E. A., Gershman, S. J., & Meck, W. H. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.08.004 · Cited by 59
Dose-response effects of d-amphetamine on effort-based decision-making and reinforcement learning
Soder, H. E., Cooper, J. A., Lopez-Gamundi, P., Hoots, J. K., Nunez, C., Lawlor, V. M., Lane, S. D., Treadway, M. T., & Wardle, M. C. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-0779-8 · Cited by 61
Functional Involvement of Human Periaqueductal Gray and Other Midbrain Nuclei in Cognitive Control
Kragel, P. A., Bianciardi, M., Hartley, L., Matthewson, G., Choi, J., Quigley, K. S., Wald, L. L., Wager, T. D., Barrett, L. F., & Satpute, A. B. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.2043-18.2019 · Cited by 61
Signaling Incentive and Drive in the Primate Ventral Pallidum for Motivational Control of Goal-Directed Action
Fujimoto, A., Hori, Y., Nagai, Y., Kikuchi, E., Oyama, K., Suhara, T., & Minamimoto, T. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.2399-18.2018 · Cited by 60
Neuroscience: Waiting for Serotonin
Ranade, S., Pi, H.-J., & Kepecs, Á. (2014) · Current Biology · doi:10.1016/j.cub.2014.07.024 · Cited by 59
Anxiety-Like Behavioural Inhibition Is Normative under Environmental Threat-Reward Correlations
Bach, D. R. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004646 · Cited by 62
Safety out of control: dopamine and defence
Lloyd, K., & Dayan, P. (2016) · Behavioral and Brain Functions · doi:10.1186/s12993-016-0099-7 · Cited by 64
Self-citation
Dopamine dynamics are dispensable for movement but promote reward responses
Cai, X., Liu, C., Tsutsui‐Kimura, I., Lee, J.-H., Guo, C., Banerjee, A., Lee, J., Amo, R., Xie, Y., Patriarchi, T., et al. (2024) · Nature · doi:10.1038/s41586-024-08038-z · Cited by 64
Greater striatal responses to medication in Parkinson׳s disease are associated with better task-switching but worse reward performance
Aarts, E., Nusselein, A. A. M., Smittenaar, P., Helmich, R. C. G., Bloem, B. R., & Cools, R. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.05.023 · Cited by 58
(Reinforcement?) Learning to forage optimally
Kolling, N., & Akam, T. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.08.008 · Cited by 61
Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales
Iigaya, K., Ahmadian, Y., Sugrue, L. P., Corrado, G. S., Loewenstein, Y., Newsome, W. T., & Fusi, S. (2019) · Nature Communications · doi:10.1038/s41467-019-09388-3 · Cited by 58
Context-Dependent Multiplexing by Individual VTA Dopamine Neurons
Kremer, Y., Flakowski, J., Rohner, C., & Lüscher, C. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.0502-20.2020 · Cited by 57
Neuroimaging of the dopamine/reward system in adolescent drug use
Ernst, M., & Luciana, M. M. (2015) · CNS Spectrums · doi:10.1017/s1092852915000395 · Cited by 59
Reward Motivation Accelerates the Onset of Neural Novelty Signals in Humans to 85 Milliseconds
Bunzeck, N., Doeller, C. F., Fuentemilla, L., Dolan, R. J., & Düzel, E. (2009) · Current Biology · doi:10.1016/j.cub.2009.06.021 · Cited by 57
The role of dopamine in risk taking: a specific look at Parkinson’s disease and gambling
Clark, C. A., & Dagher, A. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00196 · Cited by 69
A mobile approach-avoidance task
Zech, H., Rotteveel, M., DIJK, W. W. V., & van Dillen, L. F. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01379-3 · Cited by 56
Evidence integration in model-based tree search
Solway, A., & Botvinick, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1505483112 · Cited by 57
Dopaminergic Control of Motivation and Reinforcement Learning: A Closed-Circuit Account for Reward-Oriented Behavior
Morita, K., Morishima, M., Sakai, K., & Kawaguchi, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4614-12.2013 · Cited by 58
Better Get Back to Work: A Role for Motor Beta Desynchronization in Incentive Motivation
Meyniel, F., & Pessiglione, M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.1711-13.2014 · Cited by 47
Pavlovian Control of Escape and Avoidance
Millner, A. J., Gershman, S. J., Nock, M. K., & den Ouden, H. E. M. (2017) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01224 · Cited by 62
Reward Boosts Neural Coding of Task Rules to Optimize Cognitive Flexibility
Hall-McMaster, S., Muhle-Karbe, P. S., Myers, N. E., & Stokes, M. G. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.0631-19.2019 · Cited by 56
Biased belief updating and suboptimal choice in foraging decisions
Garrett, N., & Daw, N. (2019) · bioRxiv · doi:10.1038/s41467-020-16964-5 · Cited by 55
Self-citation
The cost of dopamine for dynamic cognitive control
Cools, R. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.05.007 · Cited by 79
Striatal dopamine synthesis capacity reflects smartphone social activity
Westbrook, A., Ghosh, A., van den Bosch, R., Määttä, J. I., Hofmans, L., & Cools, R. (2021) · iScience · doi:10.1016/j.isci.2021.102497 · Cited by 68
A general theory of intertemporal decision-making and the perception of time
Namboodiri, V. M. K., Mihalaş, Ş., Marton, T. M., & Shuler, M. G. H. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00061 · Cited by 61
At What Stage of Neural Processing Does Cocaine Act to Boost Pursuit of Rewards?
Hernández, G., Breton, Y.-A., Conover, K. L., & Shizgal, P. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015081 · Cited by 56
A Selective Role for Dopamine in Learning to Maximize Reward But Not to Minimize Effort: Evidence from Patients with Parkinson's Disease
Skvortsova, V., Degos, B., Welter, M., Vidailhet, M., & Pessiglione, M. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2081-16.2017 · Cited by 53
Context Processing and Cognitive Control
Chiew, K. S., & Braver, T. S. (2017) · Wiley eBooks · doi:10.1002/9781118920497.ch9 · Cited by 64
Chronic nicotine increases midbrain dopamine neuron activity and biases individual strategies towards reduced exploration in mice
Dongelmans, M., Cuttoli, R. D.-D., Nguyen, C., Côme, M., Duranté, E. K., Lemoine, D., Brito, R., Yahia, T. A., Mondoloni, S., Didienne, S., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-27268-7 · Cited by 55
Dopamine-independent effect of rewards on choices through hidden-state inference
Blanco-Pozo, M., Akam, T., & Walton, M. E. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01542-x · Cited by 69
Pathways to Late-Life Suicidal Behavior
Szántó, K., Galfalvy, H., Vanyukov, P. M., Keilp, J. G., & Dombrovski, A. Y. (2018) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.17m11611 · Cited by 54
Motivation dynamically increases noise resistance by internal feedback during movement
Manohar, S., Muhammed, K., Fallon, S. J., & Husain, M. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.07.011 · Cited by 59
Aversive Motivation and Cognitive ControlPreprint
Yee, D. M., Leng, X., Shenhav, A., & Braver, T. S. (2021) · PsyArXiv (OSF Preprints) · doi:10.1016/j.neubiorev.2021.12.016 · Cited by 59
Pharmacological evidence for the implication of noradrenaline in effort
Borderies, N., Mattioni, J., Bornert, P., Gilardeau, S., & Bouret, S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000793 · Cited by 49
Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults
Webber, H. E., Lopez-Gamundi, P., Stamatovich, S. N., de Wit, H., & Wardle, M. C. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.11.004 · Cited by 51
On the normative advantages of dopamine and striatal opponency for learning and choice
Jaskir, A., & Frank, M. J. (2023) · eLife · doi:10.7554/elife.85107 · Cited by 56
A Proposed Circuit Computation in Basal Ganglia: History‐Dependent Gain
Yttri, E. A., & Dudman, J. T. (2018) · Movement Disorders · doi:10.1002/mds.27321 · Cited by 52
Endogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability
Chew, B., Hauser, T. U., Papoutsi, M., Magerkurth, J., Dolan, R. J., & Rutledge, R. B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900872116 · Cited by 49
Abundance Compensates Kinetics: Similar Effect of Dopamine Signals on D1 and D2 Receptor Populations
Hunger, L., Kumar, A., & Schmidt, R. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.1951-19.2019 · Cited by 54
Lost in time: Relocating the perception of duration outside the brain
Robbe, D. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105312 · Cited by 53
State and rate-of-change encoding in parallel mesoaccumbal dopamine pathways
de Jong, J. W., Liang, Y., Verharen, J. P. H., Fraser, K. M., & Lammel, S. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01547-6 · Cited by 54
Weak reward source memory in depression reflects blunted activation of VTA/SN and parahippocampus
Dillon, D. G., Dobbins, I. G., & Pizzagalli, D. A. (2013) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst155 · Cited by 57
Neural processes mediating contextual influences on human choice behaviour
Rigoli, F., Friston, K. J., & Dolan, R. J. (2016) · Nature Communications · doi:10.1038/ncomms12416 · Cited by 52
Updating the role of dopamine in human motivation and apathy
Chong, T. T. ‐. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.12.010 · Cited by 53
Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making
Frömer, R., & Shenhav, A. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.12.006 · Cited by 57
Neurocomputational mechanisms of reinforcement-guided learning in humans: A review
Cohen, M. X. (2008) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.8.2.113 · Cited by 51
A benefit–cost framework of motivation for a specific activity
Studer, B., & Knecht, S. (2016) · Progress in brain research · doi:10.1016/bs.pbr.2016.06.014 · Cited by 59
Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation
Kato, A., & Morita, K. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005145 · Cited by 59
The missing, the short, and the long: Levodopa responses and dopamine actions
Albin, R. L., & Leventhal, D. (2017) · Annals of Neurology · doi:10.1002/ana.24961 · Cited by 51
Distinct Populations of Motor Thalamic Neurons Encode Action Initiation, Action Selection, and Movement Vigor
Gaidica, M., Hurst, A., Cyr, C., & Leventhal, D. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0463-18.2018 · Cited by 53
Reassessing wanting and liking in the study of mesolimbic influence on food intake
Nicola, S. M. (2016) · American Journal of Physiology-Regulatory, Integrative and Comparative Physiology · doi:10.1152/ajpregu.00234.2016 · Cited by 52
Noradrenergic deficits contribute to apathy in Parkinson’s disease through the precision of expected outcomes
Hezemans, F. H., Wolpe, N., O’Callaghan, C., Ye, R., Rua, C., Jones, P. S., Murley, A. G., Holland, N., Regenthal, R., Tsvetanov, K. A., et al. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010079 · Cited by 51
Striatal dopamine supports reward expectation and learning: A simultaneous PET/fMRI study
Calabro, F. J., Montez, D. F., Larsen, B., Laymon, C. M., Foran, W., Hallquist, M. N., Price, J. C., & Luna, B. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119831 · Cited by 56
Emergent Exploration via Novelty Management
Gordon, G., Fonio, E., & Ahissar, E. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.1872-14.2014 · Cited by 52
Error signals in the subthalamic nucleus are related to post-error slowing in patients with Parkinson's disease
Siegert, S., Ruiz, M. H., Brücke, C., Huebl, J., Schneider, G., Ullsperger, M., & Kühn, A. A. (2014) · Cortex · doi:10.1016/j.cortex.2013.12.008 · Cited by 60
Role of Dopamine Tone in the Pursuit of Brain Stimulation Reward
Hernández, G., Trujillo‐Pisanty, I., Cossette, M., Conover, K. L., & Shizgal, P. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.1051-12.2012 · Cited by 50
Endocannabinoids Promote Cocaine-Induced Impulsivity and Its Rapid Dopaminergic Correlates
Hernández, G., Oleson, E. B., Gentry, R. N., Abbas, Z., Bernstein, D. L., Arvanitogiannis, A., & Cheer, J. F. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.09.005 · Cited by 51
Tamping Ramping: Algorithmic, Implementational, and Computational Explanations of Phasic Dopamine Signals in the Accumbens
Lloyd, K., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004622 · Cited by 51
Self-citation
Limbic-motor integration by neural excitations and inhibitions in the nucleus accumbens
Morrison, S. E., McGinty, V. B., du Hoffmann, J., & Nicola, S. M. (2017) · Journal of Neurophysiology · doi:10.1152/jn.00465.2017 · Cited by 49
Midbrain dopamine neurons signal phasic and ramping reward prediction error during goal-directed navigation
Farrell, K., Lak, A., & Saleem, A. B. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.111470 · Cited by 58
Learning progress mediates the link between cognitive effort and task engagement
Sayalı, C., Heling, E., & Cools, R. (2023) · Cognition · doi:10.1016/j.cognition.2023.105418 · Cited by 57
Striatal dopamine ramping may indicate flexible reinforcement learning with forgetting in the cortico-basal ganglia circuits
Morita, K., & Kato, A. (2014) · Frontiers in Neural Circuits · doi:10.3389/fncir.2014.00036 · Cited by 55
Computational physiology of the neural networks of the primate globus pallidus: function and dysfunction
Goldberg, J. A., & Bergman, H. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.08.068 · Cited by 51
Dopamine Modulates Striato-Frontal Functioning during Temporal Processing
Jones, C. R. G., & Jahanshahi, M. (2011) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2011.00070 · Cited by 48
An amygdala-cingulate network underpins changes in effort-based decision making after a fitness program
Bernácer, J., Martínez-Valbuena, I., Martı́nez, M., Pujol, N., Luís, E. O., Ramírez-Castillo, D., & Pastor, M. A. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116181 · Cited by 49
Dissociable influences of reward and punishment on adaptive cognitive control
Leng, X., Yee, D. M., Ritz, H., & Shenhav, A. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009737 · Cited by 59

Authors: Niv, Y., Daw, N. D., Joel, D., & Dayan, P.
Affiliations: Hebrew University of Jerusalem (IL)
Gatsby Computational Neuroscience Unit (GB)
Tel Aviv University (IL)
Corresponding author: Niv, Yael
ORCID: Niv, Yael
Daw, Nathaniel D.
Joel, Daphna
Dayan, Peter
Venue: Psychopharmacology
Publisher: Springer Science and Business Media LLC
Published: 2006-10-10
Volume: 191
Issue: 3
Pages: 507-520
ISSN: 0033-3158 (ISSN-L)
1432-2072
DOI: 10.1007/s00213-006-0502-4
OpenAlex: W2156952192
Semantic Scholar: e1625352c7b6e275ec9a63984a5af25a98032671
Cited by: 1,133
References: 116
Citations by year: 2026: 49 · 2025: 65 · 2024: 75 · 2023: 59 · 2022: 76 · 2021: 80 · 2020: 64 · 2019: 63 · 2018: 69 · 2017: 69 · 2016: 84 · 2015: 52 · 2014: 62 · 2013: 61 · 2012: 44
FWCI: 9.35
Type: journal article
Language: en
Open access: closed
Licence: www.springer.com/tdm (tdm)
Locations: Psychopharmacology (publishedVersion)
PubMed (publishedVersion)
UCL Discovery (University College London) (submittedVersion)
MPG.PuRe (Max Planck Society) (submittedVersion)
Funders: Gatsby Charitable Foundation
Hebrew University of Jerusalem
Topics: Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords: tonic dopamine, response vigor, opportunity cost, optimal control, average-reward reinforcement learning, response latency, nucleus accumbens, striatum, free-operant behavior, reward rate, motivation, phasic dopamine
MeSH terms: Animals, Brain, Choice Behavior, Conditioning, Operant, Dopamine, Models, Psychological, Motivation, Reaction Time, Reinforcement, Psychology, Reinforcement Schedule, Reward, Neurotransmitter Agents, Rats

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Tonic dopamine: opportunity costs and the control of response vigor

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks