A normative perspective on motivation
Abstract
Understanding the effects of motivation on instrumental action selection, and specifically on its two main forms, goal-directed and habitual control, is fundamental to the study of decision making. Motivational states have been shown to 'direct' goal-directed behavior rather straightforwardly towards more valuable outcomes. However, how motivational states can influence outcome-insensitive habitual behavior is more mysterious. We adopt a normative perspective, assuming that animals seek to maximize the utilities they achieve, and viewing motivation as a mapping from outcomes to utilities. We suggest that habitual action selection can direct responding properly only in motivational states which pertained during behavioral training. However, in novel states, we propose that outcome-independent, global effects of the utilities can 'energize' habitual actions.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Abnormal cerebellum and basal ganglia related to tremor and akinetic-rigid symptoms lead to longer RT or MT in PD patients.
“The abnormal cerebellum and basal ganglia that related to tremor [14, 22, 43, 44] and akinetic-rigid symptoms [6–9, 42] also lead to the longer RT or MT in PD patients.”
From Evaluation of tremor interference with control of voluntary reaching movements in patients with Parkinson's disease (2019) · cites this paper as [42]
Prolonged RT and MT in PD patients with bradykinesia and rigidity have been attributed to rigidity due to abnormal basal ganglia system.
“These studies attributed the prolonged RT and MT in this kind of PD patients to the rigidity due to abnormal basal ganglia system [2, 6–9, 42].”
From Evaluation of tremor interference with control of voluntary reaching movements in patients with Parkinson's disease (2019) · cites this paper as [42]
Lack of dopamine results in slower movements that have characteristics similar to the minimum-jerk trajectory.
“Lack of dopamine has been shown to result in slower movements that had the similar characteristics of the MJT [2, 8, 41, 42].”
From Evaluation of tremor interference with control of voluntary reaching movements in patients with Parkinson's disease (2019) · cites this paper as [42]
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationSkinner, B. F., & Hull, C. L. (1944) · The American Journal of Psychology · doi:10.2307/1416955 · Cited by 4,061 · see citing works
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,238 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Self-citationThe Psychology of Animal Learning
Hill, W. F., & Mackintosh, N. J. (1976) · The American Journal of Psychology · doi:10.2307/1421765 · Cited by 2,397 · see citing works
Barto, A. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,172 · see citing works
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129 · see citing works
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citationZhang, B., Li, C., & Lin, N. (2020) · De Gruyter eBooks · doi:10.1515/9783110595567-011 · Cited by 1,939 · see citing works
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · see citing works
Motivation concepts in behavioral neuroscience
Berridge, K. C. (2004) · Physiology & Behavior · doi:10.1016/j.physbeh.2004.02.004 · Cited by 1,302 · see citing works
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · see citing works
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217 · see citing works
Motivational control of goal-directed action
Dickinson, A., & Balleine, B. (1994) · Animal Learning & Behavior · doi:10.3758/bf03199951 · Cited by 995 · see citing works
Reward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855 · see citing works
Self-citationSalamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817 · see citing works
Handbook of Contemporary Learning Theories
Mowrer, R. R. (2000) · Psychology Press eBooks · doi:10.4324/9781410600691 · Cited by 789 · see citing works
Wyvell, C. L., & Berridge, K. C. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-21-08122.2000 · Cited by 764 · see citing works
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748 · see citing works
Instrumental Responding following Reinforcer Devaluation
Adams, C. D., & Dickinson, A. (1981) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748108400816 · Cited by 687 · see citing works
Variations in the Sensitivity of Instrumental Responding to Reinforcer Devaluation
Adams, C. D. (1982) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748208400878 · Cited by 635 · see citing works
Bolles, R. C. (1967) · Medical Entomology and Zoology · Cited by 610 · see citing works
Corbit, L. H., & Balleine, B. W. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.4507-04.2005 · Cited by 589 · see citing works
A motivational view of learning, performance, and behavior modification.
Bindra, D. (1974) · Psychology Review · doi:10.1037/h0036330 · Cited by 580 · see citing works
Corbit, L. H., Muir, J. L., & Balleine, B. W. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-09-03251.2001 · Cited by 564 · see citing works
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04219.x · Cited by 447 · see citing works
Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation
Faure, A., Haberland, U., Condé, F., & Massioui, N. E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3894-04.2005 · Cited by 402 · see citing works
Relations between Pavlovian-instrumental transfer and reinforcer devaluation
Holland, P. C. (2004) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/0097-7403.30.2.104 · Cited by 388 · see citing works
The Role of Learning in the Operation of Motivational Systems
Dickinson, A., & Balleine, B. W. (2002) · Wiley eBooks · doi:10.1002/0471214426.pas0312 · Cited by 387 · see citing works
Coutureau, E., & Killcross, S. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.025 · Cited by 342 · see citing works
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326 · see citing works
Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists.
Dickinson, A., Smith, J. W., & Mirenowicz, J. (2000) · Behavioral Neuroscience · doi:10.1037//0735-7044.114.3.468 · Cited by 312 · see citing works
Brown, J. S. (1961) · McGraw-Hill Book Company · doi:10.1037/12219-000 · Cited by 304 · see citing works
Motivational control after extended instrumental training
Dickinson, A., Balleine, B. W., Watt, A. A., González, F., & Boakes, R. A. (1995) · Animal Learning & Behavior · doi:10.3758/bf03199935 · Cited by 286 · see citing works
Holland, P. C., & Gallagher, M. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.02585.x · Cited by 236 · see citing works
Balleine, B. W., & Dickinson, A. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-23-08954.2000 · Cited by 218 · see citing works
Instrumental performance following a shift in primary motivation depends on incentive learning.
Balleine, B. W. (1992) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.18.3.236 · Cited by 181 · see citing works
Instrumental responding remains sensitive to reinforcer devaluation after extensive training.
Colwill, R. M., & Rescorla, R. A. (1985) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.11.4.520 · Cited by 124 · see citing works
Motivational control of heterogeneous instrumental chains.
Balleine, B. W., Garner, C., González, F., & Dickinson, A. (1995) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.21.3.203 · Cited by 112 · see citing works
How fast to work: Response vigor, motivation and tonic dopamine
Niv, Y., Daw, N. D., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 102 · see citing works
Self-citationIncentive Processes in Instrumental Conditioning
Balleine, B. W. (2000) · Psychology Press eBooks · doi:10.4324/9781410600691-14 · Cited by 99 · see citing works
Learning, motivation, and emotion
Stevens, S. S., & Gallistel, C. R. (2002) · Medical Entomology and Zoology · Cited by 71 · see citing works
The role of response-reinforcer associations increases throughout extended instrumental training
Colwill, R. M., & Rescorla, R. A. (1988) · Animal Learning & Behavior · doi:10.3758/bf03209051 · Cited by 68 · see citing works
The nature and functioning of wants.
Tolman, E. C. (1949) · Psychological Review · doi:10.1037/h0063540 · Cited by 50 · see citing works
Incentive learning and the motivational control of instrumental performance
Dickinson, A., & Dawson, G. R. (2018) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640748908401185 · Cited by 48 · see citing works
Post-Conditioning Devaluation of an Instrumental Reinforcer has no Effect on Extinction Performance
Adams, C. D. (1980) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640748008401838 · Cited by 47 · see citing works
Weiner, I., & Joel, D. (2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-06765-9_13 · Cited by 38 · see citing works
Self-citationIncentive learning and the motivational control of instrumental performance by thirst
López, M., Balleine, B. W., & Dickinson, A. (1992) · Animal Learning & Behavior · doi:10.3758/bf03197955 · Cited by 27 · see citing works
Irrelevant Incentive Learning during Training on Ratio and Interval Schedules
Dickinson, A., & Nicholas, D. (1983) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748308400908 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · see citing works
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779 · see citing works
Self-citationHabits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · see citing works
An opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381 · see citing works
“Liking” and “wanting” food rewards: Brain substrates and roles in eating disorders
Berridge, K. C. (2009) · Physiology & Behavior · doi:10.1016/j.physbeh.2009.02.044 · Cited by 1,147 · see citing works
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · see citing works
Self-citationReinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 724 · see citing works
Self-citationReinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 690 · see citing works
Self-citationFrom prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700 · see citing works
Motivation of extended behaviors by anterior cingulate cortex
Holroyd, C. B., & Yeung, N. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.008 · Cited by 618 · see citing works
Cognitive effort: A neuroeconomic approach
Westbrook, A., & Braver, T. S. (2015) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-015-0334-y · Cited by 631 · see citing works
Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation
Mazzoni, P., Hristova, A., & Krakauer, J. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0264-07.2007 · Cited by 557 · see citing works
A unified framework for addiction: Vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0800472x · Cited by 501 · see citing works
Redish, A. D. (2016) · Nature reviews. Neuroscience · doi:10.1038/nrn.2015.30 · Cited by 463 · see citing works
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456 · see citing works
Self-citationMood as representation of momentum
Eldar, E., Rutledge, R. B., Dolan, R. J., & Niv, Y. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.07.010 · Cited by 383 · see citing works
Self-citationSerotonergic neurons signal reward and punishment on multiple timescales
Cohen, J. Y., Amoroso, M. W., & Uchida, N. (2015) · eLife · doi:10.7554/elife.06346 · Cited by 388 · see citing works
The integrative function of the basal ganglia in instrumental conditioning
Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.10.034 · Cited by 356 · see citing works
A Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299 · see citing works
Human Pavlovian–Instrumental Transfer
Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.4028-07.2008 · Cited by 302 · see citing works
Self-citationCovert expectation-of-reward in rat ventral striatum at decision points
van der Meer, M. A. A. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.001.2009 · Cited by 296 · see citing works
Contextual modulation of value signals in reward and punishment learning
Palminteri, S., Khamassi, M., Joffily, M., & Coricelli, G. (2015) · Nature Communications · doi:10.1038/ncomms9096 · Cited by 273 · see citing works
Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts
Jimura, K., Locke, H., & Braver, T. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1002007107 · Cited by 278 · see citing works
Düzel, E., Bunzeck, N., Guitart-Masip, M., & Düzel, S. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.08.006 · Cited by 249 · see citing works
Hu, H. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-070815-014106 · Cited by 253 · see citing works
Genes, hormones, and circuits: An integrative approach to study the evolution of social behavior
O’Connell, L. A., & Hofmann, H. A. (2010) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2010.12.004 · Cited by 227 · see citing works
Ackermann, H., Hage, S. R., & Ziegler, W. (2014) · Behavioral and Brain Sciences · doi:10.1017/s0140525x13003099 · Cited by 210 · see citing works
Corticostriatal Interactions during Learning, Memory Processing, and Decision Making
Pennartz, C. M. A., Berke, J. D., Graybiel, A. M., Ito, R., Lansink, C. S., van der Meer, M., Redish, A. D., Smith, K. S., & Voorn, P. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3177-09.2009 · Cited by 213 · see citing works
van der Meer, M. A. A., Johnson, A. C., Schmitzer-Torbert, N. C., & Redish, A. D. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.023 · Cited by 210 · see citing works
The neurobiology of punishment
Seymour, B., Singer, T., & Dolan, R. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2119 · Cited by 240 · see citing works
Information Processing in Decision-Making Systems
van der Meer, M. A. A., Kurth-Nelson, Z. L., & Redish, A. D. (2012) · The Neuroscientist · doi:10.1177/1073858411435128 · Cited by 186 · see citing works
A computational reward learning account of social media engagement
Lindström, B., Bellander, M., Schultner, D. T., Chang, A., Tobler, P. N., & Amodio, D. M. (2021) · Nature Communications · doi:10.1038/s41467-020-19607-x · Cited by 185 · see citing works
Niv, Y. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.018 · Cited by 179 · see citing works
Self-citationSolway, A., & Botvinick, M. (2012) · Psychological Review · doi:10.1037/a0026435 · Cited by 180 · see citing works
Pezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163 · see citing works
Interactions of Motivation and Cognitive Control
Yee, D., & Braver, T. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.11.009 · Cited by 152 · see citing works
Animal affect and decision-making
Mendl, M., & Paul, E. S. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.01.025 · Cited by 161 · see citing works
Kumaran, D., Banino, A., Blundell, C., Hassabis, D., & Dayan, P. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.052 · Cited by 155 · see citing works
Self-citationLuciana, M., Wahlstrom, D., Porter, J. N., & Collins, P. F. (2012) · Developmental Psychology · doi:10.1037/a0027432 · Cited by 156 · see citing works
Adolescent brain development in normality and psychopathology
Luciana, M. M. (2013) · Development and Psychopathology · doi:10.1017/s0954579413000643 · Cited by 157 · see citing works
Sensitivity to “sunk costs” in mice, rats, and humans
Sweis, B. M., Abram, S. V., Schmidt, B., Seeland, K. D., MacDonald, A. W., Thomas, M. J., & Redish, A. D. (2018) · Science · doi:10.1126/science.aar8644 · Cited by 142 · see citing works
Thinking as the control of imagination: a conceptual framework for goal-directed systems
Pezzulo, G., & Castelfranchi, C. (2009) · Psychological Research Psychologische Forschung · doi:10.1007/s00426-009-0237-z · Cited by 141 · see citing works
Above and beyond the concrete: The diverse representational substrates of the predictive brain
Gilead, M., Trope, Y., & Liberman, N. (2019) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19002000 · Cited by 129 · see citing works
Intention, Choice, and the Medial Frontal Cortex
Rushworth, M. F. S. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1440.014 · Cited by 132 · see citing works
The dorsomedial striatum encodes net expected return, critical for energizing performance vigor
Wang, A. Y., Miura, K., & Uchida, N. (2013) · Nature Neuroscience · doi:10.1038/nn.3377 · Cited by 128 · see citing works
Reinforcement-learning in fronto-striatal circuits
Averbeck, B. B., & O’Doherty, J. P. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-01108-0 · Cited by 127 · see citing works
Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner
Silvetti, M., Vassena, E., Abrahamse, E. L., & Verguts, T. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006370 · Cited by 130 · see citing works
Dynamic Computation of Incentive Salience: “Wanting” What Was Never “Liked”
Tindell, A. J., Smith, K. S., Berridge, K. C., & Aldridge, J. W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2499-09.2009 · Cited by 124 · see citing works
Dissociating hippocampal and striatal contributions to sequential prediction learning
Bornstein, A. M., & Daw, N. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07920.x · Cited by 120 · see citing works
Play in predictive minds: A cognitive theory of play.
Andersen, M. M., Kiverstein, J., Miller, M., & Roepstorff, A. (2023) · Psychological Review · doi:10.1037/rev0000369 · Cited by 122 · see citing works
Multiple Forms of Value Learning and the Function of Dopamine
Balleine, B. W., Daw, N. D., & O’Doherty, J. P. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00024-5 · Cited by 112 · see citing works
Interactions between deliberation and delay-discounting in rats
Papale, A. E., Stott, J. J., Powell, N. J., Regier, P. S., & Redish, A. D. (2012) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-012-0097-7 · Cited by 110 · see citing works
Mai, B., Sommer, S., & Hauber, W. (2011) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-011-0068-4 · Cited by 108 · see citing works
The Cognitive Neuroscience of Motivation and Learning
Daw, N. D., & Daphna Shohamy, A. (2008) · Social Cognition · doi:10.1521/soco.2008.26.5.593 · Cited by 103 · see citing works
How to set the switches on this thing
Dayan, P. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.011 · Cited by 99 · see citing works
Self-citationSubjective costs drive overly patient foraging strategies in rats on an intertemporal foraging task
Wikenheiser, A. M., Stephens, D. W., & Redish, A. D. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1220738110 · Cited by 96 · see citing works
Multiplicity of control in the basal ganglia: computational roles of striatal subregions
Bornstein, A. M., & Daw, N. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.009 · Cited by 97 · see citing works
A theoretical account of cognitive effects in delay discounting
Kurth-Nelson, Z. L., Bickel, W. K., & Redish, A. D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08058.x · Cited by 97 · see citing works
Lex, B., & Hauber, W. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhp151 · Cited by 94 · see citing works
Desmurget, M., & Turner, R. S. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01010.2007 · Cited by 92 · see citing works
Redish, A. D., & Mizumori, S. J. Y. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.08.014 · Cited by 98 · see citing works
Distinct Motivational Effects of Contingent and Noncontingent Rewards
Manohar, S., Finzi, D., Drew, D., & Masud Husain, B. (2017) · Psychological Science · doi:10.1177/0956797617693326 · Cited by 87 · see citing works
Frömer, R., Wolf, C. K. D., & Shenhav, A. (2019) · Nature Communications · doi:10.1038/s41467-019-12931-x · Cited by 85 · see citing works
Ventral striatum: a critical look at models of learning and evaluation
van der Meer, M. A. A., & Redish, A. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.011 · Cited by 84 · see citing works
Dopamine and the interdependency of time perception and reward
Fung, B. J., Sutlief, E., & Shuler, M. G. H. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.02.030 · Cited by 79 · see citing works
Emotions as ComputationsPreprint
Emanuel, A., & Eldar, E. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104977 · Cited by 83 · see citing works
Neural systems analysis of decision making during goal-directed navigation
Penner, M. R., & Mizumori, S. J. Y. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.08.010 · Cited by 72 · see citing works
Action simulation in the human brain: Twelve questions
Pezzulo, G., Candidi, M., Dindo, H., & Barca, L. (2013) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2013.01.004 · Cited by 73 · see citing works
Smittenaar, P., Chase, H. W., Aarts, E., Nusselein, B., Bloem, B. R., & Cools, R. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08043.x · Cited by 73 · see citing works
Values and Actions in Aversion
Dayan, P., & Seymour, B. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00012-9 · Cited by 71 · see citing works
Self-citationExcessive use of reminders: Metacognition and effort-minimisation in cognitive offloading
Sachdeva, C., & Gilbert, S. J. (2020) · Consciousness and Cognition · doi:10.1016/j.concog.2020.103024 · Cited by 72 · see citing works
Kubota, Y., Liu, J., Hu, D., DeCoteau, W. E., Eden, U. T., Smith, A. C., & Graybiel, A. M. (2009) · Journal of Neurophysiology · doi:10.1152/jn.00522.2009 · Cited by 66 · see citing works
Addiction as vulnerabilities in the decision process
Redish, A. D., Jensen, S., & Johnson, A. C. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x08004986 · Cited by 68 · see citing works
Expectancies in decision making, reinforcement learning, and ventral striatum
van der Meer, M. A. A. (2010) · Frontiers in Neuroscience · doi:10.3389/neuro.01.006.2010 · Cited by 67 · see citing works
Dopamine signals as temporal difference errors: recent advances
Starkweather, C. K., & Uchida, N. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.08.014 · Cited by 67 · see citing works
Prefrontal/accumbal catecholamine system processes high motivational salience
Puglisi‐Allegra, S., & Ventura, R. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00031 · Cited by 70 · see citing works
The Value of Foresight: How Prospection Affects Decision-Making
Pezzulo, G., & Rigoli, F. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00079 · Cited by 64 · see citing works
Theoretical and Empirical Studies of Learning
Niv, Y., & Montague, P. R. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00022-1 · Cited by 63 · see citing works
Self-citationOn the Evolution and Optimality of Mood States
Trimmer, P. C., Paul, E. S., Mendl, M., McNamara, J. M., & Houston, A. I. (2013) · Behavioral Sciences · doi:10.3390/bs3030501 · Cited by 61 · see citing works
Murphy, A., Taylor, E., & Elliott, R. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00101 · Cited by 60 · see citing works
From operant learning to cognitive enrichment in farm animal housing: bases and applicability
Manteuffel, G., Langbein, J., & Puppe, B. (2009) · Animal Welfare · doi:10.1017/s0962728600000105 · Cited by 61 · see citing works
Computational validity: using computation to translate behaviours across species
Redish, A. D., Kepecs, Á., Anderson, L. M., Calvin, O. L., Grissom, N. M., Haynos, A. F., Heilbronner, S. R., Herman, A. B., Jacob, S., Ma, S., et al. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2020.0525 · Cited by 59 · see citing works
Safety out of control: dopamine and defence
Lloyd, K., & Dayan, P. (2016) · Behavioral and Brain Functions · doi:10.1186/s12993-016-0099-7 · Cited by 59 · see citing works
Self-citationNeuroimaging of the dopamine/reward system in adolescent drug use
Ernst, M., & Luciana, M. M. (2015) · CNS Spectrums · doi:10.1017/s1092852915000395 · Cited by 55 · see citing works
Toward sophisticated basal ganglia neuromodulation: Review on basal ganglia deep brain stimulation
Cunha, C. D., Boschen, S. L., Gómez‐A, A., Ross, E. K., Gibson, W. S. J., Min, H.-K. P., Lee, K. H., & Blaha, C. D. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.02.003 · Cited by 58 · see citing works
Goal-Directed Decision Making with Spiking Neurons
Friedrich, J. H., & Lengyel, M. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.2854-15.2016 · Cited by 57 · see citing works
A neuronal mechanism for motivational control of behavior
Courtin, J., Bitterman, Y., Müller, S., Hinz, J., Hagihara, K. M., Müller, C. R., & Lüthi, A. (2022) · Science · doi:10.1126/science.abg7277 · Cited by 52 · see citing works
Action Change Theory: A Reinforcement Learning Perspective on Behavior Change
Vlaev, I., & Dolan, P. (2015) · Review of General Psychology · doi:10.1037/gpr0000029 · Cited by 55 · see citing works
Frömer, R., & Shenhav, A. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.12.006 · Cited by 50 · see citing works
Lloyd, K., & Dayan, P. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004622 · Cited by 49 · see citing works
Self-citationIncreasing farm animal welfare by positively motivated instrumental behaviour
Manteuffel, G., Langbein, J., & Puppe, B. (2009) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2009.02.014 · Cited by 49 · see citing works
Model-free and model-based learning processes in the updating of explicit and implicit evaluations
Kurdi, B., Gershman, S. J., & Banaji, M. R. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1820238116 · Cited by 50 · see citing works
Goal-directed decision making in prefrontal cortex: A computational framework.
Botvinick, M., & An, J. (2009) · PubMed · Cited by 45 · see citing works
Elucidating Poor Decision-Making in a Rat Gambling Task
Rivalan, M., Valton, V., Seriès, P., Marchand, A. R., & Dellu-Hagedorn, F. (2013) · PLoS ONE · doi:10.1371/journal.pone.0082052 · Cited by 46 · see citing works
The Neural Basis of Frustration State
Yu, R. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-800935-2.00011-7 · Cited by 45 · see citing works
Dopamine release in nucleus accumbens during rewarded task switching measured by [¹¹C]raclopride
Jonasson, L. S., Axelsson, J., Riklund, K., Braver, T. S., Ögren, M., Bäckman, L., & Nyberg, L. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.047 · Cited by 42 · see citing works
Deliberation and Procedural Automation on a Two-Step Task for Rats
Hasz, B. M., & Redish, A. D. (2018) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2018.00030 · Cited by 41 · see citing works
Dopamine promotes instrumental motivation, but reduces reward-related vigour
Grogan, J. P., Sandhu, T., Hu, M. T., & Manohar, S. (2020) · eLife · doi:10.7554/elife.58321 · Cited by 38 · see citing works
The Role of Value Systems in Decision Making
Dayan, P. (2008) · The MIT Press eBooks · doi:10.7551/mitpress/7735.003.0006 · Cited by 41 · see citing works
Self-citationIntegrating early results on ventral striatal gamma oscillations in the rat
van der Meer, M. A. A. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00028 · Cited by 41 · see citing works
- Authors
- NIV, Y., JOEL, D., & DAYAN, P.
- Affiliations
- Hebrew University of Jerusalem (IL)
- Tel Aviv University (IL)
- Gatsby Computational Neuroscience Unit (GB)
- Corresponding author
- NIV, Y
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2006-08
- Volume
- 10
- Issue
- 8
- Pages
- 375-381
- ISSN
- 1364-6613 (ISSN-L)
- OpenAlex
- W2148819999
- Cited by
- 251
- References
- 50
- Citations by year
- 2026: 9 · 2025: 5 · 2024: 11 · 2023: 10 · 2022: 10 · 2021: 12 · 2020: 17 · 2019: 12 · 2018: 12 · 2017: 9 · 2016: 13 · 2015: 15 · 2014: 14 · 2013: 11 · 2012: 21
- FWCI
- 6.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Trends in Cognitive Sciences (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- motivation, goal-directed behavior, habitual behavior, utility maximization, motivational states, outcome value, decision-making, behavioral training, normative model
- MeSH terms
- Animals, Decision Making, Goals, Habituation, Psychophysiologic, Humans, Models, Psychological, Motivation, Reinforcement, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review