Computational roles for dopamine in behavioural control
Abstract
Neuromodulators such as dopamine have a central role in cognitive disorders. In the past decade, biological findings on dopamine function have been infused with concepts taken from computational theories of reinforcement learning. These more abstract approaches have now been applied to describe the biological algorithms at play in our brains when we form value judgements and make choices. The application of such quantitative models has opened up new fields, ripe for attack by young synthesizers and theoreticians.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The findings are consistent with different functions associated with distinct regions of the striatum: afferent projections to dorsal striatum are crucial for learning correct actions to maximize reward, whereas projections to ventral striatum including nucleus accumbens have a key function in reward prediction and pavlovian learning.
“These findings are consistent with the different functions associated with distinct regions of the striatum: afferent projections to dorsal striatum are thought to be crucial for learning correct actions so as to maximize reward, whereas projections to ventral striatum, including nucleus accumbens, have a key function in reward prediction and pavlovian learning18-20,23,24.”
From Empathic neural responses are modulated by the perceived fairness of others (2006) · cites this paper as [24]
Cites
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · see citing works
Self-citationA neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Self-citationReinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
The Psychology of Learning and Motivation: Advances in Research and Theory
(2010) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/c2009-0-62209-1 · Cited by 6,035 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Di Chiara, G., & Imperato, A. (1988) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.85.14.5274 · Cited by 4,733 · see citing works
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127 · see citing works
From Neuropsychology to Mental Structure
Shallice, T. (1988) · doi:10.1017/cbo9780511526817 · Cited by 4,072 · see citing works
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027 · see citing works
Implications of Normal Brain Development for the Pathogenesis of Schizophrenia
Weinberger, D. R. (1987) · Archives of General Psychiatry · doi:10.1001/archpsyc.1987.01800190080012 · Cited by 3,724 · see citing works
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601 · see citing works
A neural system for error detection and compensation
Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1993) · Psychological Science · doi:10.1111/j.1467-9280.1993.tb00586.x · Cited by 2,862 · see citing works
A psychomotor stimulant theory of addiction.
Wise, R. A., & Bozarth, M. A. (1987) · Psychological Review · doi:10.1037/0033-295x.94.4.469 · Cited by 2,747 · see citing works
Models of Working Memory: Mechanisms of Active Maintenance and Executive Control
Miyake, A., & Shah, P. (1999) · Cited by 2,404 · see citing works
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244 · see citing works
Barto, A. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,172 · see citing works
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Self-citationZhang, B., Li, C., & Lin, N. (2020) · De Gruyter eBooks · doi:10.1515/9783110595567-011 · Cited by 1,939 · see citing works
The Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, E., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,922 · see citing works
The Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses
Gehring, W. J., & Willoughby, A. R. (2002) · Science · doi:10.1126/science.1066893 · Cited by 1,884 · see citing works
Principles of Frontal Lobe Function
Stuss, D. T. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195134971.001.0001 · Cited by 1,708 · see citing works
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · see citing works
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,486 · see citing works
Control of Cognitive Processes: Attention and Performance XVIII
Monsell, S., & Driver, J. (2000) · UCL Discovery (University College London) · Cited by 1,451 · see citing works
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450 · see citing works
Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque
Miller, E. K., Erickson, C. A., & Desimone, R. (1996) · Journal of Neuroscience · doi:10.1523/jneurosci.16-16-05154.1996 · Cited by 1,441 · see citing works
FMRI visualization of brain activity during a monetary incentive delay task
Knutson, B., Westdorp, A., Kaiser, E., & Hommer, D. (2000) · NeuroImage · doi:10.1006/nimg.2000.0593 · Cited by 1,414 · see citing works
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404 · see citing works
Addiction and the brain: The neurobiology of compulsion and its persistence
Hyman, S. E., & Malenka, R. C. (2001) · Nature reviews. Neuroscience · doi:10.1038/35094560 · Cited by 1,297 · see citing works
Self-citationSchultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291 · see citing works
Acute effects of cocaine on human brain activity and emotion
Breiter, H. C., Gollub, R. L., Weisskoff, R. M., Kennedy, D. N., Makris, N., Berke, J. D., Goodman, J. M., Kantor, H. L., Gastfriend, D. R., Riorden, J. P., et al. (1997) · Neuron · doi:10.1016/s0896-6273(00)80374-8 · Cited by 1,270 · see citing works
Self-citationModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190 · see citing works
Addiction, Dopamine, and the Molecular Mechanisms of Memory
Berke, J. D., & Hyman, S. E. (2000) · Neuron · doi:10.1016/s0896-6273(00)81056-9 · Cited by 1,167 · see citing works
Self-citationTracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · see citing works
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,103 · see citing works
Beautiful Faces Have Variable Reward Value
Aharon, I., Etcoff, N., Ariely, D., Chabris, C. F., O'Connor, E., & Breiter, H. C. (2001) · Neuron · doi:10.1016/s0896-6273(01)00491-3 · Cited by 1,087 · see citing works
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076 · see citing works
Chklovskii, D. B. (2002) · Neuron · doi:10.1016/s0896-6273(02)00703-1 · Cited by 1,074 · see citing works
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · see citing works
Neural Economics and the Biological Substrates of Valuation
Montague, P. R., & Berns, G. S. (2002) · Neuron · doi:10.1016/s0896-6273(02)00974-1 · Cited by 994 · see citing works
Self-citationTemporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · see citing works
Self-citationReward, Motivation, and Reinforcement Learning
Dayan, P., & Balleine, B. W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00963-7 · Cited by 855 · see citing works
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · see citing works
Salamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817 · see citing works
A Network Model of Catecholamine Effects: Gain, Signal-to-Noise Ratio, and Behavior
Servan‐Schreiber, D., Printz, H., & Cohen, J. D. (1990) · Science · doi:10.1126/science.2392679 · Cited by 770 · see citing works
Self-citationDissociable Neural Responses in Human Reward Systems
Elliott, R., Friston, K. J., & Dolan, R. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-16-06159.2000 · Cited by 740 · see citing works
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711 · see citing works
Self-citationAssociative processes in addiction and reward. The role of amygdala-ventral striatal subsystems
EVERITT, B. J., PARKINSON, J. A., OLMSTEAD, M. C., ARROYO, M., ROBLEDO, P., & ROBBINS, T. W. (1999) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1999.tb09280.x · Cited by 708 · see citing works
A Neural Mechanism for Working and Recognition Memory in Inferior Temporal Cortex
Miller, E. K., Li, L., & Desimone, R. (1991) · Science · doi:10.1126/science.1962197 · Cited by 706 · see citing works
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704 · see citing works
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506 · see citing works
Skinner, B. F. (1963) · Science · doi:10.1126/science.140.3570.951 · Cited by 488 · see citing works
Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals
Holroyd, C. B., Nieuwenhuis, S., Yeung, N., Nystrom, L. E., Mars, R. B., COLES, M. G. H., & Cohen, J. D. (2004) · Nature Neuroscience · doi:10.1038/nn1238 · Cited by 487 · see citing works
Self-citationUllsperger, M., & von Cramon, D. Y. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-10-04308.2003 · Cited by 484 · see citing works
Disorganisation of behaviour after frontal lobe damage
Duncan, J. (1986) · Cognitive Neuropsychology · doi:10.1080/02643298608253360 · Cited by 452 · see citing works
Activity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451 · see citing works
Self-citationErrors in reward prediction are reflected in the event-related brain potential
Holroyd, C. B., Nieuwenhuis, S., Yeung, N., & Cohen, J. D. (2003) · Neuroreport · doi:10.1097/00001756-200312190-00037 · Cited by 433 · see citing works
Self-citationA computational substrate for incentive salience
McClure, S. M., Daw, N. D., & Montague, P. R. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00177-2 · Cited by 416 · see citing works
Self-citationGambling Urges in Pathological Gambling: A Functional Magnetic Resonance Imaging Study
Potenza, M. N., Steinberg, M. A., Skudlarski, P., Fulbright, R. K., Lacadie, C. M., Wilber, M. K., Rounsaville, B. J., Gore, J. C., & Wexler, B. E. (2003) · Archives of General Psychiatry · doi:10.1001/archpsyc.60.8.828 · Cited by 414 · see citing works
Context dependence of the event‐related brain potential associated with reward and punishment
Holroyd, C. B., Larsen, J. T., & Cohen, J. D. (2004) · Psychophysiology · doi:10.1111/j.1469-8986.2004.00152.x · Cited by 376 · see citing works
Self-citationNieuwenhuis, S. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh034 · Cited by 331 · see citing works
Morphological analysis of dendritic spine development in primary cultures of hippocampal neurons
Papa, M., Bundman, M. C., Greenberger, V., & Segal, M. (1995) · Journal of Neuroscience · doi:10.1523/jneurosci.15-01-00001.1995 · Cited by 330 · see citing works
A Biologically Based Computational Model of Working Memory
O’Reilly, R. C., Braver, T. S., & Cohen, J. D. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9781139174909.014 · Cited by 323 · see citing works
Self-citationActivation of the human brain by monetary reward
Thut, G., Schultz, W., Roelcke, U., Nienhusmeier, M., Missimer, J. H., Maguire, R. P., & Leenders, K. L. (1997) · Neuroreport · doi:10.1097/00001756-199703240-00033 · Cited by 273 · see citing works
Value-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268 · see citing works
O'Reilly, R. C. (2002) · Cerebral Cortex · doi:10.1093/cercor/12.3.246 · Cited by 241 · see citing works
Balleine, B. W., & Dickinson, A. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-23-08954.2000 · Cited by 218 · see citing works
Ikemoto, S., & Panksepp, J. (1996) · Behavioral Neuroscience · doi:10.1037//0735-7044.110.2.331 · Cited by 216 · see citing works
Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals
Montague, P. R., McClure, S. M., Baldwin, P. R., Phillips, P. E. M., Budygin, E. A., Stuber, G. D., Kilpatrick, M., & Wightman, R. M. (2004) · Journal of Neuroscience · doi:10.1523/jneurosci.4279-03.2004 · Cited by 174 · see citing works
Self-citationIkemoto, S., & Panksepp, J. (1996) · Behavioral Neuroscience · doi:10.1037/0735-7044.110.2.331 · Cited by 170 · see citing works
Perspectives of Event-Related Potentials Research
Karmos, G. (1995) · Medical Entomology and Zoology · Cited by 167 · see citing works
A Computational Role for Dopamine Delivery in Human Decision-Making
Egelman, D. M., Person, C., & Montague, P. R. (1998) · Journal of Cognitive Neuroscience · doi:10.1162/089892998563022 · Cited by 95 · see citing works
Self-citationLearning representations in a gated prefrontal cortex model of dynamic task switching
Rougier, N. P., & O’Reilly, R. C. (2002) · Cognitive Science · doi:10.1207/s15516709cog2604_4 · Cited by 78 · see citing works
Poole, D. L., & Mackworth, A. K. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009258227.018 · Cited by 24 · see citing works
Dopamine Receptors and Antipsychotic Drugs
Seeman, P. M. (1997) · Nihon shinkei seishin yakurigaku zasshi · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Rescorla, R. A. & Wagner A. R. in Classical Conditioning 2: Current Research and Theory (eds Black, A. H. & Prokasy, W. F.) 64–69 (Appleton Century-Crofts, New York, 1972).
- Bayer, H. M. & Glimcher, P. W. Subjective estimates of objective rewards: using economic discounting to link behavior and brain. Soc. Neurosci. Abstr. 28, 358.6 (2002).
Cited by
Sorted by citations
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · see citing works
Self-citationBardo, M. T., & Hammer, R. P. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90226-e · Cited by 3,229 · see citing works
NEURAL MECHANISMS OF ADDICTION: The Role of Reward-Related Learning and Memory
Hyman, S. E., Malenka, R. C., & Nestler, E. J. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113009 · Cited by 2,647 · see citing works
Self-citationThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347 · see citing works
Empathic neural responses are modulated by the perceived fairness of others
Singer, T., Seymour, B., O'Doherty, J. P., Stephan, K. E., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature04271 · Cited by 1,726 · see citing works
The neural bases of momentary lapses in attention
Weissman, D. H., Roberts, K. C., Visscher, K. M., & Woldorff, M. G. (2006) · Nature Neuroscience · doi:10.1038/nn1727 · Cited by 1,566 · see citing works
A new look at habits and the habit-goal interface.
Wood, W., & Neal, D. T. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.843 · Cited by 1,470 · see citing works
Getting to Know You: Reputation and Trust in a Two-Person Economic Exchange
King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., & Montague, P. R. (2005) · Science · doi:10.1126/science.1108062 · Cited by 1,345 · see citing works
Self-citationEye movements in natural behavior
Hayhoe, M., & Ballard, D. H. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.02.009 · Cited by 1,321 · see citing works
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190 · see citing works
“Liking” and “wanting” food rewards: Brain substrates and roles in eating disorders
Berridge, K. C. (2009) · Physiology & Behavior · doi:10.1016/j.physbeh.2009.02.044 · Cited by 1,147 · see citing works
Synaptic plasticity and addiction
Kauer, J. A., & Malenka, R. C. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2234 · Cited by 1,397 · see citing works
Kober, H., Barrett, L. F., Joseph, J., Bliss-Moreau, E., Lindquist, K., & Wager, T. D. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.059 · Cited by 1,144 · see citing works
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121 · see citing works
Self-citationStriatal Plasticity and Basal Ganglia Circuit Function
Kreitzer, A. C., & Malenka, R. C. (2008) · Neuron · doi:10.1016/j.neuron.2008.11.005 · Cited by 1,009 · see citing works
Addiction: A Disease of Learning and Memory
Hyman, S. E. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.8.1414 · Cited by 980 · see citing works
Self-citationCortico-Basal Ganglia Reward Network: Microcircuitry
Sesack, S. R., & Grace, A. A. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.93 · Cited by 985 · see citing works
The somatic marker hypothesis: A critical evaluation
Dunn, B. D., Dalgleish, T., & Lawrence, A. D. (2006) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2005.07.001 · Cited by 894 · see citing works
The tempted brain eats: Pleasure and desire circuits in obesity and eating disorders
Berridge, K. C., Ho, C.-Y., Richard, J. M., & DiFeliceantonio, A. G. (2010) · Brain Research · doi:10.1016/j.brainres.2010.04.003 · Cited by 873 · see citing works
Consciousness without a cerebral cortex: A challenge for neuroscience and medicine
Merker, B. (2007) · Behavioral and Brain Sciences · doi:10.1017/s0140525x07000891 · Cited by 868 · see citing works
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809 · see citing works
Self-citationNew insights into symptoms and neurocircuit function of anorexia nervosa
Kaye, W. H., Fudge, J. L., & Paulus, M. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2682 · Cited by 843 · see citing works
Human insula activation reflects risk prediction errors as well as risk
Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4286-07.2008 · Cited by 784 · see citing works
Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits
Lerner, T. N., Shilyansky, C., Davidson, T. J., Evans, K. E., Beier, K. T., Zalocusky, K. A., Crow, A. K., Malenka, R. C., Luo, L., Tomer, R., et al. (2015) · Cell · doi:10.1016/j.cell.2015.07.014 · Cited by 794 · see citing works
“…neurons suggested that a key function of DA is to transmit reward prediction error signals (Mirenowicz and Schultz, 1996; Schultz et al., 1997; Waelti et al., 2001), a hypothesis concordant with temporal difference learning models (Montague et al., 1996, 2004; Steinberg et al., 2013; Sutton, 1988).”
Eye guidance in natural vision: reinterpreting salience
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011) · Journal of Vision · doi:10.1167/11.5.5 · Cited by 775 · see citing works
ENDOCANNABINOID-MEDIATED SYNAPTIC PLASTICITY IN THE CNS
Chevaleyre, V., Takahashi, K. A., & Castillo, P. E. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112834 · Cited by 779 · see citing works
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762 · see citing works
A Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655 · see citing works
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710 · see citing works
Witten, I. B., Steinberg, E. E., Lee, S. Y., Davidson, T. J., Zalocusky, K. A., Brodsky, M., Yizhar, O., Cho, S. L., Gong, S., Ramakrishnan, C., et al. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.028 · Cited by 680 · see citing works
Temporal self-regulation theory: A model for individual health behavior
Hall, P. A., & Fong, G. T. (2007) · Health Psychology Review · doi:10.1080/17437190701492437 · Cited by 679 · see citing works
REVIEW: Behavioural assessment of drug reinforcement and addictive features in rodents: an overview
Sanchis‐Segura, C., & Spanagel, R. (2006) · Addiction Biology · doi:10.1111/j.1369-1600.2006.00012.x · Cited by 629 · see citing works
Neurobiology of anorexia and bulimia nervosa
Kaye, W. (2008) · Physiology & Behavior · doi:10.1016/j.physbeh.2007.11.037 · Cited by 625 · see citing works
Ventral Tegmental Area Neurons in Learned Appetitive Behavior and Positive Reinforcement
Fields, H. L., Hjelmstad, G. O., Margolis, E. B., & Nicola, S. M. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094341 · Cited by 611 · see citing works
Alcaro, A., Huber, R., & Panksepp, J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.07.014 · Cited by 614 · see citing works
<i>Spatial Cognition and the Brain</i>
Burgess, N. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1440.002 · Cited by 614 · see citing works
Intense Sweetness Surpasses Cocaine Reward
Lenoir, M., Serre, F., Cantin, L., & Ahmed, S. H. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000698 · Cited by 608 · see citing works
Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories
Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., & Graybiel, A. M. (2005) · Nature · doi:10.1038/nature04053 · Cited by 595 · see citing works
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548 · see citing works
Moment-to-moment brain signal variability: A next frontier in human brain mapping?
Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W. S., Lindenberger, U., McIntosh, A. R., & Grady, C. L. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.015 · Cited by 594 · see citing works
Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008) · Psychophysiology · doi:10.1111/j.1469-8986.2008.00668.x · Cited by 581 · see citing works
Computational psychiatry: the brain as a phantastic organ
Friston, K. J., Stephan, K. E., Montague, R., & Dolan, R. J. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70275-5 · Cited by 531 · see citing works
Choosing the greater of two goods: neural currencies for valuation and decision making
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1666 · Cited by 536 · see citing works
Reward Processing in Schizophrenia: A Deficit in the Representation of Value
Gold, J. M., Waltz, J. A., Prentice, K. J., Morris, S. E., & Heerey, E. A. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn068 · Cited by 521 · see citing works
Serotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464 · see citing works
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N. D., Dayan, P., Singer, T., & Dolan, R. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 455 · see citing works
Category Learning in the Brain
Seger, C. A., & Miller, E. K. (2010) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135546 · Cited by 454 · see citing works
Opponency revisited: competition and cooperation between dopamine and serotonin
Boureau, Y.-L., & Dayan, P. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.151 · Cited by 456 · see citing works
Collerton, D., Perry, E. K., & McKeith, I. G. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000130 · Cited by 437 · see citing works
Yin, H. H., Ostlund, S. B., & Balleine, B. W. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06422.x · Cited by 416 · see citing works
The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward.
Humphries, M. D., & Prescott, T. J. (2009) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2009.11.003 · Cited by 414 · see citing works
Human aging magnifies genetic effects on executive functioning and working memory
Nagel, I. E. (2008) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.001.2008 · Cited by 415 · see citing works
Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent Plasticity
Pawlak, V., & Kerr, J. N. D. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.4402-07.2008 · Cited by 412 · see citing works
Dopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381 · see citing works
Basal ganglia orient eyes to reward
Hikosaka, O., Nakamura, K., & Nakahara, H. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00458.2005 · Cited by 377 · see citing works
Serotonin in Affective Control
Dayan, P., & Huys, Q. J. M. (2009) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.051508.135607 · Cited by 381 · see citing works
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342 · see citing works
Oakman, S., Faris, P. L., Kerr, P., Cozzari, C., & Hartman, B. (1995) · Journal of Neuroscience · doi:10.1523/jneurosci.15-09-05859.1995 · Cited by 384 · see citing works
IMAGING VALUATION MODELS IN HUMAN CHOICE
Montague, P. R., King‐Casas, B., & Cohen, J. D. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112903 · Cited by 371 · see citing works
Self-citationNeural underpinnings of within-person variability in cognitive functioning.
MacDonald, S. W. S., Li, S., & Bäckman, L. (2009) · Psychology and Aging · doi:10.1037/a0017798 · Cited by 365 · see citing works
Beyond the FRN: Broadening the time-course of EEG and ERP components implicated in reward processing
Glazer, J. E., Kelley, N. J., Pornpattananangkul, N., Mittal, V. A., & Nusslock, R. (2018) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2018.02.002 · Cited by 362 · see citing works
Kreitzer, A. C., & Malenka, R. C. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.2959-05.2005 · Cited by 358 · see citing works
Stocco, A., Lebiere, C. J., & Anderson, J. R. (2010) · Psychological Review · doi:10.1037/a0019077 · Cited by 352 · see citing works
The good, the bad and the neutral: Electrophysiological responses to feedback stimuli
Holroyd, C. B., Hajcak, G., & Larsen, J. T. (2006) · Brain Research · doi:10.1016/j.brainres.2005.12.015 · Cited by 350 · see citing works
Waltz, J. A., & Gold, J. M. (2007) · Schizophrenia Research · doi:10.1016/j.schres.2007.03.010 · Cited by 331 · see citing works
Altered Reward Processing in Women Recovered From Anorexia Nervosa
Wagner, A., Aizenstein, H., Venkatraman, V. K., Fudge, J., May, J. C., Mazurkewicz, L., Frank, G. K., Bailer, U. F., Fischer, L., Nguyen, V., et al. (2007) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2007.07040575 · Cited by 342 · see citing works
Changes and challenges in 20 years of research into the development of executive functions
Hughes, C. (2011) · Infant and Child Development · doi:10.1002/icd.736 · Cited by 347 · see citing works
Hebbian learning of cognitive control: Dealing with specific and nonspecific adaptation.
Verguts, T., & Notebaert, W. (2008) · Psychological Review · doi:10.1037/0033-295x.115.2.518 · Cited by 337 · see citing works
Learning to Attend and to Ignore Is a Matter of Gains and Losses
Della Libera, C., & Chelazzi, L. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02360.x · Cited by 329 · see citing works
Reinforcement Learning Signals Predict Future Decisions
Cohen, M. X., & Charan Ranganath, A. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.4421-06.2007 · Cited by 333 · see citing works
Animal Foraging and the Evolution of Goal‐Directed Cognition
Hills, T. T. (2006) · Cognitive Science · doi:10.1207/s15516709cog0000_50 · Cited by 331 · see citing works
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304 · see citing works
Negative Symptoms and the Failure to Represent the Expected Reward Value of Actions
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 319 · see citing works
The Dynamics of Perceptual Learning: An Incremental Reweighting Model.
Petrov, A. A., Dosher, B. A., & Lu, Z. (2005) · Psychological Review · doi:10.1037/0033-295x.112.4.715 · Cited by 313 · see citing works
Behavior needs neural variability
Waschke, L., Kloosterman, N. A., Obleser, J., & Garrett, D. D. (2021) · Neuron · doi:10.1016/j.neuron.2021.01.023 · Cited by 301 · see citing works
Goschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293 · see citing works
Cohen, J. D. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196750 · Cited by 304 · see citing works
Self-citationA Neural Computational Model of Incentive Salience
Zhang, J., Berridge, K. C., Tindell, A. J., Smith, K. S., & Aldridge, J. W. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000437 · Cited by 299 · see citing works
van den Bos, W., Cohen, M. X., Kahnt, T., & Crone, E. A. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr198 · Cited by 276 · see citing works
Embedding reward signals into perception and cognition
Pessoa, L. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00017 · Cited by 279 · see citing works
Kelly, C., de Zubicaray, G., Di Martino, A., Copland, D. A., Reiss, P. T., Klein, D. F., Castellanos, F. X., Milham, M. P., & McMahon, K. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.0810-09.2009 · Cited by 292 · see citing works
Serotonin is necessary for place memory in Drosophila
Sitaraman, D., Zars, M., LaFerriere, H., Chen, Y.-C., Sable-Smith, A., Kitamoto, T., Rottinghaus, G. E., & Zars, T. D. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0710168105 · Cited by 291 · see citing works
Goschke, T., & Bolte, A. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.07.015 · Cited by 298 · see citing works
NeuroEconomics: An overview from an economic perspective
Kenning, P., & Plaßmann, H. (2005) · Brain Research Bulletin · doi:10.1016/j.brainresbull.2005.07.006 · Cited by 287 · see citing works
Decision-making during gambling: an integration of cognitive and psychobiological approaches
Clark, L. (2010) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0147 · Cited by 294 · see citing works
Neural Response to Reward Anticipation under Risk Is Nonlinear in Probabilities
Hsu, M., Krajbich, I. M., Zhao, C., & Camerer, C. F. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5296-08.2009 · Cited by 280 · see citing works
Dopamine, learning, and reward-seeking behavior
Arias-Carrión, Ó., & Pöppel, E. (2007) · Acta Neurobiologiae Experimentalis · doi:10.55782/ane-2007-1664 · Cited by 274 · see citing works
Blunted response to feedback information in depressive illness
Steele, J. D., Kumar, P., & Ebmeier, K. P. (2007) · Brain · doi:10.1093/brain/awm150 · Cited by 261 · see citing works
Maternal neglect: oxytocin, dopamine and the neurobiology of attachment
Strathearn, L. (2011) · Journal of Neuroendocrinology · doi:10.1111/j.1365-2826.2011.02228.x · Cited by 268 · see citing works
Neural signature of fictive learning signals in a sequential investment task
Lohrenz, T. M., McCabe, K., Camerer, C. F., & Montague, P. R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0608842104 · Cited by 263 · see citing works
Self-citationMenegas, W., Babayan, B. M., Uchida, N., & Watabe‐Uchida, M. (2017) · eLife · doi:10.7554/elife.21886 · Cited by 265 · see citing works
A role for mesencephalic dopamine in activation: commentary on Berridge (2006)
Robbins, T. W., & Everitt, B. J. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0528-7 · Cited by 265 · see citing works
What is reinforced by phasic dopamine signals?
Redgrave, P., GURNEY, K. N., & Reynolds, J. N. J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 254 · see citing works
Control of synaptic plasticity in deep cortical networks
Roelfsema, P. R., & Holtmaat, A. J. D. G. (2018) · Nature reviews. Neuroscience · doi:10.1038/nrn.2018.6 · Cited by 245 · see citing works
Better or worse than expected? Aging, learning, and the ERN
Eppinger, B., Kray, J., Mock, B., & Mecklinger, A. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2007.09.001 · Cited by 248 · see citing works
The influence of reward associations on conflict processing in the Stroop task
Krebs, R. M., Boehler, C. N., & Woldorff, M. G. (2010) · Cognition · doi:10.1016/j.cognition.2010.08.018 · Cited by 246 · see citing works
Inferring on the intentions of others by hierarchical Bayesian learning
Diaconescu, A. O., Mathys, C., Weber, L. A. E., Daunizeau, J., Kasper, L., Lomakina, E. I., Fehr, E., & Stephan, K. E. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003810 · Cited by 219 · see citing works
Role of Dopamine Neurons in Reward and Aversion: A Synaptic Plasticity Perspective
Pignatelli, M., & Bonci, A. (2015) · Neuron · doi:10.1016/j.neuron.2015.04.015 · Cited by 244 · see citing works
Sensitivity to reward and punishment in eating disorders
Harrison, A., O'Brien, N., Lopez, C., & Treasure, J. (2010) · Psychiatry Research · doi:10.1016/j.psychres.2009.06.010 · Cited by 244 · see citing works
Uhl, G. R., Koob, G. F., & Cable, J. (2019) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13989 · Cited by 259 · see citing works
- Authors
- Montague, P. R., Hyman, S. E., & Cohen, J. D.
- Affiliations
- Baylor College of Medicine (US)
- Harvard University (US)
- University of Pittsburgh (US)
- Venue
- Nature
- Published
- 2004-10
- Volume
- 431
- Issue
- 7010
- Pages
- 760-767
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2165368112
- Semantic Scholar
- 16cd7a24d117f61da645fba1b4cc31839dff2805
- Cited by
- 966
- References
- 77
- Citations by year
- 2026: 6 · 2025: 13 · 2024: 19 · 2023: 36 · 2022: 21 · 2021: 26 · 2020: 32 · 2019: 31 · 2018: 39 · 2017: 40 · 2016: 42 · 2015: 49 · 2014: 51 · 2013: 56 · 2012: 82
- FWCI
- 4.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- dopamine, neuromodulation, reinforcement learning, computational model, value judgments, decision-making, behavioral control, cognitive disorders
- MeSH terms
- Animals, Choice Behavior, Cognition, Dopamine, Humans, Learning, Models, Neurological, Schizophrenia, Substance-Related Disorders
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review