The DOI matches its Crossref record; 180 of the 180 references it cites were found in a registry or an index.
Reinforcement learning in the brain
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,342
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
(1999) · Cited by 5,147
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?
Berridge, K. C., & Robinson, T. E. (1998) · Brain Research Reviews · doi:10.1016/s0165-0173(98)00019-8 · Cited by 4,127
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Dynamic Programming and Markov Processes.
Freimer, M., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,378
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 3,153
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Advances in Neural Information Processing Systems 37
(2024) · doi:10.52202/079017 · Cited by 2,791
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Scalar expectancy theory and Weber's law in animal timing.
Gibbon, J. (1977) · Psychological Review · doi:10.1037//0033-295x.84.3.279 · Cited by 2,221
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Cardinal, R. N., Parkinson, J. A., Hall, J., & Everitt, B. J. (2002) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(02)00007-6 · Cited by 2,129
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Punishment and aversive behavior
Campbell, B. A., & Church, R. M. (1969) · Appleton-Century-Crofts eBooks · Cited by 1,989
Grace, A. A. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90196-u · Cited by 1,831
The neural basis of loss aversion in decision-making under risk
Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007) · Science · doi:10.1126/science.1134239 · Cited by 1,733
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Logothetis, N. K., & Wandell, B. A. (2004) · Annual Review of Physiology · doi:10.1146/annurev.physiol.66.082602.092845 · Cited by 1,568
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Predictability, surprise, attention, and conditioning
Kamin, L. J. (1967) · NASA Technical Reports Server (NASA) · Cited by 1,483
Neuro-Dynamic Programming: NDP
Bertsekas, D. P. (2001) · Encyclopedia of Optimization · doi:10.1007/0-306-48332-7_333 · Cited by 1,466
Ikemoto, S., & Panksepp, J. (1999) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00023-5 · Cited by 1,450
Some studies in machine learning using the game of checkers
Samuel, A. L. (2000) · IBM Journal of Research and Development · doi:10.1147/rd.441.0206 · Cited by 1,439
Neural Correlates of Behavioral Preference for Culturally Familiar Drinks
McClure, S. M., Li, J., Tomlin, D., Cypert, K. S., Montague, L. M., & Montague, P. R. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.019 · Cited by 1,353
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Dissociation of reward anticipation and outcome with event-related fMRI
Knutson, B., Fong, G. W., Adams, C. M., Varner, J. L., & Hommer, D. (2001) · Neuroreport · doi:10.1097/00001756-200112040-00016 · Cited by 1,300
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · cited 1×
“Impressively, even in sophisticated conditioning tasks such as blocking and appetitive conditioned inhibition, dopaminergic responses are in line with the predictions of TD learning (Tobler et al., 2003, 2005; Waelti, Dickinson, & Schultz, 2001).”
ResultYin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226
Actions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
Self-citationDopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,117
Gallistel, C. R., & Gibbon, J. (2000) · Psychological Review · doi:10.1037//0033-295x.107.2.289 · Cited by 1,115
Floresco, S. B., West, A. R., Ash, B., Moore, H., & Grace, A. A. (2003) · Nature Neuroscience · doi:10.1038/nn1103 · Cited by 1,106
Animal intelligence; experimental studies
Thorndike, E. L. (1911) · doi:10.5962/bhl.title.55072 · Cited by 1,100
The role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076
The Underpinnings of the BOLD Functional Magnetic Resonance Imaging Signal
Logothetis, N. K. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-10-03963.2003 · Cited by 1,039
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022140919877 · Cited by 1,024
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Neuroeconomics: decision making and the brain
(2014) · Choice Reviews Online · doi:10.5860/choice.51-5702 · Cited by 1,009
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
Reinforcement Learning in Continuous Time and Space
Doya, K. (2000) · Neural Computation · doi:10.1162/089976600300015961 · Cited by 960
The neural basis of functional brain imaging signals
Attwell, D., & Iadecola, C. (2002) · Trends in Neurosciences · doi:10.1016/s0166-2236(02)02264-6 · Cited by 957
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939
Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens
Nicola, S. M., Surmeier, D. J., & Malenka, R. C. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.185 · Cited by 931
Residual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
A cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879
Knutson, B., Fong, G. W., Bennett, S. M., Adams, C. M., & Hommer, D. (2003) · NeuroImage · doi:10.1016/s1053-8119(02)00057-5 · Cited by 848
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli
Mirenowicz, J., & Schultz, W. (1996) · Nature · doi:10.1038/379449a0 · Cited by 829
Salamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817
Animal Intelligence. Experimental Studies
Thorndike, E. L., & Jelliffe (1912) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-191205000-00016 · Cited by 817
Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli
Ungless, M. A., Magill, P. J., & Bolam, J. P. (2004) · Science · doi:10.1126/science.1093360 · Cited by 782
The short-latency dopamine signal: a role in discovering novel actions?
Redgrave, P., & Gurney, K. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn2022 · Cited by 762
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748
Neuronal activity in monkey ventral striatum related to the expectation of reward
Schultz, W., Apicella, P., Scarnati, E., & Ljungberg, T. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04595.1992 · Cited by 741
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
Self-citationNeural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692
Conditioned Reflexes and Neuron Organization.
(1949) · Annals of Internal Medicine · doi:10.7326/0003-4819-31-6-1133_1 · Cited by 688
Instrumental Responding following Reinforcer Devaluation
Adams, C. D., & Dickinson, A. (1981) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748108400816 · Cited by 687
Neuroeconomics: Decision Making and the Brain
Rushworth, M. F. S., & Walton, M. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.005 · Cited by 658
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646
Self-citationNeuroleptic-Induced "Anhedonia" in Rats: Pimozide Blocks Reward Quality of Food
Wise, R. A., Spindler, J., deWit, H., & Gerberg, G. J. (1978) · Science · doi:10.1126/science.566469 · Cited by 636
Amygdala circuitry in attentional and representational processes
Holland, P. C., & Gallagher, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01271-6 · Cited by 618
Reinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618
Variations in the Sensitivity of Instrumental Responding to Reinforcer Devaluation
Adams, C. D. (1982) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748208400878 · Cited by 609
Temporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
A mathematical model for simple learning.
Bush, R. R., & Mosteller, F. (1951) · Psychological Review · doi:10.1037/h0054388 · Cited by 586
Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior
Goto, Y., & Grace, A. A. (2005) · Nature Neuroscience · doi:10.1038/nn1471 · Cited by 576
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
Houk, J. C., Adams, J. L., & Barto, A. G. (1994) · Models of Information Processing in the Basal Ganglia · doi:10.7551/mitpress/4708.003.0020 · Cited by 541
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
Learning and Computational Neuroscience: Foundations of Adaptive Networks
Gabriel, M., & Moore, J. (1990) · Cited by 518
Is the short-latency dopamine response too short to signal reward error?
Redgrave, P., Prescott, T. J., & Gurney, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01373-3 · Cited by 506
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Self-citationTime-Derivative Models of Pavlovian Reinforcement
Sutton, R. S., & Barto, A. G. (1990) · Cited by 499
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495
Dopamine: generalization and bonuses
Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 480
Activity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04219.x · Cited by 447
Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations
DELGADO, M. R., LOCKE, H. M., STENGER, V. A., & FIEZ, J. A. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.1.27 · Cited by 431
Joel, D., & Weiner, I. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90536-3 · Cited by 430
A computational substrate for incentive salience
McClure, S. M., Daw, N. D., & Read Montague, P. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00177-2 · Cited by 422
Christoph, G., Leonzio, R., & Wilcox, K. (1986) · The Journal of Neuroscience · doi:10.1523/jneurosci.06-03-00613.1986 · Cited by 418
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
The misbehavior of value and the discipline of the will
Dayan, P., Niv, Y., Seymour, B., & D. Daw, N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.002 · Cited by 388
Self-citationThe Role of Learning in the Operation of Motivational Systems
Dickinson, A., & Balleine, B. (2002) · Stevens' Handbook of Experimental Psychology · doi:10.1002/0471214426.pas0312 · Cited by 384
Afferents of the ventral tegmental area in the rat-anatomical substratum for integrative functions
Geisler, S., & Zahm, D. S. (2005) · The Journal of Comparative Neurology · doi:10.1002/cne.20668 · Cited by 379
Learning and Sequential Decision Making
Barto, A. G., Sutton, R. S., & Watkins, C. (1989) · Cited by 357
Romo, R., & Schultz, W. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.3.592 · Cited by 353
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346
Dialogues on prediction errors
Niv, Y., & Schoenbaum, G. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.03.006 · Cited by 344
Self-citationTwo Types of Conditioned Reflex and a Pseudo Type
Skinner, B. F. (1935) · The Journal of General Psychology · doi:10.1080/00221309.1935.9920088 · Cited by 341
The effect of the instrumental training contingency on susceptibility to reinforcer devaluation
Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/14640748308400912 · Cited by 338
Linking nucleus accumbens dopamine and blood oxygenation
Knutson, B., & Gibbs, S. E. B. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0686-7 · Cited by 337
Dopaminergic regulation of cerebral cortical microcirculation
Krimer, L. S., Muly, E. C., Williams, G. V., & Goldman-Rakic, P. S. (1998) · Nature Neuroscience · doi:10.1038/1099 · Cited by 327
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326
Inhibition of avoidance behavior.
Rescorla, R. A., & Lolordo, V. M. (1965) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0022060 · Cited by 323
Bee foraging in uncertain environments using predictive hebbian learning
Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J. (1995) · Nature · doi:10.1038/377725a0 · Cited by 311
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300
Reynolds, G. S. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-203 · Cited by 294
Wickens, J. R., Begg, A. J., & Arbuthnott, G. W. (1996) · Neuroscience · doi:10.1016/0306-4522(95)00436-m · Cited by 290
Costa, R. M., Lin, S.-C., Sotnikova, T. D., Cyr, M., Gainetdinov, R. R., Caron, M. G., & Nicolelis, M. A. L. (2006) · Neuron · doi:10.1016/j.neuron.2006.07.030 · Cited by 284
Decoding the neural substrates of reward-related decision making with functional MRI
Hampton, A. N., & O'Doherty, J. P. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0606297104 · Cited by 280
Models of information processing in the basal ganglia
Alexander, G. (1996) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0921-884x(96)89703-7 · Cited by 278
Striatal activity underlies novelty-based choice in humans
Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008) · Neuron · doi:10.1016/j.neuron.2008.04.027 · Cited by 277
Learning Options in Reinforcement Learning
Stolle, M., & Precup, D. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45622-8_16 · Cited by 261
Anatomical aspects of information processing in primate basal ganglia
Parent, A., & Hazrati, L.-N. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90135-9 · Cited by 256
A normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251
Self-citationReward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus
Nakamura, K., Matsumoto, M., & Hikosaka, O. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.0021-08.2008 · Cited by 245
Dopamine Transmission in the Human Striatum during Monetary Reward Tasks
Zald, D. H., Boileau, I., El-Dearedy, W., Gunn, R., McGlone, F., Dichter, G. S., & Dagher, A. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4643-03.2004 · Cited by 238
Conditioned Reflexes and Neuron Organization
(1949) · Journal of the American Medical Association · doi:10.1001/jama.1949.02900320075040 · Cited by 198
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 187
Dopamine and synaptic plasticity in the neostriatum
ARBUTHNOTT, G. W., INGHAM, C. A., & WICKENS, J. R. (2000) · Journal of Anatomy · doi:10.1046/j.1469-7580.2000.19640587.x · Cited by 187
Separate brain regions code for salience vs. valence during reward prediction in humans
Jensen, J., Smith, A. J., Willeit, M., Crawley, A. P., Mikulis, D. J., Vitcu, I., & Kapur, S. (2007) · Human Brain Mapping · doi:10.1002/hbm.20274 · Cited by 178
Using relative novelty to identify useful temporal abstractions in reinforcement learning
Şimşek, Ö., & Barto, A. G. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015353 · Cited by 176
Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals
Montague, P. R., McClure, S. M., Baldwin, P. R., Phillips, P. E. M., Budygin, E. A., Stuber, G. D., Kilpatrick, M. R., & Wightman, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4279-03.2004 · Cited by 176
Major attenuation of food reward with performance-sparing doses of pimozide in the rat.
Wise, R. A., Spindler, J., & Legault, L. (1978) · Canadian Journal of Psychology / Revue canadienne de psychologie · doi:10.1037/h0081678 · Cited by 175
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173
Cellular Models of Reinforcement
Wickens, J., & Kötter, R. (1994) · Models of Information Processing in the Basal Ganglia · doi:10.7551/mitpress/4708.003.0017 · Cited by 165
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153
Self-citationEmotional Cognition: From brain to behaviour
(2002) · Advances in consciousness research · doi:10.1075/aicr.44 · Cited by 149
Menon, M., Jensen, J., Vitcu, I., Graff‐Guerrero, A., Crawley, A. P., Smith, M. A., & Kapur, S. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.10.020 · Cited by 148
Reduction in the effectiveness of reinforcement after prior excitatory conditioning
Rescorla, R. A. (1970) · Learning and Motivation · doi:10.1016/0023-9690(70)90101-3 · Cited by 147
Lecourtier, L., Oranburg, T. D., & Moghaddam, B. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06130.x · Cited by 136
The Rescorla-Wagner model: Losses in associative strength in compound conditioned stimuli.
Kremer, E. F. (1978) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.4.1.22 · Cited by 126
Normative and Descriptive Models of Decision Making: Time Discounting and Risk Sensitivity
Kacelnik, A. (2007) · Novartis Foundation symposium · doi:10.1002/9780470515372.ch5 · Cited by 115
Motivational control of heterogeneous instrumental chains.
Balleine, B. W., Garner, C., González, F., & Dickinson, A. (1995) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.21.3.203 · Cited by 112
Reinforcement learning control
Barto, A. G. (1994) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(94)90138-4 · Cited by 112
Bergstrom, B. P., & Garris, P. A. (2003) · Journal of Neurochemistry · doi:10.1046/j.1471-4159.2003.02104.x · Cited by 111
How fast to work: Response vigor, motivation and tonic dopamine
Niv, Y., Daw, N. D., & Dayan, P. (2005) · MPG.PuRe (Max Planck Society) · Cited by 103
Self-citationGenetic control of instrumental conditioning by striatopallidal neuron–specific S1P receptor Gpr6
Lobo, M. K., Cui, Y., Ostlund, S. B., Balleine, B. W., & Yang, X. W. (2007) · Nature Neuroscience · doi:10.1038/nn1987 · Cited by 102
Takikawa, Y., Kawagoe, R., & Hikosaka, O. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00238.2004 · Cited by 86
Niv, Y., Daw, N. D., & Dayan, P. (2006) · Nature Neuroscience · doi:10.1038/nn0806-987 · Cited by 82
Self-citationReward Prediction Error Computation in the Pedunculopontine Tegmental Nucleus Neurons
Kobayashi, Y., & Okada, K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.003 · Cited by 79
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Cited by 74
Learning, motivation, and emotion
Stevens, S. S., & Gallistel, C. R. (2002) · Wiley eBooks · Cited by 71
Representation of Subjective Value in the Striatum
Knutson, B. D., Delgado, M. R., & Phillips, P. E. M. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00025-7 · Cited by 71
Long-Term Reward Prediction in TD Models of the Dopamine System
Daw, N. D., & Touretzky, D. S. (2002) · Neural Computation · doi:10.1162/089976602760407973 · Cited by 70
Policy Adjustment in a Dynamic Economic Game
Li, J., McClure, S. M., King‐Casas, B., & Montague, P. R. (2006) · PLoS ONE · doi:10.1371/journal.pone.0000103 · Cited by 64
Baird, I., & Leemon, C. (1993) · Defense Technical Information Center (DTIC) · doi:10.21236/ada280862 · Cited by 61
Sequential Decision Problems and Neural Networks
Barto, A. G., Sutton, R. S., & Watkins, C. (1989) · neural information processing systems · Cited by 60
The Misbehaviour of Value and the Discipline of the Will
Dayan, P. (2006) · Cited by 60
Weiner, I., & Joel, D. (2002) · Handbook of experimental pharmacology · doi:10.1007/978-3-662-06765-9_13 · Cited by 37
3. Associative representations of emotionally significant outcomes
Killcross, S., & Blundell-Birtill, P. (2002) · Advances in consciousness research · doi:10.1075/aicr.44.03kil · Cited by 26
Timing and Partial Observability in the Dopamine System
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2002) · Cited by 20
Espresso Reward Learning, Hold the Dopamine: Theoretical Comment on Robinson et al. (2005).
Berridge, K. C. (2005) · Behavioral Neuroscience · doi:10.1037/0735-7044.119.1.336 · Cited by 19
THE EFFECTS OF MOTIVATION ON HABITUAL INSTRUMENTAL BEHAVIOR
Niv, Y. (2007) · Cited by 11
Self-citation
Cited by
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
The mysterious motivational functions of mesolimbic dopamine
Salamone, J. D., & Correa, M. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.021 · Cited by 1,418
An Introduction to Deep Reinforcement Learning
Vincent, F.-L., Peter, H., Riashat, I., Marc, G. B., & Joelle, P. (2018) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000071 · Cited by 1,286
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Motivation and cognitive control: from behavior to neural mechanism
Botvinick, M., & Braver, T. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015044 · Cited by 909
Metadata
- Authors
- Niv, Y.
- Venue
- Journal of Mathematical Psychology
- Publisher
- Elsevier BV
- Published
- 2009-06
- Volume
- 53
- Issue
- 3
- Pages
- 139-154
- ISSN
- 0022-2496 (ISSN-L)
- OpenAlex
- W2081030963
- Cited by
- 717
- References
- 180
- FWCI
- 7.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.