The DOI matches its Crossref record; 165 of the 165 references it cites were found in a registry or an index.
Reinforcement learning: The Good, The Bad and The Ugly
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationLearning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Regret Theory: An Alternative Theory of Rational Choice Under Uncertainty
Loomes, G., & Sugden, R. (1982) · The Economic Journal · doi:10.2307/2232669 · Cited by 4,042
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764
Behavioral Game Theory: Experiments in Strategic Interaction
Camerer, C. F. (2003) · Cited by 3,722
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089
Regret in Decision Making under Uncertainty
Bell, D. E. (1982) · Operations Research · doi:10.1287/opre.30.5.961 · Cited by 2,977
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationAnomalies in Intertemporal Choice: Evidence and an Interpretation
Loewenstein, G., & Prelec, D. (1992) · The Quarterly Journal of Economics · doi:10.2307/2118482 · Cited by 2,305
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Self-citationLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Abstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
The neural correlates of subjective value during intertemporal choice
Kable, J. W., & Glimcher, P. W. (2007) · Nature Neuroscience · doi:10.1038/nn2007 · Cited by 1,916
Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.700 · Cited by 1,866
Context, ambiguity, and unlearning: sources of relapse after behavioral extinction
Bouton, M. E. (2002) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01546-9 · Cited by 1,662
Stevens' Handbook of Experimental Psychology
(2002) · Wiley eBooks · doi:10.1002/0471214426 · Cited by 1,632
Aron, A. R., & Poldrack, R. A. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4682-05.2006 · Cited by 1,630
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610
Frames, Biases, and Rational Decision-Making in the Human Brain
De Martino, B., Kumaran, D., Seymour, B., & Dolan, R. J. (2006) · Science · doi:10.1126/science.1128356 · Cited by 1,590
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Logothetis, N. K., & Wandell, B. A. (2004) · Annual Review of Physiology · doi:10.1146/annurev.physiol.66.082602.092845 · Cited by 1,568
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationOn-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
The Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390
A two-dimensional neuropsychology of defense: fear/anxiety and defensive distance
McNaughton, N., & Corr, P. J. (2004) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2004.03.005 · Cited by 1,308
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Behavioral game theory, experiments in strategic interaction
Canté, F. (2004) · Cuadernos de Economía · Cited by 1,265
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
The Neural Basis of Financial Risk Taking
Kuhnen, C. M., & Knutson, B. (2005) · Neuron · doi:10.1016/j.neuron.2005.08.008 · Cited by 1,182
Breland, K., & Breland, M. (1961) · American Psychologist · doi:10.1037/h0040090 · Cited by 1,181
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 4×
“It also inspired a new set of modeling investigations into free operant choice tasks, in which subjects are free to execute actions at times of their own choosing, and the dependent variables are quantities such as the rates of responding [70,71].”
BackgroundInfluentialSelf-citationCohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Hold Your Horses: Impulsivity, Deep Brain Stimulation, and Medication in Parkinsonism
Frank, M. J., Samanta, J., Moustafa, A. A., & Sherman, S. J. (2007) · Science · doi:10.1126/science.1146157 · Cited by 1,120
Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116
O'Reilly, R. C., & Frank, M. J. (2006) · Neural Computation · doi:10.1162/089976606775093909 · Cited by 1,066
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022140919877 · Cited by 1,024
Diminishing reciprocal fairness by disrupting the right prefrontal cortex
Knoch, D., Pascual-Leone, A., Meyer, K., Treyer, V., & Fehr, E. (2006) · Science · doi:10.1126/science.1129156 · Cited by 1,024
Do rats have a prefrontal cortex?
Uylings, H. B. M., Groenewegen, H. J., & Kolb, B. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.028 · Cited by 1,013
Neuroeconomics: decision making and the brain
(2014) · Choice Reviews Online · doi:10.5860/choice.51-5702 · Cited by 1,009
Reconciling the role of central serotonin neurons in human and animal behavior
Soubrié, P. (1986) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00022871 · Cited by 996
Role of 5-HT in stress, anxiety, and depression
Graeff, F. G., Guimarães, F. S., De Andrade, T. G. C. S., & Deakin, J. F. W. (1996) · Pharmacology Biochemistry and Behavior · doi:10.1016/0091-3057(95)02135-3 · Cited by 984
Aron, A. R., Behrens, T. E., Smith, S., Frank, M. J., & Poldrack, R. A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0519-07.2007 · Cited by 982
Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976
Time Discounting for Primary Rewards
McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4246-06.2007 · Cited by 971
Ainslie, G. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139164191 · Cited by 955
When fear is near: threat imminence elicits prefrontal periaqueductal gray shifts in humans.
Mobbs, D., Petrovic, P., Marchant, J. L., Hassabis, D., Weiskopf, N., Seymour, B., Dolan, R. J., & Frith, C. D. (2007) · Science · doi:10.1126/science.1144298 · Cited by 951
The primate amygdala represents the positive and negative value of visual stimuli during learning
Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman, C. D. (2006) · Nature · doi:10.1038/nature04490 · Cited by 917
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844
Self-citationSalamone, J. D., & Correa, M. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00282-6 · Cited by 817
Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli
Ungless, M. A., Magill, P. J., & Bolam, J. P. (2004) · Science · doi:10.1126/science.1093360 · Cited by 782
Social neuroeconomics: the neural circuitry of social preferences
Fehr, E., & Camerer, C. F. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.09.002 · Cited by 779
Toward a neurobiology of temporal cognition: advances and challenges
Gibbon, J., Malapani, C., Dale, C. L., & Gallistel, C. R. (1997) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(97)80005-0 · Cited by 762
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710
Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making
Frank, M. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.006 · Cited by 706
Belin, D., & Everitt, B. J. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.019 · Cited by 697
Neural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706111104 · Cited by 683
Regret and its avoidance: a neuroimaging study of choice behavior
Coricelli, G., Critchley, H. D., Joffily, M., O'Doherty, J. P., Sirigu, A., & Dolan, R. J. (2005) · Nature Neuroscience · doi:10.1038/nn1514 · Cited by 671
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
How the Brain Translates Money into Force: A Neuroimaging Study of Subliminal Motivation
Pessiglione, M., Schmidt, L., Draganski, B., Kalisch, R., Lau, H., Dolan, R. J., & Frith, C. D. (2007) · Science · doi:10.1126/science.1140459 · Cited by 634
Ethoexperimental Approaches to the Biology of Emotion
Blanchard, D. C., & Blanchard, R. J. (1988) · Annual Review of Psychology · doi:10.1146/annurev.ps.39.020188.000355 · Cited by 624
Amygdala circuitry in attentional and representational processes
Holland, P. C., & Gallagher, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01271-6 · Cited by 618
Joel, D., & Weiner, I. (2000) · Neuroscience · doi:10.1016/s0306-4522(99)00575-8 · Cited by 614
Temporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605
Self-citationRoesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation
Mazzoni, P., Hristova, A., & Krakauer, J. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0264-07.2007 · Cited by 557
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525
Model‐Based fMRI and Its Application to Reward Learning and Decision Making
O'DOHERTY, J. P., HAMPTON, A., & KIM, H. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.022 · Cited by 508
Midbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503
Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA
Bunzeck, N., & Düzel, E. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.021 · Cited by 503
Walton, M. E., Bannerman, D. M., Alterescu, K., & Rushworth, M. F. S. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-16-06475.2003 · Cited by 496
Firing modes of midbrain dopamine cells in the freely moving rat
Hyland, B. I., Reynolds, J. N. J., Hay, J., Perk, C. G., & Miller, R. (2002) · Neuroscience · doi:10.1016/s0306-4522(02)00267-1 · Cited by 492
Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics
Glimcher, P. W. (2003) · The MIT Press eBooks · doi:10.7551/mitpress/2302.001.0001 · Cited by 481
Dopamine: generalization and bonuses
Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 480
Self-citationReward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 465
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 462
Self-citationKim, H., Shimojo, S., & O'Doherty, J. P. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040233 · Cited by 453
Adaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · Models of Information Processing in the Basal Ganglia · doi:10.7551/mitpress/4708.003.0018 · Cited by 446
Medial prefrontal cell activity signaling prediction errors of action values
Matsumoto, M., Matsumoto, K., Abe, H., & Tanaka, K. (2007) · Nature Neuroscience · doi:10.1038/nn1890 · Cited by 437
Pan, W.-X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1478-05.2005 · Cited by 429
Genetically Determined Differences in Learning from Errors
Klein, T. A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D. Y., & Ullsperger, M. (2007) · Science · doi:10.1126/science.1145044 · Cited by 426
If multi-agent learning is the answer, what is the question?
Shoham, Y., Powers, R., & Grenager, T. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2006.02.006 · Cited by 410
Reynolds, S. M., & Berridge, K. C. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-16-07308.2002 · Cited by 403
Processing of Temporal Unpredictability in Human and Animal Amygdala
Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., Lüthi, A., & Seifritz, E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5218-06.2007 · Cited by 401
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
The misbehavior of value and the discipline of the will
Dayan, P., Niv, Y., Seymour, B., & D. Daw, N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.002 · Cited by 388
Self-citationBasal ganglia orient eyes to reward
Hikosaka, O., Nakamura, K., & Nakahara, H. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00458.2005 · Cited by 377
A Framework for Sequential Planning in Multi-Agent Settings
Gmytrasiewicz, P. J., & Doshi, P. (2005) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1579 · Cited by 360
Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala
Belova, M. A., Paton, J. J., Morrison, S. E., & Salzman, C. D. (2007) · Neuron · doi:10.1016/j.neuron.2007.08.004 · Cited by 349
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326
O'Doherty, J. P., Buchanan, T. W., Seymour, B., & Dolan, R. J. (2006) · Neuron · doi:10.1016/j.neuron.2005.11.014 · Cited by 322
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1025696116075 · Cited by 317
Decision theory: what “should” the nervous system do?
Körding, K. (2007) · Science · doi:10.1126/science.1142998 · Cited by 313
Human Pavlovian–Instrumental Transfer
Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4028-07.2008 · Cited by 301
Self-citationHaruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., Imamizu, H., & Kawato, M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3417-03.2004 · Cited by 300 · cited 1×
“However, a far more direct link between model-free RL and the workings of affective decision-making is apparent in the findings that the phasic activity of dopamine neurons during appetitive conditioning (and indeed the fMRI BOLD signal in the ventral striatum of humans, a key target of dopamine projections) has many…”
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300
Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation
Roesch, M. R., Taylor, A. R., & Schoenbaum, G. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.027 · Cited by 297
Mental models and counterfactual thoughts about what might have been
Byrne, R. M. J. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(02)01974-5 · Cited by 282
Reynolds, S. M., & Berridge, K. C. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-09-03261.2001 · Cited by 281
Decoding the neural substrates of reward-related decision making with functional MRI
Hampton, A. N., & O'Doherty, J. P. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0606297104 · Cited by 280
Neural signature of fictive learning signals in a sequential investment task
Lohrenz, T., McCabe, K., Camerer, C. F., & Montague, P. R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0608842104 · Cited by 265
Serotonin, inhibition, and negative mood
Dayan, P., & Huys, Q. J. M. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040004 · Cited by 263
Self-citationAnticipation of novelty recruits reward system and hippocampus while promoting recollection
Wittmann, B. C., Bunzeck, N., Dolan, R. J., & Düzel, E. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.06.038 · Cited by 262
Low-Serotonin Levels Increase Delayed Reward Discounting in Humans
Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S. C., Yamawaki, S., & Doya, K. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4982-07.2008 · Cited by 260
Risk-Sensitive Reinforcement Learning
Mihatsch, O., & Neuneier, R. (2002) · Machine Learning · doi:10.1023/a:1017940631555 · Cited by 258
Integrating hippocampus and striatum in decision-making
Johnson, A., van der Meer, M. A., & Redish, A. D. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.01.003 · Cited by 253
A normative perspective on motivation
NIV, Y., JOEL, D., & DAYAN, P. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.06.010 · Cited by 251
Self-citationSeparate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241
Chiu, P. H., Kayali, M. A., Kishida, K. T., Tomlin, D., Klinger, L. G., Klinger, M. R., & Montague, P. R. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.020 · Cited by 210
Human neural learning depends on reward prediction errors in the blocking paradigm.
Tobler, P. N., O’Doherty, J. P., Dolan, R. J., & Schultz, W. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00762.2005 · Cited by 209
Resolution of Uncertainty in Prefrontal Cortex
Yoshida, W., & Ishii, S. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.006 · Cited by 197
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 187
Tanaka, S. C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001333 · Cited by 184
Neural coding of distinct statistical properties of reward information in humans
Dreher, J.-C., Kohn, P., & Berman, K. F. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj004 · Cited by 180
Role of Dopamine in the Primate Caudate Nucleus in Reward Modulation of Saccades
Nakamura, K., & Hikosaka, O. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4853-05.2006 · Cited by 178
Separate brain regions code for salience vs. valence during reward prediction in humans
Jensen, J., Smith, A. J., Willeit, M., Crawley, A. P., Mikulis, D. J., Vitcu, I., & Kapur, S. (2007) · Human Brain Mapping · doi:10.1002/hbm.20274 · Cited by 178
Action and Outcome Encoding in the Primate Caudate Nucleus
Lau, B., & Glimcher, P. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3060-07.2007 · Cited by 177
Dynamic Gain Control of Dopamine Delivery in Freely Moving Animals
Montague, P. R., McClure, S. M., Baldwin, P. R., Phillips, P. E. M., Budygin, E. A., Stuber, G. D., Kilpatrick, M. R., & Wightman, R. M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.4279-03.2004 · Cited by 176
Niv, Y., Joel, D., Meilijson, I., & Ruppin, E. (2002) · Adaptive Behavior · doi:10.1177/10597123020101001 · Cited by 166
Self-citationPVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.
O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007) · Behavioral Neuroscience · doi:10.1037/0735-7044.121.1.31 · Cited by 160
The Role of the Amygdala in the Extinction of Conditioned Fear
Barad, M., Gean, P.-W., & Lutz, B. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.05.029 · Cited by 160
Basal Ganglia Mechanisms of Reward‐Oriented Eye Movement
HIKOSAKA, O. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.012 · Cited by 158
Schweimer, J. V., & Hauber, W. (2005) · Learning & Memory · doi:10.1101/lm.90605 · Cited by 156
The human amygdala and orbital prefrontal cortex in behavioural regulation
Dolan, R. J. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2088 · Cited by 150
Menon, M., Jensen, J., Vitcu, I., Graff‐Guerrero, A., Crawley, A. P., Smith, M. A., & Kapur, S. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.10.020 · Cited by 148
Lecourtier, L., Oranburg, T. D., & Moghaddam, B. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06130.x · Cited by 137
Adding prediction risk to the theory of reward learning
PREUSCHOFF, K., & BOSSAERTS, P. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.005 · Cited by 133
Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops
Samejima, K., & Doya, K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.024 · Cited by 130
The role of the medial prefrontal cortex in achieving goals
Matsumoto, K., & Tanaka, K. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.03.005 · Cited by 123
Orbitofrontal Cortex and the Computation of Economic Value
Padoa‐Schioppa, C. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.011 · Cited by 110
Murschall, A., & Hauber, W. (2006) · Learning & Memory · doi:10.1101/lm.127106 · Cited by 101
Neural basis of quasi-rational decision making
Lee, D. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.02.001 · Cited by 98
A direct projection from superior colliculus to substantia nigra pars compacta in the cat
McHaffie, J. G., Jiang, H., May, P. J., Coizet, V., Overton, P. G., Stein, B. E., & Redgrave, P. (2005) · Neuroscience · doi:10.1016/j.neuroscience.2005.11.015 · Cited by 95
Coizet, V., Dommett, E. J., Redgrave, P., & Overton, P. G. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2006.01.030 · Cited by 92
Moutoussis, M., Williams, J., Dayan, P., & Bentall, R. P. (2007) · Cognitive Neuropsychiatry · doi:10.1080/13546800701566686 · Cited by 91
Self-citationBrain mechanism of reward prediction under predictable and unpredictable environmental dynamics
Tanaka, S., Samejima, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.039 · Cited by 89
Learning-Related Human Brain Activations Reflecting Individual Finances
Tobler, P. N., Fletcher, P. C., Bullmore, E. T., & Schultz, W. (2007) · Neuron · doi:10.1016/j.neuron.2007.03.004 · Cited by 88
Holland, P. C., & Gallagher, M. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.0390-06.2006 · Cited by 88
Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints
Schweighofer, N., Shishida, K., Han, C. E., Okamoto, Y., Tanaka, S., Yamawaki, S., & Doya, K. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020152 · Cited by 87
Why Choose This Book?: How We Make Decisions
Montague, P. R. (2006) · Medical Entomology and Zoology · Cited by 84
Role of Substantia Nigra–Amygdala Connections in Surprise-Induced Enhancement of Attention
Lee, H. J., Youn, J. M., O, M. J., Gallagher, M., & Holland, P. C. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.1316-06.2006 · Cited by 84
El‐Amamy, H., & Holland, P. C. (2007) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05402.x · Cited by 80
Theoretical and Empirical Studies of Learning
Niv, Y., & Montague, P. R. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374176-9.00022-1 · Cited by 73
Self-citationPolicy Adjustment in a Dynamic Economic Game
Li, J., McClure, S. M., King‐Casas, B., & Montague, P. R. (2006) · PLoS ONE · doi:10.1371/journal.pone.0000103 · Cited by 64
Walton, M. E., Croxson, P. L., Rushworth, M. F. S., & Bannerman, D. M. (2005) · Behavioral Neuroscience · doi:10.1037/0735-7044.119.1.323 · Cited by 60
The Misbehaviour of Value and the Discipline of the Will
Dayan, P. (2006) · Cited by 60
Self-citationFlexible Neural Representations of Value in the Primate Brain
Salzman, C. D., Paton, J. J., Belova, M. A., & Morrison, S. E. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.034 · Cited by 58
Furuyashiki, T., Holland, P. C., & Gallagher, M. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.0319-08.2008 · Cited by 58
Neuroeconomics: a view from neuroscience.
Montague, P. R. (2007) · PubMed · Cited by 44
Simmons, J. M., Ravel, S., Shidara, M., & Richmond, B. J. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.028 · Cited by 35
Effects of novelty on activity of lateral and medial prefrontal neurons
Matsumoto, M., Matsumoto, K., & Tanaka, K. (2006) · Neuroscience Research · doi:10.1016/j.neures.2006.10.017 · Cited by 35
Shidara, M., & Richmond, B. J. (2004) · Neuroscience Research · doi:10.1016/j.neures.2004.03.008 · Cited by 21
Cited by
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
The neural bases of emotion regulation
Etkin, A., Büchel, C., & Gross, J. J. (2015) · Nature Reviews Neuroscience · doi:10.1038/nrn4044 · Cited by 1,244
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
Emotion and the prefrontal cortex: an integrative review
Dixon, M. L., Thiruchselvam, R., Todd, R., & Christoff, K. (2017) · Psychological Bulletin · doi:10.1037/bul0000096 · Cited by 745
Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli
Lammel, S., Ion, D. I., Roeper, J., & Malenka, R. C. (2011) · Neuron · doi:10.1016/j.neuron.2011.03.025 · Cited by 766
Metadata
- Authors
- Dayan, P., & Niv, Y.
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2008-04
- Volume
- 18
- Issue
- 2
- Pages
- 185-196
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W2133993682
- Cited by
- 704
- References
- 165
- FWCI
- 5.92
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.