Modulators of decision making
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Human and animal decisions are modulated by a variety of environmental and intrinsic contexts. Here I consider computational factors that can affect decision making and review anatomical structures and neurochemical systems that are related to contextual modulation of decision making. Expectation of a high reward can motivate a subject to go for an action despite a large cost, a decision that is influenced by dopamine in the anterior cingulate cortex. Uncertainty of action outcomes can promote risk taking and exploratory choices, in which norepinephrine and the orbitofrontal cortex appear to be involved. Predictable environments should facilitate consideration of longer-delayed rewards, which depends on serotonin in the dorsal striatum and dorsal prefrontal cortex. This article aims to sort out factors that affect the process of decision making from the viewpoint of reinforcement learning theory and to bridge between such computational needs and their neurophysiological substrates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, structures such as the basal ganglia, parietal cortex, and thalamus support procedural learning, visuospatial coordination, and sensory integration (Corbetta and Shulman, 2002; Doya, 2008; Sherman, 2016).”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
In the citing paper
“Basal ganglia enable procedural learning, habit formation, and reinforcement-driven behaviors (Doya, 2008), i.e., reinforcement-based decision-making, allowing operators to execute complex behaviors automatically under pressure.”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Neural recording experiments in animals reveal that midbrain dopamine neurons encode reward prediction error”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122 · cited 1×
“In many animal and human choice experiments, subjects tend to be more sensitive to differences in shorter delays than longer delays, which is well modeled by hyperbolic function”
BackgroundMarkov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · cited 1×
“Norepinephrine is also suggested to be important in regulating the decision to explore alternatives versus exploiting a known resourc”
BackgroundTheory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“Also, functional magnetic resonance imaging (fMRI) experiments using a monetary questionnair”
MethodBarto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 1×
“Uncertainty in prediction can also favor one learning framework over another—for example, model-free temporal difference learning and model-based plannin”
BackgroundRELATIVE AND ABSOLUTE STRENGTH OF RESPONSE AS A FUNCTION OF FREQUENCY OF REINFORCEMENT 1, 2
Herrnstein, R. J. (1961) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1961.4-267 · Cited by 2,281 · cited 1×
“In animal behavior studies, a well known principle is the matching la”
BackgroundCortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“Neural recording experiments in animals reveal that midbrain dopamine neurons encode reward prediction error”
BackgroundLearning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
(2020) · Machine Learning and Visual Perception · doi:10.1515/9783110595567-011 · Cited by 1,930
The neural correlates of subjective value during intertemporal choice
Kable, J. W., & Glimcher, P. W. (2007) · Nature Neuroscience · doi:10.1038/nn2007 · Cited by 1,916 · cited 1×
“However, another study found these areas to be activated by the subjective value after discounting, irrespective of the delay of the rewar”
BackgroundUncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774
Neural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
The Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390 · cited 1×
“Flattening of the utility curve, for instance by satiety, is called devaluation and is a useful tool for assessing the mechanism of valuatio”
MethodSUBJECTIVE PROBABILITY AND DELAY
Rachlin, H., Raineri, A., & Cross, D. (1991) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1991.55-233 · Cited by 1,341
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292 · cited 1×
“On the other hand, a recent recording study has clearly demonstrated that neurons in the lateral habenula respond to no-reward predictive cues as well as reward omission, in exactly the opposite way as dopamine neurons, and that stimulation of the lateral habenula causes inhibition of dopamine neuron”
BackgroundThe Neural Basis of Financial Risk Taking
Kuhnen, C. M., & Knutson, B. (2005) · Neuron · doi:10.1016/j.neuron.2005.08.008 · Cited by 1,182 · cited 1×
“How is variance or uncertainty of reward represented? Imaging studies show activity in the anterior insul”
BackgroundTonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“In deciding between actions with less-than-average rewards, an action with smaller reward in shorter time can be more appropriate because it allows moving on to the next action earlier, resulting in an apparently impulsive choic”
BackgroundRogers, R. (1999) · Neuropsychopharmacology · doi:10.1016/s0893-133x(98)00091-8 · Cited by 1,101
Matching Behavior and the Representation of Value in the Parietal Cortex
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004) · Science · doi:10.1126/science.1094765 · Cited by 1,030 · cited 1×
“In animal behavior studies, a well known principle is the matching la”
BackgroundTime Discounting for Primary Rewards
McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4246-06.2007 · Cited by 971
Complementary roles of basal ganglia and cerebellum in learning and motor control
Doya, K. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00153-7 · Cited by 964
Self-citationOptimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948
Reward processing in primate orbitofrontal cortex and basal ganglia
Schultz, W. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.272 · Cited by 942
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939
Self-citationImpulsive choice induced in rats by lesions of the nucleus accumbens core
Cardinal, R. N., Pennicott, D. R., Lakmali, C., Sugathapala, Robbins, T. W., & Everitt, B. J. (2001) · Science · doi:10.1126/science.1060818 · Cited by 923
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
Opponent interactions between serotonin and dopamine
Daw, N. D., Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00052-7 · Cited by 844 · cited 1×
“A hypothetical medium of such aversive prediction is serotonergic neuron”
BackgroundNetwork reset: a simplified overarching theory of locus coeruleus noradrenaline function
Bouret, S., & Sara, S. J. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.09.002 · Cited by 809 · cited 1×
“Norepinephrine is implicated in such 'resets' of ongoing activitie”
BackgroundPrediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004) · Nature Neuroscience · doi:10.1038/nn1279 · Cited by 779 · cited 1×
“In functional brain imaging studies using a game in a dynamic environment, the dorsolateral prefrontal cortex, dorsal premotor cortex, parietal cortex and insula are more activated in conditions requiring long-term prediction of rewards rather than in conditions requiring short-term prediction”
BackgroundSelf-citationMetalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704 · cited 1×
“The network is affected by the sensory and contextual information represented in the cortex, as well as in diffuse neurochemical systems, such as serotonin, norepinephrine and acetylcholin”
BackgroundSelf-citationReward expectancy in primate prefrental neurons
Watanabe, M. (1996) · Nature · doi:10.1038/382629a0 · Cited by 684
Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice
Winstanley, C. A., Theobald, D. E. H., Cardinal, R. N., & Robbins, T. W. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.5606-03.2004 · Cited by 675
Cognitive Inflexibility After Prefrontal Serotonin Depletion
Clarke, H. F., Dalley, J. W., Crofts, H. S., Robbins, T. W., & Roberts, A. C. (2004) · Science · doi:10.1126/science.1094987 · Cited by 632 · cited 1×
“Deficits in serotonin, especially in the medial prefrontal cortex, disturb adaptation to changes in the required action for a given cue (reversal learning) by making the subjects more likely to stick to prelearned behavior”
BackgroundExpectation of reward modulates cognitive signals in the basal ganglia
Kawagoe, R., Takikawa, Y., & Hikosaka, O. (1998) · Nature Neuroscience · doi:10.1038/1625 · Cited by 631
Parallel incentive processing: an integrated view of amygdala function
Balleine, B. W., & Killcross, S. (2006) · Trends in Neurosciences · doi:10.1016/j.tins.2006.03.002 · Cited by 622 · cited 1×
“Flattening of the utility curve, for instance by satiety, is called devaluation and is a useful tool for assessing the mechanism of valuatio”
MethodTemporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605
Intertemporal choice – toward an integrative framework
Berns, G. S., Laibson, D., & Loewenstein, G. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.08.011 · Cited by 580 · cited 1×
“In many animal and human choice experiments, subjects tend to be more sensitive to differences in shorter delays than longer delays, which is well modeled by hyperbolic function”
BackgroundSeparate neural pathways process different decision costs
Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1756 · Cited by 580 · cited 1×
“The rat T-maze experiments also found regional dissociation of ACC and OFC in effort and delay discountin”
BackgroundNeurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans
Chamberlain, S. R., Müller, U., Blackwell, A. D., Clark, L., Robbins, T. W., & Sahakian, B. J. (2006) · Science · doi:10.1126/science.1121218 · Cited by 573
Hariri, A. R., Brown, S. M., Williamson, D. E., Flory, J. D., de Wit, H., & Manuck, S. B. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.3446-06.2006 · Cited by 557
Yacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537 · cited 1×
“Human brain imaging shows response of the amygdala to expectation of losses as opposed to gain”
BackgroundDopamine-dependent plasticity of corticostriatal synapses
Reynolds, J. N. J., & Wickens, J. R. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00045-x · Cited by 535 · cited 1×
“Dopamine-dependent plasticity in the striatum seems to be important in learning of reward-predictive neural activitie”
BackgroundReward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems
Tobler, P. N., O'Doherty, J. P., Dolan, R. J., & Schultz, W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00745.2006 · Cited by 472
Differential Encoding of Losses and Gains in the Human Striatum
Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0400-07.2007 · Cited by 462 · cited 1×
“Human functional brain imaging reveals that different parts in the striatum respond to gains and losse”
BackgroundDopaminergic and Glutamatergic Regulation of Effort- and Delay-Based Decision Making
Floresco, S. B., Tse, M. T. L., & Ghods-Sharifi, S. (2008) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301565 · Cited by 435
The role of prelimbic cortex in instrumental conditioning
Corbit, L. H., & Balleine, B. W. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.023 · Cited by 423
S., M., S., B., M.-Y., H., C., B., E., S., J., D., & I., A. (2002) · Psychopharmacology · doi:10.1007/s00213-001-0983-0 · Cited by 409
Neuronal correlates of goal-based motor selection in the prefrontal cortex.
Matsumoto, K., Suzuki, W., & Tanaka, K. (2003) · Science · doi:10.1126/science.1084204 · Cited by 384
Winstanley, C. A., Dalley, J. W., Theobald, D. E., & Robbins, T. W. (2004) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300434 · Cited by 371
The role of rat medial frontal cortex in effort-based decision making
Walton, M. E., Bannerman, D. M., & Rushworth, M. F. S. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-24-10996.2002 · Cited by 364
Robinson, E. S. J., Eagle, D. M., Mar, A. C., Bari, A., Banerjee, G., Jiang, X., Dalley, J. W., & Robbins, T. W. (2008) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301487 · Cited by 359
Clarke, H., Walker, S., Dalley, J., Robbins, T., & Roberts, A. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhj120 · Cited by 351
Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala
Belova, M. A., Paton, J. J., Morrison, S. E., & Salzman, C. D. (2007) · Neuron · doi:10.1016/j.neuron.2007.08.004 · Cited by 349 · cited 1×
“However, a neural recording study in nonhuman primates showed that neurons in the amygdala respond to reward expectation as wel”
BackgroundCritical Involvement of Dopaminergic Neurotransmission in Impulsive Decision Making
van Gaalen, M. M., van Koten, R., Schoffelmeer, A. N. M., & Vanderschuren, L. J. M. J. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.06.005 · Cited by 338
Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits
BALLEINE, B. (2005) · Physiology & Behavior · doi:10.1016/j.physbeh.2005.08.061 · Cited by 326
A Role for the Macaque Anterior Cingulate Gyrus in Social Valuation
Rudebeck, P. H., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2006) · Science · doi:10.1126/science.1128197 · Cited by 312
Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F. S., & Bannerman, D. M. (2005) · Psychopharmacology · doi:10.1007/s00213-004-2059-4 · Cited by 302 · cited 1×
“Choice of a larger reward with a larger effort is impaired by a dopamine D2 receptor antagonis”
BackgroundPhasic norepinephrine: A neural interrupt signal for unexpected events
Dayan, P., & Yu, A. J. (2006) · Network: Computation in Neural Systems · doi:10.1080/09548980601004024 · Cited by 299 · cited 1×
“Norepinephrine is implicated in such 'resets' of ongoing activitie”
BackgroundCorrelated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298 · cited 1×
“However, because of their low spontaneous firing rate, their inhibitory response to prediction of no reward or omission of reward is wea”
BackgroundWickens, J. R., Begg, A. J., & Arbuthnott, G. W. (1996) · Neuroscience · doi:10.1016/0306-4522(95)00436-m · Cited by 290 · cited 1×
“Dopamine-dependent plasticity in the striatum seems to be important in learning of reward-predictive neural activitie”
BackgroundRogers, R. D., Blackshaw, A. J., Middleton, H. C., Matthews, K., Hawtin, K., Crowley, C., Hopwood, A., Wallace, C., Deakin, J. F. W., Sahakian, B. J., et al. (1999) · Psychopharmacology · doi:10.1007/pl00005494 · Cited by 289
Winstanley, C. A., Theobald, D. E. H., Dalley, J. W., Cardinal, R. N., & Robbins, T. W. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhi088 · Cited by 278 · cited 1×
“How are these regional specializations in evaluation of immediate and delayed rewards related to the serotonergic system? Serotonin efflux increases in medial prefrontal cortex while rats perform a delay discounting tas”
BackgroundNeural systems implicated in delayed and probabilistic reinforcement
Cardinal, R. N. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.03.004 · Cited by 248 · cited 1×
“However, the results of lesion and pharmacology studies are by no means simple, possibly because of autoregulatory feedback mechanisms in the serotonergic system (see ref”
BackgroundOPTIMIZATION AND THE MATCHING LAW AS ACCOUNTS OF INSTRUMENTAL BEHAVIOR
Baum, W. M. (1981) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1981.36-387 · Cited by 245 · cited 1×
“, in which an action is selected in proportion to its value: This is a nearly optimal strategy in 'baited' tasks, in which a reward becomes available at a given location with a certain probability and will stay there until it is take”
BackgroundCardinal, R. N., & Howes, N. J. (2005) · BMC Neuroscience · doi:10.1186/1471-2202-6-37 · Cited by 232
Ho, M.-Y., Mobini, S., Chiang, T.-J., Bradshaw, C. M., & Szabadi, E. (1999) · Psychopharmacology · doi:10.1007/pl00005482 · Cited by 213 · cited 1×
“Although there is no general guarantee that the three factors are independent, it is often assumed they work multiplicativel”
BackgroundControl of exploitation–exploration meta-parameter in reinforcement learning
Ishii, S., Yoshida, W., & Yoshimoto, J. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00056-4 · Cited by 204 · cited 1×
“Another common solution is 'Boltzmann selection' in which selection probabilities p are proportional to the exponentials of the estimated values: By an analogy with thermodynamic”
BackgroundTanaka, S. C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001333 · Cited by 184 · cited 1×
“Acute tryptophan depletion and loading has shown that activation of the ventral striatum for immediate reward prediction is enhanced during a low-serotonin condition, whereas activation of the dorsal striatum for delayed reward prediction is enhanced during a high-serotonin conditio”
BackgroundSelf-citationPothuizen, H. H. J., Jongen‐Rêlo, A. L., Feldon, J., & Yee, B. K. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04388.x · Cited by 183
Reinforcement learning: Computational theory and biological mechanisms
Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246 · Cited by 150 · cited 1×
“In the following sections, I first review theoretical models of the other three steps—evaluation, action selection and learning—based on the framework of reinforcement learnin”
BackgroundSelf-citationShaping of Motor Responses by Incentive Values through the Basal Ganglia
Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., Gross, C. E., & Boraud, T. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3745-06.2007 · Cited by 146
Complementary Process to Response Bias in the Centromedian Nucleus of the Thalamus
Minamimoto, T., Hori, Y., & Kimura, M. (2005) · Science · doi:10.1126/science.1109154 · Cited by 133
Kheramin, S., Body, S., Mobini, S., Ho, M.-Y., zquez-Martinez, D. V. X. E., Bradshaw, C. M., Szabadi, E., Deakin, J. F. W., & Anderson, I. M. (2002) · Psychopharmacology · doi:10.1007/s00213-002-1228-6 · Cited by 126
Auditory thalamus integrates visual inputs into behavioral gains
Komura, Y., Tamura, R., Uwano, T., Nishijo, H., & Ono, T. (2005) · Nature Neuroscience · doi:10.1038/nn1528 · Cited by 123
Kheramin, S., Body, S., Ho, M.-Y., Vel�zquez-Martinez, D. N., Bradshaw, C. M., Szabadi, E., Deakin, J. F. W., & Anderson, I. M. (2004) · Psychopharmacology · doi:10.1007/s00213-004-1813-y · Cited by 117
Effects of beta-adrenoceptor blockade on components of human decision-making
Rogers, R. D., Lancaster, M., Wakeley, J., & Bhagwagar, Z. (2004) · Psychopharmacology · doi:10.1007/s00213-003-1641-5 · Cited by 108
Acquisition and extinction in autoshaping.
Kakade, S., & Dayan, P. (2002) · Psychological Review · doi:10.1037//0033-295x.109.3.533 · Cited by 102 · cited 1×
“Unexpected changes in the environment and the degree of the subject's knowledge also affect the optimal degree of exploration and memory updatin”
BackgroundBrain mechanism of reward prediction under predictable and unpredictable environmental dynamics
Tanaka, S., Samejima, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.039 · Cited by 89 · cited 1×
“In functional brain imaging studies using a game in a dynamic environment, the dorsolateral prefrontal cortex, dorsal premotor cortex, parietal cortex and insula are more activated in conditions requiring long-term prediction of rewards rather than in conditions requiring short-term prediction”
BackgroundSelf-citationHumans Can Adopt Optimal Discounting Strategy under Real-Time Constraints
Schweighofer, N., Shishida, K., Han, C. E., Okamoto, Y., Tanaka, S. C., Yamawaki, S., & Doya, K. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020152 · Cited by 82 · cited 1×
“In deciding between actions with less-than-average rewards, an action with smaller reward in shorter time can be more appropriate because it allows moving on to the next action earlier, resulting in an apparently impulsive choic”
BackgroundSelf-citationKheramin, S., Body, S., Herrera, F. M., Bradshaw, C. M., Szabadi, E., Deakin, J. F. W., & Anderson, I. M. (2004) · Behavioural Brain Research · doi:10.1016/j.bbr.2004.05.017 · Cited by 79
van der Plasse, G., La Fors, S. S. B. M., Meerkerk, D. T. J., Joosten, R. N. J. M. A., Uylings, H. B. M., & Feenstra, M. G. P. (2007) · Psychopharmacology · doi:10.1007/s00213-007-0917-6 · Cited by 28
Even-Dar, E. (2016) · Springer eBooks · doi:10.1007/978-1-4939-2864-4_341 · Cited by 22
Humans can adopt optimal discounting strategy under real-time constraints
Schweighofer, N., Shishida, K., Cheol, H. E., Okamoto, Y., Tanaka, S., Yamawaki, S., & Doya, K. (2005) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020152.eor · Cited by 12
Self-citation
Research works printed but not linked: 1
- Doya, K., Ishii, S., Pouget, A. & Rao, R. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning (MIT Press, Cambridge, Massachusetts, USA, (2007).
Cited by
The theory of constructed emotion: an active inference account of interoception and categorization
Barrett, L. F. (2016) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsw154 · Cited by 1,426
“A brain is a network of billions of communicating neurons, bathed in chemicals called neurotransmitters, which permit neurons to pass information to one another (Doya, 2008; Bargmann, 2012).”
The glutamate homeostasis hypothesis of addiction
Kalivas, P. W. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2515 · Cited by 1,350
“However, if the behaviour fails to yield the adaptive outcome (for example, if a lever press no longer delivers food), then the limbic subcircuit is strongly engaged by the altered environmental contingencies and activity in the motor subcircuit becomes more disorganized, which in turn allows modification of the estab…”
BackgroundThe neural bases of emotion regulation
Etkin, A., Büchel, C., & Gross, J. J. (2015) · Nature Reviews Neuroscience · doi:10.1038/nrn4044 · Cited by 1,244
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
The theory of constructed emotion: an active inference account of interoception and categorization
Barrett, L. F. (2017) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsx060 · Cited by 1,009
“in chemicals called neurotransmitters, which permit neurons to pass information to one another (Doya, 2008; Bargmann, 2012).”
Metadata
- Authors
- Doya, K.
- Affiliations
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Doya, Kenji
- ORCID
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2008-03-26
- Volume
- 11
- Issue
- 4
- Pages
- 410-416
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1971976749
- Cited by
- 642
- References
- 93
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 30 · 2023: 15 · 2022: 26 · 2021: 35 · 2020: 34 · 2019: 51 · 2018: 57 · 2017: 31 · 2016: 39 · 2015: 43 · 2014: 43 · 2013: 45 · 2012: 37
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- decision-making, contextual modulation, reward anticipation, risk-taking, uncertainty, delay discounting, dopamine, anterior cingulate cortex, norepinephrine, orbitofrontal cortex, serotonin, dorsal striatum, dorsal prefrontal cortex, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Decision Making, Environment, Gyrus Cinguli, Humans, Models, Neurological, Models, Psychological, Probability Learning, Reward, Neurotransmitter Agents, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.