Modulators of decision making
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Human and animal decisions are modulated by a variety of environmental and intrinsic contexts. Here I consider computational factors that can affect decision making and review anatomical structures and neurochemical systems that are related to contextual modulation of decision making. Expectation of a high reward can motivate a subject to go for an action despite a large cost, a decision that is influenced by dopamine in the anterior cingulate cortex. Uncertainty of action outcomes can promote risk taking and exploratory choices, in which norepinephrine and the orbitofrontal cortex appear to be involved. Predictable environments should facilitate consideration of longer-delayed rewards, which depends on serotonin in the dorsal striatum and dorsal prefrontal cortex. This article aims to sort out factors that affect the process of decision making from the viewpoint of reinforcement learning theory and to bridge between such computational needs and their neurophysiological substrates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, structures such as the basal ganglia, parietal cortex, and thalamus support procedural learning, visuospatial coordination, and sensory integration (Corbetta and Shulman, 2002; Doya, 2008; Sherman, 2016).”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
In the citing paper
“Basal ganglia enable procedural learning, habit formation, and reinforcement-driven behaviors (Doya, 2008), i.e., reinforcement-based decision-making, allowing operators to execute complex behaviors automatically under pressure.”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Neural recording experiments in animals reveal that midbrain dopamine neurons encode reward prediction error”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122 · cited 1×
“In many animal and human choice experiments, subjects tend to be more sensitive to differences in shorter delays than longer delays, which is well modeled by hyperbolic function”
BackgroundMarkov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · cited 1×
“Norepinephrine is also suggested to be important in regulating the decision to explore alternatives versus exploiting a known resourc”
BackgroundTheory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“Also, functional magnetic resonance imaging (fMRI) experiments using a monetary questionnair”
Method
Research works printed but not linked: 1
- Doya, K., Ishii, S., Pouget, A. & Rao, R. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning (MIT Press, Cambridge, Massachusetts, USA, (2007).
Cited by
The theory of constructed emotion: an active inference account of interoception and categorization
Barrett, L. F. (2016) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsw154 · Cited by 1,426
“A brain is a network of billions of communicating neurons, bathed in chemicals called neurotransmitters, which permit neurons to pass information to one another (Doya, 2008; Bargmann, 2012).”
The glutamate homeostasis hypothesis of addiction
Kalivas, P. W. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2515 · Cited by 1,350
“However, if the behaviour fails to yield the adaptive outcome (for example, if a lever press no longer delivers food), then the limbic subcircuit is strongly engaged by the altered environmental contingencies and activity in the motor subcircuit becomes more disorganized, which in turn allows modification of the estab…”
BackgroundThe neural bases of emotion regulation
Etkin, A., Büchel, C., & Gross, J. J. (2015) · Nature Reviews Neuroscience · doi:10.1038/nrn4044 · Cited by 1,244
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
The theory of constructed emotion: an active inference account of interoception and categorization
Barrett, L. F. (2017) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsx060 · Cited by 1,009
“in chemicals called neurotransmitters, which permit neurons to pass information to one another (Doya, 2008; Bargmann, 2012).”
Active interoceptive inference and the emotional brain
Seth, A. K., & Friston, K. J. (2016) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2016.0007 · Cited by 945
“This may explain why superficial pyramidal cells are equipped with so many synaptic gain control mechanisms; such as NMDA receptors and classical neuromodulatory receptors like D1 dopamine receptors (44-47).”
BackgroundBenchenane, K., Peyrache, A., Khamassi, M., Tierney, P. L., Gioanni, Y., Battaglia, F. P., & Wiener, S. I. (2010) · Neuron · doi:10.1016/j.neuron.2010.05.013 · Cited by 872
“Such valence and salience information is thought to be transmitted by neuromodulators such as dopamine, which could act as a reward prediction (error) signal (Schultz et al., 1997; Dayan and Balleine, 2002; Doya, 2008).”
Reinforcement learning: The Good, The Bad and The Ugly
Dayan, P., & Niv, Y. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.08.003 · Cited by 704
“There has also been new work based on the theory [8] that the effective interest rate for time is under the influence of the neuromodulator serotonin.”
BackgroundA Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Motivation of extended behaviors by anterior cingulate cortex
Holroyd, C. B., & Yeung, N. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.008 · Cited by 618
Moment-to-moment brain signal variability: A next frontier in human brain mapping?
Garrett, D. D., Samanez-Larkin, G. R., MacDonald, S. W. S., Lindenberger, U., McIntosh, A. R., & Grady, C. L. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.02.015 · Cited by 585
The dysconnection hypothesis (2016)
Friston, K., Brown, H. R., Siemerkus, J., & Stephan, K. E. (2016) · Schizophrenia Research · doi:10.1016/j.schres.2016.07.014 · Cited by 561
“This may explain why superficial pyramidal cells (encoding prediction errors) have somany synaptic gain control mechanisms; such as NMDA receptors and classical neuromodulatory receptors like D1 dopamine receptors (Braver et al., 1999; Doya, 2008; Goldman-Rakic et al., 1992; Lidow et al., 1991).”
BackgroundAn active inference theory of allostasis and interoception in depression
Barrett, L. F., Quigley, K. S., & Hamilton, P. (2016) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2016.0011 · Cited by 554
Computational psychiatry: the brain as a phantastic organ
Friston, K. J., Stephan, K. E., Montague, R., & Dolan, R. J. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70275-5 · Cited by 531
A Neural Circuit Mechanism for Encoding Aversive Stimuli in the Mesolimbic Dopamine System
de Jong, J. W., Afjei, S. A., Pollak Dorocic, I., Peck, J. R., Liu, C., Kim, C. K., Tian, L., Deisseroth, K., & Lammel, S. (2019) · Neuron · doi:10.1016/j.neuron.2018.11.005 · Cited by 511
Varazzani, C., San-Galli, A., Gilardeau, S., & Bouret, S. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.0454-15.2015 · Cited by 452
“But what is the specific contribution of each neuromodulator? Even if noradrenaline (NA) is critical for motivation and decision making, its exact contribution remains unclear (AstonJones et al., 1991; Aston-Jones and Cohen, 2005; Yu and Dayan, 2005; Doya, 2008; Ventura et al., 2008; Bouret and Richmond, 2009).”
BackgroundCerebral hierarchies: predictive processing, precision and the pulvinar
Kanai, R., Komura, Y., Shipp, S., & Friston, K. (2015) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2014.0169 · Cited by 432
Cosmetic Use of Botulinum Toxin-A Affects Processing of Emotional Language
Havas, D. A., Glenberg, A. M., Gutowski, K. A., Lucarelli, M. J., & Davidson, R. J. (2010) · Psychological Science · doi:10.1177/0956797610374742 · Cited by 389
“At the psychological level, our account of how feedback from emotional states guides simulation in language was suggested by a recent theory of reinforcement learning (Doya, 2007, 2008).”
BackgroundDopamine, affordance and active inference
Friston, K. J., Shiner, T., FitzGerald, T., Galea, J. M., Adams, R., Brown, H., Dolan, R. J., Moran, R., Stephan, K. E., & Bestmann, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002327 · Cited by 381
“This formulation sits comfortably with the affordance competition hypothesis [2], [39] and other theoretical accounts: for example, the uncertainty processing theory of motivation [40], neurobiological accounts of decision-making [41], [42] and the plurality of roles suggested by the physiology of dopamine [43].”
BackgroundDecoding subjective decisions from orbitofrontal cortex
Rich, E. L., & Wallis, J. D. (2016) · Nature Neuroscience · doi:10.1038/nn.4320 · Cited by 371
Stoléru, S., Fonteille, V., Cornélis, C., Joyal, C., & Moulier, V. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.03.006 · Cited by 360
Hassani, S. A., Oemisch, M., Balcarras, M., Westendorff, S., Ardid, S., van der Meer, M. A., Tiesinga, P., & Womelsdorf, T. (2017) · Scientific Reports · doi:10.1038/srep40606 · Cited by 360
Friston, K., & Frith, C. (2015) · Consciousness and Cognition · doi:10.1016/j.concog.2014.12.003 · Cited by 359
Pavlovian to instrumental transfer: A neurobehavioural perspective
Holmes, N. M., Marchand, A. R., & Coutureau, E. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.03.007 · Cited by 334
A critical review of sex differences in decision-making tasks: Focus on the Iowa Gambling Task
van den Bos, R., Homberg, J., & de Visser, L. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2012.10.002 · Cited by 325
Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass
Menegas, W., Bergan, J. F., Ogawa, S. K., Isogai, Y., Umadevi Venkataraju, K., Osten, P., Uchida, N., & Watabe-Uchida, M. (2015) · eLife · doi:10.7554/elife.10032 · Cited by 322
“For instance, it has often been assumed that the striatum sends inhibitory “expectation” signals to dopamine neurons that can “cancel out” reward values when they are expected (Doya, 2008; Houk and Adams, 1995; Khamassi and Humphries, 2012; Kobayashi and Okada, 2007; Rao, 2010).”
BackgroundGender differences in reward-related decision processing under stress
Lighthall, N. R., Sakaki, M., Vasunilashorn, S., Nga, L., Somayajula, S., Chen, E. Y., Samii, N., & Mather, M. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsr026 · Cited by 310
“To de- termine the relationship between individual differences in ROI activation and other outcome measures, Pearson’s cor- relations were conducted across and within groups.”
BackgroundMethodGoschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293
“, 2010), whereas serotonergic systems appear to play a role in regulating the delay discounting rate and thus the degree of impulsivity in intertemporal decision-making (Doya, 2008; Schweighofer et al., 2008; Tanaka et al., 2007).”
BackgroundInfluentialAltered Processing of Contextual Information during Fear Extinction in PTSD: An fMRI Study
Rougemont-Bücking, A., Linnman, C., Zeffiro, T. A., Zeidan, M. A., Lebron-Milad, K., Rodriguez-Romaguera, J., Rauch, S. L., Pitman, R. K., & Milad, M. R. (2011) · CNS Neuroscience & Therapeutics · doi:10.1111/j.1755-5949.2010.00152.x · Cited by 292
Dopamine Is Required for Learning and Forgetting in Drosophila
Berry, J. A., Cervantes-Sandoval, I., Nicholas, E. P., & Davis, R. L. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.007 · Cited by 291
“, 2009), decision making (Doya, 2008; Zhang et al., 2007), arousal (Andretic et al.”
Goschke, T., & Bolte, A. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.07.015 · Cited by 288
Adaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286
van den Bos, W., Cohen, M. X., Kahnt, T., & Crone, E. A. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr198 · Cited by 276
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267
“Involvement of locus coeruleus and the neurotransmitter norepinephrine in tracking unexpected uncertainty has been conjectured a number of times and the evidence in its favor is suggestive [5,28,28], but further proof is needed.”
BackgroundReward and the serotonergic system
Kranz, G. S., Kasper, S., & Lanzenberger, R. (2010) · Neuroscience · doi:10.1016/j.neuroscience.2010.01.036 · Cited by 267
Computational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
“, dopamine, DA; acetylcholine, ACh; norepinephrine, NE), an application domain of particular relevance for psychiatry and a ‘classical’ target of computational modeling [45,46].”
BackgroundThe Exploration-Exploitation Dilemma: A Multidisciplinary Framework
Berger-Tal, O., Nathan, J., Meron, E., & Saltz, D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0095693 · Cited by 253
“This result is supported by both theoretical models of learning in stochastic environments and empirical studies with humans [20,34,35].”
BackgroundResultUncertainty, epistemics and active inference
Parr, T., & Friston, K. J. (2017) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0376 · Cited by 252
“We show through simulation that epistemic foraging is heavily influenced by the beliefs an agent has about the volatility and sensory precision of their environment [8].”
BackgroundResultRogers, R. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.165 · Cited by 249
“…parameters that determine outcome values in reinforcement learning modelsFincluding longer-term rewards rates (Daw et al, 2002) or delay parameters (Doya, 2008)Fprecisely by influencing the cognitive appraisal of dilemmas giving rise to these outcomes and the appraisal of the decision outcomes…”
BackgroundInfluentialNejati, V., Salehinejad, M. A., & Nitsche, M. A. (2018) · Neuroscience · doi:10.1016/j.neuroscience.2017.10.042 · Cited by 243
Activity patterns of serotonin neurons underlying cognitive flexibility
Matias, S., Lottem, E., Dugué, G. P., & Mainen, Z. F. (2017) · eLife · doi:10.7554/elife.20552 · Cited by 239
“This suggests that, contrary to the notion that each neuromodulator reports a completely distinct signal (Daw et al., 2002; Doya, 2008; Dayan, 2012), they have highly overlapping signals, presumably derived from partly overlapping inputs, but with more subtle differences through which their joint…”
ResultNeuromodulation of reward‐based learning and decision making in human aging
Eppinger, B., Hämmerer, D., & Li, S. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06230.x · Cited by 234
Reinforcement learning in depression: a review of computational research
Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.05.005 · Cited by 233
Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments
Avery, M. C., & Krichmar, J. L. (2017) · Frontiers in Neural Circuits · doi:10.3389/fncir.2017.00108 · Cited by 230
“Another theory posits that dopamine is related to reward prediction, while serotonin is related to temporal discounting, and that noradrenaline regulates the exploration/exploitation tradeoff, while acetylcholine controls learning rate (Doya, 2002, 2008).”
BackgroundDistributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons
Tian, J., Huang, R., Cohen, J. Y., Osakada, F., Kobak, D., Machens, C. K., Callaway, E. M., Uchida, N., & Watabe-Uchida, M. (2016) · Neuron · doi:10.1016/j.neuron.2016.08.018 · Cited by 224
“We chose to record from input-dense areas along this ventral stream that have most often been used in RPE models: the ventral and dorsal striatum (Aggarwal et al., 2012; Doya, 2008; Kawato and Samejima, 2007), ventral pallidum (Kawato and Samejima, 2007; Keiflin and”
Ackermann, H., Hage, S. R., & Ziegler, W. (2014) · Behavioral and Brain Sciences · doi:10.1017/s0140525x13003099 · Cited by 215
Dissociable effects of lesions to orbitofrontal cortex subregions on impulsive choice in the rat
Mar, A. C., Walker, A. L. J., Theobald, D. E., Eagle, D. M., & Robbins, T. W. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6620-10.2011 · Cited by 206
“The orbitofrontal cortex (OFC) has been strongly implicated in reinforcement-guided decision making (Murray et al., 2007; Wallis, 2007; Doya, 2008).”
BackgroundBaker, T. E., & Holroyd, C. B. (2011) · Biological Psychology · doi:10.1016/j.biopsycho.2011.01.010 · Cited by 203
“The finding that ACC neurons respond to choice outcomes has motivated the proposal that this region composes part of a larger system for reinforcement learning, such that reinforcement learning signals, believed to be carried by the dopamine system, shape the connectivity and function of neurons in the ACC and prefron…”
BackgroundThe Functional and Structural Neural Basis of Individual Differences in Loss Aversion
Canessa, N., Crespi, C., Motterlini, M., Baud-Bovy, G., Chierchia, G., Pantaleo, G., Tettamanti, M., & Cappa, S. F. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.0497-13.2013 · Cited by 201
“Within this network, stimuli or actions acquire a moti- vational significance for their potential “reward” value, which is anticipated when the same stimuli or actions are evaluated to make further choices (i.e., select actions) (Doya, 2008; Hare et al., 2010) (Fig.”
BackgroundComputational noise in reward-guided learning drives behavioral variability in volatile environments
Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., & Wyart, V. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0518-9 · Cited by 196
“A second effect documented in the literature consists of an adjustment of choice stochasticity to surprise (that is, the magnitude of the prediction error in RL) 16,17 .”
BackgroundHisted, M. H., Pasupathy, A., & Miller, E. K. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.019 · Cited by 187
“Thus, it has been unclear how such temporally disparate signals interact, though several ideas have been put forth, especially in the context of neuroeconomics (e.g. Rangel et al., 2008; Montague and Berns, 2002; Rushworth and Behrens, 2008; Doya, 2008).”
BackgroundA neural circuit covarying with social hierarchy in macaques
Noonan, M. P., Sallet, J., Mars, R. B., Neubert, F. X., O'Reilly, J. X., Andersson, J. L., Mitchell, A. S., Bell, A. H., Miller, K. L., & Rushworth, M. F. S. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001940 · Cited by 183
“Activity in some of the regions we identified, such as parts of the striatum, is known to encode many parameters that determine the values of choices [35], and our results suggest the same regions may also encode longer term valuation signals reflecting each individual’s status.”
BackgroundParallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards
Kim, H. F., & Hikosaka, O. (2015) · Brain · doi:10.1093/brain/awv134 · Cited by 181
“In addition to dopamine, there are other neuromodulators acting on basal ganglia circuits, including serotonin, acetylcholine, and norepinephrine (Doya, 2008).”
BackgroundNeurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects
Cinel, C., Valeriani, D., & Poli, R. (2019) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2019.00013 · Cited by 178
“Recent advances in neuroscience have also shed light on how individuals approach decision-making, their strategies and their aptitude to risk-taking behavior (Doya, 2008; Rushworth and Behrens, 2008).”
BackgroundKennerley, S. W., & Walton, M. E. (2011) · Behavioral Neuroscience · doi:10.1037/a0023575 · Cited by 176
“…humans and animals make choices to satisfy internal needs, such as metabolic state, ii) how value is represented (whether outcome specific or as an abstract “cached” value) and, iii) how these parameters can be incorporated into models of the computation of value (Doya, 2008; Rangel et al., 2008).”
BackgroundGruber, A. J., & McDonald, R. J. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00050 · Cited by 173
“It is now generally accepted that multiple forms of learning, including both habit and goal-oriented systems, are distributed among multiple brain structures and interact so as to control actions in rodents and primates (McClelland et al., 1995; Balleine and Dickinson, 1998; Wise and Murray, 2000; Cardinal et al., 200…”
BackgroundReward uncertainty enhances incentive salience attribution as sign-tracking
Anselme, P., Robinson, M. J. F., & Berridge, K. C. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2012.10.006 · Cited by 170
Hosoda, C., Tanaka, K., Nariai, T., Honda, M., & Hanakawa, T. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.0410-13.2013 · Cited by 166
“The corticobasal ganglia circuits are suggested for reward-based reinforcement learning since they receive both contextual information from the cortex and reward signals from dopaminergic neurons (Doya, 2008).”
BackgroundThe functional neuroanatomy of decision making: Prefrontal control of thought and action
Coutlee, C. G., & Huettel, S. A. (2012) · Brain Research · doi:10.1016/j.brainres.2011.05.053 · Cited by 155
“…to behavioral and neurochemical findings from the cognitivecontrol literature suggesting that reward, emotion, and control mechanisms interact to balance stable and flexible responding (Braver and Cohen, 2000; Cools and Robbins, 2004; Doya, 2008; Dreisbach and Goschke, 2004; Müller et al., 2007).”
BackgroundAkineto-rigid vs. tremor syndromes in Parkinsonism
Zaidel, A., Arkadir, D., Israel, Z., & Bergman, H. (2009) · Current Opinion in Neurology · doi:10.1097/wco.0b013e32832d9d67 · Cited by 150
The Functional Neuroanatomy of Decision-Making
Rosenbloom, M. H., Schmahmann, J. D., & Price, B. H. (2012) · The Journal of Neuropsychiatry and Clinical Neurosciences · doi:10.1176/appi.neuropsych.11060139 · Cited by 139
Pan, W. X., Mao, T., & Dudman, J. T. (2010) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2010.00147 · Cited by 150
“The projection of serotonergic neurons to the dorsal striatum has received some attention (Di Matteo et al., 2008), especially interesting due to its proposed role in mediating opponent learning (Daw et al., 2002; Doya, 2008).”
BackgroundNew insights on the subcortical representation of reward
Hikosaka, O., Bromberg-Martin, E., Hong, S., & Matsumoto, M. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.002 · Cited by 143
“As theoretical studies have suggested [48], different types of neurons appear to contribute to different aspects of reward-based learning and decision-making.”
Differential roles of caudate nucleus and putamen during instrumental learning
Brovelli, A., Nazarian, B., Meunier, M., & Boussaoud, D. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.05.059 · Cited by 142
“From a normative point of view, reinforcement learning theory (Sutton and Barto, 1998) has provided a coherent computational framework to model learning and their neural substrates (Doya, 2007, 2008).”
de Visser, L., van der Knaap, L. J., van de Loo, A. J. A. E., van der Weerd, C. M. M., Ohl, F., & van den Bos, R. (2010) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2010.01.027 · Cited by 147
Billieux, J., Lagrange, G., Van der Linden, M., Lançon, C., Adida, M., & Jeanningros, R. (2012) · Psychiatry Research · doi:10.1016/j.psychres.2012.01.001 · Cited by 137
“Crucially, it has been shown that these specific cognitive mechanisms are involved in adapted decisionmaking (Dunnet al., 2006;Doya, 2008), which is often impaired in PGs (Cavedini et al.”
BackgroundCoding of Task Reward Value in the Dorsal Raphe Nucleus
Bromberg-Martin, E. S., Hikosaka, O., & Nakamura, K. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0015-10.2010 · Cited by 147
“Serotonin release then has a potent influence on actions to gain rewards and avoid punishments (Rogers et al., 1999; Daw et al., 2002; Doya, 2008; Dayan and Huys, 2009).”
BackgroundExplanatory correlates of consciousness: theoretical and computational challenges.
Seth, A. (2009) · Cognitive Computation · doi:10.1007/s12559-009-9007-x · Cited by 146
“For example, ‘affective robotics’ describes attempts to enhance adaptive behaviour through emotional modulation of decision making, to facilitate human–robot interactions by exploiting human receptivity to emotional stimuli, as well as to enhance our understanding of the neuromodulatory interactions underlying emotion…”
BackgroundDifferent Varieties of Uncertainty in Human Decision-Making
Bland, A. R., & Schaefer, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00085 · Cited by 129
“In contrast, during unexpected changes in the environment that lead to a durable invalidity of our previous representations, one needs to take exploratory actions (Doya, 2008).”
BackgroundMethodInfluentialIto, M., & Doya, K. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.001 · Cited by 160
“Introduction The loop network composed by the cerebral cortex and the basal ganglia is now recognized as the major site for decision making and reinforcement learning [1,2].”
BackgroundSelf-citationPositive reward prediction errors during decision-making strengthen memory encoding
Jang, A. I., Nassar, M. R., Dillon, D. G., & Frank, M. J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0597-3 · Cited by 146
The role of the midcingulate cortex in monitoring others' decisions
Apps, M. A. J., Lockwood, P. L., & Balsters, J. H. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00251 · Cited by 126
“However, the ACC is also engaged by rewards (Doya, 2008), attention and salience (Davis et al., 2005), conflict, and during decision-making (Botvinick et al., 1999; Botvinick, 2007) which are inherently non-social processes.”
BackgroundInfluentialThe Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World
Krichmar, J. L. (2008) · Adaptive Behavior · doi:10.1177/1059712308095775 · Cited by 132
“Kenji Doya and colleagues have proposed a theory that encompasses multiple neuromodulatory systems (Doya, 2002, 2008).”
BackgroundDecision Threshold Modulation in the Human Brain
Domenech, P., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2371-10.2010 · Cited by 125
“One important factor that is likely to shape how a decision is adaptively tuned to its context is the predictability of the forthcoming event (Luce, 1991; Dayan and Abbott, 2001; Harrison et al., 2006; Doya, 2008).”
BackgroundDecision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition
Budaev, S. V., Jørgensen, C., Mangel, M., Eliassen, S., & Giske, J. (2019) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2019.00164 · Cited by 139
“Individual learning plays a crucial role in decision-making (Doya, 2008; Frankenhuis et al., 2018).”
BackgroundGu, R., Huang, W., Camilleri, J., Xu, P., Wei, P., Eickhoff, S. B., & Feng, C. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.02.017 · Cited by 124
“In light of previous research on the brain reward network in humans (Doya, 2008; Frank et al., 2009), we propose a domain-general neural circuitry responsible for encoding social and monetary reward anticipation.”
Dopamine release in ventral striatum of pathological gamblers losing money
Linnet, J., Peterson, E., Doudet, D. J., Gjedde, A., & Møller, A. (2010) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2010.01591.x · Cited by 127
“PG who lost money had significantly higher dopamine release in the left ventral striatum than HC, F(1, 29) = 5.”
BackgroundNeuronal Activity in the Human Subthalamic Nucleus Encodes Decision Conflict during Action Selection
Zaghloul, K. A., Weidemann, C. T., Lega, B. C., Jaggi, J. L., Baltuch, G. H., & Kahana, M. J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5815-11.2012 · Cited by 122
“Selecting actions that maximize value among competing alternatives is vital to an organism’s survival (Doya, 2008; Rangel et al., 2008; Kable and Glimcher, 2009).”
BackgroundCilia, R., Cho, S. S., van Eimeren, T., Marotta, G., Siri, C., Ko, J. H., Pellecchia, G., Pezzoli, G., Antonini, A., & Strafella, A. P. (2011) · Movement Disorders · doi:10.1002/mds.23480 · Cited by 121
“Dopaminemediated signaling into the striatum, prefrontal and cingulate cortex from midbrain ventral tegmental area is known to mediate the computation of reward prediction error and outcome expectation from future choices.(28,47,52,53) It may be argued that gambling severity is not only associated with impaired respon…”
BackgroundDecision Making: a Theoretical Review
Morelli, M., Casagrande, M., & Forte, G. (2021) · Integrative Psychological and Behavioral Science · doi:10.1007/s12124-021-09669-x · Cited by 149
“Although a causal relationship cannot always be clear in neuropsychological fields because of the different components involved (Robinson, 2016), the decision-making process can be divided into stages involving different brain areas (Doya, 2008).”
BackgroundNeuroeconomics and aging: Neuromodulation of economic decision making in old age
Mohr, P. N. C., Li, S., & Heekeren, H. R. (2009) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.05.010 · Cited by 130
“Here, we focus on dopamine and serotonin because these two neurotransmitters so far show the clearest relationship with economic behavior (Doya, 2008, for review).”
BackgroundIndividual differences in the influence of task-irrelevant Pavlovian cues on human behavior
Garofalo, S., & di Pellegrino, G. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00163 · Cited by 130
“Keywords: Pavlovian-to-instrumental transfer, cue-controlled behavior, Sign-Tracker, Goal-Tracker, reinforcement learning Introduction Goal-directed behavior can be variably influenced by external and internal factors which impact the values and priorities assigned to rewards and goals (Doya, 2008).”
Is attention really effort? Revisiting Daniel Kahneman’s influential 1973 book attention and effort.
Bruya, B., & Tang, Y.-Y. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01133 · Cited by 84
“…to be comprehensive in building cognitive models, one must take a fundamentally biological approach to cognition, recognizing the human being as a product of natural selection, guided by an intentional perspective on the world (Brentano, [1874] 1973; Dewey, 1958; Dennett, 1987; Freeman, 2000).”
BackgroundAltered function of ventral striatum during reward-based decision making in old age
Mell, T. (2009) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.034.2009 · Cited by 107
Ziegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
Distributed Adaptive Control: A theory of the Mind, Brain, Body Nexus
Verschure, P. F. M. J. (2012) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2012.04.005 · Cited by 131
“A value dependent reward signal (Doya, 2008; Schultz, 2006) modulates the activity of the memory units in order to bias action selection towards the optimal solution of the goal-oriented task.”
BackgroundBoto, T., Louis, T., Jindachomthong, K., Jalink, K., & Tomchik, S. M. (2014) · Current Biology · doi:10.1016/j.cub.2014.03.021 · Cited by 111
Diffusion Tensor Imaging and Decision Making in Cocaine Dependence
Lane, S. D., Steinberg, J. L., Ma, L., Hasan, K. M., Kramer, L. A., Zuniga, E. A., Narayana, P. A., & Moeller, F. G. (2010) · PLoS ONE · doi:10.1371/journal.pone.0011591 · Cited by 104
“The neural correlates that subserve decision making may involve reward valuation in amygdala and ventral striatum [6,7,8,9], and outcome and emotional processing in orbital/ prefrontal cortex [10,11], anterior cingulate cortex [12,13,14], and”
BackgroundBrain Switches Utilitarian Behavior: Does Gender Make the Difference?
Fumagalli, M., Vergari, M., Pasqualetti, P., Marceglia, S., Mameli, F., Ferrucci, R., Mrakic-Sposta, S., Zago, S., Sartori, G., Pravettoni, G., et al. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008865 · Cited by 106
“Dopamine is important for behavior, motivation and decision-making [64].”
BackgroundDopamine blockade impairs the exploration-exploitation trade-off in rats
Cinotti, F., Fresno, V., Aklil, N., Coutureau, É., Girard, B., Marchand, A. R., & Khamassi, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-43245-z · Cited by 111
Machens, C. K. (2012) · Science · doi:10.1126/science.1231865 · Cited by 111
Long-Range Attention Networks: Circuit Motifs Underlying Endogenously Controlled Stimulus Selection
Womelsdorf, T., & Everling, S. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.08.009 · Cited by 115
“Both aspects are more frequently studied with respect to their influence on decision making and reinforcement learning problems [128–131].”
Day, J. J., Jones, J. L., & Carelli, R. M. (2011) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07531.x · Cited by 99
“…that project to the NAc, including the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC) and basolateral amygdala (BLA), are known to process reward-related information (Critchley & Rolls, 1996; Watanabe, 1996; Behrens et al., 2007; Belova et al., 2007; Doya, 2008; Tye et al., 2008).”
BackgroundSydserff, S. G., Sutton, E. J., Song, D., Quirk, M. C., Maciag, C. M., Li, Z., Jonak, G. J., Gurley, D. A., Gordon, J. C., Christian, E. P., et al. (2009) · Biochemical Pharmacology · doi:10.1016/j.bcp.2009.07.005 · Cited by 100
“Furthermore, computational and experimental studies suggest that dopamine can enhance the eligibility trace of conditioned behaviors [25,26]—a potential mechanistic necessity for response learning under conditions of delayed reinforcement.”
BackgroundTaming the beast: extracting generalizable knowledge from computational models of cognition
Nassar, M. R., & Frank, M. J. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.04.003 · Cited by 81
“But several studies have shown that choice likelihood can deviate substantially from either of these functions (for example as a result of just a few choices driven by a different process altogether, such as attentional lapses), and the failure to take into account this irreducible noise can over-emphasize or under-em…”
BackgroundForaging across the life span: is there a reduction in exploration with aging?
Mata, R., Wilke, A., & Czienskowski, U. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00053 · Cited by 84
“Given the role of catecholamines in modulating learning, novelty seeking, and explorative behavior (Hills, 2006; Cohen et al., 2007; Doya, 2008; Eppinger et al., 2011), age-related deficits in catecholaminergic modulation could be expected to lead to age differences in foraging.”
BackgroundEntrepreneurial decision-making: new conceptual perspectives
Winnaar, K. D., & Scholtz, F. (2019) · Management Decision · doi:10.1108/md-11-2017-1152 · Cited by 96
Pharmacological manipulations of judgement bias: A systematic review and meta-analysis
Neville, V., Nakagawa, S., Zidar, J., Paul, E. S., Lagisz, M., Bateson, M., Løvlie, H., & Mendl, M. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.11.008 · Cited by 101
Contributions of ERK signaling in the striatum to instrumental learning and performance
Shiflett, M. W., & Balleine, B. W. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.12.010 · Cited by 94
“lator of decision processes, with greater ERK activation evoked by stimulus or event facilitating action initiation [103].”
Ovarian Cycle Effects on Immediate Reward Selection Bias in Humans: A Role for Estradiol
Smith, C. T., Sierra, Y., Oppler, S. H., & Boettiger, C. A. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.0014-14.2014 · Cited by 93
“However, animal studies also indicate that Now bias can be pharmacologically modulated, particularly by dopamine (DA; Dalley et al., 2008; Doya, 2008; Winstanley, 2011).”
Background
Metadata
- Authors
- Doya, K.
- Affiliations
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Doya, Kenji
- ORCID
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2008-03-26
- Volume
- 11
- Issue
- 4
- Pages
- 410-416
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1971976749
- Cited by
- 642
- References
- 93
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 30 · 2023: 15 · 2022: 26 · 2021: 35 · 2020: 34 · 2019: 51 · 2018: 57 · 2017: 31 · 2016: 39 · 2015: 43 · 2014: 43 · 2013: 45 · 2012: 37
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- decision-making, contextual modulation, reward anticipation, risk-taking, uncertainty, delay discounting, dopamine, anterior cingulate cortex, norepinephrine, orbitofrontal cortex, serotonin, dorsal striatum, dorsal prefrontal cortex, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Decision Making, Environment, Gyrus Cinguli, Humans, Models, Neurological, Models, Psychological, Probability Learning, Reward, Neurotransmitter Agents, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.