Modulators of decision making
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Human and animal decisions are modulated by a variety of environmental and intrinsic contexts. Here I consider computational factors that can affect decision making and review anatomical structures and neurochemical systems that are related to contextual modulation of decision making. Expectation of a high reward can motivate a subject to go for an action despite a large cost, a decision that is influenced by dopamine in the anterior cingulate cortex. Uncertainty of action outcomes can promote risk taking and exploratory choices, in which norepinephrine and the orbitofrontal cortex appear to be involved. Predictable environments should facilitate consideration of longer-delayed rewards, which depends on serotonin in the dorsal striatum and dorsal prefrontal cortex. This article aims to sort out factors that affect the process of decision making from the viewpoint of reinforcement learning theory and to bridge between such computational needs and their neurophysiological substrates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, structures such as the basal ganglia, parietal cortex, and thalamus support procedural learning, visuospatial coordination, and sensory integration (Corbetta and Shulman, 2002; Doya, 2008; Sherman, 2016).”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
In the citing paper
“Basal ganglia enable procedural learning, habit formation, and reinforcement-driven behaviors (Doya, 2008), i.e., reinforcement-based decision-making, allowing operators to execute complex behaviors automatically under pressure.”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Neural recording experiments in animals reveal that midbrain dopamine neurons encode reward prediction error”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122 · cited 1×
“In many animal and human choice experiments, subjects tend to be more sensitive to differences in shorter delays than longer delays, which is well modeled by hyperbolic function”
BackgroundMarkov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · cited 1×
“Norepinephrine is also suggested to be important in regulating the decision to explore alternatives versus exploiting a known resourc”
BackgroundTheory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“Also, functional magnetic resonance imaging (fMRI) experiments using a monetary questionnair”
Method
Research works printed but not linked: 1
- Doya, K., Ishii, S., Pouget, A. & Rao, R. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning (MIT Press, Cambridge, Massachusetts, USA, (2007).
Cited by
Items 101–200 of 688
A neurochemical approach to valuation sensitivity over gains and losses
Zhong, S., Israel, S., Xue, H., Sham, P. C., Ebstein, R. P., & Chew, S. H. (2009) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2009.1312 · Cited by 91
“serotonin, which may have a role in predicting short- and long-term rewards in the ventral and dorsal striatum (Tanaka et al. 2007; Doya 2008).”
BackgroundPalminteri, S., Lebreton, M., Worbe, Y., Hartmann, A., Lehéricy, S., Vidailhet, M., Grabli, D., & Pessiglione, M. (2011) · Brain · doi:10.1093/brain/awr147 · Cited by 90
“Growing evidence suggests that actor and critic modules may be represented in the dorsal and ventral striatum, respectively (Doya, 2008).”
BackgroundInsensitivity of place cells to the value of spatial goals in a two-choice flexible navigation task
Duvelle, É., Grieves, R. M., Hok, V., Poucet, B., Arleo, A., Jeffery, K., & Save, E. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.1578-18.2018 · Cited by 91
“Neurons encoding goals, their value, or different decision-making parameters such as reward expectation, have been found in other brain regions (Doya, 2008).”
BackgroundNeural basis of conditional cooperation
Suzuki, S., Niki, K., Fujisaki, S., & Akiyama, E. (2010) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsq042 · Cited by 94
“The OMPFC and the striatum are thought to be parts of a neural circuit that guides behavior based on reward feedback (Bechara et al., 1999, 2000; O’Doherty, 2004; O’Doherty et al., 2004; Doya, 2008; Hare et al., 2008; Rushworth and Behrens, 2008), which is consistent with our results.”
InfluentialMartinez, D. L., Canessa, N., Brusoni, S., Zollo, M., Hare, T. A., Alemanno, F., & Cappa, S. F. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00927 · Cited by 103
“A pivotal role in choice optimization has been attributed to noradrenergic activity in the locus coeruleus (Usher et al., 1999; Doya, 2008), although with variable interpretations.”
BackgroundBetter late than never? The effect of feedback delay on ERP indices of reward processing
Weinberg, A., Luhmann, C. C., Bress, J. N., & Hajcak, G. (2012) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-012-0104-z · Cited by 84
“When an action is taken, an eligibility trace for that action is initiated and immediately begins to decay with time (Doya, 2008).”
Vinckier, F., Gaillard, R., Palminteri, S., Rigoux, L., Salvador, A., Fornito, A., Adapa, R., Krebs, M. O., Pessiglione, M., & Fletcher, P. C. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.73 · Cited by 87
Hauser, T. U., Iannaccone, R., Dolan, R. J., Ball, J., Hättenschwiler, J., Drechsler, R., Rufer, M., Brandeis, D., Walitza, S., & Brem, S. (2017) · Psychological Medicine · doi:10.1017/s0033291716003305 · Cited by 86
“2016b) and are closely interconnected with other cortical and subcortical systems (Doya, 2008).”
BackgroundNeural substrates underlying effort computation in schizophrenia
Fervaha, G., Foussias, G., Agid, O., & Remington, G. J. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.09.001 · Cited by 88
Different representations of relative and absolute subjective value in the human brain
Grabenhorst, F., & Rolls, E. T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.06.045 · Cited by 99
“…when temporal discounting (the rate at which reward value decreases as a function of the delay until the reward is given) is evaluated in 'discounted utility' theory the calculations are performed with respect to the current absolute value of the goods (Doya, 2008; Kable and Glimcher, 2007).”
BackgroundDynamic Interplay Among Homeostatic, Hedonic, and Cognitive Feedback Circuits Regulating Body Weight
Hall, K. D., Hammond, R. A., & Rahmandad, H. (2014) · American Journal of Public Health · doi:10.2105/ajph.2014.301931 · Cited by 80
The role of the nucleus accumbens core in impulsive choice, timing, and reward processing.
Galtress, T., & Kirkpatrick, K. (2010) · Behavioral Neuroscience · doi:10.1037/a0018464 · Cited by 79
“Finally, the Acb has been directly implicated in impulsive choice (Doya, 2008; Gregorios-Pippas, Tobler, & Schultz, 2009), with AcbC lesions resulting in increased preference for smaller sooner rewards in temporal discounting tasks (Bezzina et al., 2007; Cardinal, Pennicott, Sugathapala, Robbins, &…”
MethodKoot, S., Zoratto, F., Cassano, T., Colangeli, R., Laviola, G., van den Bos, R., & Adriani, W. (2011) · Neuropharmacology · doi:10.1016/j.neuropharm.2011.11.002 · Cited by 86
“For instance, low levels of 5-HT have been associated with poor decision-making and/or poor impulse control (Baldwin and Rudge, 1995; Daw et al., 2002; Doya, 2008; Homberg et al., 2008; Lucki, 1998; Owens and Nemeroff, 1994; Soubrié, 1986; Tanaka et al., 2007; Winstanley et al., 2004).”
InfluentialWhen Giving Is Good: Ventromedial Prefrontal Cortex Activation for Others' Intentions
Cooper, J. C., Kreps, T. A., Wiebe, T., Pirkl, T., & Knutson, B. (2010) · Neuron · doi:10.1016/j.neuron.2010.06.030 · Cited by 79
“) One important experimental control involved subjective certainty about inferences, which might also modulate prefrontal activation (Doya, 2008; Rushworth and Behrens, 2008).”
BackgroundNeural systems analysis of decision making during goal-directed navigation
Penner, M. R., & Mizumori, S. J. Y. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.08.010 · Cited by 95
“, Doya , 2008 ; Farrar et al .”
The dopaminergic hypothesis of attention-deficit/hyperactivity disorder needs re-examining
Gonon, F. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2008.09.010 · Cited by 79
Krichmar, J. L. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00001 · Cited by 83
“One exception was a theory proposed by Kenji Doya (Doya, 2002, 2008).”
BackgroundStress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning
Lukšys, G., Gerstner, W., & Sandi, C. (2009) · Nature Neuroscience · doi:10.1038/nn.2374 · Cited by 86
“, and to neural activity in amygdala, striatum and anterior cingulat”
BackgroundBaker, P. M., Oh, S., Kidder, K. S., & Mizumori, S. J. Y. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00295 · Cited by 78
“This has led many to suggest that 5-HT signaling contributes to reward or aversive learning especially when behavioral expectancy signals must be updated for future behavioral choices such as when risk is involved in decision making (Doya, 2008; Robbins and Arnsten, 2009; Bari et al., 2010; Liu et al., 2014).”
BackgroundLayer- and area-specific actions of norepinephrine on cortical synaptic transmission
Salgado, H., Treviño, M., & Atzori, M. (2016) · Brain Research · doi:10.1016/j.brainres.2016.01.033 · Cited by 79
Puryear, C. B., Kim, M. J., & Mizumori, S. J. Y. (2010) · Behavioral Neuroscience · doi:10.1037/a0018865 · Cited by 71
“that would maximize the total amount of reward obtained (Doya, 2008; Ikemoto, 2007; Redgrave & Gurney, 2006).”
Evolving Concepts of Arousal: Insights from Simple Model Systems
Jing, J., Gillette, R., & Weiss, K. R. (2009) · Reviews in the Neurosciences · doi:10.1515/revneuro.2009.20.5-6.405 · Cited by 79
Instrumental vigour in punishment and reward
Dayan, P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08026.x · Cited by 72
“Even then, we have passed over a huge wealth of issues discussed in these reviews, including other influential views of serotonin, such as its potential role in temporal discounting (Tanaka et al., 2004, 2007; Doya, 2008; Schweighofer et al., 2008).”
BackgroundComplementary neural correlates of motivation in dopaminergic and noradrenergic neurons of monkeys
Bouret, S., Ravel, S., & Richmond, B. J. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00040 · Cited by 66
“ORIGINAL RESEARCH ARTICLE published: 17 July 2012 doi: 10.3389/fnbeh.2012.00040 Complementary neural correlates of motivation in dopaminergic and noradrenergic neurons of monkeys”
BackgroundInfluentialBystritsky, A., Nierenberg, A. A., Feusner, J. D., & Rabinovich, M. I. (2012) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2011.10.013 · Cited by 75
“This principle has also been extensively employed in neuroeconomics (Kenning and Plassmann, 2005), game theory (Lee, 2008), and decision-making theory (Doya, 2008).”
BackgroundDissociable forms of uncertainty-driven representational change across the human brain
Nassar, M. R., McGuire, J. T., Ritz, H., & Kable, J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1713-18.2018 · Cited by 68
“Interestingly, both reward prediction errors and relative uncertainty have been suggested to be signaled through brainstem neuromodulatory systems that could potentially have widespread effects throughout the brain (Schultz, 1997; Yu and Dayan, 2005; Doya, 2008; Nassar et al., 2012).”
BackgroundSunk Cost Effect in Individuals with Autism Spectrum Disorder
Fujino, J., Tei, S., Itahashi, T., Aoki, Y. Y., Ohta, H., Kanai, C., Kubota, M., Hashimoto, R., Nakamura, M., Kato, N., et al. (2018) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-018-3679-6 · Cited by 71
Rupp, H. A., James, T. W., Ketterson, E. D., Sengelaub, D. R., Janssen, E., & Heiman, J. R. (2009) · Hormones and Behavior · doi:10.1016/j.yhbeh.2009.03.005 · Cited by 69
“…a brain region related to reinforcementguided decision making and reward and risk evaluation (reviewed in Balleine, 2007; Cohen et al., 2005; Doya, 2008; Kringelbach and Rolls, 2004; Rushworth et al., 2007), including financial decision making (Dreher et al., 2007), and attractiveness…”
BackgroundMethodBeyond Being Real: A Sensorimotor Control Perspective on Interactions in Virtual Reality
Abtahi, P., Hough, S. Q., Landay, J. A., & Follmer, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517706 · Cited by 76
“Redundancy in the sensorimotor system ensures robustness [46] such that elimination of a feedback has minor efects on motor behavior [40]; however, perturbations of the same signal may sig-”
BackgroundOptimality and Robustness of a Biophysical Decision-Making Model under Norepinephrine Modulation
Eckhoff, P. A., Wong‐Lin, K., & Holmes, P. J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5024-08.2009 · Cited by 72
Cohen‐Hatton, S. R., Butler, P. C., & Honey, R. C. (2015) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815578266 · Cited by 77
“Alternatively, the basis for an action might be more reflexive and automatic, affected by previously established associations that have developed between situational cues, actions, and outcomes (e.g., Doya, 2008).”
BackgroundVentura, R., Coccurello, R., Andolina, D., Latagliata, E. C., Zanettini, C., Lampis, V., Battaglia, M., D'Amato, F. R., & Moles, A. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs145 · Cited by 68
“Both rapid orientation of attentive resources to novel events and increased NE release have been reported in response to all known experimental stressors (Dayan and Yu 2006; Doya 2008).”
Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning
Gustafson, N. J., & Daw, N. D. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002235 · Cited by 61
Decision neuroscience: neuroeconomics
Smith, D. V., & Huettel, S. A. (2010) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.73 · Cited by 79
Medial prefrontal cortex and the adaptive regulation of reinforcement learning parameters
Khamassi, M., Enel, P., Dominey, P. F., & Procyk, E. (2013) · Progress in brain research · doi:10.1016/b978-0-444-62604-2.00022-8 · Cited by 73
No fear no risk! Human risk behavior is affected by chemosensory anxiety signals
Haegler, K., Zernecke, R., Kleemann, A. M., Albrecht, J., Pollatos, O., Brückmann, H., & Wiesmann, M. (2010) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2010.09.019 · Cited by 65
“Neural substrates being involved in realife decision-making, more precisely in high risk behavior, showed rain activation in amygdala, thalamus, cingulate cortex, dorsolatral prefrontal cortex, cerebellum, and anterior insula (Bush et al., 002;Doya, 2008; Ernst et al., 2002; Paulus et al., 2003;Rogers et al., 004; Thu…”
BackgroundNeural mechanisms and computations underlying stress effects on learning and memory
Lukšys, G., & Sandi, C. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.03.003 · Cited by 68
“Such parameters can have neural correlates [41,42 ] and differ between individuals [43,44 ].”
BackgroundThe neural dynamics of reward value and risk coding in the human orbitofrontal cortex
Li, Y., Vanni-Mercier, G., Isnard, J., Mauguière, F., & Dreher, J.-C. (2016) · Brain · doi:10.1093/brain/awv409 · Cited by 66
“The orbitofrontal cortex (OFC) is known to code at least three different types of reward-related information: reward probability, risk and experienced value (or outcome value) (Doya, 2008; Rushworth and Behrens, 2008; Peters and Büchel, 2010; Padoa-Schioppa and Cai, 2011; Levy and Glimcher, 2012).”
BackgroundInactivating anterior insular cortex reduces risk taking
Ishii, H., Ohara, S., Tobler, P. N., Tsutsui, K.-I., & Iijima, T. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.2278-12.2012 · Cited by 64
“Recent studies have elucidated the neural basis underlying decision making under risk (for review, see Doya, 2008; Rushworth et al., 2011; Schultz, 2011).”
Backgroundde Visser, L., Baars, A. M., Lavrijsen, M., van der Weerd, C. M. M., & van den Bos, R. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.02.025 · Cited by 65
“thought to play a role during the initial exploratory phase of the task (Bechara et al., 1999; Ernst et al., 2002; Bolla et al., 2004; Ridderinkhof et al., 2004; Brand et al., 2006; van den Bos et al., 2006a, 2007; Doya, 2008; Lawrence et al., 2009; Li et al., 2010).”
BackgroundSadacca, B. F., Wikenheiser, A. M., & Schoenbaum, G. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2016.04.017 · Cited by 62
“…of that choice, or between salient outcomes and their predictors in the environment), computational theories of tonic NE function note that this uncertainty signal may play a role in driving behavioral change to cope with a changing environment (Yu and Dayan, 2005, Cohen et al., 2007, Doya, 2008).”
When mere action versus inaction leads to robust preference change.
Chen, Z., Holland, R. W., Quandt, J., Dijksterhuis, A., & Veling, H. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000158 · Cited by 70
Decision-making: from neuroscience to neuroeconomics—an overview
Serra, D. (2021) · Theory and Decision · doi:10.1007/s11238-021-09830-3 · Cited by 67
“…their studies revealed that highly localized clusters of cell bodies synthesized compounds including dopamine and serotonin, that these cells sent those compounds down their axons, and that these cells released those compounds from their terminals in response to depolarization (Doya, 2008).”
BackgroundMotivation and timing: Clues for modeling the reward system
Galtress, T., Marshall, A. T., & Kirkpatrick, K. (2012) · Behavioural Processes · doi:10.1016/j.beproc.2012.02.014 · Cited by 69
“…prediction and valuation (e.g., Peters and Büchel 2010, 2011), interval timing (e.g., Coull, Cheng, and Meck 2011), and decision-making (e.g., Doya 2008), the following sections summarize such evidence to support the interaction and integration of the reward-processing, interval-timing, and…”
BackgroundInfluentialPayzan-LeNestour, É., & Bossaerts, P. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00150 · Cited by 54
“The current study proposes a novel hypothesis about exploration that helps reconcile the findings of Payzan-LeNestour and Bossaerts (2011) and Frank et al. (2009)/Cavanagh et al.”
BackgroundResultAging and the neuroeconomics of decision making: A review
Brown, S. B. R. E., & Ridderinkhof, K. R. (2009) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.9.4.365 · Cited by 70
“In a similar vein, Doya (2008) presents a number of formal mathematical operationalizations of factors associated with decision making (e.g., the value of a given reward), linking neurotransmitter systems and brain circuits to these computational factors.”
BackgroundInfluentialA general theory of intertemporal decision-making and the perception of time
Namboodiri, V. M. K., Mihalaş, Ş., Marton, T. M., & Shuler, M. G. H. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00061 · Cited by 67
“It must also be noted that other known variables influencing subjective value like satiety (Stephens and Krebs, 1986; Doya, 2008), the non-linear utility of reward magnitudes (Stephens and Krebs, 1986; Doya, 2008) and the non-linear dependence of health/fitness on reward rates (Stephens and Krebs,…”
BackgroundNeural markers of loss aversion in resting-state brain activity
Canessa, N., Crespi, C., Baud‐Bovy, G., Dodich, A., Falini, A., Antonellis, G., & Cappa, S. F. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.11.050 · Cited by 68
“Those data supported the view of loss aversion as a stable behavioral response reflecting the structural properties of the network of reinforcement learning (Doya, 2008).”
BackgroundAge modulates the effect of COMT genotype on delay discounting behavior
Smith, C. T., & Boettiger, C. A. (2012) · Psychopharmacology · doi:10.1007/s00213-012-2653-9 · Cited by 59
“A variety of evidence links delay-discounting to dopamine (DA) and DA-modulated frontostriatal circuits (Adriani et al. 2009; Altamirano et al. 2011; Boettiger et al. 2007; Doya 2008; Kobayashi and Schultz 2008; Lee et al. 2009; Mitchell et al. 2007; Paloyelis et al. 2010; Pine et al. 2010).”
BackgroundЖаворонок, А., Popelo, O. V., Щур, Р., Ostrovska, N., & Кордзая, Н. (2022) · Ingénierie des systèmes d information · doi:10.18280/isi.270411 · Cited by 65
“Peculiarities of changes in the financial behavior of households in the regional aspect have been studied in the scientific works of such scientists as [27-33], the study of which made it possible to form a holistic view of the financial behavior of households, both at the macroeconomic level (market of financial serv…”
BackgroundCohen‐Hatton, S. R., & Honey, R. C. (2015) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000061 · Cited by 59
Pharmacological evidence for the implication of noradrenaline in effort
Borderies, N., Mattioni, J., Bornert, P., Gilardeau, S., & Bouret, S. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000793 · Cited by 61
“…to clarify the definition of effort and avoid confusion with other variables such as mere exerted force, risk, or reward, in order to assess the relative contribution of distinct neuromodulatory systems to the distinct components of decision making (Doya, 2008; Dayan, 2012; Walton & Bouret, 2018).”
BackgroundDecision‐making in multiagency multiteam systems operating in extreme environments
Waring, S., Moran, J., & Page, R. (2020) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12309 · Cited by 61
“An alternative explanation is that they rely on reflexive recognition-primed decision (RPD) processes, pattern matching environmental cues to prior experiences to rapidly identify a workable solution (Doya, 2008; House et al., 2014; Klein, 1993, 2008).”
BackgroundRailton, P. (2013) · Inquiry · doi:10.1080/0020174x.2014.858419 · Cited by 60
Dopamine D1Receptors and Nonlinear Probability Weighting in Risky Choice
Takahashi, H., Matsui, H., Camerer, C., Takano, H., Kodaka, F., Ideno, T., Okubo, S., Takemura, K., Arakawa, R., Eguchi, Y., et al. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.3933-10.2010 · Cited by 63
“…stimulation in the striatum inhibits response to avoid negative outcomes (Frank et al., 2007), and other neurotransmitters such as serotonin and noradrenaline are thought to be involved in the complex decision-making process (Trepel et al., 2005; Frank et al., 2007; Cools et al., 2008; Doya, 2008).”
Backgroundvan den Bos, R., Davies, W., Dellu-Hagedorn, F., Goudriaan, A. E., Granon, S., Homberg, J. R., Rivalan, M., Swendsen, J. D., & Adriani, W. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.07.005 · Cited by 58
Separable gain control of ongoing and evoked activity in the visual cortex by serotonergic input
Azimi, Z., Barzan, R., Spoida, K., Surdin, T., Wollenweber, P., Mark, M. D., Herlitze, S., & Jancke, D. (2020) · eLife · doi:10.7554/elife.53552 · Cited by 61
“Therefore, they can crucially influence the formation of perceptual events (Cassidy et al., 2018; Pinto et al., 2013; Yu and Dayan, 2002) and higher cognitive functions (Doya, 2008; Hesselmann et al., 2008; Hurley et al., 2004; Sadaghiani et al., 2010).”
BackgroundEmotional modulation of urgent and evaluative behaviors in risky driving scenarios
Megías‐Robles, A., Maldonado, A., Cândido, A., & Catena, A. (2010) · Accident Analysis & Prevention · doi:10.1016/j.aap.2010.10.029 · Cited by 60
Slater, C. L., Liu, Y., Weiss, E., Yu, K., & Wang, Q. (2022) · Brain Sciences · doi:10.3390/brainsci12070890 · Cited by 59
Circuit Analysis of a Drosophila Dopamine Type 2 Receptor That Supports Anesthesia-Resistant Memory
Scholz-Kornehl, S., & Schwärzel, M. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.4475-15.2016 · Cited by 58
“DARs converge onto intracellular cAMP levels in an antithetic fashion; whereas D1-type receptors augment cAMP levels, D2-type receptors can abate it (Doya, 2008; Nakano et al., 2010; Nutt et al., 2015).”
BackgroundAge-related loss of noradrenergic neurons in the brains of triple transgenic mice
Manaye, K. F., Mouton, P. R., Xu, G., Drew, A., Lei, D.-L., Sharma, Y., Rebeck, G. W., & Turner, S. (2013) · AGE · doi:10.1007/s11357-011-9343-0 · Cited by 54
“2000) and tyrosine hydroxylase-immunopositive (TH+) neurons in the locus coeruleus (LC) (Chan-Palay and Asan 1989; German et al. 1992; Manaye et al. 1995; Busch et al. 1997; Berridge and Waterhouse 2003; Yu and Dayan 2005; Doya 2008).”
BackgroundDopaminergic Neurons Encode a Distributed, Asymmetric Representation of Temperature in Drosophila
Tomchik, S. M. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.3933-12.2013 · Cited by 60
“, 2005), motor control (Hauber, 1998), motivation (Wise, 2004), addiction and obesity (Kenny, 2011; Lüscher and Malenka, 2011), and salience-based decisionmaking (Zhang et al., 2007; Doya, 2008).”
BackgroundMoustafa, A. A., Kéri, S., Herzallah, M. M., Myers, C. E., & Gluck, M. A. (2010) · Brain and Cognition · doi:10.1016/j.bandc.2010.07.013 · Cited by 55
Goal-Driven Cognition in the Brain: A Computational Framework
O’Reilly, R. C., Hazy, T. E., Mollick, J., Mackie, P., & Herd, S. A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.7591 · Cited by 31
“It is plausible that other neuromodulators may expand the dimensionality of the space (e.g., Doya, 2002Doya, 2008), but even there, v/mPFC is a very large space that has plenty of room for very high dimensional representations of affective states and goals.”
BackgroundVolition und kognitive Kontrolle
Goschke, T. (2016) · Springer eBooks · doi:10.1007/978-3-642-53898-8_9 · Cited by 58
Abnormal white matter integrity and decision-making deficits in alcohol dependence
Zorlu, N., Gelal, F. M., Kuserli, A., Cenik, E., Durmaz, E., Sarıçiçek, A., & Gülseren, Ş. (2013) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2013.06.014 · Cited by 55
Bernacer, J., Martinez-Valbuena, I., Martinez, M., Pujol, N., Luis, E. O., Ramirez-Castillo, D., & Pastor, M. A. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116181 · Cited by 49
The neural basis of impulsive discounting in pathological gamblers
Miedl, S. F., Wiswede, D., Marco‐Pallarés, J., Ye, Z., Fehr, T., Herrmann, M., & Münte, T. F. (2015) · Brain Imaging and Behavior · doi:10.1007/s11682-015-9352-1 · Cited by 52
“A likely interpretation of this pattern of results is that PG, in order to choose the LDR, have to recruit prefrontal areas that have often been associated with executive control processes (Doya 2008; Krämer et al. 2007; Marco-Pallares et al. 2008; Miller and Cohen 2001).”
Identification of animal behavioral strategies by inverse reinforcement learning
Yamaguchi, S., Naoki, H., Ikeda, M., Tsukada, Y., Nakano, S., Mori, I., & Ishii, S. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006122 · Cited by 61
Informing deep neural networks by multiscale principles of neuromodulatory systems
Mei, J., Muller, E. B., & Ramaswamy, S. (2022) · Trends in Neurosciences · doi:10.1016/j.tins.2021.12.008 · Cited by 53
Niyogi, R. K., & Wong-Lin, K. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003099 · Cited by 47
“Perceptual decisionmaking is not a simple standalone feed-forward sensorimotor integration process, but is distributed and subjected to various neuromodulatory or cognitive control processes, possibly to enhance decision performance [1,4–12].”
BackgroundEffects of α-2A adrenergic receptor agonist on time and risk preference in primates
Kim, S., Bobeica, I., Gamo, N. J., Arnsten, A. F. T., & Lee, D. (2012) · Psychopharmacology · doi:10.1007/s00213-011-2520-0 · Cited by 47
“The network of brain areas involved in decision making are diffusely innervated by multiple neurochemical systems, including dopamine, norepinephrine, and serotonin (Doya 2008).”
BackgroundInfluentialSmith, C. T., Wallace, D. L., Dang, L. C., Aarts, E., Jagust, W. J., D’Esposito, M., & Boettiger, C. A. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00261.2015 · Cited by 49
How metacontrol biases and adaptivity impact performance in cognitive search tasks
Mekern, V., Sjoerds, Z., & Hommel, B. (2018) · Cognition · doi:10.1016/j.cognition.2018.10.001 · Cited by 48
“Moreover, besides studying the role of dopamine or other neuromodulators (Avery & Krichmar, 2017; Doya, 2008; Goschke & Bolte, 2014) using more direct measures, recent insights have suggested other neural mechanisms of interest in cognitive control.”
de Gee, J. W., Corrêa, C. M. C., Weaver, M., Donner, T. H., & van Gaal, S. (2021) · Cerebral Cortex · doi:10.1093/cercor/bhab032 · Cited by 53
“…decisions, the role of these neuromodulatory systems might be to broadcast infor-mationaboutmomentaryuncertaintyandsurprise(Aston-Jones and Cohen 2005; Bouret and Sara 2005; Yu and Dayan 2005; Parikh et al. 2007; Doya 2008; Lak et al. 2017), a process that can be read out by monitoring pupil size.”
Social impact and governance of AI and neurotechnologies
Doya, K., Ema, A., Kitano, H., Sakagami, M., & Russell, S. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.05.012 · Cited by 55
Self-citationKoot, S., Baars, A., Hesseling, P., van den Bos, R., & Joëls, M. (2013) · Neuropharmacology · doi:10.1016/j.neuropharm.2013.02.008 · Cited by 46
Wagels, L., Votinov, M., Radke, S., Clemens, B., Montag, C., Jung, S., & Habel, U. (2017) · Human Brain Mapping · doi:10.1002/hbm.23685 · Cited by 49
“During decision making, neurotransmitters have been proposed to operate on different levels [Doya, 2008] modulating reward expectation (dopamine), the urgency for direct reward (serotonin) and exploration during uncertain situations (norepinephrine).”
BackgroundHanakawa, T., Goldfine, A. M., & Hallett, M. (2017) · eNeuro · doi:10.1523/eneuro.0200-17.2017 · Cited by 49
“…in a variety of tasks including, but not limited to, behavioral initiation/switch (Crinion et al., 2006), reinforcement/conditional learning (Doya, 2008; Koralek et al., 2012), sequential behaviors (Brotchie et al., 1991; Desrochers et al., 2010), timeconstrained decision-making (Glimcher,…”
BackgroundDynamic prediction during perception of everyday events
Eisenberg, M. L., Zacks, J. M., & Flores, S. E. (2018) · Cognitive Research Principles and Implications · doi:10.1186/s41235-018-0146-z · Cited by 49
“A previous report (Eisenberg & Zacks, 2016) characterized the effect of event boundaries on the size and frequency of eye movements, and on pupil diameter.”
MethodIglesias, S., Kasper, L., Harrison, S. J., Manka, R., Mathys, C., & Stephan, K. E. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117590 · Cited by 45
“…transmitters – such s dopamine, acetylcholine, serotonin or noradrenaline – might serve to s s w p 1 t ( H G H d ( s d t t r g w s t i g m a s c S c i I B o q ignal distinct computational quantities has a long history (for reviews, ee Doya, 2008 ; Dayan, 2012 ; Iglesias et al., 2017 ).”
BackgroundAltered neural encoding of prediction errors in assault-related posttraumatic stress disorder
Ross, M. C., Lenow, J. K., Kilts, C. D., & Cisler, J. M. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.05.008 · Cited by 46
“However, the weaker encoding of positive PEs in the present context might better reflect the insula's role in learning (Doya, 2008; Preuschoff et al., 2008) rather than salience detection.”
BackgroundJing, J., Vilim, F. S., Cropper, E. C., & Weiss, K. R. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.3855-08.2008 · Cited by 44
“In vertebrates, a variety of neuronal elements with different transmitter phenotypes similarly mediate various aspects of arousal (Robbins and Everitt, 1995; Pfaff, 2006) and thereby contribute to decision making (Doya, 2008).”
BackgroundOkada, K., & Kobayashi, Y. (2013) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2013.00036 · Cited by 43
“, 2004; Harris-Warrick, 2011), and also has a role in regulating motivated behavior (Rompre and Miliaressis, 1985; Kozak et al., 2005; Doya, 2008; Winn, 2008; Wilson et al., 2009; Okada et al., 2011).”
Homeostatic regulation of memory systems and adaptive decisions
Mizumori, S. J. Y., & Jo, Y. S. (2013) · Hippocampus · doi:10.1002/hipo.22176 · Cited by 50
“the brain is driven to resolve uncertainties that arise with prediction errors (Fiorillo et al., 2003; Knill and Pouget, 2004; Doya, 2008; Schultz et al., 2011; Bach and Dolan, 2012; Dayan, 2012)”
BackgroundNeural control of behavioural choice in juvenile crayfish
Liden, W. H., Phillips, M. L., & Herberholz, J. (2010) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2010.1000 · Cited by 44
“The main processes in value-based decision making are recognition of the present situation, determination of the values of competing actions (costs/benefits), action selection (based on valuations) and re-evaluation of the action based on the outcome (Doya 2008).”
BackgroundIntegrating computation into the mechanistic hierarchy in the cognitive and neural sciences
Elber‐Dorozko, L., & Shagrir, O. (2019) · Synthese · doi:10.1007/s11229-019-02230-9 · Cited by 44
Role of delay-based reward in the spatial cooperation
Wang, X., Nie, S., Jiang, L.-L., Wang, B.-H., & Chen, S. (2016) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2016.08.014 · Cited by 42
Electrocortical components of anticipation and consumption in a monetary incentive delay task
Angus, D. J., Latham, A. J., Harmon‐Jones, E., Deliano, M., Balleine, B. W., & Braddon‐Mitchell, D. (2017) · Psychophysiology · doi:10.1111/psyp.12913 · Cited by 44
“…we note that the particular context in which it has evoked is broadly consistent with the notion of an “eligibility trace,” which is thought to be produced following an action and serves as a form of working memory regarding the behavior that will—or will not—be reinforced by feedback (Doya, 2008).”
BackgroundResultInfluentialReward-driven changes in striatal pathway competition shape evidence evaluation in decision-making
Dunovan, K., Vich, C., Clapp, M., Verstynen, T., & Rubin, J. E. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006998 · Cited by 47
“In parallel, research in neuroscience 31 has shown how the selection of actions, and the use of feedback to modify selection processes, 32 both rely on a common neural substrate: cortico-basal ganglia-thalamic (CBGT) circuits (Doya, 2008; 33 Bogacz and Gurney, 2007; Balleine et al., 2007; Dunovan and Verstynen, 2016).”
BackgroundInfluentialLinnet, J. (2019) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2019.109802 · Cited by 47
Pascucci, T., Andolina, D., Mela, I. L., Conversi, D., Latagliata, E. C., Ventura, R., Puglisi‐Allegra, S., & Cabib, S. (2009) · The International Journal of Neuropsychopharmacology · doi:10.1017/s1461145709990381 · Cited by 48
“attentive resources to novel events and increased NE release has been reported in response to all known experimental stressors (Dayan & Yu, 2006 ; Doya, 2008).”
BackgroundHedging Your Bets by Learning Reward Correlations in the Human Brain
Wunderlich, K., Symmonds, M., Bossaerts, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.07.025 · Cited by 40
“Research in decision neuroscience provides extensive evidence for a neural representation of key decision variables (Doya, 2008) with a focus heretofore on value signals, putative inputs to the decision process such as action or goal values, and representations of expected outcome after a choice…”
Manifold reaching paradigm: how do we handle target redundancy?
Berret, B., Chiovetto, E., Nori, F., & Pozzo, T. (2011) · Journal of Neurophysiology · doi:10.1152/jn.01063.2010 · Cited by 45
“In the present reaching to a bar task, one may further speculate that cortex-basal ganglia network (see Doya 2008 for a review) would shape the decision/planning process to satisfy desirable physiological values (energy and smoothness).”
BackgroundThe anterior cingulate cortex: Monitoring the outcomes of others' decisions
Apps, M. A. J., Balsters, J. H., & Ramnani, N. (2012) · Social Neuroscience · doi:10.1080/17470919.2011.638799 · Cited by 39
“These areas are commonly considered as parts of a system that allow decisions to be biased by outcomes such as rewards (Doya, 2008).”
BackgroundNakao, T., Miyagi, M., Hiramoto, R., Wolff, A., Gómez‐Pilar, J., Miyatani, M., & Northoff, G. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116015 · Cited by 42
Acute stress enhances tolerance of uncertainty during decision-making
Byrne, K. A., Peters, C., Willis, H. C., Phan, D., Cornwall, A. C., & Worthy, D. A. (2020) · Cognition · doi:10.1016/j.cognition.2020.104448 · Cited by 43
“Such situations are examples of decision-making under uncertainty, or decisions in which individuals must make choices based on unanticipated variation in one's environment or incomplete knowledge about the potential likelihood or outcome of one's choice (Doya, 2008; Hsu et al., 2005; Platt & Huettel, 2008).”
BackgroundConsciousness, decision making, and volition: freedom beyond chance and necessity
Liljenström, H. (2021) · Theory in Biosciences · doi:10.1007/s12064-021-00346-6 · Cited by 44
“Based on the “prediction error”, the assigned values to the choices in the first step are revisited and learnt, possibly resulting in a change of mind (Gold and Shadlen 2007; Doya 2008).”
BackgroundNicotinic alteration of decision-making
Naudé, J., Dongelmans, M., & Faure, P. (2015) · Neuropharmacology · doi:10.1016/j.neuropharm.2014.11.021 · Cited by 39
“Theories that describe how the brain makes decisions are often based on economic theories of how optimal decisions should be made (Vlaev et al., 2011; Doya, 2008; Padoa-Schioppa, 2011; Kable and Glimcher, 2009).”
Zhang, X., & Poo, M. (2010) · Science China Life Sciences · doi:10.1007/s11427-010-0062-z · Cited by 41
Metadata
- Authors
- Doya, K.
- Affiliations
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Doya, Kenji
- ORCID
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2008-03-26
- Volume
- 11
- Issue
- 4
- Pages
- 410-416
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1971976749
- Cited by
- 642
- References
- 93
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 30 · 2023: 15 · 2022: 26 · 2021: 35 · 2020: 34 · 2019: 51 · 2018: 57 · 2017: 31 · 2016: 39 · 2015: 43 · 2014: 43 · 2013: 45 · 2012: 37
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- decision-making, contextual modulation, reward anticipation, risk-taking, uncertainty, delay discounting, dopamine, anterior cingulate cortex, norepinephrine, orbitofrontal cortex, serotonin, dorsal striatum, dorsal prefrontal cortex, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Decision Making, Environment, Gyrus Cinguli, Humans, Models, Neurological, Models, Psychological, Probability Learning, Reward, Neurotransmitter Agents, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.