Modulators of decision making
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Human and animal decisions are modulated by a variety of environmental and intrinsic contexts. Here I consider computational factors that can affect decision making and review anatomical structures and neurochemical systems that are related to contextual modulation of decision making. Expectation of a high reward can motivate a subject to go for an action despite a large cost, a decision that is influenced by dopamine in the anterior cingulate cortex. Uncertainty of action outcomes can promote risk taking and exploratory choices, in which norepinephrine and the orbitofrontal cortex appear to be involved. Predictable environments should facilitate consideration of longer-delayed rewards, which depends on serotonin in the dorsal striatum and dorsal prefrontal cortex. This article aims to sort out factors that affect the process of decision making from the viewpoint of reinforcement learning theory and to bridge between such computational needs and their neurophysiological substrates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, structures such as the basal ganglia, parietal cortex, and thalamus support procedural learning, visuospatial coordination, and sensory integration (Corbetta and Shulman, 2002; Doya, 2008; Sherman, 2016).”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
In the citing paper
“Basal ganglia enable procedural learning, habit formation, and reinforcement-driven behaviors (Doya, 2008), i.e., reinforcement-based decision-making, allowing operators to execute complex behaviors automatically under pressure.”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Neural recording experiments in animals reveal that midbrain dopamine neurons encode reward prediction error”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122 · cited 1×
“In many animal and human choice experiments, subjects tend to be more sensitive to differences in shorter delays than longer delays, which is well modeled by hyperbolic function”
BackgroundMarkov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · cited 1×
“Norepinephrine is also suggested to be important in regulating the decision to explore alternatives versus exploiting a known resourc”
BackgroundTheory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“Also, functional magnetic resonance imaging (fMRI) experiments using a monetary questionnair”
Method
Research works printed but not linked: 1
- Doya, K., Ishii, S., Pouget, A. & Rao, R. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning (MIT Press, Cambridge, Massachusetts, USA, (2007).
Cited by
Items 201–300 of 688
Quantized Attention-Gated Kernel Reinforcement Learning for Brain–Machine Interface Decoding
Wang, F., Wang, Y., Xu, K., Li, H., Liao, Y., Zhang, Q., Zhang, S., Zheng, X., & Príncipe, J. C. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2493079 · Cited by 39
“Mahmoudi and Sanchez [18] further developed the work with an internal reward represented by neural activity in the nucleus accumbens [19], [20].”
BackgroundLynn, S. K., Wormwood, J. B., Barrett, L. F., & Quigley, K. S. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00952 · Cited by 41
“The ISE Framework explicitly acknowledges that context is an important determinant of decision behavior in vivo that impacts perceivers’ estimates of event likelihood and value (for review see Doya, 2008).”
BackgroundMethodValue and probability coding in a feedback-based learning task utilizing food rewards
Tricomi, E., & Lempert, K. M. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00086.2014 · Cited by 36
“9 VALUE AND PROBABILITY REPRESENTATION 2006).”
InfluentialByrne, K. A., Norris, D. D., & Worthy, D. A. (2016) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0377-0 · Cited by 39
“Dopamine regulates feedback processing and reward learning during decision-making (e.g., Brand, Labudda, & Markowitsch, 2006; Doya, 2008; Rolls, 2000; Schultz, 2006).”
Neural and neurochemical basis of reinforcement-guided decision making
Khani, A., & Rainer, G. (2016) · Journal of Neurophysiology · doi:10.1152/jn.01113.2015 · Cited by 35
“…functions will remain the same as before for each action, the state value function will change for this state of the animal in the T-maze: 2 0.2 4 0.8 3.6 Several computational models have tried to explain reinforcement-guided decision making (Botvinick and Weinstein 2014; Doya 2008; Lee 2008).”
BackgroundInfluentialFrontal theta accounts for individual differences in the cost of conflict on decision making
Pinner, J. F. L., & Cavanagh, J. F. (2017) · Brain Research · doi:10.1016/j.brainres.2017.07.026 · Cited by 39
“…Daunizeau, & Pessiglione, 2012; Schouppe, Demanet, Boehler, Ridderinkhof, & Notebaert, 2014) and known dopaminergic moderation of these effects (Doya, 2008; Salamone, Correa, Farrar, Nunes, & Pardo, 2009; St Onge, Abhari, & Floresco, 2011; Treadway et al., 2012; Wardle, Treadway, Mayo, Zald, &…”
Active Inference, Belief Propagation, and the Bethe Approximation
Schwöbel, S., Kiebel, S., & Marković, D. (2018) · Neural Computation · doi:10.1162/neco_a_01108 · Cited by 36
“However, a strong limitation of classical reinforcement learning models is that they do not take into account the influence of various sources of uncertainty on human behavior (Rushworth & Behrens, 2008; Doya, 2008; Behrens, Woolrich, Walton, & Rushworth, 2007; Yu & Dayan, 2005).”
BackgroundNeural systems for choice and valuation with counterfactual learning signals
Tobia, M. J., Guo, R., Schwarze, U., Böhmer, W., Gläscher, J. P., Finckh, B., Marschner, A., Büchel, C., Obermayer, K., & Sommer, T. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.11.051 · Cited by 41
“A cortical–subcortical system, including the dorsal anterior cingulate cortex (dACC), orbitofrontal cortex (OFC) and ventromedial prefrontal cortex (vmPFC), as well as the striatum and mid-brain structures, is implicated in reward-based learning from expected values and reward PE (Doya, 2008; O'Doherty, 2004).”
The Dual Process model: the effect of cognitive load on the ascription of intentionality
Zucchelli, M. M., Trotti, N. M. A. A., Pavan, A., Piccardi, L., & Nori, R. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1451590 · Cited by 38
“Doya (2008) outlines four stages of the decision-making process: (a) recognize the present situation, assess potential actions and identify internal and external factors influencing them (i.e., the positive or negative outcome valence, likelihood, timeframe - short or long-term, cognitive workload,…”
Individual differences in impulsive and risky choice: Effects of environmental rearing conditions
Kirkpatrick, K., Marshall, A. T., Smith, A. P., Koči, J., & Park, Y. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2014.04.024 · Cited by 41
“Neurobiological studies have indicated partial overlap in the substrates involved in impulsive and risky choice and their component processes of delay, probability, and reward processing (see Doya, 2008; Peters & Büchel, 2011; Platt & Huettel, 2008).”
Ekman, R., Fletcher, A. K., Giota, J., Eriksson, A., Thomas, B., & Bååthe, F. (2021) · Mind Brain and Education · doi:10.1111/mbe.12305 · Cited by 45
“Pre-dictable environments should facilitate the consideration of longer-delayed rewards (Doya, 2008).”
BackgroundAutism - A Neurodevelopmental Journey from Genes to Behaviour
(2011) · InTech eBooks · doi:10.5772/974 · Cited by 36
A dual-process approach to exploring the role of delay discounting in obesity
Price, M. J., Higgs, S., Maw, J., & Lee, M. D. (2016) · Physiology & Behavior · doi:10.1016/j.physbeh.2016.02.020 · Cited by 40
“It is hypothesised to represent impulsive needs and desires, with variation in Q values indicating variation in nonlinear valuation [33].”
InfluentialNarcissism and risky decisions: a neurophysiological approach
Yang, Z., Sedikides, C., Gu, R., Luo, Y. L. L., Wang, Y., & Cai, H. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy053 · Cited by 35
“During that stage, decision-makers evaluate the consequences of their decisions and adjust their behavior patterns according to outcome feedback (Hastie, 2001; Ernst and Paulus, 2005; Doya, 2008).”
BackgroundInternal signal correlates neural populations and biases perceptual decision reports
Carnevale, F., de Lafuente, V., Romo, R., & Parga, N. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1216799109 · Cited by 38
Hilt, P. M., Berret, B., Papaxanthis, C., Stapley, P. J., & Pozzo, T. (2016) · Scientific Reports · doi:10.1038/srep23868 · Cited by 39
Risk-taking and pathological gambling behavior in Huntington’s disease
Kalkhoven, C., Sennef, C., Peeters, A., & van den Bos, R. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00103 · Cited by 38
“Nevertheless, disturbances in the limbic cortico-striatal circuit of HD patients may still promote risky decision-making in situations with uncertain outcome, as demonstrated in the IGT (Doya, 2008).”
BackgroundMethodBraunlich, K., & Seger, C. A. (2012) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.1217 · Cited by 34
Pittaras, E., Faure, A., Leray, X., Moraitopoulou, E., Cressant, A., Rabat, A., Meunier, C. N. J., Fossier, P., & Granon, S. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00171 · Cited by 34
“In all animals, rigidity significantly increases from the two first sessions to the last two [F(1) = 31.”
BackgroundInfluentialDecision-Making in Diabetes Mellitus Type 1
Rustad, J. K., Musselman, D. L., Skyler, J. S., Matheson, D., Delamater, A. M., Kenyon, N. S., Cáceda, R., & Nemeroff, C. B. (2013) · Journal of Neuropsychiatry · doi:10.1176/appi.neuropsych.12010016 · Cited by 32
Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making
Grossman, C. D., & Cohen, J. Y. (2022) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092021-125059 · Cited by 34
Tryptophan Promotes Interpersonal Trust
Colzato, L. S., Steenbergen, L., de Kwaadsteniet, E. W., Sellaro, R., Liepelt, R., & Hommel, B. (2013) · Psychological Science · doi:10.1177/0956797613500795 · Cited by 36
“, 2005), and TRP does induce people to take more risks (Doya, 2008; Murphy et al., 2009).”
Thinking and Emotion: Affective Modulation of Cognitive Processing Modes
Bolte, A., & Goschke, T. (2009) · On thinking · doi:10.1007/978-3-642-03129-8_18 · Cited by 39
Unstable Maternal Environment Affects Stress Response in Adult Mice in a Genotype-Dependent Manner
Segni, M. D., Andolina, D., Luchetti, A., Babicola, L., D'Apolito, L. I., Pascucci, T., Conversi, D., Accoto, A., D'Amato, F. R., & Ventura, R. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhv204 · Cited by 32
Satiation level affects anti-predatory decisions in foraging juvenile crayfish
Schadegg, A. C., & Herberholz, J. (2017) · Journal of Comparative Physiology A · doi:10.1007/s00359-017-1158-8 · Cited by 34
Kelm, M. K., & Boettiger, C. A. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00464 · Cited by 32
“Studies in animal models strongly implicate DA in regulating immediate reward bias, generally finding that elevating DA reduces immediate reward bias (Doya, 2008).”
BackgroundThe effects of acute fluoxetine administration on temporal discounting in youth with ADHD
Carlisi, C. O., Chantiluke, K. C., Norman, L. J., Christakou, A., Barrett, N., Giampietro, V., Brammer, M. J., Simmons, A., & Rubia, K. (2015) · Psychological Medicine · doi:10.1017/s0033291715002731 · Cited by 32
“2012), where striatal 5-HT levels have been shown to modulate choices of longer, delayed rewards (Schweighofer et al. 2007; Tanaka et al. 2007; Doya, 2008).”
BackgroundStasi, L. L. D., Díaz-Piedra, C., Morales, J. M., Kurapov, A., Tagliabue, M., Bjärtå, A., Megías‐Robles, A., Bernhardsson, J., Paschenko, S., Romero, S. F., et al. (2020) · Accident Analysis & Prevention · doi:10.1016/j.aap.2020.105785 · Cited by 31
Serotonergic modulation of cognitive computations
Doya, K., Miyazaki, K. W., & Miyazaki, K. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.02.003 · Cited by 33
“Introduction Neuromodulators(1) are a subset of neurotransmitters that release diffusely and produce various effects not limited to excitation or inhibition in various time scales [1,2].”
BackgroundSelf-citationMaking decisions with a continuous mind
Scherbaum, S., Dshemuchadse, M., & Kalis, A. (2008) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.8.4.454 · Cited by 31
“, Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004). Prediction of immediate and future rewards dif-”
BackgroundInfluentialStocco, A. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00018 · Cited by 33
“In their basic form, reinforcement learning models of the basal ganglia consider only one parameter in the selection of an action (namely, its expected value), and would not consider other parameters such as the uncertainty of its own estimates (Doya, 2008).”
BackgroundKrebs, R. M., Fias, W., Achten, E., & Boehler, C. N. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.027 · Cited by 33
“…in ACC and LC during conflict processing and their activity reversal in the context of stimulus novelty are consistent with the view that that key structures of the noradrenergic system are involved in the flexible processing of different behaviorally relevant inputs (Doya, 2008; Sara, 2009).”
Models of neuromodulation for computational psychiatry
Iglesias, S., Tomiello, S., Schneebeli, M., & Stephan, K. E. (2017) · WIREs Cognitive Science · doi:10.1002/wcs.1420 · Cited by 26
Decision-Making Dysfunctions of Counterfactuals in Depression: Who Might I have Been?
Howlett, J. R., & Paulus, M. P. (2013) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2013.00143 · Cited by 33
“For example, serotonin may be involved in consideration of delayed rewards (62).”
BackgroundTanaka, S., O’Doherty, J. P., & Sakagami, M. (2019) · Nature Communications · doi:10.1038/s41467-019-11334-2 · Cited by 32
Eckhoff, P. A., Wong‐Lin, K., & Holmes, P. J. (2011) · SIAM Journal on Applied Dynamical Systems · doi:10.1137/090770096 · Cited by 32
“Neuromodulation plays an important role in cognitive processes such as decision making [17], but there is currently no direct connection between cellular-level effects of neuromodulators and low-dimensional models of decision dynamics.”
BackgroundImpact of past experiences on decision-making in autism spectrum disorder
Fujino, J., Tei, S., Itahashi, T., Aoki, Y. Y., Ohta, H., Kubota, M., Hashimoto, R., Nakamura, M., Kato, N., & Takahashi, H. (2019) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-019-01071-4 · Cited by 30
Haines, N., Beauchaine, T. P., Galdo, M., Rogers, A. H., Hahn, H. A., Pitt, M. A., Myung, J. I., Turner, B. M., & Ahn, W. (2020) · Clinical Psychological Science · doi:10.1177/2167702620929636 · Cited by 30
Brunelin, J., & Fecteau, S. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.12.068 · Cited by 29
Decision making in Parkinson's disease: An analysis of the studies using the Iowa Gambling Task
Colautti, L., Iannello, P., Silveri, M. C., & Antonietti, A. (2021) · European Journal of Neuroscience · doi:10.1111/ejn.15497 · Cited by 28
Nakao, T., Kanayama, N., Katahira, K., Odani, M., Ito, Y., Hirata, Y., Nasuno, R., 繁, 尾., Hiramoto, R., Miyatani, M., et al. (2016) · Scientific Reports · doi:10.1038/srep32477 · Cited by 29
“Two-way repeated-measures ANOVA (two decision tasks (IDM, EDM) × two criteria (subjective occupation preference, the average salary of the presented occupation) revealed a significant interaction (F(1, 23) = 124.”
InfluentialShimokawa, T., Suzuki, K., Misawa, T., & Miyagawa, K. (2009) · Neuroscience · doi:10.1016/j.neuroscience.2009.02.079 · Cited by 30
The Iowa Gambling Task and the three fallacies of dopamine in gambling disorder
Linnet, J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00709 · Cited by 30
“However, the evidence of change in dopamine release and hedonic response in substance use disorders is more complex (Volkow et al., 1997, 2002a, 2008; Kalivas and Volkow, 2005).”
BackgroundResultInfluentialChen, G., Yang, B., Chen, J., Zhu, L., Jiang, H., Yu, W., Zang, F., Chen, Y., & Dai, Y. (2017) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2017.11.221 · Cited by 28
Wilkinson, B., Cohen‐Hatton, S. R., & Honey, R. C. (2019) · Journal of Contingencies and Crisis Management · doi:10.1111/1468-5973.12260 · Cited by 27
“…cluding those where the decisions involve high stakes, are time pressured and occur in a context of uncertainty (e.g., Klein, 1993, 2003, 2008; Klein et al., 2003; see also, Doya, 2008; Gigerenzer, 2007; Gureckis & Goldstone, 2006; Hodgkinson & Healey, 2008; Shafir, 1994; Tversky & Kahneman, 1974).”
An entropy modulation theory of creative exploration.
Hills, T. T., & Kenett, Y. N. (2025) · Psychological Review · doi:10.1037/rev0000511 · Cited by 28
Leib, R., Howard, I. S., Millard, M., & Franklin, D. W. (2023) · Comprehensive physiology · doi:10.1002/cphy.c220032 · Cited by 27
Modulation of cue-guided choices by transcranial direct current stimulation
Garofalo, S., Battaglia, S., Starita, F., & di Pellegrino, G. (2021) · Cortex · doi:10.1016/j.cortex.2021.01.004 · Cited by 28
Funamizu, A., Ito, M., Doya, K., Kanzaki, R., & Takahashi, H. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08025.x · Cited by 27
“In addition, many theoretical studies predict that NE controls the exploitation ⁄ exploration balance (Doya, 2002, 2008; Ishii et al., 2002) or uncertainty (Yu & Dayan, 2005), inferring that the LC controls the uncertaintydependent action choice.”
BackgroundSelf-citationStress induced risk-aversion is reverted by D2/D3 agonist in the rat
Morgado, P. F., Marques, F., Ribeiro, B. M., Leite‐Almeida, H., Pêgo, J. M., Rodrigues, A. J., Dalla, C., Kokras, N., Sousa, N., & Cerqueira, J. (2015) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2015.07.003 · Cited by 26
“Research in both animals and humans revealed that the attribution of value, factoring expectation (the balance between value and effort) and uncertainty (the probability of a given outcome), is carried by a network comprising the medial prefrontal cortex (mPFC) and the orbitofrontal cortex (OFC), as well as subcortica…”
Azimirad, V., Ramezanlou, M. T., Sotubadi, S. V., & Janabi‐Sharifi, F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.097 · Cited by 29
General Pavlovian-to-instrumental transfer in humans: Evidence from Bayesian inference
Degni, L., Dalbagno, D., Starita, F., Benassi, M., di Pellegrino, G., & Garofalo, S. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.945503 · Cited by 25
“Although neutral in principle, such cues acquire a motivational value through their repeated pairing with a reinforcer (e.g., a chocolate bar), and may bias future choices, driving our reward-seeking behavior (Behrens et al., 2007; Doya, 2008; Watson et al., 2018).”
BackgroundEmotional Theory of Rationality
Garcés, M., & Morgenstern, L. F. (2019) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2019.00011 · Cited by 27
“We can infer that the less time the nervous system takes to choose the most suitable response (Kahneman and Tversky, 1979; Doya, 2008; Cisek and Kalaska, 2010), albeit inhibitory (Logan et al.”
BackgroundThe neural correlates of hedonic and eudaimonic happiness: An fMRI study
Costa, T., Suardi, A. C., Diano, M., Cauda, F., Duca, S., Rusconi, M. L., & Sotgiu, I. (2019) · Neuroscience Letters · doi:10.1016/j.neulet.2019.134491 · Cited by 26
Kim, H., Kim, T.-K., Kim, J., Park, J. Y., Lee, Y., Kang, M., Kim, K., & Han, P. (2014) · Molecular Brain · doi:10.1186/s13041-014-0077-7 · Cited by 25
Internal-Time Temporal Difference Model for Neural Value-Based Decision Making
Nakahara, H., & Kaveri, S. R. (2010) · Neural Computation · doi:10.1162/neco_a_00049 · Cited by 27
“In fact, the Pavlovian case in the previous section is such an example, although the only dynamic factor was the noise.”
BackgroundMice Gamble for Food: Individual Differences in Risky Choices and Prefrontal Cortex Serotonin
Pittaras, E., Cressant, A., Serreau, P., Bruijel, J., Dellu-Hagedorn, F., Callebert, J., Rabat, A., & Granon, S. (2013) · Journal of Addiction Research & Therapy · doi:10.4172/2155-6105.s4-011 · Cited by 25
“Making decisions encompasses the integration of potential rewards, punishments and uncertainty of outcomes that may occur both in the short or the long-term [1-3].”
BackgroundCohen, D., Teichman, G., Volovich, M., Zeevi, Y., Elbaum, L., Madar, A., Louie, K., Levy, D. J., & Rechavi, O. (2019) · Nature Communications · doi:10.1038/s41467-019-11715-7 · Cited by 26
Jahn, C. I., Varazzani, C., Sallet, J., Walton, M. E., & Bouret, S. (2020) · Cerebral Cortex · doi:10.1093/cercor/bhaa089 · Cited by 28
Latham, A. J. (2022) · Noûs · doi:10.1111/nous.12444 · Cited by 23
Active Forgetting of Olfactory Memories in Drosophila
Berry, J. A., & Davis, R. L. (2014) · Progress in brain research · doi:10.1016/b978-0-444-63350-7.00002-4 · Cited by 26
Claes, N., Vlaeyen, J. W. S., & Crombez, G. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.07.004 · Cited by 26
Pajkossy, P., Szöllősi, Á., Demeter, G., & Racsmány, M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00506 · Cited by 23
“Importantly, the modulatory effect of subcortical areas on the prefrontal cortex is thought to be implemented through neurotransmitter systems which influence neural processing in several areas of the prefrontal cortex (Doya, 2008; Robbins and Arnsten, 2009).”
BackgroundThe computational pharmacology of oculomotion
Parr, T., & Friston, K. J. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05240-0 · Cited by 23
“Like most of the cortex, these areas receive distributed projections from the locus coeruleus and the basal forebrain via the cingulum (Avery and Krichmar 2017; Doya 2008).”
BackgroundIndependent and interacting value systems for reward and information in the human brain
Dezza, I. C., Cleeremans, A., & Alexander, W. H. (2022) · eLife · doi:10.7554/elife.66358 · Cited by 26
“A general organizational principle of reward value computation and comparison in prefrontal cortex (PFC) has accrued widespread empirical support in neuroeconomics and decision neuroscience ( Rangel et al., 2008 ; Doya, 2008 ; Montague et al., 2006 ).”
BackgroundChoice modulates the neural dynamics of prediction error processing during rewarded learning
Peterson, D. A., Lotz, D. T., Halgren, E., B, T. J. S. A., & Poizner, H. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.051 · Cited by 25
“Uncertainty has been postulated to have a significant influence in learning, choice, and their neural correlates (Behrens et al., 2007; Doya, 2008).”
Theory meets pigeons: The influence of reward-magnitude on discrimination-learning
Rose, J., Schmidt, R., Grabemann, M., & Güntürkün, O. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.10.038 · Cited by 30
“Countless research has been dedicated to understanding the computational principles mediating feedback-based learning and numerous models have been devised to describe these principles mathematically [36,8].”
The role of the anterior cingulate cortex in women's sexual decision making
Rupp, H. A., James, T. W., Ketterson, E. D., Sengelaub, D. R., Janssen, E., & Heiman, J. R. (2008) · Neuroscience Letters · doi:10.1016/j.neulet.2008.10.083 · Cited by 27
“Increased activation in the midbrain in response to men depicted as low risk sexual partners is consistent with previous work demonstrating increased midbrain activation in response to rewarding financial decisions [2,8].”
InfluentialBalasubramani, P. P., Chakravarthy, V. S., Ali, M. A., Ravindran, B., & Moustafa, A. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127542 · Cited by 23
“, an increased discounting and myopicity in reward prediction is related to impulsive behavior [19,60,102].”
Is There a Method of Neuroeconomics?
Rustichini, A. (2009) · American Economic Journal Microeconomics · doi:10.1257/mic.1.2.48 · Cited by 22
Okada, K., Nakamura, K., & Kobayashi, Y. (2011) · Neural Plasticity · doi:10.1155/2011/579840 · Cited by 24
“Together with the dopaminergic system, these neuromodulators are proposed to play an important role in gating movement, controlling several forms of attentional behavior [30], and the reinforcement process [28, 31].”
BackgroundA Theoretical Framework for Evaluating Psychiatric Research Strategies
Katahira, K., & Yamashita, Y. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00008 · Cited by 23
“The parameters and variables of computational models are often associated with neuromodulators (Dayan Computational Psychiatry 204 & Huys, 2008; Doya, 2008; Stephan, Iglesias, Heinzle, & Diaconescu, 2015; Yu & Dayan, 2005).”
BackgroundIchijo, H., Nakamura, T., Kawaguchi, M., & Takeuchi, Y. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00595 · Cited by 22
“This proposal is worth considering because the DA and 5-HT systems interact with each other in a complimentary fashion during behavioral flexibility, such that balanced increases in DA and 5-HT levels are correlated with ideal reversal learning performance in the orbitofrontal cortex and the striatum (Doya, 2008; Robb…”
BackgroundFaure, A., Nosjean, A., Pittaras, E., Duchêne, A., Andrieux, A., Gory‐Fauré, S., Charvériat, M., & Granon, S. (2018) · Neuropharmacology · doi:10.1016/j.neuropharm.2018.09.009 · Cited by 23
The Neuroscience of Suicidal Behavior
van Heeringen, K. (2018) · Cambridge University Press eBooks · doi:10.1017/9781316563205 · Cited by 45
Prinz, S., nder, G. G., Hilgers, R.-D., Holtemöller, O., & Vernaleken, I. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00158 · Cited by 24
“Doya (2008) mentioned several general issues that influence an individual’s decision: (1) needs/desires, (2) risk/uncertainty, (3) time spent/left for learning, (4) gains vs. losses, (5) cost/efforts, (6) risk/variance, (7) delay discounting, (8) exploration.”
van Stralen, D., & Mercer, J. T. A. (2020) · Neonatology Today · doi:10.51362/neonatology.today/2020101510126134 · Cited by 21
Applying human learning principles to user-centered IoT systems
Lee, S. W., Prenzel, O., & Bien, Z. (2013) · Computer · doi:10.1109/mc.2012.426 · Cited by 22
Her, E. S., Huh, N., Kim, J., & Jung, M. W. (2016) · Scientific Reports · doi:10.1038/srep27056 · Cited by 21
“= S t a a C t i a R t i a X t i t ( ) [ ( ) ( ) ( )] ( ), (4) i i i i 0”
Steele, C. C., Peterson, J. R., Marshall, A. T., Stuebing, S. L., & Kirkpatrick, K. (2017) · Behavioural Brain Research · doi:10.1016/j.bbr.2017.11.013 · Cited by 23
Evidence for a neural dual-process account for adverse effects of cognitive control
Zink, N., Stock, A., Colzato, L. S., & Beste, C. (2018) · Brain Structure and Function · doi:10.1007/s00429-018-1694-1 · Cited by 21
“what is attended and/ or processed), but regulates how information is processed and responses are selected (Doya 2008; Goschke and Bolte 2014).”
BackgroundDisruption of Nrxn1α within excitatory forebrain circuits drives value-based dysfunction
Alabi, O. O., Davatolhagh, M. F., Robinson, M., Fortunato, M. P., Cifuentes, L. V., Kable, J. W., & Fuccillo, M. V. (2020) · eLife · doi:10.7554/elife.54838 · Cited by 22
“…behaviors engages numerous cognitive and motor processes – reward-relevant cues from the external world must be accurately represented, internal motivational states must be weighed and these diverse input variables integrated by circuits that control eventual motor output (Doya, 2008).”
BackgroundSuárez, S., Eynard, B., & Granon, S. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.650219 · Cited by 22
“…the speed of decision-making processes; it is therefore the combination of the individual’s emotional, motivational and cognitive state as well as the characteristics of the environment that will allow the person to choose one option over another more rapidly (Ernst and Paulus, 2005; Doya, 2008).”
BackgroundApparent sunk cost effect in rational agents
Ott, T., Masset, P., Gouvêa, T. S., & Kepecs, Á. (2022) · Science Advances · doi:10.1126/sciadv.abi7004 · Cited by 21
How preference change induced by mere action versus inaction persists over time
Chen, Z., Holland, R. W., Quandt, J., Dijksterhuis, A., & Veling, H. (2021) · Judgment and Decision Making · doi:10.1017/s1930297500008366 · Cited by 23
Пахомов, А. П., & Sudin, N. (2013) · Cognitive Neurodynamics · doi:10.1007/s11571-013-9249-x · Cited by 21
“…eye-fields (Roesch and Olson 2003), supplementary eye-fields (Stuphorn et al. 2000; Roesch and Olson 2003), superior colliculus (Ikeda and Hikosaka 2003), basal ganglia (Kawagoe et al. 1998; Doya 2008), amygdale (Paton et al. 2006), and centromedian nucleus of the thalamus (Minamimoto et al. 2005).”
BackgroundUeyama, Y. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00119 · Cited by 29
“However, exploratory risk-taking is directly related to uncertainty in decision-making modulation (Doya, 2008), and the decision making itself may directly relate to motor control systems (Selen et al., 2012).”
BackgroundINDIVIDUAL FINANCIAL BEHAVIOR AND DETERMINANTS
Литвинова, В. В. (2020) · Zhurnal Economicheskoj Teorii · doi:10.31063/2073-6517/2020.17-1.19 · Cited by 20
Roy, B. L., Martin‐Krumm, C., Gille, A., Jacob, S., Vigier, C., Laborde, S., Claverie, D., Besnard, S., & Trousselard, M. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1286919 · Cited by 20
“In addition, the combination of an individual’s emotional, motivational, and cognitive states, as well as characteristics specific to the environment, are the elements that enable an individual to choose rapidly one option over another (47, 48).”
BackgroundCollaborative hunting in artificial agents with deep reinforcement learning
Tsutsui, K., Tanaka, R., Takeda, K., & Fujii, K. (2024) · eLife · doi:10.7554/elife.85694 · Cited by 23
Iguchi, Y., Kosugi, S., Nishikawa, H., Lin, Z., Minabe, Y., & Toda, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114024 · Cited by 20
“The reason why we focused on the system was that this system plays a pivotal role in both the process of decision-making in healthy animals/humans [18] and the pathophysiology of schizophrenia, major depression, and drug addiction [19–20].”
BackgroundUncertainty-Dependent Extinction of Fear Memory in an Amygdala-mPFC Neural Circuit Model
Li, Y., Nakae, K., Ishii, S., & Naoki, H. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005099 · Cited by 23
“The brackets ([]+) in the equations represent the learning signals that regulate the gain of synaptic plasticity [42] (blue lines in Fig 1D).”
BackgroundRat intersubjective decisions are encoded by frequency‐specific oscillatory contexts
Borg, J. S., Srivastava, S., Lin, L., Heffner, J., Dunson, D. B., Dzirasa, K., & de Lecea, L. (2017) · Brain and Behavior · doi:10.1002/brb3.710 · Cited by 22
“Decision- making is the result of multiple information processing systems acting in parallel (Barrett & Satpute, 2013; Doya, 2008).”
BackgroundIshizu, K., Nishimoto, S., Ueoka, Y., & Funamizu, A. (2024) · Nature Communications · doi:10.1038/s41467-024-48338-6 · Cited by 19
“In the population level, we observed a clear prior modulation of activity in the mPFC and M2, but less clearin the AC (Figs.3, 4, 5 and Supplementary Fig.”
BackgroundBadioli, M., Degni, L., Dalbagno, D., Danti, C., Starita, F., di Pellegrino, G., Benassi, M., & Garofalo, S. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105829 · Cited by 19
“In many daily life scenarios, environmental cues can guide our decision-making (Dayan and Balleine, 2002; Doya, 2008).”
BackgroundDissociable functions of reward inference in the lateral prefrontal cortex and the striatum
Tanaka, S., Pan, X., Oguchi, M., Taylor, J. E., & Sakagami, M. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00995 · Cited by 22
“Furthermore, a lot of studies support the idea of the existence of a circuit for reinforcement learning in the nigro-striatal circuit (Schultz et al., 1997; Daw et al., 2005; Doya, 2008).”
BackgroundResultCerebellar Grey Matter Volumes in Reactive Aggression and Impulsivity in Healthy Volunteers
Wolfs, E. M. L., Klaus, J., & Schutter, D. J. L. G. (2022) · The Cerebellum · doi:10.1007/s12311-021-01337-5 · Cited by 19
Relative Gains, Losses, and Reference Points in Probabilistic Choice in Rats
Marshall, A. T., & Kirkpatrick, K. (2015) · PLoS ONE · doi:10.1371/journal.pone.0117697 · Cited by 19
Amygdala activity for the modulation of goal-directed behavior in emotional contexts
Maeda, K., Kunimatsu, J., & Hikosaka, O. (2018) · PLoS Biology · doi:10.1371/journal.pbio.2005339 · Cited by 20
The Passenger Effect: Risky Driving is a Function of the Driver‐Passenger Emotional Relationship
Megías‐Robles, A., Cândido, A., Catena, A., Molinero, S., & Maldonado, A. (2013) · Applied Cognitive Psychology · doi:10.1002/acp.2989 · Cited by 20
“For example, medium risk situations are characterized by greater uncertainty, one very influential factor in risk perception (Doya, 2008; Pushkarskaya, Liu, Smithson, & Joseph, 2010).”
Background
Metadata
- Authors
- Doya, K.
- Affiliations
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Doya, Kenji
- ORCID
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2008-03-26
- Volume
- 11
- Issue
- 4
- Pages
- 410-416
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1971976749
- Cited by
- 642
- References
- 93
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 30 · 2023: 15 · 2022: 26 · 2021: 35 · 2020: 34 · 2019: 51 · 2018: 57 · 2017: 31 · 2016: 39 · 2015: 43 · 2014: 43 · 2013: 45 · 2012: 37
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- decision-making, contextual modulation, reward anticipation, risk-taking, uncertainty, delay discounting, dopamine, anterior cingulate cortex, norepinephrine, orbitofrontal cortex, serotonin, dorsal striatum, dorsal prefrontal cortex, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Decision Making, Environment, Gyrus Cinguli, Humans, Models, Neurological, Models, Psychological, Probability Learning, Reward, Neurotransmitter Agents, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.