Modulators of decision making
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Human and animal decisions are modulated by a variety of environmental and intrinsic contexts. Here I consider computational factors that can affect decision making and review anatomical structures and neurochemical systems that are related to contextual modulation of decision making. Expectation of a high reward can motivate a subject to go for an action despite a large cost, a decision that is influenced by dopamine in the anterior cingulate cortex. Uncertainty of action outcomes can promote risk taking and exploratory choices, in which norepinephrine and the orbitofrontal cortex appear to be involved. Predictable environments should facilitate consideration of longer-delayed rewards, which depends on serotonin in the dorsal striatum and dorsal prefrontal cortex. This article aims to sort out factors that affect the process of decision making from the viewpoint of reinforcement learning theory and to bridge between such computational needs and their neurophysiological substrates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, structures such as the basal ganglia, parietal cortex, and thalamus support procedural learning, visuospatial coordination, and sensory integration (Corbetta and Shulman, 2002; Doya, 2008; Sherman, 2016).”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
In the citing paper
“Basal ganglia enable procedural learning, habit formation, and reinforcement-driven behaviors (Doya, 2008), i.e., reinforcement-based decision-making, allowing operators to execute complex behaviors automatically under pressure.”
From Enhancement of modern defense capabilities by convergence of cognitive neuroscience and artificial intelligence (2026) · cites this paper as Doya, 2008
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
Theory of games and economic behavior
(2019) · 100 Years of Math Milestones · doi:10.1090/mbk/121/32 · Cited by 16,632
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“Neural recording experiments in animals reveal that midbrain dopamine neurons encode reward prediction error”
BackgroundMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Golden eggs and hyperbolic discounting
Laibson, D. (1997) · The Quarterly Journal of Economics · doi:10.1162/003355397555253 · Cited by 6,122 · cited 1×
“In many animal and human choice experiments, subjects tend to be more sensitive to differences in shorter delays than longer delays, which is well modeled by hyperbolic function”
BackgroundMarkov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · cited 1×
“Norepinephrine is also suggested to be important in regulating the decision to explore alternatives versus exploiting a known resourc”
BackgroundTheory of Games and Economic Behavior
Hawkins, D. (1945) · Philosophy of Science · doi:10.1086/286866 · Cited by 3,887
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“Also, functional magnetic resonance imaging (fMRI) experiments using a monetary questionnair”
Method
Research works printed but not linked: 1
- Doya, K., Ishii, S., Pouget, A. & Rao, R. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning (MIT Press, Cambridge, Massachusetts, USA, (2007).
Cited by
Items 301–400 of 688
Subliminal determinants of cue-guided choice
Garofalo, S., Sagliano, L., Starita, F., Trojano, L., & di Pellegrino, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-68926-y · Cited by 18
Hales, C. A., Clark, L., & Winstanley, C. A. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105083 · Cited by 19
One thought too few: An adaptive rationale for punishing negligence.
Sarin, A., & Cushman, F. A. (2024) · Psychological Review · doi:10.1037/rev0000476 · Cited by 21
Understanding decision neuroscience: A multidisciplinary perspective and neural substrates
Miyapuram, K. P., & Pammi, V. S. C. (2013) · Progress in brain research · doi:10.1016/b978-0-444-62604-2.00014-9 · Cited by 19
Stimulus Novelty Energizes Actions in the Absence of Explicit Reward
Koster, R., Seow, T. X. F., Dolan, R. J., & Düzel, E. (2016) · PLoS ONE · doi:10.1371/journal.pone.0159120 · Cited by 20
“Exploring novel options is an essential part of adaptive decision making behaviour [1,2], but is also linked to increased risk of addictive behaviour [3,4] and dopaminergic function [5,6].”
BackgroundContradiction in universal and particular reasoning
Medaglia, M. T., Tecchio, F., Seri, S., Lorenzo, G. D., Rossini, P. M., & Porcaro, C. (2009) · Human Brain Mapping · doi:10.1002/hbm.20838 · Cited by 19
“In particular, enhanced activity in the orbitofrontal cortex (OFC) is associated with risk-taking behavior [Doya, 2008].”
Evolutionary development of neural systems in vertebrates and beyond
O’Connell, L. A. (2013) · Journal of Neurogenetics · doi:10.3109/01677063.2013.789511 · Cited by 17
“As information on the molecular and neural substrates of behavior from a diversity of animals becomes available, we will fi nally be able to mechanistically determine the patterns of brain evolution that underlie behavioral diversity.”
BackgroundHassani, S. A., Tiesinga, P. H. E., & Womelsdorf, T. (2024) · Nature Communications · doi:10.1038/s41467-024-54395-8 · Cited by 17
Neiman, T., & Loewenstein, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.2068-12.2013 · Cited by 20
“These mechanisms range from high-level processes in which complex cognitive reasoning determines individual actions (Baron, 2000), through simpler incremental learning of state-action values (Sutton and Barto, 1998; Doya, 2008; Sakai and Fukai, 2008) to even simpler adaptation (Seung, 2003).”
BackgroundBridging the gap between single receptor type activity and whole‐brain dynamics
Jancke, D., Herlitze, S., Kringelbach, M. L., & Deco, G. (2021) · FEBS Journal · doi:10.1111/febs.15855 · Cited by 17
Psychological and Neuroscientific Connections with Reinforcement Learning
Shah, A. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_16 · Cited by 20
From genes to behavior: placing cognitive models in the context of biological pathways
Sáez, I., Set, E., & Hsu, M. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00336 · Cited by 17
“…so in cases where existing models have shown considerable validity at both neurophysiological and molecular levels, as in the case of reinforcement learning models of reward-guided behavior (Schultz et al., 1997; Dayan and Niv, 2008; Doya, 2008; Frank and Fossella, 2011; den Ouden et al., 2013).”
BackgroundExplaining Moral Behavior: A Minimal Moral Model
Osman, M., & Wiegmann, A. (2017) · Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) · doi:10.1027/1618-3169/a000336 · Cited by 19
The Neurobiology of Confidence: From Beliefs to Neurons
Ott, T., Masset, P., & Kepecs, Á. (2018) · Cold Spring Harbor Symposia on Quantitative Biology · doi:10.1101/sqb.2018.83.038794 · Cited by 19
“…formalize cognitive processes as a series of well-defined computations with variables that serve as proxies for those that are unobservable (e.g., confidence, value, or attention) (Sugrue et al. 2005; Daw and Doya 2006; Corrado and Doya 2007; Doya 2008; Reynolds and Heeger 2009; Carrasco 2011).”
BackgroundDezfouli, A., Morris, R. W., Ramos, F., Dayan, P., & Balleine, B. W. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/328849 · Cited by 23
“The role of different brain regions in this circuitry has been the subject of extensive research in the past [Gold and Shadlen, 2007, Doya, 2008], with one of the main challenges being that decisions – and thus the neural activity that causes them – are not only affected by the immediate events in…”
BackgroundDefining the r factor for post-trauma resilience and its neural predictors
van Rooij, S. J. H., Santos, J. L., Hinojosa, C. A., Ely, T. D., Harnett, N. G., Murty, V. P., Lebois, L. A. M., Jovanović, T., House, S. L., Bruce, S. E., et al. (2024) · Nature Mental Health · doi:10.1038/s44220-024-00242-0 · Cited by 16
A Reward-Maximizing Spiking Neuron as a Bounded Rational Decision Maker
Leibfried, F., & Braun, D. A. (2015) · Neural Computation · doi:10.1162/neco_a_00758 · Cited by 16
“…and psychological sciences, a vast literature uses the framework of reinforcement learning to interpret behavioral and neural data (Montague, Dayan, & Se-jnowski, 1996; Daw & Doya, 2006; Johnson, van der Meer, & Redish, 2007; O’Doherty, Hampton, & Kim, 2007; Doya, 2008; Rushworth & Behrens, 2008).”
BackgroundSato, N., Shidara, H., & Ogawa, H. (2019) · Scientific Reports · doi:10.1038/s41598-019-54555-7 · Cited by 18
Conductas urgentes y evaluativas en función del nivel de riesgo en situaciones de conducción
Megías‐Robles, A., López-Riañez, M., & Cândido, A. (2013) · Anales de Psicología · doi:10.6018/analesps.29.3.145451 · Cited by 16
Comparative Analysis of Behavioral Models for Adaptive Learning in Changing Environments
Marković, D., & Kiebel, S. J. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00033 · Cited by 19
“Numerous studies investigated computational principles and neuronal mechanism that underlie human decision making in changing environments (Angela, 2007; Behrens et al., 2007; Doya, 2008; Summerfield et al., 2011; Payzan-LeNestour et al., 2013).”
BackgroundCanonical cortical circuits and the duality of Bayesian inference and optimal control
Doya, K. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.07.003 · Cited by 16
Self-citationSocial Dominance Representations in the Human Brain
Ligneul, R., & Dreher, J. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805308-9.00017-8 · Cited by 17
Modeling sensory-motor decisions in natural behavior
Zhang, R., Zhang, S., Tong, M. H., Cui, Y., Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006518 · Cited by 15
“Note Eq (4) converts value function back to its simplest 222 form in [15].”
BackgroundZink, N., Bensmann, W., Arning, L., Colzato, L. S., Stock, A., & Beste, C. (2019) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyz024 · Cited by 15
“The central question here is how metacontrol is achieved, that is, how cognitive control is adapted or regulated in situations with different control requirements (Doya, 2008; Goschke and Bolte, 2014).”
BackgroundSLO potassium channels antagonize premature decision making in C. elegans
Aoki, I., Tateyama, M., Shimomura, T., Ihara, K., Kubo, Y., Nakano, S., & Mori, I. (2018) · Communications Biology · doi:10.1038/s42003-018-0124-5 · Cited by 15
“Doya, K. (2008). Modulators of decision making. Nat Neurosci 11, 410–416. Farboud, B., and Meyer, B.J. (2015). Dramatic enhancement of genome editing by CRISPR/cas9 through improved guide RNA design. Genetics 199, 959–971. Franceschetti, S., Lavazza, T., Curia, G., Aracri, P., Panzica, F., Sancini, G., Avanzini, G., a…”
BackgroundInfluentialVich, C., Dunovan, K., Verstynen, T., & Rubin, J. E. (2019) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2019.105048 · Cited by 18
Amita, H., & Matsushima, T. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2014.04.004 · Cited by 17
“To understand choice impulsiveness in a comprehensive manner, specifying the brain areas and networks that are specifically responsible for the choice of delayed reward is critical [17-19].”
Zink, N., Stock, A., Vahid, A., & Beste, C. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00411 · Cited by 15
“In this context, the central question is how information is processed and responses are selected (Doya, 2008; Goschke and Bolte, 2014) in situations with different control requirements.”
BackgroundGianelli, C., Basso, G., Manera, M. R., Poggi, P., & Canessa, N. (2021) · Addiction Biology · doi:10.1111/adb.13088 · Cited by 14
“The possible existence of a neuro-structural basis of patients' decreased LA was confirmed by their widespread atrophic pattern, encompassing regions involved both in executive functioning(19) and adaptive behavioural learning.(5) In line with a recent meta-analysis of VBM studies,(36) decreased GM density was found,…”
BackgroundResultFlexible Behavioral Decision Making of Mobile Robot in Dynamic Environment
Zhu, J., Wang, D., & Si, J. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3149602 · Cited by 14
“It shows that raphe nucleus (RN) synthesizes the serotonin (5-HT), basal forebrain (BF) synthesizes the ACh, and LC synthesizes the NE, while ventral tegmental area (VTA) and substantia nigra (SN) synthesizes the dopamine (DA) [9], [10].”
BackgroundA cortical network model of cognitive and emotional influences in human decision making
Nazir, A. H., & Liljenström, H. (2015) · Biosystems · doi:10.1016/j.biosystems.2015.07.004 · Cited by 17
Contradictory Reasoning Network: An EEG and fMRI Study
Porcaro, C., Medaglia, M. T., Thai, N. J., Seri, S., Rotshtein, P., & Tecchio, F. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092835 · Cited by 15
“These include the left precuneus and inferior parietal (bilateral angular) cortex, both associated with reading and processing of visual-verbal stimuli [48]; the bilateral prefrontal cortices (superior and medial frontal gyri), associated with executive functions, working memory and decision making processes [49–51];…”
BackgroundA novel risk-based decision-making paradigm
Morgado, P. F., Marques, F., Silva, M. B., Sousa, N., & Cerqueira, J. (2014) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2014.00045 · Cited by 16
“…of reward, three of the main components of decision-making which have been mapped, respectively, to the loop between the NAcc and the OFC (Doya, 2008), the OFC and ACC (Tremblay and Schultz, 1999; Gehring and Willoughby, 2002; Padoa-Schioppa and Assad, 2006) and the dorsal striatum…”
BackgroundInfluentialLowe, R. J., & Billing, E. (2017) · Adaptive Behavior · doi:10.1177/1059712316682999 · Cited by 15
“We present a neural network architecture that, in computing the three critical dimensions of reinforcement identified by Doya (2008) (i) magnitude, (ii) probability, (iii) (delay) timing, of reinforcement, provides an important theoretical and computational model of the ATP theory, i.”
BackgroundMethodResultInfluentialEradicating war memories: Neuroscientific reality and ethical concerns
Kroes, M. C. W., & Liivoja, R. (2018) · International Review of the Red Cross · doi:10.1017/s1816383118000437 · Cited by 17
Do people really avoid effort? A cost – benefit perspective on the principle of least effort
André, N., Baumeister, R. F., Gendolla, G. H. E., & Audiffren, M. F. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106587 · Cited by 13
Basal ganglia-cortical connectivity underlies self-regulation of brain oscillations in humans
Kasahara, K., DaSalla, C. S., Honda, M., & Hanakawa, T. (2022) · Communications Biology · doi:10.1038/s42003-022-03665-6 · Cited by 13
“indicate the involvement of the BgCN (44-46) and, possibly, the cerebellar-basal ganglia circuit (47, 48) in successful modulation of SMRs.”
BackgroundRutar, D., Colizoli, O., Selen, L. P. J., Spieß, L., Kwisthout, J., & Hunnius, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0270619 · Cited by 18
Prospective and retrospective temporal difference learning
Dayan, P. (2009) · Network Computation in Neural Systems · doi:10.1080/09548980902759086 · Cited by 16
“Standard temporal difference (TD) learning schemes (Sutton 1988) in reinforcement learning have recently been omnipresent as formal accounts of animal and human decision-making (e.g., (Daw and Doya 2006; Johnson et al. 2007; Doya 2008)).”
BackgroundThe Neurobiology of Preferences
Symmonds, M., & Dolan, R. J. (2011) · Elsevier eBooks · doi:10.1016/b978-0-12-381431-9.00001-2 · Cited by 14
Reward magnitude tracking by neural populations in ventral striatum
Fiallos, A. M., Bricault, S., Cai, L. X., Worku, H. A., Colonnese, M. T., Westmeyer, G. G., & Jasanoff, A. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.10.036 · Cited by 13
“Human fMRI studies have repeatedly implicated ventral striatum as one of a number of areas involved in processing rewards (Doya, 2008).”
BackgroundBehavior Decision of Mobile Robot With a Neurophysiologically Motivated Reinforcement Learning Model
Wang, D., Chen, S., Hu, Y., Liu, L., & Wang, H. (2020) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3035778 · Cited by 13
“Inaddition, Doya [56], [57] also suggested that DA is associated with reward prediction, while serotonin is associated with temporal discounting.”
BackgroundHypothalamic-Extended Amygdala Circuit Regulates Temporal Discounting
Li, H. E., Rossi, M. A., Watson, G. D. R., Moore, H. G., Cai, M. T., Kim, N., Vokt, K. A., Lu, D., Bartholomew, R. A., Hughes, R. N., et al. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.1836-20.2020 · Cited by 13
Ludwiczak, A., Osman, M., & Jahanshahi, M. (2020) · Behavioural Brain Research · doi:10.1016/j.bbr.2020.112474 · Cited by 14
Brain Imaging: Healthy Networks for Suicide Prevention
van Heeringen, K. (2014) · Crisis · doi:10.1027/0227-5910/a000213 · Cited by 13
Evaluating Visual Cues Modulates Their Representation in Mouse Visual and Cingulate Cortex
Wal, A., Klein, F. J., Born, G., Busse, L., & Katzner, S. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.1828-20.2021 · Cited by 12
“ACC is important for reward-guided action selection (Doya, 2008; Kolling et al., 2016; Shenhav et al., 2016).”
BackgroundZaupa, M., Nagaraj, N., Sylenko, A., Baier, H., Sawamiphak, S., & Filosa, A. (2024) · Neuron · doi:10.1016/j.neuron.2024.01.001 · Cited by 12
“16,17 A number of these neurotransmitters were shown to play crucial roles also in decision-making in rodents, 54 and possibly humans.”
BackgroundSex differences in motivational biases over instrumental actions
Degni, L., Garofalo, S., Finotti, G., Starita, F., Robbins, T. W., & di Pellegrino, G. (2024) · npj Science of Learning · doi:10.1038/s41539-024-00246-6 · Cited by 13
Matsunami, S., Ogura, Y., Amita, H., Izumi, T., Yoshioka, M., & Matsushima, T. (2012) · Behavioural Brain Research · doi:10.1016/j.bbr.2012.05.045 · Cited by 12
“Taking these experimental findings into account, Doya [17] proposes an integrated framework in which neuromodulators (5-HT, dopamine and noradrenalin) affect economical decision making.”
MethodRuocco, L. A., Carnevale, U. A. G., Sadile, A. G., Sica, A., Arra, C., Maio, A. D., Topo, E., & D’Aniello, A. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.11.029 · Cited by 13
Inattention and Uncertainty in the Predictive Brain
Kujala, T., & Lappi, O. (2021) · Frontiers in Neuroergonomics · doi:10.3389/fnrgo.2021.718699 · Cited by 16
“…such as the amygdala and the cingulate and orbitofrontal cortices (e.g., Angela and Dayan, 2005: acetylcholine and norepinephrine signals, Doya, 2008: norepinephrine and the orbitofrontal cortex, Rushworth and Behrens, 2008: prefrontal and cingulate cortex, Payzan-LeNestour et al., 2013:…”
BackgroundZoratto, F., Sinclair, E., Manciocco, A., Vitale, A., Laviola, G., & Adriani, W. (2014) · BioMed Research International · doi:10.1155/2014/927685 · Cited by 13
“…2 Bambino Gesù Children’s Hospital IRCCS, Rome, Italy 3 Istituto di Scienze e Tecnologie della Cognizione, National Research Council (ISTC-CNR), Rome, Italy Correspondence should be addressed to Walter Adriani; walter.adriani@iss.it Received 14 March 2014; Accepted 30 April 2014; Published 27 May…”
BackgroundInfluentialWhat Makes You Go Faster?: The Effect of Reward on Speeded Action under Risk
Chen, X., & Kwak, Y. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01057 · Cited by 15
“Risk, as well as reward amount and delay to reward delivery, is a key variable of economic decision making under uncertainty (Doya, 2008; Platt and Huettel, 2008).”
BackgroundMei, J., Rodriguez-Garcia, A., Takeuchi, D., Wainstein, G., Hubig, N., Mohsenzadeh, Y., & Ramaswamy, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.06762 · Cited by 15
“• 5-HT: Studies have suggested that 5-HT balances exploration and exploitation (Clarke et al., 2004; Dayan and Huys, 2008; Doya, 2008).”
A dynamic, embodied paradigm to investigate the role of serotonin in decision-making
Asher, D. E., Craig, A. B., Zaldivar, A., Brewer, A. A., & Krichmar, J. L. (2013) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2013.00078 · Cited by 11
““fnint-07-00078” — 2013/11/19 — 20:26 — page 1 — #1 REVIEW ARTICLE published: 21 November 2013 doi: 10.3389/fnint.2013.00078”
BackgroundInfluentialNazir, A. H., Kotaleski, J. H., & Liljenström, H. (2023) · Cognitive Neurodynamics · doi:10.1007/s11571-023-10019-3 · Cited by 11
“Decision making process The decision-making process (DM) is described in a simple way by Doya (2008) in five steps: (1) presentation of external/internal stimuli, (2) predicting values of the associated potential decisions, (3) selection of an option, (4) actual outcome evaluation, and finally, (5)…”
The challenge of learning adaptive mental behavior.
Hitchcock, P. F., & Frank, M. J. (2024) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000924 · Cited by 14
Ross, M., Lanyon, L., Viswanathan, J., Manoach, D. S., & Barton, J. J. S. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.08.006 · Cited by 13
“…that, along with orbitofrontal cortex, anterior ingulate cortex forms an important part of the dopamineric cortical-basal ganglia circuit involved in reward proessing (Doya, 2008; Haber and Knutson, 2010; Hewig et l., 2011; Hickey et al., 2010; Sescousse et al., 2010; allis and Kennerley, 2010).”
BackgroundComputational Model of the Transition from Novice to Expert Interaction Techniques
Bailly, G., Khamassi, M., & Girard, B. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3505557 · Cited by 15
“In neuroscience, several studies show similar human (and animal) behaviors where the decision process is not necessarily based on long-term rewards, but also on short-term rewards [32], in particular when a learning eort is required [100] or even without rewards [73], e.”
Ortmann, A. (2008) · Economics and Philosophy · doi:10.1017/s026626710800206x · Cited by 12
“…narrowly localized regions of activations (Anderson 6 Fodor (2000) seems to contradict to some extent his earlier views on the modularity of the brain (Fodor, 1983) although, as Prinz (2006) points out, that maybe partially due to the somewhat misleading choice of the title of the earlier book.”
BackgroundOutcome contingency selectively affects the neural coding of outcomes but not of tasks
Wisniewski, D., Forstmann, B. U., & Braß, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-55887-0 · Cited by 10
Learning as filtering: Implications for spike-based plasticity
Jegminat, J., Surace, S. C., & Pfister, J.-P. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009721 · Cited by 10
“Uncertainty of rewards has been studied in the prefrontal and cingulate cortex based on reinforcement learning models [32]; and several neuro-modulators have been identified that influence choices under uncertainty [33, 34].”
Lowe, R. J., Sandamirskaya, Y., & Billing, E. (2014) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2014.6983021 · Cited by 10
“[20]) compares to that identified by [21]: “the value of a reward given by an action at a state is a function of reward amount, delay and probability.”
BackgroundDickenson, J. A., Diamond, L. M., King, J. B., Jenson, K., & Anderson, J. S. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa058 · Cited by 11
Reading between the lines: Subtle stereotype threat cues can motivate performance
Seitchik, A. E., Jamieson, J. P., & Harkins, S. G. (2012) · Social Influence · doi:10.1080/15534510.2012.746206 · Cited by 13
“…information about the stochastic environment (Rangel, Camerer, &Montague, 2008), and can have myriad negative effects such as eliciting negative affect (e.g., Demerath, 1993), promoting risky decisions (e.g., Doya, 2008), and harming interpersonal interactions (e.g., Richeson & Trawalter, 2005).”
BackgroundA computational knowledge-based model for emulating human performance in the Iowa Gambling Task
Iglesias, A., del Castillo, M. D., Serrano, J. I., & Oliva, J. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.05.008 · Cited by 11
The impact of Pavlovian cues on pain avoidance: A behavioral study
Claes, N., Crombez, G., Franssen, M., & Vlaeyen, J. W. S. (2016) · Learning and Motivation · doi:10.1016/j.lmot.2016.10.001 · Cited by 11
Neural correlates of effort-based behavioral inconsistency
Bernácer, J., Martínez-Valbuena, I., Martínez, M., Pujol, N., Luís, E. O., Ramírez-Castillo, D., & Pastor, M. A. (2018) · Cortex · doi:10.1016/j.cortex.2018.12.005 · Cited by 11
Impaired prediction of ongoing events in posttraumatic stress disorder
Eisenberg, M. L., Rodebaugh, T. L., Flores, S., & Zacks, J. M. (2023) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2023.108636 · Cited by 9
“In fact, functions ranging from object recognition (e.g., Bar et al., 2006), to action guidance (e.g., Grush, 2004), and decision making (e.g., Doya, 2008) all require predictive processing for successful performance.”
BackgroundBastone, A., Mandiello, A., & Zeuli, F. (2026) · European Journal of Innovation Management · doi:10.1108/ejim-03-2025-0388 · Cited by 9
Chen, C., Moyzis, R. K., Lei, X., Chen, C., & Dong, Q. (2016) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199357376.013.22 · Cited by 12
Zhang, P., Chao, L., Chen, Y., Ma, X., Wang, W., He, J., Huang, J., & Li, Q. (2020) · Sensors · doi:10.3390/s20195528 · Cited by 9
“Moreover, Sanchez et al. [38,39] proved that the reward information in the autonomous RLBMI can be represented by the neural activities from the nucleus accumbens, and Marsh et al. [40] further showed that the reward information can also be represented by the neural activities from the primary…”
MethodWilkinson, B., Cohen‐Hatton, S. R., & Honey, R. C. (2021) · Journal of Contingencies and Crisis Management · doi:10.1111/1468-5973.12355 · Cited by 10
Müller, R., & Urbas, L. (2017) · Chemie Ingenieur Technik · doi:10.1002/cite.201700029 · Cited by 10
Building on Social Science: Theoretic Foundations for Modelers
Nyblade, B., O’Mahony, A., & Sieck, K. (2019) · Wiley eBooks · doi:10.1002/9781119485001.ch4 · Cited by 9
Lowe, R. J., Almér, A., & Balkenius, C. (2019) · Open Information Science · doi:10.1515/opis-2019-0017 · Cited by 12
D’Iorio, A., Guida, P., Maggi, G., Redgrave, P., Santangelo, G., & Obeso, I. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.04.003 · Cited by 9
Canessa, N., Basso, G., Poggi, P., & Gianelli, C. (2022) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2022.108258 · Cited by 9
Serrano, J. I., Iglesias, Á., Woods, S. P., & del Castillo, M. D. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117795 · Cited by 9
Bernal, F. A., Salgueiro, T. A., Brzostowski, A., Recart, E., Caramés, M. A., Pérez, J. M., Furman, D., Graziano, M., & Larrosa, P. N. F. (2022) · Scientific Reports · doi:10.1038/s41598-022-22707-x · Cited by 13
Macoveanu, J., Miskowiak, K. W., Kessing, L. V., Vinberg, M., & Siebner, H. R. (2015) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.140220 · Cited by 9
Context-Sensitive Computational Mechanisms of Decision Making
Chawla, M., & Miyapuram, K. P. (2018) · Journal of Experimental Neuroscience · doi:10.1177/1179069518809057 · Cited by 10
The aging investor: Insights from neuroeconomicsPreprint
Mohr, P. N. C., & Heekeren, H. R. (2012) · RePEc: Research Papers in Economics · Cited by 9
Novel Psychoactive Substances and Behavioral Addictions
Martinotti, G., Corazza, O., Achab, S., & Demetrovics, Z. (2014) · BioMed Research International · doi:10.1155/2014/534523 · Cited by 9
Bayramoğlu, G., & Öztürk, O. (2024) · Journal of Economic Surveys · doi:10.1111/joes.12676 · Cited by 9
Carannante, I., Scolamiero, M., Hjorth, J. J. J., Kozlov, A., Bekkouche, B. M. B., Guo, L., Kumar, A., Chachólski, W., & Kotaleski, J. H. (2024) · Network Neuroscience · doi:10.1162/netn_a_00394 · Cited by 9
“…and precedes dendritic atrophy in SPNs. 66 Striatum is crucial for sensori-motor integration (Wall, De La Parra, Callaway, and Kreitzer (2013), de la Torre-Martinez, Ketzef, and Silberberg (2023)), action-selection (Redgrave, Prescott, and Gurney (1999)) and reinforcement learning (Doya (2008)).”
BackgroundDecision making and risk attitude of the common marmoset in a gambling task
Tokuno, H., & Tanaka, I. (2011) · Neuroscience Research · doi:10.1016/j.neures.2011.07.1822 · Cited by 8
Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning
Cussat‐Blanc, S., & Harrington, K. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754730 · Cited by 10
“Neuromodulation has been considered in the context of RL [10, 25, 11], but this previous work has not focused on the implications of neuromodulation on problem solving capacity.”
BackgroundReward-Based Deception with Cognitive Bias
Wu, B., Cubuktepe, M., Bharadwaj, S., & Topcu, U. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029476 · Cited by 8
“The process of human decision-making typically follows several steps [30].”
BackgroundDomingues, A. V., Coimbra, B., Correia, R. G., Deseyve, C., Vieitas‐Gaspar, N., Floresco, S., Sousa, N., Soares‐Cunha, C., & Rodrigues, A. J. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-02043-4 · Cited by 10
“The NAc is known to process reward-related information [69, 70], and its outputs [21, 43, 68] are critical regulators of decisions that involve effort.”
BackgroundPavlovian bias instigates suboptimal choices in humans
Degni, L., Danti, C., Finotti, G., Starita, F., di Pellegrino, G., & Garofalo, S. (2025) · Behaviour Research and Therapy · doi:10.1016/j.brat.2025.104906 · Cited by 8
Wang, D., Si, W., & Luo, Y. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2019.2953944 · Cited by 8
“For instance, some forms of DA are considered to be linking with reward [16], [17], [34] and some forms of serotonin are considered to be linking with aversion and punishment [10], [12], [36].”
BackgroundNeuron-type specificity of dorsal raphe projections to ventral tegmental areaPreprint
Chang, A. J., Wang, L., Lucantonio, F., Adams, M., Lemire, A. L., Dudman, J. T., & Cohen, J. Y. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.01.06.425641 · Cited by 10
“, 2016), or as a slower signal used to modulate the magnitude of reward prediction errors by changing learning rate or reward rate (Daw et al., 2002; Daw and Touretzky, 2002; Doya, 2002; Tobler et al., 2005; Doya, 2008; Matias et al., 2017; Grossman et al., 2020).”
BackgroundSequential Density Ratio Estimation for Simultaneous Optimization of Speed and AccuracyPreprint
Ebihara, A. F., Miyagawa, T., Sakurai, K., & Imaoka, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05587 · Cited by 10
Complementary roles of serotonin and dopamine in model-based learning
Taira, M., & Sharpe, M. J. (2024) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2024.101464 · Cited by 7
Funamizu, A., Ito, M., Doya, K., Kanzaki, R., & Takahashi, H. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00027 · Cited by 8
“Decision processes in the cortico-basal ganglia circuit are theoretically explained by the reinforcement learning framework (Corrado and Doya, 2007; O’Doherty et al., 2007; Doya, 2008).”
BackgroundInfluentialSelf-citationNeuroeconomics, decision-making and rationality
Corcos, A., & Pannequin, F. (2011) · Économie et Institutions · doi:10.4000/ei.74 · Cited by 8
Reijgwart, M. L., Vinke, C. M., Hendriksen, C. F. M., van der Meer, M., Schoemaker, N. J., & van Zeeland, Y. R. A. (2017) · Animal Behaviour · doi:10.1016/j.anbehav.2017.10.026 · Cited by 7
“In the 3Ch set-up, this might have made the environment in both chambers unpredictable, which has been suggested to increase exploration (with the aim of learning about the environment and thereby reducing unpredictability; Doya, 2008; Inglis, 2000).”
BackgroundMartin, J. T., Whittaker, A. H., & Johnston, S. J. (2020) · Journal of Vision · doi:10.1167/jov.20.7.5 · Cited by 8
“Its utility for this purpose hinges to a large extent on our growing understanding of the functional association between non-luminance-mediated pupil size changes and the activity of the locus coeruleusnoradrenalin (LC-NA) system of the brain, which plays a pivotal role in the modulation of arousal, cognition, and aut…”
BackgroundHaines, N., & Beauchaine, T. P. (2020) · Psychopathology · doi:10.1159/000508539 · Cited by 7
Metadata
- Authors
- Doya, K.
- Affiliations
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Doya, Kenji
- ORCID
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2008-03-26
- Volume
- 11
- Issue
- 4
- Pages
- 410-416
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1971976749
- Cited by
- 642
- References
- 93
- Citations by year
- 2026: 22 · 2025: 18 · 2024: 30 · 2023: 15 · 2022: 26 · 2021: 35 · 2020: 34 · 2019: 51 · 2018: 57 · 2017: 31 · 2016: 39 · 2015: 43 · 2014: 43 · 2013: 45 · 2012: 37
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- decision-making, contextual modulation, reward anticipation, risk-taking, uncertainty, delay discounting, dopamine, anterior cingulate cortex, norepinephrine, orbitofrontal cortex, serotonin, dorsal striatum, dorsal prefrontal cortex, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Decision Making, Environment, Gyrus Cinguli, Humans, Models, Neurological, Models, Psychological, Probability Learning, Reward, Neurotransmitter Agents, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.