The DOI matches its Crossref record; 126 of the 131 references it cites were found in a registry or an index.
Decision theory, reinforcement learning, and the brain
Open accessAbstract
Decision making is a core competence for animals and humans acting and surviving in environments they only partially comprehend, gaining rewards and punishments for their troubles. Decision-theoretic concepts permeate experiments and computational models in ethology, psychology, and neuroscience. Here, we review a well-known, coherent Bayesian approach to decision making, showing how it unifies issues in Markovian decision problems, signal detection psychophysics, sequential sampling, and optimal exploration and discuss paradigmatic psychological and neural examples of each problem. We discuss computational issues concerning what subjects know about their task and how ambitious they are in seeking optimal solutions; we address algorithmic topics concerning model-based and model-free methods for making choices; and we highlight key aspects of the neural implementation of decision making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“One framework to address this challenge is Bayesian decision theory (Körding, 2007; Dayan and Daw, 2008; Daunizeau et al., 2010). In a nutshell, this identifies an optimal action as one that maximizes the “expected utility” (where “expected” refers to a weighted average; i.e., the predicted outcomes are weighted by their relative uncertainty).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Dayan and Daw, 2008
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
(1995) · Cited by 10,558
(1966) · Medical Entomology and Zoology · Cited by 10,293
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationSignal detection theory and psychophysics
Ingleby, D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,616
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Flagged first, then by citations
5 of the 131 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citationAction understanding as inverse planning
Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009) · Cognition · doi:10.1016/j.cognition.2009.07.005 · Cited by 920
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809
Self-citationInformation-seeking, curiosity, and attention: computational and neural mechanisms
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.001 · Cited by 784
Computational psychiatry: the brain as a phantastic organ
Friston, K. J., Stephan, K. E., Montague, R., & Dolan, R. J. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70275-5 · Cited by 531
The hierarchical basis of neurovisceral integration
Smith, R., Thayer, J. F., Khalsa, S. S., & Lane, R. D. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.003 · Cited by 523
Choice Certainty Is Informed by Both Evidence and Decision Time
Kiani, R., Corthell, L., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.12.015 · Cited by 470
Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression
Stephan, K. E., Manjaly, Z. M., Mathys, C. D., Weber, L. A. E., Paliwal, S., Gard, T., Tittgemeyer, M., Fleming, S. M., Haker, H., Seth, A. K., et al. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00550 · Cited by 437
“The predictions by this framework suggest that differential diagnosis could be decisively facilitated by model-based estimates of directed synaptic connectivity (effective connectivity) within interoceptive circuits and their interactions with regions potentially involved in meta-cognition.”
BackgroundGo and no-go learning in reward and punishment: Interactions between affect and effect
Guitart‐Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Düzel, E., & Dolan, R. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.024 · Cited by 448
Self-citationCognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 426
Collins, A. G. E., & Frank, M. J. (2014) · Psychological Review · doi:10.1037/a0037015 · Cited by 403
Mood as representation of momentum
Eldar, E., Rutledge, R. B., Dolan, R. J., & Niv, Y. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.07.010 · Cited by 383
Ahn, W., Haines, N., & Zhang, L. (2017) · Computational Psychiatry · doi:10.1162/cpsy_a_00002 · Cited by 385
The dissociable effects of punishment and reward on motor learning
Galea, J. M., Mallia, E., Rothwell, J. C., & Diedrichsen, J. (2015) · Nature Neuroscience · doi:10.1038/nn.3956 · Cited by 382
Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex
Wikenheiser, A. M., & Schoenbaum, G. M. (2016) · Nature reviews. Neuroscience · doi:10.1038/nrn.2016.56 · Cited by 368
Being a Beast Machine: The Somatic Basis of Selfhood
Seth, A. K., & Tsakiris, M. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.08.008 · Cited by 371
“Instrumental inference (equivalently, control-oriented inference) prescribes actions that exploit these models for ongoing control and regulation [15, 50, 51].”
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357
Self-citationThe anatomy of choice: active inference and agency
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00598 · Cited by 340
Predictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M., Gershman, S. J., & Daw, N. D. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 335
Self-citationNegative Symptoms and the Failure to Represent the Expected Reward Value of Actions
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 319
The basolateral amygdala in reward learning and addiction
Wassum, K. M., & Izquierdo, A. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.08.017 · Cited by 306
A computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304
Goschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293
Reinforcement learning can account for associative and perceptual learning on a visual-decision task
Law, C.-T., & Gold, J. I. (2009) · Nature Neuroscience · doi:10.1038/nn.2304 · Cited by 293
The anatomy of choice: dopamine and decision-making.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0481 · Cited by 289
Active inference and agency: optimal control without cost functions
Friston, K., Samothrakis, S., & Montague, R. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0512-8 · Cited by 241
Attention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241
“By including an estimate of the uncertainty that arises at each step, one can further calculate the costs of this uncertainty and the benefits of reducing it by obtaining information (Dayan and Daw, 2008; Rothkopf and Ballard, 2010; Sprague and Ballard, 2005; Tatler et al., 2011).”
BackgroundInfluentialGlutamatergic model psychoses: prediction error, learning, and inference
Corlett, P. R., Honey, G. D., Krystal, J. H., & Fletcher, P. C. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.163 · Cited by 229
Mice alternate between discrete strategies during perceptual decision-making
Ashwood, Z. C., Roy, N. A., Stone, I. R., Laboratory, T. I. B., Urai, A. E., Churchland, A. K., Pouget, A., & Pillow, J. W. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-01007-z · Cited by 229
Hogarth, L., & Chase, H. W. (2011) · Journal of Experimental Psychology: Animal Behavior Processes · doi:10.1037/a0022913 · Cited by 225
Depression: a decision-theoretic analysis
Huys, Q. J. M., Daw, N. D., & Dayan, P. (2015) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071714-033928 · Cited by 221
Self-citationWorking Memory Contributions to Reinforcement Learning Impairments in Schizophrenia
Collins, A. G. E., Brown, J. K., Gold, J. M., Waltz, J. A., & Frank, M. J. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.0989-14.2014 · Cited by 223
Is There a ‘Social’ Brain? Implementations and Algorithms
Lockwood, P. L., Apps, M. A. J., & Chang, S. W. C. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.06.011 · Cited by 218
Error and Expectation in Language Learning: The Curious Absence of Mouses in Adult Speech
Ramscar, M., Dye, M., & McCauley, S. M. (2013) · Language · doi:10.1353/lan.2013.0068 · Cited by 200
Modeling Trait Anxiety: From Computational Processes to Personality
Raymond, J. G., Steele, J. D., & Seriès, P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00001 · Cited by 209
The Neuroscience of Learning: Beyond the Hebbian Synapse
Gallistel, C. R., & Matzel, L. D. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143807 · Cited by 212
Am i self-conscious? (or does self-organization entail self-consciousness?)
Friston, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00579 · Cited by 197
The Basal Ganglia Do Not Select Reach Targets but Control the Urgency of Commitment
Thura, D., & Cisek, P. (2017) · Neuron · doi:10.1016/j.neuron.2017.07.039 · Cited by 188
A reinforcement learning diffusion decision model for value-based decisions
Fontanesi, L., Gluth, S., Spektor, M. S., & Rieskamp, J. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1554-2 · Cited by 195
Rao, R. P. N. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00146 · Cited by 196
Bayesian Prediction and Evaluation in the Anterior Cingulate Cortex
Ide, J. S., Shenoy, P., Yu, A. J., & Li, C.-S. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.2201-12.2013 · Cited by 185
Arnon, I., & Ramscar, M. (2011) · Cognition · doi:10.1016/j.cognition.2011.10.009 · Cited by 180
Estimates of projection overlap and zones of convergence within frontal-striatal circuits
Averbeck, B. B., Lehman, J., Jacobson, M., & Haber, S. N. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.5806-12.2014 · Cited by 179
Motivational neural circuits underlying reinforcement learning
Averbeck, B. B., & Costa, V. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4506 · Cited by 181
Reciprocal relations between cognitive neuroscience and formal cognitive models: opposites attract?
Forstmann, B. U., Wagenmakers, E., Eichele, T., Brown, S. D., & Serences, J. T. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.04.002 · Cited by 178
Reward-based training of recurrent neural networks for cognitive and value-based tasks
Song, H., Yang, G. R., & Wang, X. (2017) · eLife · doi:10.7554/elife.21492 · Cited by 175
Happel, M. F. K., Niekisch, H., Castiblanco Rivera, L. L., Ohl, F. W., Deliano, M., & Frischknecht, R. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1310272111 · Cited by 176
Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals
Pike, A. C., & Robinson, O. J. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.0051 · Cited by 183
Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170
Pain: A Precision Signal for Reinforcement Learning and Control
Seymour, B. (2019) · Neuron · doi:10.1016/j.neuron.2019.01.055 · Cited by 175
Panksepp, J., Lane, R. D., Solms, M., & Smith, R. S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.010 · Cited by 182
Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model
Bitzer, S., Park, H., Blankenburg, F., & Kiebel, S. J. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00102 · Cited by 168
Action Selection and Action Value in Frontal-Striatal Circuits
Seo, M., Lee, E., & Averbeck, B. B. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.037 · Cited by 170
Assessing inter individual differences with task related functional neuroimaging
Lebreton, M., Bavard, S., Daunizeau, J., & Palminteri, S. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0681-8 · Cited by 160
What’s in a word? How instructions, suggestions, and social information change pain and emotion
Koban, L., Jepma, M., Geuter, S., & Wager, T. D. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.014 · Cited by 166
Animal affect and decision-making
Mendl, M., & Paul, E. S. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.01.025 · Cited by 171
Plan, Activity, and Intent Recognition
Sukthankar, G., Geib, C., Bui, H., Pynadath, D. V., & Goldman, R. P. (2014) · Elsevier eBooks · doi:10.1016/c2012-0-01169-6 · Cited by 158
How instructed knowledge modulates the neural systems of reward learning
Li, J., Delgado, M. R., & Phelps, E. A. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014938108 · Cited by 152
The functional neuroanatomy of decision making: Prefrontal control of thought and action
Coutlee, C. G., & Huettel, S. A. (2011) · Brain Research · doi:10.1016/j.brainres.2011.05.053 · Cited by 165
Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction
Li, J., & Daw, N. D. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6316-10.2011 · Cited by 159
Self-citationJuechems, K., & Summerfield, C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.07.012 · Cited by 160
Corlett, P. R. (2009) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.012.2009 · Cited by 146
Dopaminergic and Prefrontal Basis of Learning from Sensory Confidence and Reward Value
Lak, A., Okun, M., Moss, M. M., Gurnani, H., Farrell, K., Wells, M. J., Reddy, C. B., Kepecs, Á., Harris, K. D., & Carandini, M. (2019) · Neuron · doi:10.1016/j.neuron.2019.11.018 · Cited by 137
Free Energy, Value, and Attractors
Friston, K. J., & Ao, P. (2011) · Computational and Mathematical Methods in Medicine · doi:10.1155/2012/937860 · Cited by 146
Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes
Groman, S. M., Keistler, C., Keip, A. J., Hammarlund, E., Dileone, R., Pittenger, C., Lee, D., & Taylor, J. R. (2019) · Neuron · doi:10.1016/j.neuron.2019.05.042 · Cited by 147
The Functional Neuroanatomy of Decision-Making
Rosenbloom, M. H., Schmahmann, J. D., & Price, B. H. (2012) · Journal of Neuropsychiatry · doi:10.1176/appi.neuropsych.11060139 · Cited by 149
Bayesian modelling of Jumping-to-Conclusions bias in delusional patients
Moutoussis, M., Bentall, R. P., El-Deredy, W., & Dayan, P. (2011) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2010.548678 · Cited by 137
Self-citationDecision-making under uncertainty: biases and Bayesians
Trimmer, P. C., Houston, A. I., Marshall, J. A. R., Mendl, M., Paul, E. S., & McNamara, J. M. (2011) · Animal Cognition · doi:10.1007/s10071-011-0387-4 · Cited by 145
Moral Learning: Conceptual foundations and normative relevance
Railton, P. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.015 · Cited by 136
A neural network for information seeking
White, J. K., Bromberg-Martin, E. S., Heilbronner, S. R., Zhang, K., Pai, J., Haber, S. N., & Monosov, I. E. (2019) · Nature Communications · doi:10.1038/s41467-019-13135-z · Cited by 129
Computational Rationality as a Theory of Interaction
Oulasvirta, A., Jokinen, J., & Howes, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517739 · Cited by 147
Dark control: The default mode network as a reinforcement learning agent
Dohmatob, E., Dumas, G., & Bzdok, D. (2020) · Human Brain Mapping · doi:10.1002/hbm.25019 · Cited by 121
Can the neuroeconomics revolution revolutionize psychiatry?
Hasler, G. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.04.011 · Cited by 122
Modeling Human Plan Recognition Using Bayesian Theory of Mind
Baker, C. L., & Tenenbaum, J. B. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-398532-3.00007-5 · Cited by 118
Bayesian Models of Development
Stamps, J. A., & Frankenhuis, W. E. (2016) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2016.01.012 · Cited by 114
Unconscious emotion: A cognitive neuroscientific perspective
Smith, R. S., & Lane, R. D. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.08.013 · Cited by 118
Predictive maps in rats and humans for spatial navigation
de Cothi, W., Nyberg, N., Griesbauer, E.-M., Ghanamé, C., Zisch, F., Lefort, J. M., Fletcher, L., Newton, C., Renaudineau, S., Bendor, D., et al. (2022) · Current Biology · doi:10.1016/j.cub.2022.06.090 · Cited by 109
Clark, A. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00270 · Cited by 108
Adolescent risk-taking in the context of exploration and social influence
Ciranka, S., & van den Bos, W. (2021) · Developmental Review · doi:10.1016/j.dr.2021.100979 · Cited by 127
The perception of probability.
Gallistel, C. R., Krishan, M., Liu, Y., Miller, R., & Latham, P. E. (2014) · Psychological Review · doi:10.1037/a0035232 · Cited by 114
Lapses in perceptual decisions reflect exploration
Pisupati, S., Chartarifsky, L., Khanal, A., & Churchland, A. K. (2021) · eLife · doi:10.7554/elife.55490 · Cited by 105
The neural substrate of prior information in perceptual decision making: a model-based analysis
Forstmann, B. U. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00040 · Cited by 113
A Roadmap for the Development of Applied Computational Psychiatry
Paulus, M. P., Huys, Q. J. M., & Maia, T. V. (2016) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.05.001 · Cited by 106
Lak, A., Hueske, E., Hirokawa, J., Masset, P., Ott, T., Urai, A. E., Donner, T. H., Carandini, M., Tonegawa, S., Uchida, N., et al. (2020) · eLife · doi:10.7554/elife.49834 · Cited by 98
Bolkan, S. S., Stone, I. R., Pinto, L., Ashwood, Z. C., Iravedra-García, J. M., Herman, A. L., Singh, P., Bandi, A., Cox, J., Zimmerman, C. A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01021-9 · Cited by 108
Reiter, A. M. F., Heinze, H., Schlagenhauf, F., & Deserno, L. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.95 · Cited by 104
Belief state representation in the dopamine system
Babayan, B. M., Uchida, N., & Gershman, S. J. (2018) · Nature Communications · doi:10.1038/s41467-018-04397-0 · Cited by 101
How to set the switches on this thing
Dayan, P. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.05.011 · Cited by 107
Self-citationDecision-making in schizophrenia: A predictive-coding perspective
Sterzer, P., Voss, M., Schlagenhauf, F., & Heinz, A. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.05.074 · Cited by 105
Constantino, S. M., Schlüter, M., Weber, E. U., & Wijermans, N. (2021) · Sustainability Science · doi:10.1007/s11625-021-00989-w · Cited by 119
Risk-Taking Behavior: Dopamine D2/D3 Receptors, Feedback, and Frontolimbic Activity
Kohno, M., Ghahremani, D. G., Morales, A. M., Robertson, C. L., Ishibashi, K., Morgan, A., Mandelkern, M. A., & London, E. D. (2013) · Cerebral Cortex · doi:10.1093/cercor/bht218 · Cited by 97
Discovery of hierarchical representations for efficient planning
Tomov, M. S., Yagati, S., Kumar, A., Yang, W., & Gershman, S. J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007594 · Cited by 100
From compulsivity to compulsion: the neural basis of compulsive disorders
Robbins, T. W., Banca, P., & Belin, D. (2024) · Nature reviews. Neuroscience · doi:10.1038/s41583-024-00807-z · Cited by 111
Multiplicity of control in the basal ganglia: computational roles of striatal subregions
Bornstein, A. M., & Daw, N. D. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.02.009 · Cited by 106
Self-citationOn-line confidence monitoring during decision making
Dotan, D., Meyniel, F., & Dehaene, S. (2017) · Cognition · doi:10.1016/j.cognition.2017.11.001 · Cited by 104
Understanding Interpersonal Function in Psychiatric Illness Through Multiplayer Economic Games
King‐Casas, B., & Chiu, P. H. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.03.033 · Cited by 103
The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory
Collins, A. G. E. (2018) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01238 · Cited by 97
Reward-Guided Learning with and without Causal Attribution
Jocham, G., Brodersen, K. H., Constantinescu, A. O., Kahn, M. C., Ianni, A. M., Walton, M. E., Rushworth, M. F. S., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.018 · Cited by 92
Metadata
- Authors
- Dayan, P., & Daw, N. D.
- Affiliations
- University College London (GB)
- New York University (US)
- Corresponding author
- Dayan, P.
- Daw, N. D.
- ORCID
- Venue
- Cognitive, Affective, & Behavioral Neuroscience
- Published
- 2008-12-01
- Volume
- 8
- Issue
- 4
- Pages
- 429-453
- ISSN
- 1530-7026 (ISSN-L)
- 1531-135X
- OpenAlex
- W2037917209
- Cited by
- 580
- References
- 131
- Citations by year
- 2026: 25 · 2025: 52 · 2024: 46 · 2023: 33 · 2022: 43 · 2021: 34 · 2020: 55 · 2019: 38 · 2018: 26 · 2017: 37 · 2016: 25 · 2015: 27 · 2014: 25 · 2013: 26 · 2012: 37
- FWCI
- 1.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Cognitive Affective & Behavioral Neuroscience (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- decision-making, Bayesian decision theory, Markov decision process, signal detection theory, sequential sampling, model-based learning, model-free learning, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Bayes Theorem, Brain, Cognition, Cost-Benefit Analysis, Decision Making, Decision Theory, Exploratory Behavior, Humans, Markov Chains, Models, Psychological, Problem Solving, Reinforcement, Psychology, Models, Statistical, Signal Detection, Psychological, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
