The DOI matches its Crossref record; 126 of the 131 references it cites were found in a registry or an index.
Decision theory, reinforcement learning, and the brain
Open accessAbstract
Decision making is a core competence for animals and humans acting and surviving in environments they only partially comprehend, gaining rewards and punishments for their troubles. Decision-theoretic concepts permeate experiments and computational models in ethology, psychology, and neuroscience. Here, we review a well-known, coherent Bayesian approach to decision making, showing how it unifies issues in Markovian decision problems, signal detection psychophysics, sequential sampling, and optimal exploration and discuss paradigmatic psychological and neural examples of each problem. We discuss computational issues concerning what subjects know about their task and how ambitious they are in seeking optimal solutions; we address algorithmic topics concerning model-based and model-free methods for making choices; and we highlight key aspects of the neural implementation of decision making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“One framework to address this challenge is Bayesian decision theory (Körding, 2007; Dayan and Daw, 2008; Daunizeau et al., 2010). In a nutshell, this identifies an optimal action as one that maximizes the “expected utility” (where “expected” refers to a weighted average; i.e., the predicted outcomes are weighted by their relative uncertainty).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Dayan and Daw, 2008
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
(1995) · Cited by 10,558
(1966) · Medical Entomology and Zoology · Cited by 10,293
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationSignal detection theory and psychophysics
Ingleby, D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,616
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Flagged first, then by citations
5 of the 131 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Items 501–583 of 583
Yu, E. C. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00541
Making an intuitive decision: a neurophysiological perspective
Broom, M. (2016) · British Journal of Neuroscience Nursing · doi:10.12968/bjnn.2016.12.1.30
Examining an Implicit Mechanism of Recognition Criterion Regulation
Cox, J. C. (2014) · Open Scholarship Institutional Repository (Washington University in St. Louis) · doi:10.7936/k72r3ps5
Learning of judgment and decision-making strategies
Mata, R., Mata, R., & Rieskamp, J. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139015684.011
Stamps, A., & Frankenhuis, W. E. (2016)
Reinforcement learning model in probalistically rewarded task
Enel, P., Khamassi, M., Procyk, E., & Dominey, P. F. (2010) · HAL
Neural Networks and Neurocomputational Modeling
Toutounji, H., Hertäg, L., & Durstewitz, D. (2018) · Wiley eBooks · doi:10.1002/9781119170174.epcn517
Implications of Human Irrationality for Reinforcement LearningPreprint
Chen, H., Chang, H. J., & Howes, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04072 · Cited by 1
Liu, Y., Zhao, Y., & Chen, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.05595
Prior Experience Modifies Acquisition Trajectories Via Response-Strategy SamplingPreprint
Daniels, C. W., & Balsam, P. D. (2020) · doi:10.31234/osf.io/bzu4y
Structures for Sophisticated Behaviour: Feudal Hierarchies and World Models
Ahilan, S. (2021) · UCL Discovery (University College London)
Additively Combining Utilities and Beliefs: Research Gaps and Algorithmic Developments
Ghambaryan, A., Gutkin, B. S., Klucharev, V. A., & Koechlin, É. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.704728
Physics of Complex Present: Properties of Action Strategy Cloud
Lubashevsky, I., & Plavinska, N. (2021) · Understanding complex systems · doi:10.1007/978-3-030-82612-3_6
Action, time, and uncertainty: two types of temporal uncertainty in cognitive motor control
Schüür, F. (2012) · UCL Discovery (University College London)
Osman, M. (2010) · Wiley eBooks · doi:10.1002/9781444328226.refs
Dora, J., Kuczynski, A. M., Schultz, M. E., Acuff, S. F., Murphy, J. G., & King, K. M. (2022) · doi:10.31234/osf.io/eh9j4
Influence of Visual Clue in the Motor Adaptation Process
Costa, V. T., Oliveira, S. R. J., & Soares, A. B. (2022) · World Congress on Medical Physics and Biomedical Engineering, September 7 - 12, 2009, Munich, Germany · doi:10.1007/978-3-030-70601-2_215
Chapter 9 - Theory of Mind and Joint AttentionPreprint
Pérez-Osorio, J., Wiese, E., & Wykowska, A. (2021) · doi:10.31234/osf.io/az8qy
Computational Psychiatry and the Bayesian Brain
Friston, K. J., & Dolan, R. J. (2017) · Oxford University Press eBooks · doi:10.1093/med/9780190681425.003.0072
An arbiter model of motivational selectionPreprint
McNally, G. P. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/b7eux
Log, Stock and Two Simple Lotteries: Technical SupplementPreprint
Verstyuk, S. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3120083
Schultz, W. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.727392887.793565155
Faculty Opinions recommendation of Belief state representation in the dopamine system.
Schultz, W. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.733228055.793565154
Hoppe, D. B. (2022) · doi:10.33612/diss.238906691
Internal uncertainty impacts social information use in risky choice across adolescencePreprint
Ciranka, S., & van den Bos, W. (2022) · doi:10.31234/osf.io/73dja
Accumulating evidence by sampling from temporally organized memory
Tiganj, Z. (2023) · Learning & Behavior · doi:10.3758/s13420-022-00569-7
Tradeoffs in Uncertain Decision Making
Strickland, L. J. G., & Boag, R. J. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197501061.013.19
The Emergence of Consciousness in a Physical UniversePreprint
Singh, R. K. (2024) · Qeios · doi:10.32388/1kc9th
The Emergence of Consciousness in a Physical UniversePreprint
Singh, R. K. (2024) · Qeios · doi:10.32388/1kc9th.2
Yang, Y., Yu, B., & Yu, G. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4768868
The expression of decision and learning variables in movement patterns related to decision actions
Selbing, I., & Skewes, J. (2024) · Experimental Brain Research · doi:10.1007/s00221-024-06805-y
The Emergence of Consciousness in a Physical UniversePreprint
Singh, R. K. (2024) · Qeios · doi:10.32388/1kc9th.3
Optimal and sub-optimal temporal decisions can explain procrastination in a real-world taskPreprint
Chebolu, S., & Dayan, P. (2024) · doi:10.31234/osf.io/69zhd
Self-citationUsée, F., Schmidt, S., Pané‐Farré, C. A., & Ostwald, D. (2024) · doi:10.31234/osf.io/gjtp9
Bayesian Priors in Active AvoidancePreprint
Granwald, T., Dayan, P., Lengyel, M., & Guitart‐Masip, M. (2024) · doi:10.31234/osf.io/2zcjh
Self-citationFood Reinforcement Architecture
Burger, K. S., & Papantoni, A. (2024) · doi:10.1093/oso/9780190671051.003.0020
Hong, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00301
Purposeful Regularization with Reinforcement Learning for Facial Expression Recognition In-the-Wild
Hong, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00464
Augusto, E., Kouskoff, V., Chenouard, N., Giraudet, M., Peltier, L., de Miranda, A., Louis, A., Alonso, L., & Gambino, F. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.29.620931
Usée, F., Schmidt, S., Melzig, C. A., & Ostwald, D. (2025) · Computational Brain & Behavior · doi:10.1007/s42113-024-00229-1
Neural Representation of Perceptual Decision Making
Araki, S., & Funamizu, A. (2025) · The Brain & Neural Networks · doi:10.3902/jnns.32.38
Cerrotti, F., Salvador, A., Hamroun, S., Lebreton, M. P., & Palminteri, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.15.648746
Mrhbm: Multi-Resolution Hierarchical Bayesian Modeling for High-Resolution Behavioral AnalysisPreprint
Lu, Z. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5258593
WITHDRAWNPreprint
Leizerman, S. L. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/aekqs_v1
Investigating the effects of orexin receptor 1 antagonism on decision making under uncertainty
Metha, J. A., Bertheau, M., Murawski, C., Hoyer, D., & Jacobson, L. H. (2025) · Behavioural Brain Research · doi:10.1016/j.bbr.2025.115691
How to Build a Reliable Framework to Make Intelligent Decisions About Road Maintenance
Sun, H., Yi, X., Pan, Z., Cheng, N., Shih, P.-C., & Guo, Y. (2025) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1109/icict64582.2025.00062
Computational Psychiatry and Eating Disorders
Sepúlveda, P., Aitsahalia, I. F., Dong, S., & Iigaya, K. (2025) · Oxford University Press eBooks · doi:10.1093/9780197675212.003.0006
A Spatio-temporal Schema Mechanism for Developmental Robotics
Georgeon, O. L., Gay, S. L., & Robertson, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00686-8_16
An Information-Theoretic Framework for Understanding Learning and Choice Under Uncertainty
Woo, J. H., Balaji, L., & Soltani, A. (2025) · Entropy · doi:10.3390/e27101056
Absence of Systematic Effects of Internalizing Psychopathology on Learning Under UncertaintyPreprint
Satti, M. H., Wille, K., Nassar, M. R., Cichy, R. M., Schuck, N. W., Dayan, P., & Bruckner, R. (2025) · eLife · doi:10.7554/elife.107611
Self-citationMahajan, P., & Seymour, B. (2025) · Pain · doi:10.1097/j.pain.0000000000003705
He, Z., Yin, Z., & Liu, Y. (2025) · Psychophysiology · doi:10.1111/psyp.70174
Dora, J., Sullivan‐Toole, H., Zhang, C., Opdahl, M., & King, K. M. (2025) · Judgment and Decision Making · doi:10.1017/jdm.2025.10022
Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance
Meyen, S., Göppert, F., Schrenk, C., Ulrike Von Luxburg, A., & Franz, V. H. (2025) · Open Mind · doi:10.1162/opmi.a.40 · Cited by 4
Computational Neuroscience and Neurorehabilitation
Christensen, M. S. (2025) · Springer eBooks · doi:10.1007/978-3-031-81120-3_8
Joshi, S., Polat, M., Chai, D. C., Pantis, S., Garg, R., Buch, V., & Ramayya, A. G. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.18.695315
Temporal stimulus segmentation by reinforcement learning in populations of spiking neurons
Donne, L. L., Chan, L. C., Urbanczik, R., Senn, W., & Camera, G. L. (2026) · Physical review. E · doi:10.1103/4mwt-2yb2 · Cited by 1
Accumulated evidence as an additive performance measure based on confidence ratings
Meyen, S., Lin, L., Schrenk, C., & Franz, V. H. (2026) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2026.101249
A Basal Ganglia–Inspired Reinforcement Learning Framework Using Dopamine-Modulated Plasticity
Rathore, A., Moyya, P. D., Rana, K., Kumar, A., Singh, A., Rana, J., Nikhil, N., & Kumar, S. S. (2026) · International Conference on Computational Intelligence in Data Science (ICCIDS) · doi:10.1109/iccids69108.2026.11407699
A bio-inspired minimal model for non-stationary K-armed bandits
Danieli, K., & Lepperød, M. E. (2026) · Biological Cybernetics · doi:10.1007/s00422-026-01037-5
Coordination des actions et des habitudes : approche neurocomportementale chez le rat
Tran‐Tu‐Yen, D. A. S. (2010) · theses.fr (ABES) · doi:10.70675/b4e62359z218dz47e1zb735zbeed5febcf30
Approches computationnelles d'analyse des relations entre cerveau et comportement
Brochard, J. (2021) · theses.fr (ABES) · doi:10.70675/622daac4z9213z4569z9a2eze68e530929ee
Structure learning for modeling individual behavior
James, A. (2024) · theses.fr (ABES) · doi:10.70675/b1ab658bzf3d2z4c5azad8ez44c341cc596e
Dynamiques dopaminergiques dans la prise de décision : rôle dans la variabilité et l’exploration
Côme, M. (2023) · theses.fr (ABES) · doi:10.70675/a7b03d10z7e3cz45a1z817bza6c6344893af
Alefantis, P., Guo, Y., Quazi, N., Savin, C., Angelaki, D. E., Pitkow, X., & Majaj, N. J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.04.14.718537
A Neurocomputational Model of Observation-Based Decision Making with a Focus on Trust
Nazir, A. H., Kotaleski, J. H., & Liljenström, H. (2026) · Brain Sciences · doi:10.3390/brainsci16050477 · Cited by 2
A Deep Dive into the Cognitive Soundscape of Flow: Finding Your GroovePreprint
Bartling, B. A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.13.724953
Esmaily, J., Moran, R., Roudi, Y., & Bahrami, B. (2026) · Neural Computation · doi:10.1162/neco.a.1543
Dopamine projections to the basolateral amygdala enable reward predictionPreprint
Ramírez-Armenta, K., Feng, C., Martinez, A., Andrade, J. E., Quach, J., Paredes, N., Sias, A. C., Griffin, N. K., Sharpe, M. J., & Wassum, K. M. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.21.733629
Shang, Z., Wang, Y., Li, M., Jin, F., Yang, L., Li, Z., & Yue, C. (2026) · Journal of Experimental Biology · doi:10.1242/jeb.252383
How do expectations shape actions?
Straube, B. (2026) · Trends Open · doi:10.1016/j.treopn.2026.06.017
Affective Expert Modulation for Adaptive Decision-Making in Non-stationary Bandits
Kossack, P., & Unger, H. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-26319-3_14
Modeling Drug-Induced Cognitive Biases in Neurobehavioural Decision-Making
Gupta, G., Tripathi, N., & Mishra, R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8988-2.ch011
Hierarchical resource rationality explains human reading behaviour
Bai, Y., Jin, X., Zhao, S., & Oulasvirta, A. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02534-0 · Cited by 2
FIBB: Capturing False Information Beliefs and Behaviors Across Physical and Online Environments
Jia, M. Y., Bhattacharjee, N., Kapadia, A. R., & Wang, D. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838743
SAMNet: Recognition and Mitigation of Interference Signals
Diao, C., Lan, L., Wang, Y., Zhang, X., & Liao, G. (2026) · IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM) · doi:10.1109/sam68950.2026.11687873
Decomposing response inhibition: A POMDP model
Wang, W., Tobias, K., & Dayan, P. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014068
Self-citationChen, Y., & Liu, M. (2026) · Applied Mathematics and Computation · doi:10.1016/j.amc.2026.130345
Self-relevance dynamically biases exploration and exploitation in value-based decisions
Selvaraj, E. S., Golubickis, M., Caughey, S. E., Jalalian, P., Sharma, Y., Khasa, S., Masroujah, L. S., & Macrae, C. N. (2026) · Psychological Research · doi:10.1007/s00426-026-02402-6
Anterior cingulate cortex : contributions to social cognition
Apps, M. A. J. (2012) · Royal Holloway Digital Repository (Royal Holloway University of London)
Computational Neuroscience: Capturing the Essence
Druckmann, S., Gidon, A., & Segev, I. (2013) · Springer eBooks · doi:10.1007/978-3-642-10769-6_30
Metadata
- Authors
- Dayan, P., & Daw, N. D.
- Affiliations
- University College London (GB)
- New York University (US)
- Corresponding author
- Dayan, P.
- Daw, N. D.
- ORCID
- Venue
- Cognitive, Affective, & Behavioral Neuroscience
- Published
- 2008-12-01
- Volume
- 8
- Issue
- 4
- Pages
- 429-453
- ISSN
- 1530-7026 (ISSN-L)
- 1531-135X
- OpenAlex
- W2037917209
- Cited by
- 580
- References
- 131
- Citations by year
- 2026: 25 · 2025: 52 · 2024: 46 · 2023: 33 · 2022: 43 · 2021: 34 · 2020: 55 · 2019: 38 · 2018: 26 · 2017: 37 · 2016: 25 · 2015: 27 · 2014: 25 · 2013: 26 · 2012: 37
- FWCI
- 1.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Cognitive Affective & Behavioral Neuroscience (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- decision-making, Bayesian decision theory, Markov decision process, signal detection theory, sequential sampling, model-based learning, model-free learning, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Bayes Theorem, Brain, Cognition, Cost-Benefit Analysis, Decision Making, Decision Theory, Exploratory Behavior, Humans, Markov Chains, Models, Psychological, Problem Solving, Reinforcement, Psychology, Models, Statistical, Signal Detection, Psychological, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
