The DOI matches its Crossref record; 126 of the 131 references it cites were found in a registry or an index.
Decision theory, reinforcement learning, and the brain
Open accessAbstract
Decision making is a core competence for animals and humans acting and surviving in environments they only partially comprehend, gaining rewards and punishments for their troubles. Decision-theoretic concepts permeate experiments and computational models in ethology, psychology, and neuroscience. Here, we review a well-known, coherent Bayesian approach to decision making, showing how it unifies issues in Markovian decision problems, signal detection psychophysics, sequential sampling, and optimal exploration and discuss paradigmatic psychological and neural examples of each problem. We discuss computational issues concerning what subjects know about their task and how ambitious they are in seeking optimal solutions; we address algorithmic topics concerning model-based and model-free methods for making choices; and we highlight key aspects of the neural implementation of decision making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“One framework to address this challenge is Bayesian decision theory (Körding, 2007; Dayan and Daw, 2008; Daunizeau et al., 2010). In a nutshell, this identifies an optimal action as one that maximizes the “expected utility” (where “expected” refers to a weighted average; i.e., the predicted outcomes are weighted by their relative uncertainty).”
From Allostatic Self-efficacy: A Metacognitive Theory of Dyshomeostasis-Induced Fatigue and Depression (2016) · cites this paper as Dayan and Daw, 2008
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
(1995) · Cited by 10,558
(1966) · Medical Entomology and Zoology · Cited by 10,293
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationSignal detection theory and psychophysics
Ingleby, D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,616
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,674
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Neural systems of reinforcement for drug addiction: from actions to habits to compulsion
Everitt, B. J., & Robbins, T. W. (2005) · Nature Neuroscience · doi:10.1038/nn1579 · Cited by 3,785
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764
(2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,601
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan, P., & Abbott, L. F. (2001) · Cited by 3,153
Self-citationSeparate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089
The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks
Ratcliff, R., & McKoon, G. (2008) · Neural Computation · doi:10.1162/neco.2008.12-06-420 · Cited by 3,059
Stastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. O. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,901
NEURAL MECHANISMS OF ADDICTION: The Role of Reward-Related Learning and Memory
Hyman, S. E., Malenka, R. C., & Nestler, E. J. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113009 · Cited by 2,647
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationThe debate over dopamine’s role in reward: the case for incentive salience
Berridge, K. C. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0578-x · Cited by 2,347
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Self-citationOptimal Foraging Theory: A Critical Review
Pyke, G. H. (1984) · Annual Review of Ecology and Systematics · doi:10.1146/annurev.es.15.110184.002515 · Cited by 2,205
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187
The time course of perceptual choice: The leaky, competing accumulator model.
Usher, M., & McClelland, J. L. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.550 · Cited by 2,152
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Self-citationBayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958
Bogacz, R., Brown, E., Moehlis, J., Holmes, P. J., & Cohen, J. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.700 · Cited by 1,945
The neural correlates of subjective value during intertemporal choice
Kable, J. W., & Glimcher, P. W. (2007) · Nature Neuroscience · doi:10.1038/nn2007 · Cited by 1,916
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A., & Puterman, M. L. (1995) · Technometrics · doi:10.2307/1269932 · Cited by 1,889
Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping
Ng, A. Y., Harada, D., & Russell, S. (1999) · Cited by 1,744
Neural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721
Roitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589 · cited 2×
“Figure 10 shows schematic results of an experiment (cartooned in part A) in which animals had a free choice of decision time, reporting their decision by making a saccade from a fixation point to one of two targets (Roitman and Shadlen, 2002; Gold and Shadlen, 2001, 2007).”
BackgroundMethodTemporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Self-citationBertsekas, D. P. (2001) · Springer eBooks · doi:10.1007/0-306-48332-7_333 · Cited by 1,519
Modeling Response Times for Two-Choice Decisions
Ratcliff, R., & Rouder, J. N. (1998) · Psychological Science · doi:10.1111/1467-9280.00067 · Cited by 1,478
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
The Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390
Sutton, R. S. (1990) · Elsevier eBooks · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,381
Probabilistic decision making by slow reverberation in cortical circuits
Wang, X.-J. (2002) · Neuron · doi:10.1016/s0896-6273(02)01092-9 · Cited by 1,242
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Psychology and neurobiology of simple decisions
Smith, P. L., & Ratcliff, R. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.01.006 · Cited by 1,051
A relationship between behavioral choice and the visual responses of neurons in macaque MT
Britten, K. H., Newsome, W. T., Shadlen, M. N., Celebrini, S., & Movshon, J. A. (1996) · Visual Neuroscience · doi:10.1017/s095252380000715x · Cited by 1,049
Multi-Armed Bandit Allocation Indices.
Bather, J. A., & Gittins, J. C. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982809 · Cited by 1,043
Ainslie, G. W. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139164191 · Cited by 995
Neural computations that underlie decisions about sensory stimuli
Gold, J. I., & Shadlen, M. N. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01567-9 · Cited by 881
Shadlen, M., Britten, K., Newsome, W., & Movshon, J. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-04-01486.1996 · Cited by 824
Noise characteristics and prior expectations in human visual speed perception
Stocker, A. A., & Simoncelli, E. P. (2006) · Nature Neuroscience · doi:10.1038/nn1669 · Cited by 809
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.6.1185 · Cited by 801
SENSE AND THE SINGLE NEURON: Probing the Physiology of Perception
Parker, A. J., & Newsome, W. T. (1998) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.21.1.227 · Cited by 801
Dynamic Modeling in Behavioral Ecology.
Caraco, T. B., Newman, J. A., Mangel, M., & Clark, C. (1990) · Evolution · doi:10.2307/2409517 · Cited by 761
Coordination of Actions and Habits in the Medial Prefrontal Cortex of Rats
Killcross, S. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.4.400 · Cited by 748
Motion perception: seeing and deciding.
Shadlen, M. N., & Newsome, W. T. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.2.628 · Cited by 723
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Dynamic Modeling in Behavioral Ecology
Mangel, M., & Clark, C. (1989) · Princeton University Press eBooks · doi:10.1515/9780691206967 · Cited by 623
Temporal difference models describe higher-order learning in humans
Seymour, B., O'Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., Friston, K. J., & Frackowiak, R. S. (2004) · Nature · doi:10.1038/nature02581 · Cited by 605
Self-citationTest of optimal sampling by foraging great tits
Krebs, J. R., Kacelnik, A., & Taylor, P. (1978) · Nature · doi:10.1038/275027a0 · Cited by 599
Kaelbling, L. P. (1993) · doi:10.7551/mitpress/4168.001.0001 · Cited by 599
Probabilistic reasoning by neurons
Yang, T., & Shadlen, M. N. (2007) · Nature · doi:10.1038/nature05852 · Cited by 597
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (2005) · Cited by 588
Bandit Problems, Sequential Allocation of Experiments.
Jones, P. W., Berry, D. A., & Fristedt, B. (1987) · Journal of the Operational Research Society · doi:10.2307/2582852 · Cited by 583
Gold, J. I., & Shadlen, M. N. (2002) · Neuron · doi:10.1016/s0896-6273(02)00971-6 · Cited by 561
Optimal representation of sensory information by neural populations
Jazayeri, M., & Movshon, J. A. (2006) · Nature Neuroscience · doi:10.1038/nn1691 · Cited by 548
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525
Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks
Lo, C.-C., & Wang, X.-J. (2006) · Nature Neuroscience · doi:10.1038/nn1722 · Cited by 505
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
Berger, J. O. (1989) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-20181-5_26 · Cited by 496
Dopamine: generalization and bonuses
Kakade, S. M., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 495
Self-citationThe computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 492
Self-citationAdaptive Critics and the Basal Ganglia
Barto, A. G. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4708.003.0018 · Cited by 476
Wald, A. (1947) · CERN Document Server (European Organization for Nuclear Research) · Cited by 446
Bayesian integration of visual and auditory signals for spatial localization
Battaglia, P., Jacobs, R. A., & Aslin, R. Ν. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001391 · Cited by 443
Decisions uncertainty. and the brain: The science of neuroeconomics
(2003) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(03)90161-1 · Cited by 419
NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE
Hilton, W. (1983) · Cited by 376
Dearden, R., Friedman, N., & Russell, S. (1998) · Cited by 363
Bandit Problems: Sequential Allocation of Experiments.
Bather, J., Berry, D., & Fristedt, B. E. (1986) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981558 · Cited by 336
Probabilistic Interpretation of Population Codes
Zemel, R. S., Dayan, P., & Pouget, A. (1998) · Neural Computation · doi:10.1162/089976698300017818 · Cited by 333
Self-citationOptimal integration of texture and motion cues to depth
Jacobs, R. A. (1999) · Vision Research · doi:10.1016/s0042-6989(99)00088-7 · Cited by 324
Reinforcement learning with perceptual aliasing: the perceptual distinctions approach
Chrisman, L. (1992) · Cited by 316
Bayesian Spiking Neurons I: Inference
Deneve, S. (2008) · Neural Computation · doi:10.1162/neco.2008.20.1.91 · Cited by 313
Decision theory: what “should” the nervous system do?
Körding, K. (2007) · Science · doi:10.1126/science.1142998 · Cited by 313
Probabilistic Interpretation of Population Codes
Zemel, R. S., Dayan, P., & Pouget, A. (1999) · The MIT Press eBooks · doi:10.7551/mitpress/4924.003.0011 · Cited by 308
Self-citationHaruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., Imamizu, H., & Kawato, M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3417-03.2004 · Cited by 300 · cited 2×
“…2004; Barto, 1995; Montague et al., 1996; Schultz et al., 1997; Suri and Schultz, 1998; Joel et al., 2002; Balleine et al., 2007; Daw and Doya, 2006; Haruno et al., 2004) in a way that ties together the at least equally extensive data on the psychology of instrumental choice with these neural…”
BackgroundThe application of statistical decision theory to animal behaviour
McNamara, J. M., & Houston, A. I. (1980) · Journal of Theoretical Biology · doi:10.1016/0022-5193(80)90265-9 · Cited by 300
Striatal activity underlies novelty-based choice in humans
Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008) · Neuron · doi:10.1016/j.neuron.2008.04.027 · Cited by 277
Self-citationStatistical decision theory and the selection of rapid, goal-directed movements
Trommershäuser, J., Maloney, L. T., & Landy, M. S. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001419 · Cited by 276
Dopaminergic Mechanisms in Actions and Habits: Figure 1.
Wickens, J. R., Horvitz, J. C., Costa, R. M., & Killcross, S. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1671-07.2007 · Cited by 271
Value-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268
What is reinforced by phasic dopamine signals?
Redgrave, P., GURNEY, K. N., & Reynolds, J. N. J. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.10.007 · Cited by 261
Bayesian Computation in Recurrent Neural Circuits
Rao, R. P. N. (2004) · Neural Computation · doi:10.1162/08997660460733976 · Cited by 252
Bayesian decision theory and psychophysics
Yuille, A. L., & Bülthoff, H. H. (1996) · Perception as Bayesian Inference · doi:10.1017/cbo9780511984037.006 · Cited by 250 · cited 4×
“…and Clark, 1989; Montague, 2006; Gold and Shadlen, 2002, 2007; Glimcher, 2004; Körding, 2007; Gittins, 1989; Berry and Fristedt, 1985; Wald, 1947; Yuille and Bülthoff, 1996; McNamara and Houston, 1980), and is therefore a critical tool for modeling, understanding and predicting psychological…”
BackgroundMethodInfluentialStatistical decision theory and trade-offs in the control of motor response
Maloney, L. T., Landy, M. S., & Trommershäuser, J. (2003) · Spatial Vision · doi:10.1163/156856803322467527 · Cited by 225
Resolution of Uncertainty in Prefrontal Cortex
Yoshida, W., & Ishii, S. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.006 · Cited by 201
Learning of sequential movements by neural network model with dopamine-like reinforcement signal
Suri, R. E., & Schultz, W. (1998) · Experimental Brain Research · doi:10.1007/s002210050467 · Cited by 198
Representation and timing in theories of the dopamine system
Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1637 · Cited by 173
Self-citation(2007) · MPG.PuRe (Max Planck Society) · Cited by 169
Self-citationMulti-Armed Bandit Allocation Indices
Jones, P. W., & Gittins, J. C. (1989) · Journal of the Operational Research Society · doi:10.2307/2582928 · Cited by 165
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Jaakkola, T., Jordan, M. I., & Singh, S. P. (1993) · doi:10.21236/ada276517 · Cited by 161
Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model
Wickens, J. R. (1990) · Journal of Neural Transmission · doi:10.1007/bf01245020 · Cited by 145
Plastic Corticostriatal Circuits for Action Learning
Costa, R. M. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.015 · Cited by 126
Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes
Whiteley, L., & Sahani, M. (2008) · Journal of Vision · doi:10.1167/8.3.2 · Cited by 120
Exploration Bonuses and Dual Control
Dayan, P., & B, T. J. S. A. (1996) · Machine Learning · doi:10.1023/a:1018357105171 · Cited by 120
Self-citationHumans Rapidly Estimate Expected Gain in Movement Planning
Trommershäuser, J., Landy, M. S., & Maloney, L. T. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01816.x · Cited by 113
(1993) · Choice Reviews Online · doi:10.5860/choice.31-1560 · Cited by 101
The Speed and Accuracy of a Simple Perceptual Decision: A Mathematical Primer
Shadlen, M. N., Hanks, T. D., Churchland, A. K., Kiani, R., & Yang, T. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262042383.003.0010 · Cited by 95
Doubly Distributional Population Codes: Simultaneous Representation of Uncertainty and Multiplicity
Sahani, M., & Dayan, P. (2003) · Neural Computation · doi:10.1162/089976603322362356 · Cited by 93
Self-citationCohen, J. D., McClure, S. M., & Yu, A. J. (2008) · doi:10.1093/oso/9780199230617.003.0014 · Cited by 84
Why Choose This Book?: How We Make Decisions
Montague, P. R. (2006) · Medical Entomology and Zoology · Cited by 84
Separate Neural Systems Value Immediate and Delayed Monetary Rewards
McClure, S. M., Laibson, D., Loewenstein, G. F., & Cohen, J. (2007) · doi:10.1093/oso/9780199257072.003.0022 · Cited by 66
An exploration-exploitation model based on norepinepherine and dopamine activity
McClure, S. M., Gilzenrat, M. S., & Cohen, J. D. (2005) · neural information processing systems · Cited by 61
Introduction to Stochastic Dynamic Programming: Probability and Mathematical
Ross, S. M. (1983) · Academic Press, Inc. eBooks · Cited by 51
Exact Inferences in a Neural Implementation of a Hidden Markov Model
Beck, J. M., & Pouget, A. (2007) · Neural Computation · doi:10.1162/neco.2007.19.5.1344 · Cited by 47
Probabilistic population codes and the exponential family of distributions
Beck, J. M., Ma, W. J., Latham, P. E., & Pouget, A. (2007) · Progress in brain research · doi:10.1016/s0079-6123(06)65032-2 · Cited by 40
Toward a Prefrontal Microcircuit Model for Cognitive Deficits in Schizophrenia
Wang, X. (2006) · Pharmacopsychiatry · doi:10.1055/s-2006-931501 · Cited by 35
The Speed and Accuracy of a Simple Perceptual Decision: A Mathematical Primer
Shadlen, M. N., Hanks, T. D., Churchland, A. K., Kiani, R., & Yang, T. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/1535.003.0016 · Cited by 24
Optimal Change-Detection and Spiking Neurons
Yu, A. J. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0198 · Cited by 22
Why Choose this Book? How we make decisions
Crotts, J. C. (2008) · Tourism Analysis · Cited by 6
Flagged first, then by citations
5 of the 131 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citationAction understanding as inverse planning
Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009) · Cognition · doi:10.1016/j.cognition.2009.07.005 · Cited by 920
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.11.018 · Cited by 809
Self-citationInformation-seeking, curiosity, and attention: computational and neural mechanisms
Gottlieb, J., Oudeyer, P.-Y., Lopes, M., & Baranes, A. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.001 · Cited by 784
Metadata
- Authors
- Dayan, P., & Daw, N. D.
- Affiliations
- University College London (GB)
- New York University (US)
- Corresponding author
- Dayan, P.
- Daw, N. D.
- ORCID
- Venue
- Cognitive, Affective, & Behavioral Neuroscience
- Published
- 2008-12-01
- Volume
- 8
- Issue
- 4
- Pages
- 429-453
- ISSN
- 1530-7026 (ISSN-L)
- 1531-135X
- OpenAlex
- W2037917209
- Cited by
- 580
- References
- 131
- Citations by year
- 2026: 25 · 2025: 52 · 2024: 46 · 2023: 33 · 2022: 43 · 2021: 34 · 2020: 55 · 2019: 38 · 2018: 26 · 2017: 37 · 2016: 25 · 2015: 27 · 2014: 25 · 2013: 26 · 2012: 37
- FWCI
- 1.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Cognitive Affective & Behavioral Neuroscience (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- decision-making, Bayesian decision theory, Markov decision process, signal detection theory, sequential sampling, model-based learning, model-free learning, reinforcement learning, computational models of decision making
- MeSH terms
- Algorithms, Animals, Bayes Theorem, Brain, Cognition, Cost-Benefit Analysis, Decision Making, Decision Theory, Exploratory Behavior, Humans, Markov Chains, Models, Psychological, Problem Solving, Reinforcement, Psychology, Models, Statistical, Signal Detection, Psychological, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
