The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Cortical substrates for exploratory decisions in humans
Open accessAbstract
Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The rewarding/motivating nature of “near misses” has been reported for gambling where they increase the desire to play (Clark et al., 2009; Daw et al., 2006; Kakade and Dayan, 2002).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Daw et al., 2006
In the citing paper
“A single set of parameters as frequently employed in our recent work (Daw et al., 2006; Gershman et al., 2009; Glascher et al., 2009) imposes a simple, but efficient regularization, which stabilizes the estimated model parameters.”
From States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning (2010) · cites this paper as Daw et al., 2006
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Optimal foraging, the marginal value theorem
Charnov, E. L. (1976) · Theoretical Population Biology · doi:10.1016/0040-5809(76)90040-x · Cited by 5,374
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Self-citationAbstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
Self-citationThe Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, É., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,939
Cited by
Sorted by citations
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
The neural correlates of subjective value during intertemporal choice
Kable, J. W., & Glimcher, P. W. (2007) · Nature Neuroscience · doi:10.1038/nn2007 · Cited by 1,916
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Self-citationBalleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Self-citationSocial cognition and the brain: A meta-analysis
Van Overwalle, F. (2009) · Human Brain Mapping · doi:10.1002/hbm.20547 · Cited by 1,792
The Social Brain: Neural Basis of Social Knowledge
Adolphs, R. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163514 · Cited by 1,827
Cognitive control in media multitaskers
Ophir, E., Nass, C., & Wagner, A. D. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0903620106 · Cited by 1,810
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Self-citationFrontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
The proactive brain: using analogies and associations to generate predictions
Bar, M. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.05.005 · Cited by 1,144
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055
Human volition: towards a neuroscience of will
Haggard, P. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2497 · Cited by 1,134
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133
Self-citationCohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Somerville, L. H., Jones, R. M., & Casey, B. J. (2010) · Brain and Cognition · doi:10.1016/j.bandc.2009.07.003 · Cited by 1,025
The Psychology and Neuroscience of Curiosity
Kidd, C., & Hayden, B. Y. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.010 · Cited by 1,026
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citationAssociative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925
Liu, X., Hairston, J., Schrier, M., & Fan, J. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.012 · Cited by 916
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Relating Introspective Accuracy to Individual Differences in Brain Structure
Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010) · Science · doi:10.1126/science.1191883 · Cited by 809
Self-citationChoice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769
Self-citationInformatic parcellation of the network involved in the computation of subjective value
Clithero, J. A., & Rangel, A. (2014) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nst106 · Cited by 706
Emotion and the prefrontal cortex: an integrative review
Dixon, M. L., Thiruchselvam, R., Todd, R., & Christoff, K. (2017) · Psychological Bulletin · doi:10.1037/bul0000096 · Cited by 745
Active inference and epistemic value
Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., & Pezzulo, G. (2015) · Cognitive Neuroscience · doi:10.1080/17588928.2015.1020053 · Cited by 731
A Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Kolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
Changes of mind in decision-making
Resulaj, A., Kiani, R., Wolpert, D. M., & Shadlen, M. N. (2009) · Nature · doi:10.1038/nature08275 · Cited by 678
Decision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679
“It will be interesting to study how, according to behavioral demands, the brain can exploit a known environment or explore optimal options in a volatile world (Daw et al., 2006; Behrens et al., 2007).”
BackgroundHumans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580
Disorders of compulsivity: A common bias towards learning habits
Voon, V, Derbyshire, K., Rück, C., Irvine, M. A., Worbe, Y., Enander, J., Schreiber, L. R. N., Gillan, C., Fineberg, N. A., Sahakian, B. J., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.44 · Cited by 643
Self-citationBoorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665
Doya, K. (2008) · Nature Neuroscience · doi:10.1038/nn2077 · Cited by 642
Drug addiction: the neurobiology of disrupted self-control
Baler, R. D., & Volkow, N. D. (2006) · Trends in Molecular Medicine · doi:10.1016/j.molmed.2006.10.005 · Cited by 640
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Self-citationNeural computations associated with goal-directed choice
Rangel, A., & Hare, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.001 · Cited by 532
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558
“The results therefore reveal two fundamentally distinct human exploration processes: first, uncontrolled exploration stochastically selecting actions within actor task sets according to a softmax policy for learning behavioral strategies that maximize action utility [3,28,41], and second, controlled exploration occurr…”
BackgroundMethodResultInfluentialRisky business: the neuroeconomics of decision making under uncertainty
Platt, M. L., & Huettel, S. A. (2008) · Nature Neuroscience · doi:10.1038/nn2062 · Cited by 556
Determining the neural substrates of goal-directed learning in the human brain
Valentin, V. V., Dickinson, A., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0564-07.2007 · Cited by 529
Self-citationUncertainty in perception and the Hierarchical Gaussian Filter
Mathys, C. D., Lomakina, E. I., Daunizeau, J., Iglesias, S., Brodersen, K. H., Friston, K. J., & Stephan, K. E. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00825 · Cited by 465
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
Self-citationHuang, V. S., Haith, A., Mazzoni, P., & Krakauer, J. W. (2011) · Neuron · doi:10.1016/j.neuron.2011.04.012 · Cited by 472
The organization of dorsal frontal cortex in humans and macaques
Sallet, J., Mars, R. B., Noonan, M. P., Neubert, F.-X., Jbabdi, S., O'Reilly, J. X., Filippini, N., Thomas, A. G., & Rushworth, M. F. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.5108-12.2013 · Cited by 416
Brain mediators of cardiovascular responses to social threat
Wager, T. D., Waugh, C. E., Lindquist, M., Noll, D. C., Fredrickson, B. L., & Taylor, S. F. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.05.043 · Cited by 402
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.4652-10.2011 · Cited by 379
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Self-citationGläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387
Self-citationNeural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.5498-10.2012 · Cited by 356
Self-citationGambling near-misses enhance motivation to gamble and recruit win-related brain circuitry
Clark, L., Lawrence, A. J., Astley-Jones, F., & Gray, N. (2009) · Neuron · doi:10.1016/j.neuron.2008.12.031 · Cited by 410
Neural representations of subjective reward value
Peters, J., & Büchel, C. (2010) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.04.031 · Cited by 405
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346
A computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324
Mapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310
Self-citationKable, J. W., & Glimcher, P. W. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00177.2009 · Cited by 316
Functionally dissociable influences on learning rate in a dynamic environment
McGuire, J. T., Nassar, M. R., Gold, J. I., & Kable, J. W. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.013 · Cited by 299
Neural computations underlying action-based decision making in the human brain
Wunderlich, K., Rangel, A., & O'Doherty, J. P. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0901077106 · Cited by 301
Self-citationEmotion and decision-making: affect-driven belief systems in anxiety and depression
Paulus, M. P., & Yu, A. J. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.009 · Cited by 287
A reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292
“If the payoffs for the same option change often and unpredictably, then rewards should be filtered on short timescales to track the fast changes in a volatile environment; in contrast, if past rewards reliably predict future ones, then they should be filtered on long timescales to exploit a stable environmen”
BackgroundAdaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286
The Role of Human Orbitofrontal Cortex in Value Comparison for Incommensurable Objects
FitzGerald, T. H. B., Seymour, B., & Dolan, R. J. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0717-09.2009 · Cited by 279
Self-citationStriatal activity underlies novelty-based choice in humans
Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008) · Neuron · doi:10.1016/j.neuron.2008.04.027 · Cited by 277
Self-citationClan mentality: evidence that the medial prefrontal cortex responds to close others
Krienen, F. M., Tu, P.-C., & Buckner, R. L. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2180-10.2010 · Cited by 256
“It is thus important to note that explicitly nonsocial tasks also engage the frontal midline, in particular those that require calculating expected reward or value of a choice (e.g., Daw et al., 2006; see also Beckmann et al., 2009 for meta-analysis).”
BackgroundLateral habenula neurons signal errors in the prediction of reward information
Bromberg-Martin, E. S., & Hikosaka, O. (2011) · Nature Neuroscience · doi:10.1038/nn.2902 · Cited by 275
The neural code of reward anticipation in human orbitofrontal cortex
Kahnt, T., Heinzle, J., Park, S. Q., & Haynes, J.-D. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912838107 · Cited by 274
Striatal dysfunction during reversal learning in unmedicated schizophrenia patients
Schlagenhauf, F., Huys, Q. J. M., Deserno, L., Rapp, M. A., Beck, A., Heinze, H.-J., Dolan, R., & Heinz, A. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.11.034 · Cited by 253
Self-citationDeconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257
Self-citationMedial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
Basal Ganglia Circuits for Reward Value–Guided Behavior
Hikosaka, O., Kim, H. F., Yasuda, M., & Yamamoto, S. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-013924 · Cited by 259
Mice alternate between discrete strategies during perceptual decision-making
Ashwood, Z. C., Roy, N. A., Stone, I. R., Laboratory, T. I. B., Urai, A. E., Churchland, A. K., Pouget, A., & Pillow, J. W. (2022) · Nature Neuroscience · doi:10.1038/s41593-021-01007-z · Cited by 229
Serotonin Selectively Modulates Reward Value in Human Decision-Making
Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0053-12.2012 · Cited by 252
Self-citationMultiple signals in anterior cingulate cortex
Kolling, N., Behrens, T., Wittmann, M., & Rushworth, M. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2015.12.007 · Cited by 249
The dopaminergic midbrain encodes the expected certainty about desired outcomes
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., & Friston, K. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu159 · Cited by 219
Self-citationDynamic updating of hippocampal object representations reflects new conceptual knowledge
Mack, M. L., Love, B. C., & Preston, A. R. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1614048113 · Cited by 223
Active inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226
“Decision-making tasks are a prime candidate for this, as they often entail a series of discrete alternatives that an agent needs to choose among (e.g., multi-arm bandit tasks [23–25], multi-step decision tasks [26]).”
BackgroundCircuit Mechanisms of Sensorimotor Learning
Makino, H., Hwang, E. J., Hedrick, N. G., & Komiyama, T. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.029 · Cited by 228
“In humans, blood oxygen level-dependent (BOLD) signals in the rostral PFC and the intraparietal sulcus increase in explorative trials during reinforcement learning (Daw et al., 2006).”
Separate Neural Mechanisms Underlie Choices and Strategic Preferences in Risky Decision Making
Venkatraman, V., Payne, J. W., Bettman, J. R., Luce, M. F., & Huettel, S. A. (2009) · Neuron · doi:10.1016/j.neuron.2009.04.007 · Cited by 222
Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6376-10.2011 · Cited by 216
Self-citationContributions of the Amygdala to Reward Expectancy and Choice Signals in Human Prefrontal Cortex
Hampton, A. N., Adolphs, R., Tyszka, J. M., & O'Doherty, J. P. (2007) · Neuron · doi:10.1016/j.neuron.2007.07.022 · Cited by 213
Self-citationBayesian Prediction and Evaluation in the Anterior Cingulate Cortex
Ide, J. S., Shenoy, P., Yu, A. J., & Li, C.-S. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.2201-12.2013 · Cited by 185
Behavioral and Neural Properties of Social Reinforcement Learning
Jones, R. M., Somerville, L. H., Li, J., Ruberry, E. J., Libby, V., Glover, G., Voss, H. U., Ballon, D. J., & Casey, B. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.2972-11.2011 · Cited by 191
Cognitive Control: Core Constructs and Current Considerations
Cohen, J. D. (2017) · The Wiley Handbook of Cognitive Control · doi:10.1002/9781118920497.ch1 · Cited by 179
Summerfield, C., & Tsetsos, K. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00070 · Cited by 167
Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement
Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. S. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08023.x · Cited by 169
Stable representations of decision variables for flexible behavior
Bari, B. A., Grossman, C. D., Lubin, E. E., Rajagopalan, A. E., Cressy, J. I., & Cohen, J. Y. (2019) · Neuron · doi:10.1016/j.neuron.2019.06.001 · Cited by 173
Gruber, A. J., & McDonald, R. J. (2012) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2012.00050 · Cited by 173
Endocannabinoid signaling is critical for habit formation
Hilário, M. R. F. (2007) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.006.2007 · Cited by 164
“Interestingly, recent studies in humans showed that activity in the caudate nucleus of the dorsal striatum (roughly the dorsomedial striatum in rodents) is correlated with the value-based exploitation, in an exploitation-exploration task (Daw et al., 2006).”
Observing the observer (I): meta-bayesian models of learning and decision-making
Daunizeau, J., den Ouden, H. E. M., Pessiglione, M., Kiebel, S. J., Stephan, K. E., & Friston, K. J. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015554 · Cited by 167
Pezzulo, G., Rigoli, F., & Chersi, F. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00092 · Cited by 163
Assessing inter individual differences with task related functional neuroimaging
Lebreton, M., Bavard, S., Daunizeau, J., & Palminteri, S. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0681-8 · Cited by 160
Behavioural and neural evidence for self-reinforcing expectancy effects on pain
Jepma, M., Koban, L., van Doorn, J., Jones, M., & Wager, T. D. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0455-8 · Cited by 155
The human amygdala and orbital prefrontal cortex in behavioural regulation
Dolan, R. J. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2088 · Cited by 150
Self-citationEarly adversity and learning: implications for typical and atypical behavioral development
Hanson, J. L., van den Bos, W., Roeber, B. J., Rudolph, K. D., Davidson, R. J., & Pollak, S. D. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12694 · Cited by 136
Functional connectivity of reward processing in the brain
Camara, E. (2008) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.019.2008 · Cited by 141
Metadata
- Authors
- Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J.
- Affiliations
- Gatsby Computational Neuroscience Unit (GB)
- University College London (GB)
- Corresponding author
- Daw, Nathaniel D.
- Venue
- Nature
- Published
- 2006-06
- Volume
- 441
- Issue
- 7095
- Pages
- 876-879
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC2635947
- OpenAlex
- W2077611535
- Cited by
- 2,224
- References
- 31
- Citations by year
- 2026: 80 · 2025: 114 · 2024: 127 · 2023: 125 · 2022: 115 · 2021: 136 · 2020: 166 · 2019: 118 · 2018: 143 · 2017: 121 · 2016: 108 · 2015: 103 · 2014: 111 · 2013: 97 · 2012: 107
- FWCI
- 40.78
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- Wellcome Trust (078865)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- exploration-exploitation balance, curiosity, frontopolar cortex, intraparietal sulcus, striatum, ventromedial prefrontal cortex, gambling task, reinforcement learning, value-based decision making, human decision-making, computational modeling of decision making
- MeSH terms
- Choice Behavior, Decision Making, Exploratory Behavior, Gambling, Humans, Magnetic Resonance Imaging, Models, Neurological, Models, Psychological, Reward, Prefrontal Cortex, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
