The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Cortical substrates for exploratory decisions in humans
Open accessAbstract
Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The rewarding/motivating nature of “near misses” has been reported for gambling where they increase the desire to play (Clark et al., 2009; Daw et al., 2006; Kakade and Dayan, 2002).”
From Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models (2011) · cites this paper as Daw et al., 2006
In the citing paper
“A single set of parameters as frequently employed in our recent work (Daw et al., 2006; Gershman et al., 2009; Glascher et al., 2009) imposes a simple, but efficient regularization, which stabilizes the estimated model parameters.”
From States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning (2010) · cites this paper as Daw et al., 2006
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Optimal foraging, the marginal value theorem
Charnov, E. L. (1976) · Theoretical Population Biology · doi:10.1016/0040-5809(76)90040-x · Cited by 5,374
Separate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Self-citationDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Self-citationAbstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
Self-citationThe Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, É., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,939
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–155 of 155
Reward-related processing in the human brain: developmental considerations
Fareri, D. S., Martin, L. N., & Delgado, M. R. (2008) · Development and Psychopathology · doi:10.1017/s0954579408000576 · Cited by 141
Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources
Beeler, J. A., Frazier, C. R. M., & Zhuang, X. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00049 · Cited by 136
The impact of engagement with social networking sites (SNSs) on cognitive skills
Alloway, T. P., & Alloway, R. G. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2012.04.015 · Cited by 137
Computational and dynamic models in neuroimaging
Friston, K. J., & Dolan, R. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.068 · Cited by 131
Self-citationZhu, L., Mathewson, K. E., & Hsu, M. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1116783109 · Cited by 125
Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder
Rothkirch, M., Tonn, J., Köhler, S., & Sterzer, P. (2017) · Brain · doi:10.1093/brain/awx025 · Cited by 124
Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game
Seo, H., Barraclough, D. J., & Lee, D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1479-09.2009 · Cited by 125
Generalization of value in reinforcement learning by humans
Wimmer, G. E., Daw, N. D., & Shohamy, D. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08017.x · Cited by 111
Self-citationThe neural system of metacognition accompanying decision-making in the prefrontal cortex
Qiu, L., Su, J., Ni, Y., Bai, Y., Zhang, X., Li, X., & Wan, X. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004037 · Cited by 125
Gershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118
Self-citationDeep active inference as variational policy gradients
Millidge, B. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102348 · Cited by 115
Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum
Diederen, K. M. J., Ziauddeen, H., Vestergaard, M. D., Spencer, T., Schultz, W., & Fletcher, P. C. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.1979-16.2016 · Cited by 110
Pu, S., Matsumura, H., Yamada, T., Ikezawa, S., Mitani, H., Adachi, A., & Nakagome, K. (2008) · Psychiatry and Clinical Neurosciences · doi:10.1111/j.1440-1819.2008.01882.x · Cited by 112
Model-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data
Gläscher, J. P., & O'Doherty, J. P. (2010) · WIREs Cognitive Science · doi:10.1002/wcs.57 · Cited by 104
Self-citationConnectivity-based parcellation of the human frontal pole with diffusion tensor imaging
Liu, H., Qin, W., Li, W., Fan, L., Wang, J., Jiang, T., & Yu, C. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4882-12.2013 · Cited by 108
Cooper, J. C., Dunne, S., Furey, T., & O'Doherty, J. P. (2012) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00114 · Cited by 101
Self-citationAn insula hierarchical network architecture for active interoceptive inference
Fermin, A. S. R., Friston, K., & Yamawaki, S. (2022) · Royal Society Open Science · doi:10.1098/rsos.220226 · Cited by 101
Play, Curiosity, and Cognition
Chu, J., & Schulz, L. E. (2020) · Annual Review of Developmental Psychology · doi:10.1146/annurev-devpsych-070120-014806 · Cited by 93
Stoy, M., Schlagenhauf, F., Schlochtermeier, L., Wrase, J., Knutson, B., Lehmkuhl, U., Huss, M., Heinz, A., & Ströhle, A. (2011) · Psychopharmacology · doi:10.1007/s00213-011-2166-y · Cited by 91
The beautiful otherness of the autistic mind
Happé, F., & Frith, U. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0009 · Cited by 93
Scholl, J., Kolling, N., Nelissen, N., Wittmann, M. K., Harmer, C. J., & Rushworth, M. F. S. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0396-15.2015 · Cited by 85
Transient activation of midbrain dopamine neurons by reward risk
Fiorillo, C. D. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.09.037 · Cited by 90
Neural circuitry of information seeking
Bromberg-Martin, E. S., & Monosov, I. E. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.07.006 · Cited by 89
Axiomatic methods, dopamine and reward prediction error
Caplin, A., & Dean, M. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.007 · Cited by 88
The Algorithmic Level Is the Bridge Between Computation and Brain
Love, B. C. (2015) · Topics in Cognitive Science · doi:10.1111/tops.12131 · Cited by 82
Mind matters: placebo enhances reward learning in Parkinson's disease
Schmidt, L., Braun, E. K., Wager, T. D., & Shohamy, D. (2014) · Nature Neuroscience · doi:10.1038/nn.3842 · Cited by 84
Striatal structure and function predict individual biases in learning to avoid pain
Eldar, E., Hauser, T. U., Dayan, P., & Dolan, R. J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1519829113 · Cited by 79
Self-citationTrudel, N., Scholl, J., Klein-Flügge, M. C., Fouragnan, E., Tankelevitch, L., Wittmann, M. K., & Rushworth, M. F. S. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0929-3 · Cited by 74
The Role of the Parietal Cortex in the Representation of Task-Reward Associations
Wisniewski, D., Reverberi, C., Momennejad, I., Kahnt, T., & Haynes, J.-D. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.4882-14.2015 · Cited by 67
Baxter, M. G., Gaffan, D., Kyriazis, D. A., & Mitchell, A. S. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06740.x · Cited by 65
Fouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J., & Philiastides, M. G. (2017) · Scientific Reports · doi:10.1038/s41598-017-04507-w · Cited by 61
Nicotinic receptors in the ventral tegmental area promote uncertainty-seeking
Naudé, J., Tolu, S., Dongelmans, M., Torquet, N., Valverde, S., Rodriguez, G., Pons, S., Maskos, U., Mourot, A., Marti, F., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4223 · Cited by 61
Lloyd, A., McKay, R., Sebastian, C. L., & Balsters, J. H. (2021) · Developmental Science · doi:10.1111/desc.13075 · Cited by 49
Jepma (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00170 · Cited by 45
A rise-to-threshold process for a relative-value decision
Vijayan, V., Wang, F., Wang, K., Chakravorty, A., Adachi, A., Akhlaghpour, H., Dickson, B. J., & Maimon, G. (2023) · Nature · doi:10.1038/s41586-023-06271-6 · Cited by 49
Predictive coding networks for temporal prediction
Millidge, B., Tang, M., Osanlouy, M., Harper, N. S., & Bogacz, R. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011183 · Cited by 44
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64
Active learning: learning a motor skill without a coach
Huang, V. S., Shadmehr, R., & Diedrichsen, J. (2008) · Journal of Neurophysiology · doi:10.1152/jn.01095.2007 · Cited by 41
Nicotinic alteration of decision-making
Naudé, J., Dongelmans, M., & Faure, P. (2015) · Neuropharmacology · doi:10.1016/j.neuropharm.2014.11.021 · Cited by 39
Conditioned associations and economic decision biases.
Guitart-Masip, M., Talmi, D., & Dolan, R. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.021 · Cited by 36
Self-citationNeural substrates of decision making as measured with the Iowa Gambling Task in men with alexithymia
Kano, M., Ito, M., & Fukudo, S. (2011) · Psychosomatic Medicine · doi:10.1097/psy.0b013e318223c7f8 · Cited by 35
Reward maximization through discrete active inference
Da Costa, L., Sajid, N., Parr, T., Friston, K., & Smith, R. (2023) · Neural Computation · doi:10.1162/neco_a_01574 · Cited by 35
Borst, J. P., Taatgen, N. A., & van Rijn, H. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.05.084 · Cited by 33
Medial Frontal Cortex Activity Predicts Information Sampling in Economic Choice
Kaanders, P., Nili, H., O'Reilly, J. X., & Hunt, L. (2021) · The Journal of Neuroscience · doi:10.1523/jneurosci.0392-21.2021 · Cited by 27
Reinen, J. M., Whitton, A. E., Pizzagalli, D. A., Slifstein, M., Abi-Dargham, A., McGrath, P. J., Iosifescu, D. V., & Schneier, F. R. (2021) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2021.08.002 · Cited by 27
Empowerment contributes to exploration behaviour in a creative video game
Brändle, F., Stocks, L. J., Tenenbaum, J. B., Gershman, S. J., & Schulz, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01661-2 · Cited by 34
Anhedonia modulates the effects of positive mood induction on reward-related brain activation
Green, I. W., Pizzagalli, D. A., Admon, R., & Kumar, P. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.063 · Cited by 23
A subcortical switchboard for perseverative, exploratory and disengaged states
Ahmadlou, M., Shirazi, M. Y., Zhang, P., Rogers, I. L. M., Dziubek, J., Young, M., & Hofer, S. B. (2025) · Nature · doi:10.1038/s41586-025-08672-1 · Cited by 25
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2023) · doi:10.1101/2023.05.17.541226 · Cited by 18
Self-citationAubert‐Teillaud, B., Bran, A., & Vaidis, D. C. (2023) · European Journal of Social Psychology · doi:10.1002/ejsp.2981 · Cited by 19
A neural correlate of strategic exploration at the onset of adolescence
Kayser, A. S., Op de Macks, Z., Dahl, R. E., & Frank, M. J. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00896 · Cited by 15
He, J. M., Huang, X. T., Yuan, H., & Chen, Y. G. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2012.08.006 · Cited by 13
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Self-citationComparing brain connections in different species using diffusion weighted imaging
Rushworth, M. F. S., Boorman, E., & Mars, R. B. (2009) · Diffusion MRI · doi:10.1016/b978-0-12-374709-9.00020-1 · Cited by 5
Infraslow histaminergic dynamics govern priming states to gate moment-to-moment memory accessibility
Morishita, Y., Takamura, Y., Nishimura, K., Yokoi, Y., Ishihama, Y., Idutsu, R., Ono, M., Matsumoto, R., Hitora-Imamura, N., Minami, M., et al. (2026) · Neuron · doi:10.1016/j.neuron.2026.05.019
Metadata
- Authors
- Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J.
- Affiliations
- Gatsby Computational Neuroscience Unit (GB)
- University College London (GB)
- Corresponding author
- Daw, Nathaniel D.
- Venue
- Nature
- Published
- 2006-06
- Volume
- 441
- Issue
- 7095
- Pages
- 876-879
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC2635947
- OpenAlex
- W2077611535
- Cited by
- 2,224
- References
- 31
- Citations by year
- 2026: 80 · 2025: 114 · 2024: 127 · 2023: 125 · 2022: 115 · 2021: 136 · 2020: 166 · 2019: 118 · 2018: 143 · 2017: 121 · 2016: 108 · 2015: 103 · 2014: 111 · 2013: 97 · 2012: 107
- FWCI
- 40.78
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- CaltechAUTHORS (California Institute of Technology) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- Wellcome Trust (078865)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Keywords
- exploration-exploitation balance, curiosity, frontopolar cortex, intraparietal sulcus, striatum, ventromedial prefrontal cortex, gambling task, reinforcement learning, value-based decision making, human decision-making, computational modeling of decision making
- MeSH terms
- Choice Behavior, Decision Making, Exploratory Behavior, Gambling, Humans, Magnetic Resonance Imaging, Models, Neurological, Models, Psychological, Reward, Prefrontal Cortex, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
