Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Abstract
Involvement of dopamine in regulating exploration during decision-making has long been hypothesized, but direct causal evidence in humans is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging to address this issue. Thirty-one healthy male participants performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150 mg of the dopamine precursor L-dopa, 2 mg of the D2 receptor antagonist haloperidol, and placebo. Choices were best explained by an extension of an established Bayesian learning model accounting for perseveration, directed exploration and random exploration. Modeling revealed attenuated directed exploration under L-dopa, while neural signatures of exploration, exploitation and prediction error were unaffected. Instead, L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates how this circuit tracks accumulating uncertainty during decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
How do you feel — now? The anterior insula and human awareness
(Bud) Craig, A. D. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2555 · Cited by 6,657
Individual differences in working memory and reading
Daneman, M., & Carpenter, P. A. (1980) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(80)90312-6 · Cited by 6,336
How do you feel? Interoception: The sense of the physiological condition of the body
Craig, A. D. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn894 · Cited by 6,006
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Conflict monitoring and anterior cingulate cortex: an update
Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.10.003 · Cited by 3,623
Anterior Cingulate Conflict Monitoring and Adjustments in Control
Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004) · Science · doi:10.1126/science.1089910 · Cited by 2,891
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Large-scale brain networks in cognition: emerging methods and principles
Bressler, S. L., & Menon, V. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.04.004 · Cited by 2,348
The expected value of control: an integrative theory of anterior cingulate cortex function
Shenhav, A., Botvinick, M. M., & Cohen, J. D. (2013) · Neuron · doi:10.1016/j.neuron.2013.07.007 · Cited by 2,321
Some aspects of the sequential design of experiments
Robbins, H. (1952) · Bulletin of the American Mathematical Society · doi:10.1090/s0002-9904-1952-09620-8 · Cited by 2,184
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Salience processing and insular cortical function and dysfunction
Uddin, L. Q. (2015) · Nature Reviews Neuroscience · doi:10.1038/nrn3857 · Cited by 2,083
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
KRINGELBACH, M. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.03.006 · Cited by 1,977
The use of analogue scales in rating subjective feelings
BOND, A., & LADER, M. (1974) · British Journal of Medical Psychology · doi:10.1111/j.2044-8341.1974.tb02285.x · Cited by 1,972
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Bartra, O., McGuire, J. T., & Kable, J. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.063 · Cited by 1,927
Grace, A. A. (1991) · Neuroscience · doi:10.1016/0306-4522(91)90196-u · Cited by 1,831
Kane, M. J., Hambrick, D. Z., Tuholski, S. W., Wilhelm, O., Payne, T. W., & Engle, R. W. (2004) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.133.2.189 · Cited by 1,679
Inverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473
Reward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426
The root of all value: a neural common currency for choice
Levy, D. J., & Glimcher, P. W. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.06.001 · Cited by 1,342
A common role of insula in feelings, empathy and uncertainty
Singer, T., Critchley, H. D., & Preuschoff, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.05.001 · Cited by 1,270
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266
The anterior cingulate as a conflict monitor: fMRI and ERP studies
VANVEEN, V, & CARTER, C. (2002) · Physiology & Behavior · doi:10.1016/s0031-9384(02)00930-7 · Cited by 1,258
Neural mechanisms of autonomic, affective, and cognitive integration
Critchley, H. D. (2005) · Journal of Comparative Neurology · doi:10.1002/cne.20749 · Cited by 1,214
Committee report: Guidelines for human startle eyeblink electromyographic studies
Blumenthal, T. D., Cuthbert, B. N., Filion, D. L., Hackley, S., Lipp, O. V., & Van Boxtel, A. (2005) · Psychophysiology · doi:10.1111/j.1469-8986.2005.00271.x · Cited by 1,169
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
Optimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948
Cools, R. (2006) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2005.03.024 · Cited by 903
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
The hidden island of addiction: the insula
Naqvi, N. H., & Bechara, A. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2008.09.009 · Cited by 833
Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
Human insula activation reflects risk prediction errors as well as risk
Preuschoff, K., Quartz, S. R., & Bossaerts, P. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4286-07.2008 · Cited by 784
Neural differentiation of expected reward and risk in human subcortical structures
Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.024 · Cited by 692
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665
Dopamine signals for reward value and risk: basic and recent data
Schultz, W. (2010) · Behavioral and Brain Functions · doi:10.1186/1744-9081-6-24 · Cited by 653
Kolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
Menon, V. (2015) · Brain Mapping · doi:10.1016/b978-0-12-397025-1.00052-x · Cited by 625
Frontal Cortex and the Hierarchical Control of Behavior
Badre, D., & Nee, D. E. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.005 · Cited by 611
Dopamine reward prediction error coding
Schultz, W. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/wschultz · Cited by 597
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580
Kellendonk, C., Simpson, E. H., Polan, H. J., Malleret, G., Vronskaya, S., Winiger, V., Moore, H., & Kandel, E. R. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.023 · Cited by 520
Unsworth, N., Redick, T. S., Heitz, R. P., Broadway, J. M., & Engle, R. W. (2009) · Memory · doi:10.1080/09658210902998047 · Cited by 479
A Possible Role for the Striatum in the Pathogenesis of the Cognitive Symptoms of Schizophrenia
Simpson, E. H., Kellendonk, C., & Kandel, E. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.014 · Cited by 478
Measuring Working Memory Capacity With Automated Complex Span Tasks
Redick, T. S., Broadway, J. M., Meier, M. E., Kuriakose, P. S., Unsworth, N., Kane, M. J., & Engle, R. W. (2012) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000123 · Cited by 457
An automatic valuation system in the human brain: Evidence from functional neuroimaging
Lebreton, M., Jorge, S., Michel, V., Thirion, B., & Pessiglione, M. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.040 · Cited by 455
Neural correlates of value, risk, and risk aversion contributing to decision making under risk
Christopoulos, G. I., Tobler, P. N., Bossaerts, P., Dolan, R. J., & Schultz, W. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2614-09.2009 · Cited by 451
Midbrain dopamine and prefrontal function in humans: interaction and modulation by COMT genotype
Meyer-Lindenberg, A., Kohn, P. D., Kolachana, B., Kippenhan, S., McInerney-Leo, A., Nussbaum, R., Weinberger, D. R., & Berman, K. F. (2005) · Nature Neuroscience · doi:10.1038/nn1438 · Cited by 428
Mesocortical dopamine modulation of executive functions: beyond working memory
Floresco, S. B., & Magyar, O. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0404-5 · Cited by 426
Effects of bromocriptine on human subjects depend on working memory capacity
Kimberg, D. Y., DʼEsposito, M., & Farah, M. J. (1997) · Neuroreport · doi:10.1097/00001756-199711100-00032 · Cited by 413
Neural representations of subjective reward value
Peters, J., & Büchel, C. (2010) · Behavioural Brain Research · doi:10.1016/j.bbr.2010.04.031 · Cited by 405
Self-citationSchönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Knowing how much you don't know: a neural organization of uncertainty estimates
Bach, D. R., & Dolan, R. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3289 · Cited by 339
Linking nucleus accumbens dopamine and blood oxygenation
Knutson, B., & Gibbs, S. E. B. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0686-7 · Cited by 337
Floresco, S. B. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00062 · Cited by 336
Working Memory Capacity Predicts Dopamine Synthesis Capacity in the Human Striatum
Cools, R., Gibbs, S. E., Miyakawa, A., Jagust, W., & D'Esposito, M. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4475-07.2008 · Cited by 302
Dopamine, Time, and Impulsivity in Humans
Pine, A., Shiner, T., Seymour, B., & Dolan, R. J. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6028-09.2010 · Cited by 300
Spontaneous eye blink rate as predictor of dopamine-related cognitive function—A review
Jongkees, B. J., & Colzato, L. S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.08.020 · Cited by 300
Jocham, G., Klein, T. A., & Ullsperger, M. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3904-10.2011 · Cited by 278
Deconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267
Dopamine modulates novelty seeking behavior during decision making.
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2014) · Behavioral Neuroscience · doi:10.1037/a0037128 · Cited by 263
D1 and D2 dopamine receptor mRNA expression in whole hemisphere sections of the human brain
Hurd, Y. L., Suzuki, M., & Sedvall, G. C. (2001) · Journal of Chemical Neuroanatomy · doi:10.1016/s0891-0618(01)00122-3 · Cited by 262
Explicit neural signals reflecting reward uncertainty
Schultz, W., Preuschoff, K., Camerer, C., Hsu, M., Fiorillo, C. D., Tobler, P. N., & Bossaerts, P. (2008) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2008.0152 · Cited by 239
Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1417219112 · Cited by 233
Contributions of the ventromedial prefrontal cortex to goal-directed action selection
O’Doherty, J. P. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06290.x · Cited by 225
A Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research
Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L., & Platt, M. L. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.108 · Cited by 219
GIBBS, S. E. B., & D'ESPOSITO, M. (2005) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.5.2.212 · Cited by 145
Role of Dopamine D2 Receptors in Human Reinforcement Learning
Eisenegger, C., Naef, M., Linssen, A., Clark, L., Gandamaneni, P. K., Müller, U., & Robbins, T. W. (2014) · Neuropsychopharmacology · doi:10.1038/npp.2014.84 · Cited by 134
Automatic versus Choice-Dependent Value Representations in the Human Brain
Grueschow, M., Polania, R., Hare, T. A., & Ruff, C. C. (2015) · Neuron · doi:10.1016/j.neuron.2014.12.054 · Cited by 130
van der Schaaf, M. E., van Schouwenburg, M. R., Geurts, D. E. M., Schellekens, A. F. A., Buitelaar, J. K., Verkes, R. J., & Cools, R. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs344 · Cited by 102
Genetic Modulation of Transcranial Direct Current Stimulation Effects on Cognition
Wiegand, A., Nieratschker, V., & Plewnia, C. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00651 · Cited by 52
Gibbs, S. E. B., & D’Esposito, M. (2006) · Neuroscience · doi:10.1016/j.neuroscience.2005.11.055 · Cited by 49
Cited by
A subcortical switchboard for perseverative, exploratory and disengaged states
Ahmadlou, M., Shirazi, M. Y., Zhang, P., Rogers, I. L. M., Dziubek, J., Young, M., & Hofer, S. B. (2025) · Nature · doi:10.1038/s41586-025-08672-1 · Cited by 25
Lloyd, A., Roiser, J. P., Skeen, S., Freeman, Z., Badalova, A., Agunbiade, A., Busakhwe, C., DeFlorio, C., Marcu, A., Pirie, H., et al. (2024) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-024-01186-9 · Cited by 17
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Environmental dynamics shape human learning: change points versus random walks
Foucault, C., Weber, L. A., & Hunt, L. (2025) · doi:10.1101/2025.11.26.690700 · Cited by 2
Metadata
- Authors
- Chakroun, K., Mathar, D., Wiehler, A., Ganzer, F., & Peters, J.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2020-06-02
- Volume
- 9
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W3030493409
- Cited by
- 128
- References
- 83
- FWCI
- 8.01
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.