The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation
Open accessAbstract
The basal ganglia support learning to exploit decisions that have yielded positive outcomes in the past. In contrast, limited evidence implicates the prefrontal cortex in the process of making strategic exploratory decisions when the magnitude of potential outcomes is unknown. Here we examine neurogenetic contributions to individual differences in these distinct aspects of motivated human behavior, using a temporal decision-making task and computational analysis. We show that two genes controlling striatal dopamine function, DARPP-32 (also called PPP1R1B) and DRD2, are associated with exploitative learning to adjust response times incrementally as a function of positive and negative decision outcomes. In contrast, a gene primarily controlling prefrontal dopamine function (COMT) is associated with a particular type of 'directed exploration', in which exploratory decisions are made in proportion to Bayesian uncertainty about whether other choices might produce outcomes that are better than the status quo. Quantitative model fits reveal that genetic factors modulate independent parameters of a reinforcement learning system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“These findings converge with the notion that dopamine has a key role in reinforcement learnin”
Depue, R. A., & Collins, P. F. (1999) · Behavioral and Brain Sciences · doi:10.1017/s0140525x99002046 · Cited by 2,046
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationHabits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 1×
“The findings also converge with rodent data showing that the transition to exploitative behavior is associated with the development of highly stabilized striatal firing pattern”
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Cited by
A high-coverage genome sequence from an archaic denisovan individual
Meyer, M., Kircher, M., Gansauge, M.-T., Li, H., Racimo, F., Mallick, S., Schraiber, J. G., Jay, F., Prüfer, K., de Filippo, C., et al. (2012) · Science · doi:10.1126/science.1224344 · Cited by 1,967
Inverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673
What determines our navigational abilities?
Wolbers, T., & Hegarty, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.001 · Cited by 810
Dissociable dopamine dynamics for learning and motivation
Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019) · Nature · doi:10.1038/s41586-019-1235-y · Cited by 735
Temporal structure of motor variability is dynamically regulated and predicts motor learning ability
Wu, H. G., Miyamoto, Y. R., Castro, L. N. G., Ölveczky, B. P., & Smith, M. A. (2014) · Nature Neuroscience · doi:10.1038/nn.3616 · Cited by 675
A Bayesian foundation for individual learning under uncertainty
Mathys, C. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00039 · Cited by 655
Neurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Humans use directed and random exploration to solve the explore–exploit dilemma.
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0038199 · Cited by 580
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558
“The results therefore reveal two fundamentally distinct human exploration processes: first, uncontrolled exploration stochastically selecting actions within actor task sets according to a softmax policy for learning behavioral strategies that maximize action utility [3,28,41], and second, controlled exploration occurr…”
BackgroundFair, D. A., Bathula, D., Nikolas, M. A., & Nigg, J. T. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1115365109 · Cited by 464
Learning from Sensory and Reward Prediction Errors during Motor Adaptation
Izawa, J., & Shadmehr, R. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002012 · Cited by 477
A Review of Reward Processing and Motivational Impairment in Schizophrenia
Strauss, G. P., Waltz, J. A., & Gold, J. M. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbt197 · Cited by 452
Jepma, M., & Nieuwenhuis, S. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21548 · Cited by 439
Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 417
Self-citationFrom Movement to Thought: Executive Function, Embodied Cognition, and the Cerebellum
Koziol, L. F., Budding, D. E., & Chidekel, D. (2012) · The Cerebellum · doi:10.1007/s12311-011-0321-y · Cited by 392
The neuromodulator of exploration: A unifying theory of the role of dopamine in personality
DeYoung, C. G. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00762 · Cited by 387
“Another case in which some might consider my use of the term “exploration” too broad comes in contexts where exploration has been contrasted with exploitation (Cohen et al., 2007; Frank et al., 2009).”
BackgroundForming beliefs: Why valence matters
Sharot, T., & Garrett, N. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2015.11.002 · Cited by 345
Gold, J. M. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.1269 · Cited by 317
Rostrolateral Prefrontal Cortex and Individual Differences in Uncertainty-Driven Exploration
Badre, D., Doll, B. B., Long, N. M., & Frank, M. J. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.025 · Cited by 309
Self-citationA computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305
Frontal Theta Reflects Uncertainty and Unexpectedness during Exploration and Exploitation
Cavanagh, J. F., Figueroa, C. M., Cohen, M. X., & Frank, M. J. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr332 · Cited by 291
Self-citationDeconstructing the human algorithms for exploration
Gershman, S. J. (2018) · Cognition · doi:10.1016/j.cognition.2017.12.014 · Cited by 267
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236
The neural representation of unexpected uncertainty during value-based decision making
Payzan-LeNestour, E., Dunne, S., Bossaerts, P., & O’Doherty, J. P. (2013) · Neuron · doi:10.1016/j.neuron.2013.04.037 · Cited by 257
Circuit Mechanisms of Sensorimotor Learning
Makino, H., Hwang, E. J., Hedrick, N. G., & Komiyama, T. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.029 · Cited by 228
Glutamatergic model psychoses: prediction error, learning, and inference
Corlett, P. R., Honey, G. D., Krystal, J. H., & Fletcher, P. C. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.163 · Cited by 229
A Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research
Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L., & Platt, M. L. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.108 · Cited by 219
Twenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
Dopaminergic Genes Predict Individual Differences in Susceptibility to Confirmation Bias
Doll, B. B., Hutchison, K. E., & Frank, M. J. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.6486-10.2011 · Cited by 184
Self-citationSerotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164
Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources
Beeler, J. A., Frazier, C. R. M., & Zhuang, X. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00049 · Cited by 136
Waltz, J. A., Frank, M. J., Wiecki, T. V., & Gold, J. M. (2011) · Neuropsychology · doi:10.1037/a0020882 · Cited by 132
Self-citationVive les differences! Individual variation in neural mechanisms of executive control
Braver, T. S., Cole, M. W., & Yarkoni, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.002 · Cited by 130
Transient activation of midbrain dopamine neurons by reward risk
Fiorillo, C. D. (2011) · Neuroscience · doi:10.1016/j.neuroscience.2011.09.037 · Cited by 90
Linking neurogenetics and individual differences in language learning: the dopamine hypothesis.
Wong, P. C. M., Morgan-Short, K., Ettlinger, M., & Zheng, J. (2012) · Cortex · doi:10.1016/j.cortex.2012.03.017 · Cited by 67
Nicotinic receptors in the ventral tegmental area promote uncertainty-seeking
Naudé, J., Tolu, S., Dongelmans, M., Torquet, N., Valverde, S., Rodriguez, G., Pons, S., Maskos, U., Mourot, A., Marti, F., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4223 · Cited by 61
Jepma (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00170 · Cited by 45
Cimino, S., Cerniglia, L., Ballarotto, G., Marzilli, E., Pascale, E., D’Addario, C., Adriani, W., & Tambelli, R. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00303 · Cited by 48
Depressive rumination and the C957T polymorphism of the DRD2 gene
Whitmer, A. J., & Gotlib, I. H. (2012) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-012-0112-z · Cited by 30
Lee, M. R., Gallen, C. L., Zhang, X., Hodgkinson, C. A., Goldman, D., Stein, E. A., & Barr, C. S. (2011) · PLoS ONE · doi:10.1371/journal.pone.0024203 · Cited by 27
Computational psychopharmacology: a translational and pragmatic approach
Robbins, T. W., & Cardinal, R. N. (2019) · Psychopharmacology · doi:10.1007/s00213-019-05302-3 · Cited by 25
A subcortical switchboard for perseverative, exploratory and disengaged states
Ahmadlou, M., Shirazi, M. Y., Zhang, P., Rogers, I. L. M., Dziubek, J., Young, M., & Hofer, S. B. (2025) · Nature · doi:10.1038/s41586-025-08672-1 · Cited by 25
Anhedonia modulates the effects of positive mood induction on reward-related brain activation
Green, I. W., Pizzagalli, D. A., Admon, R., & Kumar, P. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.063 · Cited by 23
A neural correlate of strategic exploration at the onset of adolescence
Kayser, A. S., Op de Macks, Z., Dahl, R. E., & Frank, M. J. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00896 · Cited by 15
“As in our previous work (Badre et al., 2012; Frank et al., 2009; Kayser, Mitchell, Weinstein, & Frank, 2015), on each trial participants observed a clock that completed a revolution over 5 seconds.”
ResultInfluentialSelf-citationColas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
(2025) · doi:10.1371/journal.pcbi.1012872 · Cited by 3
“While RL processes have been neurally linked to several brain regions including the ventral tegmental area (VTA), the striatum and cortico-basal-ganglia circuitry [23,24], WM processes have primarily been linked to the brain’s prefrontal cortical executive control systems [25–28].”
Metadata
- Authors
- Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F.
- Venue
- Nature Neuroscience
- Published
- 2009-07-20
- Volume
- 12
- Issue
- 8
- Pages
- 1062-1068
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1999611081
- Cited by
- 471
- References
- 54
- FWCI
- 11.92
- Type
- journal article
- Language
- en
- Update notices
- erratum (2010-05-01)
- Open access
- green
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.