The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation
Open accessAbstract
The basal ganglia support learning to exploit decisions that have yielded positive outcomes in the past. In contrast, limited evidence implicates the prefrontal cortex in the process of making strategic exploratory decisions when the magnitude of potential outcomes is unknown. Here we examine neurogenetic contributions to individual differences in these distinct aspects of motivated human behavior, using a temporal decision-making task and computational analysis. We show that two genes controlling striatal dopamine function, DARPP-32 (also called PPP1R1B) and DRD2, are associated with exploitative learning to adjust response times incrementally as a function of positive and negative decision outcomes. In contrast, a gene primarily controlling prefrontal dopamine function (COMT) is associated with a particular type of 'directed exploration', in which exploratory decisions are made in proportion to Bayesian uncertainty about whether other choices might produce outcomes that are better than the status quo. Quantitative model fits reveal that genetic factors modulate independent parameters of a reinforcement learning system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Depue, R. A., & Collins, P. F. (1999) · Behavioral and Brain Sciences · doi:10.1017/s0140525x99002046 · Cited by 2,046
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
Self-citationHabits, Rituals, and the Evaluative Brain
Graybiel, A. M. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112851 · Cited by 1,697 · cited 1×
“The findings also converge with rodent data showing that the transition to exploitative behavior is associated with the development of highly stabilized striatal firing pattern”
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Neural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411
Sutton, R. S. (1990) · Machine Learning Proceedings 1990 · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,314
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190
Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement
Dalley, J. W., Fryer, T. D., Brichard, L., Robinson, E. S. J., Theobald, D. E. H., Lääne, K., Peña, Y., Murphy, E. R., Shah, Y., Probst, K., et al. (2007) · Science · doi:10.1126/science.1137073 · Cited by 1,188
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“The present human genetic data provide support for the mechanisms posited in models of striatal dopamine, in which accumulated reward prediction errors over multiple trials produce speeded responses, whereas negative prediction errors slow response”
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,117
An efficient procedure for genotyping single nucleotide polymorphisms
Ye, S. (2001) · Nucleic Acids Research · doi:10.1093/nar/29.17.e88 · Cited by 1,005
High Impulsivity Predicts the Switch to Compulsive Cocaine-Taking
Belin, D., Mar, A. C., Dalley, J. W., Robbins, T. W., & Everitt, B. J. (2008) · Science · doi:10.1126/science.1158136 · Cited by 989
Frank, M. J. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929052880093 · Cited by 925
Self-citationGogos, J. A., Morgan, M., Luine, V., Santha, M., Ogawa, S., Pfaff, D., & Karayiorgou, M. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.17.9991 · Cited by 864
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706111104 · Cited by 683
Self-citationOuimet, C., Miller, P., Hemmings, H., Walaas, S., & Greengard, P. (1984) · The Journal of Neuroscience · doi:10.1523/jneurosci.04-01-00111.1984 · Cited by 656
Frank, M. J., & Claus, E. D. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.300 · Cited by 585
Self-citationSeparate neural pathways process different decision costs
Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1756 · Cited by 580
Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex
Roesch, M. R., & Olson, C. R. (2004) · Science · doi:10.1126/science.1093223 · Cited by 542
Dopamine: generalization and bonuses
Kakade, S., & Dayan, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00048-5 · Cited by 480
Midbrain dopamine and prefrontal function in humans: interaction and modulation by COMT genotype
Meyer-Lindenberg, A., Kohn, P. D., Kolachana, B., Kippenhan, S., McInerney-Leo, A., Nussbaum, R., Weinberger, D. R., & Berman, K. F. (2005) · Nature Neuroscience · doi:10.1038/nn1438 · Cited by 428
Genetically Determined Differences in Learning from Errors
Klein, T. A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D. Y., & Ullsperger, M. (2007) · Science · doi:10.1126/science.1145044 · Cited by 426
Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases
Frank, M. J., Woroch, B. S., & Curran, T. (2005) · Neuron · doi:10.1016/j.neuron.2005.06.020 · Cited by 426
Self-citationZhang, Y., Bertolino, A., Fazio, L., Blasi, G., Rampino, A., Romano, R., Lee, M.-L. T., Xiao, T., Papp, A., Wang, D., et al. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0707106104 · Cited by 423
Forbes, E. E., Brown, S. M., Kimak, M., Ferrell, R. E., Manuck, S. B., & Hariri, A. R. (2009) · Molecular Psychiatry · doi:10.1038/sj.mp.4002086 · Cited by 408
Calabresi, P., Gubellini, P., Centonze, D., Picconi, B., Bernardi, G., Chergui, K., Svenningsson, P., Fienberg, A. A., & Greengard, P. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-22-08443.2000 · Cited by 380
Probabilistic Interpretation of Population Codes
Zemel, R. S., Dayan, P., & Pouget, A. (1998) · Neural Computation · doi:10.1162/089976698300017818 · Cited by 333
Probabilistic Interpretation of Population Codes
Zemel, R. S., Dayan, P., & Pouget, A. (1999) · Neural Codes and Distributed Representations · doi:10.7551/mitpress/4924.003.0011 · Cited by 305
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Satoh, T., Nakai, S., Sato, T., & Kimura, M. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-30-09913.2003 · Cited by 298
Hirvonen, M., Laakso, A., Någren, K., Rinne, J. O., Pohjalainen, T., & Hietala, J. (2004) · Molecular Psychiatry · doi:10.1038/sj.mp.4001561 · Cited by 248
A phosphatase cascade by which rewarding stimuli control nucleosomal response
Stipanovich, A., Valjent, E., Matamales, M., Nishi, A., Ahn, J.-H., Maroteaux, M., Bertran-Gonzalez, J., Brami-Cherrier, K., Enslen, H., Corbillé, A.-G., et al. (2008) · Nature · doi:10.1038/nature06994 · Cited by 240
Genetic evidence implicating DARPP-32 in human frontostriatal structure, function, and cognition
Meyer-Lindenberg, A., Straub, R. E., Lipska, B. K., Verchinski, B. A., Goldberg, T., Callicott, J. H., Egan, M. F., Huffaker, S. S., Mattay, V. S., Kolachana, B., et al. (2007) · Journal of Clinical Investigation · doi:10.1172/jci30413 · Cited by 226
Dalley, J. W., Lääne, K., Theobald, D. E. H., Armstrong, H. C., Corlett, P. R., Chudasama, Y., & Robbins, T. W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0502080102 · Cited by 214
Resolution of Uncertainty in Prefrontal Cortex
Yoshida, W., & Ishii, S. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.006 · Cited by 197
COMT genotype predicts cortical-limbic D1 receptor availability measured with [11C]NNC112 and PET
Slifstein, M., Kolachana, B., Simpson, E. H., Tabares, P., Cheng, B., Duvall, M., Gordon Frankle, W., Weinberger, D. R., Laruelle, M., & Abi-Dargham, A. (2008) · Molecular Psychiatry · doi:10.1038/mp.2008.19 · Cited by 195
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 187
Role of Dopamine in the Primate Caudate Nucleus in Reward Modulation of Saccades
Nakamura, K., & Hikosaka, O. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4853-05.2006 · Cited by 178
A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism
Moustafa, A. A., Cohen, M. X., Sherman, S. J., & Frank, M. J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.3116-08.2008 · Cited by 169
Self-citationPVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.
O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007) · Behavioral Neuroscience · doi:10.1037/0735-7044.121.1.31 · Cited by 160
Self-citationSantesso, D. L., Evins, A. E., Frank, M. J., Schetter, E. C., Bogdan, R., & Pizzagalli, D. A. (2008) · Human Brain Mapping · doi:10.1002/hbm.20642 · Cited by 156
Self-citationProceedings of the seventh international conference (1990) on Machine learning
Morgan, M. B. (1990) · Cited by 142
Exploration Bonuses and Dual Control
Dayan, P., & B, T. J. S. A. (1996) · Machine Learning · doi:10.1023/a:1018357105171 · Cited by 120
Scheres, A., & Sanfey, A. G. (2006) · Behavioral and Brain Functions · doi:10.1186/1744-9081-2-35 · Cited by 109
Gani, J., Sarkadi, K., & Vincze, I. (1974) · Medical Entomology and Zoology · Cited by 86
Hirvonen, M., Laakso, A., Någren, K., Rinne, J. O., Pohjalainen, T., & Hietala, J. A. (2005) · Molecular Psychiatry · doi:10.1038/sj.mp.4001707 · Cited by 51
Wiecki, T. V., Riedinger, K., von Ameln-Mayerhofer, A., Schmidt, W., & Frank, M. J. (2009) · Psychopharmacology · doi:10.1007/s00213-008-1457-4 · Cited by 48
Self-citation
Cited by
A high-coverage genome sequence from an archaic denisovan individual
Meyer, M., Kircher, M., Gansauge, M.-T., Li, H., Racimo, F., Mallick, S., Schraiber, J. G., Jay, F., Prüfer, K., de Filippo, C., et al. (2012) · Science · doi:10.1126/science.1224344 · Cited by 1,967
Inverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control
Cools, R., & D'Esposito, M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.03.028 · Cited by 1,673
What determines our navigational abilities?
Wolbers, T., & Hegarty, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.001 · Cited by 810
Dissociable dopamine dynamics for learning and motivation
Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019) · Nature · doi:10.1038/s41586-019-1235-y · Cited by 735
Temporal structure of motor variability is dynamically regulated and predicts motor learning ability
Wu, H. G., Miyamoto, Y. R., Castro, L. N. G., Ölveczky, B. P., & Smith, M. A. (2014) · Nature Neuroscience · doi:10.1038/nn.3616 · Cited by 675
Metadata
- Authors
- Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F.
- Venue
- Nature Neuroscience
- Published
- 2009-07-20
- Volume
- 12
- Issue
- 8
- Pages
- 1062-1068
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W1999611081
- Cited by
- 471
- References
- 54
- FWCI
- 11.92
- Type
- journal article
- Language
- en
- Update notices
- erratum (2010-05-01)
- Open access
- green
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.