Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making
Open accessThe DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
Dual system theories suggest that behavioral control is parsed between a deliberative "model-based" and a more reflexive "model-free" system. A balance of control exerted by these systems is thought to be related to dopamine neurotransmission. However, in the absence of direct measures of human dopamine, it remains unknown whether this reflects a quantitative relation with dopamine either in the striatum or other brain areas. Using a sequential decision task performed during functional magnetic resonance imaging, combined with striatal measures of dopamine using [(18)F]DOPA positron emission tomography, we show that higher presynaptic ventral striatal dopamine levels were associated with a behavioral bias toward more model-based control. Higher presynaptic dopamine in ventral striatum was associated with greater coding of model-based signatures in lateral prefrontal cortex and diminished coding of model-free prediction errors in ventral striatum. Thus, interindividual variability in ventral striatal presynaptic dopamine reflects a balance in the behavioral expression and the neural signatures of model-free and model-based control. Our data provide a novel perspective on how alterations in presynaptic dopamine levels might be accompanied by a disruption of behavioral control as observed in aging or neuropsychiatric diseases such as schizophrenia and addiction.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“is associated with higher levels of presynaptic ventral striatal dopamine (Deserno et al., 2015).”
From The role of habit in compulsivity (2016) · cites this paper as Deserno et al., 2015
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“for example via a temporal difference reward prediction error (14, 15).”
BackgroundGraphical Evaluation of Blood-to-Brain Transfer Constants from Multiple-Time Uptake Data
Patlak, C. S., Blasberg, R. G., & Fenstermacher, J. D. (1983) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1983.1 · Cited by 3,567
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 3×
“The hypothesis of uncertainty-based competition (5) might also account for this finding under a premise that higher presynaptic dopamine levels result in larger phasic prediction error dopamine transients.”
BackgroundInfluentialA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“for example via a temporal difference reward prediction error (14, 15).”
BackgroundBy Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
The Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, E., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,838
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Self-citationBalleine, B. W., & O'Doherty, J. P. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.131 · Cited by 1,775
Patlak, C. S., & Blasberg, R. G. (1985) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1985.87 · Cited by 1,673
The principal features and mechanisms of dopamine modulation in the prefrontal cortex
Seamans, J. K., & Yang, C. R. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.05.006 · Cited by 1,592 · cited 1×
“In a similar vein, dopamine is implicated in a modulation of PFC maintenance processes via a gating of cortical gain, rendering coding of relevant environmental information more robust against noise (11, 26, 27).”
BackgroundGoal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · cited 1×
“Thus, in the face of such change, a goaldirected system can adapt quickly, whereas a habitual system needs to experience an actual outcome before it can alter behavior in an adaptive manner (4).”
BackgroundBayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03095.x · Cited by 1,226 · cited 1×
“Regarding the neural instantiation of both control systems, animal research has highlighted a dissociation between dorsolateral and dorsomedial striatum, with dorsolateral lesions disrupting habit formation, whereas dorsomedial lesions impact on goaldirected control (41, 42).”
ResultActions and habits: The development of behavioural autonomy
Dickinson, A. (1985) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1985.0010 · Cited by 1,217
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Tonic dopamine: Opportunity costs and the control of response vigor
Niv, Y., Daw, N. D., Joel, D., & Dayan, P. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0502-4 · Cited by 1,133 · cited 1×
“Furthermore, ventral striatal tonic dopamine is implicated in signaling average reward rates (29), a theoretical proposal that has received recent empirical support (e.”
BackgroundThe role of the dorsomedial striatum in instrumental conditioning
Yin, H. H., Ostlund, S. B., Knowlton, B. J., & Balleine, B. W. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04218.x · Cited by 1,099 · cited 1×
“Regarding the neural instantiation of both control systems, animal research has highlighted a dissociation between dorsolateral and dorsomedial striatum, with dorsolateral lesions disrupting habit formation, whereas dorsomedial lesions impact on goaldirected control (41, 42).”
ResultDolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Self-citationHowes, O. D., Kambeitz, J., Kim, E., Stahl, D., Slifstein, M., Abi-Dargham, A., & Kapur, S. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2012.169 · Cited by 967
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 869
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651
Neural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642
Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior
Goto, Y., & Grace, A. A. (2005) · Nature Neuroscience · doi:10.1038/nn1471 · Cited by 576 · cited 4×
“Such changes have been demonstrated in animal research (36), and it is conceivable that a long-term dominance of such activity might be reflected in higher presynaptic dopamine levels, as assessed here via F-DOPA PET.”
BackgroundInfluentialSelf-citationSerotonin and dopamine: unifying affective, activational, and decision functions.
Cools, R., Nakamura, K., & Daw, N. D. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.121 · Cited by 464
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380 · cited 5×
“These results are consistent with an engagement of a slower, computationally more costly model-based system (1, 3).”
BackgroundResultInfluentialCognitive and neurobiological mechanisms of alcohol-related aggression
Heinz, A. J., Beck, A., Meyer-Lindenberg, A., Sterzer, P., & Heinz, A. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3042 · Cited by 368 · cited 1×
“For example, serotonin interferes with aversive processing (46) and learning from negative prediction errors (47), whereas cholinergic influences are linked to an encoding of precision-weighted prediction errors (48).”
BackgroundSelf-citationOtto (2013) · doi:10.1177/0956797612463080 · Cited by 352
Linking nucleus accumbens dopamine and blood oxygenation
Knutson, B., & Gibbs, S. E. B. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0686-7 · Cited by 337 · cited 1×
“This may be due to several factors including the choice of experimental task, the type of neural measurement, and also limited homologies between neuroanatomical structures in rodents and primates (43, 44).”
BackgroundLimbic and cortical information processing in the nucleus accumbens
Goto, Y., & Grace, A. A. (2008) · Trends in Neurosciences · doi:10.1016/j.tins.2008.08.002 · Cited by 332 · cited 1×
“However, the association between a balance of behavioral control and ventral striatal presynaptic dopamine levels, as demonstrated in the present study, supports the idea that ventral striatum is an important nexus where several inputs converge (50).”
ResultSelf-citationFree Energy, Precision and Learning: The Role of Cholinergic Neuromodulation
Moran, R. J., Campo, P., Symmonds, M., Stephan, K. E., Dolan, R. J., & Friston, K. J. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4255-12.2013 · Cited by 312
Self-citationMapping value based planning and extensively trained choice in the human brain
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3068 · Cited by 310 · cited 1×
“evidence indicates these structures may encode model-based and model-free value signals (45), quantities that were not assessed here.”
BackgroundSelf-citationWorking Memory Capacity Predicts Dopamine Synthesis Capacity in the Human Striatum
Cools, R., Gibbs, S. E., Miyakawa, A., Jagust, W., & D'Esposito, M. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4475-07.2008 · Cited by 302 · cited 2×
“On the other hand, individual variation of striatal presynaptic dopamine, quantified using neurochemical imaging, is known to positively relate to variability in “prefrontal” cognitive capacities (17, 18), which might also limit the capacity for model-based learning (19).”
BackgroundResultDopamine enhances model-based over model-free choice behavior
Wunderlich, K., Smittenaar, P., & Dolan, R. J. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.042 · Cited by 294
Self-citationRakshi, J. S., Uema, T., Ito, K., Bailey, D. L., Morrish, P. K., Ashburner, J., Dagher, A., Jenkins, I. H., Friston, K. J., & Brooks, D. J. (1999) · Brain · doi:10.1093/brain/122.9.1637 · Cited by 270
Hierarchical cognitive control deficits following damage to the human frontal lobe
Badre, D., Hoffman, J., Cooney, J. W., & D'Esposito, M. (2009) · Nature Neuroscience · doi:10.1038/nn.2277 · Cited by 266
Dissociable Effects of Dopamine and Serotonin on Reversal Learning
den Ouden, H. E. M., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., Franke, B., & Cools, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.030 · Cited by 255
Smittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.009 · Cited by 244 · cited 1×
“Indeed, the importance of lateral PFC for model-based inference is supported by findings that theta-burst transcranial magnetic stimulation compromises model-based control in humans (28).”
Self-citationDepressive symptoms and presynaptic dopamine function in neuroleptic-naive schizophrenia
Hietala, J., Syvälahti, E., Vilkman, H., Vuorio, K., Räkköläinen, V., Bergman, J., Haaparanta, M., Solin, O., Kuoppamäki, M., Eronen, E., et al. (1999) · Schizophrenia Research · doi:10.1016/s0920-9964(98)00113-3 · Cited by 243
Dopaminergic control of the striatum for high-level cognition
Cools, R. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.04.002 · Cited by 221
Dopamine Modulates Reward-Related Vigor
Beierholm, U., Guitart-Masip, M., Economides, M., Chowdhury, R., Düzel, E., Dolan, R., & Dayan, P. (2013) · Neuropsychopharmacology · doi:10.1038/npp.2013.48 · Cited by 219
Self-citationMcDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., & Schoenbaum, G. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5499-10.2011 · Cited by 214
Twenty-Five Lessons from Computational Neuromodulation
Dayan, P. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.027 · Cited by 190
Chapter 19 Dopamine, cognitive control, and schizophrenia: the gating model
Braver, T. S., & Cohen, J. D. (1999) · Progress in Brain Research · doi:10.1016/s0079-6123(08)63082-4 · Cited by 159
An in vivo assay of synaptic function mediating human cognition
Moran, R. J., Symmonds, M., Stephan, K. E., Friston, K. J., & Dolan, R. J. (2011) · Current Biology · doi:10.1016/j.cub.2011.06.053 · Cited by 137 · cited 1×
“In a similar vein, dopamine is implicated in a modulation of PFC maintenance processes via a gating of cortical gain, rendering coding of relevant environmental information more robust against noise (11, 26, 27).”
Self-citationPET Studies of Cerebral Levodopa Metabolism: A Review of Clinical Findings and Modeling Approaches
Kumakura, Y., & Cumming, P. (2009) · The Neuroscientist · doi:10.1177/1073858409338217 · Cited by 134
Egerton, A., Demjaha, A., McGuire, P., Mehta, M. A., & Howes, O. D. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.058 · Cited by 134
Habitual versus goal-directed action control in Parkinson disease
de Wit, S., Barker, R. A., Dickinson, A. D., & Cools, R. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21514 · Cited by 116
Reliance on habits at the expense of goal-directed control following dopamine precursor depletion
de Wit, S., Standing, H. R., DeVito, E. E., Robinson, O. J., Ridderinkhof, K. R., Robbins, T. W., & Sahakian, B. J. (2012) · Psychopharmacology · doi:10.1007/s00213-011-2563-2 · Cited by 105
Dopamine Modulation of Hippocampal-Prefrontal Cortical Interaction Drives Memory-Guided Behavior
Goto, Y., & Grace, A. A. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm172 · Cited by 104
Self-citationSchlagenhauf, F., Rapp, M. A., Huys, Q. J. M., Beck, A., Wüstenberg, T., Deserno, L., Buchholz, H., Kalbitzer, J., Buchert, R., Bauer, M., et al. (2013) · Human Brain Mapping · doi:10.1002/hbm.22000 · Cited by 102
Self-citationIncentive motivation is associated with striatal dopamine asymmetry
Tomer, R., Goldstein, R. Z., Wang, G.-J., Wong, C., & Volkow, N. D. (2008) · Biological Psychology · doi:10.1016/j.biopsycho.2007.08.001 · Cited by 93
Martin-Soelch, C., Szczepanik, J., Nugent, A., Barhaghi, K., Rallis, D., Herscovitch, P., Carson, R. E., & Drevets, W. C. (2011) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07642.x · Cited by 89
Asymmetry in dopamine D2/3 receptors of caudate nucleus is lost with age
Vernaleken, I., Weibrich, C., Siessmeier, T., Buchholz, H., Rösch, F., Heinz, A., Cumming, P. D., Stoeter, P., Bartenstein, P., & Gründer, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.10.013 · Cited by 75
Self-citationVernaleken, I., Buchholz, H., Kumakura, Y., Siessmeier, T., Stoeter, P., Bartenstein, P., Cumming, P., & Gründer, G. (2007) · Human Brain Mapping · doi:10.1002/hbm.20325 · Cited by 62
Multiplexing signals in reinforcement learning with internal models and dopamine
Nakahara, H. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.001 · Cited by 34
Cited by
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
Self-citationPrioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393
Ventral Striatal Activation During Reward Processing in Psychosis: A Neurofunctional Meta-Analysis
Radua, J., Schmidt, A., Borgwardt, S., Heinz, A., Schlagenhauf, F., McGuire, P., & Fusar-Poli, P. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.2196 · Cited by 342
Self-citationPredictive representations can link model-based reinforcement learning to model-free mechanisms
Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J., & Daw, N. D. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005768 · Cited by 326
Dopamine Does Double Duty in Motivating Cognitive Effort
Westbrook, A., & Braver, T. S. (2016) · Neuron · doi:10.1016/j.neuron.2015.12.029 · Cited by 308
Metadata
- Authors
- Deserno, L., Huys, Q. J. M., Boehme, R., Buchert, R., Heinze, H.-J., Grace, A. A., Dolan, R. J., Heinz, A., & Schlagenhauf, F.
- Affiliations
- Max Planck Institute for Human Cognitive and Brain Sciences (DE)
- University of Zurich (CH)
- Charité – Universitätsmedizin Berlin (DE)
- University of Pittsburgh (US)
- Wellcome Centre for Human Neuroimaging (GB)
- NeuroCure Cluster of Excellence (DE)
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2015-01-20
- Volume
- 112
- Issue
- 5
- Pages
- 1595-1600
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC4321318
- OpenAlex
- W2016202155
- Cited by
- 233
- References
- 56
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 5 · 2023: 18 · 2022: 17 · 2021: 21 · 2020: 27 · 2019: 22 · 2018: 25 · 2017: 23 · 2016: 36 · 2015: 17
- FWCI
- 20.26
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Deutsche Forschungsgemeinschaft (SCHL1969/1-1, SCHL1969/2-1, HE2597/14-1)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- ventral striatum, presynaptic dopamine, model-based control, model-free control, dual-system theory, sequential decision making, prediction error coding, lateral prefrontal cortex, interindividual variability, dopamine imaging, schizophrenia, addiction, aging
- MeSH terms
- Adult, Behavior, Corpus Striatum, Decision Making, Dopamine, Female, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Positron-Emission Tomography, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.