Dopamine-associated cached values are not sufficient as the basis for action selection
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Significance: Real-world decisions frequently involve tradeoffs between multiple economic dimensions, and integration of benefits and costs into a common currency of subjective value is fundamental to action selection. Phasic dopamine is widely regarded as a critical teaching signal for learning the values assigned to actions, and these stored (“cached”) values can be read out from cue-evoked dopamine responses. Here, we observed a significant inversion between animals’ behavioral preferences and the rank ordering of dopamine-reported cached values, indicating that these cached values cannot be the sole determinant of choices in simple economic decision making. These data challenge the fundamental tenet of contemporary theories of decision making which posit that dopamine-associated cached values are sufficient to serve as the basis for action selection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · cited 1×
“Following 2000-Hz low-pass filtering, dopamine was isolated from the background (1 s before cue onset)subtracted voltammetric signal using chemometric analysis (3) using a standard training set based on stimulated dopamine release detected by chronically implanted electrodes (1).”
MethodDiscrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Dichotomous dopaminergic control of striatal synaptic plasticity
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008) · Science · doi:10.1126/science.1160575 · Cited by 1,190
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · cited 2×
“IEEE Trans Automat Contr 19(6):716–723.”
The Neurobiology of Decision: Consensus and Controversy
Kable, J. W., & Glimcher, P. W. (2009) · Neuron · doi:10.1016/j.neuron.2009.09.003 · Cited by 928 · cited 1×
“The discriminability of cue-evoked dopamine responses in the different forced trial types was analyzed at each time point using the auROC, an approach from signal-detection theory (4).”
MethodA cellular mechanism of reward-related learning
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001) · Nature · doi:10.1038/35092560 · Cited by 879
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651 · cited 1×
“Even though this system is based on a cached-value (“model-free”) architecture, there have been suggestions in the literature that it has access to “model-based” information derived from inferential online computation (47), further supporting the notion that the dopamine-associated cached values are a common currency…”
Neurobiology of economic choice: a good-based model
Padoa-Schioppa, C. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113648 · Cited by 600
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Separate neural pathways process different decision costs
Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1756 · Cited by 580
Self-citationCoincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574
Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards
Bromberg-Martin, E. S., & Hikosaka, O. (2009) · Neuron · doi:10.1016/j.neuron.2009.06.009 · Cited by 548
Separate Valuation Subsystems for Delay and Effort Decision Costs
Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2752-10.2010 · Cited by 499 · cited 1×
“Representations of costs indeed have been observed in areas such as the anterior cingulate cortex (50–54), insular cortex (55, 56), and basolateral amygdala (49, 57), all of which provide glutamatergic inputs that converge on striatal projection neurons.”
Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value
Tai, L.-H., Lee, A. M., Benavidez, N., Bonci, A., & Wilbrecht, L. (2012) · Nature Neuroscience · doi:10.1038/nn.3188 · Cited by 399 · cited 1×
“Perturbation of the cached values (9, 21, 23) to test the extent of their contribution to action selection could distinguish between these possibilities in future experiments.”
Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals
Clark, J. J., Sandberg, S. G., Wanat, M. J., Gan, J. O., Horne, E. A., Hart, A. S., Akers, C. A., Parker, J. G., Willuhn, I., Martinez, V., et al. (2010) · Nature Methods · doi:10.1038/nmeth.1412 · Cited by 378 · cited 2×
“Male Sprague– Dawley rats (Charles River Laboratories), 250–300 g upon arrival, were anesthetized with isoflurane for bilateral implantation of carbon-fiber microelectrodes (40) targeting the nucleus accumbens core (1.”
MethodSelf-citationLocalized microstimulation of primate pregenual cingulate cortex induces negative decision-making
Amemori, K.-I., & Graybiel, A. M. (2012) · Nature Neuroscience · doi:10.1038/nn.3088 · Cited by 262
Fundamental contribution by the basolateral amygdala to different forms of decision making
Ghods-Sharifi, S., St. Onge, J. R., & Floresco, S. B. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.0315-09.2009 · Cited by 257 · cited 3×
“Proc Natl Acad Sci USA 108(49):19767–19771.”
Dopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257 · cited 2×
“in which dopamine-associated cached values incorporate many economic attributes affecting animals’ behavioral preferences, such as objective expected value (24–27, 35, 36), and some subjective attributes, including risk preference (30–33), temporal discounting (24, 28, 29), flavor or reward-type preference (31), perce…”
Dopamine Reward Prediction Error Responses Reflect Marginal Utility
Stauffer, W. R., Lak, A., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.064 · Cited by 204
Two Dimensions of Value: Dopamine Neurons Represent Reward But Not Aversiveness
Fiorillo, C. D. (2013) · Science · doi:10.1126/science.1238699 · Cited by 203
Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli
Nomoto, K., Schultz, W., Watanabe, T., & Sakagami, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4828-09.2010 · Cited by 158 · cited 1×
“in which dopamine-associated cached values incorporate many economic attributes affecting animals’ behavioral preferences, such as objective expected value (24–27, 35, 36), and some subjective attributes, including risk preference (30–33), temporal discounting (24, 28, 29), flavor or reward-type preference (31), perce…”
Phasic Nucleus Accumbens Dopamine Encodes Risk-Based Decision-Making Behavior
Sugam, J. A., Day, J. J., Wightman, R. M., & Carelli, R. M. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2011.09.029 · Cited by 142
Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making
Stopper, C. M., Tse, M. T. L., Montes, D. R., Wiedman, C. R., & Floresco, S. B. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.033 · Cited by 136
Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions
de Lafuente, V., & Romo, R. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1117636108 · Cited by 117
Aversive Prediction Error Signals in the Amygdala
McHugh, S. B., Barkus, C., Huber, A., Capitao, L., Lima, J., Lowry, J. P., & Bannerman, D. M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.4465-13.2014 · Cited by 68 · cited 1×
“Representations of costs indeed have been observed in areas such as the anterior cingulate cortex (50–54), insular cortex (55, 56), and basolateral amygdala (49, 57), all of which provide glutamatergic inputs that converge on striatal projection neurons.”
Cited by
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
“5–15 nM (224, 621) versus 30–70 nM (116)], which raises additional, methodological issues.”
BackgroundInfluentialSalamone, J. D., Yohn, S. E., López-Cruz, L., San Miguel, N., & Correa, M. (2016) · Brain · doi:10.1093/brain/aww050 · Cited by 350
Dopamine Does Double Duty in Motivating Cognitive Effort
Westbrook, A., & Braver, T. S. (2016) · Neuron · doi:10.1016/j.neuron.2015.12.029 · Cited by 308
NOW vs LATER brain circuits: implications for obesity and addiction
Volkow, N. D., & Baler, R. D. (2015) · Trends in Neurosciences · doi:10.1016/j.tins.2015.04.002 · Cited by 237
Phasic dopamine signals: from subjective reward value to formal economic utility
Schultz, W., Carelli, R. M., & Wightman, R. M. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.09.006 · Cited by 88
Metadata
- Authors
- Hollon, N. G., Arnold, M. M., Gan, J. O., Walton, M. E., & Phillips, P. E. M.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2014-12-08
- Volume
- 111
- Issue
- 51
- Pages
- 18357-18362
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2050770543
- Cited by
- 52
- References
- 34
- FWCI
- 2.68
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.