Pavlovian valuation systems in learning and decision making
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
Environmental stimuli guide value-based decision making, but can do so through cognitive representation of outcomes or through general-incentive properties attributed to the cues themselves. We assert that these differences are conferred through the use of alternative associative structures differing in computational intensity. Using this framework, we review scientific evidence to discern the neural substrates of these assumed separable processes. We suggest that the contribution of the mesolimbic dopamine system to Pavlovian valuation is restricted to an affective system that is only updated through experiential feedback of stimulus-outcome pairing, whereas the orbitofrontal cortex contributes to an alternative system capable of inferential reasoning. Finally we discuss the interactions and convergence of these systems and their implications for decision making and its pathology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maps of Bounded Rationality: Psychology for Behavioral Economics
Kahneman, D. (2003) · American Economic Review · doi:10.1257/000282803322655392 · Cited by 5,377
Di Chiara, G., & Imperato, A. (1988) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.85.14.5274 · Cited by 4,733
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 1×
“Dopamine and model-free Pavlovian valuation A role for dopamine in reward learning is suggested by data obtained from multiple paradigms [19,20,21 ].”
MethodDaw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
From motivation to action: Functional interface between the limbic system and the motor system
MOGENSON, G., JONES, D., & YIM, C. (1980) · Progress in Neurobiology · doi:10.1016/0301-0082(80)90018-0 · Cited by 2,331 · cited 1×
“These circuits access common effector systems to influence motor behavior, with the mesolimbic pathway and OFC efferents converging in the ventral striatum (Figure 2), a structure previously identified as a ‘limbic-motor interface’ [72].”
BackgroundA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · cited 2×
“This framework has been applied to the study of habitual and goal-directed instrumental behavior to learn optimal actions [2,3,11].”
BackgroundMethodA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 1×
“The similarity between these dopaminergic responses and the reward-prediction errors used by model-free algorithms was articulated by Montague and colleagues [24].”
MethodThe reinstatement model of drug relapse: history, methodology and major findings
Shaham, Y., Shalev, U., Lu, L., de Wit, H., & Stewart, J. (2003) · Psychopharmacology · doi:10.1007/s00213-002-1224-x · Cited by 1,615
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 1×
“Consistent with the hypothesis that the mesolimbic dopamine pathway transmits reward-prediction-error signals, qualitatively similar signals have been extracted from blood-oxygen-level-dependent (BOLD) hemodynamic response in the human ventral striatum, following the presentation of rewards or reward-predictive cues […”
Result
Cited by
A framework for studying emotions across species
Anderson, D. J., & Adolphs, R. (2014) · Cell · doi:10.1016/j.cell.2014.03.003 · Cited by 691
“There is now a large literature from reinforcement learning, applied widely across species, that provides important intersection with the study of emotion (Clark et al, 2012; see Box 3).”
Toward a Wiring Diagram Understanding of Appetite Control
Andermann, M. L., & Lowell, B. B. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.014 · Cited by 606
Dissecting the diversity of midbrain dopamine neurons
Roeper, J. (2013) · Trends in Neurosciences · doi:10.1016/j.tins.2013.03.003 · Cited by 373
Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass
Menegas, W., Bergan, J. F., Ogawa, S. K., Isogai, Y., Umadevi Venkataraju, K., Osten, P., Uchida, N., & Watabe-Uchida, M. (2015) · eLife · doi:10.7554/elife.10032 · Cited by 322
A Unified Framework for Dopamine Signals across Timescales
Kim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui-Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe-Uchida, M., Gershman, S. J., et al. (2020) · Cell · doi:10.1016/j.cell.2020.11.013 · Cited by 302
Menegas, W., Babayan, B. M., Uchida, N., & Watabe-Uchida, M. (2017) · eLife · doi:10.7554/elife.21886 · Cited by 265
Dopamine neurons respond to errors in the prediction of sensory features of expected rewards
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M., & Schoenbaum, G. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.025 · Cited by 200
Sign Tracking, but Not Goal Tracking, is Resistant to Outcome Devaluation
Morrison, S. E., Bamkole, M. A., & Nicola, S. M. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00468 · Cited by 137
“…CS (Figure 5C; Z = 2.10, corrected p < 0.1); a significantly greater bias toward lever deflection over receptacle entry during the CS (Figure 5D; Z = 2.63, p < 0.05); and a trend toward lower latency to interact with the lever, relative to the receptacle (Figure 5E; Z = −1.67, uncorrected p < 0.1).”
BackgroundMethodInfluentialSebold, M., Schad, D. J., Nebe, S., Garbusow, M., Jünger, E., Kroemer, N. B., Kathmann, N., Zimmermann, U. S., Smolka, M. N., Rapp, M. A., et al. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00945 · Cited by 41
Metadata
- Authors
- Clark, J. J., Hollon, N. G., & Phillips, P. E.
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2012-12
- Volume
- 22
- Issue
- 6
- Pages
- 1054-1061
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W2080856558
- Cited by
- 121
- References
- 47
- FWCI
- 0.60
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.