The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning
Open accessThe DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto's (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 33×
“Thus, this theory can account for evidence suggesting that humans and animals attend toward good predictors of an outcome (Mackintosh, 1975) while also maintaining attention toward cues which are uncertain predictors of an outcome (Pearce and Hall, 1980).”
BackgroundMethodResultInfluentialA theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733 · cited 17×
“Thus, this theory can account for evidence suggesting that humans and animals attend toward good predictors of an outcome (Mackintosh, 1975) while also maintaining attention toward cues which are uncertain predictors of an outcome (Pearce and Hall, 1980).”
BackgroundMethodResultInfluentialGetting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · cited 3×
“This is because different expected values can be assigned to a particular state that are capable of containing information beyond the discrete cue that predicts reward (Bromberg-Martin et al., 2010c; Hong and Hikosaka, 2011; Aitken et al., 2016; Cone et al., 2016).”
BackgroundDiscrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · cited 2×
“Further, the magnitude of phasic activity correlates with the size of the unexpected reward (Hollerman and Schultz, 1998; Fiorillo et al., 2003; Roesch et al., 2007; Stauffer et al., 2014; Eshel et al., 2016) in a manner that reflects the value of the reward (Lak et al., 2014), value of the future…”
BackgroundContext and Behavioral Processes in Extinction: Table 1.
Bouton, M. E. (2004) · Learning & Memory · doi:10.1101/lm.78804 · Cited by 1,808 · cited 3×
“Thus, responding to the predictive cue will reemerge when the experimental circumstance no longer reflects that present in extinction (e.g., the passage of time or a physical change in context; Bouton, 2004).”
BackgroundInfluentialDopamine neuron systems in the brain: An update
Björklund, A., & Dunnett, S. B. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.006 · Cited by 1,777 · cited 2×
“Interestingly, VTA sends a particularly dense projection to the PL region (Bentivoglio and Morelli, 2005; Björklund and Dunnett, 2007).”
BackgroundSwanson, L. W. (1982) · Brain Research Bulletin · doi:10.1016/0361-9230(82)90145-9 · Cited by 1,718 · cited 2×
“A sub-nucleus of the amygdala complex, the BLA, is a region that receives extensive dopaminergic input from midbrain dopamine neurons (Swanson, 1982) and shows increases in neural activity when an unexpected event occurs whether it is rewarding or aversive (Belova et al., 2007, 2008; Herry et al.,…”
BackgroundToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 16×
“Until very recently evidence suggesting that phasic activity in midbrain dopamine neurons mimics the scalar prediction error described in Sutton and Barto (1981) has been largely, if not entirely, correlative (Schultz et al., 1997; Roesch et al., 2007; Niv and Schoenbaum, 2008; Iordanova, 2009;…”
BackgroundMethodResultInfluentialMedial Frontal Cortex Mediates Perceptual Attentional Set Shifting in the Rat
Birrell, J. M., & Brown, V. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-11-04320.2000 · Cited by 1,421 · cited 2×
“One of these odors predicted that the right well would be reinforced and the other predicted that the left well would be reinforced.”
Background
Cited by
Martins, D., Rademacher, L., Gabay, A. S., Taylor, R., Richey, J. A., Smith, D. V., Goerlich, K. S., Nawijn, L., Cremers, H. R., Wilson, R., et al. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.034 · Cited by 119
Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101
The Memory Paradox: Why Our Brains Need Knowledge in an Age of AI
Oakley, B., Johnston, M., Chen, K., Jung, E., & Sejnowski, T. (2025) · doi:10.2139/ssrn.5250447 · Cited by 31
The Functions of Dopamine in Operant Conditioned Reflexes
Maiorov, V. I. (2019) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-019-00815-y · Cited by 10
“The rate of acquisition of a Pavlovian conditioned refl ex has been proposed to depend directly on the magnitude of the prediction error of the unconditioned stimulus (changing from a maximum at the fi rst unexpected presentation of the unconditioned stimulus to a minimum on acquisition of the conditioned refl ex) [Re…”
Metadata
- Authors
- Nasser, H. M., Calu, D. J., Schoenbaum, G., & Sharpe, M. J.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2017-02-22
- Volume
- 8
- Pages
- 244-244
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2591012103
- Cited by
- 86
- References
- 58
- FWCI
- 4.23
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning. https://app.cochat.ai/papers/10.3389/fpsyg.2017.00244
- The paper
- Nasser, H. M., Calu, D. J., Schoenbaum, G., & Sharpe, M. J. (2017). The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2017.00244
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.