The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Dopamine responses comply with basic assumptions of formal learning theory
Abstract
According to contemporary learning theories, the discrepancy, or error, between the actual and predicted reward determines whether learning occurs when a stimulus is paired with a reward. The role of prediction errors is directly demonstrated by the observation that learning is blocked when the stimulus is paired with a fully predicted reward. By using this blocking procedure, we show that the responses of dopamine neurons to conditioned stimuli was governed differentially by the occurrence of reward prediction errors rather than stimulus-reward associations alone, as was the learning of behavioural reactions. Both behavioural and neuronal learning occurred predominantly when dopamine neurons registered a reward prediction error at the time of the reward. Our data indicate that the use of analytical tests derived from formal behavioural learning theory provides a powerful approach for studying the role of single neurons in learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The firing of DA cells that accompanies drug use encodes a number of reward properties, including reward expectancy [117], reward learning [118], and the consolidation of contextual memories [119].”
From Reward circuitry dysfunction in psychiatric and neurodevelopmental disorders and genetic syndromes: animal models and clinical findings (2012) · cites this paper as [119]
In the citing paper
“Similarly, increasing the probability that the animal will receive a reward in a given experiment constitutes a more rewarded context and increases the incidence of dopamine activations to unrewarded stimuli5,6,34.”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [5]
In the citing paper
“The physical resemblance of a stimulus to other stimuli known to be associated with a reward can enhance the initial dopamine activation through generalization5,6,17,19,60.”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [5]
In the citing paper
“these changes fail to occur with more-controlled arm, mouth and eye movements in a considerable range of studies2,5,9,13,151,152,154”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [5]
In the citing paper
“A by-now classic series of results suggests that the reward prediction error is encoded by midbrain dopamine cells (Glimcher, 2011; Schultz, 2006; Waelti et al., 2001).”
From Attention, learning, and the value of information (2012) · cites this paper as Waelti et al., 2001
In the citing paper
“Dopamine responsiveness appears to encode discrepancies between rewards received and those predicted, consistent with a “prediction error” teaching signal used in formal models of reinforcement learning5,6.”
From A selective role for dopamine in stimulus-reward learning (2011) · cites this paper as [6]
Citations
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationThe neural basis of drug craving: An incentive-sensitization theory of addiction
Robinson, T. (1993) · Brain Research Reviews · doi:10.1016/0165-0173(93)90013-p · Cited by 7,346
(1991) · Springer eBooks · doi:10.1007/978-3-7091-2840-4 · Cited by 5,092
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationClassical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Marr, D. (1969) · The Journal of Physiology · doi:10.1113/jphysiol.1969.sp008820 · Cited by 3,253
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Flagged first, then by citations
Research works printed but not linked: 3
- Pavlov, I. P. Conditional Reflexes (Oxford Univ. Press, London, 1927).
- Fibiger, H. C. & Phillips, A. G. in Handbook of Physiology—The Nervous System IV (ed. Bloom, F. E.) 647–675 (Williams and Wilkins, Baltimore, 1986).
- Kamin, L. J. in Fundamental Issues in Instrumental Learning (eds Mackintosh, N. J. & Honig, W. K.) 42–64 (Dalhousie Univ. Press, Dalhousie, 1969).
Cited by
Sorted by citations
Items 101–124 of 124
Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder
Rothkirch, M., Tonn, J., Köhler, S., & Sterzer, P. (2017) · Brain · doi:10.1093/brain/awx025 · Cited by 124
Ziegler, S., Pedersen, M. L., Mowinckel, A. M., & Biele, G. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.09.002 · Cited by 110
Reinforcement learning: Computational theory and biological mechanisms
Doya, K. (2007) · HFSP journal · doi:10.2976/1.2732246 · Cited by 150
Levodopa ameliorates learning and memory deficits in a murine model of Alzheimer’s disease
Ambrée, O., Richter, H., Sachser, N., Lewejohann, L., Dere, E., de Souza Silva, M. A., Herring, A., Keyvani, K., Paulus, W., & Schäbitz, W.-R. (2009) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2007.11.010 · Cited by 115
Reliance on habits at the expense of goal-directed control following dopamine precursor depletion
de Wit, S., Standing, H. R., DeVito, E. E., Robinson, O. J., Ridderinkhof, K. R., Robbins, T. W., & Sahakian, B. J. (2012) · Psychopharmacology · doi:10.1007/s00213-011-2563-2 · Cited by 105
Ketamine effects on memory reconsolidation favor a learning model of delusions
Corlett, P. R., Cambridge, V., Gardner, J. M., Piggot, J. S., Turner, D. C., Everitt, J. C., Arana, F. S., Morgan, H. L., Milton, A. L., Lee, J. L., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065088 · Cited by 105
Self-citationAsaad, W. F., & Eskandar, E. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3793-11.2011 · Cited by 105
Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum
Gregorios-Pippas, L., Tobler, P. N., & Schultz, W. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90730.2008 · Cited by 101
Self-citationThe conditions that promote fear learning: Prediction error and Pavlovian fear conditioning
Li, S. S. Y., & McNally, G. P. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2013.05.002 · Cited by 89
Axiomatic methods, dopamine and reward prediction error
Caplin, A., & Dean, M. (2008) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2008.07.007 · Cited by 88
Wood, J., & Ahmari, S. E. (2015) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2015.00171 · Cited by 85
“This signal establishes learned associations and guides behavioral selection (Houk et al., 1995; Montague et al., 1996; Schultz et al., 1997; Schultz, 1998; Waelti et al., 2001; Joel et al., 2002; Morris et al., 2006; Roesch et al., 2007; Glimcher, 2011).”
BackgroundDynamic shaping of dopamine signals during probabilistic Pavlovian conditioning
Hart, A. S., Clark, J. J., & Phillips, P. E. M. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.07.010 · Cited by 82
Dopamine neuron ensembles signal the content of sensory prediction errors
Stalnaker, T. A., Howard, J. D., Takahashi, Y. K., Gershman, S. J., Kahnt, T., & Schoenbaum, G. (2019) · eLife · doi:10.7554/elife.49315 · Cited by 74
Dissociable reward and timing signals in human midbrain and ventral striatum
Klein-Flügge, M. C., Hunt, L. T., Bach, D. R., Dolan, R. J., & Behrens, T. E. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.024 · Cited by 75
Reward Contexts Extend Dopamine Signals to Unrewarded Stimuli
Kobayashi, S., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2013.10.061 · Cited by 63
Self-citationNicotinic receptors in the ventral tegmental area promote uncertainty-seeking
Naudé, J., Tolu, S., Dongelmans, M., Torquet, N., Valverde, S., Rodriguez, G., Pons, S., Maskos, U., Mourot, A., Marti, F., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4223 · Cited by 61
Multiple functions of dopamine neurons
Schultz, W. (2010) · F1000 Biology Reports · doi:10.3410/b2-2 · Cited by 59
Self-citationCombining fMRI and behavioral measures to examine the process of human learning
Karuza, E. A., Emberson, L. L., & Aslin, R. N. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2013.09.012 · Cited by 50
Klanker, M., Sandberg, T., Joosten, R., Willuhn, I., Feenstra, M., & Denys, D. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2015.08.011 · Cited by 37
Sethi, A., Voon, V., Critchley, H. D., Cercignani, M., & Harrison, N. A. (2018) · Brain · doi:10.1093/brain/awy048 · Cited by 36
Modeling operant behavior in the Parkinsonian rat
Avila, I., Reilly, M. P., Sanabria, F., Posadas-Sánchez, D., Chavez, C. L., Banerjee, N., Killeen, P., & Castañeda, E. (2009) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.11.033 · Cited by 35
Véronneau-Veilleux, F., Robaey, P., Ursino, M., & Nekka, F. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.849323 · Cited by 32
Disrupted reinforcement learning during post-error slowing in ADHD
Chevrier, A., Bhaijiwala, M., Lipszyc, J., Cheyne, D., Graham, S., & Schachar, R. (2019) · PLOS ONE · doi:10.1371/journal.pone.0206780 · Cited by 21
The Functions of Dopamine in Operant Conditioned Reflexes
Maiorov, V. I. (2019) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-019-00815-y · Cited by 10
Metadata
- Authors
- Waelti, P., Dickinson, A., & Schultz, W.
- Affiliations
- University of Cambridge (GB)
- University of Fribourg (CH)
- Corresponding author
- Schultz, Wolfram
- Venue
- Nature
- Published
- 2001-07
- Volume
- 412
- Issue
- 6842
- Pages
- 43-48
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2136748963
- Cited by
- 1,018
- References
- 54
- Citations by year
- 2026: 13 · 2025: 15 · 2024: 25 · 2023: 30 · 2022: 30 · 2021: 25 · 2020: 40 · 2019: 42 · 2018: 37 · 2017: 48 · 2016: 50 · 2015: 45 · 2014: 41 · 2013: 56 · 2012: 43
- FWCI
- 13.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation
- James S. McDonnell Foundation
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- dopaminergic neurons, reward prediction error, learning theory, stimulus-reward associations, blocking, conditioned stimulus, single-unit recording, behavioural learning, reward prediction
- MeSH terms
- Animals, Conditioning, Classical, Dopamine, Learning, Macaca fascicularis, Male, Models, Neurological, Neurons, Photic Stimulation, Psychological Theory, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
