The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Dopamine responses comply with basic assumptions of formal learning theory
Abstract
According to contemporary learning theories, the discrepancy, or error, between the actual and predicted reward determines whether learning occurs when a stimulus is paired with a reward. The role of prediction errors is directly demonstrated by the observation that learning is blocked when the stimulus is paired with a fully predicted reward. By using this blocking procedure, we show that the responses of dopamine neurons to conditioned stimuli was governed differentially by the occurrence of reward prediction errors rather than stimulus-reward associations alone, as was the learning of behavioural reactions. Both behavioural and neuronal learning occurred predominantly when dopamine neurons registered a reward prediction error at the time of the reward. Our data indicate that the use of analytical tests derived from formal behavioural learning theory provides a powerful approach for studying the role of single neurons in learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The firing of DA cells that accompanies drug use encodes a number of reward properties, including reward expectancy [117], reward learning [118], and the consolidation of contextual memories [119].”
From Reward circuitry dysfunction in psychiatric and neurodevelopmental disorders and genetic syndromes: animal models and clinical findings (2012) · cites this paper as [119]
In the citing paper
“Similarly, increasing the probability that the animal will receive a reward in a given experiment constitutes a more rewarded context and increases the incidence of dopamine activations to unrewarded stimuli5,6,34.”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [5]
In the citing paper
“The physical resemblance of a stimulus to other stimuli known to be associated with a reward can enhance the initial dopamine activation through generalization5,6,17,19,60.”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [5]
In the citing paper
“these changes fail to occur with more-controlled arm, mouth and eye movements in a considerable range of studies2,5,9,13,151,152,154”
From Dopamine reward prediction-error signalling: A two-component response (2016) · cites this paper as [5]
In the citing paper
“A by-now classic series of results suggests that the reward prediction error is encoded by midbrain dopamine cells (Glimcher, 2011; Schultz, 2006; Waelti et al., 2001).”
From Attention, learning, and the value of information (2012) · cites this paper as Waelti et al., 2001
In the citing paper
“Dopamine responsiveness appears to encode discrepancies between rewards received and those predicted, consistent with a “prediction error” teaching signal used in formal models of reinforcement learning5,6.”
From A selective role for dopamine in stimulus-reward learning (2011) · cites this paper as [6]
Citations
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationThe neural basis of drug craving: An incentive-sensitization theory of addiction
Robinson, T. (1993) · Brain Research Reviews · doi:10.1016/0165-0173(93)90013-p · Cited by 7,346
(1991) · Springer eBooks · doi:10.1007/978-3-7091-2840-4 · Cited by 5,092
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Self-citationClassical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Marr, D. (1969) · The Journal of Physiology · doi:10.1113/jphysiol.1969.sp008820 · Cited by 3,253
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733
Barto, A. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,172
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Zhang, B., Li, C., & Lin, N. (2020) · De Gruyter eBooks · doi:10.1515/9783110595567-011 · Cited by 1,948
Steps toward Artificial Intelligence
Minsky, M. (1961) · Proceedings of the IRE · doi:10.1109/jrproc.1961.287775 · Cited by 1,796
Widrow, B. (1986) · Signal Processing · doi:10.1016/0165-1684(86)90025-3 · Cited by 1,597
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450
Neuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427
Self-citationConditioning and Associative Learning
Hill, W. F., & Mackintosh, N. J. (1984) · The American Journal of Psychology · doi:10.2307/1422540 · Cited by 1,304
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
Self-citationModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Neurobehavioural mechanisms of reward and motivation
Robbins, T. W., & Everitt, B. J. (1996) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(96)80077-8 · Cited by 1,187
Ito, M. (1989) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.12.030189.000505 · Cited by 1,167
Dopamine neurons report an error in the temporal prediction of reward during learning
Hollerman, J. R., & Schultz, W. (1998) · Nature Neuroscience · doi:10.1038/1124 · Cited by 1,161
Self-citationAnimal intelligence; experimental studies
Thorndike, E. L. (1911) · doi:10.5962/bhl.title.55072 · Cited by 1,100
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Self-citationMesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967
Contemporary Animal Learning Theory.
Abrahamson, D. (1981) · Europe PMC (PubMed Central) · Cited by 945
Contemporary Animal Learning Theory
LoLordo, V. M., & Dickinson, A. (1982) · The American Journal of Psychology · doi:10.2307/1422669 · Cited by 890
Self-citationAnimal Intelligence. Experimental Studies
Thorndike, E. L., & Jelliffe (1912) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-191205000-00016 · Cited by 824
Long-term synaptic depression in the striatum: physiological and pharmacological characterization
Calabresi, P., Maj, R., Pisani, A., Mercuri, N. B., & Bernardi, G. (1992) · Journal of Neuroscience · doi:10.1523/jneurosci.12-11-04224.1992 · Cited by 682
The involvement of nucleus accumbens dopamine in appetitive and aversive motivation
Salamone, J. D. (1994) · Behavioural Brain Research · doi:10.1016/0166-4328(94)90153-8 · Cited by 584
Learning and Computational Neuroscience: Foundations of Adaptive Networks
Gabriel, M., & Moore, J. (1990) · Cited by 518
Perspectives on Cognitive Neuroscience
Churchland, P. S., & B, T. J. S. A. (1988) · Science · doi:10.1126/science.3055294 · Cited by 505
Romo, R., & Schultz, W. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.3.592 · Cited by 353
Self-citationSchultz, W., & Romo, R. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.3.607 · Cited by 334
Self-citationThe cerebellum and VOR/OKR learning models
Kawato, M., & Gomi, H. (1992) · Trends in Neurosciences · doi:10.1016/0166-2236(92)90008-v · Cited by 323
Wickens, J. R., Begg, A., & Arbuthnott, G. W. (1996) · Neuroscience · doi:10.1016/0306-4522(95)00436-m · Cited by 298
Suri, R. E., & Schultz, W. (1999) · Neuroscience · doi:10.1016/s0306-4522(98)00697-6 · Cited by 294
Self-citation(1999) · Kybernetes · doi:10.1108/k.1999.28.5.70.1 · Cited by 291
Abnormal Synaptic Plasticity in the Striatum of Mice Lacking Dopamine D2 Receptors
Calabresi, P., Saiardi, A., Pisani, A., Baik, J., Centonze, D., Mercuri, N. B., Bernardi, G., & Borrelli, E. (1997) · Journal of Neuroscience · doi:10.1523/jneurosci.17-12-04536.1997 · Cited by 289
Localization of drug reward mechanisms by intracranial injections
Wise, R. A., & Hoffman, D. C. (1992) · Synapse · doi:10.1002/syn.890100307 · Cited by 279
Value-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268
Perspectives on cognitive neuroscience
(1992) · Choice Reviews Online · doi:10.5860/choice.30-0604 · Cited by 226
Unilateral Dopamine Denervation Blocks Corticostriatal LTP
Centonze, D., Gubellini, P., Picconi, B., Calabresi, P., Giacomini, P., & Bernardi, G. (1999) · Journal of Neurophysiology · doi:10.1152/jn.1999.82.6.3575 · Cited by 219
Inhibitory Cerebello-Olivary Projections and Blocking Effect in Classical Conditioning
Kim, J. J., Krupa, D. J., & Thompson, R. F. (1998) · Science · doi:10.1126/science.279.5350.570 · Cited by 212
Dopamine facilitates long-term depression of glutamatergic transmission in rat prefrontal cortex
Otani, S., Blond, O., Desce, J. M., & Crépel, F. (1998) · Neuroscience · doi:10.1016/s0306-4522(97)00677-5 · Cited by 196
The 28th Bartlett Memorial Lecture Causal learning: An associative analysis
Dickinson, A. (2001) · The Quarterly Journal of Experimental Psychology Section B · doi:10.1080/02724990042000010 · Cited by 187
Self-citationOtani, S., Auclair, N., Desce, J. M., Roisin, M. P., & Crépel, F. (1999) · Journal of Neuroscience · doi:10.1523/jneurosci.19-22-09788.1999 · Cited by 185
Schultz, W., & Romo, R. (1987) · Journal of Neurophysiology · doi:10.1152/jn.1987.57.1.201 · Cited by 184
Self-citationHolland, P. C. (1997) · Animal Learning & Behavior · doi:10.3758/bf03209846 · Cited by 81
Models of Information processing in the Basal Ganglia
Mitchell, I. (1996) · Journal of Chemical Neuroanatomy · doi:10.1016/0891-0618(96)00155-x · Cited by 63
Blocking observed in human eyelid conditioning.
Martin, I., & Levey, A. B. (1991) · PubMed · Cited by 40
Presynaptic long-term changes in excitability of the corticostriatal pathway
Garcia-Munoz, M., Young, S. J., & Groves, P. M. (1992) · Neuroreport · doi:10.1097/00001756-199204000-00017 · Cited by 31
Rauhut, A. S., McPhee, J. E., & Ayres, J. J. B. (1999) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.25.1.45 · Cited by 27
Flagged first, then by citations
Research works printed but not linked: 3
- Pavlov, I. P. Conditional Reflexes (Oxford Univ. Press, London, 1927).
- Fibiger, H. C. & Phillips, A. G. in Handbook of Physiology—The Nervous System IV (ed. Bloom, F. E.) 647–675 (Williams and Wilkins, Baltimore, 1986).
- Kamin, L. J. in Fundamental Issues in Instrumental Learning (eds Mackintosh, N. J. & Honig, W. K.) 42–64 (Dalhousie Univ. Press, Dalhousie, 1969).
Cited by
Sorted by citations
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
Kapur, S. (2003) · American Journal of Psychiatry · doi:10.1176/appi.ajp.160.1.13 · Cited by 2,813
Getting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Self-citationDopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396
Metadata
- Authors
- Waelti, P., Dickinson, A., & Schultz, W.
- Affiliations
- University of Cambridge (GB)
- University of Fribourg (CH)
- Corresponding author
- Schultz, Wolfram
- Venue
- Nature
- Published
- 2001-07
- Volume
- 412
- Issue
- 6842
- Pages
- 43-48
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2136748963
- Cited by
- 1,018
- References
- 54
- Citations by year
- 2026: 13 · 2025: 15 · 2024: 25 · 2023: 30 · 2022: 30 · 2021: 25 · 2020: 40 · 2019: 42 · 2018: 37 · 2017: 48 · 2016: 50 · 2015: 45 · 2014: 41 · 2013: 56 · 2012: 43
- FWCI
- 13.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation
- James S. McDonnell Foundation
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- dopaminergic neurons, reward prediction error, learning theory, stimulus-reward associations, blocking, conditioned stimulus, single-unit recording, behavioural learning, reward prediction
- MeSH terms
- Animals, Conditioning, Classical, Dopamine, Learning, Macaca fascicularis, Male, Models, Neurological, Neurons, Photic Stimulation, Psychological Theory, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review