The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Temporal Difference Models and Reward-Related Learning in the Human Brain
Open accessAbstract
Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, the signed prediction error indicates whether an outcome is better or worse than expected, and thereby plays a crucial role in changing the value allocated to cues, thereby guiding future decisions [14–21].”
From Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis (2021) · cites this paper as [17]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Many early fMRI studies into prediction errors used model-free accounts in Pavlovian paradigms and located prediction errors in striatal BOLD (Berns et al., 2001; O’Doherty, 2004; O’Doherty et al., 2003; Haruno et al., 2004).”
From Goals and Habits in the Brain (2013) · cites this paper as O’Doherty et al., 2003
In the citing paper
“These changes in striatum BOLD activity coincide with the process of learning how to predict rewards 15, 16 – an observation supported by electrophysiological properties of its dopaminergic afferents.”
From The social brain and reward: Social information processing in the human striatum (2014) · cites this paper as [15]
In the citing paper
“human fMRI studies have found that VS activity correlates with RPEs during tasks that feature exclusively Pavlovian reward associations [27,28]”
From Contributions of the striatum to learning, motivation, and performance: an associative account (2012) · cites this paper as [28]
In the citing paper
“Pupillary response dynamics are thought to reflect multiple different features, including sensory cues36,37, arousal/task engagement38,39, motor activity40,41, and motivation42,43.”
From Revealing abrupt transitions from goal-directed to habitual behavior (2026) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Self-citationRescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Spatial registration and normalization of images
Friston, K. J., Ashburner, J., Frith, C., Poline, J.-B., Heather, J. D., & Frackowiak, R. S. J. (1995) · Human Brain Mapping · doi:10.1002/hbm.460030303 · Cited by 3,744
Self-citationPearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Dissociation of reward anticipation and outcome with event-related fMRI
Knutson, B., Fong, G. W., Adams, C. M., Varner, J. L., & Hommer, D. (2001) · Neuroreport · doi:10.1097/00001756-200112040-00016 · Cited by 1,300
Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076
Self-citationVentral tegmental (A10) system: neurobiology. 1. Anatomy and connectivity
Oades, R. D., & Halliday, G. M. (1987) · Brain Research Reviews · doi:10.1016/0165-0173(87)90011-7 · Cited by 1,001
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning
Schoenbaum, G. M., Chiba, A. A., & Gallagher, M. (1998) · Nature Neuroscience · doi:10.1038/407 · Cited by 849
Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli
Mirenowicz, J., & Schultz, W. (1996) · Nature · doi:10.1038/379449a0 · Cited by 829
Duvernoy, H. M. (1999) · Springer eBooks · doi:10.1007/978-3-7091-6792-2 · Cited by 759
Dissociable Neural Responses in Human Reward Systems
Elliott, R., Friston, K. J., & Dolan, R. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-16-06159.2000 · Cited by 740
Self-citationPredictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711
Representation of Pleasant and Aversive Taste in the Human Brain
O’Doherty, J. P., Rolls, E. T., Francis, S., Bowtell, R. W., & McGlone, F. P. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.85.3.1315 · Cited by 688
Self-citationImportance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 544
Learning and Computational Neuroscience: Foundations of Adaptive Networks
Gabriel, M., & Moore, J. (1990) · Cited by 518
Time-Derivative Models of Pavlovian Reinforcement
Sutton, R. S., & Barto, A. G. (1990) · Cited by 501
Learning and selective attention
Dayan, P., Kakade, S., & Montague, P. R. (2000) · Nature Neuroscience · doi:10.1038/81504 · Cited by 495
Self-citationActivity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451
Francis, S., Rolls, E. T., Bowtell, R. W., McGlone, F. P., O’Doherty, J. P., Browning, A., Clare, S., & Smith, E. (1999) · Neuroreport · doi:10.1097/00001756-199902250-00003 · Cited by 421
Self-citationHikosaka, K. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.263 · Cited by 345
Reward prediction in primate basal ganglia and frontal cortex
Schultz, W., Tremblay, L., & Hollerman, J. R. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00071-9 · Cited by 318
Neuronal activity in monkey striatum related to the expectation of predictable environmental events
Apicella, P., Scarnati, E., Ljungberg, T., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.68.3.945 · Cited by 305
Neurochemical architecture of the human striatum
Holt, D. J., Graybiel, A. M., & Saper, C. B. (1997) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19970721)384:1 · Cited by 297
Value-dependent selection in the brain: Simulation in a synthetic neural model
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994) · Neuroscience · doi:10.1016/0306-4522(94)90592-4 · Cited by 268
Self-citationLearning about pain: The neural substrate of the prediction error for aversive events
Ploghaus, A., Tracey, I., Clare, S., Gati, J. S., Rawlins, J. N. P., & Matthews, P. M. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.160266497 · Cited by 250
Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention
Nobre, A. C., Coull, J. T., Frith, C., & Mesulam, M. M. (1999) · Nature Neuroscience · doi:10.1038/4513 · Cited by 249
Fletcher, P. C., Anderson, J. M., Shanks, D. R., Honey, R., Carpenter, T. A., Donovan, T., Papadakis, N., & Bullmore, E. T. (2001) · Nature Neuroscience · doi:10.1038/nn733 · Cited by 238
The Human Brain Stem and Cerebellum
Duvernoy, H. M. (1995) · Springer eBooks · doi:10.1007/978-3-7091-3078-0 · Cited by 110
Karachi, C., François, C., Parain, K., Bardinet, É., Tandé, D., Hirsch, É. C., & Yelnik, J. (2002) · The Journal of Comparative Neurology · doi:10.1002/cne.10312 · Cited by 94
The human brain stem and cerebellum
Paxton, R. M. (1995) · Neuroradiology · doi:10.1007/bf00593381 · Cited by 14
Cited by
Sorted by citations
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
Self-citationA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Self-citationKRINGELBACH, M. (2004) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2004.03.006 · Cited by 1,977
Metadata
- Authors
- O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J.
- Affiliations
- Gatsby Computational Neuroscience Unit (GB)
- UCL Queen Square Institute of Neurology (GB)
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2003-04
- Volume
- 38
- Issue
- 2
- Pages
- 329-337
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2007414406
- Cited by
- 1,522
- References
- 36
- Citations by year
- 2026: 51 · 2025: 50 · 2024: 51 · 2023: 46 · 2022: 53 · 2021: 55 · 2020: 54 · 2019: 64 · 2018: 87 · 2017: 65 · 2016: 97 · 2015: 71 · 2014: 70 · 2013: 93 · 2012: 97
- FWCI
- 17.96
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuron (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- Wellcome Trust
- Gatsby Charitable Foundation
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- temporal difference learning, prediction error signal, classical conditioning, appetitive conditioning, reward prediction, ventral striatum, orbitofrontal cortex, conditioned stimulus, human brain
- MeSH terms
- Adolescent, Adult, Brain, Brain Mapping, Conditioning, Classical, Corpus Striatum, Female, Frontal Lobe, Humans, Learning, Magnetic Resonance Imaging, Male, Reference Values, Reflex, Pupillary, Reward, Taste, Time Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
