Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Evaluation of both immediate and future outcomes of one's actions is a critical requirement for intelligent behavior. Using functional magnetic resonance imaging (fMRI), we investigated brain mechanisms for reward prediction at different time scales in a Markov decision task. When human subjects learned actions on the basis of immediate rewards, significant activity was seen in the lateral orbitofrontal cortex and the striatum. When subjects learned to act in order to obtain large future rewards while incurring small immediate losses, the dorsolateral prefrontal cortex, inferior parietal cortex, dorsal raphe nucleus and cerebellum were also activated. Computational model-based regression analysis using the predicted future rewards and prediction errors estimated from subjects' performance data revealed graded maps of time scale within the insula and the striatum: ventroanterior regions were involved in predicting immediate rewards and dorsoposterior regions were involved in predicting future rewards. These results suggest differential involvement of the cortico-basal ganglia loops in reward prediction at different time scales.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In neurotypical individuals, the key brain loci that have been implicated in prediction are basal ganglia (68, 69), striatum (70, 71), anterior cingulate (72), and cerebellum (73, 74).”
From Autism as a disorder of prediction (2014) · cites this paper as [69]
Citations
Cites
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“, which has been successfully used for explaining reward-predictive activations of the midbrain dopaminergic system as well as those of the cortex and the striatu”
MethodEmotion, Decision Making and the Orbitofrontal Cortex
Bechara, A. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.295 · Cited by 3,589 · cited 1×
“Damage to the prefrontal cortex often impairs daily decision making, which requires assessment of future outcome”
BackgroundBarto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244
Basal ganglia and cerebellar loops: motor and cognitive circuits
Middleton, F. (2000) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00040-5 · Cited by 1,940 · cited 1×
“The cortex and the basal ganglia both have parallel loop organization, with four major loops (limbic, cognitive, motor and oculomotor) and finer, topographic sub-loops within each major loo”
Background(2020) · Machine Learning and Visual Perception · doi:10.1515/9783110595567-011 · Cited by 1,930
The Orbitofrontal Cortex and Reward
Rolls, E. T. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.284 · Cited by 1,595
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Evidence for striatal dopamine release during a video game
Koepp, M. J., Gunn, R. N., Lawrence, A. D., Cunningham, V. J., Dagher, A., Jones, T., Brooks, D. J., Bench, C. J., & Grasby, P. M. (1998) · Nature · doi:10.1038/30498 · Cited by 1,279
Models of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Rogers, R. (1999) · Neuropsychopharmacology · doi:10.1016/s0893-133x(98)00091-8 · Cited by 1,101
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076
Insula of the old world monkey. III: Efferent cortical output and comments on function
Mesulam, M. ‐., & Mufson, E. J. (1982) · Journal of Comparative Neurology · doi:10.1002/cne.902120104 · Cited by 1,013 · cited 1×
“The insula takes a pivotal position in reward processing by receiving primary taste and visceral sensory inpu”
BackgroundComplementary roles of basal ganglia and cerebellum in learning and motor control
Doya, K. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00153-7 · Cited by 964
Self-citationImpulsive choice induced in rats by lesions of the nucleus accumbens core
Cardinal, R. N., Pennicott, D. R., Lakmali, C., Sugathapala, Robbins, T. W., & Everitt, B. J. (2001) · Science · doi:10.1126/science.1060818 · Cited by 923 · cited 1×
“Lesions in the core of the nucleus accumbens in rats result in a tendency to choose small immediate rewards over larger future reward”
BackgroundThe Anatomical Connections of the Macaque Monkey Orbitofrontal Cortex. A Review
Cavada, C. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.220 · Cited by 917 · cited 1×
“(a–c) These figures show activation maps focused on (a) the lateral OFC (red (x, y, z) = (38, 46, –14); blue (46, 47, 3)) (b) the insula (red (–36, 13, –4); blue (–30, 18, 1)), and (c) the striatum (red (18, 10, 0); blue (18, 12, 3)) where we observed significant activation in both contrasts.”
BackgroundTemporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 1×
“, which has been successfully used for explaining reward-predictive activations of the midbrain dopaminergic system as well as those of the cortex and the striatu”
MethodKnutson, B., Fong, G. W., Bennett, S. M., Adams, C. M., & Hommer, D. (2003) · NeuroImage · doi:10.1016/s1053-8119(02)00057-5 · Cited by 848
Parallel neural networks for learning sequential procedures
Hikosaka, O., Nakahara, H., Rand, M. K., Sakai, K., Lu, X., Nakamura, K., Miyachi, S., & Doya, K. (1999) · Trends in Neurosciences · doi:10.1016/s0166-2236(99)01439-3 · Cited by 799
Self-citationRogers, R. D., Owen, A. M., Middleton, H. C., Williams, E. J., Pickard, J. D., Sahakian, B. J., & Robbins, T. W. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-20-09029.1999 · Cited by 747
Dissociable Neural Responses in Human Reward Systems
Elliott, R., Friston, K. J., & Dolan, R. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-16-06159.2000 · Cited by 740
Predictability modulates human brain response to reward
Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-08-02793.2001 · Cited by 711
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704 · cited 1×
“A possible mechanism underlying these observations is that different sub-loops of the topographically organized cortico-basal ganglia network are specialized for reward prediction at different time scales and that they are differentially activated by the ascending serotonergic syste”
BackgroundSelf-citationNeural activity in the human brain relating to uncertainty and arousal during anticipation.
Critchley, H. D., Mathias, C. J., & Dolan, R. J. (2001) · Neuron · doi:10.1016/s0896-6273(01)00225-2 · Cited by 681
Neural systems engaged by planning: A PET study of the Tower of London task
Baker, S. C., Rogers, R. D., Owen, A. M., Frith, C. D., Dolan, R. J., Frackowiak, R. S. J., & Robbins, T. W. (1996) · Neuropsychologia · doi:10.1016/0028-3932(95)00133-6 · Cited by 635
Evenden, J. L., & Ryan, C. N. (1996) · Psychopharmacology · doi:10.1007/s002130050121 · Cited by 634 · cited 1×
“, and animals with lesions in the ascending serotonergic pathway tend to choose small immediate rewards over larger future reward”
BackgroundO'Doherty, J., Critchley, H., Deichmann, R., & Dolan, R. J. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-21-07931.2003 · Cited by 611
The orbital and medial prefrontal circuit through the primate basal ganglia
Haber, S., Kunishio, K., Mizobuchi, M., & Lynd-Balta, E. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-07-04851.1995 · Cited by 605 · cited 1×
“Our results suggest that the areas within the limbic loo”
ResultDopamine-dependent plasticity of corticostriatal synapses
Reynolds, J. N. J., & Wickens, J. R. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00045-x · Cited by 535
Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy
Shidara, M., & Richmond, B. J. (2002) · Science · doi:10.1126/science.1069504 · Cited by 518 · cited 1×
“Recent brain imaging and neural recording studies report involvement of a variety of cortical areas and the striatum in reward processin”
BackgroundElliott, R., Newman, J. L., Longe, O. A., & Deakin, J. F. W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-01-00303.2003 · Cited by 514
Celada, P., Puig, M. V., Casanovas, J. M., Guillazo, G., & Artigas, F. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-24-09917.2001 · Cited by 497
Ullsperger, M., & von Cramon, D. Y. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-10-04308.2003 · Cited by 490 · cited 1×
“Previous imaging studies also showed involvement of the insula, especially the ventroanterior region, in processing aversive outcome”
Activity in human ventral striatum locked to errors of reward prediction
Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002) · Nature Neuroscience · doi:10.1038/nn802 · Cited by 451
Planning and spatial working memory: A positron emission tomography study in humans
Owen, A. M., Doyon, J., Petrides, M., & Evans, A. C. (1996) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.1996.tb01219.x · Cited by 450
S., M., S., B., M.-Y., H., C., B., E., S., J., D., & I., A. (2002) · Psychopharmacology · doi:10.1007/s00213-001-0983-0 · Cited by 409 · cited 1×
“Damage to the prefrontal cortex often impairs daily decision making, which requires assessment of future outcome”
BackgroundNeuronal correlates of goal-based motor selection in the prefrontal cortex.
Matsumoto, K., Suzuki, W., & Tanaka, K. (2003) · Science · doi:10.1126/science.1084204 · Cited by 384 · cited 1×
“Recent brain imaging and neural recording studies report involvement of a variety of cortical areas and the striatum in reward processin”
BackgroundHikosaka, K. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.263 · Cited by 345 · cited 1×
“(a–c) These figures show activation maps focused on (a) the lateral OFC (red (x, y, z) = (38, 46, –14); blue (46, 47, 3)) (b) the insula (red (–36, 13, –4); blue (–30, 18, 1)), and (c) the striatum (red (18, 10, 0); blue (18, 12, 3)) where we observed significant activation in both contrasts.”
BackgroundChikama, M., McFarland, N. R., Amaral, D. G., & Haber, S. N. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-24-09686.1997 · Cited by 342
Martı́n-Ruiz, R., Puig, M. V., Celada, P., Shapiro, D. A., Roth, B. L., Mengod, G., & Artigas, F. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-24-09856.2001 · Cited by 339
Haruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., Imamizu, H., & Kawato, M. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.3417-03.2004 · Cited by 300
Self-citationReward-Related Neuronal Activity During Go-Nogo Task Performance in Primate Orbitofrontal Cortex
Tremblay, L., & Schultz, W. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.83.4.1864 · Cited by 285 · cited 1×
“This interpretation is consistent with previous studies demonstrating the OFC's role in prediction of rewards, immediately following sensorimotor event”
Result(1999) · Kybernetes · doi:10.1108/k.1999.28.5.70.1 · Cited by 282
The Role of Rostral Brodmann Area 6 in Mental-operation Tasks: an Integrative Neuroimaging Approach
Hanakawa, T. (2002) · Cerebral Cortex · doi:10.1093/cercor/12.11.1157 · Cited by 240 · cited 1×
“This interpretation is consistent with previous studies showing the role of these cortical areas in imager”
ResultMobini, S., Chiang, T.-J., Ho, M.-Y., Bradshaw, C. M., & Szabadi, E. (2000) · Psychopharmacology · doi:10.1007/s002130000542 · Cited by 228 · cited 1×
“, and animals with lesions in the ascending serotonergic pathway tend to choose small immediate rewards over larger future reward”
BackgroundBalleine, B. W., & Dickinson, A. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-23-08954.2000 · Cited by 218 · cited 1×
“and that insular lesion causes deficits in incentive learning for primary rewar”
BackgroundYokosuka, M., Okamura, H., & Hayashi, S. (1997) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19971208)389:1<81::aid-cne6>3.0.co;2-a · Cited by 198
Pears, A., Parkinson, J. A., Hopewell, L. J., Everitt, B. J., & Roberts, A. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-35-11189.2003 · Cited by 146 · cited 1×
“These results are consistent with the observations that localized damages within the limbic and cognitive loops manifest as deficits in evaluation of future reward”
ResultEagle, D. M., Humby, T., Dunnett, S. B., & Robbins, T. W. (1999) · Behavioral Neuroscience · doi:10.1037/0735-7044.113.4.718 · Cited by 83 · cited 1×
“These results are consistent with the observations that localized damages within the limbic and cognitive loops manifest as deficits in evaluation of future reward”
ResultMijnster, M. J., Raimundo, A. G. V., Koskuba, K., Klop, H., Docter, G. J., Groenewegen, H. J., & Voorn, P. (1997) · The Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19971208)389:1<1::aid-cne1>3.0.co;2-6 · Cited by 68
Compan, V., Ségu, L., Buhot, M.-C., & Daszuta, A. (1998) · Brain Research · doi:10.1016/s0006-8993(98)00168-1 · Cited by 65
Models of Information processing in the Basal Ganglia
Mitchell, I. (1996) · Journal of Chemical Neuroanatomy · doi:10.1016/0891-0618(96)00155-x · Cited by 63
Cited by
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
“Thus, sequentially over time, this suggests a mechanism through which activity occurs across reward-processing episodes during the course of learning (Tanaka et al , 2004).”
BackgroundSeparate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Metadata
- Authors
- Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S.
- Affiliations
- Nara Institute of Science and Technology (JP)
- Japan Science and Technology Agency (JP)
- Hiroshima University (JP)
- Corresponding author
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2004-07-04
- Volume
- 7
- Issue
- 8
- Pages
- 887-893
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2032610016
- Cited by
- 779
- References
- 54
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 16 · 2023: 9 · 2022: 29 · 2021: 28 · 2020: 32 · 2019: 25 · 2018: 32 · 2017: 37 · 2016: 40 · 2015: 45 · 2014: 50 · 2013: 42 · 2012: 47
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Core Research for Evolutional Science and Technology
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reward prediction, time scales, cortico-basal ganglia loop, prediction error, insula, striatum, orbitofrontal cortex, dorsolateral prefrontal cortex, dorsal raphe nucleus, cerebellum, computational modeling, reinforcement learning
- MeSH terms
- Adult, Brain, Brain Mapping, Discrimination Learning, Female, Humans, Magnetic Resonance Imaging, Male, Markov Chains, Middle Aged, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.