Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Evaluation of both immediate and future outcomes of one's actions is a critical requirement for intelligent behavior. Using functional magnetic resonance imaging (fMRI), we investigated brain mechanisms for reward prediction at different time scales in a Markov decision task. When human subjects learned actions on the basis of immediate rewards, significant activity was seen in the lateral orbitofrontal cortex and the striatum. When subjects learned to act in order to obtain large future rewards while incurring small immediate losses, the dorsolateral prefrontal cortex, inferior parietal cortex, dorsal raphe nucleus and cerebellum were also activated. Computational model-based regression analysis using the predicted future rewards and prediction errors estimated from subjects' performance data revealed graded maps of time scale within the insula and the striatum: ventroanterior regions were involved in predicting immediate rewards and dorsoposterior regions were involved in predicting future rewards. These results suggest differential involvement of the cortico-basal ganglia loops in reward prediction at different time scales.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In neurotypical individuals, the key brain loci that have been implicated in prediction are basal ganglia (68, 69), striatum (70, 71), anterior cingulate (72), and cerebellum (73, 74).”
From Autism as a disorder of prediction (2014) · cites this paper as [69]
Citations
Cites
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“, which has been successfully used for explaining reward-predictive activations of the midbrain dopaminergic system as well as those of the cortex and the striatu”
MethodEmotion, Decision Making and the Orbitofrontal Cortex
Bechara, A. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.295 · Cited by 3,589 · cited 1×
“Damage to the prefrontal cortex often impairs daily decision making, which requires assessment of future outcome”
BackgroundBarto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244
Basal ganglia and cerebellar loops: motor and cognitive circuits
Middleton, F. (2000) · Brain Research Reviews · doi:10.1016/s0165-0173(99)00040-5 · Cited by 1,940 · cited 1×
“The cortex and the basal ganglia both have parallel loop organization, with four major loops (limbic, cognitive, motor and oculomotor) and finer, topographic sub-loops within each major loo”
Background(2020) · Machine Learning and Visual Perception · doi:10.1515/9783110595567-011 · Cited by 1,930
The Orbitofrontal Cortex and Reward
Rolls, E. T. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.3.284 · Cited by 1,595
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001) · Neuron · doi:10.1016/s0896-6273(01)00303-8 · Cited by 1,404
Cited by
Items 201–300 of 798
Neural correlates of economic game playing
Krueger, F., Grafman, J., & McCabe, K. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0165 · Cited by 98
“The FPC encodes meta-cognitive representations that enable us to reflect on the long-term values linked to the outcomes of our decisions (Wood & Grafman 2003; Tanaka et al. 2004).”
BackgroundEffects of selective dopaminergic compounds on a delay-discounting task
Koffarnus, M. N., Newman, A. H., Grundt, P., Rice, K. C., & Woods, J. H. (2011) · Behavioural Pharmacology · doi:10.1097/fbp.0b013e3283473bcb · Cited by 91
Brain Hemispheres Selectively Track the Expected Value of Contralateral Options
Palminteri, S., Boraud, T., Lafargue, G., Dubois, B., & Pessiglione, M. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1500-09.2009 · Cited by 92
“…activations, the different responses in instrumental learning tasks are traditionally given with the same hand, such that their neuronal representation cannot be distinguished with fMRI (Haruno et al., 2004; O’Doherty et al., 2004; Tanaka et al., 2004; Daw et al., 2006; Kim et al., 2006).”
BackgroundUnderstanding Neural Coding through the Model-Based Analysis of Decision Making: Figure 1.
Corrado, G., & Doya, K. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1590-07.2007 · Cited by 93
“Tanakaetal.”
Self-citationBasal ganglia and autism – a translational perspective
Subramanian, K., Brandenburg, C., Orsati, F., Soghomonian, J., Hussman, J. P., & Blatt, G. J. (2017) · Autism Research · doi:10.1002/aur.1837 · Cited by 88
Brom, M., Both, S., Laan, E. T. M., Everaerd, W. T. A. M., & Spinhoven, P. (2013) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.10.014 · Cited by 105
Orbitofrontal cortex lesions disrupt risk assessment in a novel serial decision-making task for rats
Pais-Vieira, M., Lima, D., & Galhardo, V. (2007) · Neuroscience · doi:10.1016/j.neuroscience.2006.11.058 · Cited by 105
Neural basis of quasi-rational decision making
Lee, D. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.02.001 · Cited by 107
“For example, the caudate nucleus changes its activation according to the reward prediction error in stochastic decision-making tasks [22 ,23 ], but alters the timing of its activation according to the expectation of benevolent reciprocity during a trust game (Figures 2b and 3; Box 2; [44 ]).”
BackgroundThe prefrontal–limbic network in depression: Modulation by hypothalamus, basal ganglia and midbrain
Bennett, M. R. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.01.006 · Cited by 104
“7, that involves projections from ventromedial prefrontal cortex (vmPFC; BA medial 10) and medial orbitofrontal cortex (BA medial 11, 12, 13) of the PLN (Daw et al., 2006; Hampton et al., 2006; Kim et al., 2006; Tanaka et al., 2004; Valentin et al., 2007).”
Matsunaga, M., Kawamichi, H., Koike, T., Yoshihara, K., Yoshida, Y., Takahashi, H. K., Nakagawa, E., & Sadato, N. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.04.020 · Cited by 98
“It was demonstrated that participants' predictions of the value of future rewards are positively correlated with rACC activity (Tanaka et al., 2004).”
BackgroundThe Cerebellum Is Involved in Reward-based Reversal Learning
Thoma, P., Bellebaum, C., Koch, B., Schwarz, M., & Daum, I. (2008) · The Cerebellum · doi:10.1007/s12311-008-0046-8 · Cited by 101
“In healthy subjects, cerebellar activation was observed during the presentation of unpredicted rewards [33], the prediction of large future rewards as opposed to immediate rewards [ 34 ], and in association with rewarding sexual experiences [35].”
BackgroundExpecting unpleasant stimuli–an fMRI study
Herwig, U., Abler, B., Walter, H., & Erk, S. (2007) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2006.02.007 · Cited by 92
“This adds to reports of ventral striatum activation to be associated with the anticipation of positive stimuli (O'Doherty et al., 2002; Tanaka et al., 2004; Knutson et al., 2004).”
BackgroundVöllm, B., Richardson, P., McKie, S., Elliott, R., Dolan, M., & Deakin, B. (2007) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2007.04.008 · Cited by 92
Johnson, A. C., & Redish, A. D. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.08.009 · Cited by 100
“Recent fMRI experiments have also found that the hemodynamic (BOLD) signal in the striatum correlates with changes in the predicted value function in humans making economic choices (O’Doherty, Dayan, Critchley, & Dolan, 2003; Seymour et al., 2004; Tanaka et al., 2004 ).”
Focal basal ganglia lesions are associated with impairments in reward-based reversal learning
Bellebaum, C., Koch, B., Schwarz, M., & Daum, I. (2008) · Brain · doi:10.1093/brain/awn011 · Cited by 89
Evolved to satisfy our immediate needs: Self-control and the rewarding properties of food
van den Bos, R., & de Ridder, D. (2006) · Appetite · doi:10.1016/j.appet.2006.02.008 · Cited by 91
“…studies using different but related paradigms (temporal discounting paradigm, McClure, Laibson, Loewenstein, & Cohen, 2004; Markov decision task, Tanaka et al., 2004) have confirmed the idea of two systems being involved in the regulation of behaviour at different time-scales: the fast-acting,…”
Is Model Fitting Necessary for Model-Based fMRI?
Wilson, R. C., & Niv, Y. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004237 · Cited by 83
REVIEW: Apathy Diagnosis, Assessment, and Treatment in Alzheimer's Disease
Robert, P. H., Mulin, E., Malléa, P., & David, R. (2010) · CNS Neuroscience & Therapeutics · doi:10.1111/j.1755-5949.2009.00132.x · Cited by 81
Now or later? Striatum and insula activation to immediate versus delayed rewards.
Wittmann, M., Lovero, K. L., Lane, S. D., & Paulus, M. P. (2010) · Journal of Neuroscience, Psychology, and Economics · doi:10.1037/a0017252 · Cited by 83
“However, contrary to earlier findings with intertemporal choice tasks using a different set of time scales (Tanaka et al., 2004; Wittmann, Leland, & Paulus, 2007), we did not find differences in insular cortex activation depending on whether subjects choose immediate or delayed monetary incentives in the range of seco…”
BackgroundResultInfluentialThe neuroscience of impulsive and self-controlled decisions
Kalenscher, T., Ohmann, T., & Güntürkün, O. (2006) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2006.05.010 · Cited by 111
“This conclusion is consistent with the idea of several competing decision-making networks in the brain (McClure et al., 2004; Tanaka et al., 2004; Sanfey et al., 2006).”
ResultKühn, S., Müller, B. C. N., van Baaren, R. B., Wietzker, A., Dijksterhuis, A., & Brass, M. (2010) · Social Neuroscience · doi:10.1080/17470911003633750 · Cited by 83
“In studies focusing on reward mOFC/vmPFC has been shown to be involved in reward expectation, often by using computationally derived regressors to estimate subjects’ expectations (Behrens, Hunt, Woolrich, & Rushworth, 2008; Knutson & Cooper, 2005; Reuter et al., 2005; Tanaka et al., 2004).”
MethodHandbook of Reward and Decision Making
Dreher, J., & Tremblay, L. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.x0001-5 · Cited by 94
Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints
Schweighofer, N., Shishida, K., Han, C. E., Okamoto, Y., Tanaka, S. C., Yamawaki, S., & Doya, K. (2006) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020152 · Cited by 82
“What may be the possible neural correlates of exponential or hyperbolic discounting? We have previously found that parallel cortico–basal ganglia loops are involved in reward prediction with different discounting factors [38].”
BackgroundSelf-citationCho, S. S., Pellecchia, G., Ko, J. H., Ray, N. J., Obeso, I., Houle, S., & Strafella, A. P. (2012) · Brain stimulation · doi:10.1016/j.brs.2012.03.007 · Cited by 88
Vanyukov, P. M., Szanto, K., Hallquist, M. N., Siegle, G. J., Reynolds, C. F., Forman, S. D., Aizenstein, H. J., & Dombrovski, A. Y. (2016) · Psychological Medicine · doi:10.1017/s0033291715001890 · Cited by 83
“Some studies have argued that the neural signature of this ability resides in cognitive control areas, such as the lateral prefrontal cortex (PFC) (McClure et al. 2004; Tanaka et al. 2004; Wittmann et al. 2007).”
BackgroundDecodability of Reward Learning Signals Predicts Mood Fluctuations
Eldar, E., Roth, C., Dayan, P., & Dolan, R. J. (2018) · Current Biology · doi:10.1016/j.cub.2018.03.038 · Cited by 83
Huys, Q. J. M., Guitart-Masip, M., Dolan, R. J., & Dayan, P. (2015) · Clinical Psychological Science · doi:10.1177/2167702614562040 · Cited by 80
“One influential notion has been that the neuromodulator serotonin controls γ , and hence the degree of patience (Miyazaki, Miyazaki, & Doya, 2012; Schweighofer et al., 2008; Tanaka et al., 2004; Tanaka et al., 2007) or willingness to wait.”
BackgroundA neurochemical hypothesis for the origin of hominids
Raghanti, M. A., Edler, M. K., Stephenson, A. R., Munger, E. L., Jacobs, B., Hof, P. R., Sherwood, C. C., Holloway, R. L., & Lovejoy, C. O. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1719666115 · Cited by 83
The Different Facets of Anhedonia and Their Associations with Different Psychopathologies
Shankman, S. A., Katz, A. C., DeLizza, A. A., Sarapas, C., Gorka, S. M., & Campbell, M. L. (2014) · Springer eBooks · doi:10.1007/978-94-017-8591-4_1 · Cited by 103
Meder, D., Kolling, N., Verhagen, L., Wittmann, M. K., Scholl, J., Madsen, K. H., Hulme, O. J., Behrens, T. E. J., & Rushworth, M. F. S. (2017) · Nature Communications · doi:10.1038/s41467-017-02169-w · Cited by 85
“(1) Decision phase main effect (duration: stimuli onset until response) (2) Predict phase main effect (duration: response until outcome) (3) Outcome monitor phase main effect (duration: 3 s) (4) Parametric modulation of decision phase with reward magnitude of chosen stimulus (5) Parametric modulation of decision phase…”
BackgroundDopamine neurons learn to encode the long-term value of multiple future rewards
Enomoto, K., Matsumoto, N., Nakai, S., Satoh, T., Sato, T. K., Ueda, Y., Inokawa, H., Haruno, M., & Kimura, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014457108 · Cited by 83
“A human imaging study (16) examined brain areas for reward prediction using different time scales in a Markov decision task that used a computational modelbased regression analysis.”
BackgroundMethodThe Brain Network of Expectancy and Uncertainty Processing
Catena, A., Perales, J. C., Megías‐Robles, A., Cândido, A., Jara, E., & Maldonado, A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040252 · Cited by 85
Dopaminergic function and intertemporal choice
Joutsa, J., Voon, V., Johansson, J. J., Niemelä, S., Bergman, J. R., & Kaasinen, V. (2015) · Translational Psychiatry · doi:10.1038/tp.2014.133 · Cited by 87
Prefrontal cognitive systems in schizophrenia : Towards human genetic brain mechanisms
Tan, H.-Y., Callicott, J. H., & Weinberger, D. R. (2009) · Cognitive Neuropsychiatry · doi:10.1080/13546800903091665 · Cited by 83
“This D2 process is key in ensuring that salient new information from cognitive computations of task-goal directed or reward information, is rapidly processed and updated online; this information processing network involves the PFC, posterior cortex, and striatum (Goldman-Rakic, 1995 ; Mink, 1996 ; O'Reilly, 2006 ; Tan…”
BackgroundEfficient reinforcement learning: computational theories, neuroscience and robotics
Kawato, M., & Samejima, K. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.03.004 · Cited by 93
Time perception and impulsivity: A proposed relationship in addictive disorders
Paasche, C., Weibel, S., Wittmann, M., & Lalanne, L. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.12.006 · Cited by 68
Neural circuitry and mechanisms of waiting impulsivity: relevance to addiction
Dalley, J. W., & Ersche, K. D. (2018) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0145 · Cited by 84
Koot, S., Zoratto, F., Cassano, T., Colangeli, R., Laviola, G., van den Bos, R., & Adriani, W. (2011) · Neuropharmacology · doi:10.1016/j.neuropharm.2011.11.002 · Cited by 86
“…may be related to reward sensitivity (Hayes and Greenshaw, 2011), punishment sensitivity (Cools et al., 2008, 2011) or behavioural flexibility, and points to the interplay among different cortico-basal ganglia loops (Bechara, 2005; Doya, 2008; Homberg, 2012; Tanaka et al., 2004, 2007, 2009).”
InfluentialThe neural correlates of temporal reward discounting
Scheres, A., de Water, E., & Mies, G. W. (2013) · WIREs Cognitive Science · doi:10.1002/wcs.1246 · Cited by 75
The shortsighted brain: Neuroeconomics and the governance of choice in time
Schüll, N. D., & Zaloom, C. (2011) · Social Studies of Science · doi:10.1177/0306312710397689 · Cited by 99
“‘If you look downstream enough’, Laibson told the audience, ‘you’ll find what looks like an integrated system’ (Cohen et al., 2008; see also Hariri et al., 2006; Kable and Glimcher, 2007; McClure et al., 2004; Tanaka et al., 2004).”
BackgroundCorrelation between insula activation and self-reported quality of orgasm in women
Ortigue, S., Grafton, S. T., & Bianchi‐Demicheli, F. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.05.026 · Cited by 88
Neural Correlates of Impulsivity in Healthy Males and Females with Family Histories of Alcoholism
DeVito, E. E., Meda, S. A., Jiantonio‐Kelly, R. E., Potenza, M. N., Krystal, J. H., & Pearlson, G. D. (2013) · Neuropsychopharmacology · doi:10.1038/npp.2013.92 · Cited by 84
“The insula is implicated in impulsivity-related constructs, including, but not limited to, choice impulsivity (Tanaka et al, 2004) and risk and uncertainty assessment (Clark et al, 2008).”
BackgroundBell, P. T., Gilat, M., O’Callaghan, C., Copland, D. A., Frank, M. J., Lewis, S. J. G., & Shine, J. M. (2014) · Human Brain Mapping · doi:10.1002/hbm.22701 · Cited by 85
“Previous work has demonstrated that the striatal nuclei are subdivided into specialized territories associated with specific functions related to their cortical connections: the ventral striatum is implicated in motivation and reward (Schultz et al., 1997; Tanaka et al., 2004), the head of the caudate nucleus in cogni…”
BackgroundCortico‐basal ganglia mechanisms for overcoming innate, habitual and motivational behaviors
Isoda, M., & Hikosaka, O. (2011) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07698.x · Cited by 75
“Moreover, the value of a reward is higher if it will be available ‘now’ as opposed to later, a phenomenon known as temporal discounting (Mazur, 1984; Green & Myerson, 2004; McClure et al., 2004; Tanaka et al., 2004).”
BackgroundHippocampal Neural Correlates for Values of Experienced Events
Lee, H., Ghim, J.-W., Kim, H., Lee, D., & Jung, M. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.2806-12.2012 · Cited by 79
“Moreover, BOLD signals in the hippocampus were correlated with state value (reward prediction in a Markov decision task) (Tanaka et al., 2004) as well as prediction error that was computed with an RL model in probabilistic learning tasks (Dickerson et al., 2011; Foerde and Shohamy, 2011).”
MethodBrain mechanism of reward prediction under predictable and unpredictable environmental dynamics
Tanaka, S., Samejima, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S., & Doya, K. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.039 · Cited by 89
Self-citationvan Duuren, E., van der Plasse, G., Lankelma, J., Joosten, R. N. J. M. A., Feenstra, M. G. P., & Pennartz, C. M. A. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.0005-09.2009 · Cited by 78
“…activate the orbitofrontal cortex (OFC) (Rogers et al., 1999; Ernst et al., 2004), an area of the prefrontal cortex that has been strongly implicated in the assessment of reward value (O’Doherty et al., 2001, 2003) and in the planning of actions leading to immediate rewards (Tanaka et al., 2004).”
BackgroundMeade, C. S., Lowen, S. B., MacLean, R. R., Key, M. D., & Lukas, S. E. (2011) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2010.12.011 · Cited by 79
“Functional magnetic resonance imaging (fMRI) studies in healthy adults suggest that cortical areas, particularly the lateral prefrontal and posterior parietal cortices, are involved in hard decisions between similarly valued immediate and delayed rewards (McClure et al., 2004; Tanaka et al., 2004).”
BackgroundMarsh, B. T., Tarigoppula, V. S. A., Chen, C., & Francis, J. T. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.1802-14.2015 · Cited by 88
“…a variety of cortical and noncortical regions, many of which connect to M1 (Platt and Glimcher, 1999; Roesch and Olson, 2003; Musallam et al., 2004; Tanaka et al., 2004; Campos et al., 2005; Shuler and Bear, 2006; Louie et al., 2011; Mizuhiki et al., 2012), and dopamine receptors exist in M1…”
BackgroundTemporal discounting and inter-temporal choice in rhesus monkeys
Hwang, J., Kim, S., & Lee, D. (2009) · Frontiers in Behavioral Neuroscience · doi:10.3389/neuro.08.009.2009 · Cited by 86
“Similarly, neuroimaging studies in human subjects have suggested that the dorsolateral prefrontal cortex might play an important role in evaluating the value of delayed reward (Luhmann et al., 2008; McClure et al., 2004, 2007; Tanaka et al., 2004).”
BackgroundTemporal-Difference Reinforcement Learning with Distributed Representations
Kurth-Nelson, Z. L., & Redish, A. D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007362 · Cited by 92
“The model is also consistent with recent fMRI studies suggesting that the striatum consists of functional ‘‘slices’’ reflecting a range of discounting factors [30,31].”
BackgroundResultInfluentialAdvances in understanding ventromedial prefrontal function: The accountant joins the executive
Fellows, L. K. (2007) · Neurology · doi:10.1212/01.wnl.0000257835.46290.57 · Cited by 79
Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning
Lukšys, G., Gerstner, W., & Sandi, C. (2009) · Nature Neuroscience · doi:10.1038/nn.2374 · Cited by 86
Cohen, M. X. (2007) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsl021 · Cited by 78
“This method has been previously used to study the putative neural correlates of prediction errors (O’Doherty et al., 2003; O’Doherty et al., 2004; Seymour et al., 2004; Tanaka et al., 2004; Glascher and Buchel, 2005; Haruno and Kawato, 2006).”
MethodNeuroimaging of interval timing
Meck, W. H., & Malapani, C. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.07.010 · Cited by 86
Smolker, H. R., Depue, B. E., Reineberg, A. E., Orr, J. M., & Banich, M. T. (2015) · Brain Structure and Function · doi:10.1007/s00429-014-0723-y · Cited by 77
“…considered to support EF (however, see work regarding cognitive flexibility, Kehagia et al. 2010), but rather is a region implicated in emotion, motivation, and reward processing (Glascher et al. 2009; Plassmann et al. 2007; Sescousse et al. 2010; Tanaka et al. 2004; Wunderlich et al. 2010).”
BackgroundMoreno-Rius, J. (2019) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2019.100774 · Cited by 82
Subjective neuronal coding of reward: temporal value discounting and risk
Schultz, W. (2010) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07282.x · Cited by 74
Nakagawa, Y., & Chiba, K. (2016) · Journal of Pharmacology and Experimental Therapeutics · doi:10.1124/jpet.116.234476 · Cited by 79
The neural basis of motivational influences on cognitive control
Parro, C., Dixon, M. L., & Christoff, K. (2018) · Human Brain Mapping · doi:10.1002/hbm.24348 · Cited by 81
Reward‐related activity in the human motor cortex
Kapogiannis, D., Campion, P., Grafman, J., & Wassermann, E. M. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06147.x · Cited by 70
“Reward signals are thought to drive the plastic changes that underlie learning (Waelti et al., 2001; Schultz, 2004) in cortical (Ueki et al., 2006) and striatal (Haruno et al., 2004; Tanaka et al., 2004; Rodriguez et al., 2006) neurons.”
BackgroundFaulkner, P., & Deakin, J. F. W. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.07.024 · Cited by 89
Eriksson, T. M., Delagrange, P., Spedding, M., Popoli, M., Mathé, A. A., Ögren, S. O., & Svenningsson, P. (2012) · Molecular Psychiatry · doi:10.1038/mp.2010.131 · Cited by 73
Instrumental vigour in punishment and reward
Dayan, P. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08026.x · Cited by 72
“Even then, we have passed over a huge wealth of issues discussed in these reviews, including other influential views of serotonin, such as its potential role in temporal discounting (Tanaka et al., 2004, 2007; Doya, 2008; Schweighofer et al., 2008).”
BackgroundMorita, K., Morishima, M., Sakai, K., & Kawaguchi, Y. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.04.009 · Cited by 78
The brain structural disposition to social interaction
Lebreton, M., Barnes, A., Miettunen, J., Peltonen, L., Ridler, K., Veijola, J., Tanskanen, P., Suckling, J., Jarvelin, M., Jones, P. B., et al. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06782.x · Cited by 70
“, 2003); and also in anticipation of a simpler reward, money (Tanaka et al., 2004).”
Li, W., Li, Y., Yang, W., Zhang, Q., Wei, D., Li, W., Hitchman, G., & Qiu, J. (2015) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2015.02.019 · Cited by 76
“This region also has been shown to be involved in predicting future rewards (Tanaka et al., 2004) and successful emotion regulation or impulse control processes (Staudinger et al., 2011).”
BackgroundMacDonald, C. J., Cheng, R., & Meck, W. H. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00010 · Cited by 74
“For example, the VS contribute to inhibiting behavior that is made inappropriate so that novel behavioral strategies may be acquired (Tanaka et al., 2004, 2007; Floresco et al., 2006).”
BackgroundThe cerebellum in drug craving
Moreno, J. M. M., & Miquel, M. (2017) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2016.12.028 · Cited by 77
“predictive function of the cerebellum seems to be shared with several other brain cortical and subcortical areas, such as striatal zones (Tanaka et al., 2004), parietal lobes (Blakemore and Sirigu, 2003), and prefrontal cortices, especially the orbitofrontal cortex (Eppinger et al.”
BackgroundIntegrating early results on ventral striatal gamma oscillations in the rat
van der Meer, M. A. A., Kalenscher, T., Lansink, C. S., Pennartz, C. M. A., Berke, J. D., & Redish, A. D. (2010) · Frontiers in Neuroscience · doi:10.3389/fnins.2010.00300 · Cited by 78
“, 2007) supported by differences in recording and imaging studies during behavior (O’Doherty et al., 2004; Tanaka et al., 2004; Kimchi and Laubach, 2009; van der Meer et al., 2009).”
Neuroeconomics and the Study of Addiction
Monterosso, J. R., Piray, P., & LUO, S. H. (2012) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.03.012 · Cited by 79
Intertemporal Decision-Making Involves Prefrontal Control Mechanisms Associated with Working Memory
Jimura, K., Chushak, M. S., Westbrook, A., & Braver, T. S. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx015 · Cited by 80
“Prior work has demonstrated that increasing dlPFC activity predicts delayed reward selection in intertemporal decisionmaking (Tanaka et al. 2004; McClure et al. 2004, 2007; Hare et al. 2014).”
BackgroundIsik, A. I., & Vessel, E. A. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.676032 · Cited by 79
A rodent version of the Iowa Gambling Task: 7 years of progress
Bos, R. V. D., Koot, S., & Visser, L. D. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00203 · Cited by 61
“Furthermore, the insular cortex has connections with the dorsal and ventral striatum and thereby may exert an effect on immediate and longterm focus (see Tanaka et al., 2004).”
Background“Shall We Play a Game?”: Improving Reading Through Action Video Games in Developmental Dyslexia
Franceschini, S., Bertoni, S., Ronconi, L., Molteni, M., Gori, S., & Facoetti, A. (2015) · Current Developmental Disorders Reports · doi:10.1007/s40474-015-0064-4 · Cited by 78
Bitanihirwe, B. K. Y., Weber, L., Feldon, J., & Meyer, U. (2010) · International Journal of Neuropsychopharmacology · doi:10.1017/s1461145710000192 · Cited by 64
“2004, 2005), and dopamine D2 receptors are suggested to play a complementary role to D1 receptors in the regulation of working memory (Floresco & Magyar, 2006 ; Goldman-Rakic, 1996 ; Tanaka et al. 2004).”
The neural basis of sex differences in sexual behavior: A quantitative meta-analysis
Poeppl, T. B., Langguth, B., Rupprecht, R., Safron, A., Bzdok, D., Laird, A. R., & Eickhoff, S. B. (2016) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2016.10.001 · Cited by 74
“Corticobasal ganglia loops are believed to mediate the perceived urge to express sexual behavior and therefore sexual motivation (Kühn and Gallinat, 2011; Stoléru, 2014; Tanaka et al., 2004).”
BackgroundKann, S. J., Zhang, S., Manza, P., Leung, H., & Li, C. R. (2016) · Brain Connectivity · doi:10.1089/brain.2016.0443 · Cited by 72
“The striatum and insula are heavily connected anatomically (Chikama et al., 1997) and both structures are associated with reward predictions (Tachibana and Hikosaka, 2012; Tanaka et al., 2004; Wittmann et al., 2011).”
BackgroundDoya, K., & Uchibe, E. (2005) · Adaptive Behavior · doi:10.1177/105971230501300206 · Cited by 69
“A recent brain imaging experiment suggests that there are parallel reinforcement learning pathways in the human brain, each specialized for reward prediction at different time scales (Tanaka et al., 2004).”
BackgroundSelf-citationBühringer, G., Wïttchen, H., Gottlebe, K., Kufeld, C., & Goschke, T. (2008) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.246 · Cited by 76
“…results from the interaction between two distinct neural networks, one of which is tuned to immediate rewards and steeply discounts delayed rewards, and the other mediates the anticipation of future rewards and discounts delayed rewards only minimally (McClure et al., 2004; Tanaka et al., 2004).”
BackgroundThe Case for Adaptive Neuromodulation to Treat Severe Intractable Mental Disorders
Provenza, N. R., Matteson, E., Allawala, A. B., Barrios-Anderson, A., Sheth, S. A., Viswanathan, A., McIngvale, E., Storch, E. A., Frank, M. J., McLaughlin, N. C. R., et al. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00152 · Cited by 71
“…obsessive compulsive disorder, biomarkers INTRODUCTION Mental illness is a leading cause of disability and mortality that affects approximately 13–17% of individuals worldwide (Insel, 2009; Whiteford et al., 2013; Steel et al., 2014; Polanczyk et al., 2015; Walker et al., 2015; Vigo et al., 2016).”
BackgroundThe Role of the Parietal Cortex in the Representation of Task-Reward Associations
Wisniewski, D., Reverberi, C., Momennejad, I., Kahnt, T., & Haynes, J.-D. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.4882-14.2015 · Cited by 67
“It is also important for the processing of future rewards in humans (Tanaka et al., 2004), playing an important role in guiding decisions based on reward values.”
BackgroundFilippone, L., Shankland, R., & Hallez, Q. (2022) · Journal of Eating Disorders · doi:10.1186/s40337-022-00698-4 · Cited by 100
“These results corroborate those of neuro-anatomic studies that found changes in the cerebral structures involved in cognitive impulsivity in individuals suffering from internet addiction [21, 53, 90].”
ResultMedial prefrontal cortex and the adaptive regulation of reinforcement learning parameters
Khamassi, M., Enel, P., Dominey, P. F., & Procyk, E. (2013) · Progress in brain research · doi:10.1016/b978-0-444-62604-2.00022-8 · Cited by 73
Effect of orbitofrontal cortex lesions on temporal discounting in rats
Jo, S., Kim, K.-U., Lee, D., & Jung, M. W. (2013) · Behavioural Brain Research · doi:10.1016/j.bbr.2013.02.014 · Cited by 71
“Brain imaging studies have also found blood oxygenation level-dependent signals related to temporally discounted values in multiple areas of the brain (McClure et al., 2004; Tanaka et al., 2004; Kable and Glimcher, 2007; McClure et al., 2007; Luhmann et al., 2008).”
BackgroundAnatomy and Connectivity of the Reward Circuit
Haber, S. N. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-805308-9.00001-4 · Cited by 72
Koelsch, S., Siebel, W. A., & Fritz, T. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199230143.003.0012 · Cited by 73
Subcortical Dopamine and Cognition in Schizophrenia: Looking Beyond Psychosis in Preclinical Models
Conn, K., Burne, T. H. J., & Kesby, J. P. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00542 · Cited by 68
Neuronal Activity Representing Temporal Prediction of Reward in the Primate Prefrontal Cortex
Tsujimoto, S., & Sawaguchi, T. (2005) · Journal of Neurophysiology · doi:10.1152/jn.01149.2004 · Cited by 64
“The present data introduce a new dimension that modulates the activities of DLPFC neurons (i.e., the time of reward), which is also a critical variable in such cognitive processes (Dickinson et al. 1976; Hollerman and Schultz 1998; McClure et al. 2004; Tanaka et al. 2004).”
BackgroundRe-examining the Mysterious Role of the Cerebellum in Pain
Li, C. N., Keay, K. A., Henderson, L. A., & Mychasiuk, R. (2024) · Journal of Neuroscience · doi:10.1523/jneurosci.1538-23.2024 · Cited by 71
Moderators of the association between brain activation and farsighted choice
LUO, S. H., Ainslie, G. W., Pollini, D., Giragosian, L., & Monterosso, J. R. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.004 · Cited by 73
“The importance of the posterior insula in LL choice is also suggested by a study of reward-based learning (Tanaka et al., 2004), in which posterior and anterior portions of insular were preferentially recruited during responses consistent with LL reward pursuit (relative to SS reward pursuit).”
de Visser, L. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00102 · Cited by 62
Serotonin and the Evaluation of Future Rewards: Theory, Experiments, and Possible Neural Mechanisms
SCHWEIGHOFER, N., TANAKA, S. C., & DOYA, K. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.011 · Cited by 61
Self-citationBehavioral and neural predictors of upcoming decisions
Cohen, M. X., & Ranganath, C. (2005) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.5.2.117 · Cited by 78
“Our results might seem to contrast with results recently reported by Tanaka and colleagues (Tanaka et al., 2004), in which they reported that a discount factor predicted brain activity in the striatum, the medial PFC, and the insula during a decision-making task.”
ResultSohn, J.-W., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2982-07.2007 · Cited by 61
“For example, the ventral striatum increases its metabolic activity when the subjects choose the options that can maximize the reward immediately (Tanaka et al., 2004; Hariri et al., 2006).”
BackgroundYeh, Y.-W., Ho, P.-S., Kuo, S.-C., Chen, C.-Y., Liang, C.-S., Yen, C.-H., Huang, C.-C., Ma, K.-H., Shiue, C.-Y., Huang, W.-S., et al. (2015) · International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyu120 · Cited by 61
“Among these, the corticobasal ganglia pathway plays a crucial role in the neuropathology of mood disorders (Marchand et al., 2012) and reward prediction (Tanaka et al., 2004), and could contribute to the response and decision of antidepressant adherence.”
BackgroundPolicy Adjustment in a Dynamic Economic Game
Li, J., McClure, S. M., King-Casas, B., & Read Montague, P. (2006) · PLoS ONE · doi:10.1371/journal.pone.0000103 · Cited by 63
Suenaga, S., Nagayama, K., Nagasawa, T., Indo, H. P., & Majima, H. J. (2016) · Japanese Dental Science Review · doi:10.1016/j.jdsr.2016.04.004 · Cited by 73
Miyazaki, K. W., Miyazaki, K., & Doya, K. (2011) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07480.x · Cited by 59
Self-citation
Metadata
- Authors
- Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S.
- Affiliations
- Nara Institute of Science and Technology (JP)
- Japan Science and Technology Agency (JP)
- Hiroshima University (JP)
- Corresponding author
- Doya, Kenji
- Venue
- Nature Neuroscience
- Published
- 2004-07-04
- Volume
- 7
- Issue
- 8
- Pages
- 887-893
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2032610016
- Cited by
- 779
- References
- 54
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 16 · 2023: 9 · 2022: 29 · 2021: 28 · 2020: 32 · 2019: 25 · 2018: 32 · 2017: 37 · 2016: 40 · 2015: 45 · 2014: 50 · 2013: 42 · 2012: 47
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Core Research for Evolutional Science and Technology
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- reward prediction, time scales, cortico-basal ganglia loop, prediction error, insula, striatum, orbitofrontal cortex, dorsolateral prefrontal cortex, dorsal raphe nucleus, cerebellum, computational modeling, reinforcement learning
- MeSH terms
- Adult, Brain, Brain Mapping, Discrimination Learning, Female, Humans, Magnetic Resonance Imaging, Male, Markov Chains, Middle Aged, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.