Neural Circuitry of Reward Prediction Error
Open accessAbstract
Dopamine neurons facilitate learning by calculating reward prediction error, or the difference between expected and actual reward. Despite two decades of research, it remains unclear how dopamine neurons make this calculation. Here we review studies that tackle this problem from a diverse set of approaches, from anatomy to electrophysiology to computational modeling and behavior. Several patterns emerge from this synthesis: that dopamine neurons themselves calculate reward prediction error, rather than inherit it passively from upstream regions; that they combine multiple separate and redundant inputs, which are themselves interconnected in a dense recurrent network; and that despite the complexity of inputs, the output from dopamine neurons is remarkably homogeneous and robust. The more we study this simple arithmetic computation, the knottier it appears to be, suggesting a daunting (but stimulating) path ahead for neuroscience more generally.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Dopaminergic signaling in the NAc has been widely studied for its involvement in motivational salience, behavioral reinforcement, and reward prediction error encoding.
“Dopaminergic signaling in the NAc has been widely studied for its involvement in motivational salience, behavioral reinforcement, and reward prediction error encoding (Schultz et al., 2015; Berridge and Robinson, 2016; Watabe-Uchida et al., 2017; Berke, 2018).”
From Ventral striatum dopamine release encodes unique properties of visual stimuli in mice (2023) · cites this paper as Watabe-Uchida et al., 2017
In “Complexities in the dopamine signal” (page unavailable), the review says nucleus accumbens dopamine encodes reward prediction errors relatively faithfully in simple tasks. “Functions of dopamine prediction error signals” describes dopamine’s role in behavioral reinforcement, but does not specifically locate that finding in the accumbens. The review associates motivational-salience signals mainly with neurons in the lateral midbrain, particularly the substantia nigra pars compacta, rather than with accumbens dopamine. (LLM, conf=0.86)
Experimental — accuracy not yet measuredD1-type dopamine receptor-expressing medium spiny neurons of the direct pathway project to and inhibit VTA GABAergic interneurons that inhibit VTA dopamine cells; activating them disinhibits dopamine projections to the forebrain.
“As originally described in the dorsal striatum, MSNs expressing D1-type dopamine receptors, canonically considered the 'direct' pathway, project to and inhibit GABAergic interneurons within the VTA that, in turn, inhibit VTA dopamine releasing cells.Activation of D1-expressing MSNs thereby promotes activation of dopamine projections back to the forebrain by disinhibiting them [220].”
From Plasticity of synapses and reward circuit function in the genesis and treatment of depression (2023) · cites this paper as [220]
D2-type dopamine receptor-expressing medium spiny neurons of the indirect pathway project to ventral pallidum GABAergic cells that project to VTA interneurons; activating them inhibits dopamine projections by disinhibiting the interneurons that silence them.
“In contrast, MSNs expressing D2-type dopamine receptors, canonically considered the 'indirect' pathway, project to GABAergic cells within the ventral pallidum, that then project to the interneurons within the VTA.Activation of D2-expressing MSNs thereby inhibits activation of dopamine projections back to the forebrain by disinhibiting the interneurons that silence them.”
From Plasticity of synapses and reward circuit function in the genesis and treatment of depression (2023) · cites this paper as [220]
Dopamine dynamics within the mesolimbic pathway play a central role in appetitive learning and approach behavior.
“Dopamine dynamics within the mesolimbic pathway play a central role in appetitive learning and approach behavior5,6.”
From Dopamine dynamics as a regulatory mechanism for shifting between defensive and reward-seeking behaviors (2026) · cites this paper as [6]
According to reinforcement learning theory, the US response is a driver of learning, while the CS response is considered a consequence of learning.
“given that, according to reinforcement learning theory, the US response is a driver of learning, while the CS response is considered a consequence of learning6,18–20.”
From Dopamine dynamics as a regulatory mechanism for shifting between defensive and reward-seeking behaviors (2026) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1× · see citing works
“…hypotheses, which are then tested against sensory evidence—has been discussed in a multitude of contexts, from visual processing to motor learning, cerebellum to cortex, simple organisms such as electric fish to complex human diseases such as schizophrenia (Friston 2012, Rao & Ballard 1999).”
Millisecond-timescale, genetically targeted optical control of neural activity
Boyden, E. S., Zhang, F., Bamberg, E., Nagel, G., & Deisseroth, K. A. (2005) · Nature Neuroscience · doi:10.1038/nn1525 · Cited by 4,831 · see citing works
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564 · see citing works
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449 · see citing works
Dopamine, learning and motivation.
Wise, R. A. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1406 · Cited by 3,379 · cited 1× · see citing works
“Recent experimental findings using newer techniques have reinforced this idea, although they by no means exclude other functions of dopamine (Wise 2004).”
Synaptic Plasticity and Memory: An Evaluation of the Hypothesis
Martin, S. J., Grimwood, P. D., & Morris, R. G. M. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.649 · Cited by 2,794 · see citing works
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · see citing works
Discrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · see citing works
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · see citing works
Wise, R. A., & Rompre, P.-P. (1989) · Annual Review of Psychology · doi:10.1146/annurev.ps.40.020189.001203 · Cited by 1,968 · see citing works
Two types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · see citing works
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318 · see citing works
Self-citationMidbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · see citing works
Input-specific control of reward and aversion in the ventral tegmental area
Lammel, S., Lim, B. K., Ran, C., Huang, K. W., Betley, M. J., Tye, K. M., Deisseroth, K., & Malenka, R. C. (2012) · Nature · doi:10.1038/nature11527 · Cited by 1,300 · see citing works
Lateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292 · see citing works
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · see citing works
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266 · see citing works
Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons
Watabe‐Uchida, M., Zhu, L., Ogawa, S. K., Vamanrao, A., & Uchida, N. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.017 · Cited by 1,221 · see citing works
Self-citationMonosynaptic Restriction of Transsynaptic Tracing from Single, Genetically Targeted Neurons
Wickersham, I. R., Lyon, D. C., Barnard, R. J. O., Mori, T., Finke, S., Conzelmann, K., Young, J. A. T., & Callaway, E. M. (2007) · Neuron · doi:10.1016/j.neuron.2007.01.033 · Cited by 1,176 · see citing works
Circuit architecture of VTA dopamine neurons revealed by systematic input-output mapping
Beier, K. T., Steinberg, E. E., DeLoach, K. E., Xie, S., Miyamichi, K., Schwarz, L., Gao, X. J., Kremer, E. J., Malenka, R. C., & Luo, L. (2015) · Cell · doi:10.1016/j.cell.2015.07.015 · Cited by 1,068 · see citing works
Dopamine reward prediction-error signalling: A two-component response
Schultz, W. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2015.26 · Cited by 1,036 · see citing works
A selective role for dopamine in stimulus-reward learning
Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., Akers, C. A., Clinton, S. M., Phillips, P. E. M., & Akil, H. (2011) · Nature · doi:10.1038/nature09588 · Cited by 1,026 · cited 2× · see citing works
“Activities consistent with RPEs have been demonstrated in monkeys (Bayer & Glimcher 2005, Enomoto et al. 2011), rats (Day et al. 2007, Flagel et al. 2011, Hart et al. 2014, Oleson et al. 2012, Roesch et al. 2007, Stuber et al. 2008), and humans (D’Ardenne et al. 2008) and appear to faithfully…”
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · see citing works
Gain Modulation from Background Synaptic Input
Chance, F. S., Abbott, L. F., & Reyes, A. D. (2002) · Neuron · doi:10.1016/s0896-6273(02)00820-6 · Cited by 1,012 · see citing works
Cortico-Basal Ganglia Reward Network: Microcircuitry
Sesack, S. R., & Grace, A. A. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.93 · Cited by 985 · see citing works
Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976 · see citing works
Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events
Horvitz, J. C. (2000) · Neuroscience · doi:10.1016/s0306-4522(00)00019-1 · Cited by 967 · see citing works
Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis
Glimcher, P. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014269108 · Cited by 955 · see citing works
Grace, A. A., & Bunney, B. S. (1983) · Neuroscience · doi:10.1016/0306-4522(83)90135-5 · Cited by 937 · see citing works
A causal link between prediction errors, dopamine neurons and learning
Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K. A., & Janak, P. H. (2013) · Nature Neuroscience · doi:10.1038/nn.3413 · Cited by 879 · see citing works
Mesolimbic dopamine signals the value of work
Hamid, A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Weele, C. M. V., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2015) · Nature Neuroscience · doi:10.1038/nn.4173 · Cited by 815 · see citing works
Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits
Lerner, T. N., Shilyansky, C., Davidson, T. J., Evans, K. E., Beier, K. T., Zalocusky, K. A., Crow, A. K., Malenka, R. C., Luo, L., Tomer, R., et al. (2015) · Cell · doi:10.1016/j.cell.2015.07.014 · Cited by 794 · cited 3× · see citing works
“Instead, they increase their firing to both rewarding and aversive events (Fiorillo et al. 2013, Horvitz 2000, Joshua et al. 2008, Lerner et al. 2015, Matsumoto & Hikosaka 2009b).”
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710 · see citing works
Jhou, T. C., Fields, H. L., Baxter, M. G., Saper, C. B., & Holland, P. C. (2009) · Neuron · doi:10.1016/j.neuron.2009.02.001 · Cited by 696 · see citing works
Witten, I. B., Steinberg, E. E., Lee, S. Y., Davidson, T. J., Zalocusky, K. A., Brodsky, M., Yizhar, O., Cho, S. L., Gong, S., Ramakrishnan, C., et al. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.028 · Cited by 680 · see citing works
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661 · see citing works
Updating dopamine reward signals
Schultz, W. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.11.012 · Cited by 651 · see citing works
GABA Neurons of the VTA Drive Conditioned Place Aversion
Tan, K. R., Yvon, C., Turiault, M., Mirzabekov, J. J., Doehner, J., Labouèbe, G., Deisseroth, K. A., Tye, K. M., & Lüscher, C. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.015 · Cited by 635 · see citing works
Parvalbumin-expressing interneurons linearly transform cortical responses to visual stimuli
Atallah, B. V., Bruns, W., Carandini, M., & Scanziani, M. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.013 · Cited by 612 · see citing works
Start/stop signals emerge in nigrostriatal circuits during sequence learning
Jin, X., & Costa, R. M. (2010) · Nature · doi:10.1038/nature09263 · Cited by 611 · see citing works
Activation of VTA GABA Neurons Disrupts Reward Consumption
van Zessen, R., Phillips, J. L., Budygin, E. A., & Stuber, G. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.02.016 · Cited by 609 · see citing works
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597 · see citing works
Dopamine reward prediction error coding
Schultz, W. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.1/wschultz · Cited by 597 · see citing works
Rapid signalling in distinct dopaminergic axons during locomotion and reward
Howe, M. W., & Dombeck, D. A. (2016) · Nature · doi:10.1038/nature18942 · Cited by 597 · see citing works
Activation of specific interneurons improves V1 feature selectivity and visual perception
Lee, S.-H., Kwan, A. C., Zhang, S., Phoumthipphavong, V., Flannery, J. G., Masmanidis, S. C., Taniguchi, H., Huang, Z. J., Zhang, F., Boyden, E. S., et al. (2012) · Nature · doi:10.1038/nature11312 · Cited by 593 · see citing works
A mathematical model for simple learning.
Bush, R. R., & Mosteller, F. C. (1951) · Psychological Review · doi:10.1037/h0054388 · Cited by 590 · see citing works
Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574 · see citing works
Division and subtraction by distinct cortical inhibitory networks in vivo
Wilson, N. R., Runyan, C. A., Wang, F. L., & Sur, M. (2012) · Nature · doi:10.1038/nature11347 · Cited by 568 · see citing works
Prolonged dopamine signalling in striatum signals proximity and value of distant rewards
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E. M., & Graybiel, A. M. (2013) · Nature · doi:10.1038/nature12475 · Cited by 565 · see citing works
Representation of negative motivational value in the primate lateral habenula
Matsumoto, M., & Hikosaka, O. (2009) · Nature Neuroscience · doi:10.1038/nn.2233 · Cited by 547 · see citing works
Importance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 544 · see citing works
Margolis, E. B., Lock, H., Hjelmstad, G. O., & Fields, H. L. (2006) · The Journal of Physiology · doi:10.1113/jphysiol.2006.117069 · Cited by 527 · see citing works
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500 · see citing works
Divisive Normalization in Olfactory Population Codes
Olsen, S. R., Bhandawat, V., & Wilson, R. I. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.009 · Cited by 500 · see citing works
Glutamatergic Afferents of the Ventral Tegmental Area in the Rat
Geisler, S., Derst, C., Veh, R. W., & Zahm, D. S. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.0012-07.2007 · Cited by 486 · see citing works
Silver, R. A. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2864 · Cited by 481 · see citing works
Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons
Ungless, M. A., & Grace, A. A. (2012) · Trends in Neurosciences · doi:10.1016/j.tins.2012.02.003 · Cited by 431 · see citing works
Influence of Reward Delays on Responses of Dopamine Neurons
Kobayashi, S., & Schultz, W. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1600-08.2008 · Cited by 431 · see citing works
Neuhoff, H., Neu, A. D., Liss, B., & Roeper, J. (2002) · Journal of Neuroscience · doi:10.1523/jneurosci.22-04-01290.2002 · Cited by 400 · see citing works
Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey
Schultz, W. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.5.1439 · Cited by 390 · cited 1× · see citing works
“One limitation in the above electrophysiological studies is that the identification of dopamine neurons was based on indirect physiological properties such as wide spike waveforms (Grace & Bunney 1983, Schultz 1986, Ungless & Grace 2012).”
Hong, S., Jhou, T. C., Smith, M., Saleem, K. S., & Hikosaka, O. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1384-11.2011 · Cited by 390 · see citing works
Arithmetic and local circuitry underlying dopamine prediction errors
Eshel, N., Bukwich, M., Rao, V., Hemmelder, V., Tian, J., & Uchida, N. (2015) · Nature · doi:10.1038/nature14855 · Cited by 386 · see citing works
Self-citationInfluence of Reward Expectation on Behavior-Related Neuronal Activity in Primate Striatum
Hollerman, J. R., Tremblay, L., & Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.2.947 · Cited by 380 · see citing works
Afferents of the ventral tegmental area in the rat-anatomical substratum for integrative functions
Geisler, S., & Zahm, D. S. (2005) · The Journal of Comparative Neurology · doi:10.1002/cne.20668 · Cited by 379 · see citing works
Prediction, perception and agency.
Friston, K. (2012) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2011.11.014 · Cited by 359 · see citing works
Lima, S. Q., Hromádka, T., Znamenskiy, P., & Zador, A. M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006099 · Cited by 356 · see citing works
Effects of Spatial Attention on Contrast Response Functions in Macaque Area V4
Williford, T., & Maunsell, J. H. R. (2006) · Journal of Neurophysiology · doi:10.1152/jn.01207.2005 · Cited by 353 · see citing works
Real-time chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli
Roitman, M. F., Wheeler, R. A., Wightman, R. M., & Carelli, R. M. (2008) · Nature Neuroscience · doi:10.1038/nn.2219 · Cited by 343 · see citing works
Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target
Parker, N. F., Cameron, C. M., Taliaferro, J. P., Lee, J., Choi, J. Y., Davidson, T. J., Daw, N. D., & Witten, I. B. (2016) · Nature Neuroscience · doi:10.1038/nn.4287 · Cited by 340 · see citing works
Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass
Menegas, W., Bergan, J. F., Ogawa, S. K., Isogai, Y., Venkataraju, K. U., Osten, P., Uchida, N., & Watabe‐Uchida, M. (2015) · eLife · doi:10.7554/elife.10032 · Cited by 324 · see citing works
Self-citationThe temporal precision of reward prediction in dopamine neurons
Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008) · Nature Neuroscience · doi:10.1038/nn.2159 · Cited by 310 · see citing works
Miyamichi, K., Shlomai, Y., Shu, M., Weissbourd, B., Luo, L., & Mizrahi, A. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.027 · Cited by 309 · see citing works
Reward-Predictive Cues Enhance Excitatory Synaptic Strength onto Midbrain Dopamine Neurons
Stuber, G. D., Klanker, M., de Ridder, B., Bowers, M. S., Joosten, R. N. J. M. A., Feenstra, M. G. P., & Bonci, A. (2008) · Science · doi:10.1126/science.1160873 · Cited by 307 · see citing works
Dopamine neurons share common response function for reward prediction error
Eshel, N., Tian, J., Bukwich, M., & Uchida, N. (2016) · Nature Neuroscience · doi:10.1038/nn.4239 · Cited by 306 · see citing works
Self-citationBrown, J. W., Bullock, D. H., & Grossberg, S. (1999) · Journal of Neuroscience · doi:10.1523/jneurosci.19-23-10502.1999 · Cited by 299 · see citing works
Barrages of Synaptic Activity Control the Gain and Sensitivity of Cortical Neurons
Shu, Y., Hasenstaub, A., Badoual, M., Bal, T., & McCormick, D. A. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10388.2003 · Cited by 298 · see citing works
Hart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.2489-13.2014 · Cited by 294 · see citing works
Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · see citing works
Joshua, M., Adler, A., Mitelman, R., Vaadia, E., & Bergman, H. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3839-08.2008 · Cited by 273 · see citing works
Oleson, E. B., Gentry, R. N., Chioma, V. C., & Cheer, J. F. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3087-12.2012 · Cited by 272 · see citing works
Dopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257 · see citing works
Dopamine Neuron-Specific Optogenetic Stimulation in Rhesus Macaques
Stauffer, W. R., Lak, A., Yang, A., Borel, M., Paulsen, O., Boyden, E. S., & Schultz, W. (2016) · Cell · doi:10.1016/j.cell.2016.08.024 · Cited by 256 · see citing works
Burton, M. J., Rolls, E. T., & Mora, F. (1976) · Experimental Neurology · doi:10.1016/0014-4886(76)90189-8 · Cited by 252 · see citing works
Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates
Holt, G. R., & Koch, C. (1997) · Neural Computation · doi:10.1162/neco.1997.9.5.1001 · Cited by 250 · see citing works
Representation of Spatial Goals in Rat Orbitofrontal Cortex
Feierstein, C. E., Quirk, M. C., Uchida, N., Sosulski, D. L., & Mainen, Z. F. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.032 · Cited by 248 · see citing works
Self-citationDistributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons
Tian, J., Huang, R., Cohen, J. Y., Osakada, F., Kobak, D., Machens, C. K., Callaway, E. M., Uchida, N., & Watabe-Uchida, M. (2016) · Neuron · doi:10.1016/j.neuron.2016.08.018 · Cited by 224 · see citing works
Self-citationA basal ganglia circuit for evaluating action outcomes
Stephenson‐Jones, M., Yu, K., Ahrens, S., Tucciarone, J., van Huijstee, A. N., Mejia, L. A., Penzo, M. A., Tai, L.-H., Wilbrecht, L., & Li, B. (2016) · Nature · doi:10.1038/nature19845 · Cited by 217 · see citing works
Normalization for Sparse Encoding of Odors by a Wide-Field Interneuron
Papadopoulou, M., Cassenaer, S., Nowotny, T., & Laurent, G. (2011) · Science · doi:10.1126/science.1201835 · Cited by 214 · see citing works
Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors
Chang, C. Y., Esber, G. R., Marrero-Garcia, Y., Yau, H., Bonci, A., & Schoenbaum, G. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4191 · Cited by 211 · see citing works
Two Dimensions of Value: Dopamine Neurons Represent Reward But Not Aversiveness
Fiorillo, C. D. (2013) · Science · doi:10.1126/science.1238699 · Cited by 203 · see citing works
Haber, S. N., Ryoo, H., Cox, C., & Lu, W. (1995) · The Journal of Comparative Neurology · doi:10.1002/cne.903620308 · Cited by 188 · see citing works
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
Bayer, H. M., Lau, B., & Glimcher, P. W. (2007) · Journal of Neurophysiology · doi:10.1152/jn.01140.2006 · Cited by 184 · see citing works
Parvalbumin-Expressing Interneurons Linearly Control Olfactory Bulb Output
Kato, H., Gillet, S. N., Peters, A. J., Isaacson, J. S., & Komiyama, T. (2013) · Neuron · doi:10.1016/j.neuron.2013.08.036 · Cited by 182 · see citing works
Volman, S. F., Lammel, S., Margolis, E. B., Kim, Y., Richard, J. M., Roitman, M. F., & Lobo, M. K. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3250-13.2013 · Cited by 168 · see citing works
Habenula Lesions Reveal that Multiple Mechanisms Underlie Dopamine Prediction Errors
Tian, J., & Uchida, N. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.028 · Cited by 165 · see citing works
Self-citationDopamine neurons encoding long-term memory of object value for habitual behavior
Kim, H. F., Ghazizadeh, A., & Hikosaka, O. (2015) · Cell · doi:10.1016/j.cell.2015.10.063 · Cited by 157 · see citing works
Danjo, T., Yoshimi, K., Funabiki, K., Yawata, S., & Nakanishi, S. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1404323111 · Cited by 153 · see citing works
PVLV: The Primary Value and Learned Value Pavlovian Learning Algorithm.
O’Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007) · Behavioral Neuroscience · doi:10.1037/0735-7044.121.1.31 · Cited by 144 · see citing works
Multiplicative Gain Changes Are Induced by Excitation or Inhibition Alone
Murphy, B. K., & Miller, K. D. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-31-10040.2003 · Cited by 144 · see citing works
Blaess, S., Bodea, G. O., Kabanova, A., Chanet, S., Mugniery, E., Derouiche, A., Stephen, D., & Joyner, A. L. (2011) · Neural Development · doi:10.1186/1749-8104-6-29 · Cited by 133 · see citing works
Wenzel, J. M., Rauscher, N. A., Cheer, J. F., & Oleson, E. B. (2015) · ACS Chemical Neuroscience · doi:10.1021/cn500255p · Cited by 131 · see citing works
Takahashi, Y. K., Langdon, A., Niv, Y., & Schoenbaum, G. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.05.015 · Cited by 114 · see citing works
Midbrain dopamine neurons signal aversion in a reward-context-dependent manner
Matsumoto, H., Tian, J., Uchida, N., & Watabe‐Uchida, M. (2016) · eLife · doi:10.7554/elife.17328 · Cited by 109 · see citing works
Self-citationNeural mechanisms of acquired phasic dopamine responses in learning
Hazy, T. E., Frank, M. J., & O’Reilly, R. C. (2009) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2009.11.019 · Cited by 103 · see citing works
A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior
Contreras-Vidal, J. L., & Schultz, W. (1999) · Journal of Computational Neuroscience · doi:10.1023/a:1008862904946 · Cited by 102 · see citing works
Chapter 15 Reward-related activity in the monkey striatum and substantia nigra
Schultz, W., Apicella, P., Ljungberg, T., Romo, R., & Scarnati, E. (1993) · Progress in brain research · doi:10.1016/s0079-6123(08)61349-7 · Cited by 99 · see citing works
Diversity and Homogeneity in Responses of Midbrain Dopamine Neurons
Fiorillo, C. D., Yun, S. R., & Song, M. R. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3886-12.2013 · Cited by 99 · see citing works
Cardin, J. A., Palmer, L. A., & Contreras, D. (2008) · Neuron · doi:10.1016/j.neuron.2008.05.002 · Cited by 90 · see citing works
Synchronization of Midbrain Dopaminergic Neurons Is Enhanced by Rewarding Events
Joshua, M., Adler, A., Prut, Y., Vaadia, E., Wickens, J. R., & Bergman, H. (2009) · Neuron · doi:10.1016/j.neuron.2009.04.026 · Cited by 87 · see citing works
Dopamine neurons learn to encode the long-term value of multiple future rewards
Enomoto, K., Matsumoto, N., Nakai, S., Satoh, T., Sato, T. K., Ueda, Y., Inokawa, H., Haruno, M., & Kimura, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014457108 · Cited by 83 · see citing works
Efficient reinforcement learning: computational theories, neuroscience and robotics
Kawato, M., & Samejima, K. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.03.004 · Cited by 80 · see citing works
Electrical Synapses between Dopaminergic Neurons of the Substantia Nigra Pars Compacta
Vandecasteele, M., Glowinski, J., & Venance, L. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.4167-04.2005 · Cited by 75 · see citing works
Gain Modulation of Neuronal Responses by Subtractive and Divisive Mechanisms of Inhibition
Ayaz, A., & Chance, F. S. (2008) · Journal of Neurophysiology · doi:10.1152/jn.90547.2008 · Cited by 72 · see citing works
Dopamine Ramps Are a Consequence of Reward Prediction Errors
Gershman, S. J. (2013) · Neural Computation · doi:10.1162/neco_a_00559 · Cited by 71 · see citing works
Reward Contexts Extend Dopamine Signals to Unrewarded Stimuli
Kobayashi, S., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2013.10.061 · Cited by 63 · see citing works
Models of Information processing in the Basal Ganglia
Mitchell, I. (1996) · Journal of Chemical Neuroanatomy · doi:10.1016/0891-0618(96)00155-x · Cited by 63 · see citing works
Coordinated Activity of Ventral Tegmental Neurons Adapts to Appetitive and Aversive Learning
Kim, Y., Wood, J. T., & Moghaddam, B. (2012) · PLoS ONE · doi:10.1371/journal.pone.0029766 · Cited by 59 · see citing works
Morita, K., Morishima, M., Sakai, K., & Kawaguchi, Y. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.4614-12.2013 · Cited by 57 · see citing works
Feeding and diurnal related activity of lateral hypothalamic neurons in freely behaving rats
Ono, T., Sasaki, K., Nishino, H., et al. (1986) · Brain Research · doi:10.1016/0006-8993(86)90319-7 · Cited by 40 · see citing works
Tan, C. O., & Bullock, D. H. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.0259-08.2008 · Cited by 40 · see citing works
Vitay, J., & Hamker, F. H. (2014) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2014.00004 · Cited by 27 · see citing works
Neural activities in the monkey basal ganglia related to attention, memory and anticipation
Hikosaka, O., & Sakamoto, M. (1986) · Brain and Development · doi:10.1016/s0387-7604(86)80069-9 · Cited by 24 · see citing works
Uchida, N., Eshel, N., & Watabe‐Uchida, M. (2013) · Neuron · doi:10.1016/j.neuron.2013.11.013 · Cited by 12 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Kamin L (1969). Selective association and conditioning. In Fundamental Issues in Associative Learning, pp. 42– 64.
Cited by
Sorted by citations
The basal ganglia and the cerebellum: nodes in an integrated network
Bostan, A. C., & Strick, P. L. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0002-7 · Cited by 786 · see citing works
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667 · see citing works
Dopamine and Cognitive Control in Prefrontal Cortex
Ott, T., & Nieder, A. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.006 · Cited by 530 · see citing works
The connectome of an insect brain
Winding, M., Pedigo, B. D., Barnes, C. L., Patsolic, H. G., Park, Y., Kazimiers, T., Fushiki, A., Andrade, I. V., Khandelwal, A., Valdes-Aleman, J., et al. (2023) · Science · doi:10.1126/science.add9330 · Cited by 435 · see citing works
Yang, H., de Jong, J. W., Tak, Y., Peck, J., Bateup, H. S., & Lammel, S. (2018) · Neuron · doi:10.1016/j.neuron.2017.12.022 · Cited by 407 · see citing works
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269 · see citing works
What, If, and When to Move: Basal Ganglia Circuits and Self-Paced Action Initiation
Klaus, A., Alves da Silva, J., & Costa, R. M. (2019) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031033 · Cited by 284 · see citing works
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236 · see citing works
Self-citationDisrupted Dorsal Mid-Insula Activation During Interoception Across Psychiatric Disorders
Nord, C. L., Lawson, R. P., & Dalgleish, T. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20091340 · Cited by 171 · see citing works
Meta-analysis of human prediction error for incentives, perception, cognition, and action
Corlett, P. R., Mollick, J. A., & Kober, H. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-021-01264-3 · Cited by 135 · see citing works
Belief state representation in the dopamine system
Babayan, B. M., Uchida, N., & Gershman, S. J. (2018) · Nature Communications · doi:10.1038/s41467-018-04397-0 · Cited by 101 · see citing works
Self-citationUnderstanding visual hallucinations: A new synthesis
Collerton, D., Barnes, J., Diederich, N. J., Dudley, R., ffytche, D., Friston, K., Goetz, C. G., Goldman, J. G., Jardri, R., Kulisevsky, J., et al. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105208 · Cited by 83 · see citing works
Dopamine promotes head direction plasticity during orienting movements
Fisher, Y. E., Marquis, M., D’Alessandro, I., & Wilson, R. I. (2022) · Nature · doi:10.1038/s41586-022-05485-4 · Cited by 67 · see citing works
Plasticity of synapses and reward circuit function in the genesis and treatment of depression
Thompson, S. M. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-022-01422-1 · Cited by 40 · see citing works
Neural networks for navigation: From connections to computations
Wilson, R. I. (2023) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-110920-032645 · Cited by 38 · see citing works
Shih, H.-C., Kuo, M.-E., Wu, C., Chao, Y.-P., Huang, H.-W., & Huang, C.-M. (2022) · Brain Sciences · doi:10.3390/brainsci12070830 · Cited by 32 · see citing works
Ventral striatum dopamine release encodes unique properties of visual stimuli in mice
Gonzalez, L. S., Fisher, A. A., D'Souza, S. P., Cotella, E. M., Lang, R. A., & Robinson, J. E. (2023) · eLife · doi:10.7554/elife.85064 · Cited by 18 · see citing works
The Functions of Dopamine in Operant Conditioned Reflexes
Maiorov, V. I. (2019) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-019-00815-y · Cited by 10 · see citing works
Suicide prevention and ketamine: insights from computational modeling
Charlton, C. E., Karvelis, P., McIntyre, R. S., & Diaconescu, A. O. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1214018 · Cited by 7 · see citing works
Rethinking Predictive Processing
Furutachi, S., & Hofer, S. B. (2026) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-102124-031410 · Cited by 3 · see citing works
Tsuruga, R., Tajika, Y., Minami, M., & Tsutsui-Kimura, I. (2026) · Communications Biology · doi:10.1038/s42003-026-10485-5 · Cited by 2 · see citing works
Reward magnitude determines reinforcement learning efficiency
Gong, S., Martell, A., Dudman, J. T., & Coddington, L. T. (2026) · Science · doi:10.1126/science.aeb0813 · Cited by 2 · see citing works
- Authors
- Watabe-Uchida, M., Eshel, N., & Uchida, N.
- Affiliations
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, Massachusetts 02138;,
- Harvard University (US)
- Venue
- Annual Review of Neuroscience
- Publisher
- Annual Reviews
- Published
- 2017-07-25
- Volume
- 40
- Issue
- 1
- Pages
- 373-394
- ISSN
- 0147-006X (ISSN-L)
- 1545-4126
- PMC
- PMC6721851
- OpenAlex
- W2609307799
- Cited by
- 437
- References
- 127
- Citations by year
- 2026: 32 · 2025: 63 · 2024: 67 · 2023: 69 · 2022: 42 · 2021: 51 · 2020: 43 · 2019: 41 · 2018: 23 · 2017: 5 · 2016: 1
- FWCI
- 3.76
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Annual Review of Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH110404, F30MH100729)
- National Institute of General Medical Sciences (T32GM007753)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- reward prediction error, dopaminergic neurons, neural circuits, learning, recurrent neural networks, computational modeling, electrophysiology, human anatomy
- MeSH terms
- Animals, Brain, Dopamine, Humans, Learning, Nerve Net, Neural Pathways, Reward
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
