The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning
Open accessThe DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto's (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 33×
“Thus, this theory can account for evidence suggesting that humans and animals attend toward good predictors of an outcome (Mackintosh, 1975) while also maintaining attention toward cues which are uncertain predictors of an outcome (Pearce and Hall, 1980).”
BackgroundMethodResultInfluentialA theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733 · cited 17×
“Thus, this theory can account for evidence suggesting that humans and animals attend toward good predictors of an outcome (Mackintosh, 1975) while also maintaining attention toward cues which are uncertain predictors of an outcome (Pearce and Hall, 1980).”
BackgroundMethodResultInfluentialGetting formal with dopamine and reward
Schultz, W. (2002) · Neuron · doi:10.1016/s0896-6273(02)00967-4 · Cited by 2,574
Dopamine in motivational control: rewarding, aversive, and alerting
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010) · Neuron · doi:10.1016/j.neuron.2010.11.022 · Cited by 2,396 · cited 3×
“This is because different expected values can be assigned to a particular state that are capable of containing information beyond the discrete cue that predicts reward (Bromberg-Martin et al., 2010c; Hong and Hikosaka, 2011; Aitken et al., 2016; Cone et al., 2016).”
BackgroundDiscrete coding of reward probability and uncertainty by dopamine neurons
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003) · Science · doi:10.1126/science.1077349 · Cited by 2,085 · cited 2×
“Further, the magnitude of phasic activity correlates with the size of the unexpected reward (Hollerman and Schultz, 1998; Fiorillo et al., 2003; Roesch et al., 2007; Stauffer et al., 2014; Eshel et al., 2016) in a manner that reflects the value of the reward (Lak et al., 2014), value of the future…”
BackgroundContext and Behavioral Processes in Extinction: Table 1.
Bouton, M. E. (2004) · Learning & Memory · doi:10.1101/lm.78804 · Cited by 1,808 · cited 3×
“Thus, responding to the predictive cue will reemerge when the experimental circumstance no longer reflects that present in extinction (e.g., the passage of time or a physical change in context; Bouton, 2004).”
BackgroundInfluentialDopamine neuron systems in the brain: An update
Björklund, A., & Dunnett, S. B. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.006 · Cited by 1,777 · cited 2×
“Interestingly, VTA sends a particularly dense projection to the PL region (Bentivoglio and Morelli, 2005; Björklund and Dunnett, 2007).”
BackgroundSwanson, L. W. (1982) · Brain Research Bulletin · doi:10.1016/0361-9230(82)90145-9 · Cited by 1,718 · cited 2×
“A sub-nucleus of the amygdala complex, the BLA, is a region that receives extensive dopaminergic input from midbrain dopamine neurons (Swanson, 1982) and shows increases in neural activity when an unexpected event occurs whether it is rewarding or aversive (Belova et al., 2007, 2008; Herry et al.,…”
BackgroundToward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · cited 16×
“Until very recently evidence suggesting that phasic activity in midbrain dopamine neurons mimics the scalar prediction error described in Sutton and Barto (1981) has been largely, if not entirely, correlative (Schultz et al., 1997; Roesch et al., 2007; Niv and Schoenbaum, 2008; Iordanova, 2009;…”
BackgroundMethodResultInfluentialMedial Frontal Cortex Mediates Perceptual Attentional Set Shifting in the Rat
Birrell, J. M., & Brown, V. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-11-04320.2000 · Cited by 1,421 · cited 2×
“One of these odors predicted that the right well would be reinforced and the other predicted that the left well would be reinforced.”
BackgroundTwo types of dopamine neuron distinctly convey positive and negative motivational signals
Matsumoto, M., & Hikosaka, O. (2009) · Nature · doi:10.1038/nature08028 · Cited by 1,378 · cited 1×
“…populations of VTA dopamine neurons that appear to show distinct and complex interactions with wider neuronal systems where they contribute to behavior in diverse ways (Matsumoto and Hikosaka, 2007; Jhou et al., 2009; Lammel et al., 2011; Eban-Rothschild et al., 2016; Parker et al., 2016).”
BackgroundNeuron-type-specific signals for reward and punishment in the ventral tegmental area
Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012) · Nature · doi:10.1038/nature10754 · Cited by 1,318
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Bayer, H. M., & Glimcher, P. W. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.020 · Cited by 1,304 · cited 1×
“These neurons show a phasic increase in activity when an unexpected reward is delivered (Ljungberg et al., 1992; Mirenowicz and Schultz, 1994, 1996) or a reward is delivered that was better than expected (Bayer and Glimcher, 2005) (Figure 1A).”
BackgroundInput-specific control of reward and aversion in the ventral tegmental area
Lammel, S., Lim, B. K., Ran, C., Huang, K. W., Betley, M. J., Tye, K. M., Deisseroth, K., & Malenka, R. C. (2012) · Nature · doi:10.1038/nature11527 · Cited by 1,300 · cited 2×
“In parallel, research has shown that distinct populations of VTA dopamine neurons receive input from the laterodorsal tegmentum (LDT) and lateral habenula (LHb), argued to underlie these appetitive and aversive responses, respectively (Lammel et al., 2012).”
BackgroundLateral habenula as a source of negative reward signals in dopamine neurons
Matsumoto, M., & Hikosaka, O. (2007) · Nature · doi:10.1038/nature05860 · Cited by 1,292
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning
Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., de Lecea, L., & Deisseroth, K. (2009) · Science · doi:10.1126/science.1168878 · Cited by 1,266 · cited 1×
“…the combination of a temporally specific optogenetic approach in addition to the cell type specific transgenic rodent lines has made it easier to manipulate dopamine neurons in a causal manner (Margolis et al., 2006; Lammel et al., 2008; Tsai et al., 2009; Witten et al., 2011; Cohen et al., 2012).”
BackgroundFrontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
Ventral tegmental area: cellular heterogeneity, connectivity and behaviour
Morales, M., & Margolis, E. B. (2017) · Nature reviews. Neuroscience · doi:10.1038/nrn.2016.165 · Cited by 1,108
Schultz, W. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.03.007 · Cited by 1,028
Dopamine responses comply with basic assumptions of formal learning theory
Waelti, P., Dickinson, A., & Schultz, W. (2001) · Nature · doi:10.1038/35083500 · Cited by 1,018 · cited 4×
“In one example (Waelti et al., 2001), monkeys first received presentations of cue A paired with a juice reward.”
BackgroundMethodInfluentialResponses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system
Lammel, S., Hetzel, A., Häckel, O., Jones, I., Liss, B., & Roeper, J. (2008) · Neuron · doi:10.1016/j.neuron.2008.01.022 · Cited by 976
Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli
Mirenowicz, J., & Schultz, W. (1996) · Nature · doi:10.1038/379449a0 · Cited by 829
Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli
Lammel, S., Ion, D. I., Roeper, J., & Malenka, R. C. (2011) · Neuron · doi:10.1016/j.neuron.2011.03.025 · Cited by 766 · cited 4×
“Additionally, the firing of dopaminergic neurons in the midbrain is suppressed when an expected reward is omitted or is worse than expected (Tobler et al., 2003; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Lammel et al., 2011, 2014; Cohen et al., 2012).”
BackgroundInfluentialAssociative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710 · cited 2×
“Further, Daw et al. (2011) recently found evidence for information consistent with a model-based account of behavior in the ventral striatum, traditionally thought to receive a model-free prediction error from VTA dopamine neurons (Suaud-Chagny et al., 1992; Day et al., 2007).”
ResultFrom prediction error to incentive salience: mesolimbic computation of reward motivation.
Berridge, K. C. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.07990.x · Cited by 700 · cited 2×
“Such learning is typically referred to as ‘model-based’ and is not under the control of the Sutton and Barto (1981) error signal which relies on cached values drawn from direct experiences with cues and outcomes (Dickinson and Balleine, 2002; Berridge, 2012; Dayan and Berridge, 2014).”
BackgroundJhou, T. C., Fields, H. L., Baxter, M. G., Saper, C. B., & Holland, P. C. (2009) · Neuron · doi:10.1016/j.neuron.2009.02.001 · Cited by 696
Witten, I. B., Steinberg, E. E., Lee, S. Y., Davidson, T. J., Zalocusky, K. A., Brodsky, M., Yizhar, O., Cho, S. L., Gong, S., Ramakrishnan, C., et al. (2011) · Neuron · doi:10.1016/j.neuron.2011.10.028 · Cited by 680 · cited 1×
“…the combination of a temporally specific optogenetic approach in addition to the cell type specific transgenic rodent lines has made it easier to manipulate dopamine neurons in a causal manner (Margolis et al., 2006; Lammel et al., 2008; Tsai et al., 2009; Witten et al., 2011; Cohen et al., 2012).”
Amygdala circuitry in attentional and representational processes
Holland, P. C., & Gallagher, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01271-6 · Cited by 618
Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007) · Nature Neuroscience · doi:10.1038/nn2013 · Cited by 597
Self-citationVTA dopaminergic neurons regulate ethologically relevant sleep-wake behaviors
Eban-Rothschild, A., Rothschild, G., Giardino, W. J., Jones, J. R., & de Lecea, L. (2016) · Nature Neuroscience · doi:10.1038/nn.4377 · Cited by 583 · cited 2×
“…populations of VTA dopamine neurons that appear to show distinct and complex interactions with wider neuronal systems where they contribute to behavior in diverse ways (Matsumoto and Hikosaka, 2007; Jhou et al., 2009; Lammel et al., 2011; Eban-Rothschild et al., 2016; Parker et al., 2016).”
BackgroundImportance of unpredictability for reward responses in primate dopamine neurons
Mirenowicz, J., & Schultz, W. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.72.2.1024 · Cited by 569
Hare, T. A., Malmaud, J., & Rangel, A. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6383-10.2011 · Cited by 556 · cited 1×
“…would likely be fruitful in understanding attentional processing across species (see e.g., Hickey et al., 2006, 2011, 2015; Jovancevic et al., 2006; Hare et al., 2011; Hickey and Theeuwes, 2011; Lim et al., 2011; Gottlieb, 2012; Gottlieb et al., 2014; Theeuwes, 2013; Tommasi et al., 2015; Wilschut…”
BackgroundMidbrain dopamine neurons encode decisions for future action
Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006) · Nature Neuroscience · doi:10.1038/nn1743 · Cited by 503 · cited 1×
“…et al., 2007; Stauffer et al., 2014; Eshel et al., 2016) in a manner that reflects the value of the reward (Lak et al., 2014), value of the future action (Morris et al., 2006) or value of the choice (Roesch et al., 2007), as assessed by the agent’s approach behavior toward to reward-predictive cue.”
BackgroundElectrophysiological Evidence of the Capture of Visual Attention
Hickey, C., McDonald, J. J., & Theeuwes, J. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.4.604 · Cited by 475
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
Disrupted prediction-error signal in psychosis: Evidence for an associative account of delusions
Corlett, P. R., Murray, G. K., Honey, G. D., Aitken, M. R. F., Shanks, D. R., Robbins, T. W., Bullmore, E. T., Dickinson, A., & Fletcher, P. C. (2007) · Brain · doi:10.1093/brain/awm173 · Cited by 460
Divergent routing of positive and negative information from the amygdala during memory retrieval
Beyeler, A., Namburi, P., Glober, G. F., Simonnet, C., Calhoon, G. G., Conyers, G. F., Luck, R., Wildes, C. P., & Tye, K. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.004 · Cited by 412
Processing of Temporal Unpredictability in Human and Animal Amygdala
Herry, C., Bach, D. R., Esposito, F., Di Salle, F., Perrig, W. J., Scheffler, K., Lüthi, A., & Seifritz, E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5218-06.2007 · Cited by 401
Context, learning, and extinction
Gershman, S. J., Blei, D. M., & Niv, Y. (2010) · Psychological Review · doi:10.1037/a0017808 · Cited by 386 · cited 4×
“, contextual cues) available on that trial (Nakahara et al., 2004; Redish et al., 2007; Gershman et al., 2010; Nakahara and Hikosaka, 2012; Nakahara, 2014), basically anything that has been directly experienced as associated with reward in the past.”
BackgroundInfluentialModel-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357 · cited 2×
“Such learning is typically referred to as ‘model-based’ and is not under the control of the Sutton and Barto (1981) error signal which relies on cached values drawn from direct experiences with cues and outcomes (Dickinson and Balleine, 2002; Berridge, 2012; Dayan and Berridge, 2014).”
BackgroundExpectation modulates neural responses to pleasant and aversive stimuli in primate amygdala
Belova, M. A., Paton, J. J., Morrison, S. E., & Salzman, C. D. (2007) · Neuron · doi:10.1016/j.neuron.2007.08.004 · Cited by 349
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · cited 2×
“While the mechanism by which burst firing in VTA dopamine neurons influence attentional processing in PL cortex remains to be clarified, the ability of phasic responses to influence activity in PL cortex suggest that prediction errors in VTA may influence activity in the PL cortex to produce an…”
BackgroundAttention and associative learning in humans: An integrative review.
Le Pelley, M. E., Mitchell, C. J., Beesley, T., George, D. N., & Wills, A. J. (2016) · Psychological Bulletin · doi:10.1037/bul0000064 · Cited by 329
Lim, S.-L., O'Doherty, J. P., & Rangel, A. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1246-11.2011 · Cited by 329 · cited 1×
“…processing across species (see e.g., Hickey et al., 2006, 2011, 2015; Jovancevic et al., 2006; Hare et al., 2011; Hickey and Theeuwes, 2011; Lim et al., 2011; Gottlieb, 2012; Gottlieb et al., 2014; Theeuwes, 2013; Tommasi et al., 2015; Wilschut et al., 2015 for a more comprehensive review…”
BackgroundSuaud-Chagny, M. F., Chergui, K., Chouvet, G., & Gonon, F. (1992) · Neuroscience · doi:10.1016/0306-4522(92)90076-e · Cited by 307
Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm
Tobler, P. N., Dickinson, A., & Schultz, W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-32-10402.2003 · Cited by 300 · cited 1×
“Additionally, the firing of dopaminergic neurons in the midbrain is suppressed when an expected reward is omitted or is worse than expected (Tobler et al., 2003; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Lammel et al., 2011, 2014; Cohen et al., 2012).”
BackgroundExpectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · cited 5×
“Thus, this information could be relayed in a top-down manner to VTA to modulate these phasic signals according to this word view (Daw et al., 2011; Takahashi et al., 2011, 2016; O’Doherty et al., 2017).”
BackgroundInfluentialSelf-citationDopamine prediction error responses integrate subjective value from different reward dimensions
Lak, A., Stauffer, W. R., & Schultz, W. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1321596111 · Cited by 257 · cited 1×
“…1998; Fiorillo et al., 2003; Roesch et al., 2007; Stauffer et al., 2014; Eshel et al., 2016) in a manner that reflects the value of the reward (Lak et al., 2014), value of the future action (Morris et al., 2006) or value of the choice (Roesch et al., 2007), as assessed by the agent’s approach…”
BackgroundAttention, learning, and the value of information
Gottlieb, J. (2012) · Neuron · doi:10.1016/j.neuron.2012.09.034 · Cited by 241 · cited 1×
“…across species (see e.g., Hickey et al., 2006, 2011, 2015; Jovancevic et al., 2006; Hare et al., 2011; Hickey and Theeuwes, 2011; Lim et al., 2011; Gottlieb, 2012; Gottlieb et al., 2014; Theeuwes, 2013; Tommasi et al., 2015; Wilschut et al., 2015 for a more comprehensive review on these…”
BackgroundMoment-to-moment tracking of state value in the amygdala
Belova, M. A., Paton, J. J., & Salzman, C. D. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1400-08.2008 · Cited by 221
Dopamine Reward Prediction Error Responses Reflect Marginal Utility
Stauffer, W. R., Lak, A., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.064 · Cited by 204
Attention, Reward, and Information Seeking
Gottlieb, J., Hayhoe, M., Hikosaka, O., & Rangel, A. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3270-14.2014 · Cited by 168 · cited 1×
“…(see e.g., Hickey et al., 2006, 2011, 2015; Jovancevic et al., 2006; Hare et al., 2011; Hickey and Theeuwes, 2011; Lim et al., 2011; Gottlieb, 2012; Gottlieb et al., 2014; Theeuwes, 2013; Tommasi et al., 2015; Wilschut et al., 2015 for a more comprehensive review on these attentional theories).”
BackgroundGradual extinction prevents the return of fear: implications for the discovery of state
Gershman, S. J., Jones, C. E., Norman, K. A., Monfils, M.-H., & Niv, Y. (2013) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2013.00164 · Cited by 160 · cited 2×
“Not only are dopamine prediction errors capable of reflecting state-specific associations, dopamine prediction errors are also theorized to contribute to the creation of new states which allow for the development of state-specific associations (Gershman et al., 2010; Gershman et al., 2013).”
BackgroundMore Is Less: A Disinhibited Prefrontal Cortex Impairs Cognitive Flexibility
Gruber, A. J., Calhoon, G. G., Shusterman, I., Schoenbaum, G., Roesch, M. R., & O'Donnell, P. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4623-10.2010 · Cited by 160
Self-citationReward guides attention to object categories in real-world scenes.
Hickey, C., Kaiser, D., & Peelen, M. V. (2015) · Journal of Experimental Psychology: General · doi:10.1037/a0038627 · Cited by 91 · cited 2×
“…it is likely that future integration of these fields would likely be fruitful in understanding attentional processing across species (see e.g., Hickey et al., 2006, 2011, 2015; Jovancevic et al., 2006; Hare et al., 2011; Hickey and Theeuwes, 2011; Lim et al., 2011; Gottlieb, 2012; Gottlieb et…”
BackgroundRisk-Responsive Orbitofrontal Neurons Track Acquired Salience
Ogawa, M., van der Meer, M. A. A., Esber, G. R., Cerri, D. H., Stalnaker, T. A., & Schoenbaum, G. (2013) · Neuron · doi:10.1016/j.neuron.2012.11.006 · Cited by 74 · cited 4×
“…from the SNc to CeA may function to produce overt behavioral changes in attention to influence rates of learning rather than modulating the rate at which a cue becomes associated with an outcome per se, which may be a point of difference from attentional processing which takes place in the BLA.”
BackgroundResultInfluentialSelf-citationReward Contexts Extend Dopamine Signals to Unrewarded Stimuli
Kobayashi, S., & Schultz, W. (2014) · Current Biology · doi:10.1016/j.cub.2013.10.061 · Cited by 63
The state of the orbitofrontal cortex
Sharpe, M. J., Wikenheiser, A. M., Niv, Y., & Schoenbaum, G. (2015) · Neuron · doi:10.1016/j.neuron.2015.12.004 · Cited by 21
Self-citation
Cited by
Martins, D., Rademacher, L., Gabay, A. S., Taylor, R., Richey, J. A., Smith, D. V., Goerlich, K. S., Nawijn, L., Cremers, H. R., Wilson, R., et al. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.034 · Cited by 119
Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101
The Memory Paradox: Why Our Brains Need Knowledge in an Age of AI
Oakley, B., Johnston, M., Chen, K., Jung, E., & Sejnowski, T. (2025) · doi:10.2139/ssrn.5250447 · Cited by 31
The Functions of Dopamine in Operant Conditioned Reflexes
Maiorov, V. I. (2019) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-019-00815-y · Cited by 10
“The rate of acquisition of a Pavlovian conditioned refl ex has been proposed to depend directly on the magnitude of the prediction error of the unconditioned stimulus (changing from a maximum at the fi rst unexpected presentation of the unconditioned stimulus to a minimum on acquisition of the conditioned refl ex) [Re…”
Metadata
- Authors
- Nasser, H. M., Calu, D. J., Schoenbaum, G., & Sharpe, M. J.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2017-02-22
- Volume
- 8
- Pages
- 244-244
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2591012103
- Cited by
- 86
- References
- 59
- FWCI
- 4.23
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.