A common mechanism for adaptive scaling of reward and novelty
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Declarative memory is remarkably adaptive in the way it maintains sensitivity to relative novelty in both unknown and highly familiar environments. However, the neural mechanisms underlying this contextual adaptation are poorly understood. On the basis of emerging links between novelty processing and reinforcement learning mechanisms, we hypothesized that responses to novelty will be adaptively scaled according to expected contextual probabilities of new and familiar events, in the same way that responses to prediction errors for rewards are scaled according to their expected range. Using functional magnetic resonance imaging in humans, we show that the influence of novelty and reward on memory formation in an incidental memory task is adaptively scaled and furthermore that the BOLD signal in orbital prefrontal and medial temporal cortices exhibits concomitant scaled adaptive coding. These findings demonstrate a new mechanism for adjusting gain and sensitivity in declarative memory in accordance with contextual probabilities and expectancies of future events. Hum Brain Mapp, 2010. © 2010 Wiley‐Liss, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Tulving, E. (1985) · Canadian Psychology / Psychologie canadienne · doi:10.1037/h0080017 · Cited by 4,638
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Mesulam, M. (1998) · Brain · doi:10.1093/brain/121.6.1013 · Cited by 3,072 · cited 1×
“Novelty is a fundamental signal associated with attracting attention, promoting memory encoding and modifying goal-directed behavior [Knight, 1996; Lisman and Grace, 2005; Mesulam, 1998; Sokolov, 1963].”
BackgroundThe Medial Temporal Lobe and Recognition Memory
Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094328 · Cited by 2,580
The hippocampal-VTA loop: controlling the entry of information into long-term memory
Lisman, J. E., & Grace, A. A. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.002 · Cited by 1,983
The Critical Importance of Retrieval for Learning
Karpicke, J. D., & Roediger, H. L. (2008) · Science · doi:10.1126/science.1152408 · Cited by 1,791 · cited 2×
“…2005], and behavioral evidence for this suggestion is the so-called ‘‘testing’’ phenomenon: this refers to the observation that the mere retrieval of memories is an efficient mechanism to improve episodic memory [Buckner et al., 2001; Chan and McDermott, 2007; Karpicke and Roediger, 2008].”
BackgroundBehavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637 · cited 1×
“…the prefrontal cortex, hippocampus, and substantia nigra/ventral tegmental area (SN/VTA) have all been implicated in both novelty processing and certain forms of reinforcement learning [Duzel et al., 2009; Lisman and Grace, 2005; Ljungberg et al., 1992; Montague et al., 2004; Schultz, 1998, 2006].”
BackgroundIkemoto, S. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.05.004 · Cited by 1,477
Reward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426 · cited 2×
“The results of Experiment I reveal that components of the human reward processing circuitry, specifically the nucleus accumbens, mOFC, and mPFC [O’Doherty, 2004], show characteristic features of adaptation previously described in the activity of DA neurons as measured in nonhuman primates [Tobler…”
BackgroundRelative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314
Adaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · cited 24×
“To test the hypothesis that novelty and reward both express contextual adaptive scaling, we conducted two fMRI studies in healthy young adults using an adaptive reward coding paradigm adapted from a study in nonhuman primates [Tobler et al., 2005].”
BackgroundMethodResultInfluentialNeural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076
Self-citationNeural mechanisms for visual memory and their role in attention
Desimone, R. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.24.13494 · Cited by 1,028 · cited 2×
“stimuli are widely observed to decrease monotonically with the number of previous exposures, as in repetition suppression [Desimone, 1996; Miller and Desimone, 1994].”
BackgroundHigher nervous functions: the orienting reflex
Sokolov, E. N. (1963) · Annual Review of Physiology · doi:10.1146/annurev.ph.25.030163.002553 · Cited by 1,026
Responses of monkey dopamine neurons during learning of behavioral reactions
Ljungberg, T., Apicella, P., & Schultz, W. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.67.1.145 · Cited by 987
Reward-motivated learning: mesolimbic activation precedes memory formation
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. E. (2006) · Neuron · doi:10.1016/j.neuron.2006.03.036 · Cited by 977
Computational roles for dopamine in behavioural control
Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004) · Nature · doi:10.1038/nature03015 · Cited by 966
Optimized EPI for fMRI studies of the orbitofrontal cortex
Deichmann, R., Gottfried, J. A., Hutton, C., & Turner, R. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00073-9 · Cited by 821
Separating the brain regions involved in recollection and familiarity in recognition memory
Yonelinas, A. P., Otten, L. J., Shaw, K. N., & Rugg, M. D. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.5295-04.2005 · Cited by 813
Functional magnetic resonance imaging of reward prediction
Knutson, B., & Cooper, J. C. (2005) · Current Opinion in Neurology · doi:10.1097/01.wco.0000173463.24758.f6 · Cited by 736 · cited 3×
“, 2005], as well as by cues predicting their occurrence [Knutson and Cooper, 2005; O’Doherty et al., 2002; Wittmann et al., 2005, 2007].”
BackgroundWittmann, B. C., Schott, B. H., Guderian, S., Frey, J. U., Heinze, H.-J., & Düzel, E. (2005) · Neuron · doi:10.1016/j.neuron.2005.01.010 · Cited by 698 · cited 5×
“, 2000; Weiner, 2003] and strengthens links between reinforcement learning and declarative long-term memory in humans [Adcock et al., 2006; Wittmann et al., 2005].”
BackgroundResultInfluentialSelf-citationSynaptic tagging: implications for late maintenance of hippocampal long-term potentiation
Frey, U., & Morris, R. G. M. (1998) · Trends in Neurosciences · doi:10.1016/s0166-2236(97)01189-2 · Cited by 689 · cited 2×
“While adaptive scaling of rewards has been suggested as allowing efficient coding of appetitive prediction errors [Barlow, 1961], we suggest that adaptive scaling of novelty arranges for efficient encoding into long-term memory of information, perhaps via the influence on longterm plasticity of DA release in the hippo…”
BackgroundDopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty
Li, S., Cullen, W. K., Anwyl, R., & Rowan, M. J. (2003) · Nature Neuroscience · doi:10.1038/nn1049 · Cited by 568 · cited 1×
“In addition, under such conditions, novelty may even act as a mechanism to enhance long-term memory for familiar items [Bunzeck and Duzel, 2006; Fenker et al., 2008; Li et al., 2003].”
BackgroundAbsolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA
Bunzeck, N., & Düzel, E. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.021 · Cited by 503
Self-citationHasselmo, M. E. (2005) · Hippocampus · doi:10.1002/hipo.20116 · Cited by 502 · cited 1×
“In fact, retrieval and encoding processes in the MTL are likely to be closely interrelated [Hasselmo, 2005], and behavioral evidence for this suggestion is the so-called ‘‘testing’’ phenomenon: this refers to the observation that the mere retrieval of memories is an efficient mechanism to improve…”
BackgroundMorris, R. G. M. (2006) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2006.04888.x · Cited by 398 · cited 2×
“While adaptive scaling of rewards has been suggested as allowing efficient coding of appetitive prediction errors [Barlow, 1961], we suggest that adaptive scaling of novelty arranges for efficient encoding into long-term memory of information, perhaps via the influence on longterm plasticity of DA release in the hippo…”
BackgroundActivity in human reward-sensitive brain areas is strongly context dependent
Nieuwenhuis, S., Heslenfeld, D. J., Alting von Geusau, N. J., Mars, R. B., Holroyd, C. B., & Yeung, N. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.043 · Cited by 289
Functional imaging of the human dopaminergic midbrain
Düzel, E., Bunzeck, N., Guitart-Masip, M., Wittmann, B., Schott, B. H., & Tobler, P. N. (2009) · Trends in Neurosciences · doi:10.1016/j.tins.2009.02.005 · Cited by 216
Self-citationThe testing effect in recognition memory: A dual process account.
Chan, J. C. K., & McDermott, K. B. (2007) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.33.2.431 · Cited by 175
Krebs, R. M., Schott, B. H., & Düzel, E. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2008.08.019 · Cited by 129
Novel scenes improve recollection and recall of words
Fenker, D. B., Frey, J. U., Schuetze, H., Heipertz, D., Heinze, H.-J., & Duzel, E. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20086 · Cited by 94 · cited 1×
“In addition, under such conditions, novelty may even act as a mechanism to enhance long-term memory for familiar items [Bunzeck and Duzel, 2006; Fenker et al., 2008; Li et al., 2003].”
Background
Cited by
Shohamy, D., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 707
Walsh, M. M., & Anderson, J. R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.008 · Cited by 544
A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP
Lisman, J., Grace, A. A., & Duzel, E. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.07.006 · Cited by 465
Self-citationA neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages
Sambrook, T. D., & Goslin, J. (2015) · Psychological Bulletin · doi:10.1037/bul0000006 · Cited by 411
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
Metadata
- Authors
- Bunzeck, N., Dayan, P., Dolan, R. J., & Duzel, E.
- Venue
- Human Brain Mapping
- Publisher
- Wiley
- Published
- 2010-08-17
- Volume
- 31
- Issue
- 9
- Pages
- 1380-1394
- ISSN
- 1065-9471 (ISSN-L)
- 1097-0193
- OpenAlex
- W1977135358
- Cited by
- 100
- References
- 32
- FWCI
- 4.71
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.