A common mechanism for adaptive scaling of reward and novelty
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Declarative memory is remarkably adaptive in the way it maintains sensitivity to relative novelty in both unknown and highly familiar environments. However, the neural mechanisms underlying this contextual adaptation are poorly understood. On the basis of emerging links between novelty processing and reinforcement learning mechanisms, we hypothesized that responses to novelty will be adaptively scaled according to expected contextual probabilities of new and familiar events, in the same way that responses to prediction errors for rewards are scaled according to their expected range. Using functional magnetic resonance imaging in humans, we show that the influence of novelty and reward on memory formation in an incidental memory task is adaptively scaled and furthermore that the BOLD signal in orbital prefrontal and medial temporal cortices exhibits concomitant scaled adaptive coding. These findings demonstrate a new mechanism for adjusting gain and sensitivity in declarative memory in accordance with contextual probabilities and expectancies of future events. Hum Brain Mapp, 2010. © 2010 Wiley‐Liss, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Tulving, E. (1985) · Canadian Psychology / Psychologie canadienne · doi:10.1037/h0080017 · Cited by 4,638
Predictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Mesulam, M. (1998) · Brain · doi:10.1093/brain/121.6.1013 · Cited by 3,072 · cited 1×
“Novelty is a fundamental signal associated with attracting attention, promoting memory encoding and modifying goal-directed behavior [Knight, 1996; Lisman and Grace, 2005; Mesulam, 1998; Sokolov, 1963].”
BackgroundThe Medial Temporal Lobe and Recognition Memory
Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094328 · Cited by 2,580
The hippocampal-VTA loop: controlling the entry of information into long-term memory
Lisman, J. E., & Grace, A. A. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.002 · Cited by 1,983
The Critical Importance of Retrieval for Learning
Karpicke, J. D., & Roediger, H. L. (2008) · Science · doi:10.1126/science.1152408 · Cited by 1,791 · cited 2×
“…2005], and behavioral evidence for this suggestion is the so-called ‘‘testing’’ phenomenon: this refers to the observation that the mere retrieval of memories is an efficient mechanism to improve episodic memory [Buckner et al., 2001; Chan and McDermott, 2007; Karpicke and Roediger, 2008].”
BackgroundBehavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637 · cited 1×
“…the prefrontal cortex, hippocampus, and substantia nigra/ventral tegmental area (SN/VTA) have all been implicated in both novelty processing and certain forms of reinforcement learning [Duzel et al., 2009; Lisman and Grace, 2005; Ljungberg et al., 1992; Montague et al., 2004; Schultz, 1998, 2006].”
BackgroundIkemoto, S. (2007) · Brain Research Reviews · doi:10.1016/j.brainresrev.2007.05.004 · Cited by 1,477
Reward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426 · cited 2×
“The results of Experiment I reveal that components of the human reward processing circuitry, specifically the nucleus accumbens, mOFC, and mPFC [O’Doherty, 2004], show characteristic features of adaptation previously described in the activity of DA neurons as measured in nonhuman primates [Tobler…”
Background
Cited by
Shohamy, D., & Adcock, R. A. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.08.002 · Cited by 707
Walsh, M. M., & Anderson, J. R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.05.008 · Cited by 544
A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP
Lisman, J., Grace, A. A., & Duzel, E. (2011) · Trends in Neurosciences · doi:10.1016/j.tins.2011.07.006 · Cited by 465
Self-citationA neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages
Sambrook, T. D., & Goslin, J. (2015) · Psychological Bulletin · doi:10.1037/bul0000006 · Cited by 411
Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning
Iglesias, S., Mathys, C., Brodersen, K. H., Kasper, L., Piccirelli, M., den Ouden, H. E. M., & Stephan, K. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.009 · Cited by 342
Dopamine modulates novelty seeking behavior during decision making.
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2014) · Behavioral Neuroscience · doi:10.1037/a0037128 · Cited by 263
Davidow, J. Y., Foerde, K., Galván, A., & Shohamy, D. (2016) · Neuron · doi:10.1016/j.neuron.2016.08.031 · Cited by 269
Computational approaches to psychiatry
Stephan, K. E., & Mathys, C. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2013.12.007 · Cited by 260
Limbrick-Oldfield, E. H., van Holst, R. J., & Clark, L. (2013) · NeuroImage: Clinical · doi:10.1016/j.nicl.2013.02.005 · Cited by 143
Oxytocin Modulates Neural Reactivity to Children's Faces as a Function of Social Salience
Wittfoth-Schardt, D., Gründing, J., Wittfoth, M., Lanfermann, H., Heinrichs, M., Domes, G., Buchheim, A., Gündel, H., & Waller, C. (2012) · Neuropsychopharmacology · doi:10.1038/npp.2012.47 · Cited by 139
“On the other hand, and in line with previous studies postulating a common neural mechanism of novelty and reward in the GP (Bunzeck et al, 2010; Guitart-Masip et al, 2010), we also observed an OT-related decrease of activation in the left GP, as well as the hippocampus and lateral temporal regions,…”
ResultDopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum
Diederen, K. M. J., Ziauddeen, H., Vestergaard, M. D., Spencer, T., Schultz, W., & Fletcher, P. C. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.1979-16.2016 · Cited by 110
Diederen, K. M. J., Spencer, T., Vestergaard, M. D., Fletcher, P. C., & Schultz, W. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.019 · Cited by 102
“Thresholding Adaptive coding effects are likely to be subtle as previous fMRI studies only reported results that were uncorrected for multiple comparisons (Bunzeck et al., 2010; Park et al., 2012).”
Aversive stimuli and loss in the mesocorticolimbic dopamine system
Brooks, A. M., & Berns, G. S. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.04.001 · Cited by 85
Behavioral specifications of reward-associated long-term memory enhancement in humans
Wittmann, B. C., Dolan, R. J., & Düzel, E. (2011) · Learning & Memory · doi:10.1101/lm.1996811 · Cited by 62
“Specifically, in a recent study reward-predicting cues could be followed by two possible reward outcomes (Bunzeck et al., 2010).”
InfluentialSelf-citation
Metadata
- Authors
- Bunzeck, N., Dayan, P., Dolan, R. J., & Duzel, E.
- Venue
- Human Brain Mapping
- Publisher
- Wiley
- Published
- 2010-08-17
- Volume
- 31
- Issue
- 9
- Pages
- 1380-1394
- ISSN
- 1065-9471 (ISSN-L)
- 1097-0193
- OpenAlex
- W1977135358
- Cited by
- 100
- References
- 32
- FWCI
- 4.71
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.