Reward-Related Learning via Multiple Memory Systems
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
The application of a neuroeconomic approach to the study of reward-related processes has provided significant insights in our understanding of human learning and decision-making. Much of this research has primarily focused on the contributions of the cortico-striatal circuitry, involved in trial and error reward learning. As a result, less consideration has been allotted to the potential influence of different neural mechanisms such as the hippocampus, or to more common ways in human society in which information is acquired and utilized to reach a decision, such as through explicit instruction rather than trial and error learning. This review examines the basic and applied value of examining the individual contributions of multiple learning and memory neural systems and their interactions during human decision-making in normal individuals and neuropsychiatric populations. Specifically, the anatomical and functional connectivity across multiple memory systems are highlighted to suggest that probing the role of the hippocampus and its interactions with the cortico-striatal circuitry via the application of model-based neuroeconomic approaches may provide novel insights into several neuropsychiatric populations who suffer from damage to one of these structures, and as a consequence have deficits in learning, memory, or decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Trust, Reciprocity, and Social History
Berg, J., Dickhaut, J., & McCabe, K. (1995) · Games and Economic Behavior · doi:10.1006/game.1995.1027 · Cited by 5,548
The reward circuit: linking primate anatomy and human imaging
Haber, S. N., & Knutson, B. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2009.129 · Cited by 3,718
Contributions of the Amygdala to Emotion Processing: From Animal Models to Human Behavior
Phelps, E. A., & LeDoux, J. E. (2005) · Neuron · doi:10.1016/j.neuron.2005.09.025 · Cited by 3,203 · cited 1×
“The amygdala is a critical structure for aversive learning (126) and emotional memory (127) that has implications for decision-making in particular contexts, such as when decisions involve ambiguity (128) or are framed as gains or losses (129).”
BackgroundSeparate neural systems value immediate and delayed monetary rewards
McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004) · Science · doi:10.1126/science.1100907 · Cited by 3,089 · cited 1×
“Deliberative processes have been posited to involve prefrontal and parietal cortices, while automatic processes have been linked with subcortical regions such as the striatum ((4, 119); although see (116)).”
BackgroundThe role of the basal ganglia in habit formation
Yin, H. H., & Knowlton, B. J. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1919 · Cited by 2,402
The human hippocampus and spatial and episodic memory
Burgess, N., Maguire, E. A., & O'Keefe, J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00830-9 · Cited by 2,401
THE AMYGDALA MODULATES THE CONSOLIDATION OF MEMORIES OF EMOTIONALLY AROUSING EXPERIENCES
McGaugh, J. L. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144157 · Cited by 2,296
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-16-j0002.2001 · Cited by 2,244
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · cited 5×
“A opular method budding from cognitive and computational neuoscience research (40,41) that is rooted in classical learning theoies (42) is the application and characterization of model-based earning signals (9).”
BackgroundMethodResultInfluentialThe human orbitofrontal cortex: linking reward to hedonic experience
Kringelbach, M. L. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1747 · Cited by 2,180 · cited 1×
“As this review primarily focused on the potential contributions of learning signals associated with the BG and MTL to decision-making, less discussion was attributed to other important regions that contribute to learning and decision-making, namely the amygdala and the prefrontal cortex (124, 125).”
BackgroundDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 4×
“Functional dissoiations between subdivisions of the striatum have also been proosed, with the dorsomedial striatum involved in processing ction-contingency (43,51), goal-directed learning (52), and instruental conditioning (46) and the dorsolateral striatum linked with abit and motor learning (49).”
BackgroundInfluentialThe hippocampal-VTA loop: controlling the entry of information into long-term memory
Lisman, J. E., & Grace, A. A. (2005) · Neuron · doi:10.1016/j.neuron.2005.05.002 · Cited by 1,983
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Frank, M. J., Seeberger, L. C., & O'Reilly, R. C. (2004) · Science · doi:10.1126/science.1102941 · Cited by 1,964
A neostriatal habit learning system in humans
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996) · Science · doi:10.1126/science.273.5280.1399 · Cited by 1,777
Swanson, L. W. (1982) · Brain Research Bulletin · doi:10.1016/0361-9230(82)90145-9 · Cited by 1,718 · cited 1×
“aminergic structures (90, 91) and an alternative, perhaps comple-”
BackgroundLearning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 2×
“Importantly, patients afflicted with PD can learn hrough different neural mechanisms, such as MTL structures inolved in declarative learning, highlighting PD as a candidate disrder for investigating learning via multiple memory systems (19).”
BackgroundDopamine-dependent prediction errors underpin reward-seeking behaviour in humans
Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature05051 · Cited by 1,584
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522 · cited 2×
“Adaptations of this game to neuroimaging settings involving repeated interactions evoke trial and error learning and suggest that reputation building is dependent on learning signals being processed in the striatum (104–106), with a shift in the magnitude of the BOLD signal in the striatum to the earliest prediction o…”
BackgroundReward representations and reward-related learning in the human brain: Insights from neuroimaging
O’Doherty, J. P. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.10.016 · Cited by 1,426 · cited 1×
“The striatum, in particular, has been inked to various types of learning involving procedural skills (47), abits (30,48,49), and reward learning (43,44,50).”
BackgroundNeural systems responding to degrees of uncertainty in human decision-making
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., & Camerer, C. F. (2005) · Science · doi:10.1126/science.1115327 · Cited by 1,411
The Role of the Dorsal Striatum in Reward and Decision-Making: Figure 1.
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1554-07.2007 · Cited by 1,390
Self-citationGetting to Know You: Reputation and Trust in a Two-Person Economic Exchange
King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., & Montague, P. R. (2005) · Science · doi:10.1126/science.1108062 · Cited by 1,345 · cited 1×
“Adaptations of this game to neuroimaging settings involving repeated interactions evoke trial and error learning and suggest that reputation building is dependent on learning signals being processed in the striatum (104–106), with a shift in the magnitude of the BOLD signal in the striatum to the earliest prediction o…”
Structure and function of declarative and nondeclarative memory systems
Squire, L. R., & Zola, S. M. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.24.13515 · Cited by 1,327 · cited 3×
“Over the past few decades, an increasingly supported theory uggests that memory is not a unitary process but rather consists of ultiple processes that engage distinct neural substrates (5,6,20).”
BackgroundDeclarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory
Squire, L. R. (1992) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1992.4.3.232 · Cited by 1,282 · cited 2×
“In contrast, nondeclarative memory encompasses several ypes of learning including habit learning, procedural, conditionng, and priming (20).”
BackgroundOxytocin shapes the neural circuitry of trust and trust adaptation in humans
Baumgartner, T., Heinrichs, M., Vonlanthen, A., Fischbacher, U., & Fehr, E. (2008) · Neuron · doi:10.1016/j.neuron.2008.04.009 · Cited by 1,258 · cited 1×
“In this situation, the additional information affects the belief about probability of reciprocation and decisions to trust (104, 105, 107).”
BackgroundPutting a spin on the dorsal-ventral divide of the striatum
Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.06.006 · Cited by 1,188
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Tracking the Hemodynamic Responses to Reward and Punishment in the Striatum
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.6.3072 · Cited by 1,116 · cited 1×
“The human striatum has been shown to be involved in various facets of reward-related processing that encompass affective learning and inform future decision making (9,44)—from anticipating or predicting a potential reward (58) to coding the differential response between positive and negative outcomes (59), particularl…”
BackgroundSelf-citationNeural Economics and the Biological Substrates of Valuation
Montague, P. R., & Berns, G. S. (2002) · Neuron · doi:10.1016/s0896-6273(02)00974-1 · Cited by 994
Reward-motivated learning: mesolimbic activation precedes memory formation
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. E. (2006) · Neuron · doi:10.1016/j.neuron.2006.03.036 · Cited by 977
Time Discounting for Primary Rewards
McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4246-06.2007 · Cited by 971
Distributed Neural Representation of Expected Value
Knutson, B., Taylor, J., Kaufman, M., Peterson, R., & Glover, G. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0642-05.2005 · Cited by 955
Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917 · cited 1×
“, classical and instrumental conditioning) where predictions and contingencies are typically acquired and updated through trial and error (46, 56, 57).”
BackgroundCools, R. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1136 · Cited by 906
‘Frontal' cognitive function in patients with Parkinson's disease ‘on' and ‘off' levodopa
GOTHAM, A. M., BROWN, R. G., & MARSDEN, C. D. (1988) · Brain · doi:10.1093/brain/111.2.299 · Cited by 835
A specific role for posterior dorsolateral striatum in human habit learning
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2009.06796.x · Cited by 769 · cited 1×
“The striatum, in particular, has been inked to various types of learning involving procedural skills (47), abits (30,48,49), and reward learning (43,44,50).”
BackgroundReward‐Related Responses in the Human Striatum
DELGADO, M. R. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1390.002 · Cited by 736
Self-citationPerceptions of moral character modulate the neural systems of reward during the trust game
Delgado, M. R., Frank, R. H., & Phelps, E. A. (2005) · Nature Neuroscience · doi:10.1038/nn1575 · Cited by 729 · cited 2×
“In this situation, the additional information affects the belief about probability of reciprocation and decisions to trust (104, 105, 107).”
BackgroundSelf-citationWhat's new with the amnesic patient H.M.?
Corkin, S. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn726 · Cited by 726
Wittmann, B. C., Schott, B. H., Guderian, S., Frey, J. U., Heinze, H.-J., & Düzel, E. (2005) · Neuron · doi:10.1016/j.neuron.2005.01.010 · Cited by 698 · cited 1×
“, those associated with a high reward value (79, 83, 86)).”
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
D'Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008) · Science · doi:10.1126/science.1150605 · Cited by 661
Pathological Gambling Caused by Drugs Used to Treat Parkinson Disease
Dodd, M. L., Klos, K. J., Bower, J. H., Geda, Y. E., Josephs, K. A., & Ahlskog, J. E. (2005) · Archives of Neurology · doi:10.1001/archneur.62.9.noc50009 · Cited by 563
Haruno, M., & Kawato, M. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00382.2005 · Cited by 433
Striatal activation during acquisition of a cognitive skill.
Poldrack, R. A., Prabhakaran, V., Seger, C. A., & Gabrieli, J. D. E. (1999) · Neuropsychology · doi:10.1037/0894-4105.13.4.564 · Cited by 428
Linking context with reward: a functional circuit from hippocampal CA3 to ventral tegmental area
Luo, A. H., Tahsili-Fahadan, P., Wise, R. A., Lupica, C. R., & Aston-Jones, G. (2011) · Science · doi:10.1126/science.1204622 · Cited by 425 · cited 1×
“etween different environmental contexts and reward (92) and the”
BackgroundMechanisms of cognitive set flexibility in Parkinson's disease
Cools, R. (2001) · Brain · doi:10.1093/brain/124.12.2503 · Cited by 406
Medication-Related Impulse Control and Repetitive Behaviors in Parkinson Disease
Voon, V., & Fox, S. H. (2007) · Archives of Neurology · doi:10.1001/archneur.64.8.1089 · Cited by 397
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Emotion, decision making, and the amygdala.
Seymour, B., & Dolan, R. (2008) · Neuron · doi:10.1016/j.neuron.2008.05.020 · Cited by 324 · cited 1×
“As this review primarily focused on the potential contributions of learning signals associated with the BG and MTL to decision-making, less discussion was attributed to other important regions that contribute to learning and decision-making, namely the amygdala and the prefrontal cortex (124, 125).”
BackgroundAmygdala Modulation of Multiple Memory Systems: Hippocampus and Caudate-Putamen
Packard, M. G., & Teather, L. A. (1998) · Neurobiology of Learning and Memory · doi:10.1006/nlme.1997.3815 · Cited by 287
Dissociating Hippocampal versus Basal Ganglia Contributions to Learning and Transfer
Myers, C. E., Shohamy, D., Gluck, M. A., Grossman, S., Kluger, A., Ferris, S., Golomb, J., Schnirman, G., & Schwartz, R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321208123 · Cited by 243 · cited 3×
“uring an associative learning task (31), highlighting the role these”
BackgroundBalleine, B. W., & Dickinson, A. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-23-08954.2000 · Cited by 218 · cited 1×
“Another influential model highlights goal-directed and habitual learning as two key systems involved in guiding choice behavior (115, 120), with goal-directed learning involving the anterior caudate nucleus and medial prefrontal regions, and habitual learning associated with the posterior putamen (48, 49, 115).”
BackgroundRole of the Basal Ganglia in Category Learning: How Do Patients With Parkinson's Disease Learn?
Shohamy, D., Myers, C. E., Onlaor, S., & Gluck, M. A. (2004) · Behavioral Neuroscience · doi:10.1037/0735-7044.118.4.676 · Cited by 186
Cited by
How musical training affects cognitive development: rhythm, reward and other modulating variables
Miendlarzewska, E. A., & Trost, W. J. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00279 · Cited by 323
Goschke, T. (2014) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1410 · Cited by 293
“…in mental disorders, research on reward-based learning and value-based choice has revealed that the assignment of subjective values to goals and actions and the selection of actions based on such values involve multiple learning and memory systems (Dayan and Daw, 2008; Delgado and Dickerson, 2012).”
BackgroundSupekar, K., Swigart, A. G., Tenison, C., Jolles, D. D., Rosenberg-Lee, M., Fuchs, L., & Menon, V. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1222154110 · Cited by 268
Influence of reward motivation on human declarative memory
Miendlarzewska, E. A., Bavelier, D., & Schwartz, S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.11.015 · Cited by 174
Social rewards and social networks in the human brain
Fareri, D. S., & Delgado, M. R. (2014) · The Neuroscientist · doi:10.1177/1073858414521869 · Cited by 124
“Findings from such investigations have the ability to elucidate mechanisms by which social network and close relationships may influence decision-making during development, over and above a more general social influence.”
BackgroundSelf-citation
Metadata
- Authors
- Delgado, M. R., & Dickerson, K. C.
- Venue
- Biological Psychiatry
- Publisher
- Elsevier BV
- Published
- 2012-07
- Volume
- 72
- Issue
- 2
- Pages
- 134-141
- ISSN
- 0006-3223 (ISSN-L)
- OpenAlex
- W1977809093
- Cited by
- 56
- References
- 54
- FWCI
- 4.45
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.