Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Summary Reinforcement learning (RL) theories posit that dopaminergic signals are integrated within the striatum to associate choices with outcomes. Often overlooked is that the amygdala also receives dopaminergic input and is involved in Pavlovian processes that influence choice behavior. To determine the relative contributions of the ventral striatum (VS) and amygdala to appetitive RL we tested rhesus macaques with VS or amygdala lesions on deterministic and stochastic versions of a two-arm bandit reversal learning task. When learning was characterized with a RL model relative to controls, amygdala lesions caused general decreases in learning from positive feedback and choice consistency. By comparison, VS lesions only affected learning in the stochastic task. Moreover, the VS lesions hastened the monkeys’ choice reaction times, which emphasized a speed-accuracy tradeoff that accounted for errors in deterministic learning. These results update standard accounts of RL by emphasizing distinct contributions of the amygdala and VS to RL.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023 · cited 2×
“, amount of attention paid to a cue) influences the rate at which conditioned associations are learned (Li et al., 2011; Pearce and Hall, 1980).”
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
From circuits to behaviour in the amygdala
Janak, P. H., & Tye, K. M. (2015) · Nature · doi:10.1038/nature14188 · Cited by 2,011 · cited 2×
“The amygdala, however, receives dopaminergic inputs (Haber and Fudge, 1997) and is highly involved in Pavlovian learning of appetitive and aversive associations (Baxter and Murray, 2002; Janak and Tye, 2015; Moscarello and LeDoux, 2014).”
Haber, S. N., Fudge, J. L., & McFarland, N. R. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-06-02369.2000 · Cited by 1,610
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
Baxter, M. G., & Murray, E. A. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn875 · Cited by 1,200
Self-citationModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126
The Nucleus Accumbens: An Interface Between Cognition, Emotion, and Action
Floresco, S. B. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010213-115159 · Cited by 938
Impulsive choice induced in rats by lesions of the nucleus accumbens core
Cardinal, R. N., Pennicott, D. R., Lakmali, C., Sugathapala, Robbins, T. W., & Everitt, B. J. (2001) · Science · doi:10.1126/science.1060818 · Cited by 923
Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking
Stuber, G. D., Sparta, D. R., Stamatakis, A. M., van Leeuwen, W. A., Hardjoprajitno, J. E., Cho, S., Tye, K. M., Kempadoo, K. A., Zhang, F., Deisseroth, K., et al. (2011) · Nature · doi:10.1038/nature10194 · Cited by 878
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Day, J. J., Roitman, M. F., Wightman, R. M., & Carelli, R. M. (2007) · Nature Neuroscience · doi:10.1038/nn1923 · Cited by 710
A circuit mechanism for differentiating positive and negative associations
Namburi, P., Beyeler, A., Yorozu, S., Calhoon, G. G., Halbert, S. A., Wichmann, R., Holden, S. S., Mertens, K. L., Anahtar, M., Felix-Ortiz, A. C., et al. (2015) · Nature · doi:10.1038/nature14366 · Cited by 614 · cited 1×
“Moreover, since amygdala projections to VS preferentially facilitate appetitive behaviors (Namburi et al., 2015), our data offer a new perspective on how interactions between the amygdala and ventral striatum enable reinforcement learning.”
Corbit, L. H., & Balleine, B. W. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.4507-04.2005 · Cited by 589
Emotion, decision making, and the amygdala.
Seymour, B., & Dolan, R. (2008) · Neuron · doi:10.1016/j.neuron.2008.05.020 · Cited by 324
Dopamine modulates novelty seeking behavior during decision making.
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2014) · Behavioral Neuroscience · doi:10.1037/a0037128 · Cited by 263 · cited 1×
“Systemic blockade of the dopamine transporter similarly does not modulate learning rates during a probabilistic, three-arm bandit task (Costa et al., 2014).”
Self-citationTesting the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244
Corbit, L. H., Janak, P. H., & Balleine, B. W. (2007) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2007.05934.x · Cited by 231
Haber, S. N., & Behrens, T. E. J. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.031 · Cited by 230
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183
Reversal learning and dopamine: a Bayesian perspective
Costa, V. D., Tran, V. L., Turchi, J., & Averbeck, B. B. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1989-14.2015 · Cited by 170
Self-citationChau, B. K. H., Sallet, J., Papageorgiou, G. K., Noonan, M. P., Bell, A. H., Walton, M. E., & Rushworth, M. F. S. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.018 · Cited by 159 · cited 2×
“Our data also counter evidence that the amygdala encodes inflexible value representations that impede reversal learning and credit assignment (Chau et al., 2015; Stalnaker et al., 2007).”
Rudebeck, P. H., & Murray, E. A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2272-08.2008 · Cited by 148
Self-citationSelective bilateral amygdala lesions in rhesus monkeys fail to disrupt object reversal learning
Izquierdo, A., & Murray, E. A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3616-06.2007 · Cited by 121 · cited 4×
“They are inconsistent with evidence that excitotoxic amygdala lesions do not disrupt object discrimination reversal learning (Izquierdo and Murray, 2007).”
InfluentialSelf-citationRygula, R., Clarke, H. F., Cardinal, R. N., Cockcroft, G. J., Xia, J., Dalley, J. W., Robbins, T. W., & Roberts, A. C. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu102 · Cited by 85
Evidence for model-based computations in the human amygdala during Pavlovian conditioning
Prévost, C., McNamee, D., Jessup, R. K., Bossaerts, P., & O'Doherty, J. P. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002918 · Cited by 79 · cited 1×
“, 2015), model-based RL (Prévost et al., 2013), and abstract context representations (Saez et al.”
Basolateral amygdala lesions facilitate reward choices after negative feedback in rats
Izquierdo, A., Darling, C., Manos, N., Pozos, H., Kim, C., Ostrander, S., Cazares, V., Stepp, H., & Rudebeck, P. H. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4942-12.2013 · Cited by 63
Cited by
Adaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286
Reinforcement learning in artificial and biological systems
Neftci, E. O., & Averbeck, B. B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0025-4 · Cited by 268
Self-citationMetformin decelerates aging clock in male monkeys
Yang, Y., Lu, X., Liu, N., Ma, S., Zhang, H., Zhang, Z., Yang, K., Jiang, M., Zheng, Z., Qiao, Y., et al. (2024) · Cell · doi:10.1016/j.cell.2024.08.021 · Cited by 227
Pan, Y., Dikker, S., Goldstein, P., Zhu, Y., Yang, C., & Hu, Y. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116657 · Cited by 187
Specializations for reward-guided decision-making in the primate ventral prefrontal cortex
Murray, E. A., & Rudebeck, P. H. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0013-4 · Cited by 169
Self-citation
Metadata
- Authors
- Costa, V. D., Dal Monte, O., Lucas, D. R., Murray, E. A., & Averbeck, B. B.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2016-10
- Volume
- 92
- Issue
- 2
- Pages
- 505-517
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2529608724
- Cited by
- 133
- References
- 27
- FWCI
- 6.27
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.