Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Summary Adaptive decision-making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here we used a novel directionally-specific viral ablation approach to investigate the function of several anatomically-defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision-making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Abstract reward and punishment representations in the human orbitofrontal cortex
O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001) · Nature Neuroscience · doi:10.1038/82959 · Cited by 2,003
Using Generalized Estimating Equations for Longitudinal Data Analysis
Ballinger, G. A. (2004) · Organizational Research Methods · doi:10.1177/1094428104263672 · Cited by 1,567 · cited 1×
“…of a working correlation matrix to account for within-subject correlation of responses on dependent variables of different distributions, including normal, binomial, and Poisson, that yields unbiased regression parameters relative to ordinary least squares regression (Ballinger, 2004).”
Ridderinkhof, K. R., van den Wildenberg, W. P. M., Segalowitz, S. J., & Carter, C. S. (2004) · Brain and Cognition · doi:10.1016/j.bandc.2004.09.016 · Cited by 1,407
Relative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314
Kelley, A. E. (2004) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2003.11.015 · Cited by 942
The primate amygdala represents the positive and negative value of visual stimuli during learning
Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman, C. D. (2006) · Nature · doi:10.1038/nature04490 · Cited by 917 · cited 2×
“Nevertheless, amygdala neurons rapidly modulate their activity following a change in stimulus value (Paton et al., 2006), and emerging evidence indicates that separable neuronal populations encode positive or negative stimulus values (Paton et al.”
BackgroundOrbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885
Prefrontal cortex and decision making in a mixed-strategy game
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004) · Nature Neuroscience · doi:10.1038/nn1209 · Cited by 678 · cited 1×
“We conducted a two-fold cross validation procedure (Ito and Doya, 2009) to compare the DF RL model to the F RL model that we have previously used (Barraclough et al., 2004; Groman et al., 2016, 2018).”
MethodSelf-citationGremel, C. M., & Costa, R. M. (2013) · Nature Communications · doi:10.1038/ncomms3264 · Cited by 618 · cited 1×
“, 2017), but the functional network of the OFC has only begun to be examined (Gremel and Costa, 2013; Lichtenberg et al., 2017).”
BackgroundDecision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580
Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01876.1999 · Cited by 579
Roitman, M. F., Wheeler, R. A., & Carelli, R. M. (2005) · Neuron · doi:10.1016/j.neuron.2004.12.055 · Cited by 453
Divergent routing of positive and negative information from the amygdala during memory retrieval
Beyeler, A., Namburi, P., Glober, G. F., Simonnet, C., Calhoon, G. G., Conyers, G. F., Luck, R., Wildes, C. P., & Tye, K. M. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.004 · Cited by 412
Orbitofrontal cortex neurons: role in olfactory and visual association learning
Rolls, E. T., Critchley, H. D., Mason, R., & Wakeman, E. A. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.75.5.1970 · Cited by 398 · cited 1×
“Previous studies have linked neural activity in the orbitofrontal cortex (OFC) to reward learning (Rolls et al., 1996; Tremblay and Schultz, 1999; O’Doherty et al., 2001; Wallis and Miller, 2003) and to specific reinforcement-learning parameters (Kennerley and Wallis, 2009; Sul et al.”
BackgroundWalton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397
Waltz, J. A., & Gold, J. M. (2007) · Schizophrenia Research · doi:10.1016/j.schres.2007.03.010 · Cited by 331
Organization of valence-encoding and projection-defined neurons in the basolateral amygdala
Beyeler, A., Chang, C.-J., Silvestre, M., Lévêque, C., Namburi, P., Wildes, C. P., & Tye, K. M. (2018) · Cell Reports · doi:10.1016/j.celrep.2017.12.097 · Cited by 306 · cited 2×
“, 2006), and emerging evidence indicates that separable neuronal populations encode positive or negative stimulus values (Paton et al., 2006; Morrison et al., 2011; Beyeler et al., 2018).”
BackgroundBoulougouris, V., Dalley, J. W., & Robbins, T. W. (2007) · Behavioural Brain Research · doi:10.1016/j.bbr.2007.02.005 · Cited by 297 · cited 1×
“The lateral portions of the OFC have been established in rats as critical for reversal learning (Schoenbaum et al., 1999, 2003; Boulougouris et al., 2007) and we targeted the dorso-lateral portion of the OFC as it has been shown to be exclusively involved in reversal learning.”
BackgroundBasolateral amygdala to orbitofrontal cortex projections enable cue-triggered reward expectations
Lichtenberg, N. T., Pennington, Z. T., Holley, S. M., Greenfield, V. Y., Cepeda, C., Levine, M. S., & Wassum, K. M. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.0486-17.2017 · Cited by 171 · cited 1×
“, 2017), but the functional network of the OFC has only begun to be examined (Gremel and Costa, 2013; Lichtenberg et al., 2017).”
BackgroundRudebeck, P. H., Saunders, R. C., Lundgren, D. A., & Murray, E. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.07.042 · Cited by 170
Chau, B. K. H., Sallet, J., Papageorgiou, G. K., Noonan, M. P., Bell, A. H., Walton, M. E., & Rushworth, M. F. S. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.018 · Cited by 159
Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning
Rodriguez, P. F., Aron, A. R., & Poldrack, R. A. (2006) · Human Brain Mapping · doi:10.1002/hbm.20186 · Cited by 122 · cited 1×
“Therefore, loss of the OFC-generated representation of the expected outcome to the NAcc could subsequently lead to disruptions in prediction error signals following negative feedback (Rodriguez et al., 2006) and the maladaptive decisionmaking reported here.”
BackgroundSelective bilateral amygdala lesions in rhesus monkeys fail to disrupt object reversal learning
Izquierdo, A., & Murray, E. A. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3616-06.2007 · Cited by 121 · cited 1×
“The involvement of the amygdala in decision making is less well established than that of the OFC, in part because of the inconsistency in results regarding the impact of amygdala lesions on reversal-learning, and other decision-making tasks (Izquierdo and Murray, 2007; Izquierdo et al., 2013; Costa et al., 2016).”
ResultKennerley, S. W., & Wallis, J. D. (2009) · Journal of Neurophysiology · doi:10.1152/jn.00273.2009 · Cited by 108 · cited 1×
“, 2001; Wallis and Miller, 2003) and to specific reinforcement-learning parameters (Kennerley and Wallis, 2009; Sul et al., 2010; Abe and Lee, 2011; Stalnaker et al., 2014; Nogueira et al., 2017; Saez et al., 2017; Massi et al., 2018), suggesting that the OFC may contribute to multiple aspects of reinforcement learnin…”
BackgroundManning, E. E., Dombrovski, A. Y., Torregrossa, M. M., & Ahmari, S. E. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-018-0307-2 · Cited by 79
Basolateral amygdala lesions facilitate reward choices after negative feedback in rats
Izquierdo, A., Darling, C., Manos, N., Pozos, H., Kim, C., Ostrander, S., Cazares, V., Stepp, H., & Rudebeck, P. H. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.4942-12.2013 · Cited by 63
Singh, T. (2010) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2010.00126 · Cited by 40 · cited 1×
“, 2008), our findings are consistent with previous evidence demonstrating that lesions to NAcc disrupt learning from negative outcomes (Schoenbaum and Setlow, 2003; Singh et al., 2010).”
Result
Cited by
Single-neuron projectome of mouse prefrontal cortex
Gao, L., Liu, S., Gou, L., Hu, Y., Liu, Y., Deng, L., Ma, D., Wang, H., Yang, Q., Chen, Z., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01041-5 · Cited by 264
Value-guided remapping of sensory cortex by lateral orbitofrontal cortex
Banerjee, A., Parente, G., Teutsch, J., Lewis, C., Voigt, F. F., & Helmchen, F. (2020) · Nature · doi:10.1038/s41586-020-2704-z · Cited by 199
Ahmari, S. E., & Rauch, S. L. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-021-01130-2 · Cited by 95
Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex
Verharen, J. P. H., den Ouden, H. E. M., Adan, R. A. H., & Vanderschuren, L. J. M. J. (2020) · Psychopharmacology · doi:10.1007/s00213-020-05454-7 · Cited by 81
Xie, C., Jia, T., Rolls, E. T., Robbins, T. W., Sahakian, B. J., Zhang, J., Liu, Z., Cheng, W., Luo, Q., Zac Lo, C.-Y., et al. (2021) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2020.08.017 · Cited by 64
Metadata
- Authors
- Groman, S. M., Keistler, C., Keip, A. J., Hammarlund, E., DiLeone, R. J., Pittenger, C., Lee, D., & Taylor, J. R.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2019-08
- Volume
- 103
- Issue
- 4
- Pages
- 734-746.e3
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2954669281
- Cited by
- 142
- References
- 29
- FWCI
- 10.95
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.