Volatility Facilitates Value Updating in the Prefrontal Cortex
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
SUMMARY Adaptation of learning and decision-making might depend on the regulation of activity in the prefrontal cortex. Here, we examined how volatility of reward probabilities influences learning and neural activity in the primate prefrontal cortex. We found that animals selected recently rewarded targets more often when reward probabilities of different options fluctuated across trials than when they were fixed. Additionally, neurons in the orbitofrontal cortex displayed more sustained activity related to the outcomes of their previous choices when reward probabilities changed over time. Such volatility also enhanced signals in the dorsolateral prefrontal cortex related to the current, but not the previous, location of the previously rewarded target. These results suggest that prefrontal activity related to choice and reward is dynamically regulated by the volatility of the environment, and underscore the role of the prefrontal cortex in identifying aspects of the environment that are responsible for previous outcomes and should be learned.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“In addition, given the role of the prefrontal cortex (PFC) in learning and decision-making (Miller and Cohen, 2001; Wallis and Kennerley, 2010), flexible regulation of activity in the PFC might facilitate different learning strategies (Doya, 2002; Yu and Dayan, 2005; Nassar et al., 2012; Tervo et al., 2014; Farashahi…”
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · cited 1×
“Specifically, neural representations of the previous trial’s outcome were strengthened during the volatile blocks in the OFC, which carries signals related to values and expected outcomes that are critical for decision-making (Kennerley et al., 2011; Padoa-Schioppa and Assad, 2006).”
Compte, A. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.9.910 · Cited by 1,143
A Recurrent Network Mechanism of Time Integration in Perceptual Decisions
Wong, K.-F., & Wang, X.-J. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.3733-05.2006 · Cited by 1,029 · cited 1×
“, 2015), and relatively small changes in the strength of recurrent connections determine whether neural circuits respond to stimuli from the recent or distant past (Chaudhuri et al., 2015; Wong and Wang, 2006; Murray et al., 2014a).”
A hierarchy of intrinsic timescales across primate cortex
Murray, J. D., Bernacchia, A., Freedman, D. J., Romo, R., Wallis, J. D., Cai, X., Padoa-Schioppa, C., Pasternak, T., Seo, H., Lee, D., et al. (2014) · Nature Neuroscience · doi:10.1038/nn.3862 · Cited by 990
Self-citationWhy neurons mix: high dimensionality for higher cognition
Fusi, S., Miller, E. K., & Rigotti, M. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.010 · Cited by 766 · cited 1×
“, 2012), these results suggest that the DLPFC plays an important role in transforming multiple types of signals relevant for decision-making in an action frame of reference and that heterogeneous conjunctive codes might be essential for this transformation (Fusi et al., 2016).”
Rational regulation of learning dynamics by pupil-linked arousal systems
Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012) · Nature Neuroscience · doi:10.1038/nn.3130 · Cited by 722 · cited 2×
“In addition, given the role of the prefrontal cortex (PFC) in learning and decision-making (Miller and Cohen, 2001; Wallis and Kennerley, 2010), flexible regulation of activity in the PFC might facilitate different learning strategies (Doya, 2002; Yu and Dayan, 2005; Nassar et al., 2012; Tervo et al., 2014; Farashahi…”
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Decision Making in Recurrent Neuronal Circuits
Wang, X.-J. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.034 · Cited by 679 · cited 2×
“Given that the timescale of persistent activity can be controlled by recurrent excitation (Wang, 2008; Murray et al., 2014b; Chaudhuri et al., 2015), volatility of the animal’s environment might influence the neural activity related to learning and decision-making by controlling the efficacy of…”
Prefrontal cortex and decision making in a mixed-strategy game
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004) · Nature Neuroscience · doi:10.1038/nn1209 · Cited by 678
Self-citationNeural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642
Fellows, L. K. (2003) · Brain · doi:10.1093/brain/awg180 · Cited by 631 · cited 1×
“The OFC is thought to play causal roles in behavioral adjustment during decision-making tasks (Schoenbaum et al., 2002; Fellows and Farah, 2003; Rudebeck et al., 2008; Camille et al., 2011), somodulation of reward signals may be critical for adjusting stimulusreward associations in volatile environments.”
Wang, M., Yang, Y., Wang, C.-J., Gamo, N. J., Jin, L. E., Mazer, J. A., Morrison, J. H., Wang, X.-J., & Arnsten, A. F. T. (2013) · Neuron · doi:10.1016/j.neuron.2012.12.032 · Cited by 489 · cited 1×
“In particular, recurrent connections in PFC networks are thought to facilitate persistent activity (Compte et al., 2000; Wang et al., 2013; Murray et al., 2014b; Chaudhuri et al., 2015), and relatively small changes in the strength of recurrent connections determine whether neural circuits respond to stimuli from the…”
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397
Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating
Rudebeck, P. H., Saunders, R. C., Prescott, A. T., Chau, L. S., & Murray, E. A. (2013) · Nature Neuroscience · doi:10.1038/nn.3440 · Cited by 383
Frontal cortex subregions play distinct roles in choices between actions and stimuli
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3541-08.2008 · Cited by 350
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324 · cited 1×
“Previous research has shown that neurons in the ACC carry signals related to learning, reward, and decision-making (Seo and Lee, 2007; Hayden et al., 2009; Hayden and Platt, 2010; Kennerley et al., 2011).”
Self-citationA reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292 · cited 3×
“In particular, the results in the present study suggest that environmental volatility might alter the timescale of reward representation in the OFC, whichmay facilitate reinforcement learning (Bernacchia et al., 2011;Murray et al., 2014b;Meder et al., 2017).”
Self-citationMurray, J. D., Anticevic, A., Gancsos, M., Ichinose, M., Corlett, P. R., Krystal, J. H., & Wang, X.-J. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs370 · Cited by 255
Neurons in anterior cingulate cortex multiplex information about reward and action
Hayden, B. Y., & Platt, M. L. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4874-09.2010 · Cited by 236
Camille, N., Tsuchida, A., & Fellows, L. K. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3164-11.2011 · Cited by 195 · cited 1×
“The OFC is thought to play causal roles in behavioral adjustment during decision-making tasks (Schoenbaum et al., 2002; Fellows and Farah, 2003; Rudebeck et al., 2008; Camille et al., 2011), somodulation of reward signals may be critical for adjusting stimulusreward associations in volatile environments.”
Heterogeneous reward signals in prefrontal cortex
Wallis, J. D., & Kennerley, S. W. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.009 · Cited by 186 · cited 1×
“In addition, given the role of the prefrontal cortex (PFC) in learning and decision-making (Miller and Cohen, 2001; Wallis and Kennerley, 2010), flexible regulation of activity in the PFC might facilitate different learning strategies (Doya, 2002; Yu and Dayan, 2005; Nassar et al.”
Chau, B. K. H., Sallet, J., Papageorgiou, G. K., Noonan, M. P., Bell, A. H., Walton, M. E., & Rushworth, M. F. S. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.018 · Cited by 159 · cited 1×
“, 2012) and the representation of behavioral strategy (Chau et al., 2015).”
Cited by
Adaptive learning under expected and unexpected uncertainty
Soltani, A., & Izquierdo, A. (2019) · Nature reviews. Neuroscience · doi:10.1038/s41583-019-0180-y · Cited by 286
Hierarchical reasoning by neural circuits in the frontal cortex
Sarafyazd, M., & Jazayeri, M. (2019) · Science · doi:10.1126/science.aav8911 · Cited by 191
Serotonin neurons modulate learning rate through uncertainty
Grossman, C. D., Bari, B. A., & Cohen, J. Y. (2022) · Current Biology · doi:10.1016/j.cub.2021.12.006 · Cited by 164
The Misestimation of Uncertainty in Affective Disorders
Pulcu, E., & Browning, M. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.07.007 · Cited by 156
“Investigation of physiological markers of this volatility adaptation process have reported activity in the dorsomedial prefrontal cortex (dmPFC) that correlates with estimated volatility [7,8,13–15], with a consistent effect found in single-neuron recordings from primates [16].”
Flexible combination of reward information across primates
Farashahi, S., Donahue, C. H., Hayden, B. Y., Lee, D., & Soltani, A. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0714-3 · Cited by 112
Self-citation
Metadata
- Authors
- Massi, B., Donahue, C. H., & Lee, D.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2018-08
- Volume
- 99
- Issue
- 3
- Pages
- 598-608.e4
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2884962962
- Cited by
- 62
- References
- 24
- FWCI
- 3.87
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.