Prefrontal Cortex Predicts State Switches during Reversal Learning
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Summary Reinforcement learning allows organisms to predict future outcomes and to update their beliefs about value in the world. The dorsal-lateral prefrontal cortex (dlPFC) integrates information carried by reward circuits, which can be used to infer the current state of the world under uncertainty. Here, we explored the dlPFC computations related to updating current beliefs during stochastic reversal learning. We recorded the activity of populations up to 1000 neurons, simultaneously, in two male macaques, while they executed a two-armed bandit reversal learning task. Behavioral analyses using a Bayesian framework showed that animals inferred reversals and switched their choice preference rapidly, rather than slowly updating choice values, consistent with state inference. Furthermore, dlPFC neural populations accurately encoded choice preference switches. These results suggest that prefrontal neurons dynamically encode decisions associated with Bayesian subjective values, highlighting the role of the PFC in representing a belief about the current state of the world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · cited 2×
“reward prediction error-RPE) to incrementally update choice-outcome mappings (Rescorla and Wagner, 1972; Sutton, 1988).”
BackgroundPearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
Effects of Different Brain Lesions on Card Sorting: The Role of the Frontal Lobes
MILNER, B. (1963) · Archives of Neurology · doi:10.1001/archneur.1963.00460070100010 · Cited by 2,217 · cited 1×
“They were originally motivated by the finding that patients with dorsal-lateral PFC damage had perseverative deficits in the Wisconsin Card Sorting task (Milner, 1963).”
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088 · cited 1×
“This is consistent with these subcortical structures mediating a model-free learning process, as has been suggested by previous work (Averbeck and Costa, 2017; Daw et al., 2006; Hampton et al., 2007; Lee et al., 2015; O'Doherty et al., 2004; Rudebeck et al., 2017a; Seo et al., 2012; Taswell et al., 2018).”
ResultA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066 · cited 4×
“This work has focused on dopamine and its projections to the striatum (Lau and Glimcher, 2008; Lee et al., 2012; Lee et al., 2015; Pessiglione et al., 2006; Samejima et al., 2005), following the finding that dopamine codes RPEs (Kim et al., 2009; Montague et al., 1996; Schultz et al., 1997).”
BackgroundInfluentialThe formation of learning sets.
Harlow, H. F. (1949) · Psychological Review · doi:10.1037/h0062474 · Cited by 1,959
Dissociation in prefrontal cortex of affective and attentional shifts
Dias, R., Robbins, T. W., & Roberts, A. C. (1996) · Nature · doi:10.1038/380069a0 · Cited by 1,527 · cited 3×
“Reversal learning tasks have long been used to study behavioral flexibility (Dias et al., 1996; Groman et al., 2019; Iversen and Mishkin, 1970; Jones and Mishkin, 1972).”
BackgroundMethodModels of Information Processing in the Basal Ganglia
(1994) · doi:10.7551/mitpress/4708.001.0001 · Cited by 1,190
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939
Research works printed but not linked: 47
- Abe, 2011, The prefrontal cortex and hybrid learning during iterative competitive games, Ann. N Y Acad. Sci., 1239, 100, 10.1111/j.1749-6632.2011.06223.x · DOI
- Asaad, 2000, Task-specific neural activity in the primate prefrontal cortex, J. Neurophysiol., 84, 451, 10.1152/jn.2000.84.1.451 · DOI · Paper page
- Averbeck, 2017, Motivational neural circuits underlying reinforcement learning, Nat. Neurosci., 20, 505, 10.1038/nn.4506 · DOI · Paper page
- Averbeck, 2007, Prefrontal neural correlates of memory for sequences, J. Neurosci., 27, 2204, 10.1523/JNEUROSCI.4483-06.2007 · DOI
- Badre, 2012, Mechanisms of hierarchical reinforcement learning in cortico-striatal circuits 2: evidence from fMRI, Cereb. Cortex, 22, 527, 10.1093/cercor/bhr117 · DOI · Paper page
- Botvinick, 2009, Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective, Cognition, 113, 262, 10.1016/j.cognition.2008.08.011 · DOI · Paper page
- Collins, 2012, How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis, Eur. J. Neurosci., 35, 1024, 10.1111/j.1460-9568.2011.07980.x · DOI · Paper page
- Costa, 2016, Amygdala and ventral striatum make distinct contributions to reinforcement learning, Neuron, 92, 505, 10.1016/j.neuron.2016.09.025 · DOI · Paper page
- Costa, 2019, Subcortical substrates of explore-exploit decisions in primates, Neuron, 103, 533, 10.1016/j.neuron.2019.05.017 · DOI · Paper page
- Daw, 2006, Cortical substrates for exploratory decisions in humans, Nature, 441, 876, 10.1038/nature04766 · DOI · Paper page
- Daw, 2011, Model-based influences on humans’ choices and striatal prediction errors, Neuron, 69, 1204, 10.1016/j.neuron.2011.02.027 · DOI · Paper page
- Dayan, 1993, Feudal reinforcement learning, Adv. Neural Inf. Process. Syst., 5, 271
- Durstewitz, 2010, Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning, Neuron, 66, 438, 10.1016/j.neuron.2010.03.029 · DOI · Paper page
- Ebitz, 2018, Exploration disrupts choice-predictive signals and alters dynamics in prefrontal cortex, Neuron, 97, 450, 10.1016/j.neuron.2017.12.007 · DOI · Paper page
- Farashahi, 2017, Metaplasticity as a neural substrate for adaptive learning and choice under uncertainty, Neuron, 94, 401, 10.1016/j.neuron.2017.03.044 · DOI · Paper page
- Frank, 2005, Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism, J. Cogn. Neurosci., 17, 51, 10.1162/0898929052880093 · DOI · Paper page
- Frank, 2012, Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis, Cereb. Cortex, 22, 509, 10.1093/cercor/bhr114 · DOI · Paper page
- Frank, 2004, By carrot or by stick: cognitive reinforcement learning in parkinsonism, Science, 306, 1940, 10.1126/science.1102941 · DOI · Paper page
- Fukushima, 2014, Differential coding of conspecific vocalizations in the ventral auditory cortical stream, J. Neurosci., 34, 4665, 10.1523/JNEUROSCI.3969-13.2014 · DOI
- Gallistel, 2001, The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect, J. Exp. Psychol. Anim. Behav. Process., 27, 354, 10.1037/0097-7403.27.4.354 · doi:10.1037/0097-7403.27.4.354 · Paper page
- Genovesio, 2005, Prefrontal cortex activity related to abstract response strategies, Neuron, 47, 307, 10.1016/j.neuron.2005.06.006 · DOI · Paper page
- Gershman, 2017, Reinforcement learning and episodic memory in humans and animals: an integrative framework, Annu. Rev. Psychol., 68, 101, 10.1146/annurev-psych-122414-033625 · DOI · Paper page
- Groman, 2019, Orbitofrontal circuits control multiple reinforcement-learning processes, Neuron, 103, 734, 10.1016/j.neuron.2019.05.042 · DOI · Paper page
- Hampton, 2007, Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex, Neuron, 55, 545, 10.1016/j.neuron.2007.07.022 · DOI · Paper page
- Iversen, 1970, Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity, Exp. Brain Res., 11, 376, 10.1007/BF00237911 · DOI · Paper page
- Jang, 2015, The role of frontal cortical and medial-temporal lobe brain areas in learning a Bayesian prior belief on reversals, J. Neurosci., 35, 11751, 10.1523/JNEUROSCI.1594-15.2015 · DOI · Paper page
- Jones, 1972, Limbic lesions and the problem of stimulus—reinforcement associations, Exp. Neurol., 36, 362, 10.1016/0014-4886(72)90030-1 · DOI · Paper page
- Karlsson, 2012, Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty, Science, 338, 135, 10.1126/science.1226518 · DOI · Paper page
- Kim, 2009, Role of striatum in updating values of chosen actions, J. Neurosci., 29, 14701, 10.1523/JNEUROSCI.2728-09.2009 · DOI · Paper page
- Lau, 2008, Value representations in the primate striatum during matching behavior, Neuron, 58, 451, 10.1016/j.neuron.2008.02.021 · DOI · Paper page
- Lee, 2012, Neural basis of reinforcement learning and decision making, Annu. Rev. Neurosci., 35, 287, 10.1146/annurev-neuro-062111-150512 · DOI · Paper page
- Lee, 2015, Injection of a dopamine type 2 receptor antagonist into the dorsal striatum disrupts choices driven by previous outcomes, but not perceptual inference, J. Neurosci., 35, 6298, 10.1523/JNEUROSCI.4561-14.2015 · DOI
- Mitz, 2017, High channel count single-unit recordings from nonhuman primate frontal cortex, J. Neurosci. Methods, 289, 39, 10.1016/j.jneumeth.2017.07.001 · DOI
- Neftci, 2019, Reinforcement learning in artificial and biological systems, Nature Machine Intelligence, 1, 133, 10.1038/s42256-019-0025-4 · DOI · Paper page
- Pessiglione, 2006, Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans, Nature, 442, 1042, 10.1038/nature05051 · DOI · Paper page
- Rothenhoefer, 2017, Effects of ventral striatum lesions on stimulus-based versus action-based reinforcement learning, J. Neurosci., 37, 6902, 10.1523/JNEUROSCI.0631-17.2017 · DOI
- Rudebeck, 2013, Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating, Nat. Neurosci., 16, 1140, 10.1038/nn.3440 · DOI · Paper page
- Rudebeck, 2017, Amygdala contributions to stimulus-reward encoding in the macaque medial and orbital frontal cortex during learning, J. Neurosci., 37, 2186, 10.1523/JNEUROSCI.0933-16.2017 · DOI
- Schoenbaum, 2003, Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals, Learn. Mem., 10, 129, 10.1101/lm.55203 · DOI · Paper page
- Schuck, 2016, Human orbitofrontal cortex represents a cognitive map of state space, Neuron, 91, 1402, 10.1016/j.neuron.2016.08.019 · DOI · Paper page
- Schultz, 1997, A neural substrate of prediction and reward, Science, 275, 1593, 10.1126/science.275.5306.1593 · DOI · Paper page
- Seo, 2012, Action selection and action value in frontal-striatal circuits, Neuron, 74, 947, 10.1016/j.neuron.2012.03.037 · DOI · Paper page
- Stalnaker, 2007, Basolateral amygdala lesions abolish orbitofrontal-dependent reversal impairments, Neuron, 54, 51, 10.1016/j.neuron.2007.02.014 · DOI · Paper page
- Steinberg, 2013, A causal link between prediction errors, dopamine neurons and learning, Nat. Neurosci., 16, 966, 10.1038/nn.3413 · DOI · Paper page
- Taswell, 2018, Ventral striatum’s role in learning from gains and losses, Proc. Natl. Acad. Sci. U S A, 115, E12398, 10.1073/pnas.1809833115 · DOI
- Willenbockel, 2010, Controlling low-level image properties: the SHINE toolbox, Behav. Res. Methods, 42, 671, 10.3758/BRM.42.3.671 · DOI · Paper page
- Wilson, 2010, Bayesian online learning of the hazard rate in change-point problems, Neural Comput., 22, 2452, 10.1162/NECO_a_00007 · DOI
Cited by
Cognitive and behavioural flexibility: Neural mechanisms and clinical considerations
Uddin, L. Q. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00428-w · Cited by 698
Medial and orbital frontal cortex in decision-making and flexible behavior
Klein-Flügge, M. C., Bongioanni, A., & Rushworth, M. F. S. (2022) · Neuron · doi:10.1016/j.neuron.2022.05.022 · Cited by 244
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64
“3, 10 and 15–17), a central theme in computational psychiatry.”
Influential
Metadata
- Authors
- Bartolo, R., & Averbeck, B. B.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2020-06
- Volume
- 106
- Issue
- 6
- Pages
- 1044-1054.e4
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W3016851284
- Cited by
- 122
- References
- 25
- FWCI
- 8.51
- Type
- journal article
- Language
- en
- Open access
- bronze
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 381 d)
- Funders
- National Institute of Mental Health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Prefrontal Cortex Predicts State Switches during Reversal Learning. https://app.cochat.ai/papers/10.1016/j.neuron.2020.03.024
- The paper
- Bartolo, R., & Averbeck, B. B. (2020). Prefrontal Cortex Predicts State Switches during Reversal Learning. Neuron. https://doi.org/10.1016/j.neuron.2020.03.024
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.