Orbitofrontal Cortex as a Cognitive Map of Task Space
Open accessAbstract
Orbitofrontal cortex (OFC) has long been known to play an important role in decision making. However, the exact nature of that role has remained elusive. Here, we propose a unifying theory of OFC function. We hypothesize that OFC provides an abstraction of currently available information in the form of a labeling of the current task state, which is used for reinforcement learning (RL) elsewhere in the brain. This function is especially critical when task states include unobservable information, for instance, from working memory. We use this framework to explain classic findings in reversal learning, delayed alternation, extinction, and devaluation as well as more recent findings showing the effect of OFC lesions on the firing of dopaminergic neurons in ventral tegmental area (VTA) in rodents performing an RL task. In addition, we generate a number of testable experimental predictions that can distinguish our theory from other accounts of OFC function.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
OFC is critical for contextual knowledge of outcome expectation, which hierarchically regulates performance of already acquired behaviors.
“In addition, it has been reported that the OFC is critical for contextual knowledge of outcome expectation, which hierarchically regulates performance of already acquired behaviors51,52.”
From Dissociable roles of prefrontal plasticity in decision-making strategy and execution of habitual behavior (2026) · cites this paper as [51]
The Introduction and Results > Reversal learning propose that OFC represents context-dependent task states, allowing previously learned action preferences to guide behavior when reward contingencies recur. The paper does not describe this as hierarchical regulation by knowledge of outcome expectations; Results > Insights into the role of OFC from dopamine firing patterns instead distinguishes OFC state representation from directly encoding expected value. The cited account is a theoretical proposal, not an established finding that OFC is critical in all such behavior. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe brain may have dedicated machinery, perhaps in prefrontal cortex, for computing belief states, which could then be provided to downstream areas, such as the basal ganglia, to perform standard RL algorithms such as TD learning.
“The brain may have dedicated machinery, perhaps in prefrontal cortex [12,13], for computing belief states, which could then be provided to downstream areas, such as the basal ganglia, to perform standard RL algorithms such as TD learning [14].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [12]
Previous work has suggested that prefrontal cortex may perform state estimation in tasks with hidden states.
“For instance, previous work has suggested that prefrontal cortex may perform state estimation in tasks with hidden states [6,7,12,13].”
From Emergence of belief-like representations through reinforcement learning (2023) · cites this paper as [12]
Positively and negatively valenced functions take place within the prefrontal cortex.
“There is excellent evidence that these positively and negatively valenced functions take place within the PFC [168][169][170].”
From Plasticity of synapses and reward circuit function in the genesis and treatment of depression (2023) · cites this paper as [168]
In the rodent (agranular) OFC, both reward value and behavioural responses are represented, making the rodent OFC very different from the primate OFC.
“Fourth, although reward value is represented in the rodent (agranular) OFC, so also apparently are behavioural responses (Schoenbaum et al., 2009; Wilson et al., 2014; Sharpe et al., 2015, 2019; Izquierdo, 2017), which makes the rodent OFC very different to the primate OFC.”
From The orbitofrontal cortex: reward, emotion and depression (2020) · cites this paper as Wilson et al., 2014
In rodents, the OFC seems much more heterogeneous with behavioural responses also represented in it.
“In contrast, in rodents, the OFC seems much more heterogeneous with behavioural responses also represented in it (Wilson et al., 2014; Sharpe et al., 2015, 2019), and again, the system is more complex because different computations are apparently intermingled in the same brain region.”
From The orbitofrontal cortex: reward, emotion and depression (2020) · cites this paper as Wilson et al., 2014
The rodent orbitofrontal cortex is involved in motor responses and actions, so it can be less specialized for representing reward value and rapidly changing it.
“the rodent ‘OFC’ is also involved in motor responses and actions (Wilson et al., 2014), so it can be less specialized for representing reward value, and rapidly changing it.”
From The orbitofrontal cortex: reward, emotion and depression (2020) · cites this paper as Wilson et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
(1994) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,542 · see citing works
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · see citing works
DESCARTES' ERROR: Emotion, Reason, and the Human Brain
Marg, E. (1995) · Optometry and Vision Science · doi:10.1002/j.1538-9235.1995.tb03988.x · Cited by 8,177 · see citing works
Rescorla, R. A. (1972) · Medical Entomology and Zoology · Cited by 7,268 · see citing works
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449 · see citing works
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149 · see citing works
MacDonald, A. W., Cohen, J. D., Stenger, V. A., & Carter, C. S. (2000) · Science · doi:10.1126/science.288.5472.1835 · Cited by 3,666 · see citing works
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · see citing works
Self-citationDescartes’ Error. Emotion, Reason and the Human Brain. New York (Grosset/Putnam) 1994.
Damasio, A. R. (1994) · Cited by 2,266 · see citing works
Fuster, J. M. (1997) · Cited by 2,095 · see citing works
Context and Behavioral Processes in Extinction: Table 1.
Bouton, M. E. (2004) · Learning & Memory · doi:10.1101/lm.78804 · Cited by 1,808 · see citing works
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · see citing works
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Balleine, B. W., & Dickinson, A. (1998) · Neuropharmacology · doi:10.1016/s0028-3908(98)00033-1 · Cited by 1,541 · see citing works
Dissociation in prefrontal cortex of affective and attentional shifts
Dias, R., Robbins, T. W., & Roberts, A. C. (1996) · Nature · doi:10.1038/380069a0 · Cited by 1,527 · see citing works
Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys
Carmichael, S. T., & Price, J. L. (1995) · Journal of Comparative Neurology · doi:10.1002/cne.903630408 · Cited by 1,197 · see citing works
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165 · see citing works
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111 · see citing works
Neural responses during anticipation of a primary taste reward
O'Doherty, J. P., Deichmann, R., Critchley, H. D., & Dolan, R. J. (2002) · Neuron · doi:10.1016/s0896-6273(02)00603-7 · Cited by 1,076 · see citing works
Rolls, E. T., Hornak, J., Wade, D., & McGrath, J. (1994) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.57.12.1518 · Cited by 1,005 · see citing works
Orbitofrontal cortex and its contribution to decision-making.
Wallis, J. D. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.30.051606.094334 · Cited by 938 · see citing works
Ferrier, D. (1876) · G P Putnam's Sons eBooks · doi:10.1037/12860-000 · Cited by 872 · see citing works
Limbic lesions and the problem of stimulus--reinforcement associations.
Jones, B., & Mishkin, M. (1972) · Experimental Neurology · doi:10.1016/0014-4886(72)90030-1 · Cited by 864 · see citing works
The orbitofrontal cortex: Neuronal activity in the behaving monkey
Thorpe, S. J., Rolls, E. T., & Maddison, S. (1983) · Experimental Brain Research · doi:10.1007/bf00235545 · Cited by 836 · see citing works
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809 · see citing works
Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys
Carmichael, S. T., & Price, J. L. (1995) · Journal of Comparative Neurology · doi:10.1002/cne.903630409 · Cited by 678 · see citing works
Do Rats Have Prefrontal Cortex? The Rose-Woolsey-Akert Program Reconsidered
Preuss, T. M. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.1.1 · Cited by 666 · see citing works
Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat
McAlonan, K., & Brown, V. J. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.019 · Cited by 618 · see citing works
Neurobiology of economic choice: a good-based model
Padoa-Schioppa, C. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113648 · Cited by 600 · see citing works
Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning
Gallagher, M., McMahan, R. W., & Schoenbaum, G. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-15-06610.1999 · Cited by 593 · see citing works
Self-citationIzquierdo, A., Suda, R. K., & Murray, E. A. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.1921-04.2004 · Cited by 588 · see citing works
Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01876.1999 · Cited by 579 · see citing works
Self-citationCoincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.012 · Cited by 574 · see citing works
Chudasama, Y., & Robbins, T. W. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-25-08771.2003 · Cited by 543 · see citing works
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498 · see citing works
Forward frontal fields: phylogeny and fundamental function
Wise, S. P. (2008) · Trends in Neurosciences · doi:10.1016/j.tins.2008.08.008 · Cited by 478 · cited 3× · see citing works
“However, it is important to note that there are large differences in anatomy across species, and OFC in rats has a very different cytoarchitecture than OFC in monkeys and humans (Wise, 2008; Wallis, 2012).”
Critchley, H., & Rolls, E. T. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.75.4.1673 · Cited by 467 · see citing works
Croxson, P. L., Johansen-Berg, H., Behrens, T. E. J., Robson, M. D., Pinsk, M. A., Gross, C. G., Richter, W., Richter, M. C., Kastner, S., & Rushworth, M. F. S. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.1311-05.2005 · Cited by 410 · see citing works
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397 · see citing works
PRICE, J. L. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.008 · Cited by 395 · see citing works
Butter, C. M. (1969) · Physiology & Behavior · doi:10.1016/0031-9384(69)90075-4 · Cited by 395 · see citing works
The representation of economic value in the orbitofrontal cortex is invariant for changes of menu
Padoa‐Schioppa, C., & Assad, J. A. (2007) · Nature Neuroscience · doi:10.1038/nn2020 · Cited by 390 · see citing works
Context, learning, and extinction
Gershman, S. J., Blei, D. M., & Niv, Y. (2010) · Psychological Review · doi:10.1037/a0017808 · Cited by 386 · cited 6× · see citing works
“Consequently, similar to the model of reversal learning, we modeled extinction with a twostate system (see also Gershman et al., 2010).”
InfluentialSelf-citationDifferent Roles for Orbitofrontal Cortex and Basolateral Amygdala in a Reinforcer Devaluation Task
Pickens, C. L., Saddoris, M. P., Setlow, B., Gallagher, M., Holland, P. C., & Schoenbaum, G. M. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-35-11078.2003 · Cited by 379 · see citing works
Self-citationOrbitofrontal Cortex Mediates Outcome Encoding in Pavlovian But Not Instrumental Conditioning
Ostlund, S. B., & Balleine, B. W. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.5443-06.2007 · Cited by 379 · cited 1× · see citing works
“Consistent with this proposal, OFC lesions impair performance in devaluation experiments, causing lesioned animals to respond equally to devalued and nondevalued cues (Gallagher et al., 1999; Pickens et al., 2003; Izquierdo et al., 2004; but see Ostlund and Balleine, 2007).”
Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making
Sul, J. H., Kim, H., Huh, N., Lee, D., & Jung, M. W. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.033 · Cited by 372 · see citing works
Non-spatial memory after selective prefrontal lesions in monkeys
Mishkin, M., & Manning, F. J. (1978) · Brain Research · doi:10.1016/0006-8993(78)90571-1 · Cited by 371 · see citing works
Cross-species studies of orbitofrontal cortex and value-based decision-making
Wallis, J. D. (2011) · Nature Neuroscience · doi:10.1038/nn.2956 · Cited by 356 · see citing works
Meunier, M. (1997) · Neuropsychologia · doi:10.1016/s0028-3932(97)00027-4 · Cited by 348 · see citing works
Redish, A. D., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007) · Psychological Review · doi:10.1037/0033-295x.114.3.784 · Cited by 346 · see citing works
Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340 · see citing works
Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values
Jones, J. L., Esber, G. R., McDannald, M. A., Gruber, A. J., Hernández, A., Mirenzi, A., & Schoenbaum, G. M. (2012) · Science · doi:10.1126/science.1227489 · Cited by 332 · see citing works
Self-citationNeurobiology of Sensation and Reward
Gottfried, J. A. (2011) · CRC Press eBooks · doi:10.1201/b10776 · Cited by 331 · see citing works
Schoenbaum, G. M., & Eichenbaum, H. (1995) · Journal of Neurophysiology · doi:10.1152/jn.1995.74.2.733 · Cited by 329 · see citing works
Self-citationPrefrontal regions play a predominant role in imposing an attentional ‘set’: evidence from fMRI
Banich, M. T., Milham, M. P., Atchley, R. A., Cohen, N. J., WEBB, A. G., Wszalek, T., Kramer, A. F., Liang, Z., Barad, V., Gullett, D., et al. (2000) · Cognitive Brain Research · doi:10.1016/s0926-6410(00)00015-x · Cited by 326 · see citing works
Learning latent structure: carving nature at its joints
Gershman, S. J., & Niv, Y. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.008 · Cited by 326 · see citing works
Self-citationOrbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations
Schoenbaum, G. M., Nugent, S. L., Saddoris, M. P., & Setlow, B. (2002) · Neuroreport · doi:10.1097/00001756-200205070-00030 · Cited by 320 · see citing works
Self-citationDissociable components of rule-guided behavior depend on distinct medial and prefrontal regions
Buckley, M. J., Mansouri, F. A., Hoda, H., Mahboubi, M., Browning, P. G. F., Kwok, S. C., Phillips, A., & Tanaka, K. (2009) · Science · doi:10.1126/science.1172377 · Cited by 319 · see citing works
Schoenbaum, G., Setlow, B., Nugent, S. L., Saddoris, M. P., & Gallagher, M. (2003) · Learning & memory (Cold Spring Harbor, N.Y.) · doi:10.1101/lm.55203 · Cited by 317 · see citing works
Self-citationConnectivity-based parcellation of the human orbitofrontal cortex
Kahnt, T., Chang, L. J., Park, S. Q., Heinzle, J., & Haynes, J.-D. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.0257-12.2012 · Cited by 299 · see citing works
Hippocampal involvement in contextual modulation of fear extinction
Ji, J., & Maren, S. (2007) · Hippocampus · doi:10.1002/hipo.20331 · Cited by 297 · see citing works
Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation
Roesch, M. R., Taylor, A. R., & Schoenbaum, G. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.027 · Cited by 297 · see citing works
Self-citationTakahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., Burke, K. A., & Schoenbaum, G. M. (2009) · Neuron · doi:10.1016/j.neuron.2009.03.005 · Cited by 284 · see citing works
Self-citationExpectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex
Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O'Donnell, P., Niv, Y., & Schoenbaum, G. (2011) · Nature Neuroscience · doi:10.1038/nn.2957 · Cited by 280 · see citing works
Self-citationPostconditioning devaluation of a reinforcer affects instrumental responding.
Colwill, R. M., & Rescorla, R. A. (1985) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.11.1.120 · Cited by 273 · see citing works
Representation of Spatial Goals in Rat Orbitofrontal Cortex
Feierstein, C. E., Quirk, M. C., Uchida, N., Sosulski, D. L., & Mainen, Z. F. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.032 · Cited by 248 · see citing works
Murray, E. A., O'Doherty, J. P., & Schoenbaum, G. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1556-07.2007 · Cited by 238 · see citing works
Self-citationMcDannald, M. A., Lucantonio, F., Burke, K. A., Niv, Y., & Schoenbaum, G. M. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.5499-10.2011 · Cited by 215 · see citing works
Self-citationThe convergence of information about rewarding and aversive stimuli in single neurons
Morrison, S. E., & Salzman, C. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1815-09.2009 · Cited by 206 · see citing works
The involvement of the orbitofrontal cortex in learning under changing task contingencies.
Kim, J., & Ragozzino, M. (2005) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2004.10.003 · Cited by 206 · see citing works
A re-examination of the effects of frontal lesions on object alternation
Mishkin, M., Vest, B., Waxler, M., & Rosvold, H. E. (1969) · Neuropsychologia · doi:10.1016/0028-3932(69)90060-8 · Cited by 198 · see citing works
Critchley, H., & Rolls, E. T. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.75.4.1659 · Cited by 190 · see citing works
Tsuchida, A., Doll, B. B., & Fellows, L. K. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1958-10.2010 · Cited by 188 · cited 1× · see citing works
“Thus, our representation generalizes to probabilistic reversal learning tasks (e.g., Tsuchida et al., 2010) in which the animal (and model) must infer what state it is in by using actions and outcomes from multiple previous trials (Gershman et al., 2010).”
Rudebeck, P. H., & Murray, E. A. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.0091-11.2011 · Cited by 179 · see citing works
Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement
Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. S. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2012.08023.x · Cited by 169 · see citing works
Rapid reacquisition in conditioning of the rabbit's nictitating membrane response.
Napier, R. M., Macrae, M., & Kehoe, E. J. (1992) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037//0097-7403.18.2.182 · Cited by 161 · see citing works
Neuroscience of Rule-Guided Behavior
Stoet, G., & Snyder, L. H. (2007) · doi:10.1093/acprof:oso/9780195314274.001.0001 · Cited by 157 · see citing works
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153 · see citing works
Self-citationInferring Relevance in a Changing World
Wilson, R. C., & Niv, Y. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00189 · Cited by 150 · see citing works
Self-citationDistinct Roles of Three Frontal Cortical Areas in Reward-Guided Behavior
Noonan, M. P., Mars, R. B., & Rushworth, M. F. S. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6456-10.2011 · Cited by 145 · see citing works
Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex
Seo, H., Barraclough, D. J., & Lee, D. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm064 · Cited by 143 · see citing works
Butter, C. M., Mishkin, M., & Rosvold, H. E. (1963) · Experimental Neurology · doi:10.1016/0014-4886(63)90094-3 · Cited by 139 · see citing works
Neural Ensemble Coding of Satiety States
de Araújo, I. E., Gutiérrez, R., Oliveira-Maia, A. J., Pereira, A., Nicolelis, M. A. L., & Simon, S. A. (2006) · Neuron · doi:10.1016/j.neuron.2006.07.009 · Cited by 136 · see citing works
Teitelbaum, H. (1964) · Experimental Neurology · doi:10.1016/0014-4886(64)90053-6 · Cited by 136 · see citing works
Brown, T. I., Ross, R. S., Keller, J. B., Hasselmo, M. E., & Stern, C. E. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.6021-09.2010 · Cited by 130 · see citing works
Reacquisition following extinction in appetitive conditioning
Ricker, S. T., & Bouton, M. E. (1996) · Animal Learning & Behavior · doi:10.3758/bf03199014 · Cited by 126 · see citing works
Neural Correlates of Olfactory Recognition Memory in the Rat Orbitofrontal Cortex
Ramus, S. J., & Eichenbaum, H. (2000) · Journal of Neuroscience · doi:10.1523/jneurosci.20-21-08199.2000 · Cited by 122 · see citing works
Schoenbaum, G., Setlow, B., & Ramus, S. J. (2003) · Behavioural Brain Research · doi:10.1016/j.bbr.2003.09.013 · Cited by 113 · see citing works
Self-citationRudebeck, P. H., & Murray, E. A. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06267.x · Cited by 106 · see citing works
Behavioral effects of sequential and one-stage ablations of orbital prefrontal cortex in the monkey
Butters, N., Butter, C. M., Rosen, J. J., et al. (1973) · Experimental Neurology · doi:10.1016/0014-4886(73)90223-9 · Cited by 104 · see citing works
Monkey Orbitofrontal Cortex Encodes Response Choices Near Feedback Time
Tsujimoto, S., Genovesio, A., & Wise, S. P. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5777-08.2009 · Cited by 93 · see citing works
Rostral–caudal gradients of abstraction revealed by multi-variate pattern analysis of working memory
Nee, D. E., & Brown, J. W. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.08.034 · Cited by 83 · see citing works
Encoding of Gustatory Working Memory by Orbitofrontal Neurons
Lara, A. H., Kennerley, S. W., & Wallis, J. D. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.4637-08.2009 · Cited by 76 · see citing works
Dynamic Changes in Representations of Preceding and Upcoming Reward in Monkey Orbitofrontal Cortex
Simmons, J. M., & Richmond, B. J. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm034 · Cited by 71 · see citing works
Comparison of Strategy Signals in the Dorsolateral and Orbital Prefrontal Cortex
Tsujimoto, S., Genovesio, A., & Wise, S. P. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.5816-10.2011 · Cited by 71 · see citing works
Orbital prefrontal cortex and guidance of instrumental behaviour in rats under reversal conditions
Bohn, I., Giertler, C., & Hauber, W. (2003) · Behavioural Brain Research · doi:10.1016/s0166-4328(03)00008-1 · Cited by 65 · see citing works
Perseveration of responding and nonresponding in monkeys with orbital frontal ablations.
McEnaney, K. W., & Butter, C. M. (1969) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0027639 · Cited by 63 · see citing works
Mapping Prefrontal Cortical Systems for the Control of Cognition
Petrides, M. (2000) · Elsevier eBooks · doi:10.1016/b978-012692545-6/50008-8 · Cited by 57 · see citing works
Furuyashiki, T., Holland, P. C., & Gallagher, M. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.0319-08.2008 · Cited by 55 · see citing works
Perceptual estimation obeys Occam's razor
Gershman, S. J., & Niv, Y. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00623 · Cited by 46 · see citing works
Self-citationWhat Can Different Brains Do with Reward?
Gottfried, J. A. (2011) · CRC Press eBooks · doi:10.1201/b10776-10 · Cited by 33 · see citing works
Retention of spatial alternation following frontal lobe resections in stump-tailed macaques
Miller, M. H., & Orbach, J. (1972) · Neuropsychologia · doi:10.1016/0028-3932(72)90020-6 · Cited by 28 · see citing works
Otto, A. R., Gureckis, T. M., Markman, A. B., & Love, B. C. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.5.957 · Cited by 21 · see citing works
What Can Different Brains Do with Reward? -- Neurobiology of Sensation and Reward
Ja, G. (2011) · Cited by 5 · see citing works
The Role of the Posterior Frontolateral Cortex in Task‐Related Control
Braß, M., Derrfuß, J., & von Cramon, Y. D. (2007) · doi:10.1093/acprof:oso/9780195314274.003.0011 · Cited by 4 · see citing works
Connectional networks within the orbital and medial prefrontal cortex of macaque monkeys
Carmichael, S. T., & Price, J. L. (1996) · Journal of Comparative Neurology · doi:10.1002/(sici)1096-9861(19960722)371:2
Flagged first, then by citations
Cited by
Sorted by citations
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055 · see citing works
The neural basis of reversal learning: An updated perspective
Izquierdo, A., Brigman, J. L., Radke, A. K., Rudebeck, P. H., & Holmes, A. (2017) · Neuroscience · doi:10.1016/j.neuroscience.2016.03.021 · Cited by 670 · see citing works
Human orbitofrontal cortex represents a cognitive map of state space
Schuck, N. W., Cai, M. B., Wilson, R. C., & Niv, Y. (2016) · Neuron · doi:10.1016/j.neuron.2016.08.019 · Cited by 602 · see citing works
Self-citationThe hierarchical basis of neurovisceral integration
Smith, R., Thayer, J. F., Khalsa, S. S., & Lane, R. D. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.003 · Cited by 523 · see citing works
What the orbitofrontal cortex does not do
Stalnaker, T. A., Cooch, N. K., & Schoenbaum, G. (2015) · Nature Neuroscience · doi:10.1038/nn.3982 · Cited by 531 · see citing works
Self-citationLearning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475 · see citing works
The orbitofrontal cortex: reward, emotion and depression
Rolls, E. T., Cheng, W., & Feng, J. (2020) · Brain Communications · doi:10.1093/braincomms/fcaa196 · Cited by 474 · see citing works
Rudebeck, P. H., & Murray, E. A. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.049 · Cited by 405 · see citing works
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356 · see citing works
Self-citationOn staying grounded and avoiding quixotic dead ends
Barsalou, L. W. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1028-3 · Cited by 331 · see citing works
Sequential replay of nonspatial task states in the human hippocampus
Schuck, N. W., & Niv, Y. (2019) · Science · doi:10.1126/science.aaw5181 · Cited by 304 · see citing works
Self-citationAn integrative perspective on the role of dopamine in schizophrenia
Maia, T. V., & Frank, M. J. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2016.05.021 · Cited by 293 · see citing works
Dynamic Interaction between Reinforcement Learning and Attention in Multidimensional Environments
Leong, Y. C., Radulescu, A., Daniel, R., DeWoskin, V., & Niv, Y. (2017) · Neuron · doi:10.1016/j.neuron.2016.12.040 · Cited by 294 · see citing works
Self-citationReward value comparison via mutual inhibition in ventromedial prefrontal cortex
Strait, C. E., Blanchard, T. C., & Hayden, B. Y. (2014) · Neuron · doi:10.1016/j.neuron.2014.04.032 · Cited by 294 · see citing works
Structuring knowledge with cognitive maps and cognitive graphs
Peer, M., Brunec, I. K., Newcombe, N. S., & Epstein, R. A. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.10.004 · Cited by 283 · see citing works
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269 · see citing works
Gershman, S. J., & Uchida, N. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0220-7 · Cited by 236 · see citing works
Depression: a decision-theoretic analysis
Huys, Q. J. M., Daw, N. D., & Dayan, P. (2015) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071714-033928 · Cited by 221 · see citing works
Reinforcement learning in depression: a review of computational research
Chen, C., Takahashi, T., Nakagawa, S., Inoue, T., & Kusumi, I. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.05.005 · Cited by 233 · see citing works
Abstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183 · see citing works
Specializations for reward-guided decision-making in the primate ventral prefrontal cortex
Murray, E. A., & Rudebeck, P. H. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0013-4 · Cited by 169 · see citing works
Gilboa, A., & Moscovitch, M. (2021) · Neuron · doi:10.1016/j.neuron.2021.04.025 · Cited by 174 · see citing works
Basolateral amygdala to orbitofrontal cortex projections enable cue-triggered reward expectations
Lichtenberg, N. T., Pennington, Z. T., Holley, S. M., Greenfield, V. Y., Cepeda, C., Levine, M. S., & Wassum, K. M. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.0486-17.2017 · Cited by 171 · see citing works
Spatial goal coding in the hippocampal formation
Nyberg, N., Duvelle, É., Barry, C., & Spiers, H. J. (2022) · Neuron · doi:10.1016/j.neuron.2021.12.012 · Cited by 141 · see citing works
Namboodiri, V. M. K., Otis, J. M., van Heeswijk, K., Voets, E. S., Alghorazi, R. A., Rodriguez-Romaguera, J., Mihalas, S., & Stuber, G. D. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0408-1 · Cited by 150 · see citing works
Discovering latent causes in reinforcement learning
Gershman, S. J., Norman, K. A., & Niv, Y. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.07.007 · Cited by 151 · see citing works
Self-citationThe medial prefrontal cortex shapes dopamine reward prediction errors under state uncertainty
Starkweather, C. K., Gershman, S. J., & Uchida, N. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.036 · Cited by 136 · see citing works
Memory consolidation as an adaptive process
Cowan, E. T., Schapiro, A. C., Dunsmoor, J. E., & Murty, V. P. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-021-01978-x · Cited by 133 · see citing works
The case against economic values in the orbitofrontal cortex (or anywhere else in the brain).
Hayden, B. Y., & Niv, Y. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000448 · Cited by 127 · see citing works
Self-citationBelief state representation in the dopamine system
Babayan, B. M., Uchida, N., & Gershman, S. J. (2018) · Nature Communications · doi:10.1038/s41467-018-04397-0 · Cited by 101 · see citing works
A probability distribution over latent causes, in the orbitofrontal cortex
Chan, S. C. Y., Niv, Y., & Norman, K. A. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.0659-16.2016 · Cited by 97 · see citing works
Self-citationGeorge, D., Rikhye, R. V., Gothoskar, N., Guntupalli, J. S., Dedieu, A., & Lázaro-Gredilla, M. (2021) · Nature Communications · doi:10.1038/s41467-021-22559-5 · Cited by 109 · see citing works
Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex
Masset, P., Ott, T., Lak, A., Hirokawa, J., & Kepecs, A. (2020) · Cell · doi:10.1016/j.cell.2020.05.022 · Cited by 109 · see citing works
Enhanced deviant responses in patterned relative to random sound sequences
Southwell, R., & Chait, M. (2018) · Cortex · doi:10.1016/j.cortex.2018.08.032 · Cited by 80 · see citing works
Brain structure in pediatric Tourette syndrome
and The Tourette Association of America Neuroimaging Consortium, Greene, D. J., Williams III, A. C., Koller, J. M., Schlaggar, B. L., & Black, K. J. (2017) · Molecular Psychiatry · doi:10.1038/mp.2016.194 · Cited by 89 · see citing works
Reward-Guided Learning with and without Causal Attribution
Jocham, G., Brodersen, K. H., Constantinescu, A. O., Kahn, M. C., Ianni, A. M., Walton, M. E., Rushworth, M. F. S., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.018 · Cited by 92 · see citing works
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79 · see citing works
Saez, R. A., Saez, A., Paton, J. J., Lau, B., & Salzman, C. D. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.012 · Cited by 75 · see citing works
Trudel, N., Scholl, J., Klein-Flügge, M. C., Fouragnan, E., Tankelevitch, L., Wittmann, M. K., & Rushworth, M. F. S. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0929-3 · Cited by 74 · see citing works
Panayi, M. C., & Killcross, S. (2018) · eLife · doi:10.7554/elife.37357 · Cited by 69 · see citing works
Hervig, M. E., Fiddian, L., Piilgaard, L., Božič, T., Blanco-Pozo, M., Knudsen, C., Olesen, S. F., Alsiö, J., & Robbins, T. W. (2020) · Cerebral Cortex · doi:10.1093/cercor/bhz144 · Cited by 65 · see citing works
Human ventromedial prefrontal lesions alter incentivisation by reward
Manohar, S. G., & Husain, M. (2016) · Cortex · doi:10.1016/j.cortex.2016.01.005 · Cited by 62 · see citing works
Is the core function of orbitofrontal cortex to signal values or make predictions?
Zhou, J., Gardner, M. P., & Schoenbaum, G. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.02.011 · Cited by 57 · see citing works
Self-citationTo be specific: The role of orbitofrontal cortex in signaling reward identity.
Howard, J. D., & Kahnt, T. (2021) · Behavioral Neuroscience · doi:10.1037/bne0000455 · Cited by 57 · see citing works
Action information contributes to metacognitive decision-making
Wokke, M. E., Achoui, D., & Cleeremans, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-60382-y · Cited by 56 · see citing works
From remembering to reconstruction: The transformative neural representation of episodic memory.
Xue, G. (2022) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2022.102351 · Cited by 49 · see citing works
Emergence of belief-like representations through reinforcement learning
Hennig, J. A., Romero Pinto, S. A., Yamaguchi, T., Linderman, S. W., Uchida, N., & Gershman, S. J. (2023) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011067 · Cited by 41 · see citing works
Dopamine, inference, and uncertainty
Gershman, S. J. (2017) · Neural Computation · doi:10.1162/neco_a_01023 · Cited by 40 · see citing works
Plasticity of synapses and reward circuit function in the genesis and treatment of depression
Thompson, S. M. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-022-01422-1 · Cited by 40 · see citing works
Vaidya, A. R., & Fellows, L. K. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.1607-15.2015 · Cited by 37 · see citing works
“This allowed us to test how attention is biased by reward independent of explicit task goals, shedding light on a potential mechanism underlying previous observations of impaired value-based learning and decision-making after VMF damage (Zald and Andreotti, 2010; Fellows, 2011).”
BackgroundSerotonin depletion impairs both Pavlovian and instrumental reversal learning in healthy humans
Kanen, J. W., Apergis-Schoute, A. M., Yellowlees, R., Arntz, F. E., van der Flier, F. E., Price, A., Cardinal, R. N., Christmas, D. M., Clark, L., Sahakian, B. J., et al. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01240-9 · Cited by 34 · see citing works
Brain network dynamics during spontaneous strategy shifts and incremental task optimization
Allegra, M., Seyed-Allaei, S., Schuck, N. W., Amati, D., Laio, A., & Reverberi, C. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116854 · Cited by 27 · see citing works
The state of the orbitofrontal cortex
Sharpe, M. J., Wikenheiser, A. M., Niv, Y., & Schoenbaum, G. (2015) · Neuron · doi:10.1016/j.neuron.2015.12.004 · Cited by 21 · see citing works
Self-citationMahmoodi, A., Luo, S., Harbison, C., Piray, P., & Rushworth, M. F. S. (2024) · Neuron · doi:10.1016/j.neuron.2024.09.001 · Cited by 17 · see citing works
A probabilistic successor representation for context-dependent learning
Geerts, J. P., Gershman, S. J., Burgess, N., & Stachenfeld, K. L. (2024) · Psychological Review · doi:10.1037/rev0000414 · Cited by 14 · see citing works
Peterson, S., Chavira, J., Garcia Arango, J. A., Seamans, D., Cimino, E. D., & Keiflin, R. (2024) · Current Biology · doi:10.1016/j.cub.2024.10.049 · Cited by 7 · see citing works
Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning
Liu, Y., Spiering, L., Luo, S., Kingston, N., Grohn, J., & Rushworth, M. F. S. (2026) · Neuron · doi:10.1016/j.neuron.2026.07.029 · Cited by 1 · see citing works
Asaoka, N., Pagano, D., & Hayashi, Y. (2026) · Nature Communications · doi:10.1038/s41467-026-75706-1
A neuro‐behavioural model of neophobia
Dorfman, A., Subach, A., & Scharf, I. (2026) · Biological Reviews · doi:10.1002/brv.70151
- Authors
- Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y.
- Affiliations
- Princeton University (US)
- University of Maryland, Baltimore (US)
- Corresponding author
- Wilson, Robert C.
- Niv, Yael
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2014-01
- Volume
- 81
- Issue
- 2
- Pages
- 267-279
- ISSN
- 0896-6273 (ISSN-L)
- PMC
- PMC4001869
- OpenAlex
- W2119259385
- Cited by
- 885
- References
- 107
- Citations by year
- 2026: 72 · 2025: 81 · 2024: 81 · 2023: 62 · 2022: 67 · 2021: 112 · 2020: 95 · 2019: 80 · 2018: 66 · 2017: 60 · 2016: 52 · 2015: 37 · 2014: 20
- FWCI
- 31.41
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuron (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health
- National Institute of Mental Health (R01MH098861, T32MH065214)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- orbitofrontal cortex, cognitive maps, reinforcement learning, decision-making, working memory, reversal learning, delayed alternation, extinction, currency devaluation, dopaminergic neurons, ventral tegmental area
- MeSH terms
- Action Potentials, Animals, Cognition, Decision Making, Humans, Learning, Models, Psychological, Reinforcement, Psychology, Prefrontal Cortex, Ventral Tegmental Area, Dopaminergic Neurons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
