Value representations in the rodent orbitofrontal cortex drive learning, not choice
Open accessPublished version of a preprint
The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Humans and animals make predictions about the rewards they expect to receive in different situations. In formal models of behavior, these predictions are known as value representations, and they play two very different roles. Firstly, they drive choice : the expected values of available options are compared to one another, and the best option is selected. Secondly, they support learning : expected values are compared to rewards actually received, and future expectations are updated accordingly. Whether these different functions are mediated by different neural representations remains an open question. Here, we employ a recently developed multi-step task for rats that computationally separates learning from choosing. We investigate the role of value representations in the rodent orbitofrontal cortex, a key structure for value-based cognition. Electrophysiological recordings and optogenetic perturbations indicate that these representations do not directly drive choice. Instead, they signal expected reward information to a learning process elsewhere in the brain that updates choice mechanisms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.03.013 · Cited by 4,147
Neurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598
Visual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408
Relative reward preference in primate orbitofrontal cortex
Tremblay, L., & Schultz, W. (1999) · Nature · doi:10.1038/19525 · Cited by 1,314
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex
Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2003) · Science · doi:10.1126/science.1087919 · Cited by 1,165
Dolan, R. J., & Dayan, P. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.007 · Cited by 977
Orbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885
The orbitofrontal cortex: Neuronal activity in the behaving monkey
Thorpe, S. J., Rolls, E. T., & Maddison, S. (1983) · Experimental Brain Research · doi:10.1007/bf00235545 · Cited by 836
Gremel, C. M., & Costa, R. M. (2013) · Nature Communications · doi:10.1038/ncomms3264 · Cited by 618
Neurobiology of economic choice: a good-based model
Padoa-Schioppa, C. (2011) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-061010-113648 · Cited by 600
A new perspective on the role of the orbitofrontal cortex in adaptive behaviour
Schoenbaum, G., Roesch, M. R., Stalnaker, T. A., & Takahashi, Y. K. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2753 · Cited by 585
Choosing the greater of two goods: neural currencies for valuation and decision making
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1666 · Cited by 536
What the orbitofrontal cortex does not do
Stalnaker, T. A., Cooch, N. K., & Schoenbaum, G. (2015) · Nature Neuroscience · doi:10.1038/nn.3982 · Cited by 531
Actor–critic models of the basal ganglia: new anatomical and computational perspectives
Joel, D., Niv, Y., & Ruppin, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00047-3 · Cited by 500
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Neurons in the frontal lobe encode the value of multiple decision variables
Kennerley, S. W., Dahmubed, A. F., Lara, A. H., & Wallis, J. D. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21100 · Cited by 453
Rudebeck, P. H., & Murray, E. A. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.049 · Cited by 405
Orbitofrontal cortex, associative learning and expectancies
Schoenbaum, G., & Roesch, M. (2005) · Neuron · doi:10.1016/j.neuron.2005.07.018 · Cited by 404
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397
PRICE, J. L. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1401.008 · Cited by 395
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393
Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340
Miller, K. J., Shenhav, A., & Ludvig, E. A. (2019) · Psychological Review · doi:10.1037/rev0000120 · Cited by 280
Self-citationDoes the orbitofrontal cortex signal value?
Schoenbaum, G., Takahashi, Y., Liu, T., & McDannald, M. A. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06210.x · Cited by 232
Retrospective revaluation in sequential decision making: a tale of two systems
Gershman, S. J., Markman, A. B., & Otto, A. R. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0030844 · Cited by 217
When Does Model-Based Control Pay Off?
Kool, W., Cushman, F. A., & Gershman, S. J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005090 · Cited by 211
Value-guided remapping of sensory cortex by lateral orbitofrontal cortex
Banerjee, A., Parente, G., Teutsch, J., Lewis, C., Voigt, F. F., & Helmchen, F. (2020) · Nature · doi:10.1038/s41586-020-2704-z · Cited by 199
Rudebeck, P. H., Saunders, R. C., Lundgren, D. A., & Murray, E. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.07.042 · Cited by 170
Specializations for reward-guided decision-making in the primate ventral prefrontal cortex
Murray, E. A., & Rudebeck, P. H. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0013-4 · Cited by 169
Noonan, M. P., Chau, B. K. H., Rushworth, M. F. S., & Fellows, L. K. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.0692-17.2017 · Cited by 167
Research works printed but not linked: 54
- Akaishi, 2014, Autonomous mechanism of internal choice estimate underlies decision inertia, Neuron, 81, 195, 10.1016/j.neuron.2013.10.018 · DOI · Paper page
- Akam, 2020, Anterior Cingulate Cortex Represents Action-State Predictions and Causally Mediates Model-Based Reinforcement Learning in a Two-Step Decision Task, 10.1101/126292 · DOI
- Akrami, 2017, Posterior Parietal Cortex Represents Sensory History and Mediates Its Effects on Behavior, 10.1101/182246 · DOI
- Ballesta, 2020, Values encoded in orbitofrontal cortex are causally related to economic choices, Nature, 588, 450, 10.1038/s41586-020-2880-x · DOI · Paper page
- Blanchard, 2015, Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity, Neuron, 85, 602, 10.1016/j.neuron.2014.12.050 · DOI · Paper page
- Cai, 2011, Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice, Neuron, 69, 170, 10.1016/j.neuron.2010.11.041 · DOI · Paper page
- Chan, 2016, A probability distribution over latent causes, in the orbitofrontal cortex, The Journal of Neuroscience, 36, 7817, 10.1523/JNEUROSCI.0659-16.2016 · DOI · Paper page
- Constantinople, 2019, Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases, eLife, 8, 10.7554/eLife.49744 · DOI
- Costa, 2020, Primate orbitofrontal cortex codes information relevant for managing explore-exploit tradeoffs, The Journal of Neuroscience, 40, 2553, 10.1523/JNEUROSCI.2355-19.2020 · DOI · Paper page
- Daw, 2011, Model-based influences on humans’ choices and striatal prediction errors, Neuron, 69, 1204, 10.1016/j.neuron.2011.02.027 · DOI · Paper page
- Daw, 2014, Neuroeconomics, 393, 10.1016/B978-0-12-416008-8.00021-8 · DOI
- Dezfouli, 2019, Learning the structure of the world: the adaptive nature of state-space and action representations in multi-stage decision-making, PLOS Computational Biology, 15, 10.1371/journal.pcbi.1007334 · DOI
- Gardner, 2017, Lateral orbitofrontal inactivation dissociates devaluation-sensitive behavior and economic choice, Neuron, 96, 1192, 10.1016/j.neuron.2017.10.026 · DOI · Paper page
- Gardner, 2019, Real-time value integration during economic choice is regulated by orbitofrontal cortex, Current Biology, 29, 4315, 10.1016/j.cub.2019.10.058 · DOI
- Gardner, 2020, Processing in lateral orbitofrontal cortex is required to estimate subjective preference during initial, but not established, economic choice, Neuron, 108, 526, 10.1016/j.neuron.2020.08.010 · DOI
- Gelman, 2013, Bayesian Data Analysis, 10.1201/b16018 · doi:10.1201/b16018 · Paper page
- Groman, 2019, Neurochemical and behavioral dissections of decision-making in a rodent multistage task, The Journal of Neuroscience, 39, 295, 10.1523/JNEUROSCI.2219-18.2018 · DOI
- Hanks, 2015, Distinct relationships of parietal and prefrontal cortices to evidence accumulation, Nature, 520, 220, 10.1038/nature14066 · DOI · Paper page
- Hasz, 2018, Deliberation and procedural automation on a two-step task for rats, Frontiers in Integrative Neuroscience, 12, 10.3389/fnint.2018.00030 · DOI
- Jones, 2012, Orbitofrontal cortex supports behavior and learning using inferred but not cached values, Science, 338, 953, 10.1126/science.1227489 · DOI · Paper page
- Kennerley, 2011, Double dissociation of value computations in orbitofrontal and anterior cingulate neurons, Nature Neuroscience, 14, 1581, 10.1038/nn.2961 · DOI · Paper page
- Kobayashi, 2010, Adaptation of reward sensitivity in orbitofrontal neurons, The Journal of Neuroscience, 30, 534, 10.1523/JNEUROSCI.4009-09.2010 · DOI · Paper page
- Kopec, 2015, Cortical and subcortical contributions to short-term memory for orienting movements, Neuron, 88, 367, 10.1016/j.neuron.2015.08.033 · DOI · Paper page
- Kuwabara, 2020, Neural mechanisms of economic choices in mice, eLife, 9, 10.7554/eLife.49669 · DOI · Paper page
- Lak, 2020, Reinforcement biases subsequent perceptual decisions when confidence is low, a widespread behavioral phenomenon, eLife, 9, 10.7554/eLife.49834 · DOI
- Lee, 2012, Neural basis of reinforcement learning and decision making, Annual Review of Neuroscience, 35, 287, 10.1146/annurev-neuro-062111-150512 · DOI · Paper page
- Lombrozo, 2017, Learning by Thinking” in Science and in Everyday Life’, in The scientific imagination
- Ludvig, 2017, Associative Learning from Replayed Experience, 10.1101/100800 · DOI
- McDaniel, 2007, Prospective Memory: An Overview and Synthesis of an Emerging Field, 10.4135/9781452225913 · DOI · Paper page
- McDannald, 2011, Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning, The Journal of Neuroscience, 31, 2700, 10.1523/JNEUROSCI.5499-10.2011 · DOI · Paper page
- Miller, 2016, Identifying Model-Based and Model-Free Patterns in Behavior on Multi-Step Tasks, 10.1101/096339 · DOI
- Miller, 2017, Dorsal hippocampus contributes to model-based planning, Nature Neuroscience, 20, 1269, 10.1038/nn.4613 · DOI · Paper page
- Miller, 2021, Multi-step planning in the brain, Current Opinion in Behavioral Sciences, 38, 29, 10.1016/j.cobeha.2020.07.003 · DOI
- Miranda, 2020, Combined model-free and model-sensitive reinforcement learning in non-human primates, PLOS Computational Biology, 16, 10.1371/journal.pcbi.1007944 · DOI
- Murray, 2015, Specialized areas for value updating and goal selection in the primate orbitofrontal cortex, eLife, 4, 10.7554/eLife.11695 · DOI · Paper page
- O’Doherty, 2007, Lights, camembert, action! the role of human orbitofrontal cortex in encoding stimuli, rewards, and choices, Annals of the New York Academy of Sciences, 1121, 254, 10.1196/annals.1401.036 · DOI · Paper page
- Padoa-Schioppa, 2013, Neuronal origins of choice variability in economic decisions, Neuron, 80, 1322, 10.1016/j.neuron.2013.09.013 · DOI · Paper page
- Padoa-Schioppa, 2017, Orbitofrontal cortex: A neural circuit for economic decisions, Neuron, 96, 736, 10.1016/j.neuron.2017.09.031 · DOI · Paper page
- Paxinos, 2006, The Rat Brain in Stereotaxic Coordinates: Hard Cover Edition
- Qian J. 2013. Glmnet for matlab. Hastie. GPL-2. https://hastie.su.domains/glmnet_matlab/.
- Rich, 2014, Medial-lateral organization of the orbitofrontal cortex, Journal of Cognitive Neuroscience, 26, 1347, 10.1162/jocn_a_00573 · DOI · Paper page
- Rich, 2016, Decoding subjective decisions from orbitofrontal cortex, Nature Neuroscience, 19, 973, 10.1038/nn.4320 · DOI · Paper page
- Roesch, 2004, Neuronal activity related to reward value and motivation in primate frontal cortex, Science, 304, 307, 10.1126/science.1093223 · DOI · Paper page
- Rudebeck, 2013, Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex, Neuron, 80, 1519, 10.1016/j.neuron.2013.09.036 · DOI · Paper page
- Rustichini, 2015, A neuro-computational model of economic decisions, Journal of Neurophysiology, 114, 1382, 10.1152/jn.00184.2015 · DOI · Paper page
- Schoenbaum, 1998, Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning, Nature Neuroscience, 1, 155, 10.1038/407 · DOI · Paper page
- Schuck, 2016, Human orbitofrontal cortex represents a cognitive map of state space, Neuron, 91, 1402, 10.1016/j.neuron.2016.08.019 · DOI · Paper page
- Song, 2017, Reward-based training of recurrent neural networks for cognitive and value-based tasks, eLife, 6, 10.7554/eLife.21492 · DOI · Paper page
- Sul, 2010, Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making, Neuron, 66, 449, 10.1016/j.neuron.2010.03.033 · DOI · Paper page
- Takahashi, 2009, The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes, Neuron, 62, 269, 10.1016/j.neuron.2009.03.005 · DOI · Paper page
- Wallis, 2003, Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task, The European Journal of Neuroscience, 18, 2069, 10.1046/j.1460-9568.2003.02922.x · DOI · Paper page
- Wallis, 2007, Orbitofrontal cortex and its contribution to decision-making, Annual Review of Neuroscience, 30, 31, 10.1146/annurev.neuro.30.051606.094334 · DOI · Paper page
- Wallis, 2012, Cross-species studies of orbitofrontal cortex and value-based decision-making, Nature Neuroscience, 15, 13, 10.1038/nn.2956 · DOI · Paper page
- Walton, 2011, Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world, Annals of the New York Academy of Sciences, 1239, 14, 10.1111/j.1749-6632.2011.06257.x · DOI · Paper page
Cited by
A catalog of single nucleotide changes distinguishing modern humans from archaic hominins
Kuhlwilm, M., & Boeckx, C. (2019) · Scientific Reports · doi:10.1038/s41598-019-44877-x · Cited by 108
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64
Neural mechanisms of economic choices in mice
Kuwabara, M., Kang, N., Holy, T. E., & Padoa-Schioppa, C. (2020) · eLife · doi:10.7554/elife.49669 · Cited by 56
Peterson, S., Chavira, J., Garcia Arango, J. A., Seamans, D., Cimino, E. D., & Keiflin, R. (2024) · Current Biology · doi:10.1016/j.cub.2024.10.049 · Cited by 7
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Metadata
- Authors
- Miller, K. J., Botvinick, M. M., & Brody, C. D.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2022-08-17
- Volume
- 11
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W4292324303
- Cited by
- 66
- References
- 31
- FWCI
- 4.43
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- creativecommons.org/licenses/by/4.0/ (am)
- creativecommons.org/licenses/by/4.0/ (tdm)
- Funders
- National Institutes of Health (T-32 MH065214)
- Princeton University (Harold W Dodds Fellowship)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.