CoChat

Value representations in the rodent orbitofrontal cortex drive learning, not choice

Open access
Miller, Kevin J, Botvinick, Matthew M, Brody, Carlos D · 2022 · eLife

Published version of a preprint

Cited by 66

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Abstract

Humans and animals make predictions about the rewards they expect to receive in different situations. In formal models of behavior, these predictions are known as value representations, and they play two very different roles. Firstly, they drive choice : the expected values of available options are compared to one another, and the best option is selected. Secondly, they support learning : expected values are compared to rewards actually received, and future expectations are updated accordingly. Whether these different functions are mediated by different neural representations remains an open question. Here, we employ a recently developed multi-step task for rats that computationally separates learning from choosing. We investigate the role of value representations in the rodent orbitofrontal cortex, a key structure for value-based cognition. Electrophysiological recordings and optogenetic perturbations indicate that these representations do not directly drive choice. Instead, they signal expected reward information to a learning process elsewhere in the brain that updates choice mechanisms.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.64575

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 54

  1. Akaishi, 2014, Autonomous mechanism of internal choice estimate underlies decision inertia, Neuron, 81, 195, 10.1016/j.neuron.2013.10.018 · DOI · Paper page
  2. Akam, 2020, Anterior Cingulate Cortex Represents Action-State Predictions and Causally Mediates Model-Based Reinforcement Learning in a Two-Step Decision Task, 10.1101/126292 · DOI
  3. Akrami, 2017, Posterior Parietal Cortex Represents Sensory History and Mediates Its Effects on Behavior, 10.1101/182246 · DOI
  4. Ballesta, 2020, Values encoded in orbitofrontal cortex are causally related to economic choices, Nature, 588, 450, 10.1038/s41586-020-2880-x · DOI · Paper page
  5. Blanchard, 2015, Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity, Neuron, 85, 602, 10.1016/j.neuron.2014.12.050 · DOI · Paper page
  6. Cai, 2011, Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice, Neuron, 69, 170, 10.1016/j.neuron.2010.11.041 · DOI · Paper page
  7. Chan, 2016, A probability distribution over latent causes, in the orbitofrontal cortex, The Journal of Neuroscience, 36, 7817, 10.1523/JNEUROSCI.0659-16.2016 · DOI · Paper page
  8. Constantinople, 2019, Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases, eLife, 8, 10.7554/eLife.49744 · DOI
  9. Costa, 2020, Primate orbitofrontal cortex codes information relevant for managing explore-exploit tradeoffs, The Journal of Neuroscience, 40, 2553, 10.1523/JNEUROSCI.2355-19.2020 · DOI · Paper page
  10. Daw, 2011, Model-based influences on humans’ choices and striatal prediction errors, Neuron, 69, 1204, 10.1016/j.neuron.2011.02.027 · DOI · Paper page
  11. Daw, 2014, Neuroeconomics, 393, 10.1016/B978-0-12-416008-8.00021-8 · DOI
  12. Dezfouli, 2019, Learning the structure of the world: the adaptive nature of state-space and action representations in multi-stage decision-making, PLOS Computational Biology, 15, 10.1371/journal.pcbi.1007334 · DOI
  13. Gardner, 2017, Lateral orbitofrontal inactivation dissociates devaluation-sensitive behavior and economic choice, Neuron, 96, 1192, 10.1016/j.neuron.2017.10.026 · DOI · Paper page
  14. Gardner, 2019, Real-time value integration during economic choice is regulated by orbitofrontal cortex, Current Biology, 29, 4315, 10.1016/j.cub.2019.10.058 · DOI
  15. Gardner, 2020, Processing in lateral orbitofrontal cortex is required to estimate subjective preference during initial, but not established, economic choice, Neuron, 108, 526, 10.1016/j.neuron.2020.08.010 · DOI
  16. Gelman, 2013, Bayesian Data Analysis, 10.1201/b16018 · doi:10.1201/b16018 · Paper page
  17. Groman, 2019, Neurochemical and behavioral dissections of decision-making in a rodent multistage task, The Journal of Neuroscience, 39, 295, 10.1523/JNEUROSCI.2219-18.2018 · DOI
  18. Hanks, 2015, Distinct relationships of parietal and prefrontal cortices to evidence accumulation, Nature, 520, 220, 10.1038/nature14066 · DOI · Paper page
  19. Hasz, 2018, Deliberation and procedural automation on a two-step task for rats, Frontiers in Integrative Neuroscience, 12, 10.3389/fnint.2018.00030 · DOI
  20. Jones, 2012, Orbitofrontal cortex supports behavior and learning using inferred but not cached values, Science, 338, 953, 10.1126/science.1227489 · DOI · Paper page
  21. Kennerley, 2011, Double dissociation of value computations in orbitofrontal and anterior cingulate neurons, Nature Neuroscience, 14, 1581, 10.1038/nn.2961 · DOI · Paper page
  22. Kobayashi, 2010, Adaptation of reward sensitivity in orbitofrontal neurons, The Journal of Neuroscience, 30, 534, 10.1523/JNEUROSCI.4009-09.2010 · DOI · Paper page
  23. Kopec, 2015, Cortical and subcortical contributions to short-term memory for orienting movements, Neuron, 88, 367, 10.1016/j.neuron.2015.08.033 · DOI · Paper page
  24. Kuwabara, 2020, Neural mechanisms of economic choices in mice, eLife, 9, 10.7554/eLife.49669 · DOI · Paper page
  25. Lak, 2020, Reinforcement biases subsequent perceptual decisions when confidence is low, a widespread behavioral phenomenon, eLife, 9, 10.7554/eLife.49834 · DOI
  26. Lee, 2012, Neural basis of reinforcement learning and decision making, Annual Review of Neuroscience, 35, 287, 10.1146/annurev-neuro-062111-150512 · DOI · Paper page
  27. Lombrozo, 2017, Learning by Thinking” in Science and in Everyday Life’, in The scientific imagination
  28. Ludvig, 2017, Associative Learning from Replayed Experience, 10.1101/100800 · DOI
  29. McDaniel, 2007, Prospective Memory: An Overview and Synthesis of an Emerging Field, 10.4135/9781452225913 · DOI · Paper page
  30. McDannald, 2011, Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning, The Journal of Neuroscience, 31, 2700, 10.1523/JNEUROSCI.5499-10.2011 · DOI · Paper page
  31. Miller, 2016, Identifying Model-Based and Model-Free Patterns in Behavior on Multi-Step Tasks, 10.1101/096339 · DOI
  32. Miller, 2017, Dorsal hippocampus contributes to model-based planning, Nature Neuroscience, 20, 1269, 10.1038/nn.4613 · DOI · Paper page
  33. Miller, 2021, Multi-step planning in the brain, Current Opinion in Behavioral Sciences, 38, 29, 10.1016/j.cobeha.2020.07.003 · DOI
  34. Miranda, 2020, Combined model-free and model-sensitive reinforcement learning in non-human primates, PLOS Computational Biology, 16, 10.1371/journal.pcbi.1007944 · DOI
  35. Murray, 2015, Specialized areas for value updating and goal selection in the primate orbitofrontal cortex, eLife, 4, 10.7554/eLife.11695 · DOI · Paper page
  36. O’Doherty, 2007, Lights, camembert, action! the role of human orbitofrontal cortex in encoding stimuli, rewards, and choices, Annals of the New York Academy of Sciences, 1121, 254, 10.1196/annals.1401.036 · DOI · Paper page
  37. Padoa-Schioppa, 2013, Neuronal origins of choice variability in economic decisions, Neuron, 80, 1322, 10.1016/j.neuron.2013.09.013 · DOI · Paper page
  38. Padoa-Schioppa, 2017, Orbitofrontal cortex: A neural circuit for economic decisions, Neuron, 96, 736, 10.1016/j.neuron.2017.09.031 · DOI · Paper page
  39. Paxinos, 2006, The Rat Brain in Stereotaxic Coordinates: Hard Cover Edition
  40. Qian J. 2013. Glmnet for matlab. Hastie. GPL-2. https://hastie.su.domains/glmnet_matlab/.
  41. Rich, 2014, Medial-lateral organization of the orbitofrontal cortex, Journal of Cognitive Neuroscience, 26, 1347, 10.1162/jocn_a_00573 · DOI · Paper page
  42. Rich, 2016, Decoding subjective decisions from orbitofrontal cortex, Nature Neuroscience, 19, 973, 10.1038/nn.4320 · DOI · Paper page
  43. Roesch, 2004, Neuronal activity related to reward value and motivation in primate frontal cortex, Science, 304, 307, 10.1126/science.1093223 · DOI · Paper page
  44. Rudebeck, 2013, Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex, Neuron, 80, 1519, 10.1016/j.neuron.2013.09.036 · DOI · Paper page
  45. Rustichini, 2015, A neuro-computational model of economic decisions, Journal of Neurophysiology, 114, 1382, 10.1152/jn.00184.2015 · DOI · Paper page
  46. Schoenbaum, 1998, Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning, Nature Neuroscience, 1, 155, 10.1038/407 · DOI · Paper page
  47. Schuck, 2016, Human orbitofrontal cortex represents a cognitive map of state space, Neuron, 91, 1402, 10.1016/j.neuron.2016.08.019 · DOI · Paper page
  48. Song, 2017, Reward-based training of recurrent neural networks for cognitive and value-based tasks, eLife, 6, 10.7554/eLife.21492 · DOI · Paper page
  49. Sul, 2010, Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making, Neuron, 66, 449, 10.1016/j.neuron.2010.03.033 · DOI · Paper page
  50. Takahashi, 2009, The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes, Neuron, 62, 269, 10.1016/j.neuron.2009.03.005 · DOI · Paper page
  51. Wallis, 2003, Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task, The European Journal of Neuroscience, 18, 2069, 10.1046/j.1460-9568.2003.02922.x · DOI · Paper page
  52. Wallis, 2007, Orbitofrontal cortex and its contribution to decision-making, Annual Review of Neuroscience, 30, 31, 10.1146/annurev.neuro.30.051606.094334 · DOI · Paper page
  53. Wallis, 2012, Cross-species studies of orbitofrontal cortex and value-based decision-making, Nature Neuroscience, 15, 13, 10.1038/nn.2956 · DOI · Paper page
  54. Walton, 2011, Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world, Annals of the New York Academy of Sciences, 1239, 14, 10.1111/j.1749-6632.2011.06257.x · DOI · Paper page

Cited by

See all 66

Metadata

Authors
Miller, K. J., Botvinick, M. M., & Brody, C. D.
Venue
eLife
Published
2022-08-17
Volume
11
ISSN
2050-084X (ISSN-L)
OpenAlex
W4292324303
Cited by
66
References
31
FWCI
4.43
Type
journal article
Language
en
Open access
gold
Funders
  • National Institutes of Health (T-32 MH065214)
  • Princeton University (Harold W Dodds Fellowship)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.