Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex
Abstract
Although economic theories based on utility maximization account for a range of choice behaviors, utilities must be estimated through experience. Dynamics of this learning process may account for certain discrepancies between the predictions of economic theories and real choice behaviors of humans and other animals. To understand the neural mechanisms responsible for such adaptive decision making, we trained rhesus monkeys to play a simulated matching pennies game. Small but systematic deviations of the animal's behavior from the optimal strategy were consistent with the predictions of reinforcement learning theory. In addition, individual neurons in the dorsolateral prefrontal cortex (DLPFC) encoded 3 different types of signals that can potentially influence the animal's future choices. First, activity modulated by the animal's previous choices might provide the eligibility trace that can be used to attribute a particular outcome to its causative action. Second, activity related to the animal's rewards in the previous trials might be used to compute an average reward rate. Finally, activity of some neurons was modulated by the computer's choices in the previous trials and may reflect the process of updating the value functions. These results suggest that the DLPFC might be an important node in the cortical network of decision making.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Neural activity in all three cortical areas carries the information of past reward events.
“Neural activity in all three cortical areas carries the information of past reward events19-22.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [20]
In the matching pennies task, the animals made their decisions largely on the basis of their reward history.
“Although the global optimal strategy for the matching pennies task is to choose randomly and therefore does not require memory, the animals made their decisions largely on the basis of their reward history11,19-22.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [20]
Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions.
“Although the overall performance of monkeys was nearly optimal, their trial-by-trial decisions, locally in time, were influenced by previous rewards and actions11,19-22.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [20]
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524 · see citing works
Prospect theory: An analysis of decision under risk
Kahneman, D. J., & Tversky, A. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511609220.014 · Cited by 32,700 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
System Identification: Theory for the User
Lennart Ljung, A. (1987) · Cited by 17,690 · see citing works
Theory of games and economic behavior
Garcia, S. R., & Miller, S. J. (2019) · American Mathematical Society eBooks · doi:10.1090/mbk/121/32 · Cited by 16,777 · see citing works
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · see citing works
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472 · see citing works
Cellular basis of working memory
Goldman-Rakic, P. S. (1995) · Neuron · doi:10.1016/0896-6273(95)90304-6 · Cited by 2,663 · see citing works
Behavioral Theories and the Neurophysiology of Reward
Schultz, W. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.56.091103.070229 · Cited by 1,637 · see citing works
In all likelihood : statistical modelling and inference using likelihood
Pawitan, Y. (2001) · Cited by 1,068 · see citing works
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939 · see citing works
Quantitative Variation of Incentive and Performance in the White Rat
Crespi, L. P. (1942) · The American Journal of Psychology · doi:10.2307/1417120 · Cited by 894 · see citing works
Prefrontal cortex and decision making in a mixed-strategy game
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004) · Nature Neuroscience · doi:10.1038/nn1209 · Cited by 678 · see citing works
Self-citationAn experimental study of representative factors in monkeys.
Tinklepaugh, O. L. (1928) · Journal of comparative psychology · doi:10.1037/h0075798 · Cited by 578 · see citing works
Adaptation-level as a basis for a quantitative theory of frames of reference.
Helson, H. (1948) · Psychological Review · doi:10.1037/h0056721 · Cited by 571 · see citing works
Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior
Bair, W., Zohary, E., & Newsome, W. T. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-05-01676.2001 · Cited by 515 · see citing works
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485 · see citing works
The computational neurobiology of learning and reward
Daw, N. D., & Doya, K. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.006 · Cited by 462 · see citing works
Average reward reinforcement learning: Foundations, algorithms, and empirical results
Mahadevan, S. (1996) · Machine Learning · doi:10.1007/bf00114727 · Cited by 386 · see citing works
Lee, D., Port, N., Kruse, W., et al. (1998) · Journal of Neuroscience · doi:10.1523/jneurosci.18-03-01161.1998 · Cited by 326 · see citing works
Self-citationIncentive contrast: A review of behavioral changes following shifts in reward
Flaherty, C. F. (1982) · Animal Learning & Behavior · doi:10.3758/bf03212282 · Cited by 317 · see citing works
Response latency as a function of the amount of reinforcement.
Zeaman, D. (1949) · Journal of Experimental Psychology · doi:10.1037/h0060477 · Cited by 265 · see citing works
LINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS
Corrado, G. S., Sugrue, L. P., Seung, H. S., & Newsome, W. T. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.23-05 · Cited by 206 · see citing works
Reinforcement learning and decision making in monkeys during a competitive game
Lee, D., Conroy, M. L., McGreevy, B. P., & Barraclough, D. J. (2004) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2004.07.007 · Cited by 156 · see citing works
Self-citationGoldberg, M. E., Bushnell, M. C., & Bruce, C. J. (1986) · Experimental Brain Research · doi:10.1007/bf00237584 · Cited by 155 · see citing works
Activity in prefrontal cortex during dynamic selection of action sequences
Averbeck, B. B., Sohn, J.-W., & Lee, D. (2006) · Nature Neuroscience · doi:10.1038/nn1634 · Cited by 141 · see citing works
Self-citationA Biophysically Based Neural Model of Matching Law Behavior: Melioration by Stochastic Synapses
Soltani, A., & Wang, X. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.5159-05.2006 · Cited by 127 · see citing works
Learning and decision making in monkeys during a rock–paper–scissors game
Lee, D., McGreevy, B. P., & Barraclough, D. J. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.07.003 · Cited by 123 · see citing works
Self-citationSaito, N., Mushiake, H., Sakamoto, K., Itoyama, Y., & Tanji, J. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhi032 · Cited by 121 · see citing works
Rapid decision threshold modulation by reward rate in a neural network
Simen, P., Cohen, J. D., & Holmes, P. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.038 · Cited by 116 · see citing works
Neural basis of quasi-rational decision making
Lee, D. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.02.001 · Cited by 98 · see citing works
Self-citationDifferent forms of blinks and their two-stage control
Manning, K. A., & Evinger, C. L. (1986) · Experimental Brain Research · doi:10.1007/bf00340495 · Cited by 87 · see citing works
Neural mechanism for stochastic behaviour during a competitive game
Soltani, A., Lee, D., & Wang, X.-J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.044 · Cited by 77 · see citing works
Self-citationEffects of reward expectancy on sequential eye movements in monkeys
Sohn, J.-W., & Lee, D. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.04.005 · Cited by 21 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Neuronal Reward and Decision Signals: From Theories to Data
Schultz, W. (2015) · Physiological Reviews · doi:10.1152/physrev.00023.2014 · Cited by 1,126 · see citing works
A hierarchy of intrinsic timescales across primate cortex
Murray, J. D., Bernacchia, A., Freedman, D. J., Romo, R., Wallis, J. D., Cai, X., Padoa-Schioppa, C., Pasternak, T., Seo, H., Lee, D., et al. (2014) · Nature Neuroscience · doi:10.1038/nn.3862 · Cited by 990 · see citing works
Self-citationOrbitofrontal Cortex as a Cognitive Map of Task Space
Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., & Niv, Y. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.005 · Cited by 885 · see citing works
Learning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475 · see citing works
Neurons in the frontal lobe encode the value of multiple decision variables
Kennerley, S. W., Dahmubed, A. F., Lara, A. H., & Wallis, J. D. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21100 · Cited by 453 · see citing works
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H., & Rushworth, M. F. S. (2010) · Neuron · doi:10.1016/j.neuron.2010.02.027 · Cited by 397 · see citing works
Plassmann, H., O'Doherty, J. P., & Rangel, A. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0788-10.2010 · Cited by 356 · see citing works
Seo, H., & Lee, D. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2369-07.2007 · Cited by 324 · see citing works
Self-citationNeural computations underlying action-based decision making in the human brain
Wunderlich, K., Rangel, A., & O'Doherty, J. P. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0901077106 · Cited by 301 · see citing works
A reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292 · see citing works
Self-citationInterplay of approximate planning strategies
Huys, Q. J. M., Lally, N., Faulkner, P., Eshel, N., Seifritz, E., Gershman, S. J., Dayan, P., & Roiser, J. P. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1414219112 · Cited by 207 · see citing works
Beyond working memory: the role of persistent activity in decision making
Curtis, C. E., & Lee, D. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.03.006 · Cited by 191 · see citing works
Self-citationHeterogeneous reward signals in prefrontal cortex
Wallis, J. D., & Kennerley, S. W. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.02.009 · Cited by 186 · see citing works
A neural circuit covarying with social hierarchy in macaques
Noonan, M. P., Sallet, J., Mars, R. B., Neubert, F. X., O'Reilly, J. X., Andersson, J. L., Mitchell, A. S., Bell, A. H., Miller, K. L., & Rushworth, M. F. S. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001940 · Cited by 183 · see citing works
Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game
Seo, H., Barraclough, D. J., & Lee, D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.1479-09.2009 · Cited by 125 · see citing works
Self-citationFunctional neuroimaging of intertemporal choice models: a review
Carter, R. M., Meyer, J. R., & Huettel, S. A. (2010) · Journal of Neuroscience Psychology and Economics · doi:10.1037/a0018046 · Cited by 124 · see citing works
Asaad, W. F., & Eskandar, E. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3793-11.2011 · Cited by 105 · see citing works
Reward-Guided Learning with and without Causal Attribution
Jocham, G., Brodersen, K. H., Constantinescu, A. O., Kahn, M. C., Ianni, A. M., Walton, M. E., Rushworth, M. F. S., & Behrens, T. E. J. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.018 · Cited by 92 · see citing works
Noradrenergic α1-Adrenoceptor Actions in the Primate Dorsolateral Prefrontal Cortex
Datta, D., Yang, S.-T., Galvin, V. C., Solder, J., Luo, F., Morozov, Y. M., Arellano, J., Duque, A., Rakic, P., Arnsten, A. F. T., et al. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.2472-18.2019 · Cited by 44 · see citing works
Baxter, M. G., Gaffan, D., Kyriazis, D. A., & Mitchell, A. S. (2008) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2008.06353.x · Cited by 37 · see citing works
Valuation of uncertain and delayed rewards in primate prefrontal cortex
Kim, S., Hwang, J., Seo, H., & Lee, D. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.010 · Cited by 37 · see citing works
Self-citation
- Authors
- Seo, H., Barraclough, D. J., & Lee, D.
- Affiliations
- Yale University (US)
- University of Rochester (US)
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-06-04
- Volume
- 17
- Issue
- suppl 1
- Pages
- i110-i117
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2165440754
- Cited by
- 143
- References
- 34
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 1 · 2023: 4 · 2022: 4 · 2021: 4 · 2020: 4 · 2019: 9 · 2018: 10 · 2017: 9 · 2016: 8 · 2015: 10 · 2014: 10 · 2013: 5 · 2012: 4
- FWCI
- 6.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Cerebral Cortex (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (MH073246, R01MH073246)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurotransmitter Receptor Influence on Behavior
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- dorsolateral prefrontal cortex, adaptive decision-making, reinforcement learning, eligibility traces, rhesus monkeys, utility maximization, neural basis of decision making
- MeSH terms
- Animals, Choice Behavior, Electrophysiology, Female, Learning, Macaca mulatta, Male, Models, Neurological, Photic Stimulation, Psychomotor Performance, Reinforcement, Psychology, Reward, Saccades, Prefrontal Cortex
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
