The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Rostrolateral Prefrontal Cortex and Individual Differences in Uncertainty-Driven Exploration
Open accessAbstract
How do individuals decide to act based on a rewarding status quo versus an unexplored choice that might yield a better outcome? Recent evidence suggests individuals may strategically explore as a function of the relative uncertainty about the expected value of options. However, the neural mechanisms supporting uncertainty-driven exploration remain underspecified. The present fMRI study scanned a reinforcement learning task in which participants stop a rotating clock hand in order to win points. Reward schedules were such that expected value could increase, decrease, or remain constant with respect to time. We fit several mathematical models to subject behavior to generate trial-by-trial estimates of exploration as a function of relative uncertainty. These estimates were used to analyze our fMRI data. Results indicate that rostrolateral prefrontal cortex tracks trial-by-trial changes in relative uncertainty, and this pattern distinguished individuals who rely on relative uncertainty for their exploratory decisions versus those who do not.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Cortical Surface-Based Analysis
Dale, A. M., Fischl, B., & Sereno, M. I. (1999) · NeuroImage · doi:10.1006/nimg.1998.0395 · Cited by 10,910
“Risk, Ambiguity, and the Savage Axioms.”
Ellsberg, D. (1961) · The Quarterly Journal of Economics · doi:10.2307/1884324 · Cited by 7,891
Cortical Surface-Based Analysis
Fischl, B., Sereno, M. I., & Dale, A. M. (1999) · NeuroImage · doi:10.1006/nimg.1998.0396 · Cited by 6,147
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Cortical substrates for exploratory decisions in humans
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006) · Nature · doi:10.1038/nature04766 · Cited by 2,224
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
The Architecture of Cognitive Control in the Human Prefrontal Cortex
Koechlin, E., Ody, C., & Kouneiher, F. (2003) · Science · doi:10.1126/science.1088545 · Cited by 1,838
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774
Practice-related Changes in Human Brain Functional Anatomy during Nonmotor Learning
Raichle, M. E., Fiez, J. A., Videen, T. O., MacLeod, A.-M. K., Pardo, J. V., Fox, P. T., & Petersen, S. E. (1994) · Cerebral Cortex · doi:10.1093/cercor/4.1.8 · Cited by 1,593
Temporal difference models and reward-related learning in the human brain
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003) · Neuron · doi:10.1016/s0896-6273(03)00169-7 · Cited by 1,522
An information theoretical approach to prefrontal executive function
Koechlin, E., & Summerfield, C. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.04.005 · Cited by 1,183
Anterior prefrontal cortex: insights into function from anatomy and neuroimaging
Ramnani, N., & Owen, A. M. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1343 · Cited by 1,170
Cohen, J. D., McClure, S. M., & Yu, A. J. (2007) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2098 · Cited by 1,121
The role of the anterior prefrontal cortex in human cognition
Koechlin, E., Basso, G., Pietrini, P., Panzer, S., & Grafman, J. (1999) · Nature · doi:10.1038/20178 · Cited by 1,054
Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes
Badre, D. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.004 · Cited by 980
Self-citationTemporal Prediction Errors in a Passive Learning Task Activate Human Striatum
McClure, S. M., Berns, G. S., & Montague, P. R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00154-5 · Cited by 917
Neural Signatures of Economic Preferences for Risk and Ambiguity
Huettel, S. A., Stowe, C. J., Gordon, E. M., Warner, B. T., & Platt, M. L. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.024 · Cited by 712
Transient neural activity in human parietal cortex during spatial attention shifts
Yantis, S., Schwarzbach, J., Serences, J. T., Carlson, R. L., Steinmetz, M. A., Pekar, J. J., & Courtney, S. M. (2002) · Nature Neuroscience · doi:10.1038/nn921 · Cited by 710
Functional Specialization within Rostral Prefrontal Cortex (Area 10): A Meta-analysis
Gilbert, S. J., Spengler, S., Simons, J. S., Steele, J. D., Lawrie, S. M., Frith, C. D., & Burgess, P. W. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.6.932 · Cited by 683
Christoff, K., & Gabrieli, J. D. E. (2000) · Psychobiology · doi:10.3758/bf03331976 · Cited by 682
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665
Rostrolateral Prefrontal Cortex Involvement in Relational Integration during Reasoning
Christoff, K., Prabhakaran, V., Dorfman, J., Zhao, Z., Kroger, J. K., Holyoak, K. J., & Gabrieli, J. D. E. (2001) · NeuroImage · doi:10.1006/nimg.2001.0922 · Cited by 650
Kroger, J. K. (2002) · Cerebral Cortex · doi:10.1093/cercor/12.5.477 · Cited by 486
DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS
Lau, B., & Glimcher, P. W. (2005) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2005.110-04 · Cited by 485
Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009) · Nature Neuroscience · doi:10.1038/nn.2342 · Cited by 471
Self-citationThe Role of Frontopolar Cortex in Subgoal Processing during Working Memory
Braver, T. S., & Bongiolatti, S. R. (2002) · NeuroImage · doi:10.1006/nimg.2001.1019 · Cited by 409
Schönberg, T., Daw, N. D., Joel, D., & O'Doherty, J. P. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2496-07.2007 · Cited by 390
Bunge, S. A. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh126 · Cited by 358
Cortical mechanisms of space-based and object-based attentional control
Yantis, S., & Serences, J. T. (2003) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(03)00033-3 · Cited by 344
Frank, M. J., & Badre, D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr114 · Cited by 337
Self-citationEvaluating Self-Generated Information: Anterior Prefrontal Contributions to Human Cognition.
Christoff, K., Ream, J. M., Geddes, L. P. T., & Gabrieli, J. D. E. (2003) · Behavioral Neuroscience · doi:10.1037/0735-7044.117.6.1161 · Cited by 306
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267
Testing the Reward Prediction Error Hypothesis with an Axiomatic Model
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1747-10.2010 · Cited by 244
Prefrontal Coding of Temporally Discounted Values during Intertemporal Choice
Kim, S., Hwang, J., & Lee, D. (2008) · Neuron · doi:10.1016/j.neuron.2008.05.010 · Cited by 239
Strauss, G. P., Frank, M. J., Waltz, J. A., Kasanova, Z., Herbener, E. S., & Gold, J. M. (2011) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.10.015 · Cited by 233
Self-citationMechanisms of Hierarchical Reinforcement Learning in Cortico-Striatal Circuits 2: Evidence from fMRI
Badre, D., & Frank, M. J. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr117 · Cited by 209
Self-citationFrontal pole cortex: encoding ends at the end of the endbrain
Tsujimoto, S., Genovesio, A., & Wise, S. P. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.02.001 · Cited by 206
Resolution of Uncertainty in Prefrontal Cortex
Yoshida, W., & Ishii, S. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.006 · Cited by 197
Reinforcement learning, conditioning, and the brain: Successes and challenges
Maia, T. V. (2009) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.9.4.343 · Cited by 187
Frank, M. J., Scheres, A., & Sherman, S. J. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2058 · Cited by 166
Self-citationA Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism
Moustafa, A. A., Cohen, M. X., Sherman, S. J., & Frank, M. J. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.3116-08.2008 · Cited by 156
Self-citationExploration Bonuses and Dual Control
Dayan, P., & B, T. J. S. A. (1996) · Machine Learning · doi:10.1023/a:1018357105171 · Cited by 120
Gershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118
A hierarchy for relational reasoning in the prefrontal cortex
Krawczyk, D. C., McClelland, M. M., & Donovan, C. M. (2010) · Cortex · doi:10.1016/j.cortex.2010.04.008 · Cited by 93
Handbook of Reward and Decision Making
Dreher, J., & Tremblay, L. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.x0001-5 · Cited by 88
Jepma (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00170 · Cited by 45
The basal ganglia in reward and decision making
Doll, B. B., & Frank, M. J. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374620-7.00019-4 · Cited by 30
Self-citationAbstract relational categories, graded persistence, and prefrontal cortical representation
Speed, A. (2010) · Cognitive Neuroscience · doi:10.1080/17588921003660728 · Cited by 28
Han, S., Huettel, S. A., & Dobbins, I. G. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21060 · Cited by 21
Comparing the Bird in the Hand with the Ones in the Bush
Bunge, S. A., & Wendelken, C. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.020 · Cited by 17
Jepma, M., Beek, E. T. T., Wagenmakers, E., Gerven, J. M. A. V., & Nieuwenhuis, S. (2010) · Leiden Repository (Leiden University) · Cited by 12
Wallis, J. D. (2009) · Nature Neuroscience · doi:10.1038/nn0110-7 · Cited by 7
Xiong (1999) · doi:10.1002/(sici)1097-0193(1999)8:2/3
Cited by
Emotion and the prefrontal cortex: an integrative review
Dixon, M. L., Thiruchselvam, R., Todd, R., & Christoff, K. (2017) · Psychological Bulletin · doi:10.1037/bul0000096 · Cited by 745
Neural computations underlying arbitration between model-based and model-free learning
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.028 · Cited by 642
Frontal Cortex and the Hierarchical Control of Behavior
Badre, D., & Nee, D. E. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.005 · Cited by 611
Self-citationLearning, reward, and decision making
O'Doherty, J. P., Cockburn, J., & Pauli, W. M. (2017) · Annual Review of Psychology · doi:10.1146/annurev-psych-010416-044216 · Cited by 475
Contributions of the striatum to learning, motivation, and performance: an associative account
Liljeholm, M., & O’Doherty, J. P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.007 · Cited by 317
Metadata
- Authors
- Badre, D., Doll, B. B., Long, N. M., & Frank, M. J.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2012-02
- Volume
- 73
- Issue
- 3
- Pages
- 595-607
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W2098613458
- Cited by
- 309
- References
- 55
- FWCI
- 7.37
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.