An evolutionary computational theory of prefrontal executive function in decision-making
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
The prefrontal cortex subserves executive control and decision-making, that is, the coordination and selection of thoughts and actions in the service of adaptive behaviour. We present here a computational theory describing the evolution of the prefrontal cortex from rodents to humans as gradually adding new inferential Bayesian capabilities for dealing with a computationally intractable decision problem: exploring and learning new behavioural strategies versus exploiting and adjusting previously learned ones through reinforcement learning (RL). We provide a principled account identifying three inferential steps optimizing this arbitration through the emergence of (i) factual reactive inferences in paralimbic prefrontal regions in rodents; (ii) factual proactive inferences in lateral prefrontal regions in primates and (iii) counterfactual reactive and proactive inferences in human frontopolar regions. The theory clarifies the integration of model-free and model-based RL through the notion of strategy creation. The theory also shows that counterfactual inferences in humans yield to the notion of hypothesis testing, a critical reasoning ability for approximating optimal adaptive processes and presumably endowing humans with a qualitative evolutionary advantage in adaptive behaviour.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533 · cited 3×
“…importantly, selective models may be recalibrated according to current rewarding values of action outcomes through a model-based RL process [20]: selective and predictive models serve as action and outcome predictors, respectively, for implementing covert RL, whereby outcome rewarding…”
BackgroundInfluentialDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004) · Science · doi:10.1126/science.1094285 · Cited by 2,088
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774 · cited 1×
“The concepts of default-likelihood and absolute reliability generalize the notion of expected/unexpected uncertainty proposed by Yu & Dayan [51] and provide the computational foundations of the present theory.”
BackgroundA Core System for the Implementation of Task Sets
Dosenbach, N. U. F., Visscher, K. M., Palmer, E. D., Miezin, F. M., Wenger, K. K., Kang, H. C., Burgund, E. D., Grimes, A. L., Schlaggar, B. L., & Petersen, S. E. (2006) · Neuron · doi:10.1016/j.neuron.2006.04.031 · Cited by 1,772 · cited 1×
“More generally, the ACC is involved in responding to surprising outcomes triggering behavioural switches [39] and in starting the execution of new tasks [40].”
BackgroundLearning and Memory Functions of the Basal Ganglia
Packard, M. G., & Knowlton, B. J. (2002) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.25.112701.142937 · Cited by 1,717 · cited 1×
“As generally agreed, the premotor cortex along with the dorsal striatum encodes and stores selective models of behavioural strategies [26–28], whereas the dorsal and ventral striatum implement RL adjusting the actor selective model (see above).”
BackgroundMedial prefrontal cortex as an action-outcome predictor
Alexander, W. H., & Brown, J. W. (2011) · Nature Neuroscience · doi:10.1038/nn.2921 · Cited by 1,000
Representation of Action-Specific Reward Values in the Striatum
Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005) · Science · doi:10.1126/science.1115270 · Cited by 939
Different time courses of learning-related activity in the prefrontal cortex and striatum.
Pasupathy, A., & Miller, E. K. (2005) · Nature · doi:10.1038/nature03287 · Cited by 712
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665 · cited 2×
“Consisten-tly, neuroimaging studies show that the lFPC is engaged in cognitive branching, that is, holding on the execution of one task during the performance of another task [10,49] and in monitoring the opportunity to switch to alternative courses of action [34].”
BackgroundKolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629 · cited 1×
“…and human neuroimaging studies indicate that the ACC is involved in monitoring when to switch from exploitation to exploration behaviours [35,36], whereas adjacentmedial (pre)supplementary motor regions are involved in inhibiting established behavioural responses [37] and promoting…”
BackgroundNeural Activity in the Primate Prefrontal Cortex during Associative Learning
Asaad, W. F., Rainer, G., & Miller, E. K. (1998) · Neuron · doi:10.1016/s0896-6273(00)80658-3 · Cited by 593
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558 · cited 3×
“Collins & Koechlin [25] showed that this computational algorithm predicts human choices in recurrent and new situations featuring uncertain and variable contingencies possibly associated with the occurrences of contextual cues.”
BackgroundInfluentialSelf-citationConfidence in value-based choice
De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3279 · Cited by 534 · cited 1×
“They may be related to the psychological notions of metacognitive processes and confidence judgements [52].”
BackgroundHampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1010-06.2006 · Cited by 498
Multiple model-based reinforcement learning
Doya, K., Samejima, K., Katagiri, K.-I., & Kawato, M. (2002) · Neural Computation · doi:10.1162/089976602753712972 · Cited by 476
Neubert, F.-X., Mars, R. B., Thomas, A. G., Sallet, J., & Rushworth, M. F. S. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.012 · Cited by 416 · cited 2×
“…of the left–right asymmetry yielding to the notion of Broca’s area [3,4] subserving human language [5] and bilaterally, in its most anterior portion, a polar region [6,7] (lateral frontopolar cortex, lFPC) which apparently has no homologues in monkeys [8,9] and subserves human reasoning [10].”
BackgroundSeparate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J., & Rushworth, M. F. S. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012246107 · Cited by 340
Contributions of the striatum to learning, motivation, and performance: an associative account
Liljeholm, M., & O’Doherty, J. P. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.07.007 · Cited by 317
Separate neural substrates for skill learning and performance in the ventral and dorsal striatum
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007) · Nature Neuroscience · doi:10.1038/nn1817 · Cited by 241
Prefrontal–parietal function: from foraging to foresight
Genovesio, A., Wise, S. P., & Passingham, R. E. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.11.007 · Cited by 137
Cited by
Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition
Pezzulo, G., & Cisek, P. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.013 · Cited by 419
Managing competing goals — a key role for the frontopolar cortex
Mansouri, F. A., Koechlin, E., Rosa, M. G. P., & Buckley, M. J. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2017.111 · Cited by 327
Self-citationAbstract context representations in primate amygdala and prefrontal cortex
Saez, A., Rigotti, M., Ostojic, S., Fusi, S., & Salzman, C. D. (2015) · Neuron · doi:10.1016/j.neuron.2015.07.024 · Cited by 183
“…contexts, or if a task employs unreliable feedback, immediate switching becomes impossible, and exploration is necessary to figure out the new context (Koechlin, 2014; Yu and Dayan, 2005) (also see Collins and Koechlin, 2012; Donoso et al., 2014) (these papers refer to a context as a ‘‘task set’’).”
BackgroundA psychoengineering paradigm for the neurocognitive mechanisms of biofeedback and neurofeedback
Gaume, A., Vialatte, A., Mora-Sánchez, A., Ramdani, C., & Vialatte, F. B. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.06.012 · Cited by 133
Curiosity as a Metacognitive Feeling
Goupil, L., & Proust, J. (2023) · Cognition · doi:10.1016/j.cognition.2022.105325 · Cited by 89
Metadata
- Authors
- Koechlin, E.
- Venue
- Philosophical Transactions of the Royal Society B: Biological Sciences
- Publisher
- The Royal Society
- Published
- 2014-11-05
- Volume
- 369
- Issue
- 1655
- Pages
- 20130474
- ISSN
- 0962-8436 (ISSN-L)
- 1471-2970
- OpenAlex
- W1993637338
- Cited by
- 83
- References
- 20
- FWCI
- 4.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.