The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex
Open accessAbstract
Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available-a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Meeting of minds: the medial frontal cortex and social cognition
Amodio, D. M., & Frith, C. D. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1884 · Cited by 3,782 · cited 1×
“In the social domain, two recent investigations [15,21] have revealed signed prediction error responses in the superior temporal sulcus (STS) and dorsomedial prefrontal cortex (DMPFC)—brain regions implicated in theory of mind tasks [52,53]—when participants have to learn about the behavior of another individual.”
BackgroundA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
Self-citationAdaptive Coding of Reward Value by Dopamine Neurons
Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005) · Science · doi:10.1126/science.1105370 · Cited by 1,276 · cited 1×
“The discrepancy between the two, called the prediction error, is thought to be fundamental for learning because it partly determines the degree to which future reward expectations for the choice should be revised [6,7].”
BackgroundAnterior prefrontal cortex: insights into function from anatomy and neuroimaging
Ramnani, N., & Owen, A. M. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1343 · Cited by 1,170
Associative learning of social value
Behrens, T. E. J., Hunt, L. T., Woolrich, M. W., & Rushworth, M. F. S. (2008) · Nature · doi:10.1038/nature07538 · Cited by 925
Self-citationUniquely human social cognition
Saxe, R. (2006) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2006.03.001 · Cited by 919
Choice, uncertainty and value in prefrontal and cingulate cortex
Rushworth, M. F. S., & Behrens, T. E. J. (2008) · Nature Neuroscience · doi:10.1038/nn2066 · Cited by 866
Self-citationOptimized EPI for fMRI studies of the orbitofrontal cortex
Deichmann, R., Gottfried, J. A., Hutton, C., & Turner, R. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00073-9 · Cited by 821
Beckmann, M., Johansen-Berg, H., & Rushworth, M. F. S. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3328-08.2009 · Cited by 789
Self-citationNeural repetition suppression reflects fulfilled perceptual expectations
Summerfield, C., Trittschuh, E. H., Monti, J. M., Mesulam, M.-M., & Egner, T. (2008) · Nature Neuroscience · doi:10.1038/nn.2163 · Cited by 692
Boorman, E. D., Behrens, T. E. J., Woolrich, M. W., & Rushworth, M. F. S. (2009) · Neuron · doi:10.1016/j.neuron.2009.05.014 · Cited by 665
Self-citationThe Computation of Social Behavior
Behrens, T. E. J., Hunt, L. T., & Rushworth, M. F. S. (2009) · Science · doi:10.1126/science.1169694 · Cited by 407
Self-citationActive maintenance in prefrontal area 46 creates distractor-resistant memory
Sakai, K., Rowe, J. B., & Passingham, R. E. (2002) · Nature Neuroscience · doi:10.1038/nn846 · Cited by 390
Gläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387
A Dual Role for Prediction Error in Associative Learning
den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R., & Stephan, K. E. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn161 · Cited by 306
Research works printed but not linked: 38
- H Seo, 2008, Cortical mechanisms for reinforcement learning in competitive games., Philos Trans R Soc Lond B Biol Sci, 363, 3845, 10.1098/rstb.2008.0158 · DOI
- G Schoenbaum, 2009, A new perspective on the role of the orbitofrontal cortex in adaptive behaviour., Nat Rev Neurosci, 10, 885, 10.1038/nrn2753 · DOI · Paper page
- J. W Kable, 2009, The neurobiology of decision: consensus and controversy., Neuron, 63, 733, 10.1016/j.neuron.2009.09.003 · DOI · Paper page
- E Koechlin, 2007, Anterior prefrontal function and the limits of human decision-making., Science, 318, 594, 10.1126/science.1142995 · DOI · Paper page
- S Charron, 2010, Divided representation of concurrent goals in the human frontal lobes., Science, 328, 360, 10.1126/science.1183614 · DOI · Paper page
- T Lohrenz, 2007, Neural signature of fictive learning signals in a sequential investment task., Proc Natl Acad Sci U S A, 104, 9493, 10.1073/pnas.0608842104 · DOI · Paper page
- B. Y Hayden, 2009, Fictive reward signals in the anterior cingulate cortex., Science, 324, 948, 10.1126/science.1168488 · DOI · Paper page
- C. F Camerer, 1999, Experience-Weighted Attraction (EWA) learning in normal-form games., Econometrica, 67, 827, 10.1111/1468-0262.00054 · DOI · Paper page
- M. F Rushworth, 2009, General mechanisms for making decisions?, Curr Opin Neurobiol, 19, 75, 10.1016/j.conb.2009.02.005 · DOI
- C Summerfield, 2008, A neural representation of prior information during perceptual inference., Neuron, 59, 336, 10.1016/j.neuron.2008.05.021 · DOI · Paper page
- A. N Hampton, 2008, Neural correlates of mentalizing-related computations during strategic interactions in humans., Proc Natl Acad Sci U S A, 105, 6741, 10.1073/pnas.0711099105 · DOI · Paper page
- H. E. M den Ouden, 2009, Striatal prediction error activity drives cortical connectivity changes during associative learning., Hum Brain Mapp Abstracts
- N Camille, 2004, The involvement of the orbitofrontal cortex in the experience of regret., Science, 304, 1167, 10.1126/science.1094550 · DOI · Paper page
- G Coricelli, 2005, Regret and its avoidance: a neuroimaging study of choice behavior., Nat Neurosci, 8, 1255, 10.1038/nn1514 · DOI · Paper page
- A Rangel, 2010, Neural computations associated with goal-directed choice., Curr Opin Neurobiol, 20, 262, 10.1016/j.conb.2010.03.001 · DOI · Paper page
- E Koechlin, 1999, The role of the anterior prefrontal cortex in human cognition., Nature, 399, 148, 10.1038/20178 · DOI · Paper page
- K Sakai, 2006, Prefrontal set activity predicts rule-specific neural processing during subsequent cognitive performance., J Neurosci, 26, 1211, 10.1523/JNEUROSCI.3887-05.2006 · DOI · Paper page
- P. W Burgess, 2003, The role of the rostral frontal cortex (area 10) in prospective memory: a lateral versus medial dissociation., Neuropsychologia, 41, 906, 10.1016/S0028-3932(02)00327-5 · DOI · Paper page
- J. S Simons, 2006, Differential components of prospective memory? Evidence from fMRI., Neuropsychologia, 44, 1388, 10.1016/j.neuropsychologia.2006.01.005 · DOI · Paper page
- C. S Soon, 2008, Unconscious determinants of free decisions in the human brain., Nat Neurosci, 11, 543, 10.1038/nn.2112 · DOI · Paper page
- J. B Rowe, 2007, Is the prefrontal cortex necessary for establishing cognitive sets?, J Neurosci, 27, 13303, 10.1523/JNEUROSCI.2349-07.2007 · DOI
- S Tsujimoto, 2010, Evaluating self-generated decisions in frontal pole cortex of monkeys., Nat Neurosci, 13, 120, 10.1038/nn.2453 · DOI · Paper page
- E Koechlin, 2000, Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning., Proc Natl Acad Sci U S A, 97, 7651, 10.1073/pnas.130177397 · DOI · Paper page
- J Parvizi, 2006, Neural connections of the posteromedial cortex in the macaque., Proc Natl Acad Sci U S A, 103, 1563, 10.1073/pnas.0507729103 · DOI · Paper page
- M Petrides, 2007, Efferent association pathways from the rostral prefrontal cortex in the macaque monkey., J Neurosci, 27, 11573, 10.1523/JNEUROSCI.2419-07.2007 · DOI · Paper page
- J. M Pearson, 2009, Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task., Curr Biol, 19, 1532, 10.1016/j.cub.2009.07.048 · DOI · Paper page
- T Jubault, 2007, Serial organization of human behavior in the inferior parietal cortex., J Neurosci, 27, 11028, 10.1523/JNEUROSCI.1986-07.2007 · DOI
- T. H FitzGerald, 2009, The role of human orbitofrontal cortex in value comparison for incommensurable objects., J Neurosci, 29, 8388, 10.1523/JNEUROSCI.0717-09.2009 · DOI · Paper page
- N. D Daw, 2006, Cortical substrates for exploratory decisions in humans., Nature, 441, 876, 10.1038/nature04766 · DOI · Paper page
- A. N Hampton, 2006, The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans., J Neurosci, 26, 8360, 10.1523/JNEUROSCI.1010-06.2006 · DOI · Paper page
- P. H Chiu, 2008, Smokers' brains compute, but ignore, a fictive error signal in a sequential investment task., Nat Neurosci, 11, 514, 10.1038/nn2067 · DOI
- J. F Bates, 1993, Prefrontal connections of medial motor areas in the rhesus monkey., J Comp Neurol, 336, 211, 10.1002/cne.903360205 · DOI · Paper page
- K Friston, 2005, A theory of cortical responses., Philos Trans R Soc Lond B Biol Sci, 360, 815, 10.1098/rstb.2005.1622 · DOI · Paper page
- B Seymour, 2008, Anchors, scales and the relative coding of value in the brain., Curr Opin Neurobiol, 18, 173, 10.1016/j.conb.2008.07.010 · DOI
- M. W Woolrich, 2001, Temporal autocorrelation in univariate linear modeling of FMRI data., Neuroimage, 14, 1370, 10.1006/nimg.2001.0931 · DOI · Paper page
- M Jenkinson, 2001, A global optimisation method for robust affine registration of brain images., Med Image Anal, 5, 143, 10.1016/S1361-8415(01)00036-6 · DOI · Paper page
- C. F Beckmann, 2003, General multilevel linear modeling for group analysis in FMRI., Neuroimage, 20, 1052, 10.1016/S1053-8119(03)00435-X · DOI · Paper page
- M. W Woolrich, 2004, Multilevel linear modelling for FMRI group analysis using Bayesian inference., Neuroimage, 21, 1732, 10.1016/j.neuroimage.2003.12.023 · DOI · Paper page
Cited by
Frontal cortex and reward-guided learning and decision-making
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011) · Neuron · doi:10.1016/j.neuron.2011.05.014 · Cited by 1,111
Self-citationNeurophysiology of performance monitoring and adaptive behavior
Ullsperger, M., Danielmeier, C., & Jocham, G. (2014) · Physiological Reviews · doi:10.1152/physrev.00041.2012 · Cited by 634
Kolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
Self-citationReasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558
Neural Basis of Reinforcement Learning and Decision Making
Lee, D., Seo, H., & Jung, M. W. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150512 · Cited by 546
Metadata
- Authors
- Boorman, E. D., Behrens, T. E., & Rushworth, M. F.
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-06-28
- Volume
- 9
- Issue
- 6
- Pages
- e1001093
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- OpenAlex
- W2012319162
- Cited by
- 218
- References
- 18
- FWCI
- 7.69
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.