Hierarchical Error Representation: A Computational Model of Anterior Cingulate and Dorsolateral Prefrontal Cortex
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Anterior cingulate and dorsolateral prefrontal cortex (ACC and dlPFC, respectively) are core components of the cognitive control network. Activation of these regions is routinely observed in tasks that involve monitoring the external environment and maintaining information in order to generate appropriate responses. Despite the ubiquity of studies reporting coactivation of these two regions, a consensus on how they interact to support cognitive control has yet to emerge. In this letter, we present a new hypothesis and computational model of ACC and dlPFC. The error representation hypothesis states that multidimensional error signals generated by ACC in response to surprising outcomes are used to train representations of expected error in dlPFC, which are then associated with relevant task stimuli. Error representations maintained in dlPFC are in turn used to modulate predictive activity in ACC in order to generate better estimates of the likely outcomes of actions. We formalize the error representation hypothesis in a new computational model based on our previous model of ACC. The hierarchical error representation (HER) model of ACC/dlPFC suggests a mechanism by which hierarchically organized layers within ACC and dlPFC interact in order to solve sophisticated cognitive tasks. In a series of simulations, we demonstrate the ability of the HER model to autonomously learn to perform structured tasks in a manner comparable to human performance, and we show that the HER model outperforms current deep learning networks by an order of magnitude.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Baddeley, A. D., & Hitch, G. (1974) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60452-1 · Cited by 9,860
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“Specifically, the dlPFC learns to maintain representations of stimuli that reliably co-occur with prediction errors (Friston, 2010).”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Aston-Jones, G., & Cohen, J. D. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.28.061604.135709 · Cited by 4,472
Holroyd, C. B., & Coles, M. G. H. (2002) · Psychological Review · doi:10.1037/0033-295x.109.4.679 · Cited by 4,027
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
MacDonald, A. W., Cohen, J. D., Stenger, V. A., & Carter, C. S. (2000) · Science · doi:10.1126/science.288.5472.1835 · Cited by 3,666
Anterior Cingulate Cortex, Error Detection, and the Online Monitoring of Performance
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., & Cohen, J. D. (1998) · Science · doi:10.1126/science.280.5364.747 · Cited by 3,273 · cited 1×
“The AX and 1-2AX continuous performance task (CPT) are frequently used in the study of WM and in diagnosing behavioral and cognitive deficits related to WM dysfunction (Carter et al., 1998; Servan-Schreiber, Cohen, & Steingard, 1996).”
MethodPearce, J. M., & Hall, G. (1980) · Psychological Review · doi:10.1037/0033-295x.87.6.532 · Cited by 3,023
A theory of attention: Variations in the associability of stimuli with reinforcement
Mackintosh, N. J. (1975) · Psychological Review · doi:10.1037/h0076778 · Cited by 2,733 · cited 1×
“This approach is distinct from classical models of associative learning (Mackintosh, 1975; Pearce & Hall, 1980; Rescorla, 1971) in which stimuli and their consequences are effectively presented simultaneously, that is, trial-level models.”
MethodA framework for mesencephalic dopamine systems based on predictive Hebbian learning
Montague, P., Dayan, P., & Sejnowski, T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-05-01936.1996 · Cited by 2,066
Learning the value of information in an uncertain world
Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007) · Nature Neuroscience · doi:10.1038/nn1954 · Cited by 2,053
The neural basis of error detection: Conflict monitoring and the error-related negativity
Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.931 · Cited by 1,924
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Uncertainty, Neuromodulation, and Attention
Yu, A. J., & Dayan, P. (2005) · Neuron · doi:10.1016/j.neuron.2005.04.026 · Cited by 1,774
Overlapping mechanisms of attention and spatial working memory
Awh, E., & Jonides, J. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01593-x · Cited by 1,229
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes
Badre, D. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.004 · Cited by 980
Optimal decision making and the anterior cingulate cortex
Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006) · Nature Neuroscience · doi:10.1038/nn1724 · Cited by 948
Is the rostro-caudal axis of the frontal lobe hierarchical?
Badre, D., & D'Esposito, M. (2009) · Nature Reviews Neuroscience · doi:10.1038/nrn2667 · Cited by 946
Learned predictions of error likelihood in the anterior cingulate cortex
Brown, J. W., & Braver, T. S. (2005) · Science · doi:10.1126/science.1105783 · Cited by 928
Self-citationPrefrontal Dopamine D1Receptors and Working Memory in Schizophrenia
Abi-Dargham, A., Mawlawi, O., Lombardo, I., Gil, R., Martinez, D., Huang, Y., Hwang, D.-R., Keilp, J., Kochan, L., Van Heertum, R., et al. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-09-03708.2002 · Cited by 724 · cited 1×
“Dysfunction of the DA system is implicated in a range of behavioral pathologies, including obsessive-compulsive disorder and schizophrenia (Abi-Dargham et al., 2002; Cohen & Servan-Schreiber, 1993; Denys, Zohar, & Westenberg, 2003; Goodman et al., 1990).”
BackgroundKolling, N., Behrens, T. E. J., Mars, R. B., & Rushworth, M. F. S. (2012) · Science · doi:10.1126/science.1216930 · Cited by 629
Brunel, N., & Wang, X.-J. (2001) · Journal of Computational Neuroscience · doi:10.1023/a:1011204814320 · Cited by 620
Motivation of extended behaviors by anterior cingulate cortex
Holroyd, C. B., & Yeung, N. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.008 · Cited by 618
Badre, D., & D'Esposito, M. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.2082 · Cited by 615
Motivation and cognitive control in the human prefrontal cortex
Kouneiher, F., Charron, S., & Koechlin, E. (2009) · Nature Neuroscience · doi:10.1038/nn.2321 · Cited by 590
Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making
Collins, A., & Koechlin, E. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001293 · Cited by 558
Hierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532 · cited 1×
“In general, HRL models attempt to learn to accomplish tasks composed of abstract goals, rules, and concrete elementary behaviors organized into behavioral schemata (Barto & Mahadevan, 2003; Botvinick, 2008; Botvinick, Niv, & Barto, 2009; Jong & Stone, 2008).”
BackgroundProspective coding for objects in primate prefrontal cortex
Rainer, G., Rao, S. C., & Miller, E. K. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-13-05493.1999 · Cited by 432
Cognitive control over learning: Creating, clustering, and generalizing task-set structure.
Collins, A. G. E., & Frank, M. J. (2013) · Psychological Review · doi:10.1037/a0030852 · Cited by 426
Murase, S., Grenhoff, J., Chouvet, G., Gonon, F. G., & Svensson, T. H. (1993) · Neuroscience Letters · doi:10.1016/0304-3940(93)90641-w · Cited by 387 · cited 1×
“In particular, activation in mPFC elicited either through electrical (Gariano & Groves, 1988; Taber & Fibiger, 1993) or chemical stimulation (Murase, Grenhoff, Chouvet, Gonon, & Svensson, 1993) has been observed to drive burst firing in the ventral tegmental area (VTA), a major source of dopamine (DA) projections to c…”
BackgroundRiggall, A. C., & Postle, B. R. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.1892-12.2012 · Cited by 385
Frontal cortex subregions play distinct roles in choices between actions and stimuli
Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., & Rushworth, M. F. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.3541-08.2008 · Cited by 350
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1025696116075 · Cited by 317
Williams, S. M., & Goldman-Rakic, P. S. (1993) · Cerebral Cortex · doi:10.1093/cercor/3.3.199 · Cited by 284
Contributions of the ventromedial prefrontal cortex to goal-directed action selection
O’Doherty, J. P. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06290.x · Cited by 225
Neuroimaging evidence of the anatomo-functional organization of the human cingulate motor areas
Amiez, C., & Petrides, M. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs329 · Cited by 203
Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex
Seo, H., Barraclough, D. J., & Lee, D. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm064 · Cited by 143
Nee, D. E., Kastner, S., & Brown, J. W. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.08.027 · Cited by 139
Self-citationElectrical Stimulation of the Medial Prefrontal Cortex Increases Dopamine Release in the Striatum
Taber, M. T., & Fibiger, H. C. (1993) · Neuropsychopharmacology · doi:10.1038/npp.1993.63 · Cited by 99
Cited by
Toward a rational and mechanistic account of mental effort
Shenhav, A., Musslick, S., Lieder, F., Kool, W., Griffiths, T. L., Cohen, J. D., & Botvinick, M. M. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031526 · Cited by 1,077
Frontal Cortex and the Hierarchical Control of Behavior
Badre, D., & Nee, D. E. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.005 · Cited by 611
Dorsal anterior cingulate cortex and the value of control
Shenhav, A., Cohen, J. D., & Botvinick, M. M. (2016) · Nature Neuroscience · doi:10.1038/nn.4384 · Cited by 565
Hierarchical active inference: A theory of motivated control
Pezzulo, G., Rigoli, F., & Friston, K. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.01.009 · Cited by 334
Herding Brains: A Core Neural Mechanism for Social Alignment
Shamay-Tsoory, S. G., Saporta, N., Marton-Alper, I. Z., & Gvirts, H. Z. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.01.002 · Cited by 289
Metadata
- Authors
- Alexander, W. H., & Brown, J. W.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2015-11
- Volume
- 27
- Issue
- 11
- Pages
- 2354-2410
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2167031142
- Cited by
- 116
- References
- 42
- FWCI
- 5.72
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.