Orthogonal representations for robust context-dependent task performance in brains and neural networks
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
How do neural populations code for multiple, potentially conflicting tasks? Here we used computational simulations involving neural networks to define "lazy" and "rich" coding solutions to this context-dependent decision-making problem, which trade off learning speed for robustness. During lazy learning the input dimensionality is expanded by random projections to the network hidden layer, whereas in rich learning hidden units acquire structured representations that privilege relevant over irrelevant features. For context-dependent decision-making, one rich solution is to project task representations onto low-dimensional and orthogonal manifolds. Using behavioral testing and neuroimaging in humans and analysis of neural signals from macaque prefrontal cortex, we report evidence for neural coding patterns in biological brains whose dimensionality and neural geometry are consistent with the rich learning regime.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Monsell, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00028-7 · Cited by 2,984 · cited 2×
“Human studies havemapped the brain regions that exert control during task performance (Koechlin et al., 2003; Kerns et al., 2004; Yeung et al., 2006; Cole et al., 2016) or measured the processing costs incurred by task switches (Monsell, 2003; Brown et al., 2007).”
Anterior Cingulate Conflict Monitoring and Adjustments in Control
Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004) · Science · doi:10.1126/science.1089910 · Cited by 2,891 · cited 1×
“Human studies havemapped the brain regions that exert control during task performance (Koechlin et al., 2003; Kerns et al., 2004; Yeung et al., 2006; Cole et al., 2016) or measured the processing costs incurred by task switches (Monsell, 2003; Brown et al.”
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 5×
“We do this by drawing upon recent work in machine learning research, which distinguishes among the learning regimes that occur when deep networks are initialized with high-variance (lazy) or low-variance (rich) weights (Rigotti et al., 2013; Fusi et al., 2016; Chizat et al., 2018; Jacot et al., 2018; Arora et al., 201…”
InfluentialNeural Tangent Kernel: Convergence and Generalization in Neural Networks
Jacot, A., Gabriel, F., & Hongler, C. (2018) · arXiv · doi:10.48550/arxiv.1806.07572 · Cited by 1,562
Correlated neuronal discharge rate and its implications for psychophysical performance
Zohary, E., Shadlen, M. N., & Newsome, W. T. (1994) · Nature · doi:10.1038/370140a0 · Cited by 1,237 · cited 1×
“, 2020) where correlated patterns of firing confer robustness on the population code (Zohary et al., 1994).”
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · cited 1×
“There has been a recent resurgence of interest in neural networks (or ‘‘deep learning models’’) as computational theories of biological brains (Richards et al., 2019; Saxe et al., 2021).”
Self-citationGuclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
An adaptive coding model of neural function in prefrontal cortex
Duncan, J. (2001) · Nature Reviews Neuroscience · doi:10.1038/35097575 · Cited by 874 · cited 2×
“…is wholly or partially filtered out in ways that minimize interference between tasks (Cohen et al., 1990; Miller and Cohen, 2001), consistent with accounts emphasizing that neural codes are sculpted by task demands (Duncan, 2001) or through attention to scenes and objects (Çukur et al., 2013).”
Why neurons mix: high dimensionality for higher cognition
Fusi, S., Miller, E. K., & Rigotti, M. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.010 · Cited by 766 · cited 3×
“Previous analyses of single-cell data from macaque PFC have emphasized that neural selectivity is mixed and representations are high dimensional, in seeming contradiction to the findings reported here (Rigotti et al., 2013; Fusi et al., 2016).”
Cited by
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
The neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233
The role of population structure in computations through neural dynamics
Dubreuil, A., Valente, A., Beiran, M., Mastrogiuseppe, F., & Ostojic, S. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01088-4 · Cited by 237
Abstract representations emerge naturally in neural networks trained to perform multiple tasks
Johnston, W. J., & Fusi, S. (2023) · Nature Communications · doi:10.1038/s41467-023-36583-0 · Cited by 89
Curiosity: primate neural circuits for novelty and information seeking
Monosov, I. E. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00784-9 · Cited by 44
“Theoretical work suggests that, when learning to perform many tasks, explicit representations of abstract variables (such as uncertainty) could be useful for task performance and task acquisition 99–102 .”
A model of conceptual bootstrapping in human cognition
Zhao, B., Lucas, C. G., & Bramley, N. R. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01719-1 · Cited by 30
Shi, D., Shu, L., & Yu, Q. (2025) · doi:10.1101/2025.11.26.690703 · Cited by 1
Flexible Prefrontal Control over Hippocampal Episodic Memory for Goal-Directed Generalization
Zheng, Y., Wolf, N., Ranganath, C., O'Reilly, R. C., & McKee, K. L. (2025) · Proceedings of Cognitive Computational Neuroscience 2025 · doi:10.32470/jnqbpro
“…as neural networks typically train better with interleaved learning to prevent catastrophic forgetting (McClelland et al., 1995), constantly switching between different tasks can create switch costs (Russin et al., 2022; Flesch et al., 2022) that result in noisy hidden state representations.”
Metadata
- Authors
- Flesch, T., Juechems, K., Dumbalska, T., Saxe, A., & Summerfield, C.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2022-04
- Volume
- 110
- Issue
- 7
- Pages
- 1258-1270.e11
- ISSN
- 0896-6273 (ISSN-L)
- OpenAlex
- W4210510200
- Cited by
- 249
- References
- 20
- FWCI
- 24.69
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.