Task representations in neural networks trained to perform many cognitive tasks
Open accessAbstract
The brain has the ability to flexibly perform many tasks, but the underlying mechanism cannot be elucidated in traditional experimental and modeling studies designed for one task at a time. Here, we trained single network models to perform 20 cognitive tasks that depend on working memory, decision making, categorization, and inhibitory control. We found that after training, recurrent units can develop into clusters that are functionally specialized for different cognitive processes, and we introduce a simple yet effective measure to quantify relationships between single-unit neural representations of tasks. Learning often gives rise to compositionality of task representations, a critical feature for cognitive flexibility, whereby one task can be performed by recombining instructions for other tasks. Finally, networks developed mixed task selectivity similar to recorded prefrontal neurons after learning multiple tasks sequentially with a continual-learning technique. This work provides a computational platform to investigate neural representations of many cognitive tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The RNNs are initialized with normally distributed weights to facilitate comparisons with the computational neuroscience literature.
“In contrast to the orthogonal initialization typically employed in the mainstream machine learning literature 95 , we initialize our RNNs with normally distributed weights to facilitate comparisons with the computational neuroscience literature 75,96,97 .”
From Emergent behaviour and neural dynamics in artificial agents tracking odour plumes (2023) · cites this paper as [75]
The Methods (Training procedure; page unavailable) say the input and output weights are initialized as independent Gaussian random variables. But the recurrent weights are initialized as a scaled identity matrix, and the paper does not say that Gaussian initialization facilitates comparison with computational neuroscience literature. (LLM, conf=0.88)
Experimental — accuracy not yet measuredRNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks.
“RNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks (Driscoll et al., 2022; Yang et al., 2019)”
From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Yang et al., 2019
The Results section, “Functional clusters encode subsets of tasks” (page unavailable), reports clusters of recurrent units dedicated to processes shared across tasks, supporting the modular-population and reuse portions of the claim. The Discussion describes these clusters as potential building blocks, but our copy does not establish that dynamical motifs are reused across tasks; it also notes that sequential training produces more mixed representations. (LLM, conf=0.86)
Experimental — accuracy not yet measuredRNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks.
“For example, RNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks (Driscoll et al., 2022; Yang et al., 2019).”
From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Yang et al., 2019
The Results section, “Functional clusters encode subsets of tasks in reference networks” (page not available), reports clusters of recurrent units dedicated to processes shared across tasks. The paper supports modular populations and their reuse, but the passages examined do not establish that dynamical motifs are reused across tasks. (LLM, conf=0.86)
Experimental — accuracy not yet measuredModular structure has been shown to spontaneously emerge in some end-to-end neural network systems in domains other than language.
“Modular structure has been shown to spontaneously emerge in some end-to-end neural network systems in domains other than language [e.g., 203, 204]”
From Dissociating language and thought in large language models (2024) · cites this paper as [203]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · see citing works
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · see citing works
Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex
Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989) · Journal of Neurophysiology · doi:10.1152/jn.1989.61.2.331 · Cited by 2,685 · see citing works
Fuster, J. M. (1997) · Cited by 2,095 · see citing works
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works
Context-dependent computation by recurrent dynamics in prefrontal cortex
Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013) · Nature · doi:10.1038/nature12742 · Cited by 1,807 · see citing works
Self-citationContinual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784 · see citing works
Multi-task connectivity reveals flexible hubs for adaptive task control
Cole, M. W., Reynolds, J. R., Power, J. D., Repovs, G., Anticevic, A., & Braver, T. S. (2013) · Nature Neuroscience · doi:10.1038/nn.3470 · Cited by 1,704 · see citing works
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · see citing works
Self-citationNeurons in the orbitofrontal cortex encode economic value
Padoa-Schioppa, C., & Assad, J. A. (2006) · Nature · doi:10.1038/nature04676 · Cited by 1,598 · see citing works
Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque
Miller, E. K., Erickson, C. A., & Desimone, R. (1996) · Journal of Neuroscience · doi:10.1523/jneurosci.16-16-05154.1996 · Cited by 1,441 · see citing works
Look away: the anti-saccade task and the voluntary control of eye movement
Munoz, D. P., & Everling, S. (2004) · Nature reviews. Neuroscience · doi:10.1038/nrn1345 · Cited by 1,423 · see citing works
Probabilistic decision making by slow reverberation in cortical circuits
Wang, X.-J. (2002) · Neuron · doi:10.1016/s0896-6273(02)01092-9 · Cited by 1,242 · see citing works
Self-citationZipser, D., & Andersen, R. A. (1988) · Nature · doi:10.1038/331679a0 · Cited by 1,187 · see citing works
Single neurons in prefrontal cortex encode abstract rules
Wallis, J. D., Anderson, K. C., & Miller, E. K. (2001) · Nature · doi:10.1038/35082081 · Cited by 1,075 · see citing works
A Large-Scale Model of the Functioning Brain
Eliasmith, C., Stewart, T. C., Choo, X., Bekolay, T., DeWolf, T., Tang, Y., & Rasmussen, D. (2012) · Science · doi:10.1126/science.1225266 · Cited by 934 · see citing works
Functional specificity in the human brain: a window into the functional architecture of the mind
Kanwisher, N. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1005062107 · Cited by 929 · see citing works
Neuronal correlates of parametric working memory in the prefrontal cortex
Romo, R., Brody, C. D., Hernández, A., & Lemus, L. (1999) · Nature · doi:10.1038/20939 · Cited by 822 · see citing works
Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks
Sussillo, D., & Barak, O. (2013) · Neural Computation · doi:10.1162/neco_a_00409 · Cited by 575 · see citing works
A Simple Way to Initialize Recurrent Networks of Rectified Linear UnitsPreprint
Le, Q. V., Jaitly, N., & Hinton, G. E. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00941 · Cited by 521 · see citing works
Role of frontal eye fields in countermanding saccades: visual, movement, and fixation activity
Hanes, D. P., Patterson, W. F., & Schall, J. D. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.79.2.817 · Cited by 492 · see citing works
A category-free neural population supports evolving demands during decision-making
Raposo, D. C., Kaufman, M. T., & Churchland, A. K. (2014) · Nature Neuroscience · doi:10.1038/nn.3865 · Cited by 474 · see citing works
Cortical information flow during flexible sensorimotor decisions
Siegel, M., Buschman, T. J., & Miller, E. K. (2015) · Science · doi:10.1126/science.aab0551 · Cited by 445 · see citing works
Task Set and Prefrontal Cortex
Sakai, K. (2008) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.31.060407.125642 · Cited by 390 · see citing works
Inhibition, Spike Threshold, and Stimulus Selectivity in Primary Visual Cortex
Priebe, N. J., & Ferster, D. (2008) · Neuron · doi:10.1016/j.neuron.2008.02.005 · Cited by 359 · see citing works
Recurrent network models of sequence generation and memory
Rajan, K., Harvey, C. D., & Tank, D. W. (2016) · Neuron · doi:10.1016/j.neuron.2016.02.009 · Cited by 351 · see citing works
Song, H. F., Yang, G. R., & Wang, X.-J. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004792 · Cited by 281 · see citing works
Self-citationAnatomy of the Prefrontal Cortex
Fuster, J. M. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407815-4.00002-7 · Cited by 272 · see citing works
Computational principles of synaptic memory consolidation
Benna, M. K., & Fusi, S. (2016) · Nature Neuroscience · doi:10.1038/nn.4401 · Cited by 240 · see citing works
Cole, M. W., Laurent, P. A., & Stocco, A. (2012) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-012-0125-7 · Cited by 195 · see citing works
Rigotti, M. (2010) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2010.00024 · Cited by 184 · see citing works
Reward-based training of recurrent neural networks for cognitive and value-based tasks
Song, H., Yang, G. R., & Wang, X. (2017) · eLife · doi:10.7554/elife.21492 · Cited by 169 · see citing works
Self-citationChaisangmongkon, W., Swaminathan, S. K., Freedman, D. J., & Wang, X. (2017) · Neuron · doi:10.1016/j.neuron.2017.03.002 · Cited by 159 · see citing works
Self-citationDrivers and modulators from push-pull and balanced synaptic input
Abbott, L. F., & Chance, F. S. (2005) · Progress in brain research · doi:10.1016/s0079-6123(05)49011-1 · Cited by 153 · see citing works
Compositionality of Rule Representations in Human Prefrontal Cortex
Reverberi, C., Görgen, K., & Haynes, J. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr200 · Cited by 139 · see citing works
Neuronal Mechanisms of Visual Categorization: An Abstract View on Decision Making
Freedman, D. J., & Assad, J. A. (2016) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071714-033919 · Cited by 138 · see citing works
Carnevale, F., de Lafuente, V., Romo, R., Barak, O., & Parga, N. (2015) · Neuron · doi:10.1016/j.neuron.2015.04.014 · Cited by 119 · see citing works
Rapid Transfer of Abstract Rules to Novel Contexts in Human Lateral Prefrontal Cortex
Cole, M. W., Etzel, J. A., Zacks, J. M., Schneider, W., & Braver, T. S. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00142 · Cited by 95 · see citing works
Tschentscher, N., Mitchell, D. J., & Duncan, J. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2478-16.2017 · Cited by 53 · see citing works
A Dataset and Architecture for Visual Reasoning with a Working Memory
Yang, G. R., Ganichev, I., Wang, X., Shlens, J., & Sussillo, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_44 · Cited by 43 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [3]Wang X-J in Principles of Frontal Lobe Function (Stuss DT & Knight RT eds.) (Cambridge Univ. Press, New York, 2013).
- [40]Lake BM & Baroni M. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. ICML 80, 2873– 2882 (2017).
Cited by
Sorted by citations
Computation Through Neural Population Dynamics
Vyas, S., Golub, M. D., Sussillo, D., & Shenoy, K. V. (2020) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092619-094115 · Cited by 671 · see citing works
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345 · see citing works
The role of population structure in computations through neural dynamics
Dubreuil, A., Valente, A., Beiran, M., Mastrogiuseppe, F., & Ostojic, S. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01088-4 · Cited by 237 · see citing works
Flexible multitask computation in recurrent networks utilizes shared dynamical motifs
Driscoll, L., Shenoy, K., & Sussillo, D. (2022) · doi:10.1101/2022.08.15.503870 · Cited by 217 · see citing works
Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way
Willett, F. R., Deo, D. R., Avansino, D. T., Rezaii, P., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2020) · Cell · doi:10.1016/j.cell.2020.02.043 · Cited by 171 · see citing works
Brain-like functional specialization emerges spontaneously in deep neural networks
Dobs, K., Martinez, J., Kell, A. J. E., & Kanwisher, N. (2022) · Science Advances · doi:10.1126/sciadv.abl8913 · Cited by 129 · see citing works
A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping
Michaels, J. A., Schaffelhofer, S., Agudelo-Toro, A., & Scherberger, H. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2005087117 · Cited by 103 · see citing works
Abstract representations emerge naturally in neural networks trained to perform multiple tasks
Johnston, W. J., & Fusi, S. (2023) · Nature Communications · doi:10.1038/s41467-023-36583-0 · Cited by 89 · see citing works
Neocortical layer 5 subclasses: from cellular properties to roles in behavior
Moberg, S., & Takahashi, N. (2022) · Frontiers in Synaptic Neuroscience · doi:10.3389/fnsyn.2022.1006773 · Cited by 62 · see citing works
Emergent behaviour and neural dynamics in artificial agents tracking odour plumes
Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00599-w · Cited by 61 · see citing works
“Third, multitask training should produce agents with richer behaviours and more complex neural activity structures with shared and task-specific adaptations 75,76 .”
BackgroundMethodTasks and their role in visual neuroscience
Kay, K., Bonnen, K., Denison, R. N., Arcaro, M. J., & Barack, D. L. (2023) · Neuron · doi:10.1016/j.neuron.2023.03.022 · Cited by 61 · see citing works
Evolving the olfactory system with machine learning
Wang, P. Y., Sun, Y., Axel, R., Abbott, L. F., & Yang, G. R. (2021) · Neuron · doi:10.1016/j.neuron.2021.09.010 · Cited by 54 · see citing works
Self-citationDiscovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64 · see citing works
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38 · see citing works
Lappalainen, J. K., Tschopp, F. D., Prakhya, S., McGill, M., Nern, A., Shinomiya, K., Takemura, S.-Y., Gruntman, E., Macke, J. H., & Turaga, S. C. (2023) · doi:10.1101/2023.03.11.532232 · Cited by 31 · see citing works
Eckstein, M. K., Summerfield, C., Daw, N. D., & Miller, K. J. (2023) · doi:10.1101/2023.05.17.541226 · Cited by 18 · see citing works
Remapping in a recurrent neural network model of navigation and context inference
Low, I. I., Giocomo, L. M., & Williams, A. H. (2023) · eLife · doi:10.7554/elife.86943 · Cited by 15 · see citing works
“32,33 When the RNN architecture is explicitly modular in its design, these subpopulations can improve model performance on particular types of tasks.”
BackgroundState and capacity in neural models of cognition and consciousness
Wang, P., Liu, H., Ilievski, F., Ganushchak, L., Zou, L., Laureys, S., & Paas, F. (2026) · Trends Open · doi:10.1016/j.treopn.2026.04.004 · Cited by 8 · see citing works
Remapping in a recurrent neural network model of navigation and context inference
Low, I. I. C., Giocomo, L. M., & Williams, A. H. (2023) · doi:10.1101/2023.01.25.525596 · Cited by 3 · see citing works
Evidence from formal logical reasoning reveals that the language of thought is not natural language
Kean, H., Fung, A., Jaggers, P., Chen, J., Rule, J. S., Benn, Y., Tenenbaum, J. B., Piantadosi, S. T., Varley, R. A., & Fedorenko, E. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2520095123 · Cited by 2 · see citing works
Remapping in a recurrent neural network model of navigation and context inference
Low, I. I., Giocomo, L. M., & Williams, A. H. (2023) · eLife · doi:10.7554/elife.86943.3 · Cited by 1 · see citing works
- Authors
- Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X.-J.
- Affiliations
- Mortimer B. Zuckerman Mind Brain Behavior Institute (US)
- Courant Institute of Mathematical Sciences (US)
- Google DeepMind (United Kingdom) (GB)
- Howard Hughes Medical Institute (US)
- New York University (US)
- Corresponding author
- Wang, Xiao-Jing
- Venue
- Nature Neuroscience
- Published
- 2019-01-14
- Volume
- 22
- Issue
- 2
- Pages
- 297-306
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- PMC
- PMC11549734
- OpenAlex
- W2908124316
- Cited by
- 549
- References
- 43
- Citations by year
- 2026: 75 · 2025: 88 · 2024: 86 · 2023: 93 · 2022: 56 · 2021: 65 · 2020: 41 · 2019: 39 · 2018: 5 · 2017: 1
- FWCI
- 29.22
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01MH062349)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- task representation, cognitive flexibility, working memory, decision-making, categorization, inhibitory control, recurrent neural networks, functional specialization, compositional representations, prefrontal cortex, continual learning
- MeSH terms
- Brain, Cognition, Computer Simulation, Decision Making, Humans, Learning, Memory, Short-Term, Models, Neurological, Neurons, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
