PreprintCited by 217Neural Networks and ApplicationsNeural dynamics and brain functionMemory and Neural Mechanisms

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Open access
Driscoll, Laura, Shenoy, Krishna, Sussillo, David·2022·Nature Neuroscience

Abstract

Flexible computation is a hallmark of intelligent behavior. Yet, little is known about how neural networks contextually reconfigure for different computations. Humans are able to perform a new task without extensive training, presumably through the composition of elementary processes that were previously learned. Cognitive scientists have long hypothesized the possibility of a compositional neural code, where complex neural computations are made up of constituent components; however, the neural substrate underlying this structure remains elusive in biological and artificial neural networks. Here we identified an algorithmic neural substrate for compositional computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses of networks revealed learned computational strategies that mirrored the modular subtask structure of the task-set used for training. Dynamical motifs such as attractors, decision boundaries and rotations were reused across different task computations. For example, tasks that required memory of a continuous circular variable repurposed the same ring attractor. We show that dynamical motifs are implemented by clusters of units and are reused across different contexts, allowing for flexibility and generalization of previously learned computation. Lesioning these clusters resulted in modular effects on network performance: a lesion that destroyed one dynamical motif only minimally perturbed the structure of other dynamical motifs. Finally, modular dynamical motifs could be reconfigured for fast transfer learning. After slow initial learning of dynamical motifs, a subsequent faster stage of learning reconfigured motifs to perform novel tasks. This work contributes to a more fundamental understanding of compositional computation underlying flexible general intelligence in neural systems. We present a conceptual framework that establishes dynamical motifs as a fundamental unit of computation, intermediate between the neuron and the network. As more whole brain imaging studies record neural activity from multiple specialized systems simultaneously, the framework of dynamical motifs will guide questions about specialization and generalization across brain regions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/2022.08.15.503870

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • RNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks.

    “RNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks (Driscoll et al., 2022; Yang et al., 2019)”

    From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Driscoll et al., 2022

  • RNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks.

    “For example, RNNs trained in many computational tasks develop modular neural populations and dynamical motifs that are re-used across tasks (Driscoll et al., 2022; Yang et al., 2019).”

    From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Driscoll et al., 2022

  • Neurons in the medial entorhinal cortex typically represent one or more features such as spatial position, heading direction, and environmental landmarks and are therefore thought to support navigation.

    “neurons in the medial entorhinal cortex typically represent one or more features such as spatial position, heading direction, and environmental landmarks and are therefore thought to support navigation.[15][16][17][18][19][20][21][22]”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Bower, G. H.) vol. 24 109-165 (Academic Press, 1989).
  2. Goudar, V., Peysakhovich, B., Freedman, D. J. & Buffalo, E. A. Elucidating the neural mechanisms of Learning-to-Learn. bioRxiv (2021).
  3. Guangyu Robert Yang, Madhura R Joglekar, H Francis Song, William T Newsome, and Xiao-Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience, 22(2):297, 2019.
  4. Matthew Golub and David Sussillo. Fixedpointfinder: A tensorflow toolbox for identifying and character- izing fixed points in recurrent neural networks. Journal of Open Source Software, 3(31):1003, 2018.
  5. Valerio Mante, David Sussillo, Krishna V Shenoy, and William T Newsome. Context-dependent computa- tion by recurrent dynamics in prefrontal cortex. nature, 503(7474):78-84, 2013.

Cited by

Sorted by citations

Authors
Driscoll, L., Shenoy, K., & Sussillo, D.
Affiliations
  • Stanford University (US)
  • Howard Hughes Medical Institute (US)
Corresponding author
Driscoll, Laura
Venue
Nature Neuroscience
Publisher
openRxiv
Published
2022-08-15
Volume
27
Issue
7
Pages
1349-1363
ISSN
1097-6256 (ISSN-L)
OpenAlex
W4400462656
Cited by
217
References
62
Citations by year
2026: 78 · 2025: 58 · 2024: 45 · 2023: 31 · 2022: 4
FWCI
26.71
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Nature Neuroscience (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Simons Foundation (543049, 1155867, 543045)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords
recurrent neural networks, multi-task learning, modular computation, ring attractor, attractor dynamics, decision boundaries, transfer learning
MeSH terms
Algorithms, Animals, Learning, Models, Neurological, Nerve Net, Neurons, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review