Abstract representations emerge naturally in neural networks trained to perform multiple tasks
Open accessPublished version of a preprint
Abstract
Humans and other animals demonstrate a remarkable ability to generalize knowledge across distinct contexts and objects during natural behavior. We posit that this ability to generalize arises from a specific representational geometry, that we call abstract and that is referred to as disentangled in machine learning. These abstract representations have been observed in recent neurophysiological studies. However, it is unknown how they emerge. Here, using feedforward neural networks, we demonstrate that the learning of multiple tasks causes abstract representations to emerge, using both supervised and reinforcement learning. We show that these abstract representations enable few-sample learning and reliable generalization on novel tasks. We conclude that abstract representations of sensory and cognitive variables may emerge from the multiple behaviors that animals exhibit in the natural world, and, as a consequence, could be pervasive in high-level brain regions. We also make several specific predictions about which variables will be represented abstractly.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Computational work highlights the importance of feature-invariant representations for transfer.
“This interpretation is broadly consistent with computational work highlighting the importance of feature-invariant representations for transfer (Johnston & Fusi, 2023; Yosinski et al., 2014).”
From The asymmetry of working memory training transfer: A systematic review and meta-analysis (2026) · cites this paper as Johnston & Fusi, 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826 · see citing works
Harris, C., Millman, K., van der Walt, S., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N., et al. (2020) · RePEc: Research Papers in Economics · Cited by 17,844 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · see citing works
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial NetsPreprint
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 3,379 · see citing works
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · see citing works
Self-citationUsing goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621 · see citing works
Sparse coding of sensory inputs
Olshausen, B. A., & Field, D. J. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.07.007 · Cited by 1,404 · see citing works
Visual neurones responsive to faces in the monkey temporal cortex
Perrett, D. I., Rolls, E. T., & Caan, W. (1982) · Experimental Brain Research · doi:10.1007/bf00239352 · Cited by 1,386 · see citing works
Sparse Coding and Decorrelation in Primary Visual Cortex During Natural Vision
Vinje, W. E., & Gallant, J. L. (2000) · Science · doi:10.1126/science.287.5456.1273 · Cited by 1,288 · see citing works
A Cortical Region Consisting Entirely of Face-Selective Cells
Tsao, D. Y., Freiwald, W. A., Tootell, R. B. H., & Livingstone, M. S. (2006) · Science · doi:10.1126/science.1119983 · Cited by 1,131 · see citing works
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · see citing works
Challenging Common Assumptions in the Unsupervised Learning of Disentangled RepresentationsPreprint
Locatello, F., Bauer, S., Lučić, M., Rätsch, G., Gelly, S., Schölkopf, B., & Bachem, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.12359 · Cited by 880 · see citing works
Oscillations and Sparsening of Odor Representations in the Mushroom Body
Pérez-Orive, J., Mazor, O., Turner, G. C., Cassenaer, S., Wilson, R. I., & Laurent, G. (2002) · Science · doi:10.1126/science.1070502 · Cited by 792 · see citing works
Why neurons mix: high dimensionality for higher cognition
Fusi, S., Miller, E. K., & Rigotti, M. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.010 · Cited by 766 · see citing works
Self-citationDeep Convolutional Inverse Graphics NetworkPreprint
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03167 · Cited by 708 · see citing works
Efficient coding of natural sounds
Lewicki, M. S. (2002) · Nature Neuroscience · doi:10.1038/nn831 · Cited by 667 · see citing works
The Code for Facial Identity in the Primate Brain
Chang, L., & Tsao, D. Y. (2017) · Cell · doi:10.1016/j.cell.2017.05.011 · Cited by 644 · see citing works
Smith, E. C., & Lewicki, M. S. (2006) · Nature · doi:10.1038/nature04485 · Cited by 625 · see citing works
High-dimensional geometry of population responses in visual cortex
Stringer, C., Pachitariu, M., Steinmetz, N. A., Carandini, M., & Harris, K. D. (2019) · Nature · doi:10.1038/s41586-019-1346-5 · Cited by 591 · see citing works
Task representations in neural networks trained to perform many cognitive tasks
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X.-J. (2019) · Nature Neuroscience · doi:10.1038/s41593-018-0310-2 · Cited by 549 · see citing works
Experience-dependent representation of visual categories in parietal cortex
Freedman, D. J., & Assad, J. A. (2006) · Nature · doi:10.1038/nature05078 · Cited by 542 · see citing works
Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex
Rolls, E. T., & Tovée, M. J. (1995) · Journal of Neurophysiology · doi:10.1152/jn.1995.73.2.713 · Cited by 500 · see citing works
The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex
Bernardi, S., Benna, M. K., Rigotti, M., Munuera, J., Fusi, S., & Salzman, C. D. (2020) · Cell · doi:10.1016/j.cell.2020.09.031 · Cited by 496 · see citing works
Self-citationA category-free neural population supports evolving demands during decision-making
Raposo, D. C., Kaufman, M. T., & Churchland, A. K. (2014) · Nature Neuroscience · doi:10.1038/nn.3865 · Cited by 474 · see citing works
Seeing 3D Chairs: Exemplar Part-Based 2D-3D Alignment Using a Large Dataset of CAD Models
Aubry, M., Maturana, D., Efros, A. A., Russell, B. C., & Šivic, J. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.487 · Cited by 471 · see citing works
Multi-Task Learning with Deep Neural Networks: A Survey
Crawshaw, M. (2020) · arXiv · doi:10.48550/arxiv.2009.09796 · Cited by 428 · see citing works
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
Bouchacourt, D., Tomioka, R., & Nowozin, S. (2017) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11867 · Cited by 343 · see citing works
Towards the neural population doctrine
Saxena, S., & Cunningham, J. P. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.02.002 · Cited by 317 · see citing works
Understanding disentangling in $β$-VAEPreprint
Burgess, C., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., & Lerchner, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03599 · Cited by 317 · see citing works
Neural population geometry: An approach for understanding biological and artificial neural networks
Chung, S., & Abbott, L. F. (2021) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2021.10.010 · Cited by 285 · see citing works
The population doctrine in cognitive neuroscience
Ebitz, R. B., & Hayden, B. Y. (2021) · Neuron · doi:10.1016/j.neuron.2021.07.011 · Cited by 273 · see citing works
Flesch, T., Juechems, K., Dumbalska, T., Saxe, A., & Summerfield, C. (2022) · Neuron · doi:10.1016/j.neuron.2022.01.005 · Cited by 257 · see citing works
The role of population structure in computations through neural dynamics
Dubreuil, A., Valente, A., Beiran, M., Mastrogiuseppe, F., & Ostojic, S. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01088-4 · Cited by 237 · see citing works
Flexible multitask computation in recurrent networks utilizes shared dynamical motifs
Driscoll, L., Shenoy, K., & Sussillo, D. (2022) · doi:10.1101/2022.08.15.503870 · Cited by 217 · see citing works
Flexible multitask computation in recurrent networks utilizes shared dynamical motifs
Driscoll, L. N., Shenoy, K. V., & Sussillo, D. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01668-6 · Cited by 214 · see citing works
Preferential encoding of visual categories in parietal cortex compared with prefrontal cortex
Swaminathan, S. K., & Freedman, D. J. (2012) · Nature Neuroscience · doi:10.1038/nn.3016 · Cited by 180 · see citing works
Trade-Off between Object Selectivity and Tolerance in Monkey Inferotemporal Cortex
Zoccolan, D., Kouh, M., Poggio, T., & DiCarlo, J. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1897-07.2007 · Cited by 169 · see citing works
Higgins, I., Chang, L., Langston, V., Hassabis, D., Summerfield, C., Tsao, D., & Botvinick, M. (2021) · Nature Communications · doi:10.1038/s41467-021-26751-5 · Cited by 162 · see citing works
Woloszyn, L., & Sheinberg, D. L. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.032 · Cited by 149 · see citing works
The macaque face patch system: a turtle’s underbelly for the brain
Hesse, J. K., & Tsao, D. Y. (2020) · Nature reviews. Neuroscience · doi:10.1038/s41583-020-00393-w · Cited by 139 · see citing works
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped ObservationsPreprint
Bouchacourt, D., Tomioka, R., & Nowozin, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08841 · Cited by 127 · see citing works
Are Disentangled Representations Helpful for Abstract Visual Reasoning?Preprint
van Steenkiste, S., Locatello, F., Schmidhuber, J., & Bachem, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12506 · Cited by 119 · see citing works
Recanatesi, S., Farrell, M., Lajoie, G., Denève, S., Rigotti, M., & Shea‐Brown, E. (2021) · Nature Communications · doi:10.1038/s41467-021-21696-1 · Cited by 88 · see citing works
Nonlinear mixed selectivity supports reliable neural computation
Johnston, W. J., Palmer, S. E., & Freedman, D. J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007544 · Cited by 75 · see citing works
Self-citationNeural state space alignment for magnitude generalization in humans and recurrent networks
Sheahan, H., Luyckx, F., Nelli, S., Teupe, C., & Summerfield, C. (2021) · Neuron · doi:10.1016/j.neuron.2021.02.004 · Cited by 67 · see citing works
The geometry of cortical representations of touch in rodents
Nogueira, R., Rodgers, C. C., Bruno, R. M., & Fusi, S. (2023) · Nature Neuroscience · doi:10.1038/s41593-022-01237-9 · Cited by 66 · see citing works
Self-citationHow to study the neural mechanisms of multiple tasks
Yang, G. R., Cole, M. W., & Rajan, K. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.07.001 · Cited by 54 · see citing works
On the principles of Parsimony and Self-consistency for the emergence of intelligence
Ma, Y., Tsao, D. Y., & Shum, H. (2022) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200297 · Cited by 46 · see citing works
Symmetry-Based Representations for Artificial and Biological General Intelligence
Higgins, I., Racanière, S., & Rezende, D. J. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.836498 · Cited by 34 · see citing works
Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task LearningPreprint
Huang, W., Mordatch, I., Abbeel, P., & Pathak, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03062 · Cited by 24 · see citing works
The neural code for face memoryPreprint
She, L., Benna, M. K., Shi, Y., Fusi, S., & Tsao, D. Y. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.12.435023 · Cited by 18 · see citing works
Self-citationCTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
Dai, X., Tong, S., Li, M., et al. (2022) · Entropy · doi:10.3390/e24040456 · Cited by 18 · see citing works
Subspace alignment as a mechanism for bindingPreprint
Fine, J. M., Yoo, S. B. M., Ebitz, R. B., & Hayden, B. Y. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.07.07.451472 · Cited by 4 · see citing works
Subspace orthogonalization as a mechanism for binding values to spacePreprint
Johnston, W. J., Fine, J. M., Yoo, S. B. M., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.06769 · Cited by 4 · see citing works
Self-citationIncremental Learning of Structured Memory via Closed-Loop TranscriptionPreprint
Tong, S., Dai, X., Wu, Z., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05411 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- [16]Boyle, L., Posani, L., Irfan, S., Siegelbaum, S. A. & Fusi, S. The geometry of hippocampal ca2 representations enables abstract coding of social familiarity and identity. bioRxiv https://www.biorxiv.org/content/10.1101/2022.01.24.477361v2 (2022). · DOI
- [18]Higgins, I. et al. β-VAE: learning basic visual concepts with a constrained variational framework. In: ICLR (2017).
- [37]Kim, H. & Mnih, A. Disentangling by factorising. In: International Conference on Machine Learning, 2649–2658 (PMLR, 2018).
- [38]Gao, P. et al. A theory of multineuronal dimensionality, dynamics and measurement. https://www.biorxiv.org/content/10.1101/214262v2#:~:text=This%20theory%20reveals%20conceptual%20insights,future%20large%2Dscale%20experimental%20design (2017). · DOI
- [41]Higgins, I. et al. beta-vae: learning basic visual concepts with a constrained variational framework. https://openreview.net/forum?id=Sy2fzU9gl (2016).
- [61]Abadi, M. et al. Tensorflow: a system for large-scale machine learning. In: 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), 265–283 (2016).
Cited by
Sorted by citations
The asymmetry of working memory training transfer: A systematic review and meta-analysis
Fu, J., Zhou, Y., Yuan, M., & Cai, Y. (2026) · Psychonomic Bulletin & Review · doi:10.3758/s13423-026-02994-5
“This interpretation is broadly consistent with computational work highlighting the importance of fea-ture-invariant representations for transfer (Johnston & Fusi, 2023; Yosinski et al., 2014).”
Circuit mechanisms for disentangled representations and generalization in the hippocampus
Tang, W., Chang, H., Liu, C., Perez-Hernandez, S., Zheng, W. Y., Park, J., Karaba, L. A., Oliva, A., & Fernandez-Ruiz, A. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02491-x
- Authors
- Johnston, W. J., & Fusi, S.
- Affiliations
- Mortimer B. Zuckerman Mind Brain Behavior Institute (US)
- Corresponding author
- Johnston, W. Jeffrey
- Fusi, Stefano
- Venue
- Nature Communications
- Published
- 2023-02-23
- Volume
- 14
- Issue
- 1
- Pages
- 1040-1040
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC9950464
- OpenAlex
- W4321610373
- Cited by
- 89
- References
- 58
- Citations by year
- 2026: 24 · 2025: 29 · 2024: 22 · 2023: 13 · 2022: 1
- Type
- conference abstract
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Nature Communications (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (1707398)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- abstract representation, disentangled representations, representational geometry, multi-task learning, emergent representations, few-shot learning, task generalization, reinforcement learning, supervised learning, feedforward neural networks
- MeSH terms
- Animals, Brain, Humans, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
