CoChat
Sign Up
PreprintCited by 27Memory and Neural MechanismsNeural dynamics and brain functionNeural Networks and Applications

Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks

Open access
Cueva, Christopher J., Wang, Peter Y., Chin, Matthew, Wei, Xue-Xin·2019·arXiv
DOI 10.48550/arxiv.1912.10189arXiv 1912.10189
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent work suggests goal-driven training of neural networks can be used to model neural activity in the brain. While response properties of neurons in artificial neural networks bear similarities to those in the brain, the network architectures are often constrained to be different. Here we ask if a neural network can recover both neural representations and, if the architecture is unconstrained and optimized, the anatomical properties of neural circuits. We demonstrate this in a system where the connectivity and the functional organization have been characterized, namely, the head direction circuits of the rodent and fruit fly. We trained recurrent neural networks (RNNs) to estimate head direction through integration of angular velocity. We found that the two distinct classes of neurons observed in the head direction system, the Compass neurons and the Shifter neurons, emerged naturally in artificial neural networks as a result of training. Furthermore, connectivity analysis and in-silico neurophysiology revealed structural and mechanistic similarities between artificial networks and the head direction system. Overall, our results show that optimization of RNNs in a goal-driven task can recapitulate the structure and function of biological circuits, suggesting that artificial neural networks can be used to study the brain at the level of both neural activity and anatomical organization.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1912.10189

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The RNN must maintain an estimate of a 1D circular position, which is best achieved through approximate ring attractors.

    “First, the RNN must maintain an estimate of a 1D circular position, which is best achieved through approximate ring attractors (Cueva et al., 2019).”

    From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Cueva et al., 2019

    The Model Structure section (p. 2) says the RNN estimates head direction by integrating angular velocity. The Appendix (p. 17) describes a compass-like attractor geometry that stores the angle. These passages support the circular-position and attractor parts of the claim, but do not establish that approximate ring attractors are the best way to perform the task. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 59 · Cited by 27

Research works printed but not linked: 53

  1. Shun-ichi Amari. Dynamics of pattern formation in lateral-inhibition type neural fields. Biological cybernetics , 27(2):77-87, 1977.
  2. Fred Attneave. Some informational aspects of visual perception. Psychological review , 61(3):183, 1954.
  3. Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J Chadwick, Thomas Degris, Joseph Modayil, et al. Vector-based navigation using grid-like representations in artificial agents. Nature , 557(7705):429, 2018.
  4. Horace B Barlow. Possible principles underlying the transformation of sensory messages. Sensory communication , pp. 217-234, 1961.
  5. Joshua P Bassett and Jeffrey S Taube. Neural correlates for angular head velocity in the rat dorsal tegmental nucleus. Journal of Neuroscience , 21(15):5740-5751, 2001.
  6. John A Bender and Michael H Dickinson. A comparison of visual and haltere-mediated feedback in the control of body saccades in drosophila melanogaster. Journal of Experimental Biology , 209 (23):4597-4606, 2006.
  7. Hugh T Blair and Patricia E Sharp. Anticipatory head direction signals in anterior thalamus: evidence for a thalamocortical circuit that integrates angular head motion to compute head direction. Journal of Neuroscience , 15(9):6260-6270, 1995.
  8. Hugh T Blair, Brian W Lipscomb, and Patricia E Sharp. Anticipatory time intervals of head-direction cells in the anterior thalamus of the rat: implications for path integration in the head-direction circuit. Journal of neurophysiology , 78(1):145-159, 1997.
  9. Hugh T Blair, Jeiwon Cho, and Patricia E Sharp. Role of the lateral mammillary nucleus in the rat head direction circuit: a combined single unit recording and lesion study. Neuron , 21(6): 1387-1397, 1998.
  10. Charles F Cadieu, Ha Hong, Daniel LK Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, and James J DiCarlo. Deep neural networks rival the representation of primate it cortex for core visual object recognition. PLoS computational biology , 10(12):e1003963, 2014.
  11. Christopher J Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. ICLR , 2018.
  12. Christopher J Cueva, Alex Saez, Encarni Marcos, Aldo Genovesio, Mehrdad Jazayeri, Ranulfo Romo, C Daniel Salzman, Michael N Shadlen, and Stefano Fusi. Low dimensional dynamics for working memory and time encoding. bioRxiv doi: 10.1101/504936 , 2019. · DOI
  13. Winfried Denk, James H Strickler, and Watt W Webb. Two-photon laser scanning fluorescence microscopy. Science , 248(4951):73-76, 1990.
  14. Arseny Finkelstein, Dori Derdikman, Alon Rubin, Jakob N Foerster, Liora Las, and Nachum Ulanovsky. Three-dimensional head-direction coding in the bat brain. Nature , 517(7533):159, 2015.
  15. Arseny Finkelstein, Nachum Ulanovsky, Misha Tsodyks, and Johnatan Aljadeff. Optimal dynamic coding by mixed-dimensionality neurons in the head-direction system of bats. Nature communications , 9(1):3590, 2018.
  16. Jonathan Green, Atsuko Adachi, Kunal K Shah, Jonathan D Hirokawa, Pablo S Magani, and Gaby Maimon. A neural circuit architecture for angular integration in drosophila. Nature , 546(7656): 101, 2017.
  17. Umut Güçlü and Marcel AJ van Gerven. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience , 35(27):10005-10014, 2015.
  18. Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS computational biology , 10(11):e1003915, 2014.
  19. Sung Soo Kim, Hervé Rouault, Shaul Druckmann, and Vivek Jayaraman. Ring attractor dynamics in the drosophila central brain. Science , 356(6340):849-853, 2017.
  20. D P Kingma and J L Ba. Adam: a method for stochastic optimization. International Conference on Learning Representations , 2015.
  21. Nikolaus Kriegeskorte. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annual Review of Vision Science , 1:417-446, 2015.
  22. Simon Laughlin. A simple coding procedure enhances a neuron's information capacity. Zeitschrift für Naturforschung c , 36(9-10):910-912, 1981.
  23. Chih-Yung Lin, Chao-Chun Chuang, Tzu-En Hua, Chun-Chao Chen, Barry J Dickson, Ralph J Greenspan, and Ann-Shyn Chiang. A comprehensive wiring diagram of the protocerebral bridge for visual information processing in the drosophila brain. Cell reports , 3(5):1739-1753, 2013.
  24. Ralph Linsker. Self-organization in a perceptual network. Computer , 21(3):105-117, 1988.
  25. Niru Maheswaranathan, Alex H Williams, Matthew D Golub, Surya Ganguli, and David Sussillo. Universality and individuality in neural dynamics across large populations of recurrent networks. arXiv preprint arXiv:1907.08549 , 2019. · arXiv
  26. Valerio Mante, David Sussillo, Krishna V Shenoy, and William T Newsome. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature , 503(7474):78-84, 2013.
  27. James Martens and Ilya Sutskever. Learning recurrent neural networks with hessian-free optimization. pp. 1033-1040, 2011.
  28. Sohie Lee Moody, Steven P. Wise, Giuseppe di Pellegrino, and David Zipser. A model that accounts for activity in primate frontal cortex during a delayed matching-to-sample task. Journal of Neuroscience , 18(1):399-410, 1998. ISSN 0270-6474. doi: 10.1523/JNEUROSCI.18-01-00399. 1998. URL https://www.jneurosci.org/content/18/1/399 . · DOI
  29. Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature , 381(6583):607, 1996.
  30. A Emin Orhan and Wei Ji Ma. A diverse range of factors affect the nature of neural representations underlying short-term memory. Nature neuroscience , 22(2):275, 2019.
  31. Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, et al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature , 572(7767):106-111, 2019.
  32. Adrien Peyrache, Marie M Lacroix, Peter C Petersen, and György Buzsáki. Internally organized mechanisms of the head direction sense. Nature neuroscience , 18(4):569, 2015.
  33. Keram Pfeiffer and Uwe Homberg. Organization and functional roles of the central complex in the insect brain. Annual review of entomology , 59:165-184, 2014.
  34. Florian Raudies and Michael E Hasselmo. Modeling boundary vector cell firing given optic flow as a cue. PLoS computational biology , 8(6):e1002553, 2012.
  35. Audrey J Sederberg and Ilya Nemenman. Randomly connected networks generate emergent selectivity and predict decoding properties of large populations of neurons. arXiv preprint arXiv:1909.10116 , 2019. · arXiv
  36. Patricia E Sharp, Hugh T Blair, and Jeiwon Cho. The anatomical and computational basis of the rat head-direction cell signal. Trends in neurosciences , 24(5):289-294, 2001.
  37. William E Skaggs, James J Knierim, Hemant S Kudrimoti, and Bruce L McNaughton. A model of the neural basis of the rat's sense of direction. In Advances in neural information processing systems , pp. 173-180, 1995.
  38. H Francis Song, Guangyu R Yang, and Xiao-Jing Wang. Training excitatory-inhibitory recurrent neural networks for cognitive tasks: a simple and flexible framework. PLoS computational biology , 12(2):e1004792, 2016.
  39. Pengcheng Song and Xiao-Jing Wang. Angular path integration by moving 'hill of activity': a spiking neuron model without recurrent excitation of the head-direction system. Journal of Neuroscience , 25(4):1002-1014, 2005.
  40. Robert W Stackman and Jeffrey S Taube. Firing properties of rat lateral mammillary single units: head direction, head pitch, and angular head velocity. Journal of Neuroscience , 18(21):9020-9037, 1998.
  41. David Sussillo and Omri Barak. Opening the black box: low-dimensional dynamics in highdimensional recurrent neural networks. Neural computation , 25(3):626-649, 2013.
  42. David Sussillo, Mark M Churchland, Matthew T Kaufman, and Krishna V Shenoy. A neural network that finds a naturalistic solution for the production of muscle activity. Nature neuroscience , 18(7): 1025-1033, 2015.
  43. Jeffrey S Taube. Head direction cells recorded in the anterior thalamic nuclei of freely moving rats. Journal of Neuroscience , 15(1):70-86, 1995.
  44. Jeffrey S Taube and Robert U Muller. Comparisons of head direction cell activity in the postsubiculum and anterior thalamus of freely moving rats. Hippocampus , 8(2):87-108, 1998.
  45. Jeffrey S Taube, Robert U Muller, and James B Ranck. Head-direction cells recorded from the postsubiculum in freely moving rats. i. description and quantitative analysis. Journal of Neuroscience , 10(2):420-435, 1990a.
  46. Jeffrey S Taube, Robert U Muller, and James B Ranck. Head-direction cells recorded from the postsubiculum in freely moving rats. ii. effects of environmental manipulations. Journal of Neuroscience , 10(2):436-447, 1990b.
  47. Alan Turing. The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences , 237(641):37-72, 1952.
  48. Tanya Wolff, Nirmala A Iyer, and Gerald M Rubin. Neuroarchitecture and neuroanatomy of the drosophila central complex: A gal4-based dissection of protocerebral bridge neurons and circuits. Journal of Comparative Neurology , 523(7):997-1037, 2015.
  49. Xiaohui Xie, Richard HR Hahnloser, and H Sebastian Seung. Double-ring network model of the head-direction system. Physical Review E , 66(4):041902, 2002.
  50. Daniel LK Yamins and James J DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience , 19(3):356-365, 2016.
  51. Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences , 111(23):8619-8624, 2014.
  52. Guangyu Robert Yang, Madhura R Joglekar, H Francis Song, William T Newsome, and Xiao-Jing Wang. Task representations in neural networks trained to perform many cognitive tasks. Nature neuroscience , 22(2):297, 2019.
  53. Kechen Zhang. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. Journal of Neuroscience , 16(6):2112-2126, 1996.

Authors
Cueva, C. J., Wang, P. Y., Chin, M., & Wei, X.-X.
Affiliations
Columbia University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1912.10189
arXiv
1912.10189
OpenAlex
W2996170727
Cited by
27
References
59
Citations by year
2025: 2 · 2024: 3 · 2023: 5 · 2022: 4 · 2021: 6 · 2020: 7
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (1707398)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
head direction system, recurrent neural networks, neural circuit architecture, connectivity analysis, rodents, fruit flies, neural representations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review