Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework
Open accessThe DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
Abstract
The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze model recurrent neural networks (RNNs) that have been optimized to perform the same tasks as behaving animals. Because the optimization of network parameters specifies the desired output but not the manner in which to achieve this output, "trained" networks serve as a source of mechanistic hypotheses and a testing ground for data analyses that link neural computation to behavior. Complete access to the activity and connectivity of the circuit, and the ability to manipulate them arbitrarily, make trained networks a convenient proxy for biological circuits and a valuable platform for theoretical investigation. However, existing RNNs lack basic biological features such as the distinction between excitatory and inhibitory units (Dale's principle), which are essential if RNNs are to provide insights into the operation of biological circuits. Moreover, trained networks can achieve the same behavioral performance but differ substantially in their structure and dynamics, highlighting the need for a simple and flexible framework for the exploratory training of RNNs. Here, we describe a framework for gradient descent-based training of excitatory-inhibitory RNNs that can incorporate a variety of biological knowledge. We provide an implementation based on the machine learning library Theano, whose automatic differentiation capabilities facilitate modifications and extensions. We validate this framework by applying it to well-known experimental paradigms such as perceptual decision-making, context-dependent integration, multisensory integration, parametric working memory, and motor sequence generation. Our results demonstrate the wide range of neural activity patterns and behavior that can be modeled, and suggest a unified setting in which diverse cognitive computations and mechanisms can be studied.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



![Fig 4 Context-dependent integration task. (A) Psychometric function, showing the percentage of trials on which the network chose choice 1 as a function of the signed motion (upper) and signed color (lower) coherence in motion-context (black) and color-context (blue) trials. (B) Average population responses in state space during the stimulus period, projected to the 3-dimensional subspace capturing variance due to choice, motion, and color as in [5]. Only correct trials were included. The task-related axes were obtained through a linear regression analysis. Note that “choice” here has a unit-specific meaning that depends on the preferred choice of the unit as determined by the selectivity index (Eq 30). For both motion (black) and color (blue), coherences increase from light to dark. Upper plots show trials during the motion context, and lower plots show trials during the color context. (C) Normalized responses of four recurrent units during the stimulus period show mixed representation of task variables. Solid lines indicate the preferred choice and dashed lines the nonpreferred choice of each unit. (D) Denoised regression coefficients from the linear regression analysis. By definition, the coefficients for choice are almost exclusively positive.](https://figures.cochat.ai/paper-figures/ca3a2b623c1e4966aef22596dc894038/training-excitatory-inhibitory-recurrent-neural-networks-for-cognitive-tasks-a-figure-4.jpg)





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“and which can be viewed as an approximation to continuous time firing rate models of neural circuits (Song et al., 2016).”
From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Song et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“For instance, machine learning applications typically employ a type of RNN known as Long Short-Term Memory (LSTM), which uses multiplicative gates to facilitate learning of long-term dependencies and currently represents one of the most powerful methods for solving sequence-related problems [72].”
MethodReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“Combining RNN training with reinforcement learning methods [75 – 77] will be essential to bringing network training closer to the reward-based manner in which animals are trained.”
MethodReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Even so, SGD with the objective function given in Eqs 15 and 16 often fails to converge to a solution when the network must learn dependencies between distant time points [60].”
BackgroundOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 4×
“To remedy this problem, which is due to some gradient components being too large ( exploding gradients) and some gradient components being too small ( vanishing gradients), we follow [35] in making two modifications.”
BackgroundMethodInfluential(2013) · Cited by 3,740
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
Interneurons of the neocortical inhibitory system
Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1519 · Cited by 2,954 · cited 2×
“Local cortical circuits have distance [48], layer [26, 49, 50], and cell-type [23, 25, 27, 51] dependent patterns of connectivity and different overall levels of sparseness for excitatory to excitatory, inhibitory to excitatory, excitatory to inhibitory, and inhibitory to inhibitory connections [52, 53].”
Background
Research works printed but not linked: 2
- Bengio Y. 2015 Towards biologically plausible deep learning;Available from: http://arxiv.org/abs/1502.0415.
- DE Rumelhart, 1986, Parallel Distrib. Process, Vol. 1, 318
Cited by
Computation Through Neural Population Dynamics
Vyas, S., Golub, M. D., Sussillo, D., & Shenoy, K. V. (2020) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092619-094115 · Cited by 671
“The authors developed a set of tools for reward-based training of RNNs, which match behavioral and electrophysiological recordings from an array of experimental paradigms and thus might serve as a powerful framework for studying value-based neural computations (Song et al. 2017).”
BackgroundFrontal Cortex and the Hierarchical Control of Behavior
Badre, D., & Nee, D. E. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.005 · Cited by 611
“As noted above, many solutions surely exist to hierarchical control problems, including those not discussed here, such as recurrent networks [100].”
BackgroundTask representations in neural networks trained to perform many cognitive tasks
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X.-J. (2019) · Nature Neuroscience · doi:10.1038/s41593-018-0310-2 · Cited by 549
Self-citationArtificial Neural Networks for Neuroscientists: A Primer
Yang, G. R., & Wang, X.-J. (2020) · Neuron · doi:10.1016/j.neuron.2020.09.005 · Cited by 376
Self-citationWilliams, A. H., Kim, T. H., Wang, F., Vyas, S., Ryu, S. I., Shenoy, K. V., Schnitzer, M., Kolda, T. G., & Ganguli, S. (2018) · Neuron · doi:10.1016/j.neuron.2018.05.015 · Cited by 310
Cueva, C. J., & Wei, X.-X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07770 · Cited by 183
“Given the abundant recurrent connections in the brain, it seems a particular fruitful avenue to take advantage of the recent development in RNNs to help with neuroscience questions (Mante et al., 2013; Song et al., 2016; Miconi, 2017; Sussillo et al., 2015).”
BackgroundFlexible Sensorimotor Computations through Rapid Reconfiguration of Cortical Dynamics
Remington, E. D., Narain, D., Hosseini, E. A., & Jazayeri, M. (2018) · Neuron · doi:10.1016/j.neuron.2018.05.020 · Cited by 269
Recurrent neural networks as versatile tools of neuroscience research
Barak, O. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.06.003 · Cited by 244
Inhibitory Plasticity: Balance, Control, and Codependence
Hennequin, G., Agnes, E. J., & Vogels, T. P. (2017) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-072116-031005 · Cited by 228
Bayesian Computation through Cortical Latent Dynamics
Sohn, H., Narain, D., Meirhaeghe, N., & Jazayeri, M. (2019) · Neuron · doi:10.1016/j.neuron.2019.06.012 · Cited by 231
Two views on the cognitive brain
Barack, D. L., & Krakauer, J. W. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00448-6 · Cited by 210
A unifying perspective on neural manifolds and circuits for cognition
Langdon, C., Genkin, M., & Engel, T. A. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00693-x · Cited by 207
Bridging Neural and Computational Viewpoints on Perceptual Decision-Making
O’Connell, R. G., Shadlen, M. N., Wong-Lin, K., & Kelly, S. P. (2018) · Trends in Neurosciences · doi:10.1016/j.tins.2018.06.005 · Cited by 203
Circuit mechanisms for the maintenance and manipulation of information in working memory
Masse, N. Y., Yang, G. R., Song, H. F., Wang, X.-J., & Freedman, D. J. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0414-3 · Cited by 214
“Here, we train biologically inspired RNN models, consisting of excitatory and inhibitory like neurons 30 and dynamic synapses governed by STP 18 , to solve a variety of widely studied WM based tasks.”
MethodInfluentialSelf-citationDeep neural networks in computational neuroscience
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019) · Oxford Research Encyclopedia of Neuroscience · doi:10.1093/acrefore/9780190264086.013.46 · Cited by 186
Reward-based training of recurrent neural networks for cognitive and value-based tasks
Song, H. F., Yang, G. R., & Wang, X.-J. (2017) · eLife · doi:10.7554/elife.21492 · Cited by 175
Self-citationMacpherson, T., Churchland, A. K., Sejnowski, T., DiCarlo, J. J., Kamitani, Y., Takahashi, H., & Hikida, T. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.018 · Cited by 190
Low-dimensional dynamics for working memory and time encoding
Cueva, C. J., Saez, A., Marcos, E., Genovesio, A., Jazayeri, M., Romo, R., Salzman, C. D., Shadlen, M. N., & Fusi, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915984117 · Cited by 181
Reconstructing computational system dynamics from neural data with recurrent neural networks
Durstewitz, D., Koppe, G., & Thurm, M. I. (2023) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00740-7 · Cited by 230
“Various RNN architectures have been proposed over the past few decades, some formulated in discrete time (as time-recursive equations) 35,136,137 and some in continuous time (as systems of ordinary or partial differential equations) 83,130,138,139 .”
BackgroundJoglekar, M. R., Mejias, J. F., Yang, G. R., & Wang, X.-J. (2018) · Neuron · doi:10.1016/j.neuron.2018.02.031 · Cited by 153
Self-citationCortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference
Echeveste, R., Aitchison, L., Hennequin, G., & Lengyel, M. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0671-1 · Cited by 171
Kim, R., & B, T. J. S. A. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-00753-w · Cited by 149
Miconi, T. (2017) · eLife · doi:10.7554/elife.20899 · Cited by 143
“, 2013), and also used by Song and colleagues (Song et al., 2016) (see Figure 5).”
BackgroundMethodInfluentialComputational principles and models of multisensory integration
Chandrasekaran, C. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.11.002 · Cited by 103
A Dynamical Systems Perspective on Flexible Motor Timing
Remington, E. D., Egger, S. W., Narain, D., Wang, J., & Jazayeri, M. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.07.010 · Cited by 111
Laminar differences in decision-related neural activity in dorsal premotor cortex
Chandrasekaran, C., Peixoto, D., Newsome, W. T., & Shenoy, K. V. (2017) · Nature Communications · doi:10.1038/s41467-017-00715-0 · Cited by 103
Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network
Gilra, A., & Gerstner, W. (2017) · eLife · doi:10.7554/elife.28295 · Cited by 100
“, 2014) or stochastic gradient descent (Song et al., 2016) have also been applied in recurrent networks of neurons, but with limited biological plausibility of the published learning rules.”
BackgroundA Review of Formal Methods applied to Machine LearningPreprint
Urban, C., & Miné, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02466 · Cited by 131
“They focus on proving safety properties of vanilla recurrent neural networks with ReLU activations trained for cognitive tasks [144].”
BackgroundKoppe, G., Toutounji, H., Kirsch, P., Lis, S., & Durstewitz, D. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007263 · Cited by 120
“neural rate equations that have been used to model various neural and cognitive phenomena [37,38], and–due to their piecewise-linear structure–can also be easily translated into an equivalent continuous-time neural rate model [see 69].”
MethodThalamus-driven functional populations in frontal cortex support decision-making
Yang, W., Tipparaju, S. L., Chen, G., & Li, N. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01171-w · Cited by 86
Quantum Reservoir Computing Using Arrays of Rydberg Atoms
Bravo, R. A., Najafi, K., Gao, X., & Yelin, S. F. (2022) · PRX Quantum · doi:10.1103/prxquantum.3.030325 · Cited by 119
Psychiatric Illnesses as Disorders of Network Dynamics
Durstewitz, D., Huys, Q. J. M., & Koppe, G. (2020) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2020.01.001 · Cited by 100
Local online learning in recurrent networks with random feedback
Murray, J. M. (2019) · eLife · doi:10.7554/elife.43299 · Cited by 126
Synaptic wiring motifs in posterior parietal cortex support decision-making
Kuan, A. T., Bondanelli, G., Driscoll, L. N., Han, J., Kim, M., Hildebrand, D. G. C., Graham, B. J., Wilson, D. E., Thomas, L. A., Panzeri, S., et al. (2024) · Nature · doi:10.1038/s41586-024-07088-7 · Cited by 92
“4e), and trained the connection weights of the RNNs to reproduce the measured calcium activity for left and right trials 39,40 (Methods).”
MethodBlauch, N. M., Behrmann, M., & Plaut, D. C. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2112566119 · Cited by 74
Ito, T., Yang, G. R., Laurent, P. A., Schultz, D. H., & Cole, M. W. (2022) · Nature Communications · doi:10.1038/s41467-022-28323-7 · Cited by 95
Self-citationEfficient probabilistic inference in generic neural networks trained with non-probabilistic feedback
Orhan, A. E., & Ma, W. J. (2017) · Nature Communications · doi:10.1038/s41467-017-00181-8 · Cited by 77
“More powerful architectures that combine a neural network controller with an external memory can both learn faster and generalize better [13], offering a promising direction for modeling the generalization patterns of observers in simple psychophysical tasks.”
BackgroundSimple framework for constructing functional spiking recurrent neural networks
Kim, R., Li, Y., & B, T. J. S. A. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905926116 · Cited by 91
“Gradient descent-based methods, 34 including Hessian-free methods, have been also successfully applied to train rate networks in a 35 supervised manner and to replicate the computational dynamics observed in networks from be36 having animals [7, 12, 13].”
BackgroundMethodInfluentialDeep Neural Networks in Computational NeurosciencePreprint
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/133504 · Cited by 108
“Recurrent architectures are an exciting tool for computational neuroscience and will likely allow for key insights into the recurrent computational dynamics of the brain, from sensory processing to flexible cognitive tasks (Song, Yang, & Wang, 2016, 2017).”
MethodModels that learn how humans learn: The case of decision-making and its disorders
Dezfouli, A., Griffiths, K. R., Ramos, F. T., Dayan, P., & Balleine, B. W. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006903 · Cited by 91
“, 2018), perceptual decision-making, performance in cognitive tasks, 377 working-memory (Miconi, 2017; Carnevale et al., 2015; Mante et al., 2013; Song et al., 2016; Barak et al., 378 2013; Yang et al., 2017), motor patterns, motor reach and timing (Sussillo et al.”
BackgroundMethodUnderstanding the computation of time using neural network models
Bi, Z., & Zhou, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1921609117 · Cited by 72
“We stopped training as soon as the performance of the network reached criterion (23, 25) (performance examples are in SI Appendix , Fig.”
Durstewitz, D. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005542 · Cited by 87
Measurement, manipulation and modeling of brain-wide neural population dynamics
Shenoy, K. V., & Kao, J. C. (2021) · Nature Communications · doi:10.1038/s41467-020-20371-1 · Cited by 73
Wang, C., Zhang, T., Chen, X., He, S., Li, S., & Wu, S. (2023) · eLife · doi:10.7554/elife.86365 · Cited by 70
Hong, C., Wei, X., Wang, J., Deng, B., Yu, H., & Che, Y. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919662 · Cited by 63
Connectome-based reservoir computing with the conn2res toolbox
Suárez, L. E., Mihalik, Á., Milisav, F., Marshall, K., Li, M., Vértes, P. E., Lajoie, G., & Mišić, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44900-4 · Cited by 82
The computational foundations of dynamic coding in working memory
Stroud, J. P., Duncan, J., & Lengyel, M. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.02.011 · Cited by 65
Cerebro-cerebellar networks facilitate learning through feedback decoupling
Boven, E., Pemberton, J., Chadderton, P. T., Apps, R., & Costa, R. P. (2023) · Nature Communications · doi:10.1038/s41467-022-35658-8 · Cited by 59
How to study the neural mechanisms of multiple tasks
Yang, G. R., Cole, M. W., & Rajan, K. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.07.001 · Cited by 61
“How to train neural networks to be specialized or flexible In neuroscience, it is increasingly common to compare biological neural circuits with artificial ones [65,64,40,53,54].”
BackgroundSelf-citationFronto-parietal networks shape human conscious report through attention gain and reorienting
Liu, J., Bayle, D. J., Spagna, A., Sitt, J. D., Bourgeois, A., Lehongre, K., Fernández-Vidal, S., Adam, C., Lambrecq, V., Navarro, V., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-05108-2 · Cited by 60
“The connections between hidden units were constrained according to Dale ’ s principle: neurotransmitters tend to be either excitatory or inhibitory such that the post-synaptic weights of each recurrent unit are all of the same sign 78 .”
MethodOptimal information loading into working memory explains dynamic coding in the prefrontal cortex
Stroud, J. P., Watanabe, K., Suzuki, T., Stokes, M. G., & Lengyel, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2307991120 · Cited by 66
The cortical connectome of primate lateral prefrontal cortex
Xu, R., Bichot, N. P., Takahashi, A., & Desimone, R. (2021) · Neuron · doi:10.1016/j.neuron.2021.10.018 · Cited by 58
Latent circuit inference from heterogeneous neural responses during cognitive tasks
Langdon, C., & Engel, T. A. (2025) · Nature Neuroscience · doi:10.1038/s41593-025-01869-7 · Cited by 63
“nature neuroscience https://doi.org/10.1038/s41593-025-01869-7”
InfluentialGeometry of neural computation unifies working memory and planning
Ehrlich, D. B., & Murray, J. D. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2115610119 · Cited by 51
“Neurobiologically motivated properties, such as Dale’s principle (57, 58), short-term synaptic plasticity (59), or multiregional connectivity constraints (60), could be explored for their effects on the emergence or structure of contingency representations.”
BackgroundFascianelli, V., Battista, A., Stefanini, F., Tsujimoto, S., Genovesio, A., & Fusi, S. (2024) · Nature Communications · doi:10.1038/s41467-024-50503-w · Cited by 64
Brain-Inspired Motion Learning in Recurrent Neural Network With Emotion Modulation
Huang, X., Wu, W. L., Qiao, H., & Ji, Y. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2843563 · Cited by 48
“For training these networks, there are two classes of methods: 1) supervised learning [8], [11], [12] and”
BackgroundMethodLearning to live with Dale’s principle: ANNs with separate excitatory and inhibitory unitsPreprint
Cornford, J. H., Kalajdzievski, D., Leite, M., Lamarquette, A., Kullmann, D. M., & Richards, B. A. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.11.02.364968 · Cited by 62
The dynamics and geometry of choice in the premotor cortex
Genkin, M., Shenoy, K. V., Chandrasekaran, C., & Engel, T. A. (2025) · Nature · doi:10.1038/s41586-025-09199-1 · Cited by 50
Goulas, A., Damicelli, F., & Hilgetag, C. C. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.011 · Cited by 50
Shahnazian, D., & Holroyd, C. B. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1280-1 · Cited by 45
“Although notoriously difficult to train (Sussillo, 2014), recent advances (e.g., Martens, 2010; Martens & Sutskever, 2011; Song, Yang, & Wang, 2016) have been encouraging (Abbott, DePasquale, & Memmesheimer, 2016; Ardid & Wang, 2013; Song et al., 2016; Sussillo & Abbott, 2009).”
Recent Advances at the Interface of Neuroscience and Artificial Neural Networks
Cohen, Y., Engel, T. A., Langdon, C., Lindsay, G. W., Ott, T., Peters, M. A. K., Shine, J. M., Breton‐Provencher, V., & Ramaswamy, S. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.1503-22.2022 · Cited by 49
“3) (Mante et al., 2013; Song et al., 2016; Wang et al., 2018; Yang et al., 2019) and how networks leverage previously learned tasks for continual learning (Driscoll et al., 2022).”
BackgroundTraining dynamically balanced excitatory-inhibitory networks
Ingrosso, A., & Abbott, L. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0220547 · Cited by 40
Neural inhibition for continual learning and memory
Barron, H. C. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.09.007 · Cited by 46
“Incorporating separate excitatory and inhibitory populations [55] and the full diversity of different inhibitory subtypes in artificial neural networks may yield further insight into the functional role of neural inhibition in continual learning.”
Mnemonic Encoding and Cortical Organization in Parietal and Prefrontal Cortices
Masse, N. Y., Hodnefield, J. M., & Freedman, D. J. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.3903-16.2017 · Cited by 45
“Thus, we trained recurrent neural networks using the Pycog framework (Song et al., 2016) to perform a variant of the DMS task described above (see Materials and Methods).”
BackgroundMethodInfluentialDeistler, M., Kadhim, K. L., Pals, M., Beck, J., Huang, Z., Gloeckler, M., Lappalainen, J. K., Schröder, C., Berens, P., Gonçalves, P. J., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02895-w · Cited by 59
Beyond dimension reduction: stable electric fields emerge from and allow representational drift
Pinotsis, D. A., & Miller, E. K. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119058 · Cited by 37
Choice selective inhibition drives stability and competition in decision circuits
Roach, J. P., Churchland, A. K., & Engel, T. A. (2023) · Nature Communications · doi:10.1038/s41467-023-35822-8 · Cited by 40
Inhibitory Plasticity: From Molecules to Computation and Beyond
Gandolfi, D., Bigiani, A., Porro, C. A., & Mapelli, J. (2020) · International Journal of Molecular Sciences · doi:10.3390/ijms21051805 · Cited by 36
“Additionally, the reciprocal and recurrent broad connections between inhibitory and excitatory neurons are ideal to condition large circuits areas [40].”
BackgroundModelling human behaviour in cognitive tasks with latent dynamical systems
Jaffe, P. I., Poldrack, R. A., Schafer, R. J., & Bissett, P. G. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01510-8 · Cited by 51
“in which the densities defined in equations (6) and (13) have been inserted into equation (14).”
BackgroundDistributed functions of prefrontal and parietal cortices during sequential categorical decisions
Zhou, Y., Rosen, M. C., Swaminathan, S. K., Masse, N. Y., Zhu, O., & Freedman, D. J. (2021) · eLife · doi:10.7554/elife.58782 · Cited by 38
Ehrlich, D. B., Stone, J. T., Brandfonbrener, D., Atanasov, A., & Murray, J. D. (2020) · eNeuro · doi:10.1523/eneuro.0427-20.2020 · Cited by 39
“Third, task definitions in PyCog are not themselves modular, making experiments which are trivial to implement in PsychRNN more laborious and cumbersome for the user.”
BackgroundMethodInfluentialThe controllosphere: The neural origin of cognitive effort.
Holroyd, C. B. (2024) · Psychological Review · doi:10.1037/rev0000467 · Cited by 39
Massot, C., Jagadisan, U. K., & Gandhi, N. J. (2019) · Communications Biology · doi:10.1038/s42003-019-0527-y · Cited by 34
Natural statistics support a rational account of confidence biases
Webb, T. W., Miyoshi, K., So, T. Y., Rajananda, S., & Lau, H. (2023) · Nature Communications · doi:10.1038/s41467-023-39737-2 · Cited by 31
Liebe, S., Niediek, J., Pals, M., Reber, T. P., Faber, J., Boström, J. P., Elger, C. E., Macke, J. H., & Mormann, F. (2025) · Nature Neuroscience · doi:10.1038/s41593-025-01893-7 · Cited by 38
How thoughts arise from sights: inferotemporal and prefrontal contributions to vision
Kornblith, S., & Tsao, D. Y. (2017) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.08.016 · Cited by 32
Liquid-Based Memory Devices for Next-Generation Computing
Kim, D., & Lee, J. (2023) · ACS Applied Electronic Materials · doi:10.1021/acsaelm.2c01636 · Cited by 30
Identifying nonlinear dynamical systems with multiple time scales and long-range dependenciesPreprint
Schmidt, D., Koppe, G., Monfared, Z., Beutelspacher, M., & Durstewitz, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03471 · Cited by 32
“This specific formulation is originally motivated by firing rate (population) models in computational neuroscience (Song et al., 2016; Durstewitz, 2017), where latent states z t may represent membrane voltages or currents, A the neurons’ passive time constants, W the synaptic coupling among neurons,…”
Cueva, C. J., Wang, P. Y., Chin, M., & Wei, X.-X. (2019) · arXiv · doi:10.48550/arxiv.1912.10189 · Cited by 27
“…trained on cognitive tasks have also been used to account for neural response characteristics in various domains (Mante et al., 2013; Sussillo et al., 2015; Song et al., 2016; Cueva & Wei, 2018; Banino et al., 2018; Remington et al., 2018; Wang et al., 2018; Orhan & Ma, 2019; Yang et al., 2019).”
Towards biologically constrained attractor models of schizophrenia
Stein, H., Barbosa, J., & Compte, A. (2021) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2021.10.013 · Cited by 28
Multiplexing working memory and time in the trajectories of neural networks
Zhou, S., Seay, M. J., Taxidis, J., Golshani, P., & Buonomano, D. V. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01592-y · Cited by 29
Improving multi-layer spiking neural networks by incorporating brain-inspired rules
Zeng, Y., Zhang, T., & Xu, B. (2017) · Science China Information Sciences · doi:10.1007/s11432-016-0439-4 · Cited by 26
Considerations in using recurrent neural networks to probe neural dynamics
Kao, J. C. (2019) · Journal of Neurophysiology · doi:10.1152/jn.00467.2018 · Cited by 26
Adaptive time scales in recurrent neural networks
Quax, S. C., D’Asaro, M., & van Gerven, M. A. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-68169-x · Cited by 34
Encoding time in neural dynamic regimes with distinct computational tradeoffs
Zhou, S., Masmanidis, S. C., & Buonomano, D. V. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009271 · Cited by 26
Distributed context-dependent choice information in mouse posterior cortex
Orlandi, J. G., Abdolrahmani, M., Aoki, R., Lyamzin, D. R., & Benucci, A. (2023) · Nature Communications · doi:10.1038/s41467-023-35824-6 · Cited by 28
One Step Back, Two Steps Forward: Interference and Learning in Recurrent Neural Networks
Beer, C., & Barak, O. (2019) · Neural Computation · doi:10.1162/neco_a_01222 · Cited by 26
“In order to elucidate simple learning tasks, recurrent neural networks were designed [8–11] and trained [2, 12–15] to memorize analog values or periodic patterns.”
MethodJeon, I., & Kim, T. (2023) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2023.1092185 · Cited by 27
“In ANN applications (Song et al., 2016; Ingrosso and Abbott, 2019; Tian et al., 2020), balanced inputs are utilized to optimize neural networks for better performance, with the advantages shown in BNN models.”
MethodGabbassov, E., Moreno-Molina, M., Comas, I., Libbrecht, M. W., & Chindelevitch, L. (2021) · Microbial Genomics · doi:10.1099/mgen.0.000607 · Cited by 21
Podlaski, W. F., & Machens, C. K. (2024) · Neural Computation · doi:10.1162/neco_a_01658 · Cited by 22
Rungratsameetaweemana, N., Kim, R., Chotibut, T., & B, T. J. S. A. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2316745122 · Cited by 24
Mediodorsal thalamus regulates task uncertainty to enable cognitive flexibility
Zhang, X., Mukherjee, A., Halassa, M. M., & Chen, Z. S. (2025) · Nature Communications · doi:10.1038/s41467-025-58011-1 · Cited by 42
“Recurrent neuralnetworks (RNNs) have become an important reverse-engineering tool for dissecting circuit functions 24 – 30 .”
Learning better with Dale’s Law: A Spectral Perspective
Li, P., Cornford, J. H., Ghosh, A., & Richards, B. (2023) · neural information processing systems · doi:10.52202/075280-0045 · Cited by 25
Slow manifolds within network dynamics encode working memory efficiently and robustly
Ghazizadeh, E., & Ching, S. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009366 · Cited by 23
Trained recurrent neural networks develop phase-locked limit cycles in a working memory task
Pals, M., Macke, J. H., & Barak, O. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011852 · Cited by 34
“Having described the dynamics, we next aimed to understand the population structure that gives rise to these dynamics.”
Mice Preferentially Use Increases in Cerebral Cortex Spiking to Detect Changes in Visual Stimuli
Cone, J. J., Bade, M. L., Masse, N. Y., Page, E. A., Freedman, D. J., & Maunsell, J. H. R. (2020) · Journal of Neuroscience · doi:10.1523/jneurosci.1124-20.2020 · Cited by 17
“The activity of the recurrent neurons was modeled to follow the dynamical system (Song et al., 2016) as follows:”
MethodInfluentialModeling Neurodegeneration in silico With Deep Learning
Tuladhar, A., Moore, J. A., Ismail, Z., & Forkert, N. D. (2021) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2021.748370 · Cited by 21
“, 2019; Marcin and Maciej, 2020), or specifically targeting excitatory (positive) or inhibitory (negative) connections (Song et al., 2016; Mackwood et al., 2021) may be instructive.”
BackgroundRecurrent neural network models of multi-area computation underlying decision-makingPreprint
Kleinman, M., Chandrasekaran, C., & Kao, J. C. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/798553 · Cited by 19
“Prior studies modeling cognitive tasks have employed early stopping to reproduce animal behavior Mante et al. (2013), Song et al. (2016).”
BackgroundMethodInfluentialWang, Q., Li, Z., Nie, D., Mu, X., Wang, Y., Jiang, Y., Zhang, Y., & Lu, Z. (2024) · Molecular Pain · doi:10.1177/17448069241254201 · Cited by 22
“23”
Adaptive robustness through incoherent signaling mechanisms in a regenerative brain
Bray, S. R., Wyss, L. S., Chai, C., Lozada, M. E., & Wang, B. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114580 · Cited by 15
“45 Planarians have densely packed peptidergic neurons collectively expressing a suite of over 60 neuropeptides, most, if not all, of which are coexpressed with small-molecule neuromodulator synthesis genes.”
Influential
Metadata
- Authors
- Song, H. F., Yang, G. R., & Wang, X.-J.
- Affiliations
- New York University (US)
- New York University Shanghai (CN)
- Corresponding author
- Wang, Xiao-Jing
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-02-29
- Volume
- 12
- Issue
- 2
- Pages
- e1004792
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC4771709
- OpenAlex
- W2280725856
- Cited by
- 281
- References
- 91
- Citations by year
- 2026: 23 · 2025: 33 · 2024: 39 · 2023: 42 · 2022: 26 · 2021: 28 · 2020: 23 · 2019: 27 · 2018: 14 · 2017: 21 · 2016: 5
- FWCI
- 14.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Office of Naval Research (N00014–13–1-0297, N00014)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- Dale's principle, neural circuit model, cognitive tasks, perceptual decision making, multisensory integration, working memory, neural population dynamics, Theano, automatic differentiation, gradient-based learning
- MeSH terms
- Algorithms, Animals, Haplorhini, Cognition, Computer Simulation, Models, Neurological, Neurons, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.