Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework
Open accessThe DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
Abstract
The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze model recurrent neural networks (RNNs) that have been optimized to perform the same tasks as behaving animals. Because the optimization of network parameters specifies the desired output but not the manner in which to achieve this output, "trained" networks serve as a source of mechanistic hypotheses and a testing ground for data analyses that link neural computation to behavior. Complete access to the activity and connectivity of the circuit, and the ability to manipulate them arbitrarily, make trained networks a convenient proxy for biological circuits and a valuable platform for theoretical investigation. However, existing RNNs lack basic biological features such as the distinction between excitatory and inhibitory units (Dale's principle), which are essential if RNNs are to provide insights into the operation of biological circuits. Moreover, trained networks can achieve the same behavioral performance but differ substantially in their structure and dynamics, highlighting the need for a simple and flexible framework for the exploratory training of RNNs. Here, we describe a framework for gradient descent-based training of excitatory-inhibitory RNNs that can incorporate a variety of biological knowledge. We provide an implementation based on the machine learning library Theano, whose automatic differentiation capabilities facilitate modifications and extensions. We validate this framework by applying it to well-known experimental paradigms such as perceptual decision-making, context-dependent integration, multisensory integration, parametric working memory, and motor sequence generation. Our results demonstrate the wide range of neural activity patterns and behavior that can be modeled, and suggest a unified setting in which diverse cognitive computations and mechanisms can be studied.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



![Fig 4 Context-dependent integration task. (A) Psychometric function, showing the percentage of trials on which the network chose choice 1 as a function of the signed motion (upper) and signed color (lower) coherence in motion-context (black) and color-context (blue) trials. (B) Average population responses in state space during the stimulus period, projected to the 3-dimensional subspace capturing variance due to choice, motion, and color as in [5]. Only correct trials were included. The task-related axes were obtained through a linear regression analysis. Note that “choice” here has a unit-specific meaning that depends on the preferred choice of the unit as determined by the selectivity index (Eq 30). For both motion (black) and color (blue), coherences increase from light to dark. Upper plots show trials during the motion context, and lower plots show trials during the color context. (C) Normalized responses of four recurrent units during the stimulus period show mixed representation of task variables. Solid lines indicate the preferred choice and dashed lines the nonpreferred choice of each unit. (D) Denoised regression coefficients from the linear regression analysis. By definition, the coefficients for choice are almost exclusively positive.](https://figures.cochat.ai/paper-figures/ca3a2b623c1e4966aef22596dc894038/training-excitatory-inhibitory-recurrent-neural-networks-for-cognitive-tasks-a-figure-4.jpg)





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“and which can be viewed as an approximation to continuous time firing rate models of neural circuits (Song et al., 2016).”
From Remapping in a recurrent neural network model of navigation and context inference (2023) · cites this paper as Song et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“For instance, machine learning applications typically employ a type of RNN known as Long Short-Term Memory (LSTM), which uses multiplicative gates to facilitate learning of long-term dependencies and currently represents one of the most powerful methods for solving sequence-related problems [72].”
MethodReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“Combining RNN training with reinforcement learning methods [75 – 77] will be essential to bringing network training closer to the reward-based manner in which animals are trained.”
MethodReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Even so, SGD with the objective function given in Eqs 15 and 16 often fails to converge to a solution when the network must learn dependencies between distant time points [60].”
BackgroundOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 4×
“To remedy this problem, which is due to some gradient components being too large ( exploding gradients) and some gradient components being too small ( vanishing gradients), we follow [35] in making two modifications.”
BackgroundMethodInfluential(2013) · Cited by 3,740
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
Interneurons of the neocortical inhibitory system
Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1519 · Cited by 2,954 · cited 2×
“Local cortical circuits have distance [48], layer [26, 49, 50], and cell-type [23, 25, 27, 51] dependent patterns of connectivity and different overall levels of sparseness for excitatory to excitatory, inhibitory to excitatory, excitatory to inhibitory, and inhibitory to inhibitory connections [52, 53].”
Background
Research works printed but not linked: 2
- Bengio Y. 2015 Towards biologically plausible deep learning;Available from: http://arxiv.org/abs/1502.0415.
- DE Rumelhart, 1986, Parallel Distrib. Process, Vol. 1, 318
Cited by
Items 101–200 of 273
Neural timescales from a computational perspective
Zeraati, R., Levina, A., Macke, J. H., & Gao, R. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02343-8 · Cited by 37
“These ideas can be further extended to study networks endowed with more sophisticated and biologically-realistic constraints such as Dale’s law [114, 156] and dendritic compartmentalization [157], hypothesized computational mechanisms such as oscillatory phase coding [158, 159], as well as better…”
Learning Long Temporal Sequences in Spiking Networks by Multiplexing Neural Oscillations
Vincent‐Lamarre, P., Calderini, M., & Thivierge, J. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00078 · Cited by 18
Boven, E., & Cerminara, N. L. (2023) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2023.1211530 · Cited by 18
“Artificial recurrent neural network models have been successful at approximating the cerebrum as a dynamical system (Laje and Buonomano, 2013; Mante et al., 2013; Rajan et al., 2016; Frontiers in Systems Neuroscience 03 frontiersin.org 10.3389/fnsys.2023.1211530 Song et al., 2016).”
BackgroundInfluentialvan Holk, M., & Mejías, J. F. (2024) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2024.101351 · Cited by 18
“…neural networks (ANNs) designed for processing sequential data, RNNs retain a good level of neurobiological plausibility compared with other ANNs such as deep networks, due to their recurrent connection structure (Figure 1a) [16] and additional realistic features introduced in recent works [17–19].”
A multi-region recurrent circuit for evidence accumulation in rats
Gupta, D., Kopec, C. D., Bondy, A., Luo, T. Z., Elliott, V. A., & Brody, C. D. (2026) · Neuron · doi:10.1016/j.neuron.2025.12.029 · Cited by 18
Anwar, H., Caby, S., Durá-Bernal, S., D’Onofrio, D., Haşegan, D., Deible, M., Grunblatt, S., Chadderdon, G. L., Kerr, C. C., Lakatos, P., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0265808 · Cited by 16
Single-unit activations confer inductive biases for emergent circuit solutions to cognitive tasks
Tolmachev, P., & Engel, T. A. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01127-2 · Cited by 14
“We used three common activation functions (ReLU, sigmoid and tanh) and, for each, trained RNNs with and without Dale’s law constraint on the connectivity (restricting units to be either excitatory or inhibitory) 3,27 , a fundamental feature of cortical circuits.”
InfluentialExcitatory-inhibitory recurrent dynamics produce robust visual grids and stable attractors
Zhang, X., Long, X., Zhang, S., & Chen, Z. S. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.111777 · Cited by 14
“Figure 2: Impact of Sequence Length on Grid Patterns in Trained RNNs (A) Grid-like patterns were robust with respect to a wide range of sequence lengths (200 ms, 400 ms, 600 ms, and 1000 ms) and activation functions in the trained E/I-RNN.”
MethodInfluentialYu, H., & Zhao, Q. (2023) · Cognitive Neurodynamics · doi:10.1007/s11571-023-09932-4 · Cited by 12
“To accurately perform the same tasks as behaving animals, Song et al. (2016) developed a simple and flexible framework for training excitatory-in-hibitory RNNs that incorporate a variety of biological knowledge, and applied it to cognitive tasks of perceptual decision-making, working memory and…”
Rajakumar, A. A. A., Rinzel, J., & Chen, Z. S. (2021) · Neural Computation · doi:10.1162/neco_a_01418 · Cited by 13
“To avoid local minima, we trained multiple excitatory-inhibitory RNNs with different random seeds, hyperparameters, and initial conditions in parallel.”
BackgroundMethodInfluentialXue, X., Wimmer, R. D., Halassa, M. M., & Chen, Z. S. (2022) · Cognitive Computation · doi:10.1007/s12559-022-09994-2 · Cited by 12
“Mean±SD statistics were computed from 10 untrained and 10 trained SRNNs. * **, p < 10 −3 .”
BackgroundMethodResultInfluentialMei, J., Rodriguez-Garcia, A., Takeuchi, D., Wainstein, G., Hubig, N., Mohsenzadeh, Y., & Ramaswamy, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.06762 · Cited by 15
“A few DNN and recurrent neural network (RNN) architectures have incorporated these features, often adopting the biological brain’s experimental 80:20 excitation:inhibition ratio (Cornford et al., 2020; Kao, 2019; Li et al., 2023; Song et al., 2016).”
Flexible gating between subspaces in a neural network model of internally guided task switching
Liu, Y., & Wang, X. (2024) · Nature Communications · doi:10.1038/s41467-024-50501-y · Cited by 15
Self-citationDynamical mechanisms of how an RNN keeps a beat, uncovered with a low-dimensional reduced model
Zemlianova, K. O., Bose, A., & Rinzel, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-77849-x · Cited by 12
Gating Attractor Dynamics of Frontal Cortex Under Acupuncture via Recurrent Neural Network
Li, K., Wang, J., Hu, Z., Deng, B., & Yu, H. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3158963 · Cited by 11
“[20] train excitatory-inhibitory RNN to decode paradigms such as perceptual decision-making and multisensory integration.”
BackgroundOn the neural networks of self and other bias and their role in emergent social interactions
Forbes, C. E. (2024) · Cortex · doi:10.1016/j.cortex.2024.05.002 · Cited by 11
31st Annual Computational Neuroscience Meeting: CNS*2022
Corsi, M., Sorrentino, P., Schwartz, D., George, N., Hugueville, L., Kahn, A. E., Dupont, S., Bassett, D. S., Jirsa, V., & Fallani, F. D. V. (2023) · Journal of Computational Neuroscience · doi:10.1007/s10827-022-00841-9 · Cited by 13
Neuromodulators Generate Multiple Context-Relevant Behaviors in Recurrent Neural Networks
Tsuda, B., Pate, S. C., Tye, K. M., Siegelmann, H. T., & B, T. J. S. A. (2026) · Neural Computation · doi:10.1162/neco.a.1489 · Cited by 21
“Consistent with biological neural networks, we implemented Dale’s Law using the method in Song et al. (2016) such that each neuron was either excitatory or inhibitory.”
Dynamics of spiking map-based neural networks in problems of supervised learning
Pugavko, M. M., Maslennikov, O. V., & Nekorkin, V. I. (2020) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2020.105399 · Cited by 11
Wainstein, G., Whyte, C. J., Martens, K. A. E., Müller, E. J., Munn, B., Medel, V., Anderson, B., Stöttinger, E., Danckert, J., & Shine, J. M. (2024) · eLife · doi:10.7554/elife.93191.4 · Cited by 13
Multi-context blind source separation by error-gated Hebbian rule
Isomura, T., & Toyoizumi, T. (2019) · Scientific Reports · doi:10.1038/s41598-019-43423-z · Cited by 11
Biologically Plausible Training Mechanisms for Self-Supervised Learning in Deep Networks
Tang, M., Yang, Y., & Amit, Y. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.789253 · Cited by 14
“, 2016; Yamins and DiCarlo, 2016) and those using recurrent neural networks (RNNs) as models of generic cortical computation (Song et al., 2016; Masse et al., 2019) have suggested that ANNs are not only able to replicate behavioral activities (e.”
BackgroundComputational mechanisms of distributed value representations and mixed learning strategies
Farashahi, S. G., & Soltani, A. (2021) · Nature Communications · doi:10.1038/s41467-021-27413-2 · Cited by 11
Zhang, X., Liu, S., & Chen, Z. S. (2021) · iScience · doi:10.1016/j.isci.2021.102919 · Cited by 12
Implementation of a Minimal Recurrent Spiking Neural Network in a Solid-State Device
Stoliar, P., Schneegans, O., & Rozenberg, M. J. (2021) · Physical Review Applied · doi:10.1103/physrevapplied.16.034030 · Cited by 12
Cortical interneurons: fit for function and fit to function? Evidence from development and evolution
Keijser, J., & Sprekeler, H. (2023) · Frontiers in Neural Circuits · doi:10.3389/fncir.2023.1172464 · Cited by 9
“Recently, 301 such deep network models have also started to include circuit-level structure such as separate excitatory and 302 inhibitory populations [131, 132], different neuronal timescales [133, 134], and short-term plasticity [135, 15].”
BackgroundDeep Learning Neural Encoders for Motor Cortex
Liang, K.-F., & Kao, J. C. (2019) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2955722 · Cited by 9
Short-term memory by transient oscillatory dynamics in recurrent neural networks
Ichikawa, K., & Kaneko, K. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.033193 · Cited by 12
A Midbrain Inspired Recurrent Neural Network Model for Robust Change Detection
Sawant, Y., Kundu, J. N., Babu, R. V., & Sridharan, D. (2022) · Journal of Neuroscience · doi:10.1523/jneurosci.0164-22.2022 · Cited by 11
“These include cognitive tasks involving perceptual decision-making, multisensory integration, working memory (Song et al., 2016), delayed estimation, change detection, forced choice comparison (Orhan and Ma, 2019), signal detection and context dependent discrimination (Mastrogiuseppe and Ostojic,…”
BackgroundMethodInfluentialMaslennikov, O. V., & Nekorkin, V. I. (2020) · Nonlinear Dynamics · doi:10.1007/s11071-020-05787-0 · Cited by 9
Sensory processing and categorization in cortical and deep neural networks
Pinotsis, D. A., Siegel, M., & Miller, E. K. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116118 · Cited by 9
Miconi, T. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/057729 · Cited by 9
“This task was also used as an example application by Song and colleagues for their recurrent network training framework (Song et al., 2016) .”
MethodInfluentialAn Evolutionary Framework for Replicating Neurophysiological Data with Spiking Neural Networks
Rounds, E. L., Scott, E. O., Alexander, A. S., Jong, K. A. D., Nitz, D. A., & Krichmar, J. L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45823-6_50 · Cited by 8
“Existing work in the area of SNN synthesis has either trained recurrent networks to match high-level animal behavior in cognitive tasks [7,13,17], or it has focused on tuning the parameters of individual neuron models to match electrophysiological data [8,14–16].”
BackgroundWierda, T. S., Dora, S., Pennartz, C. M. A., & Mejías, J. F. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013559 · Cited by 8
“We trained 200 Recurrent neural networks (RNNs) based on earlier work [26,28].”
InfluentialAn Overview of Open Source Deep Learning-Based Libraries for Neuroscience
Tshimanga, L. F., Pup, F. D., Corbetta, M., & Atzori, M. (2023) · Applied Sciences · doi:10.3390/app13095472 · Cited by 8
Non-linear motor control by local learning in spiking neural networksPreprint
Gilra, A., & Gerstner, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.10158 · Cited by 8
“…of non-linear systems seem to use non-local rules (Sanner & Slotine, 1992; Slotine & Weiping Li, 1987; Slo-tine & Coetsee, 1986; DeWolf et al., 2016; Zerkaoui et al., 2009; Hennequin et al., 2014; Song et al., 2016) or use abstract networks (Berniker & Kording, 2015; Hanuschkin et al., 2013).”
BackgroundUnconditional stability of a recurrent neural circuit implementing divisive normalization
Rawat, S., Heeger, D. J., & Martiniani, S. (2024) · neural information processing systems · doi:10.52202/079017-0470 · Cited by 12
“Song et al. [56] incorporated Dale’s law into the vanilla RNN architecture, which was successfully trained across a variety of cognitive tasks.”
Decoding Covert Visual Attention in Space and Time from Neural Signals
Hamed, S. B. (2025) · Annual Review of Vision Science · doi:10.1146/annurev-vision-101322-011902 · Cited by 7
Battista, A., Padoa‐Schioppa, C., & Wang, X.-J. (2026) · Neuron · doi:10.1016/j.neuron.2025.12.010 · Cited by 7
“Our model consists of a continuous-time recurrent neural network of the “vanilla” type (with no GRUs) that adheres to Dale’s law with 80% excitatory (E) and 20% inhibitory (I) neurons and long-range excitatory projections [19, 20].”
Self-citationLi, Y., Kim, R., & B, T. J. S. A. (2021) · Neural Computation · doi:10.1162/neco_a_01409 · Cited by 7
Emergent computations in trained artificial neural networks and real brains
Parga, N., Serrano-Fernández, L., & Falcó-Roget, J. (2023) · Journal of Instrumentation · doi:10.1088/1748-0221/18/02/c02060 · Cited by 8
“The success achieved in understanding the network mechanisms at work in various cognitive tasks [16, 17, 18, 19, 6, 7, 20, 13, 21, 22, 23] encourages”
Augmenting flexibility: mutual inhibition between inhibitory neurons expands functional diversity
Liu, B., White, A. J., & Lo, C. (2025) · iScience · doi:10.1016/j.isci.2024.111718 · Cited by 9
“In light of this, an efficient design is for neural networks to be capable of flexibly switching 24 between various functions upon demand [1, 2, 3, 4, 5, 6, 7, 8, 9], instead of having each task been performed by a 25 different network [1, 3, 10, 11, 12].”
BackgroundUnification of MAP Estimation and Marginal Inference in Recurrent Neural Networks
Yu, Z., Chen, F., & Deng, F. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805813 · Cited by 7
“A RNN is a ubiquitous motif of cortical microcircuits, which can perform the same cognitive tasks as animals [27].”
A Nonequilibrium-Potential Approach to Competition in Neural Populations
Deza, R. R., Deza, I., Martínez, N., Mejías, J. F., & Wio, H. S. (2019) · Frontiers in Physics · doi:10.3389/fphy.2018.00154 · Cited by 7
Testing the Tools of Systems Neuroscience on Artificial Neural NetworksPreprint
Lindsay, G. W. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.07035 · Cited by 8
“In addition to the caveats already given, there are ways, for example, to train networks that have different neuronal subtypes and obey Dale’s law [Song et al., 2016, Cornford et al., 2021].”
BackgroundEffects of noise and metabolic cost on cortical task representationsPreprint
Stroud, J. P., Wójcik, M. J., Jensen, K. T., Kusunoki, M., Kadohisa, M., Buckley, M. J., Duncan, J., Stokes, M. G., & Lengyel, M. (2024) · eLife · doi:10.7554/elife.94961 · Cited by 8
Zhang, Y., Feng, J., & Min, B. (2024) · eLife · doi:10.7554/elife.103636.3 · Cited by 8
Zou, L., Palomero‐Gallagher, N., Zhou, D., Li, S., & Mejías, J. F. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.12.26.573347 · Cited by 8
“…(Brunton et al., 2013; Roach et al., 2023; Wang, 2002; Wong and Wang, 2006) to context-dependent choices (Mante et al., 2013; Siegel et al., 2015; Song et al., 2016), multisensory integration (Chandrasekaran, 2017; Meijer et al., 2020; Raposo et al., 2014; Wierda et al., 2023) or even perceptual…”
BackgroundSoo, W. W. M., Goudar, V., & Wang, X.-J. (2023) · neural information processing systems · doi:10.52202/075280-1391 · Cited by 17
Self-citationKuroki, S., & Isomura, T. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00083 · Cited by 6
“where θ is the vectorized parameter set of the policy network for the optimization, E (θ)is the objective function of the policy network, Ntrials is the number of trials, Jn (θ)is the expected reward prediction, n (θ) is regularization term used to preserve the size of the gradients as errors are propagated through…”
BackgroundMethodInfluentialUsing Stigmergy to Incorporate the Time into Artificial Neural Networks
Galatolo, F. A., Cimino, M. G. C. A., & Vaglini, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-05918-7_22 · Cited by 6
“However, the training process of recurrent NNs strongly depends on the set of constraints and regularizations used in the optimization process, resulting in a not entirely unbiased task with respect to the “how” [5].”
BackgroundInferring Dynamical Systems with Long-Range Dependencies through Line Attractor Regularization.Preprint
Schmidt, D., Koppe, G., Beutelspacher, M., & Durstewitz, D. (2019) · arXiv (Cornell University) · Cited by 6
Deep Continuous NetworksPreprint
Tömen, N., Pintea, S. L., & van Gemert, J. C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.01557 · Cited by 6
“Similarly, optimization via gradient-based algorithms such as the Hessian-free method (HF) or stochastic gradient-descent (SGD) have been adopted to replicate experimental observations (Mante et al., 2013; Barak et al., 2013; Song et al., 2016).”
MethodStochastic activity in low-rank recurrent neural networks
Mastrogiuseppe, F., Carmona, J., & Machens, C. K. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013371 · Cited by 7
DropCircuit : A Modular Regularizer for Parallel Circuit Networks
Phan, K. T., Maul, T., Vu, T. T., & Lai, W. K. (2017) · Neural Processing Letters · doi:10.1007/s11063-017-9677-4 · Cited by 5
“Given that most biological neurons are either excitatory or inhibitory (and therefore not all non-negative) [38], we propose extending these concepts further.”
BackgroundMethodInfluentialOptimal learning with excitatory and inhibitory synapses
Ingrosso, A. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008536 · Cited by 7
“Even though the precise functional role of Dale’s law has not yet been understood, the importance of synaptic sign constraints is pivotal in constructing biologically plausible models of synaptic plasticity in the brain [1–5].”
BackgroundLiu, Q., Ulloa, A., & Horwitz, B. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.876652 · Cited by 5
Ensemble perspective for understanding temporal credit assignment
Zou, W., Li, C., & Huang, H. (2023) · Physical review. E · doi:10.1103/physreve.107.024307 · Cited by 6
“The noise term is also called the external sensory noise of strength σin, commonly observed in brain circuits [21, 22], but is absent in engineering applications.”
BackgroundMethodJarne, C., & Laje, R. (2023) · Journal of Computational Neuroscience · doi:10.1007/s10827-023-00857-9 · Cited by 6
“…and (Funahashi & Nakamura, 1993; Funahashi, 1989)) to model the mechanisms of a variety of systems, such as brain processes (Gerstner et al., 2012; Song et al., 2016; Michaels et al., 2016; Hoellinger et al., 2013; Pehlevan et al., 2018; Sussillo et al., 2014; Remington et al., 2018), stability…”
BackgroundMethodInfluentialLiu, Q., Chen, W., Qu, Y., & Liang, Z. (2024) · Advances in neurobiology · doi:10.1007/978-3-031-69188-1_3 · Cited by 6
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?
Bhatt, G., Bansal, H., Singh, R., & Agarwal, S. (2020) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2020.acl-srw.33 · Cited by 7
Flexible cognition in context-modulated reservoir networksPreprint
Masse, N. Y., Rosen, M. C., Tsao, D. Y., & Freedman, D. J. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.05.09.491102 · Cited by 6
Remapping in a recurrent neural network model of navigation and context inference
Low, I. I. C., Giocomo, L. M., & Williams, A. H. (2023) · doi:10.1101/2023.01.25.525596 · Cited by 3
“circuits.(Song et al., 2016) At each time index the activation vector of hidden t ∈ {1, ···, T} N”
Artificial intelligence as a surrogate brain: bridging neural dynamical models and data
Zhang, Y., Liu, D., Liang, Z., Cheng, J., Lou, K., Duan, J., Gao, T., Hu, B., & Liu, Q. (2025) · National Science Review · doi:10.1093/nsr/nwaf457 · Cited by 5
“As shown by Song et al. , enforcing this biological fidelity stabilizes dynamics and structures temporal representations of cortical circuits [18 ].”
Tomoda, Y., Tsuda, I., & Yamaguti, Y. (2025) · Cognitive Neurodynamics · doi:10.1007/s11571-025-10377-0 · Cited by 4
“…dynamics similar to biological circuits, have a long history in computational neuroscience (Caianiello 1961; Amari 1972; Hopfield 1982; Elman 1990), and recent developments enable modeling of complex neural dynamics (Sussillo 2014; Song et al. 2016; Yang et al. 2019; Yamaguti and Tsuda 2021).”
Effects of noise and metabolic cost on cortical task representations
Stroud, J. P., Wójcik, M. J., Jensen, K. T., Kusunoki, M., Kadohisa, M., Buckley, M. J., Duncan, J., Stokes, M. G., & Lengyel, M. (2025) · eLife · doi:10.7554/elife.94961.2 · Cited by 7
Recurrent neural networks with transient trajectory explain working memory encoding mechanisms
Liu, C., Jia, S., Liu, H., Zhao, X., Li, C., Xu, B., & Zhang, T. (2025) · Communications Biology · doi:10.1038/s42003-024-07282-3 · Cited by 6
Bolhasani, E., Aboutalebi, S. H., & Merrikhi, Y. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202500147 · Cited by 5
Brain-like border ownership signals support prediction of natural videos
Ye, Z., Weßel, R., & Franken, T. P. (2025) · iScience · doi:10.1016/j.isci.2025.112199 · Cited by 4
“34 Inhibitory neurons may mediate the competition between neurons with opposite preferred sides of ownership in the brain.”
Training Spiking Neural Networks in the Strong Coupling Regime
Kim, C. M., & Chow, C. C. (2021) · Neural Computation · doi:10.1162/neco_a_01379 · Cited by 5
“…2018); backpropagation through time (BPTT) was adapted to rectify the synaptic weights in order to produce zero gradients in the opposite sign regime (Song et al., 2016); in cases where only the read-out and/or feedback weights were trained, the strong recurrent synapses of the initial network…”
BackgroundMethodInfluentialNetwork Mechanism Supporting Long-Distance-Dependencies
Dubreuil, A. M., & Leblois, A. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534151 · Cited by 4
hebbRNN: A Reward-Modulated Hebbian Learning Rule for Recurrent Neural Networks
Michaels, J. A., & Scherberger, H. (2016) · The Journal of Open Source Software · doi:10.21105/joss.00060 · Cited by 4
Network dynamics underlying OFF responses in the auditory cortexPreprint
Bondanelli, G., Deneux, T., Bathellier, B., & Ostojic, S. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/810655 · Cited by 4
Operative dimensions in unconstrained connectivity of recurrent neural networksPreprint
Krause, R., Cook, M., Kollmorgen, S., Mante, V., & Indiveri, G. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.06.03.494670 · Cited by 5
Training stochastic stabilized supralinear networks by dynamics-neutral growthPreprint
Soo, W. W. M., & Lengyel, M. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.10.19.512820 · Cited by 4
“…outright saturating (e.g. tanh) or at best lacked the superlinear characteristics of cortical neurons (as in ReLUs, or the rectified softplus) [9–11, 15, 16], had noiseless dynamics [12], no separation of excitatory (E) and inhibitory (I) cells [9, 10, 15], and sometimes even employed…”
BackgroundBalwani, A., Wang, A. Q., Najafi, F., & Choi, H. (2025) · Science Advances · doi:10.1126/sciadv.adw4970 · Cited by 9
29th Annual Computational Neuroscience Meeting: CNS*2020
Zeldenrust, F., Huang, C., Uijtewaal, P., Englitz, B., & Çelikel, T. (2020) · BMC Neuroscience · doi:10.1186/s12868-020-00593-1 · Cited by 6
Jarne, C., & Caruso, M. (2023) · Cognitive Neurodynamics · doi:10.1007/s11571-023-09956-w · Cited by 4
“…ratios were considered: the balanced case (half of each type, as in the studies from Ingrosso and Abbott (2019); Rajan and Abbott (2006); Ingrosso (2021)), and the case where excitatory neurons outnumber inhibitory neurons (2/3 excitatory vs. 1/3 of the inhibitory, inspired by Song et al. (2016)).”
InfluentialZhang, Y., Xue, L., Luo, X., Li, P., & Qu, H. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105808 · Cited by 3
Inter-trial convergence of neural task states supports cognitive flexibility
Ritz, H., Jha, A., Daw, N. D., & Cohen, J. D. (2026) · Current Biology · doi:10.1016/j.cub.2026.07.028 · Cited by 3
Diversity in the impact of heterogeneities on recurrent networks performing a cognitive task
Santhosh, A., & Narayanan, R. (2026) · Patterns · doi:10.1016/j.patter.2026.101685 · Cited by 4
Bubblewrap: Online tiling and real-time flow prediction on neural manifolds
Draelos, A., Gupta, P., Jun, N. Y., Sriworarat, C., & Pearson, J. (2021) · PubMed · doi:10.48550/arxiv.2108.13941 · Cited by 5
“On the neural side, this has resulted in a host of new modeling and analysis approaches that aim to match the complexity of these data, typically using artificial neural network models as proxies for biological neural computation [6, 7, 8].”
MethodDynamic tuning of neural stability for cognitive control
Xu, M., Hosokawa, T., Tsutsui, K., & Aihara, K. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2409487121 · Cited by 4
“This folding back may reflect the change in the mean population activity and might be accounted for by incorporating biologically plausible mechanisms, such as sparse coding ( 17 , 18 ).”
Huang, X., Wu, W. L., Yin, P., & Qiao, H. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7965944 · Cited by 3
“where the eligibility trace for input weight W out li is inspired by the work of [23]:”
BackgroundMethodInfluentialUnderstanding the functional and structural differences across excitatory and inhibitory neurons
Minni, S., Ji-An, L., Moskovitz, T., Lindsay, G. W., Miller, K. D., Dipoppa, M., & Yang, G. R. (2019) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2019.1265-0 · Cited by 3
Self-citationProbing Human Brain Function with Artificial Neural Networks
Güçlü, U., & van Gerven, M. A. J. (2017) · Wiley eBooks · doi:10.1002/9781119159193.ch30 · Cited by 3
Mininni, C. J., & Zanutto, B. S. (2021) · Scientific Reports · doi:10.1038/s41598-021-82964-0 · Cited by 3
Bridging physiological and perceptual views of autism by means of sampling-based Bayesian inference
Echeveste, R., Ferrante, E., Milone, D. H., & Samengo, I. (2021) · Network Neuroscience · doi:10.1162/netn_a_00219 · Cited by 4
Modeling Cognitive Processes with Neural Reinforcement LearningPreprint
Bosch, S., Seeliger, K., & van Gerven, M. A. J. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/084111 · Cited by 3
Models that learn how humans learn: the case of decision-making and its disordersPreprint
Dezfouli, A., Griffiths, K., Ramos, F., Dayan, P., & Balleine, B. W. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/285221 · Cited by 3
“…et al., 2018), perceptual decision-making, performance in cognitive tasks, working-memory (Miconi, 2017; Carnevale et al., 2015; Mante et al., 2013; Song et al., 2016; Barak et al., 2013; Yang et al., 2017), motor patterns, motor reach and timing (Sussillo et al., 2015; Hennequin et al., 2014;…”
BackgroundHow connectivity structure shapes rich and lazy learning in neural circuits.Preprint
Liu, Y. H., Baratin, A., Cornford, J. H., Mihalaş, Ş., Shea‐Brown, E., & Lajoie, G. (2024) · PubMed · Cited by 5
Kleinman, M., Wang, T., Xiao, D., Feghhi, E., Lee, E., Carr, N., Li, Y., Hadidi, N., Chandrasekaran, C., & Kao, J. C. (2025) · eLife · doi:10.7554/elife.89369.2 · Cited by 4
Nonlinear feedback modulation contributes to the optimization of flexible decision-making
Wu, X., & Zhou, Y. (2024) · eLife · doi:10.7554/elife.96402.3 · Cited by 3
Continuous-Time Piecewise-Linear Recurrent Neural NetworksPreprint
Brändle, A., Eisenmann, L., Götz, F., & Durstewitz, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.15649 · Cited by 3
“Within each subregion, the dynamics is governed by the linear map Reformulating the discrete-time PLRNN as a continuous-time system, we obtain an equation as typically used for neural population dynamics (e.g. Song et al. (2016)): We call this the continuous-time PLRNN (cPLRNN) .”
Aduca, K. R. F., Silva, D. L., & Diona, R. L. (2023) · IEEE International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM) · doi:10.1109/hnicem60674.2023.10589155 · Cited by 2
“Positive weights were described as excitatory weights while negative weights were described as inhibitory weights [26].”
BackgroundNeural dynamics of cognitive control: Current tensions and future promise
Zhou, D., Cosme, D., Kang, Y., Stanoi, O. A., Lydon‐Staley, D. M., Mucha, P. J., Falk, E. B., Ochsner, K. N., & Bassett, D. S. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106909 · Cited by 2
Luo, C., Chu, C., Wu, J., Rasheed, M. T., & Wang, J. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109245 · Cited by 2
Low-Dimensional Manifolds Support Multiplexed Integrations in Recurrent Neural Networks
Fanthomme, A., & Monasson, R. (2021) · Neural Computation · doi:10.1162/neco_a_01366 · Cited by 4
“…time-series, have been studied for several decades in neuroscience, both experimentally (Robinson, 1989; Aksay et al., 2007; Wong and Wang, 2006) and theoretically (Elman, 1990; Seung, 1996; Lee et al., 1997), and more recently, numerically, in the context of machine learning (Song et al., 2016).”
BackgroundSpiking mode-based neural networks
Lin, Z., & Huang, H. (2024) · Physical review. E · doi:10.1103/physreve.110.024306 · Cited by 2
“This biological law could be considered as a matrix product between a non-negative matrix and a diagonal matrix specifying the cell types [29].”
BackgroundExploring Flip Flop memories and beyond: training Recurrent Neural Networks with key insights
Jarne, C. (2024) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2024.1269190 · Cited by 2
Metadata
- Authors
- Song, H. F., Yang, G. R., & Wang, X.-J.
- Affiliations
- New York University (US)
- New York University Shanghai (CN)
- Corresponding author
- Wang, Xiao-Jing
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-02-29
- Volume
- 12
- Issue
- 2
- Pages
- e1004792
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC4771709
- OpenAlex
- W2280725856
- Cited by
- 281
- References
- 91
- Citations by year
- 2026: 23 · 2025: 33 · 2024: 39 · 2023: 42 · 2022: 26 · 2021: 28 · 2020: 23 · 2019: 27 · 2018: 14 · 2017: 21 · 2016: 5
- FWCI
- 14.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Office of Naval Research (N00014–13–1-0297, N00014)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- Dale's principle, neural circuit model, cognitive tasks, perceptual decision making, multisensory integration, working memory, neural population dynamics, Theano, automatic differentiation, gradient-based learning
- MeSH terms
- Algorithms, Animals, Haplorhini, Cognition, Computer Simulation, Models, Neurological, Neurons, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.