The DOI matches its DataCite record.
Warming up recurrent neural networks to maximise reachable multistability greatly improves learning
Open accessAbstract
Training recurrent neural networks is known to be difficult when time dependencies become long. In this work, we show that most standard cells only have one stable equilibrium at initialisation, and that learning on tasks with long time dependencies generally occurs once the number of network stable equilibria increases; a property known as multistability. Multistability is often not easily attained by initially monostable networks, making learning of long time dependencies between inputs and outputs difficult. This insight leads to the design of a novel way to initialise any recurrent cell connectivity through a procedure called "warmup" to improve its capability to learn arbitrarily long time dependencies. This initialisation procedure is designed to maximise network reachable multistability, i.e., the number of equilibria within the network that can be reached through relevant input trajectories, in few gradient steps. We show on several information restitution, sequence classification, and reinforcement learning benchmarks that warming up greatly improves learning speed and performance, for multiple recurrent cells, but sometimes impedes precision. We therefore introduce a double-layer architecture initialised with a partial warmup that is shown to greatly improve learning of long time dependencies while maintaining high levels of precision. This approach provides a general framework for improving learning abilities of any recurrent cell when long time dependencies are present. We also show empirically that other initialisation and pretraining procedures from the literature implicitly foster reachable multistability of recurrent cells.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Conversely, other works (Bakker, 2001; Lambrechts et al., 2021; Esslinger et al., 2022; Hung et al., 2018; Raposo et al., 2021; Ren et al., 2022) actually entangle memory and credit assignment together, despite their intention to evaluate one of the two.”
From When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment (2023) · cites this paper as Lambrechts et al., 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 35
- Bakker, B. (2001). Reinforcement learning with long short-term memory. Advances in Neural Information Processing Systems, 14.
- Bengio, Y., Frasconi, P., & Simard, P. (1993). The problem of learning long- term dependencies in recurrent networks. In IEEE International Conference on Neural Networks (pp. 1183-1188). IEEE.
- Ceni, A., Ashwin, P., & Livi, L. (2020). Interpreting recurrent neural networks behaviour via excitable network attractors. Cognitive Computation, 12(2), 330-356.
- Chen, J., & Chaudhari, N. (2009). Segmented-memory recurrent neural networks. IEEE Transactions on Neural Networks, 20(8), 1267-1280. http://dx.doi.org/10. 1109/TNN.2009.2022980, URL http://ieeexplore.ieee.org/document/5164893/. · DOI
- Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259. · arXiv
- Chung, J., Ahn, S., & Bengio, Y. (2017). Hierarchical multiscale recurrent neu- ral networks. http://dx.doi.org/10.48550/arXiv.1609.01704, URL arXiv:1609. 01704[cs. · DOI · Paper page
- Doya, K. (1993). Bifurcations of recurrent neural networks in gradient descent learning. IEEE Transactions on Neural Networks, 1(75), 218.
- Hausknecht, M., & Stone, P. (2015). Deep recurrent Q-learning for partially observable MDPs. In 2015 AAAI Fall Symposium Series.
- Hihi, S., & Bengio, Y. (1995). Hierarchical recurrent neural networks for long-term dependencies. 8, In Advances in Neural Information Process- ing Systems. MIT Press, URL https://proceedings.neurips.cc/paper/1995/hash/ c667d53acd899a97a85de0c201ba99be-Abstract.html.
- Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.
- Ienco, D., Interdonato, R., & Gaetano, R. (2019). Supervised level-wise pretraining for recurrent neural network initialization in multi-class classification. URL arXiv:1911.01071[cs, stat]. · arXiv
- Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1-2), 99-134.
- Katz, G. E., & Reggia, J. A. (2017). Using directional fibers to locate fixed points of recurrent neural networks. IEEE Transactions on Neural Networks and Learning Systems, 29(8), 3636-3646.
- Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. · arXiv
- Koutnik, J., Greff, K., Gomez, F., & Schmidhuber, J. (2014). A clockwork RNN. In Proceedings of the 31st International Conference on Machine Learning (pp. 1863-1871). PMLR, URL https://proceedings.mlr.press/v32/koutnik14.html, ISSN: 1938-7228.
- Lin, T., Horne, B. G., Tino, P., & Giles, C. L. (1996). Learning long-term de- pendencies in NARX recurrent neural networks. IEEE Transactions on Neural Networks, 7(6), 1329-1338. http://dx.doi.org/10.1109/72.548162. · DOI
- Maheswaranathan, N., Williams, A., Golub, M., Ganguli, S., & Sussillo, D. (2019). Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics. Advances In Neural Information Processing Systems, 32.
- Menezes, J. M. P., & Barreto, G. A. (2008). Long-term time series pre- diction with the NARX network: An empirical evaluation. Neurocomput- ing, 71(16), 3335-3343. http://dx.doi.org/10.1016/j.neucom.2008.01.030, URL https://www.sciencedirect.com/science/article/pii/S0925231208003081. · DOI
- Mikolov, T., Joulin, A., Chopra, S., Mathieu, M., & Ranzato, M. (2015). Learning longer memory in recurrent neural networks. URL arXiv:1412.7753[cs]. · arXiv
- Ong, B. T., Sugiura, K., & Zettsu, K. (2014). Dynamic pre-training of deep recurrent neural networks for predicting environmental monitoring data. In 2014 IEEE International Conference on Big Data (Big Data) (pp. 760-765).
- Washington, DC, USA: IEEE, http://dx.doi.org/10.1109/BigData.2014.7004302, URL http://ieeexplore.ieee.org/document/7004302/. · DOI
- Pasa, L., & Sperduti, A. (2014). Pre-training of recurrent neural networks via linear autoencoders. 27, In Advances in Neural Information Processing Systems.
- Curran Associates, Inc., URL https://proceedings.neurips.cc/paper/2014/hash/ f0fcf351df4eb6786e9bb6fc4e2dee02-Abstract.html.
- Pasa, L., Testolin, A., & Sperduti, A. (2015). Neural networks for sequential data: a pre-training approach based on hidden Markov models. Neurocomputing, 169, 323-333. http://dx.doi.org/10.1016/j.neucom.2014.11.081, URL https:// linkinghub.elsevier.com/retrieve/pii/S0925231215003689. · DOI
- Pascanu, R., Mikolov, T., & Bengio, Y. (2013). On the difficulty of training recurrent neural networks. In International Conference on Machine Learning (pp. 1310-1318). PMLR.
- Sagheer, A., & Kotb, M. (2019). Unsupervised Pre-training of a deep LSTM- based stacked autoencoder for multivariate time series forecasting problems. Scientific Reports, 9(1), 19038. http://dx.doi.org/10.1038/s41598-019-55320- 6, URL https://www.nature.com/articles/s41598-019-55320-6, Number: 1 Publisher: Nature Publishing Group. · DOI · Paper page
- Smallwood, R. D., & Sondik, E. J. (1973). The optimal control of partially observable Markov processes over a finite horizon. Operations Research, 21(5), 1071-1088.
- Sussillo, D., & Barak, O. (2013). Opening the black box: low-dimensional dy- namics in high-dimensional recurrent neural networks. Neural Computation, 25(3), 626-649.
- Tallec, C., & Ollivier, Y. (2018). Can recurrent neural networks warp time? In International Conference on Learning Representations.
- Tang, Z., Wang, D., & Zhang, Z. (2016). Recurrent neural network training with dark knowledge transfer. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 5900-5904). Shanghai: IEEE, http://dx.doi. org/10.1109/ICASSP.2016.7472809, URL http://ieeexplore.ieee.org/document/ 7472809/. · DOI
- Trinh, T. H., Dai, A. M., Luong, M.-T., & Le, Q. V. (2018). Learning longer-term dependencies in RNNs with auxiliary losses. URL arXiv:1803.00144[cs, stat]. · arXiv
- Van Der Westhuizen, J., & Lasenby, J. (2018). The unreasonable effectiveness of the forget gate. arXiv preprint arXiv:1804.04849. · arXiv
- Werbos, P. J. (1990). Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10), 1550-1560.
- Williams, R. J., & Zipser, D. (1995). Gradient-based learning algorithms for recurrent networks and their computational complexity. In Backpropaga- tion: theory, architectures, and applications (pp. 433-486). USA: L. Erlbaum Associates Inc..
- Zhou, G.-B., Wu, J., Zhang, C.-L., & Zhou, Z.-H. (2016). Minimal gated unit for recurrent neural networks. International Journal of Automation and Computing, 13(3), 226-234.
Cited by
Sorted by citations
When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment
Ni, T., Ma, M., Eysenbach, B., & Bacon, P.-L. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2194 · Cited by 7
Metadata
- Authors
- Lambrechts, G., De Geeter, F., Vecoven, N., Ernst, D., & Drion, G.
- Affiliations
- University of Liège (BE)
- Télécom Paris (FR)
- Corresponding author
- Lambrechts, Gaspard
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Volume
- 166
- Pages
- 645-669
- ISSN
- 0893-6080 (ISSN-L)
- arXiv
- 2106.01001
- OpenAlex
- W4385623827
- Cited by
- 9
- References
- 0
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 1 · 2023: 1 · 2022: 1
- FWCI
- 0.57
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- ORBi (University of Liège) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Fonds De La Recherche Scientifique - FNRS (1117545)
- Fédération Wallonie-Bruxelles (1117545)
- Fonds pour la Formation à la Recherche dans l’Industrie et dans l’Agriculture
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- recurrent neural networks, multistability, stable equilibrium, sequence learning, reinforcement learning, learning speed
- MeSH terms
- Intelligence, Learning, Reinforcement, Psychology, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Warming up recurrent neural networks to maximise reachable multistability greatly improves learning. https://app.cochat.ai/papers/10.48550/arxiv.2106.01001
- The paper
- Lambrechts, G., De Geeter, F., Vecoven, N., Ernst, D., & Drion, G. (2021). Warming up recurrent neural networks to maximise reachable multistability greatly improves learning. arXiv. https://doi.org/10.48550/arxiv.2106.01001
