The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Optimization and applications of echo state networks with leaky- integrator neurons
Abstract
Standard echo state networks (ESNs) are built from simple additive units with a sigmoid activation function. Here we investigate ESNs whose reservoir units are leaky integrator units. Units of this type have individual state dynamics, which can be exploited in various ways to accommodate the network to the temporal characteristics of a learning task. We present stability conditions, introduce and investigate a stochastic gradient descent method for the optimization of the global learning parameters (input and output feedback scalings, leaking rate, spectral radius) and demonstrate the usefulness of leaky-integrator ESNs for (i) learning very slow dynamic systems and replaying the learnt system at different speeds, (ii) classifying relatively slow and noisy time series (the Japanese Vowel dataset--here we obtain a zero test error rate), and (iii) recognizing strongly time-warped dynamic patterns.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,394
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811
Self-citationMinimum prediction residual principle applied to speech recognition
Itakura, F. (1975) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1975.1162641 · Cited by 1,620
The neural basis of temporal processing
Mauk, M. D., & Buonomano, D. V. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144247 · Cited by 976
Adaptive Filters Theory and Applications
Farhang‐Boroujeny, B. (1999) · doi:10.1002/9781118591352 · Cited by 965
Gradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 578
Readings in Speech Recognition
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-27652-5 · Cited by 574
Research works printed but not linked: 3
- Jaeger, 2003, Adaptive nonlinear system identification with echo state networks, Vol. 15, 593
- Levin, 1992, Vol. 4, 151
- Sun, 1993, Vol. 5, 180
Cited by
Sorted by citations
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Self-citationRecurrent Neural Network RegularizationPreprint
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.2329 · Cited by 3,174
Information processing using a single dynamical node as complex system
Appeltant, L., Soriano, M. C., der Sande, G. V., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., & Fischer, I. (2011) · Nature Communications · doi:10.1038/ncomms1476 · Cited by 1,785
Ghosh‐Dastidar, S., & Adeli, H. (2009) · International Journal of Neural Systems · doi:10.1142/s0129065709002002 · Cited by 1,066
State-dependent computations: spatiotemporal processing in cortical networks
Buonomano, D. V., & Maass, W. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2558 · Cited by 950
Metadata
- Authors
- Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U.
- Affiliations
- Constructor University (DE)
- Planet (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2007-04
- Volume
- 20
- Issue
- 3
- Pages
- 335-352
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2001263627
- Semantic Scholar
- 370984d3910e61d41ee8318714cd610b689de726
- Cited by
- 948
- References
- 32
- Citations by year
- 2026: 61 · 2025: 87 · 2024: 90 · 2023: 65 · 2022: 71 · 2021: 68 · 2020: 68 · 2019: 81 · 2018: 51 · 2017: 38 · 2016: 44 · 2015: 33 · 2014: 37 · 2013: 28 · 2012: 42
- FWCI
- 24.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- echo state networks, reservoir computing, stability conditions, stochastic gradient descent, spectral radius, time series classification, time warping
- MeSH terms
- Animals, Feedback, Humans, Learning, Models, Neurological, Nerve Net, Neurons, Stochastic Processes, Neural Networks, Computer, Nonlinear Dynamics, Recognition, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
