The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Optimization and applications of echo state networks with leaky- integrator neurons
Abstract
Standard echo state networks (ESNs) are built from simple additive units with a sigmoid activation function. Here we investigate ESNs whose reservoir units are leaky integrator units. Units of this type have individual state dynamics, which can be exploited in various ways to accommodate the network to the temporal characteristics of a learning task. We present stability conditions, introduce and investigate a stochastic gradient descent method for the optimization of the global learning parameters (input and output feedback scalings, leaking rate, spectral radius) and demonstrate the usefulness of leaky-integrator ESNs for (i) learning very slow dynamic systems and replaying the learnt system at different speeds, (ii) classifying relatively slow and noisy time series (the Japanese Vowel dataset--here we obtain a zero test error rate), and (iii) recognizing strongly time-warped dynamic patterns.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
In the citing paper
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
In the citing paper
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,394
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811
Self-citationMinimum prediction residual principle applied to speech recognition
Itakura, F. (1975) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1975.1162641 · Cited by 1,620
The neural basis of temporal processing
Mauk, M. D., & Buonomano, D. V. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144247 · Cited by 976
Adaptive Filters Theory and Applications
Farhang‐Boroujeny, B. (1999) · doi:10.1002/9781118591352 · Cited by 965
Gradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 578
Readings in Speech Recognition
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-27652-5 · Cited by 574
The combining classifier: to train or not to train?
Duin, R. P. W. (2003) · doi:10.1109/icpr.2002.1048415 · Cited by 348
Analysis and Design of Echo State Networks
Ozturk, M. C., Xu, D., & Príncipe, J. C. (2006) · Neural Computation · doi:10.1162/neco.2007.19.1.111 · Cited by 316
Pattern Extraction for Time Series Classification
Geurts, P. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44794-6_10 · Cited by 296
Training Recurrent Networks by Evolino
Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. J. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.757 · Cited by 253
Computational Aspects of Feedback in Neural Circuits
Maass, W., Joshi, P., & Sontag, E. D. (2005) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020165 · Cited by 252
Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus
Stanley, G. B., Li, F. F., & Dan, Y. (1999) · Journal of Neuroscience · doi:10.1523/jneurosci.19-18-08036.1999 · Cited by 244
Short term memory in echo state networks
Jaeger, H. (2022) · Fraunhofer-Publica (Fraunhofer-Gesellschaft) · doi:10.24406/publica-fhg-291107 · Cited by 197
Self-citationA tighter bound for the echo state property
Buehner, M. R., & Young, P. M. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.872357 · Cited by 154
Multidimensional curve classification using passing-through regions
Kudo, M., Toyama, J., & Shimbo, M. (1999) · Pattern Recognition Letters · doi:10.1016/s0167-8655(99)00077-x · Cited by 139
Jaeger, H. (2002) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 99
Self-citationThe Case for an Internal Dynamics Model versus Equilibrium Point Control in Human Movement
Hinder, M. R., & Milner, T. E. (2003) · The Journal of Physiology · doi:10.1113/jphysiol.2002.033845 · Cited by 96
A Learning Rule for the Emergence of Stable Dynamics and Timing in Recurrent Networks
Buonomano, D. V. (2005) · Journal of Neurophysiology · doi:10.1152/jn.01250.2004 · Cited by 83
Analyzing the weight dynamics of recurrent learning algorithms
Schiller, U., & Steil, J. J. (2004) · Neurocomputing · doi:10.1016/j.neucom.2004.04.006 · Cited by 53
Trajectory generation and modulation using dynamic neural networks
Zegers, P., & Sundareshan, M. K. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.810603 · Cited by 40
Time Warping Invariant Echo State Networks
Lukoševičius, M., Popovici, D., Jaeger, H., & Siewert, U. (2006) · Cited by 38
Self-citationTime Warping Invariant Neural Networks
Sun, G.-Z., Chen, H.-H., & Lee, Y.-C. (1992) · neural information processing systems · Cited by 17
Finding good Echo State Networks to control an underwater robot using evolutionary computations
van der Zant, T., Bečanović, V., Ishii, K., Kobialka, H., & Plöger, P. G. (2004) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)31978-x · Cited by 14
Dynamic Bayesian Networks with Deterministic Latent Tables
Barber, D. (2002) · neural information processing systems · Cited by 7
Self-Organizing Neural Networks for Sequence Processing
Marc Strickert, A. (2005) · Cited by 6
Time-Warping Network: A Hybrid Framework for Speech Recognition
Levin, E., Pieraccini, R., & Bocchieri, E. (1991) · neural information processing systems · Cited by 5
Finding good echo state networks to control an underwater robot using evolutionary computations
Zant, T. V. D., Becanovic, V., Ishii, K., et al. (2004) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 1
Research works printed but not linked: 3
- Jaeger, 2003, Adaptive nonlinear system identification with echo state networks, Vol. 15, 593
- Levin, 1992, Vol. 4, 151
- Sun, 1993, Vol. 5, 180
Cited by
Sorted by citations
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Self-citationRecurrent Neural Network RegularizationPreprint
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.2329 · Cited by 3,174
Information processing using a single dynamical node as complex system
Appeltant, L., Soriano, M. C., der Sande, G. V., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., & Fischer, I. (2011) · Nature Communications · doi:10.1038/ncomms1476 · Cited by 1,785
Ghosh‐Dastidar, S., & Adeli, H. (2009) · International Journal of Neural Systems · doi:10.1142/s0129065709002002 · Cited by 1,066
State-dependent computations: spatiotemporal processing in cortical networks
Buonomano, D. V., & Maass, W. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2558 · Cited by 950
Metadata
- Authors
- Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U.
- Affiliations
- Constructor University (DE)
- Planet (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2007-04
- Volume
- 20
- Issue
- 3
- Pages
- 335-352
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2001263627
- Semantic Scholar
- 370984d3910e61d41ee8318714cd610b689de726
- Cited by
- 948
- References
- 32
- Citations by year
- 2026: 61 · 2025: 87 · 2024: 90 · 2023: 65 · 2022: 71 · 2021: 68 · 2020: 68 · 2019: 81 · 2018: 51 · 2017: 38 · 2016: 44 · 2015: 33 · 2014: 37 · 2013: 28 · 2012: 42
- FWCI
- 24.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- echo state networks, reservoir computing, stability conditions, stochastic gradient descent, spectral radius, time series classification, time warping
- MeSH terms
- Animals, Feedback, Humans, Learning, Models, Neurological, Nerve Net, Neurons, Stochastic Processes, Neural Networks, Computer, Nonlinear Dynamics, Recognition, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
