The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Minimum Complexity Echo State Network
Abstract
Reservoir computing (RC) refers to a new class of state-space models with a fixed state transition structure (the reservoir) and an adaptable readout form the state space. The reservoir is supposed to be sufficiently complex so as to capture a large number of features of the input stream that can be exploited by the reservoir-to-output readout mapping. The field of RC has been growing rapidly with many successful applications. However, RC has been criticized for not being principled enough. Reservoir construction is largely driven by a series of randomized model-building stages, with both researchers and practitioners having to rely on a series of trials and errors. To initialize a systematic study of the field, we concentrate on one of the most popular classes of RC methods, namely echo state network, and ask: What is the minimal complexity of reservoir construction for obtaining competitive models and what is the memory capacity (MC) of such simplified reservoirs? On a number of widely used time series benchmarks of different origin and characteristics, as well as by conducting a theoretical analysis we show that a simple deterministically constructed cycle reservoir is comparable to the standard echo state network methodology. The (short-term) MC of linear cyclic reservoirs can be made arbitrarily close to the proved optimal value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“However, there is a well known problem associated with parameter fitting in the state-transition function [31]: briefly, in order to ‘la tch’ an important piece of past information for the future use, the state-transition dynamics should have an attractive se t.”
BackgroundNovel approach to nonlinear/non-Gaussian Bayesian state estimation
Gordon, N. J., Salmond, D. J., & Smith, A. F. M. (1993) · IEE Proceedings F Radar and Signal Processing · doi:10.1049/ip-f-2.1993.0015 · Cited by 7,550
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 1×
“Liquid state machines (LSMs) [33] also mostly have a linear readout and the reservoirs are driven by the dynamics of a set of coupled spiking neuron models.”
BackgroundHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811
A two-dimensional mapping with a strange attractor
Hénon, M. (1976) · Communications in Mathematical Physics · doi:10.1007/bf01608556 · Cited by 2,987
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
An experimental unification of reservoir computing methods
Verstraeten, D., Schrauwen, B., D’Haene, M., & Stroobandt, D. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.003 · Cited by 1,135
Optimization and applications of echo state networks with leaky- integrator neurons
Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.016 · Cited by 948
An overview of reservoir computing: theory, applications and implementations
Schrauwen, B., Verstraeten, D., & Campenhout, J. V. (2007) · Ghent University Academic Bibliography (Ghent University) · Cited by 553
New results on recurrent network training: unifying the algorithms and accelerating convergence
Atiya, A. F., & Parlos, A. G. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.846741 · Cited by 521
A computational model of filtering, detection, and compression in the cochlea
Lyon, R. (2005) · ICASSP '82. IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1982.1171644 · Cited by 411
Analysis and Design of Echo State Networks
Ozturk, M. C., Xu, D., & Príncipe, J. C. (2007) · Neural Computation · doi:10.1162/neco.2007.19.1.111 · Cited by 308
Memory traces in dynamical systems
Ganguli, S., Huh, D., & Sompolinsky, H. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0804451105 · Cited by 303
Training Recurrent Networks by Evolino
Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.757 · Cited by 248
Improving reservoirs using intrinsic plasticity
Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J. J., & Stroobandt, D. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.12.020 · Cited by 227
Decoupled echo state networks with lateral inhibition
Xue, Y., Yang, L., & Haykin, S. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.014 · Cited by 186
The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition
Schrauwen, B., Defour, J., Verstraeten, D., & Van Campenhout, J. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74690-4_48 · Cited by 181
Special issue on echo state networks and liquid state machines
Jaeger, H., Maass, W., & Principe, J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.001 · Cited by 167
Steil, J. J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.011 · Cited by 165
Learning grammatical structure with Echo State Networks
Tong, M. H., Bickett, A. D., Christiansen, E. M., & Cottrell, G. W. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.013 · Cited by 162
Collective Behavior of a Small-World Recurrent Neural System With Scale-Free Distribution
Deng, Z., & Zhang, Y. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.894082 · Cited by 129
Fading memory and kernel properties of generic cortical microcircuit models
Maass, W., Natschläger, T., & Markram, H. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.020 · Cited by 110
Echo state networks with filter neurons and a delay&sum readout
Holzmann, G., & Häuser, H. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.004 · Cited by 98
Liquid state machines and cultured cortical networks: The separation property
Dockendorf, K. P., Park, I. M., He, P., Príncipe, J. C., & DeMarse, T. B. (2008) · Biosystems · doi:10.1016/j.biosystems.2008.08.001 · Cited by 82
Echo state networks: appeal and challenges
Prokhorov, D. V. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556091 · Cited by 79
Stable Output Feedback in Reservoir Computing Using Ridge Regression
wyffels, F., Schrauwen, B., & Stroobandt, D. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87536-9_83 · Cited by 77
Identification of motion with echo state network
Ishu, K., van der Zant, T., Bečanović, V., & Plöger, P. G. (2004) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans.2004.1405751 · Cited by 68
Is there a Liquid State Machine in the Bacterium Escherichia Coli?
Jones, B., Stekel, D. J., Rowe, J., & Fernando, C. (2007) · IEEE Symposium on Artificial Life (ALIFE) · doi:10.1109/alife.2007.367795 · Cited by 67
Minimum mean squared error time series classification using an echo state network prediction model
Skowronski, M. D., & Harris, J. G. (2006) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2006.1693294 · Cited by 64
Modeling reward functions for incomplete state representations via echo state networks
Bush, K., & Anderson, C. W. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556402 · Cited by 45
Slutzky, M. W., Cvitanović, P., & Mogul, D. J. (2003) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2003.810701 · Cited by 41
Lukoševičius, M., & Jaeger, H. (2007) · Cited by 33
Häusler, S., Markram, H., & Maass, W. (2003) · Complexity · doi:10.1002/cplx.10089 · Cited by 27
Predictive Modeling with Echo State Networks
Čerňanský, M., & Peter Tiňo, B. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87536-9_80 · Cited by 25
Reservoir optimization in recurrent neural networks using kronecker kernels
Rad, A. A., Jalili, M., & Hasler, M. (2008) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2008.4541556 · Cited by 11
Cited by
Information processing using a single dynamical node as complex system
Appeltant, L., Soriano, M. C., Van der Sande, G., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., & Fischer, I. (2011) · Nature Communications · doi:10.1038/ncomms1476 · Cited by 1,716
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2926625 · Cited by 1,117
Parallel photonic information processing at gigabyte per second data rates using transient states
Brunner, D., Soriano, M. C., Mirasso, C. R., & Fischer, I. (2013) · Nature Communications · doi:10.1038/ncomms2368 · Cited by 938
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930
ECG-based heartbeat classification for arrhythmia detection: A survey
Luz, E. J. D. S., Schwartz, W. R., Cámara-Chávez, G., & Menotti, D. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.12.008 · Cited by 878
Metadata
- Authors
- Rodan, A., & Tino, P.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 2011-01
- Volume
- 22
- Issue
- 1
- Pages
- 131-144
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2159682675
- Cited by
- 710
- References
- 36
- FWCI
- 22.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.