The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Minimum Complexity Echo State Network
Abstract
Reservoir computing (RC) refers to a new class of state-space models with a fixed state transition structure (the reservoir) and an adaptable readout form the state space. The reservoir is supposed to be sufficiently complex so as to capture a large number of features of the input stream that can be exploited by the reservoir-to-output readout mapping. The field of RC has been growing rapidly with many successful applications. However, RC has been criticized for not being principled enough. Reservoir construction is largely driven by a series of randomized model-building stages, with both researchers and practitioners having to rely on a series of trials and errors. To initialize a systematic study of the field, we concentrate on one of the most popular classes of RC methods, namely echo state network, and ask: What is the minimal complexity of reservoir construction for obtaining competitive models and what is the memory capacity (MC) of such simplified reservoirs? On a number of widely used time series benchmarks of different origin and characteristics, as well as by conducting a theoretical analysis we show that a simple deterministically constructed cycle reservoir is comparable to the standard echo state network methodology. The (short-term) MC of linear cyclic reservoirs can be made arbitrarily close to the proved optimal value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“However, there is a well known problem associated with parameter fitting in the state-transition function [31]: briefly, in order to ‘la tch’ an important piece of past information for the future use, the state-transition dynamics should have an attractive se t.”
BackgroundNovel approach to nonlinear/non-Gaussian Bayesian state estimation
Gordon, N. J., Salmond, D. J., & Smith, A. F. M. (1993) · IEE Proceedings F Radar and Signal Processing · doi:10.1049/ip-f-2.1993.0015 · Cited by 7,550
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 1×
“Liquid state machines (LSMs) [33] also mostly have a linear readout and the reservoirs are driven by the dynamics of a set of coupled spiking neuron models.”
BackgroundHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811
A two-dimensional mapping with a strange attractor
Hénon, M. (1976) · Communications in Mathematical Physics · doi:10.1007/bf01608556 · Cited by 2,987
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
An experimental unification of reservoir computing methods
Verstraeten, D., Schrauwen, B., D’Haene, M., & Stroobandt, D. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.003 · Cited by 1,135
Optimization and applications of echo state networks with leaky- integrator neurons
Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.016 · Cited by 948
An overview of reservoir computing: theory, applications and implementations
Schrauwen, B., Verstraeten, D., & Campenhout, J. V. (2007) · Ghent University Academic Bibliography (Ghent University) · Cited by 553
Cited by
Information processing using a single dynamical node as complex system
Appeltant, L., Soriano, M. C., Van der Sande, G., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., & Fischer, I. (2011) · Nature Communications · doi:10.1038/ncomms1476 · Cited by 1,716
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2926625 · Cited by 1,117
Parallel photonic information processing at gigabyte per second data rates using transient states
Brunner, D., Soriano, M. C., Mirasso, C. R., & Fischer, I. (2013) · Nature Communications · doi:10.1038/ncomms2368 · Cited by 938
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930
ECG-based heartbeat classification for arrhythmia detection: A survey
Luz, E. J. D. S., Schwartz, W. R., Cámara-Chávez, G., & Menotti, D. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.12.008 · Cited by 878
A Practical Guide to Applying Echo State Networks
Lukoševičius, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_36 · Cited by 867
Larger, L., Soriano, M. C., Brunner, D., Appeltant, L., Gutierrez, J. M., Pesquera, L., Mirasso, C. R., & Fischer, I. (2012) · Optics Express · doi:10.1364/oe.20.003241 · Cited by 864
Optoelectronic Reservoir Computing
Paquot, Y., Duport, F., Smerieri, A., Dambre, J., Schrauwen, B., Haelterman, M., & Massar, S. (2012) · Scientific Reports · doi:10.1038/srep00287 · Cited by 783
Dynamic memristor-based reservoir computing for high-efficiency temporal signal processing
(2021) · doi:10.1038/s41467-020-20692-1 · Cited by 597
Advances in photonic reservoir computing
Van der Sande, G., Brunner, D., & Soriano, M. C. (2017) · Nanophotonics · doi:10.1515/nanoph-2016-0132 · Cited by 591
An Experimental Review on Deep Learning Architectures for Time Series Forecasting
Lara-Benítez, P., Carranza-García, M., & Riquelme, J. C. (2021) · International Journal of Neural Systems · doi:10.1142/s0129065721300011 · Cited by 532
Ahmad, T., Zhang, H., & Yan, B. (2020) · Sustainable Cities and Society · doi:10.1016/j.scs.2020.102052 · Cited by 504
All-optical reservoir computing
Duport, F., Schneider, B., Smerieri, A., Haelterman, M., & Massar, S. (2012) · Optics Express · doi:10.1364/oe.20.022783 · Cited by 502
Physical reservoir computing—an introductory perspective
Nakajima, K. (2020) · Japanese Journal of Applied Physics · doi:10.35848/1347-4065/ab8d4f · Cited by 422
Randomness in neural networks: an overview
Scardapane, S., & Wang, D. (2017) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1200 · Cited by 352
Emerging opportunities and challenges for the future of reservoir computing
Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., & Sun, J. (2024) · Nature Communications · doi:10.1038/s41467-024-45187-1 · Cited by 346
Self-citationRecurrent Neural Networks for Short-Term Load Forecasting: An Overview and Comparative Analysis
Bianchi, F. M., Maiorino, E., Kampffmeyer, M. C., Rizzi, A., & Jenssen, R. (2017) · SpringerBriefs in Computer Science · doi:10.1007/978-3-319-70338-1 · Cited by 280
A Fast Machine Learning Model for ECG-Based Heartbeat Classification and Arrhythmia Detection
Alfaras, M., Soriano, M. C., & Ortín, S. (2019) · Frontiers in Physics · doi:10.3389/fphy.2019.00103 · Cited by 265
Design of deep echo state networks
Gallicchio, C., Micheli, A., & Pedrelli, L. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.08.002 · Cited by 262
Echo State Networks for Proactive Caching in Cloud-Based Radio Access Networks With Mobile Users
Chen, M., Saad, W., Yin, C., & Debbah, M. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2683482 · Cited by 176
“Different from the existing works in [33] and [34] that use an independent and identically distributed input stream to derive the ESN memory capacity, we formulate the ESN memory capacity with a periodic input stream.”
MethodNonlinearity in memristors for neuromorphic dynamic systems
Yang, K., Joshua Yang, J., Huang, R., & Yang, Y. (2022) · Small Science · doi:10.1002/smsc.202100049 · Cited by 151
The unfolding argument: Why IIT and other causal structure theories cannot explain consciousness
Doerig, A., Schurger, A., Hess, K., & Herzog, M. H. (2019) · Consciousness and Cognition · doi:10.1016/j.concog.2019.04.002 · Cited by 140
Using echo state networks for classification: A case study in Parkinson's disease diagnosis
Lacy, S. E., Smith, S. L., & Lones, M. A. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.02.002 · Cited by 55
Applications of Metaheuristics in Reservoir Computing Techniques: A Review
Bala, A., Ismail, I., Ibrahim, R., & Sait, S. M. (2018) · IEEE Access · doi:10.1109/access.2018.2873770 · Cited by 53
“The HNN consists of a simple cycle reservoir (SCR) ESN [68] at the first stage and the ELM at the second stage.”
Metadata
- Authors
- Rodan, A., & Tino, P.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 2011-01
- Volume
- 22
- Issue
- 1
- Pages
- 131-144
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2159682675
- Cited by
- 710
- References
- 36
- FWCI
- 22.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.