Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 301–400 of 1,009
Optimal Sequence Memory in Driven Random Networks
Schuecker, J., Goedeke, S., & Helias, M. (2018) · Physical Review X · doi:10.1103/physrevx.8.041029 · Cited by 125
“Their information processing capabilities have been a focus in neuroscience [5] and in machine learning [6], and they show optimal performance close to the transition to chaos [7–9].”
BackgroundNeural Networks and Deep Learning
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.007 · Cited by 111
MFRFNN: Multi-Functional Recurrent Fuzzy Neural Network for Chaotic Time Series Prediction
Nasiri, H., & Ebadzadeh, M. M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.08.032 · Cited by 106
Deep photonic reservoir computing recurrent network
Shen, Y.-W., Li, R., Liu, G., Yu, J., He, X., Yi, L., & Wang, C. (2023) · Optica · doi:10.1364/optica.506635 · Cited by 126
“Reservoir computing (RC) is such a kind of RNN, but with fixed weights in the input layer and in the hidden reservoir layers [10,11].”
BackgroundRandomly distributed embedding making short-term high-dimensional data predictable
Ma, H., Leng, S., Aihara, K., Lin, W. J., & Chen, L. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1802987115 · Cited by 123
Mesaritakis, C., Papataxiarhis, V., & Syvridis, D. (2013) · Journal of the Optical Society of America B · doi:10.1364/josab.30.003048 · Cited by 103
Functional echo state network for time series classification
Ma, Q., Shen, L., Chen, W., Wang, J., Wei, J., & Yu, Z. (2016) · Information Sciences · doi:10.1016/j.ins.2016.08.081 · Cited by 112
Period-aware content attention RNNs for time series forecasting with missing values
Cinar, Y. G., Mirisaee, H., Goswami, P., Gaussier, É., & Aït-Bachir, A. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.090 · Cited by 112
Watt, S., & Kostylev, M. (2020) · Physical Review Applied · doi:10.1103/physrevapplied.13.034057 · Cited by 112
“The reservoir computer (RC) is a computational framework based on the recursive neural network [1, 2] and as such is suitable for modelling complex dynamical systems.”
Embodied Intelligence: A Synergy of Morphology, Action, Perception and Learning
Liu, H., Guo, D., & Cangelosi, A. (2025) · ACM Computing Surveys · doi:10.1145/3717059 · Cited by 200
“Reservoir computing is the most commonly used method in morphological computation [81], which belongs to the dynamic system analysis method.”
Intelligent stock trading system based on improved technical analysis and Echo State Network
Lin, X., Yang, Z., & Song, Y. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.03.001 · Cited by 102
A Two-Stage Online Prediction Method for a Blast Furnace Gas System and Its Application
Zhao, J., Wang, W., Liu, Y., & Pedrycz, W. (2010) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2010.2051545 · Cited by 102
“Sound prediction capabilities of ESN when applied to some chaotic time series have been reported in [10].”
BackgroundEmbedding Multiple Trajectories in Simulated Recurrent Neural Networks in a Self-Organizing Manner
Liu, J. K., & Buonomano, D. V. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2358-09.2009 · Cited by 107
“…computations(Buonomano,2000;MedinaandMauk,2000;Maass etal.,2002),andthatcarefullycontrollingthefeedbackfromoutput unitsintotherecurrentnetworkoffersapromisingwaytocontrol dynamicsintheabsenceofsynapticplasticitywithintherecurrent network(JaegerandHaas,2004;Maassetal.,2007).”
Deterministic Neural Classification
Toh, K. (2008) · Neural Computation · doi:10.1162/neco.2007.04-07-508 · Cited by 109
“…from the gradient-based and batch search methods, advance in computing and memory technologies have also facilitated the realization of other powerful compact network and learning methods (Broomhead & Lowe, 1988; Pao & Takefuji, 1992; Jaeger, 2002; Jaeger & Haas, 2004; Huang, Chen, & Siew, 2006).”
BackgroundMethodInfluentialA Survey on Silicon Photonics for Deep Learning
Sunny, F., Taheri, E., Nikdast, M., & Pasricha, S. (2021) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3459009 · Cited by 114
A perspective on machine learning in turbulent flows
Pandey, S., Schumacher, J., & Sreenivasan, K. R. (2020) · Journal of Turbulence · doi:10.1080/14685248.2020.1757685 · Cited by 101
Memory versus non-linearity in reservoirs
Verstraeten, D., Dambre, J., Dutoit, X., & Schrauwen, B. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596492 · Cited by 125
Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis
Bianchi, F. M., Livi, L., & Alippi, C. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2630802 · Cited by 110
“MG Time Series: The prediction of the MG time series is a common benchmark where ESNs achieve good performance in terms of prediction accuracy [47], [48].”
Physics-informed echo state networks
Doan, N. A. K., Polifke, W., & Magri, L. (2020) · Journal of Computational Science · doi:10.1016/j.jocs.2020.101237 · Cited by 101
“These are typically sparse matrices constructed so that the reservoir verifies the Echo State Property [20].”
BackgroundNokkala, J., Martínez‐Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2021) · Communications Physics · doi:10.1038/s42005-021-00556-w · Cited by 151
Reservoir computing on a silicon platform with a ferroelectric field-effect transistor
Toprasertpong, K., Nako, E., Wang, Z., Nakane, R., Takenaka, M., & Takagi, S. (2022) · Communications Engineering · doi:10.1038/s44172-022-00021-8 · Cited by 110
“(5)) and thus the nonlinear property originates from the nonlinear dynamics in the FeFET.”
BackgroundMethodReservoir Computing on Spin-Torque Oscillator Array
Kanao, T., Suto, H., Mizushima, K., Goto, H., Tanamoto, T., & Nagasawa, T. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.12.024052 · Cited by 108
“The performance can become higher than a standard neural network model known as an echo-state network (ESN) [1, 2].”
BackgroundFast and Deep Graph Neural Networks
Gallicchio, C., & Micheli, A. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i04.5803 · Cited by 104
Energy consumption prediction of office buildings based on echo state networks
Shi, G., Liu, D., & Wei, Q. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.08.004 · Cited by 95
Time series forecasting based on echo state network and empirical wavelet transformation
Gao, R., Du, L., Duru, O., & Yuen, K. F. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107111 · Cited by 109
Dynamic ensemble deep echo state network for significant wave height forecasting
Gao, R., Li, R., Hu, M., Suganthan, P. N., & Yuen, K. F. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120261 · Cited by 95
Reservoir riddles: suggestions for echo state network research (extended abstract)
Jaeger, H. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556090 · Cited by 96
Self-citationEcho state networks with filter neurons and a delay&sum readout
Holzmann, G., & Häuser, H. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.004 · Cited by 119
“For a comparison of the Mackey-Glass time series prediction to other techniques, one has to consider [15], where it was shown that for the τ = 17 task ESNs vastly outperform all other known methods.”
BackgroundMethodInfluentialControlling Recurrent Neural Networks by ConceptorsPreprint
Jaeger, H. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.3369 · Cited by 133
“Note that it used to be a challenging task to make an RNN learn any single of these attractors [56]; to my knowledge, training a single RNN to generate several different chaotic attractors has not been attempted before.”
BackgroundMethodSelf-citationCellular automata as convolutional neural networks
Gilpin, W. (2019) · Physical review. E · doi:10.1103/physreve.100.032402 · Cited by 108
Photonic neuromorphic information processing and reservoir computing
Lugnan, A., Katumba, A., Laporte, F., Freiberger, M., Sackesyn, S., Ma, C., Gooskens, E., Dambre, J., & Bienstman, P. (2020) · APL Photonics · doi:10.1063/1.5129762 · Cited by 104
“51,52 ).”
Physical Implementation of Reservoir Computing through Electrochemical Reaction
Kan, S., Nakajima, K., Asai, T., & Akai‐Kasaya, M. (2021) · Advanced Science · doi:10.1002/advs.202104076 · Cited by 95
Nokkala, J., Martínez-Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2020) · arXiv · doi:10.48550/arxiv.2006.04821 · Cited by 94
Brain-Inspired Wireless Communications: Where Reservoir Computing Meets MIMO-OFDM
Mosleh, S., Liu, L., Sahin, C., Zheng, Y. R., & Yi, Y. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766162 · Cited by 105
“Although they share the same basic concept, these methods have their own unique implementation: ESNs [21], [24], [32], [33], liquid state machines [17], backpropagation–decorrelation learning [34], temporal RNNs [35], and delay line feed-back [36]–[38].”
BackgroundA recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems
Chicca, E., & Indiveri, G. (2020) · Applied Physics Letters · doi:10.1063/1.5142089 · Cited by 103
“Examples of theoretical neural processing frameworks that require variability can be found in the domain of ensemble learning 53 , reservoir computing 54 and liquid state machines 55 .”
BackgroundTime-series quantum reservoir computing with weak and projective measurements
Mujal, P., Martínez‐Peña, R., Giorgi, G. L., Soriano, M. C., & Zambrini, R. (2023) · npj Quantum Information · doi:10.1038/s41534-023-00682-z · Cited by 161
Reservoir computing system with double optoelectronic feedback loops
Chen, Y., Yi, L., Ke, J., Zhao, Y., Yang, Y., Huang, L., Zhuge, Q., & Hu, W. (2019) · Optics Express · doi:10.1364/oe.27.027431 · Cited by 101
An introduction to quantum machine learning: from quantum logic to quantum deep learning
Alchieri, L., Badalotti, D., Bonardi, P., & Bianco, S. (2021) · Quantum Machine Intelligence · doi:10.1007/s42484-021-00056-8 · Cited by 125
“It should be noted that the idea of using random non linear features is not new to machine learning, and has been already implemented for the development of specific Convolutional Neural Networks (see Ranzato et al. 2007; Jaeger and Haas 2004).”
BackgroundComputational Models of Timing Mechanisms in the Cerebellar Granular Layer
Yamazaki, T., & Tanaka, S. (2009) · The Cerebellum · doi:10.1007/s12311-009-0115-7 · Cited by 96
“The random projection model is a member of liquid-state machines [54] or echo-state networks [55], which has been recently proposed to provide a new paradigm of modern neural computation (see [56, 57] for recent work).”
BackgroundDetection of Integrity Attacks in Cyber-Physical Critical Infrastructures Using Ensemble Modeling
Ntalampiras, S. (2015) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2367322 · Cited by 93
A Theory of Sequence Indexing and Working Memory in Recurrent Neural Networks
Frady, E. P., Kleyko, D. V., & Sommer, F. T. (2018) · Neural Computation · doi:10.1162/neco_a_01084 · Cited by 98
“The ability of recurrent neural networks to memorize past inputs in the current activity state is the basis of working memory in artificial neural systems for signal processing (Jaeger and Haas, 2004), cognitive reasoning (Eliasmith et al.”
Dynamic ensemble wind speed prediction model based on hybrid deep reinforcement learning
Chen, C., & Liu, H. (2021) · Advanced Engineering Informatics · doi:10.1016/j.aei.2021.101290 · Cited by 91
Event detection and localization for small mobile robots using reservoir computing
Antonelo, E. A., Schrauwen, B., & Stroobandt, D. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.06.010 · Cited by 105
“…2001b) and a broad range of applications (Verstraeten et al., 2007) (which sometimes even drastically outperform the current state-of-the-art (Jaeger and Haas, 2004)) show that RC is very powerful and overcomes many of the problems of traditional RNN training such as slow convergence,…”
Deep Echo State Network (DeepESN): A Brief SurveyPreprint
Gallicchio, C., & Micheli, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04323 · Cited by 130
“[3, 4]), the Echo State Network (ESN) [5, 6] certainly represents the most widely known model, with a strong theoretical ground (e.”
BackgroundArtificial neural networks for photonic applications—from algorithms to implementation: tutorial
Freire, P. J., Manuylovich, E. S., Prilepsky, J. E., & Turitsyn, S. K. (2023) · Advances in Optics and Photonics · doi:10.1364/aop.484119 · Cited by 111
Fading memory and kernel properties of generic cortical microcircuit models
Maass, W., Natschläger, T., & Markram, H. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2005.09.020 · Cited by 111
“1% in [20]) in order to achieve that the network decomposes into many loosely coupled subcircuits, thereby establishing a richly structured reservoir of excitable dynamics.”
BackgroundMethodPredicting non-linear dynamics by stable local learning in a recurrent spiking neural network
Gilra, A., & Gerstner, W. (2017) · eLife · doi:10.7554/elife.28295 · Cited by 100
“Our approach toward learning of recurrent spiking networks is situated at the crossroads of reservoir computing (Jaeger, 2001; Maass et al., 2002; Legenstein et al., 2003; Maass and Markram, 2004; Jaeger and Haas, 2004; Joshi and Maass, 2005; Legenstein and Maass, 2007), FORCE learning (Sussillo and Abbott, 2009, 2012…”
BackgroundInfluentialDeep time-delay reservoir computing: Dynamics and memory capacity
Goldmann, M., Köster, F., Lüdge, K., & Yanchuk, S. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0017974 · Cited by 109
Complexity in electro-optic delay dynamics: modelling, design and applications
Larger, L. (2013) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2012.0464 · Cited by 101
“Moreover, performances at the level of the most efficient classical neural network approaches have been achieved, and sometimes they even outperform them [37].”
BackgroundAraujo, F. A., Riou, M., Torrejón, J., Tsunegi, S., Querlioz, D., Yakushiji, K., Fukushima, A., Kubota, H., Yuasa, S., Stiles, M. D., et al. (2020) · Scientific Reports · doi:10.1038/s41598-019-56991-x · Cited by 93
Dynamics of random recurrent networks with correlated low-rank structure
Schuessler, F., Dubreuil, A. M., Mastrogiuseppe, F., Ostojic, S., & Barak, O. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.013111 · Cited by 119
“This is particularly important for learning algorithms like the echo state framework or FORCE, which by construction only allow for the adaptation of the vector n [20,21].”
BackgroundMethodInfluentialBorghi, M., Biasi, S., & Pavesi, L. (2021) · Scientific Reports · doi:10.1038/s41598-021-94952-5 · Cited by 116
Fabiani, G., Calabrò, F., Russo, L., & Siettos, C. I. (2021) · Journal of Scientific Computing · doi:10.1007/s10915-021-01650-5 · Cited by 88
Lexical surprisal as a general predictor of reading time
Monsalve, I. F., Frank, S. L., & Vigliocco, G. (2012) · UCL Discovery (University College London) · Cited by 135
Prediction of telephone calls load using Echo State Network with exogenous variables
Bianchi, F. M., Scardapane, S., Uncini, A., Rizzi, A., & Sadeghian, A. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.08.010 · Cited by 95
“As forecast method we use a standard Echo State Network (ESN) (Butcher, Verstraeten, Schrauwen, Day, & Haycock, 2013; Jaeger & Haas, 2004; Lukoševičius & Jaeger, 2009; Verstraeten, Schrauwen, d’Haene, & Stroobandt, 2007), which is a particular class of Recurrent Neural Network (RNN).”
MethodHuang, J., Cao, Y., Xiong, C., & Zhang, H. (2018) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2018.2867939 · Cited by 90
“However, when the testing data change slightly away from the training data, ESNs may be ill-posed and accompanied with large output weights, which weakens their generalization capability [31].”
BackgroundMachine Learning Roadmap for Perovskite Photovoltaics
Srivastava, M., Howard, J. M., Gong, T., Rebello Sousa Dias, M., & Leite, M. S. (2021) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.1c01961 · Cited by 85
The CRISP theory of hippocampal function in episodic memory
Cheng, S. (2013) · Frontiers in Neural Circuits · doi:10.3389/fncir.2013.00088 · Cited by 93
“The computational advantage of our suggestion is that learning in feedforward networks is far simpler than in recurrent networks (Jaeger and Haas, 2004; Cheng and Frank, 2011) and feedforward memory networks have higher memory capacities than recurrent networks (Palm and Sommer, 1992).”
BackgroundIntelligent dual stream CNN and echo state network for anomaly detection
Ullah, W., Hussain, T., Khan, Z. A., Haroon, U., & Baik, S. W. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109456 · Cited by 88
Prediction for noisy nonlinear time series by echo state network based on dual estimation
Sheng, C., Zhao, J., Liu, Y., & Wang, W. (2012) · Neurocomputing · doi:10.1016/j.neucom.2011.11.021 · Cited by 86
Online Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalization
Antonik, P., Duport, F., Hermans, M., Smerieri, A., Haelterman, M., & Massar, S. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2598655 · Cited by 87
“The channel equalization task [31], [32], [35]–[38], in addition to its practical interest, does not require the use of large reservoirs to obtain the state-of-the-art results [14], [17], [19], [21].”
BackgroundA substrate-independent framework to characterize reservoir computers
Dale, M., Miller, J. F., Stepney, S., & Trefzer, M. A. (2019) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2018.0723 · Cited by 99
“The nonlinear Channel Equalization task introduced in [45] has benchmarked both simulated and physical reservoir systems [31].”
BackgroundClifford Support Vector Machines for Classification, Regression, and Recurrence
Bayro-Corrochano, E. J., & Arana‐Daniel, N. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2060352 · Cited by 85
Echo state graph neural networks with analogue random resistive memory arrays
Wang, S., Li, Y., Wang, D., Zhang, W., Chen, X., Dong, D., Wang, S., Zhang, X., Lin, P., Gallicchio, C., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00609-5 · Cited by 84
Machine Learning for Perovskites' Reap-Rest-Recovery Cycle
Howard, J. M., Tennyson, E. M., Neves, B. R. A., & Leite, M. S. (2019) · Joule · doi:10.1016/j.joule.2018.11.010 · Cited by 85
Nonintrusive reduced order modeling framework for quasigeostrophic turbulence
Rahman, S. M., Pawar, S. A., San, O., Rasheed, A., & Iliescu, T. (2019) · Physical review. E · doi:10.1103/physreve.100.053306 · Cited by 89
“The recurrent neural network (RNN) is one of the widely used neural network architectures in ROMs which is designed to operate on input information as well as the previously stored observations to predict the dependencies among the temporal data sequences [93, 94].”
BackgroundApplication of complex systems topologies in artificial neural networks optimization: An overview
Kaviani, S., & Sohn, I. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115073 · Cited by 97
Microring resonators with external optical feedback for time delay reservoir computing
Donati, G. P., Mirasso, C. R., Mancinelli, M., Pavesi, L., & Argyris, A. (2021) · Optics Express · doi:10.1364/oe.444063 · Cited by 92
“in photonic reservoir computing (RC) concepts [15, 16], in integrated platforms [17] and for applications in transmission channel equalization [18].”
BackgroundMethodInformation Systems Opportunities in Brain–Machine Interface Decoders
Kao, J. C., Stavisky, S. D., Sussillo, D., Nuyujukian, P., & Shenoy, K. V. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2307357 · Cited by 86
“In particular, the echo state network (ESN) [66] has seen wide spread application and has been investigated in both offline demonstrations (e.g., [67]) and closed-loop BMI systems (e.g., [39]).”
BackgroundUsing reservoir computers to distinguish chaotic signals
Carroll, T. L. (2018) · Physical review. E · doi:10.1103/physreve.98.052209 · Cited by 84
Learning Universal Computations with Spikes
Thalmeier, D., Uhlmann, M., Kappen, H. J., & Memmesheimer, R.-M. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004895 · Cited by 99
“This requires an overall nonlinear relaxation dynamics suitable for computations [2].”
BackgroundMethodInfluentialOnline Learning Control of Hydraulic Excavators Based on Echo-State Networks
Park, J., Lee, B.-J., Kang, S., Kim, P. Y., & Kim, H. J. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2582213 · Cited by 83
“To the best of our knowledge, this is the first time that a nonmodel-based approach has been reported to the control of excavators and the first time that an online learning control framework based on ESNs has been experimented on a real-time application.”
BackgroundThe fractal brain: scale-invariance in structure and dynamics
Grosu, G. F., Hopp, A. V., Moca, V. V., Bârzan, H., Ciuparu, A., Ercsey-Ravasz, M., Winkel, M., Linde, H., & Muresan, R. C. (2022) · Cerebral Cortex · doi:10.1093/cercor/bhac363 · Cited by 101
“Similarly, echo state networks (Jaeger and Haas 2004) with feedback connectivity present the ability to perform online computation on stochastic stimuli, like their biological counterpart (Boedecker et al.”
BackgroundAlexander, K., Vaerenbergh, T. V., Fiers, M., Mechet, P., Dambre, J., & Bienstman, P. (2013) · Optics Express · doi:10.1364/oe.21.026182 · Cited by 84
Nonlinear System Modeling With Random Matrices: Echo State Networks Revisited
Zhang, B., Miller, D. J., & Wang, Y. (2011) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2011.2178562 · Cited by 91
“In this brief, we applied random matrix theory to examine the properties of the random reservoirs used by ESNs, including different reservoir topologies (sparse or fully connected) and different connection weights (Bernoulli or Gaussian).”
InfluentialTime scales of memory, learning, and plasticity
Tetzlaff, C., Kolodziejski, C., Markelic, I., & Wörgötter, F. (2012) · Biological Cybernetics · doi:10.1007/s00422-012-0529-z · Cited by 81
Computational analysis of memory capacity in echo state networks
Farkaš, I., Bosák, R., & Gergeľ, P. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.07.012 · Cited by 88
“A number of numerical and theoretical studies have shown that, in case of randomly connected spiking networks, STM is much longer in the neutrally stable regime than in the converging or diverging regimes (Bertschinger & Natschläger, 2004; Büsing et al., 2010; Legenstein & Maass, 2007; Maass, Legenstein, & Bertschinge…”
BackgroundMethodInfluentialUnsupervised EEG feature extraction based on echo state network
Sun, L., Jin, B., Yang, H., Tong, J., Liu, C., & Xiong, H. (2018) · Information Sciences · doi:10.1016/j.ins.2018.09.057 · Cited by 86
Machine learning based on reservoir computing with time-delayed optoelectronic and photonic systems
Chembo, Y. K. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120788 · Cited by 93
Natural quantum reservoir computing for temporal information processing
Suzuki, Y., Gao, Q., Pradel, K. C., Yasuoka, K., & Yamamoto, N. (2022) · Scientific Reports · doi:10.1038/s41598-022-05061-w · Cited by 150
“As a special type of RNN, reservoir computing (RC) 11 has been actively studied, which originate from echo state networks (ESNs) 12 and liquid state machines (LSMs) 13 .”
BackgroundPhotonic neuromorphic computing using vertical cavity semiconductor lasers
Skalli, A., Robertson, J., Owen-Newns, D., Hejda, M., Porté, X., Reitzenstein, S., Hurtado, A., & Brunner, D. (2022) · Optical Materials Express · doi:10.1364/ome.450926 · Cited by 101
Recurrent Kernel Machines: Computing with Infinite Echo State Networks
Hermans, M., & Schrauwen, B. (2011) · Neural Computation · doi:10.1162/neco_a_00200 · Cited by 101
“…and Harris, 2007; Verstraeten et al., 2006), phoneme recognition (Triefenbach et al., 2010), robot navigation (Antonelo et al., 2008), time series prediction (wyffels and Schrauwen, 2010), and even yields state-of-the-art performance on the modeling of a chaotic attractor (Jaeger and Haas, 2004).”
BackgroundA neurodynamical model for working memory
Pascanu, R., & Jaeger, H. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.10.003 · Cited by 99
“RC has demonstrated its usefulness for a variety of time series processing tasks (Jaeger and Haas, 2004; Lukosevicius et al., 2007; Verstraeten et al., 2006), is becoming a standard tool in recurrent neural network modeling (Jaeger et al.”
BackgroundSelf-citationMorphological Computation and Morphological Control: Steps Toward a Formal Theory and Applications
Füchslin, R. M., Dzyakanchuk, A., Flumini, D., Hauser, H., Hunt, K. J., Luchsinger, R. H., Reller, B., Scheidegger, S., & Walker, R. (2013) · Artificial Life · doi:10.1162/artl_a_00079 · Cited by 83
“For example, it is well established that the dynamics of artificial neural networks can be used to perform complex computations [14, 20].”
BackgroundEcho State Networks for data-driven downhole pressure estimation in gas-lift oil wells
Antonelo, E. A., Camponogara, E., & Foss, B. A. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.09.009 · Cited by 91
“Both tasks are solved by using the same efficient black-box RC-based architecture [23] for modeling the partic-”
MethodModel-based Deep Reinforcement Learning for Dynamic Portfolio Optimization
Yu, P., Sern, L. J., Kulyatin, I., Shi, Z., & Dasgupta, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08740 · Cited by 58
“The RNN layer (of a reservoir computing type architecture) (Jaeger, 2003; Jaeger & Haas, 2004) is used to create nonlinear feature transforms of the historical time-series and update the bias Table 2.”
MethodShort-Term Load Forecasting With Seasonal Decomposition Using Evolution for Parameter Tuning
Høverstad, B. A., Tidemann, A., Helge Langseth, A., & Öztürk, P. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2395822 · Cited by 81
“Studies of reservoir models have demonstrated their capability for pattern recognition and classification [33] and time series forecasting tasks [32], including short-term power load forecasting [34].”
BackgroundGallicchio, C., & Micheli, A. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.08.017 · Cited by 97
ResInNet: A Novel Deep Neural Network With Feature Reuse for Internet of Things
Sun, X., Gui, G., Li, Y., Liu, R. P., & An, Y. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2853663 · Cited by 86
Wilting, J., & Priesemann, V. (2019) · Cerebral Cortex · doi:10.1093/cercor/bhz049 · Cited by 93
Haluszczynski, A., & Rath, C. W. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5118725 · Cited by 88
Wikner, A., Pathak, J., Hunt, B., Girvan, M., Arcomano, T., Szunyogh, I., Pomerance, A., & Ott, E. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0005541 · Cited by 98
Gottwald, G. A., & Reich, S. (2021) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2021.132911 · Cited by 107
“In the context of dynamical systems the framework of random feature maps was extended to include internal degrees of freedom with their own dynamics in so called echo-state networks [13, 14, 15, 16].”
Unifying framework for information processing in stochastically driven dynamical systems
Kubota, T., Takahashi, H., & Nakajima, K. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.043135 · Cited by 109
Reservoir Computing approach to Great Lakes water level forecasting
Coulibaly, P. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.11.027 · Cited by 89
Position-Based Content Attention for Time Series Forecasting with Sequence-to-Sequence RNNs
Cinar, Y. G., Mirisaee, H., Goswami, P., Gaussier, E., Aït-Bachir, A., & Strijov, V. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70139-4_54 · Cited by 74
“Subse-quently, various RNN-based models were developed for different time series, as noisy foreign exchange rate prediction [18], chaotic time series prediction in communication engineering [19] or stock price prediction [20].”
MethodPsychiatric Illnesses as Disorders of Network Dynamics
Durstewitz, D., Huys, Q. J. M., & Koppe, G. (2020) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2020.01.001 · Cited by 100
Iwana, B. K., Frinken, V., & Uchida, S. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.104971 · Cited by 85
“Gated Recurrent Units (GRU) [27] and Echo State Networks (ESN) [28] are more recent approaches to RNNs that have shown to have similar results to RNNs and LSTMs but with easier to train structures [29, 30, 31].”
ResultWu, Z., Li, Q., & Xia, X. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.2987096 · Cited by 85
“The multi-timescale echo state network (MTS-ESN) algo-rithm is proposed with the multi-tasking perspective, in which ESN is an effective and simple form of recurrent neural network (RNN) [19].”
BackgroundGradient based hyperparameter optimization in Echo State Networks
Thiede, L., & Parlitz, U. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.02.001 · Cited by 92
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.