Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 201–300 of 1,009
Nonlinearity in memristors for neuromorphic dynamic systems
Yang, K., Joshua Yang, J., Huang, R., & Yang, Y. (2022) · Small Science · doi:10.1002/smsc.202100049 · Cited by 151
Electrocardiogram Classification Using Reservoir Computing With Logistic Regression
Escalona-Morán, M., Soriano, M. C., Fischer, I., & Mirasso, C. R. (2014) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2014.2332001 · Cited by 165
Physical reservoir computing using magnetic skyrmion memristor and spin torque nano-oscillator
Jiang, W., Chen, L., Zhou, K., Li, L., Fu, Q., Du, Y., & Liu, R. (2019) · Applied Physics Letters · doi:10.1063/1.5115183 · Cited by 159
Capacity and Trainability in Recurrent Neural Networks
Collins, J., Sohl‐Dickstein, J., & Sussillo, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.09913 · Cited by 74
“Previous work on short term memory traces explores the tradeoffs between memory fidelity and duration, for the case that a new input is presented to the RNN at every time step (Jaeger & Haas, 2004; Maass et al., 2002; White et al., 2004; Ganguli et al., 2008; Charles et al., 2014).”
BackgroundA new boosting algorithm for improved time-series forecasting with recurrent neural networks
Assaad, M. A., Boné, R., & Cardot, H. (2006) · Information Fusion · doi:10.1016/j.inffus.2006.10.009 · Cited by 170
“…the many neural network architectures employed for time series prediction we can mention MLPs with a time window in the input, MLPs with finite impulse response (FIR) connections both from the input to the hidden layer and from the hidden layer to the output [18], recurrent networks obtained by…”
BackgroundDifferential Encoding of Time by Prefrontal and Striatal Network Dynamics
Bakhurin, K. I., Goudar, V., Shobe, J. L., Claar, L. D., Buonomano, D. V., & Masmanidis, S. C. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1789-16.2016 · Cited by 176
Constructing optimized binary masks for reservoir computing with delay systems
Appeltant, L., Van der Sande, G., Danckaert, J., & Fischer, I. (2014) · Scientific Reports · doi:10.1038/srep03629 · Cited by 144
Wind speed and wind direction forecasting using echo state network with nonlinear functions
Chitsazan, M. A., Fadali, M. S., & Trzynadlowski, A. M. (2018) · Renewable Energy · doi:10.1016/j.renene.2018.07.060 · Cited by 143
Jiang, J., & Lai, Y. (2019) · Physical Review Research · doi:10.1103/physrevresearch.1.033056 · Cited by 168
Qadir, Z., Khan, S. I., Khalaji, E., Munawar, H. S., Al-Turjman, F. M., Mahmud, M. A. P., Kouzani, A. Z., & Le, K. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.01.018 · Cited by 159
Hinaut, X., & Dominey, P. F. (2013) · PLoS ONE · doi:10.1371/journal.pone.0052946 · Cited by 140
“Reservoir computing is a machine learning technique in which a recurrent network with fixed connections is used to encode the spatiotemporal structure of an input sequence, and connections to a readout layer are trained using efficient algorithms to produce a desired output in response to input sequences [35,36].”
MethodIn search of the neural circuits of intrinsic motivation
Kaplan, F. (2007) · Frontiers in Neuroscience · doi:10.3389/neuro.01.1.1.017.2007 · Cited by 140
“Maas and Markram suggested o view a column as a liquid state machine (LSM) (Maas et al., 2002) (which s somewhat similar to Echo State Networks described by Jaeger (Jaeger, 001; Jaeger and Haas, 2004)).”
Nguimdo, R. M., Verschaffelt, G., Danckaert, J., & der Sande, G. V. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2404346 · Cited by 149
“We use nonlinear channel equalization [2], [7], [24] and Santa-Fe time series prediction [5], [6], [17] as benchmark tasks in this brief to demonstrate the parallel computational abilities of SRLs.”
BackgroundMethodInfluentialThe neurobench framework for benchmarking neuromorphic computing algorithms and systems
Yik, J., den Berghe, K. V., den Blanken, D., Bouhadjar, Y., Fabre, M., Hueber, P., Ke, W., Khoei, M. A., Kleyko, D. V., Pacik-Nelson, N., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56739-4 · Cited by 213
“Mackey-Glass has been widely adopted as a benchmark for evaluating temporal predictors, including neuromorphic models 29 – 31 .”
InfluentialHierarchy and dynamics of neural networks
Kaiser, M. (2010) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2010.00112 · Cited by 136
“They demonstrate that the stability of Echo State Networks (Jaeger and Haas, 2004) is potentially enhanced, as indicated by the range of spectral radius values, when the networks are structured in a multi-modular, hierarchical way.”
BackgroundToyoizumi, T., & Abbott, L. F. (2011) · Physical Review E · doi:10.1103/physreve.84.051908 · Cited by 140
Inferring collective dynamical states from widely unobserved systems
Wilting, J., & Priesemann, V. (2018) · Nature Communications · doi:10.1038/s41467-018-04725-4 · Cited by 158
The challenges of modern computing and new opportunities for optics
Li, C., Zhang, X., Li, J., Fang, T., & Dong, X. (2021) · PhotoniX · doi:10.1186/s43074-021-00042-0 · Cited by 152
Hoerzer, G. M., Legenstein, R., & Maass, W. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs348 · Cited by 139
“…been shown that if such supervised learning is applied to readout neurons whose axons are also connected to other neurons within the network, such generic networks can also learn to generate periodic patterns and working memory (Jaeger and Haas 2004; Maass et al. 2007; Sussillo and Abbott 2009).”
A Neuromorphic Event-Based Neural Recording System for Smart Brain-Machine-Interfaces
Corradi, F., & Indiveri, G. (2015) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2015.2479256 · Cited by 157
“The first layer of the network is composed of a recurrent pool of 128 neurons that acts as a spiking reservoir [30]: it receives spikeevent inputs from the neural recording system, and it exploits the analog dynamics of the neurons and synapses to enhance the temporal properties of input patterns.”
BackgroundOptimal Forecast Combination Based on Neural Networks for Time Series Forecasting
Wang, L., Wang, Z., Qu, H., & Liu, S. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.02.004 · Cited by 148
Synchronization of chaotic systems and their machine-learning models
Weng, T., Yang, H., Gu, C., Zhang, J., & SMALL, M. P. (2019) · Physical review. E · doi:10.1103/physreve.99.042203 · Cited by 134
Chen, P., Liu, R., Aihara, K., & Chen, L. (2020) · Nature Communications · doi:10.1038/s41467-020-18381-0 · Cited by 160
Reservoir-based techniques for speech recognition
Verstraeten, D., Schrauwen, B., & Stroobandt, D. (2006) · The 2006 IEEE International Joint Conference on Neural Network Proceedings · doi:10.1109/ijcnn.2006.1716215 · Cited by 131
“Also, we have applied an ESN to the same task which allowed us to investigate the influence of some important interconnection parameters on the performance of the reservoir, and to formulate some very interesting conclusions about the connection between the regime the reservoir operates in and its recognition accuracy.”
BackgroundMethodInfluentialFuruta, T., Fujii, K., Nakajima, K., Tsunegi, S., Kubota, H., Suzuki, Y., & Miwa, S. (2018) · Physical Review Applied · doi:10.1103/physrevapplied.10.034063 · Cited by 146
“Additionally, an echo-state network [13] is introduced for comparison with the system using MTJs, where the following function is employed for Eq.”
MethodReconstructing computational system dynamics from neural data with recurrent neural networks
Durstewitz, D., Koppe, G., & Thurm, M. I. (2023) · Nature reviews. Neuroscience · doi:10.1038/s41583-023-00740-7 · Cited by 230
Nguimdo, R. M., Verschaffelt, G., Danckaert, J., & der Sande, G. V. (2014) · Optics Express · doi:10.1364/oe.22.008672 · Cited by 141
Bits from Brains for Biologically Inspired Computing
Wibral, M., Lizier, J. T., & Priesemann, V. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00005 · Cited by 128
Locomotion Without a Brain: Physical Reservoir Computing in Tensegrity Structures
Caluwaerts, K., D'Haene, M., Verstraeten, D., & Schrauwen, B. (2013) · Artificial Life · doi:10.1162/artl_a_00080 · Cited by 134
“Most importantly, we distinguish liquid state machines [43] and echo state networks [34, 35].”
BackgroundCompact hardware liquid state machines on FPGA for real-time speech recognition
Schrauwen, B., D’Haene, M., Verstraeten, D., & Campenhout, J. V. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.12.009 · Cited by 162
Collective Behavior of a Small-World Recurrent Neural System With Scale-Free Distribution
Deng, Z., & Zhang, Y. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.894082 · Cited by 129
“modeling of nonlinear dynamic system [18], [19].”
BackgroundMethodInfluentialExploiting short-term memory in soft body dynamics as a computational resource
Nakajima, K., Li, T., Hauser, H., & Pfeifer, R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0437 · Cited by 128
“A number of different implementations for reservoirs have been proposed: for example, abstract dynamical systems for echo state networks (ESN) [8, 9], or models of neurons for liquid state machines [10].”
BackgroundExtreme events in dynamical systems and random walkers: A review
Chowdhury, S. N., Ray, A., Dana, S. K., & Ghosh, D. (2022) · Physics Reports · doi:10.1016/j.physrep.2022.04.001 · Cited by 145
“[234, 235, 393] for further understanding of the reservoir computing.”
BackgroundMethodRotating neurons for all-analog implementation of cyclic reservoir computing
Liang, X., Zhong, Y., Tang, J., Liu, Z., Yao, P., Sun, K., Zhang, Q., Gao, B., Heidari, H., Qian, H., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-29260-1 · Cited by 149
Reinforced recurrent neural networks for multi-step-ahead flood forecasts
Chen, P.-A., Chang, L., & Chang, F. (2013) · Journal of Hydrology · doi:10.1016/j.jhydrol.2013.05.038 · Cited by 133
“Because most observational disciplines tend to infer properties of an uncertain system from the analysis of time-dependent data, analytical technologies for extracting the meaningful characteristics of time series data have some inherent limitations, which has been a widely discussed issue for a long time (Brockwell a…”
BackgroundAdaptive Elastic Echo State Network for Multivariate Time Series Prediction
Xu, M., & Han, M. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2467167 · Cited by 127
“aforementioned issues, called echo state network (ESN), was proposed by Jaeger [13] and Jaeger and Haas [14].”
BackgroundInfluentialReservoir Computing Universality With Stochastic Inputs
Gonon, L., & Ortega, J. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2899649 · Cited by 177
“An important particular case of the RC systems in (1) and (2) is echo state networks (ESNs) introduced, in different contexts, in [8], [11], and [12], and that are built using the transformations xt = σ(Axt−1 + Czt + ζ ) yt = w xt (3)”
BackgroundA survey of fractional calculus applications in artificial neural networks
Joshi, M. P., Bhosale, S. R., & Vyawahare, V. A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10474-8 · Cited by 137
“Other weights are considered to be random values (Jaeger and Haas 2004; Jaeger 2001; Zhang et al. 2014).”
BackgroundHow Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation
Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.7906 · Cited by 103
“…forward and backwards in time) and that correspondingly average their gradients over longer horizons, or that could be done implicitly thanks to the long-term memory that architectures such as the echo-state network / liquid memory recurrence (Maass et al. , 2002; Jaeger and Haas, 2004) provide.”
BackgroundA random matrix approach to neural networks
Louart, C., Liao, Z., & Couillet, R. (2018) · The Annals of Applied Probability · doi:10.1214/17-aap1328 · Cited by 231
Optical Reservoir Computing Using Multiple Light Scattering for Chaotic Systems Prediction
Dong, J., Rafayelyan, M. S., Krząkała, F., & Gigan, S. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2936281 · Cited by 153
“Random matrix multiplications are present in Echo-State Networks [7], [8] and Liquid-State Machines [9].”
BackgroundLearning Precisely Timed Spikes
Memmesheimer, R.-M., Rubin, R., Ölveczky, B. P., & Sompolinsky, H. (2014) · Neuron · doi:10.1016/j.neuron.2014.03.026 · Cited by 145
“…introduces variability in the spike timing during training, which forces the learning dynamics to converge onto a stable solution, similar to the stabilizing effects of noise on learning in nonspiking recurrent networks (Jaeger and Haas, 2004; Jaeger et al., 2007; Sussillo and Abbott, 2009).”
BackgroundSymmetry Breaking in Space-Time Hierarchies Shapes Brain Dynamics and Behavior
Pillai, A. S., & Jirsa, V. (2017) · Neuron · doi:10.1016/j.neuron.2017.05.013 · Cited by 140
Preparatory activity and the expansive null-space
Churchland, M. M., & Shenoy, K. V. (2024) · Nature reviews. Neuroscience · doi:10.1038/s41583-024-00796-z · Cited by 164
Pruning and regularization in reservoir computing
Dutoit, X., Schrauwen, B., Campenhout, J. V., Stroobandt, D., Brussel, H. V., & Nuttin, M. (2009) · Neurocomputing · doi:10.1016/j.neucom.2008.12.020 · Cited by 144
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Soltoggio, A., Stanley, K. O., & Risi, S. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.013 · Cited by 121
“(2007), showing that the evolved RNNs were more compact and resulted in better learning than randomly initialized echo state networks (Jaeger and Haas, 2004).”
BackgroundSimple Deterministically Constructed Cycle Reservoirs with Regular Jumps
Rodan, A., & Tiňo, P. (2012) · Neural Computation · doi:10.1162/neco_a_00297 · Cited by 141
Topological and Dynamical Complexity of Random Neural Networks
Wainrib, G., & Touboul, J. (2013) · Physical Review Letters · doi:10.1103/physrevlett.110.118101 · Cited by 124
Short-term wind speed forecasting based on spectral clustering and optimised echo state networks
Liu, D., Wang, J., & Wang, H. (2015) · Renewable Energy · doi:10.1016/j.renene.2015.01.022 · Cited by 123
Inverse-Free Extreme Learning Machine With Optimal Information Updating
Li, S., You, Z., Guo, H., Luo, X., & Zhao, Z. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2434841 · Cited by 136
“, liquid state machines [34] and echo state networks [35], [36].”
BackgroundEmbedding and approximation theorems for echo state networks
Hart, A., Hook, J., & Dawes, J. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.05.013 · Cited by 155
Data-driven prediction in dynamical systems: recent developments
Ghadami, A., & Epureanu, B. I. (2022) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2021.0213 · Cited by 184
“One of the most important early results in the field of predicting dynamics using neural network came from Herbert Jaeger & Harald Haass [108] where a particular kind of neural network, called a reservoir computer, used to forecast the Mackey–Glass dynamical system.”
MethodMultilayered Echo State Machine: A Novel Architecture and Algorithm
Malik, Z. K., Hussain, A., & Wu, Q. M. J. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2533545 · Cited by 133
“where the stepsize δ typically set toδ = 1/10 [20], [23].”
BackgroundComputing with networks of nonlinear mechanical oscillators
Coulombe, J. C., York, M. C. A., & Sylvestre, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178663 · Cited by 121
“In a particularly simple network architecture called “reservoir computing” [1], an output function is formed using a weight vector w ∈ R N .”
Exploiting the Dynamics of Soft Materials for Machine Learning
Nakajima, K., Häuser, H., Li, T., & Pfeifer, R. (2018) · Soft Robotics · doi:10.1089/soro.2017.0075 · Cited by 141
“11 To show the relevance of the multiplexing technique clearly, a setting without multiplexing is also demonstrated, in which we take 10 sensory values, f s 1 k s þ ( s (cid:2) 1) , . . . , s 10 k s þ ( s (cid:2) 1) g , and 1 bias term as a computational resource for each computational timestep k…”
BackgroundMethodChattering-free fuzzy sliding-mode control strategy for uncertain chaotic systems
Yau, H., & Chen, C. (2006) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2006.03.077 · Cited by 126
“The simulation results with initial value X0 = [3 4 2] are shown in Figs.”
BackgroundMethodInformation Processing Using Transient Dynamics of Semiconductor Lasers Subject to Delayed Feedback
Hicke, K., Escalona-Morán, M., Brunner, D., Soriano, M. C., Fischer, I., & Mirasso, C. R. (2013) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2013.2241738 · Cited by 124
“All these aspects are crucial for the computational performance of RC [4].”
BackgroundMethodInfluentialfull-FORCE: A target-based method for training recurrent networks
DePasquale, B., Cueva, C. J., Rajan, K., Escola, G. S., & Abbott, L. F. (2018) · PLoS ONE · doi:10.1371/journal.pone.0191527 · Cited by 159
Laplacian Echo State Network for Multivariate Time Series Prediction
Han, M., & Xu, M. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2574963 · Cited by 120
“To substantiate the effectiveness of the LAESN, we also evaluate general ESNs [6], ESGP [10], PCAESN [12], RESN [13], support vector echo state machine (SVESM) [30], as well as regularized simple cycle reservoir (RSCR) model [31].”
BackgroundMethodResultInfluentialArtificial Intelligence and Advanced Materials
López, C. (2023) · Advanced Materials · doi:10.1002/adma.202208683 · Cited by 144
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
“For example, “liquid-” (Maass et al., 2002) or “echo-state machines” (Jaeger and Haas, 2004) are randomly connected recurrent networks that form a basis set (also known as a “reservoir”) of random filters, which can be harnessed for learning with tunable readout weights.”
BackgroundCarnevale, F., de Lafuente, V., Romo, R., Barak, O., & Parga, N. (2015) · Neuron · doi:10.1016/j.neuron.2015.04.014 · Cited by 130
“This builds on the framework of state-dependent or reservoir computing (Buonomano and Maass, 2009; Jaeger and Haas, 2004; Maass et al., 2007) in which computations arise from the interaction between external stimuli and the internally generated dynamics produced by the network’s recurrent connectivity.”
All-Optical Reservoir Computing on a Photonic Chip Using Silicon-Based Ring Resonators
Coarer, F. D.-L., Sciamanna, M., Katumba, A., Freiberger, M., Dambre, J., Bienstman, P., & Rontani, D. (2018) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2018.2836985 · Cited by 119
A Novel Framework of Reservoir Computing for Deterministic and Probabilistic Wind Power Forecasting
Wang, J., Niu, T., Lu, H., Yang, W., & Du, P. (2019) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2019.2890875 · Cited by 123
“Echo state network (ESN), as an advanced RNN, was first proposed in [21], with the merits of high efficiency and accuracy for time series forecasting.”
BackgroundTemporal Information Processing on Noisy Quantum Computers
Chen, J., Nurdin, H. I., & Yamamoto, N. (2020) · Physical Review Applied · doi:10.1103/physrevapplied.14.024065 · Cited by 168
A decentralized training algorithm for Echo State Networks in distributed big data applications
Scardapane, S., Wang, D., & Panella, M. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.07.006 · Cited by 122
“ESNs have been applied successfully to a wide range of domains, including chaotic time-series prediction (Jaeger and Haas, 2004; Li et al., 2012), grammatical inference (Tong et al.”
Enhanced performance of a reservoir computer using polarization dynamics in VCSELs
Vatin, J., Rontani, D., & Sciamanna, M. (2018) · Optics Letters · doi:10.1364/ol.43.004497 · Cited by 118
Nanoscale resistive switching devices for memory and computing applications
Lee, S. H., Zhu, X., & Lu, W. D. (2020) · Nano Research · doi:10.1007/s12274-020-2616-0 · Cited by 125
Deep networks for system identification: A survey
Pillonetto, G., Aravkin, A. Y., Gedon, D., Ljung, L., Ribeiro, A. H., & Schön, T. B. (2024) · Automatica · doi:10.1016/j.automatica.2024.111907 · Cited by 210
Second type of criticality in the brain uncovers rich multiple-neuron dynamics
Dahmen, D., Grün, S., Diesmann, M., & Helias, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1818972116 · Cited by 149
“Such networks provide a wealth of transformations on the input and, therefore, support the concept of reservoir computing (50, 51).”
BackgroundQuantum reservoir computing with a single nonlinear oscillator
Govia, L. C. G., Ribeill, G. J., Rowlands, G. E., Krovi, H., & Ohki, T. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.013077 · Cited by 160
Machine Learning with Neural Networks: An Introduction for Scientists and Engineers
Mehlig, B. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108860604 · Cited by 201
Generalized synchronization of chaos for secure communication: Remarkable stability to noise
Москаленко, О. И., Koronovskii, A. A., & Hramov, A. E. (2010) · Physics Letters A · doi:10.1016/j.physleta.2010.05.024 · Cited by 116
Echo State Networks With Orthogonal Pigeon-Inspired Optimization for Image Restoration
Duan, H., & Wang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2479117 · Cited by 123
“…models, including dynamical canonical nonlinear programming circuit [23], Adachi neural network [24], Elman network [25], and echo state network (ESN) [26], have been proposed to guarantee the optimality, improve the convergence properties, extend the applicability, and reduce the complexity.”
BackgroundMadhiarasan, M., & Deepa, S. N. (2016) · Artificial Intelligence Review · doi:10.1007/s10462-016-9506-6 · Cited by 125
“Table 6 infers that compared to the other existing model in the literature the proposed model achieves better accuracy with minimal error value in comparison with that of the extreme learning machine model (Wang and Hu 2015), echo state network model (Jaeger and Haas 2004), wavelet neuralmodel (Meng et al.”
ResultRanganathan, G. N., Apostolides, P. F., Harnett, M. T., Xu, N.-L., Druckmann, S., & Magee, J. C. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0254-6 · Cited by 100
Learning Without Neurons in Physical Systems
Stern, M., & Murugan, A. (2023) · Annual Review of Condensed Matter Physics · doi:10.1146/annurev-conmatphys-040821-113439 · Cited by 177
An experimental characterization of reservoir computing in ambient assisted living applications
Bacciu, D., Barsocchi, P., Chessa, S., Gallicchio, C., & Micheli, A. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1364-4 · Cited by 128
“RC comprises several approaches, including the popular Echo State Networks (ESNs) (see Jaeger and Haas (2004); Jaeger (2001)), Liquid State Machines (see Maass et al (2002)) and BackPropagation Decorrelation (Steil, 2004, 2006), among the others.”
InfluentialFrozen Pretrained Transformers as Universal Computation Engines
Lu, K., Grover, A., Abbeel, P., & Mordatch, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20729 · Cited by 113
“Similarly to the FPT setup and Global Workspace Theory, in reservoir computing (Tanaka et al. 2019) and echo state networks (Jaeger 2001; Jaeger and Haas 2004), a random recurrent network is frozen and only the output readout layer is trained.”
MethodJames, C. D., Aimone, J. B., Miner, N. E., Vineyard, C. M., Rothganger, F. H., Carlson, K. D., Mulder, S. A., Draelos, T. J., Faust, A., Marinella, M. J., et al. (2017) · Biologically Inspired Cognitive Architectures · doi:10.1016/j.bica.2016.11.002 · Cited by 153
Edge-of-chaos learning achieved by ion-electron–coupled dynamics in an ion-gating reservoir
Nishioka, D., Tsuchiya, T., Namiki, W., Takayanagi, M., Imura, M., Koide, Y., Higuchi, T., & Terabe, K. (2022) · Science Advances · doi:10.1126/sciadv.ade1156 · Cited by 127
Chatzis, S., & Demiris, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2162109 · Cited by 120
“However, for most practical purposes, the echo state property can be easily satisfied by merely ensuring that the reservoir weight matrix W is contractive, i.e., by scaling the reservoir weight matrix so that its spectral radius ρ(W) (that is, its largest absolute eigenvalue) is less than 1 [17].”
BackgroundAn Augmented Echo State Network for Nonlinear Adaptive Filtering of Complex Noncircular Signals
Xia, Y., Jelfs, B., Hulle, M. M. V., Príncipe, J. C., & Mandic, D. P. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2085444 · Cited by 113
“Recently, a class of discrete time RNNs, called Echo State Networks (ESNs) have been introduced with the aim to reduce the complexity of computation encountered by standard RNNs [4].”
BackgroundInfluentialReservoir Computing Using Multiple Lasers With Feedback on a Photonic Integrated Circuit
Sugano, C., Kanno, K., & Uchida, A. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2929179 · Cited by 118
“R ESERVOIR computing has been investigated intensively as a new technology of artificial intelligence in recent years [1], [2].”
BackgroundA tutorial on deep learning-based data analytics in manufacturing through a welding case study
Wang, Q., Jiao, W., Wang, P. E., & Zhang, Y. (2020) · Journal of Manufacturing Processes · doi:10.1016/j.jmapro.2020.04.044 · Cited by 125
High-efficiency chaotic time series prediction based on time convolution neural network
Cheng, W. X., Wang, Y., Peng, Z., Ren, X., Shuai, Y., Zang, S., Liu, H., Cheng, H., & Wu, J. (2021) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2021.111304 · Cited by 121
DeePr-ESN: A deep projection-encoding echo-state network
Ma, Q., Shen, L., & Cottrell, G. W. (2019) · Information Sciences · doi:10.1016/j.ins.2019.09.049 · Cited by 118
Recurrent neural network training with dark knowledge transfer
Tang, Z., Wang, D., & Zhang, Z. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472809 · Cited by 125
“In the echo state network (ESN) architecture proposed by [9], the hidden-to-hidden weights a re not learned in the training so the problem of odd gradients does not exist.”
BackgroundInterval Type-2 Fuzzy Neural Networks for Chaotic Time Series Prediction: A Concise Overview
Han, M., Zhong, K., Qiu, T., & Han, B. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2834356 · Cited by 114
“ral networks [16], multilayer perceptron neural networks [17], echo state network [2], [18], ensemble neural network [19], and fuzzy neural networks (FNNs) [20].”
BackgroundDeep belief echo-state network and its application to time series prediction
Sun, X., Li, T., Li, Q., Huang, Y., & Li, Y. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.05.022 · Cited by 110
Antonik, P., Gulina, M., Pauwels, J., & Massar, S. (2018) · Physical review. E · doi:10.1103/physreve.98.012215 · Cited by 127
“The reservoir computer used in this work is a discretetime echo state network, as introduced in [1, 6].”
BackgroundMethodPower‐Efficient Multisensory Reservoir Computing Based on Zr‐Doped HfO2 Memcapacitive Synapse Arrays
Pei, M., Zhu, Y., Liu, S., Cui, H., Li, Y., Yan, Y., Li, Y., Wan, C., & Wan, Q. C. (2023) · Advanced Materials · doi:10.1002/adma.202305609 · Cited by 116
Quaternion-Valued Echo State Networks
Xia, Y., Jahanchahi, C., & Mandic, D. P. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2320715 · Cited by 113
Deep learning algorithm for data-driven simulation of noisy dynamical system
Yeo, K., & Melnyk, I. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.024 · Cited by 120
Polaritonic Neuromorphic Computing Outperforms Linear Classifiers
Ballarini, D., Gianfrate, A., Panico, R., Opala, A., Ghosh, S., Dominici, L., Ardizzone, V., Giorgi, M. D., Lerario, G., Gigli, G., et al. (2020) · Nano Letters · doi:10.1021/acs.nanolett.0c00435 · Cited by 136
Defect Detection in Reinforced Concrete Using Random Neural Architectures
Butcher, J. B., Day, C. R., Austin, J. C., Haycock, P. W., Verstraeten, D., & Schrauwen, B. (2014) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12039 · Cited by 106
“Using such a simple and efficient training procedure has been shown to provide improved performance for a variety of tasks (Huang et al., 2006; Jaeger, 2001; Jaeger and Haas, 2004).”
MethodDesign and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing
Kudithipudi, D., Saleh, Q., Merkel, C., Thesing, J., & Wysocki, B. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00502 · Cited by 105
“This idea was developed in tandem by two independent research groups, Jaeger (Jaeger, 2001; Jaeger and Haas, 2004) as the Echo State Network and Maass as the Liquid State Machine (Maass et al., 2002; Buonomano and Maass, 2009).”
MethodChallenges in complex systems science
Miguel, M. S., Johnson, J. H., Kertész, J., Kaski, K. K., Díaz-Guilera, A., MacKay, R. S., Loreto, V., Érdi, P., & Helbing, D. (2012) · The European Physical Journal Special Topics · doi:10.1140/epjst/e2012-01694-y · Cited by 138
Optimizing the echo state network with a binary particle swarm optimization algorithm
Wang, H., & Yan, X. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.06.003 · Cited by 113
Integrated photonic neuromorphic computing: opportunities and challenges
Farmakidis, N., Dong, B., & Bhaskaran, H. (2024) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-024-00050-9 · Cited by 140
“Here, the reservoir is treated as a black box, with random, temporally fixed nonlinear nodes wherein training the system entails the modification of a set of nodes that form the output layer 125 .”
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.