Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 401–500 of 1,009
Physics-enhanced neural networks learn order and chaos
Choudhary, A., Lindner, J. F., Holliday, E. G., Miller, S. T., Sinha, S., & Ditto, W. L. (2020) · Physical review. E · doi:10.1103/physreve.101.062207 · Cited by 102
“current neural networks of reservoir computing [26–28].”
Game Theory for Next Generation Wireless and Communication Networks: Modeling, Analysis, and Design
Han, Z., Niyato, D. T., Saad, W., & Başar, T. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108277402 · Cited by 90
Discovering multiscale dynamical features with hierarchical Echo State Networks
Jaeger, H. (2007) · Cited by 112
Self-citationFPGA based spike-time dependent encoder and reservoir design in neuromorphic computing processors
Yi, Y., Liao, Y., Wang, B., Fu, X., Shen, F., Hou, H., & Liu, L. (2016) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2016.03.009 · Cited by 79
Time Series Classification Using Time Warping Invariant Echo State Networks
Tanisaro, P., & Heidemann, G. (2016) · 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2016.0149 · Cited by 75
“Convergence is slow and not always guaranteed [19].”
BackgroundNetwork structure effects in reservoir computers
Carroll, T. L., & Pecora, L. M. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5097686 · Cited by 79
Lyapunov spectra of chaotic recurrent neural networks
Engelken, R., Wolf, F., & Abbott, L. F. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.043044 · Cited by 140
“Such a transition is relevant for information processing because the network loses its dependence on initial conditions, which could be a desirable feature if the network must reliably generate different output trajectories for different input patterns [14–16].”
BackgroundDo reservoir computers work best at the edge of chaos?
Carroll, T. L. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0038163 · Cited by 79
Anticipating synchronization with machine learning
Fan, H., Kong, L.-W., Lai, Y., & Wang, X. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.023237 · Cited by 91
“The idea and principle of exploiting reservoir computing for predicting the state evolution of chaotic systems were first laid out about two decades ago [11, 12].”
BackgroundReservoir computing as digital twins for nonlinear dynamical systems
Kong, L.-W., Weng, Y., Glaz, B. J., Haile, M., & Lai, Y. (2023) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0138661 · Cited by 102
“Reservoir computing, 18–20 a kind of recurrent neural network, naturally stands out as a suitable candidate, which in recent years has been extensively studied for predicting chaotic systems.”
BackgroundEcho state networks: appeal and challenges
Prokhorov, D. V. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556091 · Cited by 79
A θ–γ Oscillation Code for Neuronal Coordination during Motor Behavior
Igarashi, J., Isomura, Y., Arai, K., Harukuni, R., & Fukai, T. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.2126-13.2013 · Cited by 96
“…may reflect the functional difference between the hippocampal circuit specialized for memory processing (Nakashiba et al., 2012) and the “general-purpose neocortical circuit,” such as transient memory storage and its readout (Maass et al., 2002; Jaeger and Haas, 2004; Sussillo and Abbott, 2009).”
BackgroundState-Dependent Computation Using Coupled Recurrent Networks
Rutishauser, U., & Douglas, R. J. (2009) · Neural Computation · doi:10.1162/neco.2008.03-08-734 · Cited by 82
“Another approach has been to project the input into a high dimensional space using randomly connected units (the liquid state machine, LSM) (Maass et al., 2007; Jaeger and Haas, 2004).”
BackgroundLeijnen, S., & Fjodor, V. V. (2020) · International Society for the Study of Information Summit (IS4SI Summit) · doi:10.3390/proceedings2020047009 · Cited by 89
Sackesyn, S., Ma, C., Dambre, J., & Bienstman, P. (2021) · Optics Express · doi:10.1364/oe.435013 · Cited by 82
Computational State Space Models for Activity and Intention Recognition. A Feasibility Study
Krüger, F., Nyolt, M., Yordanova, K., Hein, A., & Kirste, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109381 · Cited by 83
“Therefore, interestingly, the HMM indeed had more degrees of freedom to adapt to the data than the CSSM. (Nevertheless, we think there are other data-driven models, such as Echo State Networks [40], which will provide a much bigger challenge to the CSSM method with respect to sheer performance.)”
Modeling neural plasticity in echo state networks for classification and regression
Yusoff, M.-H., Chrol-Cannon, J., & Jin, Y. (2015) · Information Sciences · doi:10.1016/j.ins.2015.11.017 · Cited by 80
“In this study, we use Mackey–Glass (MG) time series dataset for evaluating the performance of the ESNs for prediction tasks, as many studies in the literature showed that ESNs work is well suited for time series prediction compared to other methods [11,14].”
BackgroundMethodRobust Optimization and Validation of Echo State Networks for learning chaotic dynamics
Racca, A., & Magri, L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.05.004 · Cited by 96
Parallel and deep reservoir computing using semiconductor lasers with optical feedback
Hasegawa, H., Kanno, K., & Uchida, A. (2022) · Nanophotonics · doi:10.1515/nanoph-2022-0440 · Cited by 89
“There has been a considerable rise in the demand and subsequent interest in reservoir computing in the past two decades [7, 8].”
BackgroundReproducible Sequence Generation In Random Neural Ensembles
Huerta, R., & Rabinovich, M. (2004) · Physical Review Letters · doi:10.1103/physrevlett.93.238104 · Cited by 77
Synthesis of recurrent neural networks for dynamical system simulation
Trischler, A., & D’Eleuterio, G. M. T. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.04.001 · Cited by 95
“This set includes backpropagation through time (Werbos, 1990), real-time recurrent learning (Williams & Zipser, 1989), the extended Kalman filter (Feldkamp et al., 1998), reservoir computing (Jaeger & Haas, 2004), and phase-space learning (Tsung & Cottrell, 1995).”
MethodImaging and computing with disorder
Gigan, S. (2022) · Nature Physics · doi:10.1038/s41567-022-01681-1 · Cited by 100
“convolutions, matrix vector multiplications and photonic integrated unitary circuits 58,59 .”
BackgroundCognitive Computation with Autonomously Active Neural Networks: An Emerging Field
Gros, C. (2009) · Cognitive Computation · doi:10.1007/s12559-008-9000-9 · Cited by 82
“The ‘echo-state’ approach using discrete-time rate-encoding neurons [22,23], and the ‘liquid state machine’ using continuous-time spiking neurons [24,25].”
MethodEncoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks
Goudar, V., & Buonomano, D. V. (2018) · eLife · doi:10.7554/elife.31134 · Cited by 81
Machine-learning inference of fluid variables from data using reservoir computing
Nakai, K., & Saiki, Y. (2018) · Physical review. E · doi:10.1103/physreve.98.023111 · Cited by 89
Fading memory echo state networks are universal
Gonon, L., & Ortega, J. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.01.025 · Cited by 115
Sequential and efficient neural-population coding of complex task information
Koay, S. A., Charles, A. S., Thiberge, S. Y., Brody, C. D., & Tank, D. W. (2021) · Neuron · doi:10.1016/j.neuron.2021.10.020 · Cited by 91
Phase Space Graph Convolutional Network for Chaotic Time Series Learning
Ren, W., Jin, N., & OuYang, L. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3363089 · Cited by 75
“A lightweight task is to predict the system parameter or output for a given system [4], [5], [6], [7], [8], [9], [10], [11].”
BackgroundFading memory and time series prediction in recurrent networks with different forms of plasticity
Lazăr, A., Pipa, G., & Triesch, J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.020 · Cited by 94
Noise-Robust Automatic Speech Recognition Using a Predictive Echo State Network
Skowronski, M. D., & Harris, J. G. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.896669 · Cited by 80
“The ESN has been used to model nonlinear dynamical systems [23] and has also been applied to the problem of channel optimization [24].”
MethodMovement Generation with Circuits of Spiking Neurons
Joshi, P., & Maass, W. (2005) · Neural Computation · doi:10.1162/0899766054026684 · Cited by 90
“used instead of neural circuits, and to the melody-generation and prediction of chaotic time series with artificial neural networks in discrete time of (Jäger, 2002; Jäger and Haas, 2004).”
MethodLian, C., Zeng, Z., Yao, W., Tang, H., & Chen, C. L. P. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2512283 · Cited by 75
“as well as an echo state network [37] for a recurrent neural”
BackgroundReservoir Computing with an Ensemble of Time-Delay Reservoirs
Ortín, S., & Pesquera, L. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9463-7 · Cited by 82
“To analyze the computational abilities of a delay-based reservoir with several TDRs, we have chosen four benchmark tasks: the linear memory capacity [11, 18], the NARMA-10 [3, 8, 19], the nonlinear channel equalization [1, 3, 11, 19] and the Santa Fe prediction [3, 10, 12, 20].”
Local online learning in recurrent networks with random feedback
Murray, J. M. (2019) · eLife · doi:10.7554/elife.43299 · Cited by 126
Deep Learning Techniques for Cyber Security Intrusion Detection : A Detailed Analysis
Ferrag, M. A., Maglaras, L. A., Janicke, H., & Smith, R. (2019) · Electronic workshops in computing · doi:10.14236/ewic/icscsr19.16 · Cited by 78
“networks proposed by Jordan (1997) and Echo State networks proposed by Jaeger and Haas (2004).”
BackgroundApproximation bounds for random neural networks and reservoir systems
Gonon, L., Grigoryeva, L., & Ortega, J. (2023) · The Annals of Applied Probability · doi:10.1214/22-aap1806 · Cited by 133
An RUL prediction approach for lithium-ion battery based on SADE-MESN
Ji, Y., Chen, Z., Shen, Y., Yang, K., Wang, Y., & Cui, J. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107195 · Cited by 74
Implementing a Magnonic Reservoir Computer Model Based on Time-Delay Multiplexing
Watt, S., Kostylev, M., Ustinov, A. B., & Kalinikos, B. A. (2021) · Physical Review Applied · doi:10.1103/physrevapplied.15.064060 · Cited by 87
“A reservoir computer (RC) is a brain-inspired machine learning model which aims to simplify the construction and training of recurrent neural networks by outsourcing learning to a readout layer only [1, 2].”
BackgroundAhn, Y., & Kim, B. S. (2021) · Energy and Buildings · doi:10.1016/j.enbuild.2021.111717 · Cited by 80
Astronomia ex machina: a history, primer and outlook on neural networks in astronomy
Smith, M. J., & Geach, J. E. (2023) · Royal Society Open Science · doi:10.1098/rsos.221454 · Cited by 112
“First, Weddell and Webb (2008) successfully used an echo state network (Jaeger and Haas, 2004) to predict the point spread function of a target object in a wide field of view.”
MethodEvolino: hybrid neuroevolution / optimal linear search for sequence learning
Schmidhuber, J., Wierstra, D., & Gomez, F. J. (2005) · International Joint Conference on Artificial Intelligence · Cited by 87
An approach to reservoir computing design and training
Ferreira, A. A., Ludermir, T. B., & de Aquino, R. R. B. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.01.029 · Cited by 85
On Learning Navigation Behaviors for Small Mobile Robots With Reservoir Computing Architectures
Antonelo, E. A., & Schrauwen, B. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2323247 · Cited by 84
A comparative study of Reservoir Computing strategies for monthly time series prediction
wyffels, F., & Schrauwen, B. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.01.016 · Cited by 92
Neural networks in wireless networks: Techniques, applications and guidelines
Ahad, N., Qadir, J., & Ahsan, N. (2016) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2016.04.006 · Cited by 84
Metastable attractors explain the variable timing of stable behavioral action sequences
Recanatesi, S., Pereira, U., Murakami, M., Mainen, Z. F., & Mazzucato, L. (2021) · Neuron · doi:10.1016/j.neuron.2021.10.011 · Cited by 87
Patel, D., Canaday, D., Girvan, M., Pomerance, A., & Ott, E. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0042598 · Cited by 86
Memory in linear recurrent neural networks in continuous time
Hermans, M., & Schrauwen, B. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.08.008 · Cited by 75
“For instance speech recognition (Verstraeten et al., 2006), robotics applications (Antonelo et al., 2008), and for certain tasks (predicting chaotic time series) it outperforms all other state-of-the-art techniques (Jaeger and Haas, 2004).”
BackgroundExperimental reservoir computing using VCSEL polarization dynamics
Vatin, J., Rontani, D., & Sciamanna, M. (2019) · Optics Express · doi:10.1364/oe.27.018579 · Cited by 88
Temporal Pattern Recognition with Delayed-Feedback Spin-Torque Nano-Oscillators
Riou, M., Torrejón, J., Garitaine, B., Araujo, F. A., Bortolotti, P., Cros, V., Tsunegi, S., Yakushiji, K., Fukushima, A., Kubota, H., et al. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.12.024049 · Cited by 76
“III, we emulate a reservoir computer with a single, time-multiplexed oscillator [25], [26], and demonstrate the performance of the extrinsic memory by recognizing temporal patterns composed of random sine and square waveforms.”
MethodA novel photovoltaic power forecasting model based on echo state network
Yao, X., Wang, Z., & Zhang, H. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.10.022 · Cited by 74
Evolutionary aspects of reservoir computing
Seoane, L. F. (2019) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2018.0377 · Cited by 74
“Despite the simplicity of this method, RC-trained RNN have been robustly used for a plethora of tasks including data classification [40– 42], systems control [41, 43–45], time-series prediction [46, 47], uncovering grammar and other linguistic and speech features [41, 48–51] etc.”
MethodInformation Processing Capacity of Spin-Based Quantum Reservoir Computing Systems
Martínez‐Peña, R., Nokkala, J., Giorgi, G. L., Zambrini, R., & Soriano, M. C. (2020) · Cognitive Computation · doi:10.1007/s12559-020-09772-y · Cited by 121
“prediction [30,31,7].”
SuperMind: a survey of the potential of superconducting electronics for neuromorphic computing
Schneider, M. L., Toomey, E., Rowlands, G. E., Shainline, J. M., Tschirhart, P., & Segall, K. (2022) · Superconductor Science and Technology · doi:10.1088/1361-6668/ac4cd2 · Cited by 83
“RC emerged as an alternative training methodology for recurrent neural networks (RNNs) that are difficult to train using traditional back-propagation methods [112, 113].”
Quantum reservoir computing implementation on coherently coupled quantum oscillators
Dudas, J., Carles, B., Plouet, E., Mizrahi, F. A., Grollier, J., & Marković, D. (2023) · npj Quantum Information · doi:10.1038/s41534-023-00734-4 · Cited by 115
AI meets physics: a comprehensive survey
Jiao, L., Song, X., You, C., Liu, X., Li, L., Chen, P., Tang, X., Feng, Z., Liu, F., Guo, Y., et al. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10874-4 · Cited by 99
Minimal approach to neuro-inspired information processing
Soriano, M. C., Brunner, D., Escalona-Morán, M., Mirasso, C. R., & Fischer, I. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00068 · Cited by 84
“Almost at the same time, a similar approach, known as echo state networks, was independently developed by Jaeger and Haas (2004).”
BackgroundNonlinear Dynamic Boltzmann Machines for Time-Series Prediction
Dasgupta, S., & Osogami, T. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10806 · Cited by 71
“30th order nonlinear autoregressive moving average (NARMA): In this task, we train the RNN-G-DyBM for one step prediction with a one dimensional nonlinear timeseries.”
BackgroundMethodInfluentialHybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction
Xu, M., Han, M., Qiu, T., & Lin, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2825253 · Cited by 73
“For example, Jaeger and Haas [17] proposed a randomized algorithm for training neural networks, which is called echo state networks (ESNs).”
BackgroundZhang, H., & Vargas, D. V. (2023) · IEEE Access · doi:10.1109/access.2023.3299296 · Cited by 106
“As explained in section 1, our approach is comparable to a reservoir computing approach, which normally uses randomly coupled non-linear elements as a computational resource (Jaeger, 2002; Maass et al., 2002; Jaeger and Haas, 2004).”
BackgroundMethodInfluentialAkai‐Kasaya, M., Takeshima, Y., Kan, S., Nakajima, K., Oya, T., & Asai, T. (2021) · Neuromorphic Computing and Engineering · doi:10.1088/2634-4386/ac4339 · Cited by 72
Mengü, D., Rahman, M. S. S., Luo, Y., Li, J., Kulce, O., & Özcan, A. (2022) · Advances in Optics and Photonics · doi:10.1364/aop.450345 · Cited by 80
Sheng, C., Zhao, J., Wang, W., & Leung, H. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2250299 · Cited by 70
“Recently, the reservoir computing network (RCN) has demonstrated a good ability in describing the nonlinear or non-Gaussian system dynamics due to its special reservoir consisting of a large number of sparsely connected neurons [15], [16].”
BackgroundTrainable hardware for dynamical computing using error backpropagation through physical media
Hermans, M., Burm, M., Vaerenbergh, T. V., Dambre, J., & Bienstman, P. (2015) · Nature Communications · doi:10.1038/ncomms7729 · Cited by 72
Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.09249 · Cited by 138
“Those early CM systems, however, did not yet use powerful RNNs such as LSTM.”
ResultInfluentialStock market index prediction based on reservoir computing models
Wang, W., Tang, Y., Xiong, J. J., & Zhang, Y. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115022 · Cited by 83
Quantum Neuromorphic Computing with Reservoir Computing Networks
Ghosh, S., Nakajima, K., Krisnanda, T., Fujii, K., & Liew, T. C. H. (2021) · Advanced Quantum Technologies · doi:10.1002/qute.202100053 · Cited by 90
Nonlinear Spiking Neural Systems With Autapses for Predicting Chaotic Time Series
Liu, Q., Peng, H. J., Long, L., Wang, J., Yang, Q., Pérez-Jímenez, M. J., & Orellana-Martín, D. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3270873 · Cited by 69
“Fig.8(a) and (b) show the prediction curves and absolute error curves, respectively, of NSNP-AU, LSTM, GRU, and [32], OESN [46], SCR [47], DESN [48], RR-ESN [49], Lasso-ESN [50], and BPSO-ESN [51].”
BackgroundMethodLi, S., & Li, R. (2017) · Sustainability · doi:10.3390/su9071181 · Cited by 69
Nanoarchitectonic atomic switch networks for unconventional computing
Demis, E. C., Aguilera, R., Scharnhorst, K. S., Aono, M., Stieg, A. Z., & Gimzewski, J. K. (2016) · Japanese Journal of Applied Physics · doi:10.7567/jjap.55.1102b2 · Cited by 78
Deep Randomized Neural Networks
Gallicchio, C., & Scardapane, S. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-43883-8_3 · Cited by 103
Optimizing a quantum reservoir computer for time series prediction
Kutvonen, A., Fujii, K., & Sagawa, T. (2020) · Scientific Reports · doi:10.1038/s41598-020-71673-9 · Cited by 106
Performance boost of time-delay reservoir computing by non-resonant clock cycle
Stelzer, F., Röhm, A., Lüdge, K., & Yanchuk, S. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.01.010 · Cited by 72
Reservoir computing with diverse timescales for prediction of multiscale dynamics
Tanaka, G., MATSUMORI, T., Yoshida, H., & Aihara, K. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.l032014 · Cited by 82
Physical reservoir computing with FORCE learning in a living neuronal culture
Yada, Y., Shusaku, Y., & Takahashi, H. (2021) · Applied Physics Letters · doi:10.1063/5.0064771 · Cited by 72
Ensemble Deep Random Vector Functional Link Neural Network for Regression
Hu, M., Chion, J. H., Suganthan, P. N., & Katuwal, R. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3213628 · Cited by 68
“The number of neurons is chosen among the following values [32 , 256].”
BackgroundFeedback Control by Online Learning an Inverse Model
Waegeman, T., wyffels, F., & Schrauwen, B. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2208655 · Cited by 75
Peng, Y., Miao, L., Li, J., & Peng, X. (2012) · Neural Computing and Applications · doi:10.1007/s00521-012-1291-9 · Cited by 67
Synaptic information transfer in computer models of neocortical columns
Neymotin, S. A., Jacobs, K. M., Fenton, A. A., & Lytton, W. W. (2010) · Journal of Computational Neuroscience · doi:10.1007/s10827-010-0253-4 · Cited by 73
“4.6 Relation to echo state networks Recent work on echo state networks (ESNs, Jaeger and Haas 2004; Buonomano and Maass 2009; Buonomano 2009) showed that these networks do extremely well with time-series prediction.”
BackgroundMethodInfluentialTraining Echo State Networks with Regularization Through Dimensionality Reduction
Løkse, S., Bianchi, F. M., & Jenssen, R. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9450-z · Cited by 74
Gallicchio, C., & Micheli, A. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489464 · Cited by 65
“…(TESN) [27] is an efficient approach to the design of RecNN based on the Reservoir Computing (RC) paradigm [28] and on the Echo State Network (ESN) [29] model, whereas the recursive part is implemented by a randomized fixed-weight (obeying stability condition for their values) reservoir of units…”
BackgroundMethodFour-channels reservoir computing based on polarization dynamics in mutually coupled VCSELs system
Guo, X., Xiang, S., Zhang, Y. H., Lin, L., Wen, A. J., & Hao, Y. (2019) · Optics Express · doi:10.1364/oe.27.023293 · Cited by 70
Sunada, S., & Uchida, A. (2021) · Optica · doi:10.1364/optica.434918 · Cited by 77
Hybrid quantum-classical reservoir computing of thermal convection flow
Pfeffer, P., Heyder, F., & Schumacher, J. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.033176 · Cited by 99
“The reservoir state ψ t is updated as follows [26, 28, 35], see eq.”
BackgroundEffects of spectral radius and settling time in the performance of echo state networks
Venayagamoorthy, G. K., & Bashyal, S. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.021 · Cited by 66
Improved extreme learning machine for multivariate time series online sequential prediction
Wang, X., & Han, M. (2015) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.12.013 · Cited by 66
“Because of the universal approximation capability and distributed computing characteristic, neural networks have become one of the most influential prediction tools (Jaeger and Haas, 2004; Zemouri et al., 2003; Niska et al., 2004).”
BackgroundLocal Lyapunov exponents of deep echo state networks
Gallicchio, C., Micheli, A., & Silvestri, L. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.11.073 · Cited by 75
A survey of approaches for implementing optical neural networks
Xu, R., Lv, P., Xu, F., & Shi, Y. (2020) · Optics & Laser Technology · doi:10.1016/j.optlastec.2020.106787 · Cited by 78
Hebbian Learning in a Random Network Captures Selectivity Properties of the Prefrontal Cortex
Lindsay, G. W., Rigotti, M., Warden, M. R., Miller, E. K., & Fusi, S. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.1222-17.2017 · Cited by 70
Microfabricated Neuroaccelerometer: Integrating Sensing and Reservoir Computing in MEMS
Barazani, B., Dion, G., Morissette, J.-F., Beaudoin, L., & Sylvestre, J. (2020) · Journal of Microelectromechanical Systems · doi:10.1109/jmems.2020.2978467 · Cited by 74
“While sensory functions are implemented using fairly conventional MEMS designs, computing functions exploit the non-linear dynamics of a mechanical resonator in the MEMS, to implement a form of machine learning known as reservoir computing (RC) [1].”
MethodStock Market Prediction Using Machine Learning and Deep Learning Techniques: A Review
Saberironaghi, M., Ren, J., & Saberironaghi, A. (2025) · AppliedMath · doi:10.3390/appliedmath5030076 · Cited by 98
A novel model of leaky integrator echo state network for time-series prediction
Lun, S., Yao, X., Qi, H.-Y., & Hu, H. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.029 · Cited by 65
Xia, Y., Sun, C., & Zheng, W. X. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2184800 · Cited by 65
“Remark 2: There exist the echo state network (ESN) [36] and the extreme learning machine (ELM) algorithm [37].”
BackgroundFPGA-Based Stochastic Echo State Networks for Time-Series Forecasting
Alomar, M. L., Canals, V., Pérez-Mora, N., Martínez-Moll, V., & Rosselló, J. L. (2015) · Computational Intelligence and Neuroscience · doi:10.1155/2016/3917892 · Cited by 72
Large-Scale Spatiotemporal Photonic Reservoir Computer for Image Classification
Antonik, P., Marsal, N., & Rontani, D. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2924138 · Cited by 72
“Reservoir Computing (RC) is a Machine Learning paradigm, which includes echo-state networks (ESN) and liquid-state machines (LSM), to design and train artificial recurrent neural networks [11], [12].”
BackgroundMethodPhotonic reservoir computing based on nonlinear wave dynamics at microscale
Sunada, S., & Uchida, A. (2019) · Scientific Reports · doi:10.1038/s41598-019-55247-y · Cited by 83
“the microcavity laser is maximized around the edge of the phase transition, W∞/Wth ≈ 1.79. The decrease of NMCν is explained by the loss of the consistency [31], or the so-called echo-state property [2], which is an important condition for RC [29, 32]. In the multimode lasing regime, the spontaneous multi-modal oscill…”
BackgroundInfluentialPerspectives on adaptive dynamical systems
Sawicki, J., Berner, R., Loos, S. A. M., Anvari, M., Bader, R., Barfuß, W., Botta, N., Brede, N., Franović, I., Gauthier, D. J., et al. (2023) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0147231 · Cited by 84
Lv, S.-X., Peng, L., & Wang, L. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.08.024 · Cited by 67
Theory of Gating in Recurrent Neural Networks
Krishnamurthy, K., Can, T., & Schwab, D. J. (2022) · Physical Review X · doi:10.1103/physrevx.12.011011 · Cited by 93
Mustaqeem, M., Ishaq, M., & Kwon, S. (2022) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2022.102275 · Cited by 76
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.