Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
We present a method for learning nonlinear systems, echo state networks (ESNs). ESNs employ artificial recurrent neural networks in a way that has recently been proposed independently as a learning mechanism in biological brains. The learning method is computationally efficient and easy to use. On a benchmark task of predicting a chaotic time series, accuracy is improved by a factor of 2400 over previous techniques. The potential for engineering applications is illustrated by equalizing a communication channel, where the signal error rate is improved by two orders of magnitude.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although work in artificial intelligence and machine-learning research has shown that similar or even more complex outcomes can be achieved with a variety of techniques, including deep neural networks with trained weights (Gibney, 2015), reservoir networks with a trained output mapping (Maass et al., 2002;Jaeger & Haas, 2004), networks that are evolved using a genetic algorithm (Iizuka & Di Paolo,”
From A potential mechanism for Gibsonian resonance: behavioral entrainment emerges from local homeostasis in an unsupervised reservoir network (2024) · cites this paper as Jaeger & Haas, 2004
In the citing paper
“Therefore, a sequence production solely by a recurrent neural network can be unstable against small fluctuations in activities and unwanted noises (Jaeger and Haas, 2004).”
From The Cerebro-Cerebellum as a Locus of Forward Model: A Review (2020) · cites this paper as Jaeger and Haas, 2004
Citations
Cites
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Oscillation and Chaos in Physiological Control Systems
Mackey, M. C., & Glass, L. (1977) · Science · doi:10.1126/science.267326 · Cited by 4,135
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Real-time prediction of hand trajectory by ensembles of cortical neurons in primates
Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A., & Nicolelis, M. A. L. (2000) · Nature · doi:10.1038/35042582 · Cited by 1,487
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
Applying LSTM to Time Series Predictable through Time-Window Approaches
Gers, F. A., Eck, D., & Schmidhuber, J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44668-0_93 · Cited by 334
Computational neuroscience: a comprehensive approach
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5284 · Cited by 279
Research works printed but not linked: 10
- K. Doya, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 796–800.
- J. Vesanto in Proc. WSOM'97 (1997); www.cis.hut.fi/projects/monitor/publications/papers/wsom97.ps.
- L. Chudy, I. Farkas, Neural Network World8, 481 (1998).
- H. Bersini M. Birattari G. Bontempi in Proc. IEEE World Congr. on Computational Intelligence (IJCNN'98) (1997) pp. 2102–2106; ftp://iridia.ulb.ac.be/pub/lazy/papers/IridiaTr1997-13_2.ps.gz.
- V. J. Mathews, J. Lee, in Advanced Signal Processing: Algorithms, Architectures, and Implementations V (Proc. SPIE Vol. 2296), (SPIE, San Diego, CA, 1994), pp. 317–327.
- J. Hertzberg H. Jaeger F. Schönherr in Proc. 15th Europ. Conf. on Art. Int. (ECAI 02) F. van Harmelen Ed. (IOS Press Amsterdam 2002) pp. 708–712; www.ais.fhg.de/∼schoenhe/papers/ECAI02.pdf.
- H. Jaeger, in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, K. Obermayer, Eds. (MIT Press, Cambridge, MA, 2003) pp. 593–600.
- G. E. Hinton, in Parallel Models of Associative Memory, G. E. Hinton, J. A. Anderson, Eds. (Erlbaum, Hillsdale, NJ, 1981), pp. 161–187.
- M. Kawato, in The Handbook of Brain Theory and Neural Networks, M. Arbib, Ed. (MIT Press, Cambridge, MA, 1995), pp. 172–178.
- G. B. Stanley, Neurocomputing38–40, 1703 (2001).
Cited by
Items 501–600 of 1,009
An extended echo state network using Volterra filtering and principal component analysis
Boccato, L., Lopes, A., Attux, R., & Zuben, F. J. V. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.028 · Cited by 66
Data-driven based model for flow prediction of steam system in steel industry
Liu, Y., Liu, Q., Wei, W., Zhao, J., & Leung, H. (2012) · Information Sciences · doi:10.1016/j.ins.2011.12.031 · Cited by 65
“ln pðDja;bÞ 1⁄4 aE W bE MP D 1 2 lnðdet AÞ þ L 2 lnðaÞ þ n 2 lnðbÞ n 2 lnð2pÞ ð21Þ The optimal values of the hyper-parameters can be obtained by differentiating the derived log evidence, see [12] for more calculation details.”
Dynamical Basis of Irregular Spiking in NMDA-Driven Prefrontal Cortex Neurons
Durstewitz, D., & Gabriel, T. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhk044 · Cited by 65
“If PFC networks would reside in a close to or slightly chaotic regime with complex deterministic temporal structure, this may strongly facilitate the processing of temporal sequences (Bertschinger and Natschläger, 2004; Jaeger and Haas, 2004).”
BackgroundTraining enhanced reservoir computing predictor for landslide displacement
Yao, W., Zeng, Z., Lian, C., & Tang, H. (2014) · Engineering Geology · doi:10.1016/j.enggeo.2014.11.008 · Cited by 63
“Reservoir computing is a fairly effective technique for dynamic modeling, as demonstrated by numerical experiments (Jaeger and Haas, 2004).”
BackgroundMethodComputation by Switching in Complex Networks of States
Neves, F. S., & Timme, M. (2012) · Physical Review Letters · doi:10.1103/physrevlett.109.018701 · Cited by 65
Antonik, P., Haelterman, M., & Massar, S. (2017) · Physical Review Applied · doi:10.1103/physrevapplied.7.054014 · Cited by 56
“Following [1], we used the x -coordinate trajectory for training and testing, that we scaled by a factor of 0 .”
BackgroundMethodInfluentialReservoir Computing with Untrained Convolutional Neural Networks for Image Recognition
Tong, Z., & Tanaka, G. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545471 · Cited by 82
“The reservoir computing originates from the independently proposed models: the echo state networks (ESNs) [9], [10] and the liquid state machines [11], [12].”
BackgroundMethodKatumba, A., Yin, X. L., Dambre, J., & Bienstman, P. (2019) · Journal of Lightwave Technology · doi:10.1109/jlt.2019.2900568 · Cited by 70
A complete, parallel and autonomous photonic neural network in a semiconductor multimode laser
Porté, X., Skalli, A., Haghighi, N., Reitzenstein, S., Lott, J. A., & Brunner, D. (2021) · Journal of Physics Photonics · doi:10.1088/2515-7647/abf6bd · Cited by 85
“W in and W int do not partake in task-specific optimization, and we therefore implement a reservoir computer [17].”
BackgroundMethodCooperative Optimal Control of the Following Operation of High-Speed Trains
Liu, H., Yang, L., & Yang, H. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3163971 · Cited by 62
“[30], which consists of an input layer, reservoir, and output layer.”
BackgroundWang, S., Ma, H., Zhang, Y., Li, S., & He, W. (2023) · Energy · doi:10.1016/j.energy.2023.128984 · Cited by 68
A local Echo State Property through the largest Lyapunov exponent
Wainrib, G., & Galtier, M. N. (2016) · Neural Networks · doi:10.1016/j.neunet.2015.12.013 · Cited by 64
“Echo State Networks (ESN) are neural networks designed for performing complex non-linear regression or classification tasks, such as non-linear time-series forecasting [1, 2].”
BackgroundFeed forward neural network with random quaternionic neurons
Minemoto, T., Isokawa, T., Nishimura, H., & Matsui, N. (2016) · Signal Processing · doi:10.1016/j.sigpro.2016.11.008 · Cited by 62
Nonuniform State Space Reconstruction for Multivariate Chaotic Time Series
Han, M., Ren, W., Xu, M., & Qiu, T. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2816657 · Cited by 66
“The echo state network (ESN) is a commonly used recurrent neural network [52], popular in time series prediction and classification problems.”
BackgroundMinimum mean squared error time series classification using an echo state network prediction model
Skowronski, M. D., & Harris, J. G. (2006) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2006.1693294 · Cited by 64
“Nonlinear dynamic models have been applied to many problems, including system identification [1], [2], prediction [3], channel equalization [4], detection [5], and classification [6]–[8].”
Integrated photonic reservoir computing based on hierarchical time-multiplexing structure
Zhang, H., Feng, X., Li, B., Wang, Y., Cui, K., Liu, F., Dou, W., & Huang, Y. (2014) · Optics Express · doi:10.1364/oe.22.031356 · Cited by 61
Hinaut, X., Petit, M., Pointeau, G., & Dominey, P. F. (2014) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2014.00016 · Cited by 71
“Further development was realized in related systems including the Liquid State Machine (Maass et al., 2002), and Echo State Network (Jaeger, 2001; Jaeger and Haas, 2004).”
MethodUsing a Single Dendritic Neuron to Forecast Tourist Arrivals to Japan
Chen, W., Jian, S. U. N., Gao, S., Cheng, J., Wang, J., & Todo, Y. (2016) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2016edp7152 · Cited by 65
“However, the traditional recurrent ANN algorithms based on the gradient descent approach are well known for their slow convergence and high computational costs [41], thus it is di ffi cult to utilize them in actual applications.”
BackgroundLow-Loss Photonic Reservoir Computing with Multimode Photonic Integrated Circuits
Katumba, A., Heyvaert, J., Schneider, B., Uvin, S., Dambre, J., & Bienstman, P. (2018) · Scientific Reports · doi:10.1038/s41598-018-21011-x · Cited by 71
Reservoir computing with dipole-coupled nanomagnets
Nomura, H., Furuta, T., Tsujimoto, K., Kuwabiraki, Y., Peper, F., Tamura, E., Miwa, S., Goto, M., Nakatani, R., & Suzuki, Y. (2019) · Japanese Journal of Applied Physics · doi:10.7567/1347-4065/ab2406 · Cited by 65
Optimal modularity and memory capacity of neural reservoirs
Rodriguez, N., Izquierdo, E. J., & Ahn, Y. (2019) · Network Neuroscience · doi:10.1162/netn_a_00082 · Cited by 77
Quantized Minimum Error Entropy Criterion
Chen, B., Xing, L., Zheng, N., & Príncipe, J. C. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2868812 · Cited by 61
Reservoir computing model of two-dimensional turbulent convection
Pandey, S., & Schumacher, J. (2020) · Physical Review Fluids · doi:10.1103/physrevfluids.5.113506 · Cited by 63
Input-driven bifurcations and information processing capacity in spintronics reservoirs
Akashi, N., Yamaguchi, T., Tsunegi, S., Taniguchi, T., Nishida, M., Sakurai, R., Wakao, Y., & Nakajima, K. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.043303 · Cited by 63
Predicting amplitude death with machine learning
Xiao, R., Kong, L.-W., Sun, Z., & Lai, Y. (2021) · Physical review. E · doi:10.1103/physreve.104.014205 · Cited by 62
Embedding theory of reservoir computing and reducing reservoir network using time delays
Duan, X.-Y., Xiong, Y., Leng, S., Kurths, J., Lin, W. J., & Ma, H. (2023) · Physical Review Research · doi:10.1103/physrevresearch.5.l022041 · Cited by 71
Emotion recognition from speech signals via a probabilistic echo-state network
Trentin, E., Scherer, S. J., & Schwenker, F. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.10.015 · Cited by 65
Multi-step-ahead host load prediction using autoencoder and echo state networks in cloud computing
Yang, Q., Zhou, Y., Yu, Y., Yuan, J., Xing, X., & Du, S. (2015) · The Journal of Supercomputing · doi:10.1007/s11227-015-1426-8 · Cited by 63
“In our approach, we choose echo state network (ESN) as our basic prediction model, which is a kind of recurrent neural network (RNN) belonging to a collection of techniques called Reservoir Computing [13].”
BackgroundMethodInfluentialSong, Q., & Feng, Z. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.01.015 · Cited by 64
Information storage, loop motifs, and clustered structure in complex networks
Lizier, J. T., Atay, F. M., & Jost, J. (2012) · Physical Review E · doi:10.1103/physreve.86.026110 · Cited by 66
A new echo state network with variable memory length
Lun, S., Yao, X., & Hu, H. (2016) · Information Sciences · doi:10.1016/j.ins.2016.07.065 · Cited by 60
Bertschinger, N., Natschläger, T., & Legenstein, R. A. (2004) · neural information processing systems · Cited by 60
Adaptive feature fusion for time series classification
Wang, T., Liu, Z., Zhang, T., Hussain, S. F., Waqas, M. M., & Li, Y. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108459 · Cited by 62
Potential and limitations of quantum extreme learning machines
Innocenti, L., Lorenzo, S., Palmisano, I., Ferraro, A., Paternostro, M., & Palma, G. M. (2023) · Communications Physics · doi:10.1038/s42005-023-01233-w · Cited by 94
Convolutional Multitimescale Echo State Network
Ma, Q., Chen, E., Lin, Z., Yan, J., Yu, Z., & Ng, W. W. Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2919648 · Cited by 60
“THE ECHO state network (ESN) [1], [2] is one of the most well-known reservoir computing (RC) [3] models, which”
BackgroundMethodFabry-Perot Lasers as Enablers for Parallel Reservoir Computing
Bogris, A., Mesaritakis, C., Deligiannidis, S., & Li, P. (2020) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2020.3011879 · Cited by 61
Coupled Nonlinear Delay Systems as Deep Convolutional Neural Networks
Penkovsky, B., Porté, X., Jacquot, M., Larger, L., & Brunner, D. (2019) · Physical Review Letters · doi:10.1103/physrevlett.123.054101 · Cited by 69
Parallel Machine Learning for Forecasting the Dynamics of Complex Networks
Srinivasan, K., Coble, N. J., Hamlin, J., Antonsen, T. M., Ott, E., & Girvan, M. (2022) · Physical Review Letters · doi:10.1103/physrevlett.128.164101 · Cited by 65
Neurocomputational models of interval and pattern timing
Hardy, N. F., & Buonomano, D. V. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.01.012 · Cited by 90
Neuromorphic Computing in Ginzburg-Landau Polariton-Lattice Systems
Opala, A., Ghosh, S., Liew, T. C. H., & Matuszewski, M. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.11.064029 · Cited by 66
A long short-term memory embedding for hybrid uplifted reduced order models
Ahmed, S. E., San, O., Rasheed, A., & Iliescu, T. (2020) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2020.132471 · Cited by 60
“In particular, LSTMs have shown great success in learning maps of sequential data and time-series [62–66].”
BackgroundA systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics
Platt, J. A., Penny, S. G., Smith, T. A., Chen, T., & Abarbanel, H. D. I. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.06.025 · Cited by 92
DePasquale, B., Sussillo, D., Abbott, L. F., & Churchland, M. M. (2023) · Neuron · doi:10.1016/j.neuron.2022.12.007 · Cited by 71
2D-Material-Based Volatile and Nonvolatile Memristive Devices for Neuromorphic Computing
Xia, X., Huang, W., Hang, P., Guo, T., Yan, Y., Yang, J., Yang, D., Yu, X., & Li, X. (2023) · ACS Materials Letters · doi:10.1021/acsmaterialslett.2c01026 · Cited by 59
Chemical reservoir computation in a self-organizing reaction network
Baltussen, M. G., de Jong, T. J., Duez, Q., Robinson, W. E., & Huck, W. T. S. (2024) · Nature · doi:10.1038/s41586-024-07567-x · Cited by 84
“In the considered example, the authors assumed that the Lorentz attractor describes changes in u(t), and they focused their attention on restoring this input.”
Improving liquid state machines through iterative refinement of the reservoir
Norton, D. P., & Ventura, D. A. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.08.005 · Cited by 67
“Follow this and additional works at: https://scholarsarchive.byu.edu/etd Part of the Computer Sciences Commons”
BackgroundModeling systems with internal state using evolino
Wierstra, D., Gomez, F. J., & Schmidhuber, J. (2005) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1068009.1068315 · Cited by 68
“This simple approach is currently the title holder in the Mackey-Glass time-series benchmark, improving on the accuracy of all other methods by as much as three orders of magnitude [10].”
BackgroundMethodAdvanced X-ray CT scanning can boost tree ring research for earth system sciences
den Bulcke, J. V., Boone, M. A., Dhaene, J., Loo, D. V., Hoorebeke, L. V., Boone, M., wyffels, F., Beeckman, H., Acker, J. V., & Mil, T. D. (2019) · Annals of Botany · doi:10.1093/aob/mcz126 · Cited by 61
Deep Temporal Convolution Network for Time Series Classification
Koh, B. H. D., Lim, C. L. P., Rahimi, H., Woo, W. L., & Gao, B. (2021) · Sensors · doi:10.3390/s21020603 · Cited by 63
“Other networks that have been proposed for the time series classification include the fully convolutional networks (FCN) [27], multichannel deep convolution neural network (MC-DCNN) [28], residual network (ResNet) [29], and the echo state network (ESN) [30].”
BackgroundLong, L., Liu, Q., Peng, H. J., Wang, J., & Yang, Q. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.04.030 · Cited by 58
Continual Deep Learning for Time Series Modeling
Ao, S.-I., & Fayek, H. M. (2023) · Sensors · doi:10.3390/s23167167 · Cited by 76
“Jaeger and Haas [50] invented the ESN in 2004 for time series forecasting for wireless communication tasks.”
BackgroundArtificial Neural Network Models
Peter Tiňo, B., Beňušková, Ľ., & Sperduti, A. (2015) · Springer eBooks · doi:10.1007/978-3-662-43505-2_27 · Cited by 61
Reducing the phase sensitivity of laser-based optical reservoir computing systems
Nguimdo, R. M., Verschaffelt, G., Danckaert, J., & der Sande, G. V. (2016) · Optics Express · doi:10.1364/oe.24.001238 · Cited by 57
Röhm, A., Gauthier, D. J., & Fischer, I. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0065813 · Cited by 61
Echo State Networks for Practical Nonlinear Model Predictive Control of Unknown Dynamic Systems
Jordanou, J. P., Antonelo, E. A., & Camponogara, E. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3136357 · Cited by 59
Koryakin, D., Lohmann, J., & Butz, M. V. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.08.008 · Cited by 65
Reservoir Computing as a Model for In-Materio Computing
Dale, M., Miller, J. F., & Stepney, S. (2016) · Emergence, complexity and computation · doi:10.1007/978-3-319-33924-5_22 · Cited by 61
“Mackey-Glass chaotic time-series prediction is another common benchmark, where the system is trained to predict one time-step into the future (for examples see [44, 49, 116]).”
BackgroundNeural Decomposition of Time-Series Data for Effective Generalization
Godfrey, L. B., & Gashler, M. S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2709324 · Cited by 72
“Echo state networks (ESNs) have also performed particularly well at this task [26], [27], [28].”
BackgroundNguimdo, R. M., Lacot, É., Jacquin, O., Hugon, O., der Sande, G. V., & de Chatellus, H. G. (2017) · Optics Letters · doi:10.1364/ol.42.000375 · Cited by 55
Online adaptive dynamic programming based on echo state networks for dissolved oxygen control
Bo, Y.-C., & Zhang, X. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.09.015 · Cited by 55
Model Predictive Control Design for Dynamical Systems Learned by Echo State Networks
Armenio, L. B., Terzi, E., Farina, M., & Scattolini, R. (2019) · IEEE Control Systems Letters · doi:10.1109/lcsys.2019.2920720 · Cited by 55
“ESNs have been already explored with significant results in speech recognition [2], time-series prediction [6], reinforcement learning [8], and language modelling [15].”
BackgroundPhotonic Reservoir Computing: Optical Recurrent Neural Networks
der Sande, G. V., & Soriano, M. C. (2019) · De Gruyter eBooks · doi:10.1515/9783110583496 · Cited by 63
Using echo state networks for classification: A case study in Parkinson's disease diagnosis
Lacy, S. E., Smith, S. L., & Lones, M. A. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.02.002 · Cited by 55
“In particular they have demonstrated a strong ability to forecast chaotic time series, with considerable success on the well-known Mackey-Glass benchmark equation [13, 9].”
BackgroundGottwald, G. A., & Reich, S. (2021) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0066080 · Cited by 71
“…in so called echo-state networks and reservoir computers with remarkable success in forecasting dynamical systems (Maass et al., 2002; Jaeger, 2002; Jaeger and Haas, 2004; Pathak et al., 2018a; J¨ungling et al., 2019; Algar et al., 2019; Nadiga, 2020; Bollt, 2021; Gauthier et al., 2021; Platt et…”
Fundamental physics and applications of skyrmions: A review
Wang, K., Bheemarasetty, V. S., Duan, J., Zhou, S., & Xiao, G. (2022) · Journal of Magnetism and Magnetic Materials · doi:10.1016/j.jmmm.2022.169905 · Cited by 67
Reservoir computing based on electric-double-layer coupled InGaZnO artificial synapse
Yang, Y., Cui, H., Ke, S., Pei, M., Shi, K., Wan, C., & Wan, Q. (2023) · Applied Physics Letters · doi:10.1063/5.0137647 · Cited by 56
A reservoir of foraging decision variables in the mouse brain
Cazettes, F., Mazzucato, L., Murakami, M., Morais, J. P., Augusto, E., Renart, A., & Mainen, Z. F. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01305-8 · Cited by 65
Predicting turbulent dynamics with the convolutional autoencoder echo state network
Racca, A., Doan, N. A. K., & Magri, L. (2023) · Journal of Fluid Mechanics · doi:10.1017/jfm.2023.716 · Cited by 73
Reinforcement Learning with Echo State Networks
Szita, I., Gyenes, V., & Lörincz, A. (2006) · Lecture notes in computer science · doi:10.1007/11840817_86 · Cited by 64
Spatio-Temporal Interpolated Echo State Network for Meteorological Series Prediction
Xu, M., Yang, Y., Han, M., Qiu, T., & Lin, H. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869131 · Cited by 55
“Fortunately, ESNs are proposed by Jaeger and Haas [20], in the year of 2001, for the purpose of chaotic attractor modeling.”
BackgroundA model of temporal scaling correctly predicts that motor timing improves with speed
Hardy, N. F., Goudar, V., Romero-Sosa, J. L., & Buonomano, D. V. (2018) · Nature Communications · doi:10.1038/s41467-018-07161-6 · Cited by 64
“Wealsotrained three Echo State Networks 26,27 (ESNs) to produce a sinusoidal outputs (ESN ’ s are not well-suited to produce long aperiodic patterns) at three different frequencies: 5, 10, and 15 Hz.”
MethodBai, K., & Yi, Y. (2018) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3264659 · Cited by 63
“The performance metrics of reservoir computing outperforms conventional RNNs by three orders and two orders of magnitude more accurate in chaotic dynamic predictions and nonlinear channel equalization, respectively (Jaeger and Haas 2004; Lukoševicius 2012).”
MethodWang, Z., Zeng, Y., Wang, S., & Wang, L. (2019) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2019.02.009 · Cited by 55
Delayed dynamical systems: networks, chimeras and reservoir computing
Hart, J. D., Larger, L., Murphy, T. E., & Roy, R. (2019) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2018.0123 · Cited by 68
“RC has already achieved many successes, revealing its computational potential both in ESN numerical simulations [70,71], and also in physical hardware implementation.”
BackgroundTan, X.-S., Hou, Y., Wu, Z.-M., & Xia, G.-Q. (2019) · Optics Express · doi:10.1364/oe.27.026070 · Cited by 56
Ros, F. D., Ranzini, S. M., Bülow, H., & Zibar, D. (2020) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2020.2975607 · Cited by 60
“of recent works, especially for a nonlinear wireless transmission [33], [38].”
BackgroundMulti-reservoir echo state networks with sequence resampling for nonlinear time-series prediction
Li, Z., & Tanaka, G. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.122 · Cited by 65
Principles of Neuromorphic Photonics
Shastri, B. J., Tait, A. N., de Lima, T. F., Nahmias, M. A., Peng, H.-T., & Prucnal, P. R. (2018) · Springer eBooks · doi:10.1007/978-1-4939-6883-1_702 · Cited by 57
Spatiotemporal features for asynchronous event-based data
Lagorce, X., Ieng, S.-H., Clady, X., Pfeiffer, M., & Benosman, R. B. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00046 · Cited by 60
“As suggested in Jaeger and Haas (2004), training of the read-out weights W k out can be achieved with a standard recursive least squares algorithm (here a version described in Farhang-Boroujeny (2013) was used).”
MethodThe architecture of dynamic reservoir in the echo state network
Cui, H., Liu, X., & Li, L. (2012) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.4746765 · Cited by 59
Angelov, P. P., & Sperduti, A. (2016) · Lancaster EPrints (Lancaster University) · Cited by 60
Bird species identification via transfer learning from music genres
Ntalampiras, S. (2018) · Ecological Informatics · doi:10.1016/j.ecoinf.2018.01.006 · Cited by 56
Dynamical Learning of Dynamics
Klos, C., Kossio, Y. F. K., Goedeke, S., Gilra, A., & Memmesheimer, R.-M. (2020) · Physical Review Letters · doi:10.1103/physrevlett.125.088103 · Cited by 61
“They are continually fed back to the network, allowing their autonomous generation [31].”
BackgroundMethodHua, Z., Zheng, Z., Pahon, E., Péra, M., & Gao, F. (2021) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2021.3121179 · Cited by 52
“Consequently, a new paradigm known as ESN has been proposed, and the hidden layers of RNN are replaced by a reservoir that is built randomly and left untrained [32].”
BackgroundA Multi-scale Approach to Gesture Detection and Recognition
Neverova, N., Wolf, C., Paci, G., Sommavilla, G., Taylor, G. W., & Nebout, F. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.69 · Cited by 52
“To address this issue, several techniques such as Long Short-term Memory RNN (LSTM-RNN) [10] and EchoState Networks (ESN) [12] have been proposed.”
MethodLiu, L., Wang, Z., Yao, X., & Zhang, H. (2018) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2018.2817495 · Cited by 52
“As a result, a mass of researchers are committed to studying the ESNs [63], [65], [68].”
BackgroundA Distributed Algorithm for the Cooperative Prediction of Power Production in PV Plants
Rosato, A., Panella, M., & Araneo, R. (2019) · IEEE Transactions on Energy Conversion · doi:10.1109/tec.2018.2873009 · Cited by 52
“time series prediction [44], grammatical inference [45], stock price prediction [46], among others.”
BackgroundOnline sequential echo state network with sparse RLS algorithm for time series prediction
Yang, C., Qiao, J., Ahmad, Z., Nie, K., & Wang, L. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.05.006 · Cited by 52
Classification of time-domain waveforms using a speckle-based optical reservoir computer
Paudel, U., Luengo-Kovac, M., Pilawa, J., Shaw, T. J., & Valley, G. C. (2019) · Optics Express · doi:10.1364/oe.379264 · Cited by 61
Han, M., Li, W., Feng, S., Qiu, T., & Chen, C. L. P. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3004253 · Cited by 52
“We intend to simulate the chaotic characteristic by ESN [28].”
MethodLong, L., Lugu, R., Xiong, X., Liu, Q., Peng, H. J., Wang, J., Orellana-Martín, D., & Pérez-Jímenez, M. J. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109568 · Cited by 51
Millimeter-Level Pick and Peg-in-Hole Task Achieved by Aerial Manipulator
Wang, M., Chen, Z., Guo, K., Yu, X. Y., Zhang, Y. M., Guo, L., & Wang, W. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3338956 · Cited by 74
“One of the popular methods is the least mean squares (LMS) algorithm [49].”
BackgroundMethodWater Inflow Forecasting using the Echo State Network: a Brazilian Case Study
Sacchi, R., Ozturk, M. C., Príncipe, J. C., Carneiro, A. A. F. M., & da Silva, I. N. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4371334 · Cited by 51
“Recently, a new utilization of recurrent network topologies that aims at addressing the difficulties in RNN training has been introduced by Jaeger under the name of echo state networks (ESN) [1][7].”
BackgroundNanophotonic Reservoir Computing With Photonic Crystal Cavities to Generate Periodic Patterns
Fiers, M., Vaerenbergh, T. V., wyffels, F., Verstraeten, D., Schrauwen, B., Dambre, J., & Bienstman, P. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2274670 · Cited by 54
“…and P. Bienstman are with the Department of Information Technology of Ghent University, Ghent 9000, Belgium, and also with the Center for Nano and Biophotonics, Ghent University, Ghent B-9000, Belgium (e-mail: martin.fiers@intec.ugent.be; thomas.vanvaerenbergh@intec.ugent.be;…”
BackgroundAutomatic identification of integrity attacks in cyber-physical systems
Ntalampiras, S. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.04.006 · Cited by 57
Neural networks within multi-core optic fibers
Cohen, E., Malka, D., Shemer, A., Shahmoon, A., Zalevsky, Z., & London, M. (2016) · Scientific Reports · doi:10.1038/srep29080 · Cited by 51
A novel echo state network for multivariate and nonlinear time series prediction
Shen, L., Chen, J., Zeng, Z., Yang, J., & Jin, J. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.10.038 · Cited by 51
PAM-4 Transmission at 1550 nm Using Photonic Reservoir Computing Post-Processing
Argyris, A., Bueno, J., & Fischer, I. (2019) · IEEE Access · doi:10.1109/access.2019.2905422 · Cited by 51
“Hardware-friendly ML techniques, such as RC [21], represent attractive alternatives to the conventional ML approaches.”
MethodEvaluation of memory capacity of spin torque oscillator for recurrent neural networks
Tsunegi, S., Taniguchi, T., Miwa, S., Nakajima, K., Yakushiji, K., Fukushima, A., Yuasa, S., & Kubota, H. (2018) · Japanese Journal of Applied Physics · doi:10.7567/jjap.57.120307 · Cited by 56
Broad echo state network for multivariate time series prediction
Yao, X., & Wang, Z. (2019) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2019.01.027 · Cited by 55
Metadata
- Authors
- Jaeger, H., & Haas, H.
- Affiliations
- Constructor University (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Science
- Published
- 2004-04-02
- Volume
- 304
- Issue
- 5667
- Pages
- 78-80
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2118706537
- Cited by
- 3,811
- References
- 18
- Citations by year
- 2026: 257 · 2025: 314 · 2024: 315 · 2023: 332 · 2022: 300 · 2021: 355 · 2020: 259 · 2019: 309 · 2018: 237 · 2017: 160 · 2016: 142 · 2015: 128 · 2014: 120 · 2013: 107 · 2012: 100
- FWCI
- 33.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Chaos control and synchronization
- Keywords
- echo state networks, chaotic time series prediction, recurrent neural networks, channel equalization, wireless communication, energy efficiency, reservoir computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.