Optimization and applications of echo state networks with leaky- integrator neurons
Abstract
Standard echo state networks (ESNs) are built from simple additive units with a sigmoid activation function. Here we investigate ESNs whose reservoir units are leaky integrator units. Units of this type have individual state dynamics, which can be exploited in various ways to accommodate the network to the temporal characteristics of a learning task. We present stability conditions, introduce and investigate a stochastic gradient descent method for the optimization of the global learning parameters (input and output feedback scalings, leaking rate, spectral radius) and demonstrate the usefulness of leaky-integrator ESNs for (i) learning very slow dynamic systems and replaying the learnt system at different speeds, (ii) classifying relatively slow and noisy time series (the Japanese Vowel dataset--here we obtain a zero test error rate), and (iii) recognizing strongly time-warped dynamic patterns.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A randomly connected circuit model, akin to a 'reservoir' network, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or 'edge of chaos').
“Finally, we show that a randomly connected circuit model, akin to a “reservoir” network13-15, can reproduce the observed distribution of timescales, provided that the network operates at the critical point (or “edge of chaos”)16-18.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
Reservoir models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.
“Those models predict that the memory of input signals is stored in a large, recurrent and heterogeneous network (reservoir), in a distributed manner, and that a desired output is obtained by a trainable combination of the response signals in the reservoir.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning, but experimental support that such networks are adopted by the brain has been lacking.
“Reservoir-type networks have been the subject of active research in computational neuroscience and machine learning13-15, but so far experimental support that such networks are adopted by the brain has been lacking.”
From A reservoir of time constants for memory traces in cortical neurons (2011) · cites this paper as [14]
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · see citing works
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,312 · see citing works
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · see citing works
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · see citing works
Self-citationMinimum prediction residual principle applied to speech recognition
Itakura, F. (1975) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1975.1162641 · Cited by 1,620 · see citing works
The neural basis of temporal processing
Mauk, M. D., & Buonomano, D. V. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144247 · Cited by 976 · see citing works
Adaptive Filters Theory and Applications
Farhang‐Boroujeny, B. (1999) · doi:10.1002/9781118591352 · Cited by 965 · see citing works
Gradient calculations for dynamic recurrent neural networks: a survey
Pearlmutter, B. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.410363 · Cited by 578 · see citing works
Readings in Speech Recognition
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-27652-5 · Cited by 562 · see citing works
The combining classifier: to train or not to train?
Duin, R. P. W. (2003) · doi:10.1109/icpr.2002.1048415 · Cited by 340 · see citing works
Analysis and Design of Echo State Networks
Ozturk, M. C., Xu, D., & Príncipe, J. C. (2006) · Neural Computation · doi:10.1162/neco.2007.19.1.111 · Cited by 311 · see citing works
Pattern Extraction for Time Series Classification
Geurts, P. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44794-6_10 · Cited by 296 · see citing works
Training Recurrent Networks by Evolino
Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. J. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.757 · Cited by 248 · see citing works
Computational Aspects of Feedback in Neural Circuits
Maass, W., Joshi, P., & Sontag, E. D. (2005) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0020165 · Cited by 241 · see citing works
Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus
Stanley, G. B., Li, F. F., & Dan, Y. (1999) · Journal of Neuroscience · doi:10.1523/jneurosci.19-18-08036.1999 · Cited by 240 · see citing works
Short term memory in echo state networks
Jaeger, H. (2022) · Fraunhofer-Publica (Fraunhofer-Gesellschaft) · doi:10.24406/publica-fhg-291107 · Cited by 197 · see citing works
Self-citationA tighter bound for the echo state property
Buehner, M. R., & Young, P. M. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.872357 · Cited by 154 · see citing works
Multidimensional curve classification using passing-through regions
Kudo, M., Toyama, J., & Shimbo, M. (1999) · Pattern Recognition Letters · doi:10.1016/s0167-8655(99)00077-x · Cited by 139 · see citing works
Jaeger, H. (2002) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 99 · see citing works
Self-citationThe Case for an Internal Dynamics Model versus Equilibrium Point Control in Human Movement
Hinder, M. R., & Milner, T. E. (2003) · The Journal of Physiology · doi:10.1113/jphysiol.2002.033845 · Cited by 96 · see citing works
A Learning Rule for the Emergence of Stable Dynamics and Timing in Recurrent Networks
Buonomano, D. V. (2005) · Journal of Neurophysiology · doi:10.1152/jn.01250.2004 · Cited by 81 · see citing works
Analyzing the weight dynamics of recurrent learning algorithms
Schiller, U., & Steil, J. J. (2004) · Neurocomputing · doi:10.1016/j.neucom.2004.04.006 · Cited by 53 · see citing works
Trajectory generation and modulation using dynamic neural networks
Zegers, P., & Sundareshan, M. K. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.810603 · Cited by 39 · see citing works
Time Warping Invariant Echo State Networks
Lukoševičius, M., Popovici, D., Jaeger, H., & Siewert, U. (2006) · Cited by 38 · see citing works
Self-citationTime Warping Invariant Neural Networks
Sun, G.-Z., Chen, H.-H., & Lee, Y.-C. (1992) · neural information processing systems · Cited by 17 · see citing works
Finding good Echo State Networks to control an underwater robot using evolutionary computations
van der Zant, T., Bečanović, V., Ishii, K., Kobialka, H., & Plöger, P. G. (2004) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)31978-x · Cited by 14 · see citing works
Dynamic Bayesian Networks with Deterministic Latent Tables
Barber, D. (2002) · neural information processing systems · Cited by 7 · see citing works
Self-Organizing Neural Networks for Sequence Processing
Strickert, M. (2005) · Cited by 5 · see citing works
Time-Warping Network: A Hybrid Framework for Speech Recognition
Levin, E., Pieraccini, R., & Bocchieri, E. (1991) · neural information processing systems · Cited by 5 · see citing works
Finding good echo state networks to control an underwater robot using evolutionary computations
Zant, T. V. D., Becanovic, V., Ishii, K., et al. (2004) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 1 · see citing works
Flagged first, then by citations
1 of the 32 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Jaeger, 2003, Adaptive nonlinear system identification with echo state networks, Vol. 15, 593
- Levin, 1992, Vol. 4, 151
- Sun, 1993, Vol. 5, 180
Cited by
Sorted by citations
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956 · see citing works
Self-citationRecurrent Neural Network RegularizationPreprint
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv.org · doi:10.48550/arxiv.1409.2329 · Cited by 3,086 · see citing works
Information processing using a single dynamical node as complex system
Appeltant, L., Soriano, M. C., der Sande, G. V., Danckaert, J., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., & Fischer, I. (2011) · Nature Communications · doi:10.1038/ncomms1476 · Cited by 1,719 · see citing works
Ghosh‐Dastidar, S., & Adeli, H. (2009) · International Journal of Neural Systems · doi:10.1142/s0129065709002002 · Cited by 1,066 · see citing works
State-dependent computations: spatiotemporal processing in cortical networks
Buonomano, D. V., & Maass, W. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2558 · Cited by 919 · see citing works
A Practical Guide to Applying Echo State Networks
Lukoševičius, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_36 · Cited by 880 · see citing works
Self-citationOptoelectronic Reservoir Computing
Paquot, Y., Duport, F., Smerieri, A., Dambre, J., Schrauwen, B., Haelterman, M., & Massar, S. (2012) · Scientific Reports · doi:10.1038/srep00287 · Cited by 866 · see citing works
Minimum Complexity Echo State Network
Rodan, A., & Peter Tiňo, B. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2089641 · Cited by 710 · see citing works
All-optical reservoir computing
Duport, F., Schneider, B., Smerieri, A., Haelterman, M., & Massar, S. (2012) · Optics Express · doi:10.1364/oe.20.022783 · Cited by 527 · see citing works
Deep reservoir computing: A critical experimental analysis
Gallicchio, C., Micheli, A., & Pedrelli, L. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.089 · Cited by 513 · see citing works
Re-visiting the echo state property
Yildiz, I. B., Jaeger, H., & Kiebel, S. J. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.07.005 · Cited by 495 · see citing works
Self-citationAdvances in optimizing recurrent networks
Bengio, Y., Boulanger-Lewandowski, N., & Pascanu, R. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639349 · Cited by 478 · see citing works
Jaeger, H. (2007) · Scholarpedia · doi:10.4249/scholarpedia.2330 · Cited by 459 · see citing works
Self-citationInterference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach
Challita, U., Saad, W., & Bettstetter, C. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2900035 · Cited by 441 · see citing works
Lukoševičius, M., Jaeger, H., & Schrauwen, B. (2012) · Künstliche Intell. · doi:10.1007/s13218-012-0204-5 · Cited by 387 · see citing works
Self-citationAn overview of reservoir computing: theory, applications and implementations
Schrauwen, B., Verstraeten, D., & Campenhout, J. V. (2007) · Ghent University Academic Bibliography (Ghent University) · Cited by 369 · see citing works
Randomness in neural networks: an overview
Scardapane, S., & Wang, D. (2017) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1200 · Cited by 355 · see citing works
Reservoir observers: Model-free inference of unmeasured variables in chaotic systems
Lu, Z., Pathak, J., Hunt, B. R., Girvan, M., Brockett, R. W., & Ott, E. (2017) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.4979665 · Cited by 340 · see citing works
Information processing via physical soft body
Nakajima, K., Hauser, H., Li, T., & Pfeifer, R. (2015) · Scientific Reports · doi:10.1038/srep10487 · Cited by 324 · see citing works
Learning to ExecutePreprint
Zaremba, W., & Sutskever, I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.4615 · Cited by 314 · see citing works
Analysis and Design of Echo State Networks
Ozturk, M. C., Xu, D., & Príncipe, J. C. (2006) · Neural Computation · doi:10.1162/neco.2007.19.1.111 · Cited by 311 · see citing works
A reservoir of time constants for memory traces in cortical neurons
Bernacchia, A., Seo, H., Lee, D., & Wang, X.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2752 · Cited by 292 · see citing works
Computing with Spiking Neuron Networks
Paugam‐Moisy, H., & Bohté, S. M. (2012) · Springer eBooks · doi:10.1007/978-3-540-92910-9_10 · Cited by 283 · see citing works
Design of deep echo state networks
Gallicchio, C., Micheli, A., & Pedrelli, L. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.08.002 · Cited by 262 · see citing works
Training Recurrent Networks by Evolino
Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. J. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.757 · Cited by 248 · see citing works
Han, S.-I., & Lee, J. M. (2013) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2013.2253072 · Cited by 241 · see citing works
Dong, S. S., & Li, Z. (2021) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2021.114129 · Cited by 226 · see citing works
Manjunath, G. P., & Jaeger, H. (2012) · Neural Computation · doi:10.1162/neco_a_00411 · Cited by 218 · see citing works
Self-citationVedmedenko, E. Y., Kawakami, R. K., Sheka, D. D., Gambardella, P., Kirilyuk, A. I., Hirohata, A., Binek, C., Chubykalo‐Fesenko, O., Sanvito, S., Kirby, B. J., et al. (2020) · Journal of Physics D Applied Physics · doi:10.1088/1361-6463/ab9d98 · Cited by 214 · see citing works
Fully analogue photonic reservoir computer
Duport, F., Smerieri, A., Akrout, A., Haelterman, M., & Massar, S. (2016) · Scientific Reports · doi:10.1038/srep22381 · Cited by 203 · see citing works
Short-term stock price prediction based on echo state networks
Lin, X., Yang, Z., & Song, Y. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.09.049 · Cited by 202 · see citing works
Learning Longer Memory in Recurrent Neural NetworksPreprint
Mikolov, T., Joulin, A., Chopra, S., Mathieu, M., & Ranzato, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.7753 · Cited by 200 · see citing works
Growing Echo-State Network With Multiple Subreservoirs
Qiao, J., Li, F., Han, H., & Li, W. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514275 · Cited by 190 · see citing works
Liu, D., Wang, H., Peng, Y., Xie, W. H., & Liao, H. (2013) · Energies · doi:10.3390/en6083654 · Cited by 186 · see citing works
The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition
Schrauwen, B., Defour, J., Verstraeten, D., & Campenhout, J. V. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74690-4_48 · Cited by 181 · see citing works
Connectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons
Buesing, L., Schrauwen, B., & Legenstein, R. A. (2009) · Neural Computation · doi:10.1162/neco.2009.01-09-947 · Cited by 176 · see citing works
An Integrated Probabilistic Approach to Lithium-Ion Battery Remaining Useful Life Estimation
Liu, D., Xie, W. H., Liao, H., & Peng, Y. (2014) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2014.2348613 · Cited by 171 · see citing works
Reservoir computing and extreme learning machines for non-linear time-series data analysis
Butcher, J. B., Verstraeten, D., Schrauwen, B., Day, C. R., & Haycock, P. W. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.11.011 · Cited by 169 · see citing works
Reservoir Computing with Random Skyrmion Textures
Pinna, D., Bourianoff, G. I., & Everschor‐Sitte, K. (2020) · Physical Review Applied · doi:10.1103/physrevapplied.14.054020 · Cited by 168 · see citing works
Host load prediction with long short-term memory in cloud computing
Song, B., Yu, Y., Zhou, Y., Wang, Z., & Du, S. (2017) · The Journal of Supercomputing · doi:10.1007/s11227-017-2044-4 · Cited by 166 · see citing works
Echo State Property of Deep Reservoir Computing Networks
Gallicchio, C., & Micheli, A. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9461-9 · Cited by 163 · see citing works
Chen, Q., Shi, H., & Sun, M. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2931877 · Cited by 165 · see citing works
Observing spatio-temporal dynamics of excitable media using reservoir computing
Zimmermann, R. S., & Parlitz, U. (2018) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5022276 · Cited by 157 · see citing works
Remaining useful life prediction of PEMFC systems under dynamic operating conditions
Hua, Z., Zheng, Z., Pahon, E., Péra, M., & Gao, F. (2021) · Energy Conversion and Management · doi:10.1016/j.enconman.2021.113825 · Cited by 160 · see citing works
RISE-Based Asymptotic Prescribed Performance Tracking Control of Nonlinear Servo Mechanisms
Wang, S., Na, J., & Ren, X. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2769683 · Cited by 159 · see citing works
Forecasting chaotic systems with very low connectivity reservoir computers
Griffith, A., Pomerance, A., & Gauthier, D. J. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120710 · Cited by 154 · see citing works
Human activity recognition using multisensor data fusion based on Reservoir Computing
Palumbo, F., Gallicchio, C., Pucci, R., & Micheli, A. (2016) · Journal of Ambient Intelligence and Smart Environments · doi:10.3233/ais-160372 · Cited by 157 · see citing works
Phoneme Recognition with Large Hierarchical Reservoirs
Triefenbach, F., Jalalvand, A., Schrauwen, B., & Martens, J. (2010) · Cited by 150 · see citing works
The promise of spintronics for unconventional computing
Finocchio, G., Di Ventra, M., Camsari, K. Y., Everschor-Sitte, K., Khalili Amiri, P., & Zeng, Z. (2021) · Journal of Magnetism and Magnetic Materials · doi:10.1016/j.jmmm.2020.167506 · Cited by 173 · see citing works
Deep Reinforcement Learning for Interference-Aware Path Planning of Cellular-Connected UAVs
Challita, U., Saad, W., & Bettstetter, C. (2018) · International Conference on Communications · doi:10.1109/icc.2018.8422706 · Cited by 145 · see citing works
Javed, K., Gouriveau, R., Noureddine Zerhouni, A., & Hissel, D. (2016) · Journal of Power Sources · doi:10.1016/j.jpowsour.2016.05.092 · Cited by 143 · see citing works
Wind speed and wind direction forecasting using echo state network with nonlinear functions
Chitsazan, M. A., Fadali, M. S., & Trzynadlowski, A. M. (2018) · Renewable Energy · doi:10.1016/j.renene.2018.07.060 · Cited by 143 · see citing works
Hinaut, X., & Dominey, P. F. (2013) · PLoS ONE · doi:10.1371/journal.pone.0052946 · Cited by 139 · see citing works
Han, M., Feng, S., Chen, C. L. P., Xu, M., & Qiu, T. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2866149 · Cited by 134 · see citing works
Learning Precisely Timed Spikes
Memmesheimer, R.-M., Rubin, R., Ölveczky, B. P., & Sompolinsky, H. (2014) · Neuron · doi:10.1016/j.neuron.2014.03.026 · Cited by 125 · see citing works
Pruning and regularization in reservoir computing
Dutoit, X., Schrauwen, B., Campenhout, J. V., Stroobandt, D., Brussel, H. V., & Nuttin, M. (2009) · Neurocomputing · doi:10.1016/j.neucom.2008.12.020 · Cited by 125 · see citing works
Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps
Rodan, A., & Peter Tiňo, B. (2012) · Neural Computation · doi:10.1162/neco_a_00297 · Cited by 124 · see citing works
A Novel Framework of Reservoir Computing for Deterministic and Probabilistic Wind Power Forecasting
Wang, J., Niu, T., Lu, H., Yang, W., & Du, P. (2019) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2019.2890875 · Cited by 119 · see citing works
Nanoscale resistive switching devices for memory and computing applications
Lee, S. H., Zhu, X., & Lu, W. D. (2020) · Nano Research · doi:10.1007/s12274-020-2616-0 · Cited by 118 · see citing works
Quantum reservoir computing with a single nonlinear oscillator
Govia, L. C. G., Ribeill, G. J., Rowlands, G. E., Krovi, H., & Ohki, T. (2021) · Physical Review Research · doi:10.1103/physrevresearch.3.013077 · Cited by 117 · see citing works
FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural NetworkPreprint
Kusupati, A., Singh, M., Bhatia, K., Kumar, A., Jain, P., & Varma, M. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1901.02358 · Cited by 208 · see citing works
An experimental characterization of reservoir computing in ambient assisted living applications
Bacciu, D., Barsocchi, P., Chessa, S., Gallicchio, C., & Micheli, A. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1364-4 · Cited by 114 · see citing works
Echo State Networks With Orthogonal Pigeon-Inspired Optimization for Image Restoration
Duan, H., & Wang, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2479117 · Cited by 115 · see citing works
Chatzis, S., & Demiris, Y. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2162109 · Cited by 113 · see citing works
A Systematic Review of Echo State Networks From Design to Application
Sun, C., Song, M., Cai, D., Zhang, B., Hong, S., & Li, H. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3225780 · Cited by 112 · see citing works
DeePr-ESN: A deep projection-encoding echo-state network
Ma, Q., Shen, L., & Cottrell, G. W. (2019) · Information Sciences · doi:10.1016/j.ins.2019.09.049 · Cited by 111 · see citing works
Quaternion-Valued Echo State Networks
Xia, Y., Jahanchahi, C., & Mandic, D. P. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2320715 · Cited by 108 · see citing works
Optimizing the echo state network with a binary particle swarm optimization algorithm
Wang, H., & Yan, X. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.06.003 · Cited by 106 · see citing works
Automated EEG-Based Diagnosis of Neurological Disorders: Inventing the Future of Neurology
Adeli, H., & Ghosh‐Dastidar, S. (2010) · Thammasat University Digital Collections · Cited by 103 · see citing works
Intelligent stock trading system based on improved technical analysis and Echo State Network
Lin, X., Yang, Z., & Song, Y. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.03.001 · Cited by 102 · see citing works
Tian, Y., Xu, Y., Yang, Z., Wang, G., & Zhu, Q. (2018) · Water · doi:10.3390/w10111655 · Cited by 99 · see citing works
Memory versus non-linearity in reservoirs
Verstraeten, D., Dambre, J., Dutoit, X., & Schrauwen, B. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596492 · Cited by 99 · see citing works
Echo state networks with filter neurons and a delay&sum readout
Holzmann, G., & Häuser, H. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.004 · Cited by 95 · see citing works
Chen, Q., Shi, L., Na, J., Ren, X., & Nan, Y. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.09.083 · Cited by 95 · see citing works
Energy consumption prediction of office buildings based on echo state networks
Shi, G., Liu, D., & Wei, Q. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.08.004 · Cited by 95 · see citing works
Zhang, H., Liu, C., Su, H., & Zhang, K. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2958484 · Cited by 95 · see citing works
Zhong, S., Xie, X., Lin, L., & Wang, F. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.053 · Cited by 93 · see citing works
Shahi, S., Fenton, F. H., & Cherry, E. M. (2022) · Machine Learning with Applications · doi:10.1016/j.mlwa.2022.100300 · Cited by 93 · see citing works
Freire, P. J., Srivallapanondh, S., Spinnler, B., Napoli, A., Costa, N. M. S., Prilepsky, J. E., & Turitsyn, S. K. (2024) · Journal of Lightwave Technology · doi:10.1109/jlt.2024.3386886 · Cited by 90 · see citing works
Event detection and localization for small mobile robots using reservoir computing
Antonelo, E. A., Schrauwen, B., & Stroobandt, D. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.06.010 · Cited by 91 · see citing works
Controlling Recurrent Neural Networks by ConceptorsPreprint
Jaeger, H. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.3369 · Cited by 95 · see citing works
Self-citationOnline Training of an Opto-Electronic Reservoir Computer Applied to Real-Time Channel Equalization
Antonik, P., Duport, F., Hermans, M., Smerieri, A., Haelterman, M., & Massar, S. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2598655 · Cited by 86 · see citing works
Recurrent Neural Networks for Edge Intelligence
Lalapura, V. S., Amudha, J., & Satheesh, H. S. (2021) · ACM Computing Surveys · doi:10.1145/3448974 · Cited by 84 · see citing works
Artificial neural networks for photonic applications—from algorithms to implementation: tutorial
Freire, P. J., Manuylovich, E. S., Prilepsky, J. E., & Turitsyn, S. K. (2023) · Advances in Optics and Photonics · doi:10.1364/aop.484119 · Cited by 90 · see citing works
Short-Term Load Forecasting With Seasonal Decomposition Using Evolution for Parameter Tuning
Høverstad, B. A., Tidemann, A., Helge Langseth, A., & Öztürk, P. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2395822 · Cited by 81 · see citing works
Morando, S., Jemeï, S., Hissel, D., Gouriveau, R., & Noureddine Zerhouni, A. (2015) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2015.06.009 · Cited by 83 · see citing works
Han, S.-I., & Lee, J. M. (2012) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2012.2218560 · Cited by 82 · see citing works
Recurrent Kernel Machines: Computing with Infinite Echo State Networks
Hermans, M., & Schrauwen, B. (2011) · Neural Computation · doi:10.1162/neco_a_00200 · Cited by 82 · see citing works
A neurodynamical model for working memory
Pascanu, R., & Jaeger, H. (2010) · Neural Networks · doi:10.1016/j.neunet.2010.10.003 · Cited by 82 · see citing works
Self-citationEcho State Networks for data-driven downhole pressure estimation in gas-lift oil wells
Antonelo, E. A., Camponogara, E., & Foss, B. A. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.09.009 · Cited by 82 · see citing works
Gradient based hyperparameter optimization in Echo State Networks
Thiede, L., & Parlitz, U. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.02.001 · Cited by 80 · see citing works
Discovering multiscale dynamical features with hierarchical Echo State Networks
Jaeger, H. (2007) · Cited by 80 · see citing works
Self-citationTime Series Classification Using Time Warping Invariant Echo State Networks
Tanisaro, P., & Heidemann, G. (2016) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2016.0149 · Cited by 79 · see citing works
Modeling neural plasticity in echo state networks for classification and regression
Yusoff, M.-H., Chrol-Cannon, J., & Jin, Y. (2015) · Information Sciences · doi:10.1016/j.ins.2015.11.017 · Cited by 78 · see citing works
Modified Neural Dynamic Surface Approach to Output Feedback of MIMO Nonlinear Systems
Sun, G., Li, D., & Ren, X. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2312001 · Cited by 77 · see citing works
Scalable Spatiotemporal Graph Neural Networks
Cini, A., Marisca, I., Bianchi, F. M., & Alippi, C. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25880 · Cited by 77 · see citing works
On Learning Navigation Behaviors for Small Mobile Robots With Reservoir Computing Architectures
Antonelo, E. A., & Schrauwen, B. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2323247 · Cited by 74 · see citing works
Line-of-Sight-Based Guidance and Adaptive Neural Path-Following Control for Sailboats
Deng, Y., Zhang, X., & Zhang, G. (2019) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2019.2923502 · Cited by 76 · see citing works
A comparative study of Reservoir Computing strategies for monthly time series prediction
wyffels, F., & Schrauwen, B. (2010) · Neurocomputing · doi:10.1016/j.neucom.2010.01.016 · Cited by 74 · see citing works
Recurrent Residual Learning for Sequence Classification
Wang, Y., & Tian, F. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1093 · Cited by 75 · see citing works
- Authors
- Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U.
- Affiliations
- Constructor University (DE)
- Planet (DE)
- Corresponding author
- Jaeger, Herbert
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2007-04
- Volume
- 20
- Issue
- 3
- Pages
- 335-352
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2001263627
- Semantic Scholar
- 370984d3910e61d41ee8318714cd610b689de726
- Cited by
- 948
- References
- 32
- Citations by year
- 2026: 61 · 2025: 87 · 2024: 90 · 2023: 65 · 2022: 71 · 2021: 68 · 2020: 68 · 2019: 81 · 2018: 51 · 2017: 38 · 2016: 44 · 2015: 33 · 2014: 37 · 2013: 28 · 2012: 42
- FWCI
- 24.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- echo state networks, reservoir computing, stability conditions, stochastic gradient descent, spectral radius, time series classification, time warping
- MeSH terms
- Animals, Feedback, Humans, Learning, Models, Neurological, Nerve Net, Neurons, Stochastic Processes, Neural Networks, Computer, Nonlinear Dynamics, Recognition, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
