The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Deep reservoir computing: A critical experimental analysis
Open accessAbstract
In this paper, we propose an empirical analysis of deep recurrent neural network (RNN) architectures with stacked layers. The main aim is to address some fundamental open research issues on the significance of creating deep layered architectures in RNN and to characterize the inherent hierarchical representation of time in such models, especially for efficient implementations. In particular, the analysis aims at the study and proposal of approaches to develop and enhance hierarchical dynamics in deep architectures within the efficient Reservoir Computing (RC) framework for RNN modeling. The effect of a deep layered organization of RC models is investigated in terms of both occurrence of multiple time-scale and increasing of richness of the dynamics. It turns out that a deep layering of recurrent models allows an effective diversification of temporal representations in the layers of the hierarchy, by amplifying the effects of the factors influencing the time-scales and the richness of the dynamics, measured as the entropy of recurrent units activations. The advantages of the proposed approach are also highlighted by measuring the increment of the short-term memory capacity of the RC models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
(2016) · Cited by 11,679
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811
Reservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Proceedings IEEE International Conference on Acoustics Speech and Signal Processing
Zheng, F., McLaughlin, S., & Mulgrew, B. (1991) · Cited by 1,905
Generating Text with Recurrent Neural Networks
Sutskever, I., Martens, J., & Hinton, G. E. (2011) · International Conference on Machine Learning · Cited by 1,783
An experimental unification of reservoir computing methods
Verstraeten, D., Schrauwen, B., D’Haene, M., & Stroobandt, D. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.003 · Cited by 1,135
Optimization and applications of echo state networks with leaky- integrator neurons
Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.016 · Cited by 948
(2014) · International Conference on Learning Representations · Cited by 634
Training and Analysing Deep Recurrent Neural Networks
Hermans, M., & Schrauwen, B. (2013) · neural information processing systems · Cited by 631
(1997) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · Cited by 598
Proceedings of the International Conference on Artificial Neural Networks
Dorffner, G., Bischof, H., & Hornik, K. (2001) · Cited by 377
Hierarchical Recurrent Neural Networks for Long-Term Dependencies
Hihi, S. E., & Bengio, Y. (1995) · Cited by 325
Generalized distances between rankings
Kumar, R., & Vassilvitskii, S. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772749 · Cited by 273
Improving reservoirs using intrinsic plasticity
Schrauwen, B., Wardermann, M., Verstraeten, D., Steil, J. J., & Stroobandt, D. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.12.020 · Cited by 227
Architectural and Markovian factors of echo state networks
Gallicchio, C., & Micheli, A. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.02.002 · Cited by 198
Self-citationSteil, J. J. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.011 · Cited by 165
A Gradient Rule for the Plasticity of a Neuron’s Intrinsic Excitability
Triesch, J. (2005) · Lecture notes in computer science · doi:10.1007/11550822_11 · Cited by 154
Perspectives of Neural-Symbolic Integration
Hammer, B., & Hitzler, P. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-73954-8 · Cited by 95
Acoustic Modeling With Hierarchical Reservoirs
Triefenbach, F., Jalalvand, A., Demuynck, K., & Martens, J. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2280209 · Cited by 80
Angelov, P. P., & Sperduti, A. (2016) · Lancaster EPrints (Lancaster University) · Cited by 60
A Dynamical Approach to Temporal Pattern Processing
Stornetta, W. S., Hogg, T., & Huberman, B. A. (1987) · neural information processing systems · Cited by 52
Deep reservoir computing: A critical analysis
Gallicchio, C., & Micheli, A. (2016) · CINECA IRIS Institutial research information system (University of Pisa) · Cited by 38
Self-citationMarkovian Bias of Neural-based Architectures With Feedback Connections
Peter Tiňo, B., Hammer, B., & Mikael Bodén, A. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-73954-8_5 · Cited by 33
Cited by
Recent advances in physical reservoir computing: A review
Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., & Hirose, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.005 · Cited by 2,146
Embedded Devices for Neuromorphic Time-Series Assessment
Mazumder, A. N., Hosseini, M., & Mohsenin, T. (2022) · Research Explorer (The University of Manchester) · doi:10.13016/m2ksy0-wxr3 · Cited by 652
Temporal data classification and forecasting using a memristor-based reservoir computing system
Moon, J., Ma, W., Shin, J. H., Cai, F., Du, C., Lee, S. H., & Lu, W. D. (2019) · Nature Electronics · doi:10.1038/s41928-019-0313-3 · Cited by 645
Emerging opportunities and challenges for the future of reservoir computing
Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., & Sun, J. (2024) · Nature Communications · doi:10.1038/s41467-024-45187-1 · Cited by 346
Random vector functional link neural network based ensemble deep learning
Shi, Q., Katuwal, R., Suganthan, P. N., & Tanveer, M. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107978 · Cited by 286
Metadata
- Authors
- Gallicchio, C., Micheli, A., & Pedrelli, L.
- Venue
- Neurocomputing
- Publisher
- Elsevier BV
- Published
- 2017-12
- Volume
- 268
- Pages
- 87-99
- ISSN
- 0925-2312 (ISSN-L)
- OpenAlex
- W2608997467
- Cited by
- 519
- References
- 29
- FWCI
- 17.22
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.