Reservoir Computing Approaches for Representation and Classification of Multivariate Time Series
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Classification of multivariate time series (MTS) has been tackled with a large variety of methodologies and applied to a wide range of scenarios. Reservoir computing (RC) provides efficient tools to generate a vectorial, fixed-size representation of the MTS that can be further processed by standard classifiers. Despite their unrivaled training speed, MTS classifiers based on a standard RC architecture fail to achieve the same accuracy of fully trainable neural networks. In this article, we introduce the reservoir model space, an unsupervised approach based on RC to learn vectorial representations of MTS. Each MTS is encoded within the parameters of a linear model trained to predict a low-dimensional embedding of the reservoir dynamics. Compared with other RC methods, our model space yields better representations and attains comparable computational performance due to an intermediate dimensionality reduction procedure. As a second contribution, we propose a modular RC framework for MTS classification, with an associated open-source Python library. The framework provides different modules to seamlessly implement advanced RC architectures. The architectures are compared with other MTS classifiers, including deep learning models and time series kernels. Results obtained on the benchmark and real-world MTS data sets show that RC classifiers are dramatically faster and, when implemented using our proposed representation, also achieve superior classification accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 11×
“The second-best accuracy is obtained by oneNN with DTW, while the classifiers based on LSTM and GRU perform only better than lESN.”
BackgroundInfluentialSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Gated RNN architectures Despite the theoretical capability of basic RNNs to model any dynamical system, in practice their effectiveness is hampered by the difficulty of training their parameters [37, 38].”
BackgroundRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025 · cited 5×
“For this reason, several authors proposed to replace the linear decoding function g(·) in (5) with a nonlinear model, such as SVMs [12] or MLPs [41]–[43].”
BackgroundInfluentialDeep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342 · cited 1×
“We implement baselines based on popular deep learning architectures (MLP, FCN, and ResNets) [53], [54] and report results where available on the original articles for BOSS [55], PROP [56], COTE [57], an advanced deep learning architecture that combines an LSTM with attention to a CNN architecture (LSTM-FCN) [58], a mo…”
MethodReservoir computing approaches to recurrent neural network training
Lukoševičius, M., & Jaeger, H. (2009) · Computer Science Review · doi:10.1016/j.cosrev.2009.03.005 · Cited by 2,956
Time series classification from scratch with deep neural networks: A strong baseline
Wang, Z., Yan, W., & Oates, T. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966039 · Cited by 2,299 · cited 1×
“We implement baselines based on popular deep learning architectures (MLP, FCN, and ResNets) [53], [54] and report results where available on the original articles for BOSS [55], PROP [56], COTE [57], an advanced deep learning architecture that combines an LSTM with attention to a CNN architecture (LSTM-FCN) [58], a mo…”
MethodBagnall, A., Lines, J., Bostrom, A., Large, J., & Keogh, E. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-016-0483-9 · Cited by 1,377 · cited 1×
“The problem has been tackled by a wealth of different approaches, spanning from the definition of tailored distance measures over MTS [9], to the identification and modeling of short-time patterns in the form of dictionaries or shapelets [10].”
BackgroundGenerative embedding for model-based classification of fMRI data
Brodersen, K. H., Schofield, T. M., Leff, A. P., Ong, C. S., Lomakina, E. I., Buhmann, J. M., & Stephan, K. E. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002079 · Cited by 165 · cited 1×
“Usually, in a hybrid generative-discriminative approach where data are assumed to be generated by a parametric distribution, the subsequent discriminative model cannot be specified independently from the generative model type, without introducing biases in the classification [60].”
BackgroundMultiplex visibility graphs to investigate recurrent neural network dynamics
Bianchi, F. M., Livi, L., Alippi, C., & Jenssen, R. (2017) · Scientific Reports · doi:10.1038/srep44037 · Cited by 41 · cited 1×
“the hyperparameter setting [47], a fine-tuning with independent cross validation for each task is usually more important in”
BackgroundSelf-citation
Cited by
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342
Emerging opportunities and challenges for the future of reservoir computing
Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., & Sun, J. (2024) · Nature Communications · doi:10.1038/s41467-024-45187-1 · Cited by 346
Fixed complexity tiny reservoir heterogeneous network for on-line ECG learning of anomalies
Abdennadher, N., Pau, D., & Bruna, A. (2021) · 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE) · doi:10.1109/gcce53005.2021.9622022 · Cited by 9
Metadata
- Authors
- Bianchi, F. M., Scardapane, S., Lokse, S., & Jenssen, R.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2021-05
- Volume
- 32
- Issue
- 5
- Pages
- 2169-2179
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2789601762
- Cited by
- 224
- References
- 13
- FWCI
- 9.19
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.