The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Deep Neural Network Ensembles for Time Series Classification
Open accessAbstract
Deep neural networks have revolutionized many fields such as computer vision and natural language processing. Inspired by this recent success, deep learning started to show promising results for Time Series Classification (TSC). However, neural networks are still behind the state-of-the-art TSC algorithms, that are currently composed of ensembles of 37 non deep learning based classifiers. We attribute this gap in performance due to the lack of neural network ensembles for TSC. Therefore in this paper, we show how an ensemble of 60 deep learning models can significantly improve upon the current state-of-the-art performance of neural networks for TSC, when evaluated over the UCR/UEA archive: the largest publicly available benchmark for time series analysis. Finally, we show how our proposed Neural Network Ensemble (NNE) is the first time series classifier to outperform COTE while reaching similar performance to the current state-of-the-art ensemble HIVE-COTE.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“we find that classification performance of a single IE model shows high standard deviation due to the random weight initialization in the experiment [68].”
From IENet: a robust convolutional neural network for EEG based brain-computer interfaces (2022) · cites this paper as [68]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
(2010) · Cited by 14,134
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 12,051
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342 · cited 10×
“All of these architectures were implemented in a common framework during our empirical study [19], containing originally 9 different deep learning approaches for TSC.”
BackgroundMethodInfluentialSelf-citationTime series classification from scratch with deep neural networks: A strong baseline
Wang, Z., Yan, W., & Oates, T. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966039 · Cited by 2,299
Bagnall, A., Lines, J., Bostrom, A., Large, J., & Keogh, E. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-016-0483-9 · Cited by 1,377
10 CHALLENGING PROBLEMS IN DATA MINING RESEARCH
Yang, Q., & Wu, X. (2006) · International Journal of Information Technology & Decision Making · doi:10.1142/s0219622006002258 · Cited by 956
Convolutional neural networks for time series classification
College of Electronic Science and Engineering, N. U. O. D. T., Zhao, B., Lu, H., College of Electronic Science and Engineering, N. U. O. D. T., Chen, S., College of Electronic Science and Engineering, N. U. O. D. T., Liu, J., College of Electronic Science and Engineering, N. U. O. D. T., Wu, D., & College of Electronic Science and Engineering, N. U. O. D. T. (2017) · Journal of Systems Engineering and Electronics · doi:10.21629/jsee.2017.01.18 · Cited by 888
The Loss Surfaces of Multilayer Networks
Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., & LeCun, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.0233 · Cited by 733
Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks
Zheng, Y., Liu, Q., Chen, E., Ge, Y., & Zhao, J. L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08010-9_33 · Cited by 706
Multi-Scale Convolutional Neural Networks for Time Series ClassificationPreprint
Cui, Z.-C., Chen, W.-L., & Chen, Y. (2016) · arXiv.org · doi:10.48550/arxiv.1603.06995 · Cited by 637
Time series classification with ensembles of elastic distance measures
Lines, J., & Bagnall, A. J. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0361-2 · Cited by 513
A Bag-of-Features Framework to Classify Time Series
Baydogan, M. G., Runger, G., & Tuv, E. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.72 · Cited by 361
Using dynamic time warping distances as features for improved time series classification
Kate, R. J. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0418-x · Cited by 355
Data Augmentation for Time Series Classification using Convolutional Neural Networks
Guennec, A. L., Malinowski, S., & Tavenard, R. (2016) · HAL · Cited by 341
Lines, J., Taylor, S., & Bagnall, A. (2018) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3182382 · Cited by 333
Transfer learning for time series classification
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2018) · 2018 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2018.8621990 · Cited by 319
Self-citationShould we really use post-hoc tests based on mean-ranks?Preprint
BenavoliAlessio, CoraniGiorgio, & MangiliFrancesca (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02288 · Cited by 318
Data augmentation using synthetic data for time series classification with deep residual networksPreprint
Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., & Müller, P.-A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.02455 · Cited by 187
Self-citationProximity Forest: an effective and scalable distance-based classifier for time series
Lucas, B., Shifaz, A., Pelletier, C., O’Neill, L., Zaidi, N., Goethals, B., Petitjean, F., & Webb, G. I. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00617-3 · Cited by 183
Adversarial Attacks on Deep Neural Networks for Time Series Classification
Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., & Müller, P.-A. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851936 · Cited by 170
Self-citationEnsemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition
Wen, G., Hou, Z., Li, H., Li, D., Jiang, L., & Xun, E. (2017) · Cognitive Computation · doi:10.1007/s12559-017-9472-6 · Cited by 164
Proximity Forest: an effective and scalable distance-based classifier for time series
Lucas, B., Shifaz, A., Pelletier, C., O’Neill, L., Zaidi, N. A., (15827960), B. G., Petitjean, F., & (14480133), G. W. (2024) · Figshare · Cited by 147
Evaluating Surgical Skills from Kinematic Data Using Convolutional Neural Networks
Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., & Müller, P.-A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_25 · Cited by 121
Self-citationTowards a Universal Neural Network Encoder for Time Series
Joan, S., Santiago, P., & Karatzoglou, A. (2018) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-918-8-120 · Cited by 119
Towards a universal neural network encoder for time seriesPreprint
Serrà, J., Pascual, S., & Karatzoglou, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03908 · Cited by 92
Binary Shapelet Transform for Multiclass Time Series Classification
Bostrom, A., & Bagnall, A. J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22729-0_20 · Cited by 85
Time Series Classification Using Time Warping Invariant Echo State Networks
Tanisaro, P., & Heidemann, G. (2016) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2016.0149 · Cited by 79
Time-series classification with COTE: The collective of transformation-based ensembles
Bagnall, A. J., Lines, J., Hills, J., & Bostrom, A. G. (2016) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2016.7498418 · Cited by 63
Ensemble Deep Learning for Biomedical Time Series Classification
Jin, L.-P., & Dong, J. (2016) · Computational Intelligence and Neuroscience · doi:10.1155/2016/6212684 · Cited by 60
Deep Dilated Convolution on Multimodality Time Series for Human Activity Recognition
Xi, R., Hou, M., Fu, M., Qu, H., & Liu, D. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489540 · Cited by 56
Gharghabi, S., Imani, S., Bagnall, A. J., Darvishzadeh, A., & Keogh, E. (2020) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-020-00695-8 · Cited by 34
Text-To-Text Generative Adversarial Networks
Li, C., Su, Y., & Liu, W. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489624 · Cited by 33
Dau, H. A., Silva, D. F., Petitjean, F., Forestier, G., Bagnall, A. J., & Keogh, E. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258009 · Cited by 26
Self-citationMNRD: A Merged Neural Model For Rumor Detection In Social Media
Xu, N., Chen, G., & Mao, W. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489582 · Cited by 24
Predicting Bucket-Filling Control Actions of a Wheel-Loader Operator Using a Neural Network Ensemble
Dadhich, S., Sandin, F., & Bodin, U. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489388 · Cited by 20
Time Series Trend Detection and Forecasting Using Complex Network Topology Analysis
Anghinoni, L., Zhao, L., Zheng, Q., & Zhang, J. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489167 · Cited by 15
Semi-Supervised Clustering With Multiresolution Autoencoders
Ienco, D., & Pensa, R. G. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489353 · Cited by 12
Ensemble of binary tree structured deep convolutional network for image classification
Lee, J.-E., Min-Joo, K., & Kang, J. (2017) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8282260 · Cited by 8
Deep neural network ensemble by data augmentation and bagging for skin lesion classificationPreprint
Goyal, M., & Rajapakse, J. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.05496 · Cited by 7
Hierarchical Tree Long Short-Term Memory for Sentence Representations
Wang, X., Li, C., & Xu, B. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489082 · Cited by 4
Multi ROI and Multi Map Networks for Accurate and Efficient Pedestrian Detection
Qiu, Z., & Gu, X. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489310 · Cited by 3
Stochastic Multiple Choice Learning for Acoustic Modeling
Liul, B., Nie, S., Liang, S., Yang, Z., & Liu, W. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489454 · Cited by 3
Cited by
Sorted by citations
Inceptiontime: finding alexnet for time series classification
Ismail Fawaz, H., Lucas, B., Forestier, G., Pelletier, C., Schmidt, D. F., Weber, J., Webb, G. I., Idoumghar, L., Muller, P.-A., & Petitjean, F. (2020) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-020-00710-y · Cited by 1,788
Self-citationDempster, A., Petitjean, F., & Webb, G. I. (2020) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-020-00701-z · Cited by 1,165
Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey
Foumani, N. M., Miller, L., Tan, C. W., Webb, G. I., Forestier, G., & Salehi, M. (2024) · ACM Computing Surveys · doi:10.1145/3649448 · Cited by 328
Self-citationAdversarial Attacks on Deep Neural Networks for Time Series Classification
Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., & Müller, P.-A. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851936 · Cited by 170
Self-citationDeep Learning for Satellite Image Time-Series Analysis: A review
Miller, L., Pelletier, C., & Webb, G. I. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3393010 · Cited by 136
Metadata
- Authors
- Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A.
- Affiliations
- IRIMAS, Université Haute-Alsace, Mulhouse, France
- Université de Haute-Alsace (FR)
- Venue
- 2019 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2019-07
- Pages
- 1-6
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2978259066
- Cited by
- 103
- References
- 47
- Citations by year
- 2026: 3 · 2025: 10 · 2024: 15 · 2023: 21 · 2022: 17 · 2021: 21 · 2020: 14 · 2019: 2
- FWCI
- 10.35
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Nvidia
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- time series classification, neural network ensembles, deep neural network, ensemble learning, CotE, ensemble classifiers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
