The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Time Series Classification with HIVE-COTE: The Hierarchical Vote Collective of Transformation-Based Ensembles
Open accessAbstract
A recent experimental evaluation assessed 19 time series classification (TSC) algorithms and found that one was significantly more accurate than all others: the Flat Collective of Transformation-based Ensembles (Flat-COTE). Flat-COTE is an ensemble that combines 35 classifiers over four data representations. However, while comprehensive, the evaluation did not consider deep learning approaches. Convolutional neural networks (CNN) have seen a surge in popularity and are now state of the art in many fields and raises the question of whether CNNs could be equally transformative for TSC. We implement a benchmark CNN for TSC using a common structure and use results from a TSC-specific CNN from the literature. We compare both to Flat-COTE and find that the collective is significantly more accurate than both CNNs. These results are impressive, but Flat-COTE is not without deficiencies. We significantly improve the collective by proposing a new hierarchical structure with probabilistic voting, defining and including two novel ensemble classifiers built in existing feature spaces, and adding further modules to represent two additional transformation domains. The resulting classifier, the Hierarchical Vote Collective of Transformation-based Ensembles (HIVE-COTE), encapsulates classifiers built on five data representations. We demonstrate that HIVE-COTE is significantly more accurate than Flat-COTE (and all other TSC algorithms that we are aware of) over 100 resamples of 85 TSC problems and is the new state of the art for TSC. Further analysis is included through the introduction and evaluation of 3 new case studies and extensive experimentation on 1,000 simulated datasets of 5 different types.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This shows how much the community lacks of an overview of the current performance of deep learning models for solving the TSC problem (Lines et al., 2018).”
From Deep learning for time series classification: a review (2019) · cites this paper as Lines et al., 2018
In the citing paper
“It is worthwhile noting that cliques formed by the Wilcoxon Signed Rank Test with Holm's alpha correction do not necessary reflect the rank order (Lines et al., 2018).”
From Deep learning for time series classification: a review (2019) · cites this paper as Lines et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 12,051
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Experiments with a new boosting algorithm
Freund, Y., & Schapire, R. E. (1996) · Cited by 8,189
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Theano: A Python framework for fast computation of mathematical expressionsPreprint
The Theano Development Team, Al‐Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.02688 · Cited by 2,176
Rotation Forest: A New Classifier Ensemble Method
Rodríguez, J. J., Kuncheva, L. I., & Alonso, C. J. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.211 · Cited by 1,931
Graph-based approach for airborne light detection and ranging segmentation
Vilarino, D. L., Cabaleiro, J. C., Sánchez, J. M., Rivera, F. F., & Pena, T. F. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.015020 · Cited by 1,760
Static Analysis of Shape in TensorFlow ProgramsPreprint
The Theano Development Team, Al‐Rfou, R., Alain, G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2020.15 · Cited by 1,629
Bagnall, A., Lines, J., Bostrom, A., Large, J., & Keogh, E. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-016-0483-9 · Cited by 1,377
Self-citationGarcía, S., & Herrera, F. (2008) · Cited by 1,277
An audio-visual corpus for speech perception and automatic speech recognition
Cooke, M. P., Barker, J., Cunningham, S. P., & Shao, X. (2006) · The Journal of the Acoustical Society of America · doi:10.1121/1.2229005 · Cited by 1,146
A Convolutional Neural Network for Modelling SentencesPreprint
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.2188 · Cited by 967
MultiBoosting: A Technique for Combining Boosting and Wagging
Webb, G. I. (2000) · Machine Learning · doi:10.1023/a:1007659514849 · Cited by 703
Weighted dynamic time warping for time series classification
Jeong, Y., Jeong, M. K., & Omitaomu, O. A. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.022 · Cited by 697
Multi-Scale Convolutional Neural Networks for Time Series ClassificationPreprint
Cui, Z.-C., Chen, W.-L., & Chen, Y. (2016) · arXiv.org · doi:10.48550/arxiv.1603.06995 · Cited by 637
A time series forest for classification and feature extraction
Deng, H., Runger, G. C., Tuv, E., & Vladimir, M. (2013) · Information Sciences · doi:10.1016/j.ins.2013.02.030 · Cited by 632
The BOSS is concerned with time series classification in the presence of noise
Schäfer, P. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0377-7 · Cited by 564
Classification of time series by shapelet transformation
Hills, J., Lines, J., Baranauskas, E., Mapp, J., & Bagnall, A. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-013-0322-1 · Cited by 538
Self-citationProceedings 2001 IEEE International Conference on Data Mining
(2001) · doi:10.1109/icdm.2001 · Cited by 515
Time series classification with ensembles of elastic distance measures
Lines, J., & Bagnall, A. J. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0361-2 · Cited by 513
Self-citationLearning time-series shapelets
Grabocka, J., Schilling, N., Wistuba, M., & Schmidt-Thieme, L. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623613 · Cited by 489
Time-Series Classification with COTE: The Collective of Transformation-Based Ensembles
Bagnall, A. J., Lines, J., Hills, J., & Bostrom, A. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2416723 · Cited by 433
Self-citationTime series shapelets: a novel technique that allows accurate, interpretable and fast classification
Ye, L., & Keogh, E. (2011) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0179-5 · Cited by 406
Three Myths about Dynamic Time Warping Data Mining
Ratanamahatana, C. A., & Keogh, E. (2005) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972757.50 · Cited by 398
A shapelet transform for time series classification
Lines, J., Davis, L. M., Hills, J., & Bagnall, A. J. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339579 · Cited by 380
Self-citationHighly Comparative Feature-Based Time-Series Classification
Fulcher, B., & Jones, N. S. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2316504 · Cited by 363
A Bag-of-Features Framework to Classify Time Series
Baydogan, M. G., Runger, G., & Tuv, E. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.72 · Cited by 361
CID: an efficient complexity-invariant distance for time series
Batista, G. E. A. P. A., Keogh, E., Tataw, O. M., & Souza, V. M. A. (2013) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-013-0312-3 · Cited by 361
Using dynamic time warping distances as features for improved time series classification
Kate, R. J. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0418-x · Cited by 355
Time Warp Edit Distance with Stiffness Adjustment for Time Series Matching
Marteau, P.-F. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.76 · Cited by 337
Should we really use post-hoc tests based on mean-ranks?Preprint
BenavoliAlessio, CoraniGiorgio, & MangiliFrancesca (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02288 · Cited by 318
Rotation-invariant similarity in time series using bag-of-patterns representation
Lin, J., Khade, R., & Li, Y. (2012) · Journal of Intelligent Information Systems · doi:10.1007/s10844-012-0196-5 · Cited by 305
A periodogram-based metric for time series classification
Caiado, J., Crato, N., & Peña, D. (2005) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2005.04.012 · Cited by 258
Proceedings of the 2014 SIAM International Conference on Data Mining
(2014) · doi:10.1137/1.9781611973440 · Cited by 234
Time series clustering and classification by the autoregressive metric
Corduas, M., & Piccolo, D. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.06.001 · Cited by 199
The Move-Split-Merge Metric for Time Series
Stefan, A., Athitsos, V., & Das, G. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.88 · Cited by 189
Lines, J., Taylor, S. L., & Bagnall, A. J. (2016) · Proceedings · doi:10.1109/icdm.2016.0133 · Cited by 177
Self-citationTime series representation and similarity based on local autopatterns
Baydoğan, M. G., & Runger, G. C. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0425-y · Cited by 176
Transformation Based Ensembles for Time Series Classification
Bagnall, A. J., Davis, L. M., Hills, J., & Lines, J. (2012) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972825.27 · Cited by 106
Self-citationNon-isometric transforms in time series classification using DTW
Górecki, T., & Łuczak, M. (2014) · Knowledge-Based Systems · doi:10.1016/j.knosys.2014.02.011 · Cited by 95
Binary Shapelet Transform for Multiclass Time Series Classification
Bostrom, A., & Bagnall, A. J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22729-0_20 · Cited by 85
Self-citationProceedings of the 12th SIAM International Conference on Data Mining, SDM 2012
Zhou, T., Shan, H., Banerjee, A., & Sapiro, G. (2012) · SIAM International Conference on Data Mining · Cited by 78
Villar, J. R., Vergara, P. M., Menéndez‐González, M., de la Cal, E., González, V. M., & Sedano, J. (2016) · International Journal of Neural Systems · doi:10.1142/s0129065716500374 · Cited by 64
A Run Length Transformation for Discriminating Between Auto Regressive Time Series
Bagnall, A. J., & Janacek, G. J. (2013) · Journal of Classification · doi:10.1007/s00357-013-9135-6 · Cited by 39
Self-citationBagnall, A. J., Bostrom, A. G., Large, J., & Lines, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09480 · Cited by 3
Self-citation
Research works printed but not linked: 1
- 10.1023/A:1018054314350 · DOI · Paper page
Cited by
Sorted by citations
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342
A Transformer-based Framework for Multivariate Time Series Representation Learning
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467401 · Cited by 1,278
Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., & Keogh, E. (2019) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2019.1911747 · Cited by 1,128
Self-citationDempster, A., Petitjean, F., & Webb, G. I. (2020) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-020-00701-z · Cited by 1,165
Jin, M., Koh, H. Y., Wen, Q., Zambon, D., Alippi, C., Webb, G. I., King, I., & Pan, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3443141 · Cited by 567
Metadata
- Authors
- Lines, J., Taylor, S., & Bagnall, A.
- Affiliations
- University of East Anglia (GB)
- Venue
- ACM Transactions on Knowledge Discovery from Data
- Published
- 2018-07-05
- Volume
- 12
- Issue
- 5
- Pages
- 1-35
- ISSN
- 1556-4681 (ISSN-L)
- 1556-472X
- OpenAlex
- W2786161686
- Cited by
- 333
- References
- 48
- Citations by year
- 2026: 19 · 2025: 42 · 2024: 59 · 2023: 48 · 2022: 45 · 2021: 50 · 2020: 42 · 2019: 27 · 2018: 1
- FWCI
- 21.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Knowledge Discovery from Data (publishedVersion, cc-by)
- UEA Digital Repository (University of East Anglia) (submittedVersion)
- Funders
- UK Engineering and Physical Sciences Research Council (EP/M015807/1 and EP/M014053/1)
- Research and Specialist Computing Support service at the University of East Anglia
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- time series classification, ensemble classifiers, convolutional neural networks, probabilistic voting, data representation, benchmark comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
