CoChat

SAND: streaming subsequence anomaly detection

Boniol, Paul, Paparrizos, John, Palpanas, Themis, Franklin, Michael J. · 2021 · Proceedings of the VLDB Endowment
Cited by 119

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

Abstract

With the increasing demand for real-time analytics and decision making, anomaly detection methods need to operate over streams of values and handle drifts in data distribution. Unfortunately, existing approaches have severe limitations: they either require prior domain knowledge or become cumbersome and expensive to use in situations with recurrent anomalies of the same type. In addition, subsequence anomaly detection methods usually require access to the entire dataset and are not able to learn and detect anomalies in streaming settings. To address these problems, we propose SAND, a novel online method suitable for domain-agnostic anomaly detection. SAND aims to detect anomalies based on their distance to a model that represents normal behavior. SAND relies on a novel steaming methodology to incrementally update such model, which adapts to distribution drifts and omits obsolete data. The experimental results on several real-world datasets demonstrate that SAND correctly identifies single and recurrent anomalies without prior knowledge of the characteristics of these anomalies. SAND outperforms by a large margin the current state-of-the-art algorithms in terms of accuracy while achieving orders of magnitude speedups.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.14778/3467861.3467863

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 51

  1. D. Abboud M. Elbadaoui W.A. Smith and R.B. Randall. 2019. Advanced bearing diagnostics: A comparative study of two powerful approaches. MSSP 114 (2019). D. Abboud M. Elbadaoui W.A. Smith and R.B. Randall. 2019. Advanced bearing diagnostics: A comparative study of two powerful approaches. MSSP 114 (2019). · DOI
  2. 10.1177/1475921710395811 · doi:10.1177/1475921710395811
  3. Antoni Jerome, 2019, A statistical methodology for the design of condition indicators. Mechanical Systems and Signal Processing
  4. Bagnall Anthony J., 2019, Data Series Management (Dagstuhl Seminar 19282). Dagstuhl Reports (9(7)
  5. Bahaadini S., 2018, Machine learning for Gravity Spy: Glitch classification and dataset. Information Sciences 444 (1 5
  6. V. Barnet and T. Lewis. 1994. Outliers in Statistical Data. John Wiley and Sons Inc. V. Barnet and T. Lewis. 1994. Outliers in Statistical Data. John Wiley and Sons Inc.
  7. Boniol Paul, 2020, Automated Anomaly Detection in Large Sequences. In 36th IEEE International Conference on Data Engineering, ICDE 2020
  8. Boniol Paul, 2020, SAD: An Unsupervised System for Subsequence Anomaly Detection. In ICDE.
  9. Boniol Paul, 2021, Unsupervised and scalable subsequence anomaly detection in large data series. The VLDB Journal
  10. 10.14778/3407790.3407792 · DOI
  11. 10.14778/3415478.3415514 · DOI
  12. Bu Yingyi, 2007, Ada Wai-Chee Fu, Eamonn J. Keogh, Jian Pei, and Sam Meshkin.
  13. 10.1145/956750.956808 · DOI · Paper page
  14. Daouayry Nassia, Data-Centric Helicopter Failure Anticipation: The MGB Oil Pressure Virtual Sensor Case
  15. Yan, 2016, ICDM
  16. 10.1007/11811305_3 · DOI
  17. 10.1109/TKDE.2003.1198387 · DOI · Paper page
  18. Hadjem Medina, 2016, Khokhar
  19. 10.1109/ICDM.2005.79 · DOI · Paper page
  20. 10.1007/s10618-006-0049-3 · DOI
  21. 10.1109/ICDE.2011.5767923 · DOI · Paper page
  22. 10.5555/2969033.2969177 · doi:10.5555/2969033.2969177
  23. Linardi Michele, 2020, Keogh
  24. Liu Fei Tony, 2008, Kai Ming Ting, and Zhi-Hua Zhou
  25. 10.1007/978-3-642-00672-2_62 · DOI
  26. 10.5555/2022850.2022862 · doi:10.5555/2022850.2022862
  27. Haoran Ma Benyamin Ghojogh Maria N. Samad Dongyu Zheng and Mark Crowley. 2020. Isolation Mondrian Forest for Batch and Online Anomaly Detection. arXiv:2003.03692 [cs.LG] Haoran Ma Benyamin Ghojogh Maria N. Samad Dongyu Zheng and Mark Crowley. 2020. Isolation Mondrian Forest for Batch and Online Anomaly Detection. arXiv:2003.03692 [cs.LG] · arXiv
  28. Mirylenka Katsiaryna, 2013, Envelope-Based Anomaly Detection for High-Speed Manufacturing Processes. European Advanced Process Control and Manufacturing Conference
  29. G. B. Moody and R. G. Mark. 2001. The impact of the MIT-BIH Arrhythmia Database. IEEE Engineering in Medicine and Biology Magazine (2001). G. B. Moody and R. G. Mark. 2001. The impact of the MIT-BIH Arrhythmia Database. IEEE Engineering in Medicine and Biology Magazine (2001). · DOI · Paper page
  30. Mueen Abdullah, 2009, Exact Discovery of Time Series Motifs. In SDM
  31. 10.1109/ACCESS.2018.2886457 · DOI · Paper page
  32. Themis Palpanas. 2013. Real-Time Data Analytics in Sensor Networks. In Managing and Mining Sensor Data. 173--210. Themis Palpanas. 2013. Real-Time Data Analytics in Sensor Networks. In Managing and Mining Sensor Data. 173--210. · DOI
  33. 10.1145/2814710.2814719 · DOI
  34. 10.1145/3377391.3377400 · DOI
  35. 10.14778/3342263.3342648 · DOI
  36. 10.1145/2723372.2737793 · DOI · Paper page
  37. 10.1145/3044711 · DOI · Paper page
  38. 10.1145/3318464.3389760 · DOI
  39. 10.14778/2824032.2824078 · DOI · Paper page
  40. Peng Botao, 2021, Fast Data Series Indexing for In-Memory Data. VLDBJ
  41. Peng Botao, 2021, SING: Sequence Indexing Using GPUs. In ICDE.
  42. Peng Botao, 2020, Data Series Indexing on Multi-core Architectures. TKDE
  43. Scott D. W., Multivariate Density Estimation. Theory, Practice, and Visualization
  44. 10.1145/3051126 · DOI
  45. 10.5555/1182635.1164145 · doi:10.5555/1182635.1164145
  46. 10.14778/2994509.2994526 · DOI
  47. Wang J., 2013, SLPAT
  48. 10.1145/3447548.3467317 · DOI
  49. Dragomir Yankov Eamonn J. Keogh Jose Medina Bill Yuan-chi Chiu and Victor B. Zordan. [n.d.]. Detecting time series motifs under uniform scaling. In ACM. Dragomir Yankov Eamonn J. Keogh Jose Medina Bill Yuan-chi Chiu and Victor B. Zordan. [n.d.]. Detecting time series motifs under uniform scaling. In ACM.
  50. 10.1109/ICDM.2007.61 · DOI
  51. Michael Yeh Chin-Chia, 2016, Diego Furtado Silva, Abdullah Mueen, and Eamonn J. Keogh.

Cited by

See all 119

Metadata

Authors
Boniol, P., Paparrizos, J., Palpanas, T., & Franklin, M. J.
Venue
Proceedings of the VLDB Endowment
Published
2021-06
Volume
14
Issue
10
Pages
1717-1729
ISSN
2150-8097 (ISSN-L)
OpenAlex
W3176476506
Cited by
119
References
6
FWCI
5.82
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.