SAND: streaming subsequence anomaly detection
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
With the increasing demand for real-time analytics and decision making, anomaly detection methods need to operate over streams of values and handle drifts in data distribution. Unfortunately, existing approaches have severe limitations: they either require prior domain knowledge or become cumbersome and expensive to use in situations with recurrent anomalies of the same type. In addition, subsequence anomaly detection methods usually require access to the entire dataset and are not able to learn and detect anomalies in streaming settings. To address these problems, we propose SAND, a novel online method suitable for domain-agnostic anomaly detection. SAND aims to detect anomalies based on their distance to a model that represents normal behavior. SAND relies on a novel steaming methodology to incrementally update such model, which adapts to distribution drifts and omits obsolete data. The experimental results on several real-world datasets demonstrate that SAND correctly identifies single and recurrent anomalies without prior knowledge of the characteristics of these anomalies. SAND outperforms by a large margin the current state-of-the-art algorithms in terms of accuracy while achieving orders of magnitude speedups.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 5×
“, methods that take as input the entire data series): Isolation Forest (IF) [28], NormA [10], Series2Graph (S2G) [11], and STOMP [60]; as detailed below, we use the parameters suggested in the original papers.”
BackgroundMethodInfluentialLOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335388 · Cited by 4,149 · cited 1×
“Following this idea, general methods for multi-dimensional points outlier detection has been proposed [13, 28, 31].”
MethodDau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., & Keogh, E. (2019) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2019.1911747 · Cited by 1,128
Reliability of Scoring Respiratory Disturbance Indices and Sleep Staging
Whitney, C. W., Gottlieb, D. J., Redline, S., Norman, R. G., Dodge, R. R., Shahar, E., Surovec, S., & Nieto, F. J. (1998) · Sleep · doi:10.1093/sleep/21.7.749 · Cited by 327 · cited 1×
“length, l , we can calculate their ED , as follows [15, 35, 55, 57, 58]: ED (A, B) = √∑l i ( Ai,1−μA σA − Bi,1−μB σB ) 2, where μ and σ represent the mean and standard deviation of the sequences.”
Research works printed but not linked: 51
- D. Abboud M. Elbadaoui W.A. Smith and R.B. Randall. 2019. Advanced bearing diagnostics: A comparative study of two powerful approaches. MSSP 114 (2019). D. Abboud M. Elbadaoui W.A. Smith and R.B. Randall. 2019. Advanced bearing diagnostics: A comparative study of two powerful approaches. MSSP 114 (2019). · DOI
- 10.1177/1475921710395811 · doi:10.1177/1475921710395811
- Antoni Jerome, 2019, A statistical methodology for the design of condition indicators. Mechanical Systems and Signal Processing
- Bagnall Anthony J., 2019, Data Series Management (Dagstuhl Seminar 19282). Dagstuhl Reports (9(7)
- Bahaadini S., 2018, Machine learning for Gravity Spy: Glitch classification and dataset. Information Sciences 444 (1 5
- V. Barnet and T. Lewis. 1994. Outliers in Statistical Data. John Wiley and Sons Inc. V. Barnet and T. Lewis. 1994. Outliers in Statistical Data. John Wiley and Sons Inc.
- Boniol Paul, 2020, Automated Anomaly Detection in Large Sequences. In 36th IEEE International Conference on Data Engineering, ICDE 2020
- Boniol Paul, 2020, SAD: An Unsupervised System for Subsequence Anomaly Detection. In ICDE.
- Boniol Paul, 2021, Unsupervised and scalable subsequence anomaly detection in large data series. The VLDB Journal
- 10.14778/3407790.3407792 · DOI
- 10.14778/3415478.3415514 · DOI
- Bu Yingyi, 2007, Ada Wai-Chee Fu, Eamonn J. Keogh, Jian Pei, and Sam Meshkin.
- 10.1145/956750.956808 · DOI · Paper page
- Daouayry Nassia, Data-Centric Helicopter Failure Anticipation: The MGB Oil Pressure Virtual Sensor Case
- Yan, 2016, ICDM
- 10.1007/11811305_3 · DOI
- 10.1109/TKDE.2003.1198387 · DOI · Paper page
- Hadjem Medina, 2016, Khokhar
- 10.1109/ICDM.2005.79 · DOI · Paper page
- 10.1007/s10618-006-0049-3 · DOI
- 10.1109/ICDE.2011.5767923 · DOI · Paper page
- 10.5555/2969033.2969177 · doi:10.5555/2969033.2969177
- Linardi Michele, 2020, Keogh
- Liu Fei Tony, 2008, Kai Ming Ting, and Zhi-Hua Zhou
- 10.1007/978-3-642-00672-2_62 · DOI
- 10.5555/2022850.2022862 · doi:10.5555/2022850.2022862
- Haoran Ma Benyamin Ghojogh Maria N. Samad Dongyu Zheng and Mark Crowley. 2020. Isolation Mondrian Forest for Batch and Online Anomaly Detection. arXiv:2003.03692 [cs.LG] Haoran Ma Benyamin Ghojogh Maria N. Samad Dongyu Zheng and Mark Crowley. 2020. Isolation Mondrian Forest for Batch and Online Anomaly Detection. arXiv:2003.03692 [cs.LG] · arXiv
- Mirylenka Katsiaryna, 2013, Envelope-Based Anomaly Detection for High-Speed Manufacturing Processes. European Advanced Process Control and Manufacturing Conference
- G. B. Moody and R. G. Mark. 2001. The impact of the MIT-BIH Arrhythmia Database. IEEE Engineering in Medicine and Biology Magazine (2001). G. B. Moody and R. G. Mark. 2001. The impact of the MIT-BIH Arrhythmia Database. IEEE Engineering in Medicine and Biology Magazine (2001). · DOI · Paper page
- Mueen Abdullah, 2009, Exact Discovery of Time Series Motifs. In SDM
- 10.1109/ACCESS.2018.2886457 · DOI · Paper page
- Themis Palpanas. 2013. Real-Time Data Analytics in Sensor Networks. In Managing and Mining Sensor Data. 173--210. Themis Palpanas. 2013. Real-Time Data Analytics in Sensor Networks. In Managing and Mining Sensor Data. 173--210. · DOI
- 10.1145/2814710.2814719 · DOI
- 10.1145/3377391.3377400 · DOI
- 10.14778/3342263.3342648 · DOI
- 10.1145/2723372.2737793 · DOI · Paper page
- 10.1145/3044711 · DOI · Paper page
- 10.1145/3318464.3389760 · DOI
- 10.14778/2824032.2824078 · DOI · Paper page
- Peng Botao, 2021, Fast Data Series Indexing for In-Memory Data. VLDBJ
- Peng Botao, 2021, SING: Sequence Indexing Using GPUs. In ICDE.
- Peng Botao, 2020, Data Series Indexing on Multi-core Architectures. TKDE
- Scott D. W., Multivariate Density Estimation. Theory, Practice, and Visualization
- 10.1145/3051126 · DOI
- 10.5555/1182635.1164145 · doi:10.5555/1182635.1164145
- 10.14778/2994509.2994526 · DOI
- Wang J., 2013, SLPAT
- 10.1145/3447548.3467317 · DOI
- Dragomir Yankov Eamonn J. Keogh Jose Medina Bill Yuan-chi Chiu and Victor B. Zordan. [n.d.]. Detecting time series motifs under uniform scaling. In ACM. Dragomir Yankov Eamonn J. Keogh Jose Medina Bill Yuan-chi Chiu and Victor B. Zordan. [n.d.]. Detecting time series motifs under uniform scaling. In ACM.
- 10.1109/ICDM.2007.61 · DOI
- Michael Yeh Chin-Chia, 2016, Diego Furtado Silva, Abdullah Mueen, and Eamonn J. Keogh.
Cited by
TranAD: deep transformer networks for anomaly detection in multivariate time series data
Tuli, S., Casale, G., & Jennings, N. R. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3514061.3514067 · Cited by 1,063
Anomaly detection in time series: a comprehensive evaluation
Schmidl, S., Wenig, P., & Papenbrock, T. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3538598.3538602 · Cited by 590
ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection
Chen, Y., Zhang, C., Ma, M., Liu, Y., Ding, R., Li, B., He, S., Rajmohan, S., Lin, Q., & Zhang, D. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3632093.3632101 · Cited by 139
A survey on outlier explanations
Panjei, E., Gruenwald, L., Leal, E., Nguyen, C., & Silvia, S. (2022) · The VLDB Journal · doi:10.1007/s00778-021-00721-1 · Cited by 67
Smart Anomaly Detection Using Data-Driven Techniques in IoT Edge: A Survey
Manokaran, J., & Vairavel, G. (2022) · Lecture Notes in Electrical Engineering · doi:10.1007/978-981-16-8862-1_45 · Cited by 13
Metadata
- Authors
- Boniol, P., Paparrizos, J., Palpanas, T., & Franklin, M. J.
- Venue
- Proceedings of the VLDB Endowment
- Published
- 2021-06
- Volume
- 14
- Issue
- 10
- Pages
- 1717-1729
- ISSN
- 2150-8097 (ISSN-L)
- OpenAlex
- W3176476506
- Cited by
- 119
- References
- 6
- FWCI
- 5.82
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SAND: streaming subsequence anomaly detection. https://app.cochat.ai/papers/10.14778/3467861.3467863
- The paper
- Boniol, P., Paparrizos, J., Palpanas, T., & Franklin, M. J. (2021). SAND: streaming subsequence anomaly detection. Proceedings of the VLDB Endowment. https://doi.org/10.14778/3467861.3467863
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.