The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark
Open accessAbstract
Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations, examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687
A symbolic representation of time series, with implications for streaming algorithms
Lin, J., Keogh, E., Lonardi, S., & Chiu, B. (2003) · Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery · doi:10.1145/882082.882086 · Cited by 1,799
Percentage Points for a Generalized ESD Many-Outlier Procedure
Rosner, B. (1983) · Technometrics · doi:10.1080/00401706.1983.10487848 · Cited by 1,268
Hawkins, J., & Blakeslee, S. (2004) · Cited by 1,012
Musa, T. H. A., & Bouras, A. (2022) · Lecture Notes in Networks and Systems · doi:10.1007/978-981-16-2102-4_36 · Cited by 958
HOT SAX: Efficiently Finding the Most Unusual Time Series Subsequence
Keogh, E., Lin, J., & Fu, A. (2006) · Fifth IEEE International Conference on Data Mining (ICDM'05) · doi:10.1109/icdm.2005.79 · Cited by 696
A symbolic representation of time series, with implications for streaming algorithms
Lin, J., Keogh, E., Lonardi, S., & Chiu, B. (2003) · doi:10.1145/882085.882086 · Cited by 642
Generic and Scalable Framework for Automated Time-series Anomaly Detection
Laptev, N., Amizadeh, S., & Flint, I. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788611 · Cited by 425
Toward Credible Evaluation of Anomaly-Based Intrusion-Detection Methods
Tavallaee, M., Stakhanova, N., & Ghorbani, A. A. (2010) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2010.2048428 · Cited by 280
Comparative Evaluation of Anomaly Detection Techniques for Sequence Data
Chandola, V., Mithal, V., & Kumar, V. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.151 · Cited by 167
Finding anomalous periodic time series: An application to catalogs of periodic variable stars
Rebbapragada, U. D., Protopapas, P., Brodley, C. E., & Alcock, C. R. (2008) · Machine Learning · doi:10.1007/s10994-008-5093-3 · Cited by 166
Serdio, F., Lughofer, E. D., Pichler, K., Buchegger, T., Pichler, M., & Efendic, H. (2014) · Information Fusion · doi:10.1016/j.inffus.2014.03.006 · Cited by 97
Cited by
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219845 · Cited by 1,771
Unsupervised real-time anomaly detection for streaming data
Ahmad, S., Lavin, A., Purdy, S., & Agha, Z. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.04.070 · Cited by 995
Self-citationA Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series
Munir, M., Siddiqui, S. A., Dengel, A., & Ahmed, S. (2019) · IEEE Access · doi:10.1109/access.2018.2886457 · Cited by 696
40 years of cognitive architectures: core cognitive abilities and practical applications
Kotseruba, I., & Tsotsos, J. K. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-018-9646-y · Cited by 539
Metadata
- Authors
- Lavin, A., & Ahmad, S.
- Venue
- 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 38-44
- OpenAlex
- W2191950414
- Cited by
- 457
- References
- 12
- FWCI
- 11.93
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.