Anomaly Detection and Classification using Distributed Tracing and Deep Learning
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Artificial Intelligence for IT Operations (AIOps) combines big data and machine learning to replace a broad range of IT Operations tasks including availability, performance, and monitoring of services. By exploiting log, tracing, metric, and network data, AIOps enable detection of faults and issues of services. The focus of this work is on detecting anomalies based on distributed tracing records that contain detailed information for the availability and the response time of the services. In large-scale distributed systems, where a service is deployed on heterogeneous hardware and has multiple scenarios of normal operation, it becomes challenging to detect such anomalous cases. We address the problem by proposing unsupervised, response time anomaly detection based on deep learning data modeling techniques; unsupervised dynamic error threshold approach; tolerance module for false positive reduction; and descriptive classification of the anomalies. The evaluation shows that the approach achieves high accuracy and solid performance in both, experimental testbed and large-scale production cloud.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 1×
“[22] F.”
An efficient k-means clustering algorithm: analysis and implementation
Kanungo, T., Mount, D. M., Netanyahu, N. S., Piatko, C. D., Silverman, R., & Wu, A. Y. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017616 · Cited by 5,660 · cited 1×
“[21] T.”
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219845 · Cited by 1,771 · cited 1×
“Furthermore, Hundman et al. [27] show the use of LSTM recurrent neural networks for spacecraft anomalies on multi-variate telemetry data.”
Background
Cited by
Soldani, J., & Brogi, A. (2023) · ACM Computing Surveys · doi:10.1145/3501297 · Cited by 256
“[60] DT supervised learning P S previous and runtime traces Bogatinovski et al.”
BackgroundMethodInfluentialUnsupervised Detection of Microservice Trace Anomalies through Service-Level Deep Bayesian Networks
Liu, P., Xu, H., Ouyang, Q., Jiao, R., Chen, Z., Zhang, S., Yang, J., Mo, L., Zeng, J., Xue, W., et al. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00014 · Cited by 252
“However, most previous works [4]–[8], [30] do not try a unified detection approach.”
BackgroundMethodInfluentialA survey of AIOps methods for failure management
Notaro, P., Cardoso, J., & Gerndt, M. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3483424 · Cited by 142
Self-citationTraceGra: A trace-based anomaly detection for microservice using graph deep learning
Chen, J., Liu, F., Jiang, J., Zhong, G., Xu, D., Tan, Z., & Shi, S. (2023) · Computer Communications · doi:10.1016/j.comcom.2023.03.028 · Cited by 55
Self-Supervised Anomaly Detection from Distributed Traces
Bogatinovski, J., Nedelkoski, S., Cardoso, J., & Kao, O. (2020) · 2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc48980.2020.00054 · Cited by 50
Self-citation
Metadata
- Authors
- Nedelkoski, S., Cardoso, J., & Kao, O.
- Venue
- 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 241-250
- OpenAlex
- W2955169411
- Cited by
- 84
- References
- 7
- FWCI
- 7.79
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.