CoChat
Sign Up
Cited by 78

The DOI matches its Crossref record.

An Approach for Anomaly Diagnosis Based on Hybrid Graph Model with Logs for Distributed Services

Jia, Tong, Chen, Pengfei, Yang, Lin, Li, Ying, Meng, Fanjing, Xu, Jingmin·2017·2017 IEEE International Conference on Web Services (ICWS)
DOI 10.1109/icws.2017.12
Open this paper in CoChat — free account

Abstract

Detecting runtime anomalies is very important to monitoring and maintenance of distributed services. People often use execution logs for troubleshooting and problem diagnosis manually, which is time consuming and error-prone. In this paper, we propose an approach for automatic anomaly detection based on logs. We first mine a hybrid graph model that captures normal execution flows inter and intra services, and then raise anomaly alerts on observing deviations from the hybrid model. We evaluate the effectiveness of our approach by leveraging logs from an IBM public cloud production platform and two simulated systems in the lab environment. Evaluation results show that our hybrid graph model mining performs over 80% precision and 70% recall and anomaly detection performs nearly 90% precision and 80% recall on average.

Overview

Integrity signals

Registry

Crossref · 10.1109/icws.2017.12

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 34

  1. 10.1109/IPOM.2003.1251233 · DOI
  2. 10.1109/CLOUD.2017.64 · DOI
  3. spirtes, 1993, Causation, prediction, and search[J], Lecture Notes in Statistics, 81, 272
  4. causality, 2011, Models reasoning and inference, 79, 289
  5. 10.1007/BFb0014140 · DOI
  6. 10.1145/1740390.1740411 · DOI
  7. animesh, 2016, Anomaly Detection Using Program Control Flow Graph Mining From Execution Logs[C], SIGKDD
  8. deraalst, 2004, workflow mining: Fdis Discovering process models from event logs[C], TKDE
  9. gunther, 2007, Fuzzy mining-adaptive process simplification based on multi-perspective metrics[C], BPM
  10. 10.1145/1835804.1835883 · DOI
  11. levenshtein, 1965, Binary codes capable of correcting deletions, insertions and reversals [J], Problems of Information Transmission, 707
  12. ester, 1996, A Density-Based Algorithm for Discovering Clusters, Large Spatial Databases with Noise[C], 226
  13. 10.1145/2038633.2038636 · DOI
  14. 0, Hadoop
  15. kalisch, 2007, Estimating high-dimensional directed acyclic graphs with the pc-algorithm[J], Mach Learn Res, 8, 613
  16. chen, 2008, Automating network application dependency discovery: Experiences, limitations, and new solutions[C], Proc 8th USENIX Conf Operating Syst Des Implementation, 117
  17. wei, 2010, Detecting large-scale system problems by mining console logs[C], International Conference on Machine Learning, 37
  18. barham, 2004, Using magpie for request extraction and workload modelling[C], Proc 6th Conf Symp Opearting Syst Des Implementation, 18
  19. 10.1109/ICDM.2009.19 · DOI
  20. 10.1145/2872362.2872407 · DOI
  21. barham, 2008, Constellation Automated Discovery of Service and Host Dependencies in Networked Systems
  22. lin, 2016, Log Clustering based Problem Identification for Online Service Systems[C], International Conference on Software Engineering
  23. 0, OpenStack
  24. 10.1109/IC2E.2016.12 · DOI
  25. lim, 2008, A log mining approach to failure analysis of enterprise telephony systems[C], Dependable Systems and Networks With FTCS and DCC 2008 DSN 2008 IEEE International Conference on, 398
  26. 0, Cloud Foundry
  27. 10.1109/ICDM.2009.60 · DOI
  28. 10.1198/016214508000000193 · DOI
  29. 10.1109/DSN.2005.44 · DOI
  30. cohen, 2004, Correlating instrumentation data to system states: A building block for automated diagnosis and control[C], Proc 6th Conf Symp Opearting Syst Des Implementation, 16
  31. bhlmann, 2008, Robustification of the pc-algorithm for directed acyclic graphs[J], Journal of Computational & Graphical Statistics, 17, 773, 10.1198/106186008X381927 · DOI
  32. cohen, 2005, Capturing, indexing, clustering, and retrieving system history[C], ACM SIGOPS Operating Syst Rev, 39, 105, 10.1145/1095809.1095821 · DOI
  33. fonseca, 2007, X-trace: A pervasive network tracing framework, Proc 4th USENIX Conf Netw Syst Des Implementation, 20
  34. 10.1109/TSC.2016.2607739 · DOI

Cited by

Sorted by citations

Metadata

Authors
Jia, T., Chen, P., Yang, L., Li, Y., Meng, F., & Xu, J.
Venue
2017 IEEE International Conference on Web Services (ICWS)
Publisher
IEEE
Published
2017-06
Pages
25-32
DOI
10.1109/icws.2017.12
Cited by
78
References
0

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review