LogSed: Anomaly Diagnosis through Mining Time-Weighted Control Flow Graph in Logs
The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
Detecting execution anomalies is very important to monitoring and maintenance of cloud systems. People often use execution logs for troubleshooting and problem diagnosis, which is time consuming and error-prone. There is great demand for automatic anomaly detection based on logs. In this paper, we mine a time-weighted control flow graph (TCFG) that captures healthy execution flows of each component in cloud, and automatically raise anomaly alerts on observing deviations from TCFG. We outlined three challenges that are solved in this paper, including how to deal with the interleaving of multiple threads in logs, how to identify operational logs that do not contain any transactional information, and how to split the border of each transaction flow in the TCFG. We evaluate the effectiveness of our approach by leveraging logs from an IBM public cloud production platform and two simulated systems in the lab environment. The evaluation results show that our TCFG mining and anomaly diagnosis both perform over 80% precision and recall on average.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332 · cited 1×
“Other works ([5][6]) leverage a white-box approach where source code is available and a duplicated controllable environment is available.”
Background
Cited by
A Survey on Automated Log Analysis for Reliability Engineering
He, S., He, P., Chen, Z., Yang, T., Su, Y., & Lyu, M. R. (2022) · ACM Computing Surveys · doi:10.1145/3460345 · Cited by 334
Soldani, J., & Brogi, A. (2023) · ACM Computing Surveys · doi:10.1145/3501297 · Cited by 256
“This technique is adopted by OASIS [58], Jia et al. [33], and LogSed et al. [34], which we discuss hereafter.”
MethodInfluentialQiu, J., Du, Q., Yin, K., Zhang, S.-L., & Qian, C. (2020) · Applied Sciences · doi:10.3390/app10062166 · Cited by 96
“According to the research in [9–11], the log mining method also plays an important role in the RCA of cloud applications.”
ResultAn Approach for Anomaly Diagnosis Based on Hybrid Graph Model with Logs for Distributed Services
Jia, T., Chen, P., Yang, L., Li, Y., Meng, F., & Xu, J. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.12 · Cited by 78
Self-citationJia, T., Wu, Y., Hou, C., & Li, Y. (2021) · 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre52982.2021.00021 · Cited by 55
Self-citationLogDC: Problem Diagnosis for Declartively-Deployed Cloud Applications with Log
Xu, J., Chen, P., Yang, L., Meng, F., & Wang, P. (2017) · 2017 IEEE 14th International Conference on e-Business Engineering (ICEBE) · doi:10.1109/icebe.2017.52 · Cited by 39
“We have implemented an advanced template mining approach in our previous work [6, 12].”
MethodSelf-citationAnomaly Detection in Microservice-Based Systems
Nobre, J., Pires, E. J. S., & Reis, A. (2023) · Applied Sciences · doi:10.3390/app13137891 · Cited by 34
SXAD: Shapely eXplainable AI-Based Anomaly Detection Using Log Data
Alam, K., Kifayat, K., Sampedro, G. A., Karovič, V., & Naeem, T. (2024) · IEEE Access · doi:10.1109/access.2024.3425472 · Cited by 32
“In addition, Jia et al. [25], proposed anomaly diagnosis through a method called ‘‘Logsed,’’ which involves mining control graphs that are time-weighted in transactional operational logs.”
BackgroundMethodAcLog: An Approach to Detecting Anomalies from System Logs with Active Learning
Duan, C., Jia, T., Li, Y., & Huang, G. (2023) · 2023 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws60048.2023.00062 · Cited by 20
Self-citationAnalysis of Log Files to Enable Smart-Troubleshooting in Industry 4.0: A Systematic Mapping Study
Partovian, S., Bucaioni, A., Flammini, F., & Thornadtsson, J. (2024) · IEEE Access · doi:10.1109/access.2023.3342365 · Cited by 16
Failures and fixes: a study of software system incident response
Sillito, J., & Kutomi, E. (2020) · 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme46990.2020.00027 · Cited by 14
A Trace-Log-Clusterings-Based Fault Localization Approach to Microservice Systems
Sun, C.-A., Zeng, T., Zuo, W., & Liu, H. (2023) · 2023 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws60048.2023.00013 · Cited by 7
“[16], [17] proposed an approach to mine and cluster log templates to build a time-weighted control flow graph for each service, enabling anomaly detection by comparing actual behavior with the expected behavior modeled in the graph.”
MethodToward Context-Aware Anomaly Detection for AIOps in Microservices Using Dynamic Knowledge Graphs
Moens, P., Steenwinckel, B., Ongenae, F., Volckaert, B., & Hoecke, S. V. (2026) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2026.3652304 · Cited by 6
“They used these time-weighted graphs to detect deviations from normal application behavior by matching captured interval times with the previously established time weights in the learned normal CFGs. LogSed [14] uses a benchmark microservice application, namely Acme Air [15].”
Metadata
- Authors
- Jia, T., Yang, L., Chen, P., Li, Y., Meng, F., & Xu, J.
- Venue
- 2017 IEEE 10th International Conference on Cloud Computing (CLOUD)
- Publisher
- IEEE
- Published
- 2017-06
- Pages
- 447-455
- ISSN
- 2159-6190 (ISSN-L)
- OpenAlex
- W2755791218
- Cited by
- 86
- References
- 1
- FWCI
- 4.60
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.