LogCluster - A data clustering and pattern mining algorithm for event logs
The DOI matches its Crossref record.
Abstract
Modern IT systems often produce large volumes of event logs, and event pattern discovery is an important log management task. For this purpose, data mining methods have been suggested in many previous works. In this paper, we present the LogCluster algorithm which implements data clustering and line pattern mining for textual event logs. The paper also describes an open source implementation of LogCluster.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 20
- adetokunbo, 0, Clustering Event Logs using Iterative Partitioning, Proc ACM SIGKDD Conf Knowledge Disc Data Mining 2009, 1255
- makanju, 2012, Exploring Event Log Analysis With Minimum Apriori Information
- klemettinen, 1999, A knowledge discovery methodology for telecommunication network alarm databases
- zheng, 0, Intelligent Search of Correlated Alarms from Database Containing Noise Data, Proc 2002 IEEE Network Operations and Management Symposium, 405
- 10.1109/ICDE.2001.914829 · DOI · Paper page
- 10.1007/11856214_1 · DOI
- 10.1109/MILCOM.2000.904991 · doi:10.1109/milcom.2000.904991
- 10.1109/CLUSTR.2004.1392628 · DOI · Paper page
- adetokunbo, 2008, LogView: Visualizing Event Log Clusters, Proceedings - 6th Annual Conference on Privacy Security and Trust, 99
- 10.1145/1644893.1644897 · DOI
- 10.1109/NOMS.2008.4575281 · DOI
- 10.1007/978-3-540-30179-0_27 · DOI
- thomas, 0, Identifying Symptoms of Recurrent Faults in Log Files of Distributed Information Systems, IEEE Proceedings of Network Operations and Management Symposium 2010, 187
- 10.1109/CNSM.2010.5691262 · DOI
- reidemeister, 2012, Fault Diagnosis in Enterprise Software Systems Using Discrete Monitoring Data
- thomas, 0, Mining Unstructured Log Files for Recurrent Fault Diagnosis, Proceedings of the 2011 IEEE/IFIP International Symposium on Integrated Network Management, 377
- 10.1109/IPOM.2003.1251233 · DOI · Paper page
- 10.1109/MILCOM.2014.53 · doi:10.1109/milcom.2014.53
- xu, 2008, Mining Console Logs for Large-Scale System Problem Detection, Proceedings of the 3rd Workshop on Tackling Computer Systems Problems with Machine Learning Techniques
- 10.1007/s00607-013-0282-8 · DOI
Cited by
Tools and benchmarks for automated log parsing
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00021 · Cited by 574
“…approaches have been proposed from both academia and industry, including frequent pattern mining (SLCT [20], and its extension LogCluster [21]), iterative partitioning (IPLoM [22]), hierarchical clustering (LKE [23]), longest common subsequence computation (Spell [24]), parsing tree…”
BackgroundLog-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
“There are many log parsing techniques, including frequent pattern mining [17], [18], [19], clustering [20], [21], [22], language modeling [23], heuristics [24], [25], [26], etc.”
BackgroundA Survey on Automated Log Analysis for Reliability Engineering
He, S., He, P., Chen, Z., Yang, T., Su, Y., & Lyu, M. R. (2022) · ACM Computing Surveys · doi:10.1145/3460345 · Cited by 334
Loghub: a large collection of system log datasets for AI-driven log analytics
Zhu, J., He, S., He, P., Liu, J., & Lyu, M. R. (2023) · 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre59848.2023.00071 · Cited by 302
Log-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257
Anomaly Detection Using Program Control Flow Graph Mining From Execution Logs
Nandi, A., Mandal, A., Atreja, S., Dasgupta, G. B., & Bhattacharya, S. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939712 · Cited by 154
Deep Learning or Classical Machine Learning? An Empirical Study on Log-Based Anomaly Detection
Yu, B., Yao, J., Fu, Q., Zhong, Z., Xie, H., Wu, Y., Ma, Y., & He, P. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3623308 · Cited by 65
Impact of log parsing on deep learning-based anomaly detection
Khan, Z. A., Shin, D., Bianculli, D., & Briand, L. C. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10533-w · Cited by 49
“To address this problem, many automated log-parsing approaches, which take as input log messages and identify their log templates using different heuristics, have been proposed in the literature (e.g., AEL (Jiang et al. 2008), Drain (He et al. 2017), IPLoM (Makanju et al. 2009), LenMa (Shima 2016), LFA (Nagappan and V…”
BackgroundMethodInfluentialMLog: Mogrifier LSTM-Based Log Anomaly Detection Approach Using Semantic Representation
Fu, Y., Liang, K., & Xu, J. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3289488 · Cited by 49
SXAD: Shapely eXplainable AI-Based Anomaly Detection Using Log Data
Alam, K., Kifayat, K., Sampedro, G. A., Karovič, V., & Naeem, T. (2024) · IEEE Access · doi:10.1109/access.2024.3425472 · Cited by 32
“In [30] proposed LogCluster, a system for organizing log sequences that handle atypical sequences to help developers spot issues at a glance.”
BackgroundAI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges
Cheng, Q., Sahoo, D., Saha, A., Yang, W., Liu, C., Woo, G., Singh, M., Saverese, S., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2304.04661 · Cited by 23
LogOnline: A Semi-Supervised Log-Based Anomaly Detector Aided with Online Learning Mechanism
Wang, X., Song, J., Zhang, X., Tang, J., Gao, W., & Lin, Q. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00043 · Cited by 18
The Insider Threat Detection Method of University Website Clusters Based on Machine Learning
Li, Y., & Su, Y. (2023) · 2023 6th International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd57115.2023.10206282 · Cited by 14
“…have proposed improved methods, such as Xu et al. using PCA [19] to detect large-scale system issues by mining console logs, Lou et al. [20] detecting system issues by mining invariants from console logs, and Vaarandi et al. [21] proposing an event log data clustering and pattern mining algorithm.”
MethodExplainable Anomaly Detection for System Logs in Distributed Environments
Gu, Z., Yue, W., & Liu, C. (2026) · Computers, Materials & Continua · doi:10.32604/cmc.2026.077388
“LogCluster [23] was excluded from the comparison because its requirement for centralized access to raw logs conflicts with the principle of federated data localization.”
Metadata
- Authors
- Vaarandi, R., & Pihelgas, M.
- Venue
- 2015 11th International Conference on Network and Service Management (CNSM)
- Publisher
- IEEE
- Published
- 2015-11
- Pages
- 1-7
- OpenAlex
- W2208211896
- Cited by
- 246
- References
- 0
- FWCI
- 5.85
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.