The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
LogSig: generating system events from raw textual logs
Abstract
Modern computing systems generate large amounts of log data. System administrators or domain experts utilize the log data to understand and optimize system behaviors. Most system logs are raw textual and unstructured. One main fundamental challenge in automated log analysis is the generation of system events from raw textual logs. Log messages are relatively short text messages but may have a large vocabulary, which often result in poor performance when applying traditional text clustering techniques to the log data. Other related methods have various limitations and only work well for some particular system logs. In this paper, we propose a message signature based algorithm logSig to generate system events from textual log messages. By searching the most representative message signatures, logSig categorizes log messages into a set of event types. logSig can handle various types of log data, and is able to incorporate human's domain knowledge to achieve a high performance. We conduct experiments on five real system log data. Experiments show that logSig outperforms other alternative algorithms in terms of the overall performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Log parsers could produce inconsistent results depend on the preprocessing step and the set of parameters [16], [15].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [16]
In the citing paper
“The log parsing accuracy highly influences the performance of log mining [16].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [16]
Citations
Cites
Data mining: concepts and techniques
(2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,531
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 11,392
(2008) · Data Mining and Market Intelligence for Optimal Marketing Returns · doi:10.4324/9780080878096-12 · Cited by 6,954
Introduction to modern information retrieval
Dillon, M. (1983) · Information Processing & Management · doi:10.1016/0306-4573(83)90062-6 · Cited by 5,909
An Introduction to Data Mining
Stephens, C. R., & Sukumar, R. (2006) · The Handbook of Marketing Research · doi:10.4135/9781412973380.n22 · Cited by 4,562
Constrained K-means Clustering with Background Knowledge
Wagstaff, K. L., Cardie, C., Rogers, S., & Schrödl, S. (2001) · Cited by 2,469
Introduction to Data Mining, (First Edition)
Tan, P., Steinbach, M. S., & Kumar, V. (2005) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,889
Integrating constraints and metric learning in semi-supervised clustering
Bilenko, M., Basu, S., & Mooney, R. J. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015360 · Cited by 849
Hamerly, G., & Elkan, C. (2003) · Cited by 841
What Supercomputers Say: A Study of Five System Logs
Oliner, A., & Stearley, J. (2007) · 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07) · doi:10.1109/dsn.2007.103 · Cited by 620
Research works printed but not linked: 4
- 10.5555/3121576.3121612 · doi:10.5555/3121576.3121612
- 10.5555/1169220.1169382 · doi:10.5555/1169220.1169382
- 10.5555/1111682.1111732 · doi:10.5555/1111682.1111732
- S. Theodoridis and r. e. Konstantinos Koutroumbas. Pattern Recognition . Academic Press , 2006 . S. Theodoridis and r. e. Konstantinos Koutroumbas. Pattern Recognition. Academic Press, 2006.
Cited by
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835
Drain: an online log parsing approach with fixed depth tree
He, P., Zhu, J., Zheng, Z., & Lyu, M. R. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.13 · Cited by 943
“Thus, for the other four data sets, as with the existing work [20], [25], we evaluate LKE’s accuracy on sample data sets with 2k log messages randomly extracted from the original ones, while all parsers are evaluated on the 2k sample data sets in our previous paper [20].”
MethodRobust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761
Experience Report: System Log Analysis for Anomaly Detection
He, S., Zhu, J., He, P., & Lyu, M. R. (2016) · 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2016.21 · Cited by 593
Tools and benchmarks for automated log parsing
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00021 · Cited by 574
Spell: Streaming Parsing of System Event Logs
Du, M., & Li, F. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0103 · Cited by 423
Loghub: a large collection of system log datasets for AI-driven log analytics
Zhu, J., He, S., He, P., Liu, J., & Lyu, M. R. (2023) · 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre59848.2023.00071 · Cited by 302
A Survey on Automated Log Analysis for Reliability Engineering
He, S., He, P., Chen, Z., Yang, T., Su, Y., & Lyu, M. R. (2022) · ACM Computing Surveys · doi:10.1145/3460345 · Cited by 334
Log-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
“He et al [16] evaluated four widely used log parsers, including SLCT [57], IPLoM [26], LKE [58] and LogSig [20].”
BackgroundMethodLog-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257
Towards Automated Log Parsing for Large-Scale Log Data Analysis
He, P., Zhu, J., He, S., Li, J., & Lyu, M. R. (2018) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2017.2762673 · Cited by 230
Log Parsing with Prompt-based Few-shot Learning
Le, V.-H., & Zhang, H. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00204 · Cited by 129
Root-Cause Metric Location for Microservice Systems via Log Anomaly Detection
Wang, L., Zhao, N., Chen, J., Li, P., Zhang, W., & Sui, K. (2020) · 2020 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws49710.2020.00026 · Cited by 84
An empirical investigation of practical log anomaly detection for online service systems
Zhao, N., Wang, H., Li, Z., Peng, X., Wang, G., Pan, Z., Wu, Y., Feng, Z., Wen, X., Zhang, W., et al. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3473933 · Cited by 77
Jia, T., Wu, Y., Hou, C., & Li, Y. (2021) · 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre52982.2021.00021 · Cited by 55
Impact of log parsing on deep learning-based anomaly detection
Khan, Z. A., Shin, D., Bianculli, D., & Briand, L. C. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10533-w · Cited by 49
Zhang, C., Wang, X., Zhang, H., Zhang, H., & Han, P. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3125967 · Cited by 49
AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges
Cheng, Q., Sahoo, D., Saha, A., Yang, W., Liu, C., Woo, G., Singh, M., Saverese, S., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2304.04661 · Cited by 23
Metadata
- Authors
- Tang, L., Li, T., & Perng, C.-S.
- Affiliations
- Florida International University (US)
- IBM (United States) (US)
- ORCID
- Venue
- Proceedings of the 20th ACM international conference on Information and knowledge management
- Publisher
- ACM
- Published
- 2011-10-24
- Pages
- 785-794
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W2153470728
- Cited by
- 244
- References
- 29
- Citations by year
- 2026: 12 · 2025: 25 · 2024: 29 · 2023: 34 · 2022: 20 · 2021: 46 · 2020: 35 · 2019: 14 · 2018: 12 · 2017: 8 · 2016: 4 · 2013: 3 · 2012: 2
- FWCI
- 1.42
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Information and Knowledge Management (CIKM) (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- log analysis, document clustering, system logs, domain knowledge integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.