The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
LogSig: generating system events from raw textual logs
Abstract
Modern computing systems generate large amounts of log data. System administrators or domain experts utilize the log data to understand and optimize system behaviors. Most system logs are raw textual and unstructured. One main fundamental challenge in automated log analysis is the generation of system events from raw textual logs. Log messages are relatively short text messages but may have a large vocabulary, which often result in poor performance when applying traditional text clustering techniques to the log data. Other related methods have various limitations and only work well for some particular system logs. In this paper, we propose a message signature based algorithm logSig to generate system events from textual log messages. By searching the most representative message signatures, logSig categorizes log messages into a set of event types. logSig can handle various types of log data, and is able to incorporate human's domain knowledge to achieve a high performance. We conduct experiments on five real system log data. Experiments show that logSig outperforms other alternative algorithms in terms of the overall performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Log parsers could produce inconsistent results depend on the preprocessing step and the set of parameters [16], [15].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [16]
In the citing paper
“The log parsing accuracy highly influences the performance of log mining [16].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [16]
Citations
Cites
Data mining: concepts and techniques
(2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,531
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 11,392
(2008) · Data Mining and Market Intelligence for Optimal Marketing Returns · doi:10.4324/9780080878096-12 · Cited by 6,954
Introduction to modern information retrieval
Dillon, M. (1983) · Information Processing & Management · doi:10.1016/0306-4573(83)90062-6 · Cited by 5,909
An Introduction to Data Mining
Stephens, C. R., & Sukumar, R. (2006) · The Handbook of Marketing Research · doi:10.4135/9781412973380.n22 · Cited by 4,562
Constrained K-means Clustering with Background Knowledge
Wagstaff, K. L., Cardie, C., Rogers, S., & Schrödl, S. (2001) · Cited by 2,469
Introduction to Data Mining, (First Edition)
Tan, P., Steinbach, M. S., & Kumar, V. (2005) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,889
Integrating constraints and metric learning in semi-supervised clustering
Bilenko, M., Basu, S., & Mooney, R. J. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015360 · Cited by 849
Hamerly, G., & Elkan, C. (2003) · Cited by 841
What Supercomputers Say: A Study of Five System Logs
Oliner, A., & Stearley, J. (2007) · 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07) · doi:10.1109/dsn.2007.103 · Cited by 620
The Complexity of Some Problems on Subsequences and Supersequences
Maier, D. (1978) · Journal of the ACM · doi:10.1145/322063.322075 · Cited by 616
Pattern Recognition, Third Edition
Theodoridis, S., & Koutroumbas, K. (2006) · Academic Press, Inc. eBooks · Cited by 582
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244302760200687 · Cited by 526
Clustering event logs using iterative partitioning
Makanju, A. A. O., Zincir-Heywood, A. N., & Milios, E. E. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557154 · Cited by 338
Towards informatic analysis of syslogs
Stearley, J. (2005) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/clustr.2004.1392628 · Cited by 117
Discovering actionable patterns in event data
Hellerstein, J. L., Ma, S., & Perng, C.-S. (2002) · IBM Systems Journal · doi:10.1147/sj.413.0475 · Cited by 106
Self-citationMining console logs for large-scale system problem detection
Xu, W., Huang, L., Fox, A., Patterson, D. A., & Jordan, M. I. (2008) · Cited by 105
Alert Detection in System Logs
Oliner, A. J., Aiken, A., & Stearley, J. (2008) · Proceedings · doi:10.1109/icdm.2008.132 · Cited by 88
An integrated framework on mining logs files for computing system management
Li, T., Liang, F., Ma, S., & Peng, W. (2005) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1081870.1081972 · Cited by 73
Self-citationOne Graph Is Worth a Thousand Logs: Uncovering Hidden Structures in Massive System Event Logs
Aharon, M., Barash, G., Cohen, I., & Mordechai, E. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04180-8_32 · Cited by 70
Constructing comprehensive summaries of large event sequences
Kiernan, J., & Terzi, E. (2009) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1631162.1631169 · Cited by 58
LogTree: A Framework for Generating System Events from Raw Textual Logs
Tang, L., & Li, T. (2010) · Proceedings · doi:10.1109/icdm.2010.76 · Cited by 53
Self-citationAn algorithmic approach to event summarization
Wang, P., Wang, H., Majin, L., & Wang, W. (2010) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1807167.1807189 · Cited by 47
Gao, J., Jiang, G., Chen, H., & Han, J. (2009) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2009.56 · Cited by 42
Event summarization for system management
Peng, W., Perng, C., Li, T., et al. (2007) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281305 · Cited by 33
Self-citationConstructing comprehensive summaries of large event sequences
Kiernan, J., & Terzi, E. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1401943 · Cited by 27
Incremental learning of system log formats
Zhu, K. Q., Fisher, K. S., & Walker, D. (2010) · ACM SIGOPS Operating Systems Review · doi:10.1145/1740390.1740410 · Cited by 27
RNA multiple structural alignment with longest common subsequences
Bereg, S., Kubica, M., Waleń, T., & Zhu, B.-H. (2006) · Journal of combinatorial optimization · doi:10.1007/s10878-006-9020-x · Cited by 26
Ning, K., Ng, H. K., & Leong, H. (2006) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2006.253315 · Cited by 12
Research works printed but not linked: 4
- 10.5555/3121576.3121612 · doi:10.5555/3121576.3121612
- 10.5555/1169220.1169382 · doi:10.5555/1169220.1169382
- 10.5555/1111682.1111732 · doi:10.5555/1111682.1111732
- S. Theodoridis and r. e. Konstantinos Koutroumbas. Pattern Recognition . Academic Press , 2006 . S. Theodoridis and r. e. Konstantinos Koutroumbas. Pattern Recognition. Academic Press, 2006.
Cited by
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835
Drain: an online log parsing approach with fixed depth tree
He, P., Zhu, J., Zheng, Z., & Lyu, M. R. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.13 · Cited by 943
“Thus, for the other four data sets, as with the existing work [20], [25], we evaluate LKE’s accuracy on sample data sets with 2k log messages randomly extracted from the original ones, while all parsers are evaluated on the 2k sample data sets in our previous paper [20].”
MethodRobust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761
Experience Report: System Log Analysis for Anomaly Detection
He, S., Zhu, J., He, P., & Lyu, M. R. (2016) · 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2016.21 · Cited by 593
Tools and benchmarks for automated log parsing
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00021 · Cited by 574
Metadata
- Authors
- Tang, L., Li, T., & Perng, C.-S.
- Affiliations
- Florida International University (US)
- IBM (United States) (US)
- ORCID
- Venue
- Proceedings of the 20th ACM international conference on Information and knowledge management
- Publisher
- ACM
- Published
- 2011-10-24
- Pages
- 785-794
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W2153470728
- Cited by
- 244
- References
- 29
- Citations by year
- 2026: 12 · 2025: 25 · 2024: 29 · 2023: 34 · 2022: 20 · 2021: 46 · 2020: 35 · 2019: 14 · 2018: 12 · 2017: 8 · 2016: 4 · 2013: 3 · 2012: 2
- FWCI
- 1.42
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Information and Knowledge Management (CIKM) (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- log analysis, document clustering, system logs, domain knowledge integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.