DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Anomaly detection is a critical step towards building a secure and trustworthy system. The primary purpose of a system log is to record system states and significant events at various critical points to help debug system failures and perform root cause analysis. Such log data is universally available in nearly all computer systems. Log data is an important and valuable resource for understanding system status and performance issues; therefore, the various system logs are naturally excellent source of information for online monitoring and anomaly detection. We propose DeepLog, a deep neural network model utilizing Long Short-Term Memory (LSTM), to model a system log as a natural language sequence. This allows DeepLog to automatically learn log patterns from normal execution, and detect anomalies when log patterns deviate from the model trained from log data under normal execution. In addition, we demonstrate how to incrementally update the DeepLog model in an online fashion so that it can adapt to new log patterns over time. Furthermore, DeepLog constructs workflows from the underlying system log so that once an anomaly is detected, users can diagnose the detected anomaly and perform root cause analysis effectively. Extensive experimental evaluations over large log data have shown that DeepLog has outperformed other existing log-based anomaly detection methods based on traditional data mining methodologies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Besides, Log2Vec adopts word2vec-based model that ignores the contextual information in sentences [47], thus it cannot fully understand the semantic meaning of log messages.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [47]
In the citing paper
“Since DeepLog represents log messages as the indexes of log templates, it cannot prevent semantic information loss and could produce many wrong detection results [47], [48].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [47]
In the citing paper
“Techniques such as DeepLog [67] have been adapted to compute anomaly scores in microservice systems [68]”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [67]
In the citing paper
“Its paradigm has shifted from traditional rule matching to deep Natural Language Processing (NLP) models, represented by DeepLog [12] and LogRobust [13] .”
From A Survey of Anomaly Detection and Root Cause Analysis Techniques for Artificial Intelligence for IT Operations (AIOps) in Large-Scale Software Systems (2026) · cites this paper as [12]
In the citing paper
“DeepLog[27] is an early representative in this field; it templates logs into Log Keys and uses LSTMs to learn the transition probabilities of normal sequences, treating deviations from predictions as anomalies. However, it can only provide anomaly judgments and lacks deep semantic extraction from semi-structured texts.”
From From Correlation to Intervention: A Survey on Log-Based Causal Analysis for Security and AIOps (2026) · cites this paper as [27]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
(2016) · Cited by 11,679
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 2×
“In recent years, neural language models that use recurrent neural networks have been shown to be highly effective across various NLP tasks [3, 25] language model, a LSTM-based one can encode more intricate patterns and maintain long-range state over a sequence [34].”
BackgroundInfluentialA neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
LSTM neural networks for language modeling
Sundermeyer, M., Schlüter, R., & Ney, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-65 · Cited by 1,963 · cited 1×
“In recent years, neural language models that use recurrent neural networks have been shown to be highly effective across various NLP tasks [3, 25] language model, a LSTM-based one can encode more intricate patterns and maintain long-range state over a sequence [34].”
BackgroundDetecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332 · cited 10×
“Principal Component Analysis (PCA) method [39] assumes that there are different “sessions” in a log file that can be easily identified by a session id attached to each log entry.”
BackgroundMethodInfluentialA Primer on Neural Network Models for Natural Language Processing
Goldberg, Y. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4992 · Cited by 1,201
Research works printed but not linked: 1
- Elastic Stack. 2017. The Open Source Elastic Stack. (2017). https://www.elastic.co/products[Online; accessed 16-May-2017].
Cited by
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
“ch Campus [88] 317,398 frames N/A 5.38% Video [88] Video surveillance UCF-Crime [146] 1,900 videos (13.8m frames) N/A 13 crimes Video [146] System log analysis HDFS Log [165] 11.2m N/A 2.90% Sequence [41] System log analysis OpenStack log 1.3m N/A 7.00% Sequence [41] 8 CONCLUSIONS AND FUTURE OPPORTUNITIES In this work…”
MethodInfluentialDeep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240
“Several studies have been conducted to understand the theoretical and practical applications of deep learning in medical and bio-informatics (Min et al. [2017], Cao et al. [2018a], Zhao et al. [2016], Khan and Yairi [2018]).”
BackgroundMethodPyTorch distributed: experiences on accelerating data parallel training
Li, S., Zhao, Y., Varma, R., Salpekar, O., Noordhuis, P., Li, T., Paszke, A., Smith, J., Vaughan, B., Damania, P., et al. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3415478.3415530 · Cited by 465
Robust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761
Machine Learning for Anomaly Detection: A Systematic Review
Nassif, A. B., Talib, M. A., Nasir, Q., & Dakalbab, F. M. (2021) · IEEE Access · doi:10.1109/access.2021.3083060 · Cited by 748
DÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663
“Most applications leverage Long Short-Term Memory (LSTM) networks for detecting anomalies in time series [45], aircraft data [46] or system logs [47].”
BackgroundTools and benchmarks for automated log parsing
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00021 · Cited by 574
“With close collaborations with our industrial partners, as well as some peer researchers (authors from [10], [18], [30]), we collect a large set of logs (over 70GB in total) produced by 16 different systems spanning distributed systems, supercomputers, operating systems, mobile systems, server…”
BackgroundInfluentialProgress in Outlier Detection Techniques: A Survey
Wang, H., Bah, M. J., & Hammad, M. (2019) · IEEE Access · doi:10.1109/access.2019.2932769 · Cited by 523
Robust Deep Learning Methods for Anomaly Detection
Chalapathy, R., Khoa, N. L. D., & Chawla, S. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3406704 · Cited by 502
LogBERT: Log Anomaly Detection via BERT
Guo, H., Yuan, S., & Wu, X. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534113 · Cited by 454
“Regarding baselines, we leverage the package Loglizer [4] to evaluate PCA, OCSVM, iForest as well as LogCluster for anomaly detection and adopt the open source deep learning-based log analysis toolkit to evaluate DeepLog and LogAnomaly 3 .”
InfluentialA Comprehensive Survey on Deep Graph Representation Learning
Ju, W., Fang, Z., Gu, Y., Liu, Z., Long, Q., Qiao, Z., Qin, Y., Shen, J., Sun, F., Xiao, Z., et al. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106207 · Cited by 341
A Distributed Deep Learning System for Web Attack Detection on Edge Devices
Tian, Z., Luo, C., Qiu, J., Du, X., & Guizani, M. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2938778 · Cited by 367
Log-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
“For example, DeepLog [5] first applies the Spell [27] parser to extract log templates.”
BackgroundMethodResultInfluentialDeep Learning-based Anomaly Detection in Cyber-physical Systems: Progress and Opportunities
Luo, Y., Xiao, Y., Cheng, L., Peng, G., & Yao, D. (. (2022) · ACM Computing Surveys · doi:10.1145/3453155 · Cited by 268
A Survey on Automated Log Analysis for Reliability Engineering
He, S., He, P., Chen, Z., Yang, T., Su, Y., & Lyu, M. R. (2022) · ACM Computing Surveys · doi:10.1145/3460345 · Cited by 334
Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study
Preuveneers, D., Rimmer, V., Tsingenopoulos, I., Spooren, J., Joosen, W., & Ilie-Zudor, E. (2018) · Applied Sciences · doi:10.3390/app8122663 · Cited by 323
Deep learning for misinformation detection on online social networks: a survey and new perspectives
Islam, M. R., Liu, S., Wang, X., & Xu, G. (2020) · Social Network Analysis and Mining · doi:10.1007/s13278-020-00696-x · Cited by 289
Liu, F., Wen, Y., Zhang, D., Jiang, X., Xing, X., & Meng, D. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3363224 · Cited by 319
Task Failure Prediction in Cloud Data Centers Using Deep Learning
Gao, J., Wang, H., & Shen, H. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2020.2993728 · Cited by 219
Loghub: a large collection of system log datasets for AI-driven log analytics
Zhu, J., He, S., He, P., Liu, J., & Lyu, M. R. (2023) · 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre59848.2023.00071 · Cited by 302
Semi-Supervised Log-Based Anomaly Detection via Probabilistic Label Estimation
Yang, L., Chen, J., Wang, Z., Wang, W., Jiang, J., Dong, X., & Zhang, W. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00130 · Cited by 244
Graph Neural Networks for Intrusion Detection: A Survey
Bilot, T., Madhoun, N. E., Agha, K. A., & Zouaoui, A. (2023) · IEEE Access · doi:10.1109/access.2023.3275789 · Cited by 227
SHADEWATCHER: Recommendation-guided Cyber Threat Analysis using System Audit Records
Zengy, J., Wang, X., Liu, J., Chen, Y., Liang, Z., Chua, T.-S., & Chua, Z. L. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833669 · Cited by 288
Deep learning for anomaly detection in log data: A survey
Landauer, M., Onder, S., Skopik, F., & Wurzenberger, M. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100470 · Cited by 202
Pitfalls in Machine Learning for Computer Security
Arp, D., Quiring, E., Pendlebury, F., Warnecke, A., Pierazzi, F., Wressnegger, C., Cavallaro, L., & Rieck, K. (2024) · Communications of the ACM · doi:10.1145/3643456 · Cited by 299
THREATRACE: Detecting and Tracing Host-Based Threats in Node Level Through Provenance Graph Learning
Wang, S., Wang, Z., Zhou, T., Sun, H., Yin, X., Han, D., Zhang, H., Shi, X., & Yang, J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3208815 · Cited by 228
Log-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257
Soldani, J., & Brogi, A. (2023) · ACM Computing Surveys · doi:10.1145/3501297 · Cited by 256
“relying on an external monitoring tool [20]”
Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network
Lu, S., Wei, X., Li, Y., & Wang, L. (2018) · 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech) · doi:10.1109/dasc/picom/datacom/cyberscitec.2018.00037 · Cited by 220
Unsupervised Detection of Microservice Trace Anomalies through Service-Level Deep Bayesian Networks
Liu, P., Xu, H., Ouyang, Q., Jiao, R., Chen, Z., Zhang, S., Yang, J., Mo, L., Zeng, J., Xue, W., et al. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00014 · Cited by 252
“DeepLog [8] uses a neural network model utilizing LSTM to detect execution path anomaly.”
BackgroundMethodInfluentialHitAnomaly: Hierarchical Transformers for Anomaly Detection in System Log
Huang, S., Liu, Y., Fung, C., He, R., Zhao, Y., Yang, H., & Luan, Z. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3034647 · Cited by 228
A Federated Learning Approach to Anomaly Detection in Smart Buildings
Sater, R. A., & Hamza, A. B. (2021) · ACM Transactions on Internet of Things · doi:10.1145/3467981 · Cited by 243
DeepTraLog: trace-log combined microservice anomaly detection through graph-based deep learning
Zhang, C., Peng, X., Sha, C., Zhang, K., Fu, Z., Wu, X., Lin, Q., & Zhang, D. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510180 · Cited by 207
Deep neural networks in the cloud: Review, applications, challenges and research directions
Chan, K. Y., Abu-Salih, B., Qaddoura, R., Al-Zoubi, A. M., Palade, V., Pham, D.-S., Ser, J. D., & Muhammad, K. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126327 · Cited by 199
Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection
Brown, A., Tuor, A., Hutchinson, B., & Nichols, N. (2018) · Proceedings of the First Workshop on Machine Learning for Computing Systems · doi:10.1145/3217871.3217872 · Cited by 198
Ma, Z., Ge, H., Liu, Y., Zhao, M., & Ma, J. (2019) · IEEE Access · doi:10.1109/access.2019.2896003 · Cited by 215
IoT-KEEPER: Detecting Malicious IoT Network Activity Using Online Traffic Analysis at the Edge
Hafeez, I., Antikainen, M., Ding, A. Y., & Tarkoma, S. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.2966951 · Cited by 198
Adversarial Attacks and Defenses in Deep Learning: From a Perspective of Cybersecurity
Zhou, S., Liu, C., Ye, D., Zhu, T., Zhou, W., & Yu, P. S. (2022) · ACM Computing Surveys · doi:10.1145/3547330 · Cited by 244
Logram: Efficient Log Parsing Using n-Gram Dictionaries
Dai, H., Li, H., Chen, C. S., Shang, W., & Chen, T.-H. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3007554 · Cited by 170
Kairos: Practical Intrusion Detection and Investigation using Whole-system Provenance
Cheng, Z., Lv, Q., Liang, J., Wang, Y., Sun, D., Pasquier, T., & Han, X. (2024) · 2024 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp54263.2024.00005 · Cited by 132
Cyber threat detection based on artificial neural networks using event profiles
Lee, J., Kim, J., Kim, I., & Han, K. (2019) · IEEE Access · doi:10.1109/access.2019.2953095 · Cited by 188
“[39] employed the TF-IDF for learning program behavior in malicious activities”
BackgroundMethodExperience Report: Deep Learning-based System Log Analysis for Anomaly Detection
Chen, Z., Liu, J., Gu, W., Su, Y., & Lyu, M. R.-T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05908 · Cited by 60
Tiresias: Predicting Security Events Through Deep Learning
Shen, Y., Mariconti, E., Vervier, P. A., & Stringhini, G. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243811 · Cited by 139
HELAD: A novel network anomaly detection model based on heterogeneous ensemble learning
Zhong, Y., Chen, W., Wang, Z., Chen, Y., Wang, K., Li, Y., Yin, X., Shi, X., Yang, J., & Li, K. (2020) · Computer Networks · doi:10.1016/j.comnet.2019.107049 · Cited by 171
Towards Safer Smart Contracts: A Sequence Learning Approach to Detecting Security ThreatsPreprint
Tann, W. J.-W., Han, X., Gupta, S. S., & Ong, Y.-S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06632 · Cited by 228
Provenance-based Intrusion Detection Systems: A Survey
Zipperle, M., Gottwalt, F., Chang, E. J., & Dillon, T. S. (2022) · ACM Computing Surveys · doi:10.1145/3539605 · Cited by 179
K-Means-based isolation forest
Karczmarek, P., Kiersztyn, A., Pedrycz, W., & Al, E. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105659 · Cited by 169
Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs
Nedelkoski, S., Bogatinovski, J., Acker, A., Cardoso, J., & Kao, O. (2020) · 2020 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm50108.2020.00148 · Cited by 176
“Both DeepLog and PCA, have lower F1 scores in all experiments performed.”
BackgroundMethodInfluentialTask Failure Prediction in Cloud Data Centers Using Deep Learning
Gao, J., Wang, H., & Shen, H. (2019) · 2019 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata47090.2019.9006011 · Cited by 170
A survey of neural networks usage for intrusion detection systems
Drewek-Ossowicka, A., Pietrołaj, M., & Rumiński, J. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02014-x · Cited by 193
SwissLog: Robust and Unified Deep Learning Based Log Anomaly Detection for Diverse Faults
Li, X., Chen, P., Jing, L., He, Z., & Yu, G. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00018 · Cited by 143
Anomaly Detection with Graph Convolutional Networks for Insider Threat and Fraud Detection
Jiang, J., Chen, J., Gu, T., Choo, K.-K. R., Liu, C., Yu, M., Huang, W., & Mohapatra, P. (2019) · MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom47813.2019.9020760 · Cited by 166
A survey on big data-driven digital phenotyping of mental health
Liang, Y., Zheng, X., & Zeng, D. D. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.04.001 · Cited by 158
LAnoBERT: System log anomaly detection based on BERT masked language model
Lee, Y., Kim, J., & Kang, P. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110689 · Cited by 191
Anomaly Detection from System Tracing Data Using Multimodal Deep Learning
Nedelkoski, S., Cardoso, J., & Kao, O. (2019) · 2019 IEEE 12th International Conference on Cloud Computing (CLOUD) · doi:10.1109/cloud.2019.00038 · Cited by 139
Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review
Sabuhi, M., Zhou, M., Bezemer, C., & Musílek, P. (2021) · IEEE Access · doi:10.1109/access.2021.3131949 · Cited by 175
INSOMNIA: Towards Concept-Drift Robustness in Network Intrusion Detection
Andresini, G., Pendlebury, F., Pierazzi, F., Loglisci, C. A. A. M. D., Appice, A., & Cavallaro, L. (2021) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3474369.3486864 · Cited by 227
A Spatiotemporal Deep Learning Approach for Unsupervised Anomaly Detection in Cloud Systems
He, Z., Chen, P., Li, X., Wang, Y., Yu, G., Chen, C., Li, X., & Zheng, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027736 · Cited by 148
Flash: A Comprehensive Approach to Intrusion Detection via Provenance Graph Representation Learning
Rehman, M. U., Ahmadi, H., & Hassan, W. U. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00139 · Cited by 262
Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-source Data
Lee, C., Yang, T., Chen, Z., Su, Y., & Lyu, M. R. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00150 · Cited by 149
“Previous anomaly detection approaches are usually based on system logs [58]–[62] or KPIs [63]–[67], or both [68], targeting traditional distributed systems without complex invocation relationships.”
UniParser: A Unified Log Parser for Heterogeneous Log Data
Liu, Y., Zhang, X., He, S., Zhang, H., Li, L., Kang, Y., Xu, Y., Ma, M., Lin, Q., Dang, Y., et al. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511993 · Cited by 125
Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code
Zhang, Z., Chen, C., Liu, B., Liao, C., Gong, Z., Yu, H., Li, J., & Wang, R. (2024) · Transactions on machine learning research · doi:10.13140/rg.2.2.32710.69440/1 · Cited by 194
Rashid, A. B., Kausik, A. K., Al Hassan Sunny, A., & Bappy, M. H. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/8676366 · Cited by 143
A survey of AIOps methods for failure management
Notaro, P., Cardoso, J., & Gerndt, M. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3483424 · Cited by 142
Cloud-Native Computing: A Survey From the Perspective of Services
Deng, S., Zhao, H., Huang, B., Zhang, C., Chen, F., Deng, Y., Yin, J., Dustdar, S., & Zomaya, A. Y. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3353855 · Cited by 179
Internet of Things Cyber Attacks Detection using Machine Learning
Alsamiri, J., & Alsubhi, K. (2019) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2019.0101280 · Cited by 129
A Survey on Explainable Anomaly Detection
Li, Z., Zhu, Y., & Van Leeuwen, M. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3609333 · Cited by 139
“For instance, Reference [205] utilises Decision Trees [41] and DeepLog [62] to detect anomalies in system logs, and then explain the results using the Shapley value approach.”
MethodUnsupervised log message anomaly detection
Farzad, A., & Gulliver, T. A. (2020) · ICT Express · doi:10.1016/j.icte.2020.06.003 · Cited by 122
LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection
Qi, J., Huang, S., Luan, Z., Yang, S., Fung, C., Yang, H., Qian, D., Shang, J., Xiao, Z., & Wu, Z. (2023) · 2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys) · doi:10.1109/hpcc-dss-smartcity-dependsys60770.2023.00045 · Cited by 102
Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy
Du, M., Jia, R., & Song, D. X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07116 · Cited by 84
Self-citationLifelong Anomaly Detection Through Unlearning
Du, M., Chen, Z., Liu, C., Oak, R., & Song, D. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3363226 · Cited by 118
Self-citationSystem log clustering approaches for cyber security applications: A survey
Landauer, M., Skopik, F., Wurzenberger, M., & Rauber, A. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101739 · Cited by 108
A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We?
Jiang, Z., Liu, J., Huang, J., Li, Y., Huo, Y., Gu, J., Chen, Z., Zhu, J., & Lyu, M. R.-T. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652123 · Cited by 172
Log Parsing with Prompt-based Few-shot Learning
Le, V.-H., & Zhang, H. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00204 · Cited by 129
“The rich information included in log data enables a variety of log analytics tasks, such as anomaly detection [1], [2], [3], [4], root cause analysis [5], [6], failure prediction [7], [8], and log compression [9], [10].”
BackgroundMethodReLeS: A Neural Adaptive Multipath Scheduler based on Deep Reinforcement Learning
Zhang, H., Li, W., Gao, S., Wang, X., & Ye, B. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737649 · Cited by 129
Spell: Online Streaming Parsing of Large Unstructured System Logs
Du, M., & Li, F. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2875442 · Cited by 114
Self-citationMalware Detection on Highly Imbalanced Data through Sequence Modeling
Oak, R., Du, M., Yan, D., Takawale, H., & Amit, I. (2019) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3338501.3357374 · Cited by 168
Self-citationDeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications
Han, D., Wang, Z., Chen, W., Zhong, Y., Wang, S., Zhang, H., Yang, J., Shi, X., & Yin, X. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484589 · Cited by 121
“For DeepLog , we use HDFS dataset [57] which is the same dataset evaluated in its work.”
InfluentialDAPT 2020 - Constructing a Benchmark Dataset for Advanced Persistent Threats
Myneni, S., Chowdhary, A., Sabur, A., Sengupta, S., Agrawal, G., Huang, D., & Kang, M. H. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-59621-7_8 · Cited by 157
LogTransfer: Cross-System Log Anomaly Detection for Software Systems with Transfer Learning
Chen, R., Zhang, S., Li, D., Zhang, Y., Guo, F., Meng, W., Pei, D., Zhang, Y., Chen, X., & Liu, Y. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00013 · Cited by 101
Robust Failure Diagnosis of Microservice System Through Multimodal Data
Zhang, S., Jin, P., Lin, Z., Sun, Y., Zhang, B., Xia, S., Li, Z., Zhong, Z., Ma, M., Jin, W., et al. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3290018 · Cited by 118
Deep Reinforcement Adversarial Learning Against Botnet Evasion Attacks
Apruzzese, G., Andreolini, M., Marchetti, M., Venturi, A., & Colajanni, M. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3031843 · Cited by 117
“M ACHINE Learning (ML) approaches are being increasingly applied to cybersecurity where data-driven detection algorithms outperform traditional signature-based methods against novel forms of attacks [1], [2].”
BackgroundLogLens: A Real-Time Log Analysis System
Debnath, B. K., Solaimani, M., Gulzar, M. A., Arora, N., Lumezanu, C., Xu, J., Zong, B., Zhang, H., Jiang, G., & Khan, L. R. (2018) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2018.00105 · Cited by 117
Attacks Meet Interpretability: Attribute-steered Detection of Adversarial SamplesPreprint
Tao, G., Ma, S., Liu, Y., & Zhang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11580 · Cited by 149
LogEvent2vec: LogEvent-to-Vector Based Anomaly Detection for Large-Scale Logs in Internet of Things
Wang, J., Tang, Y., He, S., Zhao, C., Sharma, P. K., Alfarraj, O., & Tolba, A. (2020) · Sensors · doi:10.3390/s20092451 · Cited by 107
LightLog: A lightweight temporal convolutional network for log anomaly detection on the edge
Wang, Z., Tian, J., Fang, H., Chen, L., & Qin, J. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108616 · Cited by 116
GRU-based deep learning approach for network intrusion alert prediction
Ansari, M. S., Bartoš, V., & Lee, B. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.09.040 · Cited by 114
DivLog: Log Parsing with Prompt Enhanced In-Context Learning
Xu, J., Yang, R., Huo, Y., Zhang, C., & He, P. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639155 · Cited by 164
LogFiT: Log Anomaly Detection Using Fine-Tuned Language Models
Almodovar, C., Sabrina, F., Karimi, S., & Azad, S. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3358730 · Cited by 111
“As shown in Figure 1, log parsers [8], [9] extract string templates from the log data to build a vocabulary of all known log templates (steps 1 and 2).”
BackgroundMethodInfluentialLogzip: Extracting Hidden Structures via Iterative Clustering for Log Compression
Liu, J., Zhu, J., He, S., He, P., Zheng, Z., & Lyu, M. R. (2019) · 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00085 · Cited by 97
Brain: Log Parsing With Bidirectional Parallel Tree
Yu, S., He, P., Chen, N., & Wu, Y. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3270566 · Cited by 126
DEEPCASE: Semi-Supervised Contextual Analysis of Security Events
Ede, T. V., Aghakhani, H., Spahn, N., Bortolameotti, R., Cova, M., Continella, A., Steen, M. V., Peter, A., Kruegel, C., & Vigna, G. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833671 · Cited by 90
ZeroWall: Detecting Zero-Day Web Attacks through Encoder-Decoder Recurrent Neural Networks
Tang, R., Yang, Z., Li, Z., Meng, W., Wang, H., Li, Q. L., Sun, Y., Pei, D., Tao, W., Xu, Y., et al. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155278 · Cited by 124
Identifying bad software changes via multimodal anomaly detection for online service systems
Zhao, N., Chen, J., Yu, Z., Wang, H., Li, J., Qiu, B., Xu, H., Zhang, W., Sui, K., & Pei, D. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3468543 · Cited by 89
Anomaly Detection and Classification using Distributed Tracing and Deep Learning
Nedelkoski, S., Cardoso, J., & Kao, O. (2019) · 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) · doi:10.1109/ccgrid.2019.00038 · Cited by 84
LogGPT: Log Anomaly Detection via GPT
Han, X., Yuan, S., & Trabelsi, M. (2023) · 2023 IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386543 · Cited by 63
Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies
Liu, Y., Tao, S., Meng, W., Wang, J., Ma, W., Chen, Y., Zhao, Y., Yang, H., & Jiang, Y. (2024) · Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension · doi:10.1145/3643916.3644408 · Cited by 47
Guidelines for assessing the accuracy of log message template identification techniques
Khan, Z. A., Shin, D., Bianculli, D., & Briand, L. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510101 · Cited by 88
openGauss: an autonomous database system
Li, G., Zhou, X., Sun, J., Yu, X., Han, Y., Jin, L., Li, W., Wang, T., & Li, S. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3476311.3476380 · Cited by 112
Zero-trust based dynamic access control for cloud computing
Wang, R., Li, C., Zhang, K., & Tu, B. (2025) · Cybersecurity · doi:10.1186/s42400-024-00320-x · Cited by 111
Metadata
- Authors
- Du, M., Li, F., Zheng, G., & Srikumar, V.
- Affiliations
- University of Utah (US)
- Venue
- Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2017-10-30
- Pages
- 1285-1298
- ISSN
- 1543-7221 (ISSN-L)
- OpenAlex
- W2767094836
- Cited by
- 1,835
- References
- 48
- Citations by year
- 2026: 320 · 2025: 326 · 2024: 248 · 2023: 242 · 2022: 189 · 2021: 199 · 2020: 157 · 2019: 114 · 2018: 39
- FWCI
- 67.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion)
- Funders
- National Natural Science Foundation of China (61729202)
- National Science Foundation (1314945, 1514520)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- log anomaly detection, system logs, long short-term memory, deep neural network, online learning, incremental model updating, root cause analysis, anomaly diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.