The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Abstract
Anomaly detection is a critical step towards building a secure and trustworthy system. The primary purpose of a system log is to record system states and significant events at various critical points to help debug system failures and perform root cause analysis. Such log data is universally available in nearly all computer systems. Log data is an important and valuable resource for understanding system status and performance issues; therefore, the various system logs are naturally excellent source of information for online monitoring and anomaly detection. We propose DeepLog, a deep neural network model utilizing Long Short-Term Memory (LSTM), to model a system log as a natural language sequence. This allows DeepLog to automatically learn log patterns from normal execution, and detect anomalies when log patterns deviate from the model trained from log data under normal execution. In addition, we demonstrate how to incrementally update the DeepLog model in an online fashion so that it can adapt to new log patterns over time. Furthermore, DeepLog constructs workflows from the underlying system log so that once an anomaly is detected, users can diagnose the detected anomaly and perform root cause analysis effectively. Extensive experimental evaluations over large log data have shown that DeepLog has outperformed other existing log-based anomaly detection methods based on traditional data mining methodologies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Besides, Log2Vec adopts word2vec-based model that ignores the contextual information in sentences [47], thus it cannot fully understand the semantic meaning of log messages.”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [47]
In the citing paper
“Since DeepLog represents log messages as the indexes of log templates, it cannot prevent semantic information loss and could produce many wrong detection results [47], [48].”
From Log-based Anomaly Detection Without Log Parsing (2021) · cites this paper as [47]
In the citing paper
“Techniques such as DeepLog [67] have been adapted to compute anomaly scores in microservice systems [68]”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [67]
In the citing paper
“Its paradigm has shifted from traditional rule matching to deep Natural Language Processing (NLP) models, represented by DeepLog [12] and LogRobust [13] .”
From A Survey of Anomaly Detection and Root Cause Analysis Techniques for Artificial Intelligence for IT Operations (AIOps) in Large-Scale Software Systems (2026) · cites this paper as [12]
In the citing paper
“DeepLog[27] is an early representative in this field; it templates logs into Log Keys and uses LSTMs to learn the transition probabilities of normal sequences, treating deviations from predictions as anomalies. However, it can only provide anomaly judgments and lacks deep semantic extraction from semi-structured texts.”
From From Correlation to Intervention: A Survey on Log-Based Causal Analysis for Security and AIOps (2026) · cites this paper as [27]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
(2016) · Cited by 11,679
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
LSTM neural networks for language modeling
Sundermeyer, M., Schlüter, R., & Ney, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-65 · Cited by 1,963
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332
A Primer on Neural Network Models for Natural Language Processing
Goldberg, Y. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4992 · Cited by 1,201
Semi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
Foundations of statistical natural language processing
Weikum, G. (2002) · ACM SIGMOD Record · doi:10.1145/601858.601867 · Cited by 615
Experience Report: System Log Analysis for Anomaly Detection
He, S., Zhu, J., He, P., & Lyu, M. R. (2016) · 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2016.21 · Cited by 593
Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis
Fu, Q., Lou, J.-G., Wang, Y., & Li, J. (2009) · 2009 Ninth IEEE International Conference on Data Mining · doi:10.1109/icdm.2009.60 · Cited by 542
Log clustering based problem identification for online service systems
Lin, Q., Zhang, H., Lou, J.-G., Zhang, Y., & Chen, X. (2016) · Proceedings of the 38th International Conference on Software Engineering Companion · doi:10.1145/2889160.2889232 · Cited by 427
Spell: Streaming Parsing of System Event Logs
Du, M., & Li, F. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0103 · Cited by 423
Self-citationClustering event logs using iterative partitioning
Makanju, A. A. O., Zincir-Heywood, A. N., & Milios, E. E. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557154 · Cited by 338
Bittau, A., Belay, A., Mashtizadeh, A., Mazieres, D., & Boneh, D. (2014) · 2014 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2014.22 · Cited by 313
Mining invariants from console logs for system problem detection
Lou, J., Fu, Q., Yang, S., Xu, Y., & Li, J. (2010) · Cited by 302
Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks
Yen, T.-F., Oprea, A., Onarlioglu, K., Leetham, T., Robertson, W., Juels, A., & Kirda, E. (2013) · Proceedings of the 29th Annual Computer Security Applications Conference · doi:10.1145/2523649.2523670 · Cited by 270
LogSig: generating system events from raw textual logs
Tang, L., Li, T., & Perng, C.-S. (2011) · Proceedings of the 20th ACM international conference on Information and knowledge management · doi:10.1145/2063576.2063690 · Cited by 244
Structured comparative analysis of systems logs to diagnose performance problems
Nagaraj, K., Killian, C. E., & Neville, J. (2012) · Cited by 243
An Evaluation Study on Log Parsing and Its Use in Log Mining
He, P., Zhu, J., He, S., Li, J., & Lyu, M. R. (2016) · 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) · doi:10.1109/dsn.2016.66 · Cited by 230
LogMine: Fast Pattern Recognition for Log Analytics
Hamooni, H., Debnath, B., Xu, J., Zhang, H., Jiang, G., & Mueen, A. (2016) · Proceedings of the 25th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2983323.2983358 · Cited by 227
Dynamic syslog mining for network failure monitoring
Yamanishi, K., & Maruyama, Y. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081927 · Cited by 213
SherLog: error diagnosis by connecting clues from run-time logs
Yuan, D., Mai, H., Xiong, W., Tan, L., Zhou, Y., & Pasupathy, S. (2010) · ACM SIGARCH Computer Architecture News · doi:10.1145/1735970.1736038 · Cited by 212
Online System Problem Detection by Mining Patterns of Console Logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. (2009) · 2009 Ninth IEEE International Conference on Data Mining · doi:10.1109/icdm.2009.19 · Cited by 203
Introducing CloudLab: Scientific Infrastructure for Advancing Cloud Architectures and Applications.
Ricci, R., Eide, E. N., & CloudLab Team (2014) · ;login:: the magazine of USENIX & SAGE · Cited by 200
Inferring models of concurrent systems from logs of their behavior with CSight
Beschastnikh, I., Brun, Y., Ernst, M. D., & Krishnamurthy, A. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568246 · Cited by 181
SherLog: error diagnosis by connecting clues from run-time logs
Yuan, D., Mai, H., Xiong, W., Tan, L., Zhou, Y., & Pasupathy, S. (2010) · Proceedings of the fifteenth International Conference on Architectural support for programming languages and operating systems · doi:10.1145/1736020.1736038 · Cited by 176
Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data
Oprea, A., Li, Z., Yen, T.-F., Chin, S. H., & Alrwais, S. (2015) · 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks · doi:10.1109/dsn.2015.14 · Cited by 174
Automated IT system failure prediction: A deep learning approach
Zhang, K., Xu, J., Min, M. R., Jiang, G., Pelechrinis, K., & Zhang, H. (2016) · 2016 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2016.7840733 · Cited by 171
Event Logs for the Analysis of Software Failures: A Rule-Based Approach
Cinque, M., Cotroneo, D., & Pecchia, A. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.67 · Cited by 134
Automated System Monitoring and Notification With Swatch
Hansen, S., & Atkins, E. T. (1993) · Cited by 125
Mining program workflow from interleaved traces
Lou, J., Fu, Q., Yang, S., Jiang, L., & Wu, B. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835883 · Cited by 108
CloudSeer: Workflow Monitoring of Cloud Infrastructures via Interleaved Logs
Yu, X., Joshi, P., Xu, J., Jin, G., Zhang, H., & Jiang, G. (2016) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/2872362.2872407 · Cited by 100
Zhao, X., Rodrigues, K., Luo, Y., Yuan, D., & Stumm, M. (2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026924 · Cited by 79
Lin, Q., Lou, J., Zhang, H., & Zhang, D. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884795 · Cited by 72
Real-time log file analysis using the Simple Event Correlator (SEC)
Rouillard, J. P. (2004) · Cited by 67
Analyzing cluster log files using Logsurfer
Prewett, J. E. (2003) · Cited by 61
LogTree: A Framework for Generating System Events from Raw Textual Logs
Tang, L., & Li, T. (2010) · Proceedings · doi:10.1109/icdm.2010.76 · Cited by 53
PerfAugur: Robust diagnostics for performance anomalies in cloud services
Roy, S., König, A. C., Dvorkin, I., & Kumar, M. (2015) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2015.7113365 · Cited by 52
A primer on neural network models for natural language processing
GoldbergYoav (2016) · doi:10.5555/3176748.3176757 · Cited by 35
2024 IEEE International Conference on Data Mining (ICDM)
(2024) · doi:10.1109/icdm59182.2024 · Cited by 32
ATOM: Efficient Tracking, Monitoring, and Orchestration of Cloud Resources
Du, M., & Li, F. (2017) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2017.2652467 · Cited by 28
Self-citation2017 IEEE International Conference on Data Mining (ICDM)
(2017) · doi:10.1109/icdm41144.2017 · Cited by 24
Augmented LSTM Framework to Construct Medical Self-Diagnosis Android
Liu, C., Sun, H., Du, N., Tan, S., Fei, H., Fan, W., Yang, T., Wu, H., Li, Y., & Zhang, C. (2016) · Proceedings · doi:10.1109/icdm.2016.0036 · Cited by 24
ATOM : Efficient Tracking, Monitoring, and Orchestration of Cloud Resources
Rani, K. S., & Srinu, K. (2018) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.21733653 · Cited by 4
Research works printed but not linked: 1
- Elastic Stack. 2017. The Open Source Elastic Stack. (2017). https://www.elastic.co/products[Online; accessed 16-May-2017].
Cited by
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
Deep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240
PyTorch distributed: experiences on accelerating data parallel training
Li, S., Zhao, Y., Varma, R., Salpekar, O., Noordhuis, P., Li, T., Paszke, A., Smith, J., Vaughan, B., Damania, P., et al. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3415478.3415530 · Cited by 465
Robust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761
Machine Learning for Anomaly Detection: A Systematic Review
Nassif, A. B., Talib, M. A., Nasir, Q., & Dakalbab, F. M. (2021) · IEEE Access · doi:10.1109/access.2021.3083060 · Cited by 748
Metadata
- Authors
- Du, M., Li, F., Zheng, G., & Srikumar, V.
- Affiliations
- University of Utah (US)
- Venue
- Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2017-10-30
- Pages
- 1285-1298
- ISSN
- 1543-7221 (ISSN-L)
- OpenAlex
- W2767094836
- Cited by
- 1,835
- References
- 48
- Citations by year
- 2026: 320 · 2025: 326 · 2024: 248 · 2023: 242 · 2022: 189 · 2021: 199 · 2020: 157 · 2019: 114 · 2018: 39
- FWCI
- 67.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion)
- Funders
- National Natural Science Foundation of China (61729202)
- National Science Foundation (1314945, 1514520)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- log anomaly detection, system logs, long short-term memory, deep neural network, online learning, incremental model updating, root cause analysis, anomaly diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.