The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
Drain: An Online Log Parsing Approach with Fixed Depth Tree
Abstract
Logs, which record valuable system runtime information, have been widely employed in Web service management by service providers and users. A typical log analysis based Web service management procedure is to first parse raw log messages because of their unstructured format; and then apply data mining models to extract critical system behavior information, which can assist Web service management. Most of the existing log parsing methods focus on offline, batch processing of logs. However, as the volume of logs increases rapidly, model training of offline log parsing methods, which employs all existing logs after log collection, becomes time consuming. To address this problem, we propose an online log parsing method, namely Drain, that can parse logs in a streaming and timely manner. To accelerate the parsing process, Drain uses a fixed depth parse tree, which encodes specially designed rules for parsing. We evaluate Drain on five real-world log data sets with more than 10 million raw log messages. The experimental results show that Drain has the highest accuracy on four data sets, and comparable accuracy on the remaining one. Besides, Drain obtains 51.85%~81.47% improvement in running time compared with the state-of-the-art online parser. We also conduct a case study on an anomaly detection task using Drain in the parsing step, which determines the effectiveness of Drain in log analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This makes the agent's reasoning inspectable: a reviewer can see which logs, metrics, traces, tickets, configurations, validators, or simulators were queried, and whether the resulting evidence was sufficient for the proposed diagnosis or action [77], [60], [78], [61], [25].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [78]
In the citing paper
“First, modern production diagnosis is fundamentally query-driven, and good teams treat incident response as repeated hypothesis testing under partial observability [4], [78], [84].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [78]
In the citing paper
“This framing aligns well with how production debugging systems are used in practice, through writing discriminating query, refine the candidate set, and then write the next query [78], [61], [53], [54].”
From Large language models for agentic NetOps and AIOps: Architectures, evaluation, and safety (2027) · cites this paper as [78]
In the citing paper
“they face two key challenges: high computing overhead from template conversion via parsing algorithms like Drain, and real-time processing pressure from explosive log volume during fault concurrency, making processing efficiency the system bottleneck.”
From A Survey of Anomaly Detection and Root Cause Analysis Techniques for Artificial Intelligence for IT Operations (AIOps) in Large-Scale Software Systems (2026) · cites this paper as [9]
In the citing paper
“The industry widely adopts online clustering algorithms based on parse trees such as Drain [9] , which can compress data dimensions efficiently but has insufficient discrimination for templates with high semantic similarity.”
From A Survey of Anomaly Detection and Root Cause Analysis Techniques for Artificial Intelligence for IT Operations (AIOps) in Large-Scale Software Systems (2026) · cites this paper as [9]
In the citing paper
“While computationally lightweight, these methods struggle to adapt to evolving workloads and often suffer from high false-positive rates when applied to complex, real-world clusters.”
From LINEADAPTER: Parameter-Efficient Fine-Tuning for Log Anomaly Detection and Root Cause Analysis (2025) · cites this paper as [4]
In the citing paper
“This frequencyratio-based heuristic has been widely adopted in prior work on log anomaly detection, particularly in template blacklisting and rule construction [4], [5].”
From LINEADAPTER: Parameter-Efficient Fine-Tuning for Log Anomaly Detection and Root Cause Analysis (2025) · cites this paper as [4]
In the citing paper
“It uses the Drain algorithm for parsing logs in real time, showing superior performance across benchmarks like HDFS and Thunderbird [19].”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [19]
Citations
Cites
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Term-Weighting Approaches in Automatic Text Retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 11,421
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332
What Supercomputers Say: A Study of Five System Logs
Oliner, A. J., & Stearley, J. (2007) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2007.103 · Cited by 695
Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis
Fu, Q., Lou, J., Wang, Y., & Jiang, L. (2009) · Proceedings · doi:10.1109/icdm.2009.60 · Cited by 555
Spell: Streaming Parsing of System Event Logs
Du, M., & Li, F. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0103 · Cited by 423
An Evaluation Study on Log Parsing and Its Use in Log Mining
He, P., Zhu, J., He, S., Li, J., & Lyu, M. R.-T. (2016) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2016.66 · Cited by 274
Self-citationLogSig: generating system events from raw textual logs
Tang, L., Li, T., & Perng, C.-S. (2011) · Proceedings of the 20th ACM international conference on Information and knowledge management · doi:10.1145/2063576.2063690 · Cited by 244
Learning to Log: Helping Developers Make Informed Logging Decisions
Zhu, J., He, P., Fu, Q., Zhang, H., Lyu, M. R.-T., & Zhang, D. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.60 · Cited by 229
Self-citationEverything as a Service (XaaS) on the Cloud: Origins, Current and Future Trends
Duan, Y., Fu, G., Zhou, N., Sun, X., Narendra, N. C., & Hu, B. (2015) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2015.88 · Cited by 198
Effective Software Fault Localization Using an RBF Neural Network
Wong, W. E., Debroy, V., Golden, R. M., Xu, X., & Thuraisingham, B. M. (2011) · IEEE Transactions on Reliability · doi:10.1109/tr.2011.2172031 · Cited by 191
Mi, H., Wang, H., Zhou, Y., Lyu, M. R.-T., & Cai, H. (2013) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2013.21 · Cited by 176
Self-citationA Lightweight Algorithm for Message Type Extraction in System Application Logs
Makanju, A., Zincir-Heywood, N., & Milios, E. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.138 · Cited by 147
Incremental Mining of System Log Format
Mizutani, M. (2013) · IEEE International Conference on Services Computing · doi:10.1109/scc.2013.73 · Cited by 145
Reliable QoS monitoring based on client feedback
Jurca, R., Faltings, B. V., & Binder, W. (2007) · International World Wide Web Conference (WWW) · doi:10.1145/1242572.1242708 · Cited by 137
Performance prediction of component-based applications
Chen, S., Liu, Y., Gorton, I., & Liu, A. (2004) · Journal of Systems and Software · doi:10.1016/j.jss.2003.05.005 · Cited by 91
Log 2 : a cost-aware logging mechanism for performance diagnosis
Ding, R. F., Zhou, H., Lou, J., Zhang, H., Lin, Q., Fu, Q., Zhang, D., & Xie, T. (2015) · NOVA (University of Newcastle Australia) · Cited by 70
Business Process Mining from E-Commerce Web Logs
Poggi, N., Muthusamy, V., Carrera, D., & Khalaf, R. Y. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40176-3_7 · Cited by 64
UiLog: Improving Log-Based Fault Diagnosis by Log Analysis
Zou, D., Qin, H., & Jin, H. (2016) · Journal of Computer Science and Technology · doi:10.1007/s11390-016-1678-7 · Cited by 60
Process mining on noisy logs — Can log sanitization help to improve performance?
Cheng, H.-J., & Kumar, A. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.08.003 · Cited by 55
Deriving Protocol Models from Imperfect Service Conversation Logs
Motahari‐Nezhad, H. R., Saint-Paul, R., Benatallah, B., & Casati, F. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.87 · Cited by 44
Automotive Cloud Service Systems Based on Service-Oriented Architecture and Its Evaluation
Iwai, A., & Aoyama, M. (2011) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2011.119 · Cited by 40
System problem detection by mining console logs
Patterson, D. A., & Xu, W. (2010) · Cited by 37
Zhang, J., Iannucci, B., Hennessy, M., Gopal, K., Xiao, S., Kumar, S., Pfeffer, D., Aljedia, B., Ren, Y., Griss, M., et al. (2013) · IEEE International Conference on Services Computing · doi:10.1109/scc.2013.34 · Cited by 34
A Temporal-Aware Hybrid Collaborative Recommendation Method for Cloud Service
Meng, S., Zhou, Z., Huang, T., Li, D., Wang, S., Fei, F., Wang, W., & Dou, W. (2016) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2016.40 · Cited by 23
Personalized ranking for digital libraries based on log analysis
Sun, Y., Li, H., Councill, I. G., Huang, J. L., Lee, W.-C., & Giles, C. L. (2008) · ACM International Workshop on Web Information and Data Management (WIDM) · doi:10.1145/1458502.1458524 · Cited by 20
Modelling Collaborative Services for Business and QoS Compliance
Yao, J., Chen, S., Wang, C., Levy, D., & Žic, J. (2011) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2011.44 · Cited by 14
Prediction of Web User Behavior by Discovering Temporal Relational Rules from Web Log Data
Yu, X., Li, M., Paik, I., & Ryu, K. H. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-32597-7_3 · Cited by 12
Real-Time Anomaly Detection in Streams of Execution Traces
Zhang, W., Bastani, F., Yen, I., Hulin, K. J., Bastani, F., & Khan, L. R. (2012) · IEEE International Symposium on High Assurance Systems Engineering (HASE) · doi:10.1109/hase.2012.13 · Cited by 8
Web Services Selection in Support of Reliable Web Service Choreography
Hwang, S., Liao, W.-P., & Lee, C.-H. (2010) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2010.79 · Cited by 8
Reconfiguration of Service Failures in DAMASCo Using Dynamic Software Product Lines
Cubo, J., Gámez, N., Pimentel, E., & Fuentes, L. (2015) · IEEE International Conference on Services Computing · doi:10.1109/scc.2015.25 · Cited by 4
Flagged first, then by citations
1 of the 33 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- lang, 2013, Using SEC USENIX, Login Magazine, 38
- 0, Drain source codsvce
- 0, Amazon EC2
- 0, LLC Operational data to support and enable computer science research
- 0, Evaluation of Clustering
Cited by
Sorted by citations
Robust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761
Tools and benchmarks for automated log parsing
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00021 · Cited by 574
Self-citationLogBERT: Log Anomaly Detection via BERT
Guo, H., Yuan, S., & Wu, X. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534113 · Cited by 454
Log-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
“Table I shows examples of wrongly parsed log templates by the Drain parser [24].”
BackgroundMethodInfluentialA Survey on Automated Log Analysis for Reliability Engineering
He, S., He, P., Chen, Z., Yang, T., Su, Y., & Lyu, M. R. (2022) · ACM Computing Surveys · doi:10.1145/3460345 · Cited by 334
Self-citation
Metadata
- Authors
- He, P., Zhu, J., Zheng, Z., & Lyu, M. R.
- Affiliations
- Chinese University of Hong Kong (HK)
- Sun Yat-sen University (CN)
- Venue
- 2017 IEEE International Conference on Web Services (ICWS)
- Publisher
- IEEE
- Published
- 2017-06
- Pages
- 33-40
- OpenAlex
- W2754665629
- Cited by
- 943
- References
- 33
- Citations by year
- 2026: 164 · 2025: 184 · 2024: 184 · 2023: 157 · 2022: 84 · 2021: 91 · 2020: 52 · 2019: 18 · 2018: 7 · 2017: 2
- FWCI
- 17.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Web Services (ICWS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- log parsing, online parsing, log analysis, web service management, anomaly detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
