BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Logs are primary information resource for fault diagnosis and anomaly detection in large-scale computer systems, but it is hard to classify anomalies from system logs. Recent studies focus on extracting semantic information from unstructured log messages and converting it into word vectors. Therefore, LSTM approach is more suitable for time series data. Word2Vec is the up-to-date encoding method, but the order of words in sequences is not taken into account. In this article, we propose BERT-Log, which regards the log sequence as a natural language sequence, use pre-trained language model to learn the semantic representation of normal and anomalous logs, and a fully connected neural network is utilized to fine-tune the BERT model to detect abnormal. It can capture all the semantic information from log sequence including context and position. It has achieved the highest performance among all the methods on HDFS dataset, with an F1-score of 99.3%. We propose a new log feature extractor on BGL dataset to obtain log sequence by sliding window including node ID, window size and step size. BERT-Log approach detects anomalies on BGL dataset with an F1-score of 99.4%. It gives 19% performance improvement compared to LogRobust and 7% performance improvement compared to HitAnomaly.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 7×
“It indicates that LSTM, Bi-LSTM, Word2Vec and BERT are better suitable for capturing the semantic information of log sequence.”
MethodInfluentialDetecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332 · cited 2×
“Xu et al. (2009) use PCA algorithm to detect large-scale system problems by converting logs into count vectors.”
MethodDrain: an online log parsing approach with fixed depth tree
He, P., Zhu, J., Zheng, Z., & Lyu, M. R. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.13 · Cited by 943 · cited 7×
“In this article, we choose Drain as log parser.”
MethodInfluentialRobust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761 · cited 2×
“LogRobust (Zhang et al. 2019a) represents log events as semantic vectors and use an attention-based Bi-LSTM (Chen et al. 2022) model to detect anomalies.”
MethodTools and benchmarks for automated log parsing
Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00021 · Cited by 574 · cited 1×
“Log collection is one of the most important tasks for developers and operation engineers to monitor computer systems (Zhong, Guo, and Liu 2018; Zhu et al. 2019).”
BackgroundLogBERT: Log Anomaly Detection via BERT
Guo, H., Yuan, S., & Wu, X. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534113 · Cited by 454 · cited 1×
“LogBERT (Guo, Yuan, and Wu 2021) learns the patterns of normal log sequences by two novel self-supervised training tasks, masked log message prediction and volume of hypersphere minimization, but it does not identify and train the semantic information of abnormal logs.”
BackgroundSpell: Streaming Parsing of System Event Logs
Du, M., & Li, F. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0103 · Cited by 423
Log-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361 · cited 1×
“NeuralLog (Le and Zhang 2021) proposes a novel log-based anomaly detection approach based on a Transformer-based classification model that does not require log parsing.”
MethodTowards Automated Log Parsing for Large-Scale Log Data Analysis
He, P., Zhu, J., He, S., Li, J., & Lyu, M. R. (2018) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2017.2762673 · Cited by 230 · cited 6×
“Word2Vec (Wang et al. 2021) is the up-to-date encoding method proposed in the HitAnomaly (He et al. 2018) to map each word in log template to a vector.”
MethodInfluential
Cited by
Ferrag, M. A., Ndhlovu, M., Tihanyi, N., Cordeiro, L. C., Debbah, M., Lestable, T., & Thandi, N. S. (2024) · IEEE Access · doi:10.1109/access.2024.3363469 · Cited by 312
Security in DevSecOps: Applying Tools and Machine Learning to Verification and Monitoring Steps
Cankar, M., Petrovic, N., Pita Costa, J., Cernivec, A., Antic, J., Martincic, T., & Stepec, D. (2023) · Companion of the 2023 ACM/SPEC International Conference on Performance Engineering · doi:10.1145/3578245.3584943 · Cited by 24
A Systematic Comparison of Large Language Models Performance for Intrusion Detection
Bui, M.-T., Boffa, M., Valentim, R. V., Navarro, J. M., Chen, F., Bao, X., Houidi, Z. B., & Rossi, D. (2024) · Proceedings of the ACM on Networking · doi:10.1145/3696379 · Cited by 19
Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments
Rigaki, M., Lukáš, O., Catania, C., & Garcia, S. (2024) · Proceedings of the 16th International Conference on Agents and Artificial Intelligence · doi:10.5220/0012391800003636 · Cited by 17
“In non-security environments, LLM agents have showcased exploratory capabilities (Wang et al., 2023a; Du et al., 2023; Wu et al., 2023) in gaming environments such as Minecraft and Crafter.”
BackgroundLarge Language Models for Security Operations Centers: A Comprehensive Survey
Habibzadeh, A., Feyzi, F., & Atani, R. E. (2026) · Journal of Electrical and Computer Engineering · doi:10.1155/jece/3383674 · Cited by 13
“These models consist of two part: First part involves utilizing a LLM for extracting semantic vectors from log messages and second part involves a deep learning model for binary classification which could be MLP [201], custom Siamese network [181], or another Language model [206].”
Metadata
- Authors
- Chen, S., & Liao, H.
- Venue
- Applied Artificial Intelligence
- Publisher
- Informa UK Limited
- Published
- 2022-11-17
- Volume
- 36
- Issue
- 1
- ISSN
- 0883-9514 (ISSN-L)
- 1087-6545
- OpenAlex
- W4309623083
- Cited by
- 87
- References
- 11
- FWCI
- 7.87
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model. https://app.cochat.ai/papers/10.1080/08839514.2022.2145642
- The paper
- Chen, S., & Liao, H. (2022). BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model. Applied Artificial Intelligence. https://doi.org/10.1080/08839514.2022.2145642
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.