The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
LogBERT: Log Anomaly Detection via BERT
Abstract
Detecting anomalous events in online computer systems is crucial to protect the systems from malicious attacks or malfunctions. System logs, which record detailed information of computational events, are widely used for system status analysis. In this paper, we propose LogBERT, a self-supervised framework for log anomaly detection based on Bidirectional Encoder Representations from Transformers (BERT). LogBERT learns the patterns of normal log sequences by two novel self-supervised training tasks, masked log message prediction and volume of hypersphere minimization. After training, LogBERT is able to capture the patterns of normal log sequences and further detect anomalies where the underlying patterns deviate from expected patterns. The experimental results on three log datasets show that LogBERT outperforms state-of-the-art approaches for anomaly detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, the study reported ROC metrics only, omitting PR-AUC, calibration, and latency evaluation.”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [7]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LogBERT achieved ROC-AUC > 0.90 on large-scale log corpora (HDFS, BGL) and proved effective for unseen attack types.”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Likewise, calibrated scoring and threshold control are often omitted, even though analysts require stable, interpretable alert probabilities to manage false positives.”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“(ii) deep CNN, LSTM, and Transformer models improve temporal representation but are largely uncalibrated and offline [1,3,8,23];”
From A SIEM-Integrated Cybersecurity Prototype for Insider Threat Anomaly Detection Using Enterprise Logs and Behavioural Biometrics (2025) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LogBERT[28] innovatively introduced Masked Language Modeling (MLM), treating logs as word sequences and utilizing BERT's powerful attention mechanism to learn long-distance contextual dependencies.”
From From Correlation to Intervention: A Survey on Log-Based Causal Analysis for Security and AIOps (2026) · cites this paper as [28]
Experimental — accuracy not yet measured
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“attention sub-layer and a position-wise feed forward sub-layer, in which a residual connection is employed around each of two sub-layers, followed by layer normalization [17].”
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“Inspired by BERT [8], LogBERT leverages the Transformer encoder to model log sequences and is trained by novel self-supervised tasks to capture the patterns of normal sequences.”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“Inspired by BERT [8], LogBERT leverages the Transformer encoder to model log sequences and is trained by novel self-supervised tasks to capture the patterns of normal sequences.”
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · cited 1×
“In order to visualize the log sequences, we adopt locally linear embedding (LLE) algorithm [24] to map”
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 1×
“– IsolationForest (iForest) [7].”
Estimating the Support of a High-Dimensional Distribution
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001) · Neural Computation · doi:10.1162/089976601750264965 · Cited by 6,168
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 1×
“In 2017, the Facebook AI team [58] adopted CNN in sequence-to-sequence learning by using convolution kernels to extract the information of sequences.”
MethodDeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835 · cited 11×
“Regarding baselines, we leverage the package Loglizer [4] to evaluate PCA, OCSVM, iForest as well as LogCluster for anomaly detection and adopt the open source deep learning-based log analysis toolkit to evaluate DeepLog and LogAnomaly 3 .”
InfluentialRuff, L., Vandermeulen, R. A., Görnitz, N., Deecke, L., Siddiqui, S. A., Binder, A., Müller, E., & Kloft, M. (2018) · International Conference on Machine Learning · Cited by 1,719
Research works printed but not linked: 1
- he, 2020, Loghub A Large Collection of System Log Datasets towards Automated Log Analytics
Cited by
Log-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
“automatically detect anomalies by analyzing log data [3], [4], [5], [6], [7], [8], [9].”
MethodLog-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257
“It has been applied in [15, 26, 39] for log-based anomaly detection.”
MethodDeep learning for anomaly detection in log data: A survey
Landauer, M., Onder, S., Skopik, F., & Wurzenberger, M. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100470 · Cited by 202
“Other common loss functions include the Hyper-Sphere Objective Function (HS) where the distance to the center of a hyper-sphere represents the anomaly score [24], [39], [41], [62], the Mean Squared Error (MSE) that is used for regression [20], [27], [28], [47], [50], [53], [68], and the Kullback-Leibler Divergence (KL…”
BackgroundMethodInfluentialKheddar, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103347 · Cited by 161
LAnoBERT: System log anomaly detection based on BERT masked language model
Lee, Y., Kim, J., & Kang, P. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110689 · Cited by 189
Metadata
- Authors
- Guo, H., Yuan, S., & Wu, X.
- Affiliations
- Utah State University (US)
- University of Arkansas at Fayetteville (US)
- ORCID
- Wu, Xintao
- Venue
- 2021 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2021-07-18
- Pages
- 1-8
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W3199174176
- Cited by
- 454
- References
- 25
- Citations by year
- 2026: 122 · 2025: 124 · 2024: 94 · 2023: 81 · 2022: 26 · 2021: 7
- FWCI
- 48.02
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- Funders
- NSF (1564250,1937010,2103829)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- log anomaly detection, system logs, BERT, self-supervised learning, intrusion detection, cybersecurity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: LogBERT: Log Anomaly Detection via BERT. https://app.cochat.ai/papers/10.1109/ijcnn52387.2021.9534113
- The paper
- Guo, H., Yuan, S., & Wu, X. (2021). LogBERT: Log Anomaly Detection via BERT. 2021 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/ijcnn52387.2021.9534113
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.