The DOI matches its DataCite record; 24 of the 24 references it cites were found in a registry or an index.
LogBERT: Log Anomaly Detection via BERT
Open accessAbstract
Detecting anomalous events in online computer systems is crucial to protect the systems from malicious attacks or malfunctions. System logs, which record detailed information of computational events, are widely used for system status analysis. In this paper, we propose LogBERT, a self-supervised framework for log anomaly detection based on Bidirectional Encoder Representations from Transformers (BERT). LogBERT learns the patterns of normal log sequences by two novel self-supervised training tasks and is able to detect anomalies where the underlying patterns deviate from normal log sequences. The experimental results on three log datasets show that LogBERT outperforms state-of-the-art approaches for anomaly detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Classical unsupervised techniques, including PCA and Isolation Forest, obtained F1-scores of 0.741 and 0.718, respectively, on the HDFs dataset [6].”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While LogBERT has demonstrated state-of-the-art performance on structured and labeled datasets -such as Hadoop Distributed File System (HDFS), BlueGene/L (BGL), and Thunderbird -its applicability to heterogeneous and unlabeled Linux syslogs remains underexplored in the literature.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Guo et al. [6] followed a similar approach but with larger time windows between 1 to 5 min, and instead of following our buffering technique, they just dropped windows shorter than 10 tokens.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly to what [6] did, we conducted training and validation using 100% of the available training windows, with a 10% split reserved for validation during training.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The selection of these models aligns with the existing literature [6,14,15,22] and supports a balanced comparison between unsupervised and supervised approaches.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, Guo et al. [6] demonstrated that moderate mask ratios preserved semantic context while still being challenging for the model to reconstruct the missing information.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The study builds upon the LogBERT architecture [6], a transformer-based model leveraging self-supervised learning to detect anomalies in unstructured logs.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, LogBERT [6] employs a self-supervised masked language modeling objective combined with hypersphere minimization to detect deviations from expected patterns.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“In addition, the masking-and-reconstruction objective provides a principled anomaly score based on unexpected token predictions, which is suitable when explicit anomaly categories are unavailable.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Its performance on datasets like HDFS, BGL, and Thunderbird surpasses classical models such as PCA, DeepLog, and LogAnomaly [6], making it well-suited for operational deployment under resource constraints.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“These labels were used solely to split the datasets in the next stages so that LogBERT's [6] unsupervised training could be restricted to only sequences that contained normal events.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Although LogBERT [6] itself is trained in a completely unsupervised way, using only normal log sequences, labels are nevertheless computed at this stage for splitting purposes.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“We followed the same splitting approach as [6].”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Our evaluation confirmed the validity of the experimental design because it reproduced the performance that Guo et al. [6] reported on the HDFS, BGL, and Thunderbird datasets, where F1-scores reached 82.32%, 90.83%, and 96.64%, respectively.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“We ensured reproducibility by adopting an unsupervised training pipeline consistent with prior work [6].”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Initially, inspired by the approach of Guo et al. [6], who deleted log sequences with fewer than 10 tokens, potentially omitting valuable anomalies, we replicated their setup with a modification.”
From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [6]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601
Estimating the Support of a High-Dimensional Distribution
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001) · Neural Computation · doi:10.1162/089976601750264965 · Cited by 6,168
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Du, M., Li, F., Zheng, G., & Srikumar, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134015 · Cited by 1,835
Ruff, L., Vandermeulen, R. A., Görnitz, N., Deecke, L., Siddiqui, S. A., Binder, A., Müller, E., & Kloft, M. (2018) · International Conference on Machine Learning · Cited by 1,719
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332
Drain: an online log parsing approach with fixed depth tree
He, P., Zhu, J., Zheng, Z., & Lyu, M. R. (2017) · 2017 IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2017.13 · Cited by 943
Robust log-based anomaly detection on unstable log data
Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., et al. (2019) · Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3338906.3338931 · Cited by 761
LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs
Meng, W., Liu, Y., Zhu, Y., Zhang, S., Pei, D., Liu, Y., Chen, Y., Zhang, R., Tao, S., Sun, P., et al. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/658 · Cited by 708
What Supercomputers Say: A Study of Five System Logs
Oliner, A. J., & Stearley, J. (2007) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2007.103 · Cited by 695
Experience Report: System Log Analysis for Anomaly Detection
He, S., Zhu, J., He, P., & Lyu, M. R.-T. (2016) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2016.21 · Cited by 675
Log clustering based problem identification for online service systems
Lin, Q., Zhang, H., Lou, J., Zhang, Y., & Chen, X. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2889160.2889232 · Cited by 493
Liu, F., Wen, Y., Zhang, D., Jiang, X., Xing, X., & Meng, D. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3363224 · Cited by 319
Loghub: a large collection of system log datasets for AI-driven log analytics
Zhu, J., He, S., He, P., Liu, J., & Lyu, M. R. (2023) · 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre59848.2023.00071 · Cited by 302
Mining invariants from console logs for system problem detection
Lou, J., Fu, Q., Yang, S., Xu, Y., & Li, J. (2010) · Cited by 301
Yen, T.-F., Oprea, A., Onarlıoğlu, K., Leetham, T., Robertson, W., Juels, A., & Kirda, E. (2013) · Annual Computer Security Applications Conference · doi:10.1145/2523649.2523670 · Cited by 270
Improving one-class SVM for anomaly detection
Li, K., Huang, H., Tian, S., & Xu, W. M. (2004) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2003.1260106 · Cited by 261
Online System Problem Detection by Mining Patterns of Console Logs
Xu, W., Huang, L., Fox, A., Patterson, D. A., & Jordan, M. I. (2009) · Proceedings · doi:10.1109/icdm.2009.19 · Cited by 207
Automated IT system failure prediction: A deep learning approach
Zhang, K., Xu, J., Min, M. R., Jiang, G., Pelechrinis, K., & Zhang, H. (2016) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2016.7840733 · Cited by 193
Anomaly intrusion detection using one class SVM
Wang, Y., Wong, J., & Miner, A. S. (2005) · IEEE SMC Information Assurance Workshop (IAW) · doi:10.1109/iaw.2004.1437839 · Cited by 166
Event Logs for the Analysis of Software Failures: A Rule-Based Approach
Cinque, M., Cotroneo, D., & Pecchia, A. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.67 · Cited by 133
Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection
Wang, Z., Chen, Z., Ni, J., Liu, H., Chen, H., & Tang, J. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467125 · Cited by 56
Research works printed but not linked: 2
- [20]20. Yen, T.F., Oprea, A., Onarlioglu, K., Leetham, T., Robertson, W., Juels, A., Kirda, E.: Beehive: Large-scale log analysis for detecting suspicious activity in enterprise networks. In: Proceedings of the 29th Annual Computer Security Applications Conference. pp. 199-208 (2013)
- [24]24. He, S., Zhu, J., He, P., Lyu, M.R.: Loghub: A Large Collection of System Log Datasets towards Automated Log Analytics. arXiv:2008.06448 [cs] (Aug 2020) · arXiv
Cited by
Sorted by citations
Log-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
Deep learning for anomaly detection in log data: A survey
Landauer, M., Onder, S., Skopik, F., & Wurzenberger, M. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100470 · Cited by 286
Log-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257
Kheddar, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103347 · Cited by 240
LAnoBERT: System log anomaly detection based on BERT masked language model
Lee, Y., Kim, J., & Kang, P. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110689 · Cited by 159
Metadata
- Authors
- Guo, H., Yuan, S., & Wu, X.
- Affiliations
- Utah State University (US)
- University of Arkansas at Fayetteville (US)
- ORCID
- Wu, Xintao
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2103.04475
- OpenAlex
- W3134847483
- Cited by
- 7
- References
- 24
- Citations by year
- 2026: 1 · 2024: 2 · 2023: 2 · 2021: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- Digital Commons - USU (Utah State University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Digital Commons - USU (Utah State University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- log anomaly detection, system logs, BERT, self-supervised learning, intrusion detection, cybersecurity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
