Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 58 of the 64 references it cites were found in a registry or an index; 12 of the 36 sources its claims cite were checked for support.

Log-based Anomaly Detection Without Log Parsing

Open access
Le, Van-Hoang, Zhang, Hongyu · 2021 · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Abstract

Software systems often record important runtime information in system logs for troubleshooting purposes. There have been many studies that use log data to construct machine learning models for detecting system anomalies. Through our empirical study, we find that existing log-based anomaly detection approaches are significantly affected by log parsing errors that are introduced by 1) OOV (out-of-vocabulary) words, and 2) semantic misunderstandings. The log parsing errors could cause the loss of important information for anomaly detection. To address the limitations of existing methods, we propose NeuralLog, a novel log-based anomaly detection approach that does not require log parsing. NeuralLog extracts the semantic meaning of raw log messages and represents them as semantic vectors. These representation vectors are then used to detect anomalies through a Transformer-based classification model, which can capture the contextual information from log sequences. Our experimental results show that the proposed approach can effectively understand the semantic meaning of log messages and achieve accurate anomaly detection results. Overall, NeuralLog achieves F1-scores greater than 0.95 on four public datasets, outperforming the existing approaches.

Overview

CoChat Summary

TL;DR

NeuralLog is a log-based anomaly detection approach that skips log parsing by encoding raw log messages with BERT/WordPiece and classifying log sequences with a Transformer. It reports F1-scores above 0.95 on four public datasets and outperforms SVM, LR, IM, LogRobust, and Log2Vec on BGL, Thunderbird, and Spirit.

Key findings

  • An empirical study on BGL and Thunderbird found that existing log parsers suffer from OOV words and semantic misunderstanding; on a 60/40 BGL split, 94.12% of unique testing words were unseen in training, and these OOV words produced 1,304 unseen log events, accounting for 86.59% of testing log events.
  • NeuralLog achieved F1-scores greater than 0.95 on all four public datasets: HDFS 0.98, BGL 0.98, Thunderbird 0.96, and Spirit 0.97; it was best on BGL, Thunderbird, and Spirit and comparable on HDFS.
  • Parsing errors degraded baselines: SVM with Drain achieved F1 0.46 on BGL and 0.50 on Thunderbird, while LogRobust achieved F1 0.75 on BGL and 0.68 on Thunderbird; manually fixing parsing errors improved LogRobust by 25% and SVM by 29% on average on BGL.
  • Ablations showed NeuralLog’s raw-message encoding mattered: NeuralLog-Template reached F1 0.90 on BGL and 0.90 on Thunderbird versus NeuralLog’s 0.98 and 0.96, and NeuralLog-Word2Vec reached F1 0.80 on Thunderbird versus NeuralLog’s 0.96.

Methods

  • The paper first empirically studies log parsing errors on BGL and Thunderbird (14,672,653 log messages total, 353,394 labeled anomalies), evaluating Drain, AEL, IPLoM, and Spell and their impact on SVM and LogRobust.
  • NeuralLog preprocesses raw log messages, applies WordPiece tokenization, encodes messages with BERT base (12 transformer layers, 768 hidden units), adds positional encoding, and uses a Transformer encoder-based classifier (12 attention heads, feed-forward size 2048) for anomaly detection.
  • Evaluation uses four public datasets: HDFS, BGL, Thunderbird, and Spirit. HDFS sequences are built by block ID with an 80/20 train-test split; BGL, Thunderbird, and Spirit use the first 80% of time-sorted messages for training and the last 20% for testing, with sliding windows of length 20 and step size 1; metrics are Precision, Recall, and F1-score.

Limitations

  • Stated: the subject datasets come from a distributed system and supercomputers, so the number of systems is limited and does not cover all domains.
  • Stated: tool comparison relies on implementations and default settings from prior work, and manual labeling of the public datasets may introduce noise.
  • Stated: preprocessing removes numeric and special-character tokens, which may discard useful information such as node ID, task ID, IP address, or exit code. No additional limitations are evident from the text beyond those stated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/ase51524.2021.9678773

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 Valid · 8 Not valid · 18 Not checked · 105 Own results and interpretations

79 more claims

Create a free account to see all 132 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 30
  • Probably not a substantial claim. 18
  • The quote only partly states this claim. 4

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 64 references

Research works printed but not linked: 2

  1. radford, 2019, Language models are unsupervised multitask learners, OpenAIRE blog, 1, 9
  2. 2021, Bert pretrained models

Cited by

See all 361

Metadata

Authors
Le, V.-H., & Zhang, H.
Affiliations
University of Newcastle Australia (AU)
Venue
2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)
Publisher
IEEE
Published
2021-11
Pages
492-504
OpenAlex
W4205965165
Cited by
361
References
64
Citations by year
2026: 85 · 2025: 107 · 2024: 78 · 2023: 66 · 2022: 23 · 2021: 2
FWCI
32.57
Type
conference paper
Language
en
Open access
Locations
  • IEEE/ACM International Conference on Automated Software Engineering (ASE) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • NOVA (University of Newcastle Australia) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
Australian Research Council
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
log anomaly detection, out-of-vocabulary words, system logs, anomaly detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Log-based Anomaly Detection Without Log Parsing. https://app.cochat.ai/papers/10.1109/ase51524.2021.9678773
The paper
Le, V.-H., & Zhang, H. (2021). Log-based Anomaly Detection Without Log Parsing. 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). https://doi.org/10.1109/ase51524.2021.9678773

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.