Improving Log-Based Anomaly Detection with Component-Aware Analysis

Abstract

Logs are universally available in software systems for troubleshooting. They record system run-time states and messages of system activities. Log analysis is an effective way to diagnosis system exceptions, but it will take a long time for engineers to locate anomalies accurately through logs. Many automatic approaches have been proposed for log-based anomaly detection. However, most of the prior approaches did not consider the corresponding system component of a log message. Such component records the log location, which can help detect the location-sequence-related anomalies. In this paper, we propose LogC, a new Log -based anomaly detection approach with Component-aware analysis. LogC contains two phases: (i) turning log messages into log template sequences and component sequences, (ii) feeding such two sequences to train a combined LSTM model for detecting anomalous logs. LogC only needs normal log sequences to train the combined model. We evaluate LogC on two open-source log datasets: HDFS and ThunderBird. Experimental results show that LogC overall outperforms three baselines (i.e., PCA, IM, and DeepLog) in terms of three metrics (precision, recall, and F-measure).