CoChat
Sign Up
PreprintCited by 7Software System Performance and ReliabilityAnomaly Detection Techniques and ApplicationsNetwork Security and Intrusion Detection

LLM meets ML: Data-efficient Anomaly Detection on Unstable Logs

Open access
Hadadi, Fatemeh, Xu, Qinghua, Bianculli, Domenico, Briand, Lionel·2024·arXiv
DOI 10.48550/arxiv.2406.07467arXiv 2406.07467

Preprint — published as 10.1145/3771283

Open this paper in CoChat — free accountOpen PDF

Abstract

Most log-based anomaly detectors assume logs are stable, though logs are often unstable due to software or environmental changes. Anomaly detection on unstable logs (ULAD) is therefore a more realistic, yet under-investigated challenge. Current approaches predominantly employ machine learning (ML) models, which often require extensive labeled data for training. To mitigate data insufficiency, we propose FlexLog, a novel hybrid approach for ULAD that combines ML models -- decision tree, k-nearest neighbors, and a feedforward neural network -- with a Large Language Model (Mistral) through ensemble learning. FlexLog also incorporates a cache and retrieval-augmented generation (RAG) to further enhance efficiency and effectiveness. To evaluate FlexLog, we configured four datasets for \task, namely ADFA-U, LOGEVOL-U, SynHDFS-U, and SYNEVOL-U. FlexLog outperforms all baselines by at least 1.2 percentage points (pp) in F1 score while using much less labeled data (62.87 pp reduction). When trained on the same amount of data as the baselines, FlexLog achieves up to a 13 pp increase in F1 score on ADFA-U across varying training dataset sizes. Additionally, FlexLog maintains inference time under one second per log sequence, making it suitable for most applications, except latency-sensitive systems. Further analysis reveals the positive impact of FlexLog's key components: cache, RAG and ensemble learning.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2406.07467

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The baseline models were selected to align with existing literature and to support a balanced comparison between unsupervised and supervised approaches.

    “The selection of these models aligns with the existing literature [6,14,15,22] and supports a balanced comparison between unsupervised and supervised approaches.”

    From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [15]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 82

  1. Andrea Arcuri and Lionel Briand. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In Proceedings of the 33rd International Conference on Software Engineering, ICSE '11, page 1-10, New York, NY, USA, 2011. Association for Computing Machinery. ISBN 9781450304450. doi: 10.1145/1985793.1985795. URL https://doi.org/10.1145/1985793.1985795. · DOI
  2. Halima Oluwabunmi Bello, Adebimpe Bolatito Ige, and Maxwell Nana Ameyaw. Deep learning in high-frequency trading: conceptual challenges and solutions for real-time fraud detection. World Journal of Advanced Engineering Technology and Sciences, 12(02):035-046, 2024.
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
  4. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321-357, June 2002. ISSN 1076-9757. doi: 10.1613/jair.953. URL http://dx.doi.org/10.1613/jair.953. · DOI · Paper page
  5. Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over- sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
  6. Laming Chen, Guoxin Zhang, and Hanning Zhou. Fast greedy map inference for determinantal point process to improve recommendation diversity, 2018. URL https://arxiv.org/abs/1709.05135. · arXiv
  7. Mike Y. Chen, Alice X. Zheng, Jesper Lloyd, Michael I. Jordan, and Eric Brewer. Failure diagnosis using decision trees. International Conference on Autonomic Computing, 2004. Proceedings., pages 36-43, 2004. URL https://api. semanticscholar.org/CorpusID:56849250.
  8. Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1724-1734. ACL, 2014. doi: 10.3115/V1/D14-1179. URL https://doi.org/10.3115/v1/d14-1179. · DOI · Paper page
  9. Guohao Chu, Jiaqi Wang, Qi Qi, Haiyang Sun, Shengtao Tao, and Jun Liao. Prefix-graph: A versatile log parsing approach merging prefix tree with probabilistic graph. In Proceedings of the 37th International Conference on Data Engineering (ICDE), pages 2411-2422, Virtual Event, 2021. IEEE. URL https://ieeexplore.ieee.org/document/9458609.
  10. Michael Collier and Robin Shahan. Microsoft azure essentials-fundamentals of azure. Microsoft Press, 2015.
  11. Gideon Creech and Jiankun Hu. Generation of a new ids test dataset: Time to retire the kdd collection. In 2013 IEEE Wireless Communications and Networking Conference (WCNC), pages 4487-4492, 2013. doi: 10.1109/WCNC.2013. 6555301. · DOI
  12. Jeffrey Dean and Sanjay Ghemawat. Mapreduce: Simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, 2008. doi: 10.1145/1327452.1327492. URL http://doi.acm.org/10.1145/1327452.1327492. · DOI · Paper page
  13. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, and Xiao Bi. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. · arXiv
  14. Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391-407, 1990.
  15. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms, 2023. URL https://arxiv.org/abs/2305.14314. · arXiv
  16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding, 2019. URL https://doi.org/10.18653/v1/n19-1423. · DOI · Paper page
  17. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022. · arXiv
  18. Evelyn Fix and Joseph L. Hodges. Discriminatory analysis -nonparametric discrimination: Consistency properties. International Statistical Review, 57:238, 1989. URL https://api.semanticscholar.org/CorpusID:120323383.
  19. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, and Aiesha Letman. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783. · arXiv
  20. Gongde Guo, Hui Wang, David Bell, Yaxin Bi, and Kieran Greer. Knn model-based approach in classification. In Robert Meersman, Zahir Tari, and Douglas C. Schmidt, editors, On The Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE, pages 986-996, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. ISBN 978-3-540-39964-3.
  21. Haixuan Guo, Shuhan Yuan, and Xintao Wu. Logbert: Log anomaly detection via bert. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1-8, 2021. doi: 10.1109/IJCNN52387.2021.9534113. · DOI · Paper page
  22. Xiao Han, Shuhan Yuan, and Mohamed Trabelsi. Loggpt: Log anomaly detection via gpt. In 2023 IEEE International Conference on Big Data (BigData), pages 1117-1122, 2023. doi: 10.1109/BigData59044.2023.10386543. · DOI
  23. M. He, T. Jia, C. Duan, H Cai, Y. Li, , and G. Huang. Llmelog: An approach for anomaly detection based on llm-enriched log events. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE), Tsukuba, Japan, 2024. IEEE Computer Society.
  24. Shilin He, Jieming Zhu, Pinjia He, and Michael R. Lyu. Experience report: System log analysis for anomaly detection, 2016. URL https://doi.org/10.1109/ISSRE.2016.21. · DOI
  25. Shilin He, Jieming Zhu, Pinjia He, and Michael R. Lyu. Loghub: A Large Collection of System Log Datasets Towards Automated Log Analytics. CoRR, abs/2008.06448, 2020. URL https://arxiv.org/abs/2008.06448. · arXiv
  26. Shilin He, Pinjia He, Zhuangbin Chen, Tianyi Yang, Yuxin Su, and Michael R. Lyu. A survey on automated log analysis for reliability engineering. ACM Comput. Surv., 54(6), jul 2021. ISSN 0360-0300. doi: 10.1145/3460345. URL https://doi.org/10.1145/3460345. · DOI · Paper page
  27. Maryamsadat Hejazi and Yashwant Prasad Singh. One-class support vector machines approach to anomaly detection. Applied Artificial Intelligence, 27(5):351-366, 2013.
  28. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 9(8):1735-1780, nov 1997. ISSN 0899-7667. doi: 10.1162/neco.1997.9.8.1735. URL https://doi.org/10.1162/neco.1997.9.8.1735. · DOI · Paper page
  29. Guangbin Huang, Yan Qiu Chen, and Haroon Atique Babri. Classification ability of single hidden layer feedforward neural networks. IEEE transactions on neural networks, 11 3:799-801, 2000. URL https://api.semanticscholar.org/ CorpusID:1852628.
  30. Shaohan Huang, Yi Liu, Carol Fung, Rong He, Yining Zhao, Hailong Yang, and Zhongzhi Luan. Hitanomaly: Hierarchical transformers for anomaly detection in system log. IEEE Transactions on Network and Service Management, 17(4):2064-2076, 2020. doi: 10.1109/TNSM.2020.3034647. · DOI
  31. Y. Huo, C. Lee, Y. Su, S. Shan, J. Liu, and M. R. Lyu. Evlog: Identifying anomalous logs over software evolution. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE), pages 391-402, Los Alamitos, CA, USA, oct 2023. IEEE Computer Society. doi: 10.1109/ISSRE59848.2023.00018. URL https://doi.ieeecomputersociety. org/10.1109/ISSRE59848.2023.00018. · DOI
  32. Chandra Irugalbandara, Ashish Mahendra, Roland Daynauth, Tharuka Kasthuri Arachchige, Jayanaka Dantanarayana, Krisztian Flautner, Lingjia Tang, Yiping Kang, and Jason Mars. Scaling down to scale up: A cost-benefit analysis of replacing openai's llm with open source slms in production. In 2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 280-291, 2024. doi: 10.1109/ISPASS61541.2024.00034. · DOI
  33. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https://arxiv.org/abs/2310.06825. · arXiv
  34. Suhas Kabinna, Weiyi Shang, Cor-Paul Bezemer, and Ahmed E. Hassan. Examining the stability of logging statements. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), volume 1, pages 326-337, 2016. doi: 10.1109/SANER.2016.29. · DOI
  35. Harsurinder Kaur, Husanbir Singh Pannu, and Avleen Kaur Malhi. A systematic review on imbalanced data challenges in machine learning: Applications and solutions. ACM computing surveys (CSUR), 52(4):1-36, 2019.
  36. Zanis Ali Khan, Donghwan Shin, Domenico Bianculli, and Lionel C. Briand. Impact of log parsing on deep learning- based anomaly detection. Empirical Software Engineering, 29(6), August 2024. ISSN 1573-7616. doi: 10.1007/s10664- 024-10533-w. URL http://dx.doi.org/10.1007/s10664-024-10533-w. · DOI · Paper page
  37. Yoon Kim. Convolutional neural networks for sentence classification. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746-1751, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14-1181. URL https://aclanthology.org/D14-1181. · DOI · Paper page
  38. Herb Krasner. The cost of poor software quality in the us: A 2020 report. Proc. Consortium Inf. Softw. QualityTM (CISQTM), pages 1-46, 2021.
  39. Max Landauer, Sebastian Onder, Florian Skopik, and Markus Wurzenberger. Deep learning for anomaly detection in log data: A survey. Machine Learning with Applications, 12:100470, 2023. ISSN 2666-8270. doi: https://doi.org/10.1016/ j.mlwa.2023.100470. URL https://www.sciencedirect.com/science/article/pii/S2666827023000233. · DOI
  40. V. Le and H. Zhang. Log-based anomaly detection without log parsing, nov 2021. URL https://doi.ieeecomputersociety. org/10.1109/ASE51524.2021.9678773. · DOI · Paper page
  41. V. Le and H. Zhang. Log parsing: How far can chatgpt go? In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 1699-1704, Los Alamitos, CA, USA, sep 2023. IEEE Computer Society. doi: 10.1109/ASE56229.2023.00206. URL https://doi.ieeecomputersociety.org/10.1109/ASE56229.2023.00206. · DOI
  42. Van-Hoang Le and Hongyu Zhang. Log-based anomaly detection with deep learning: how far are we? In Proceedings of the 44th International Conference on Software Engineering, ICSE '22, page 1356-1367, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450392211. doi: 10.1145/3510003.3510155. URL https://doi.org/10. 1145/3510003.3510155. · DOI · Paper page
  43. Van-Hoang Le and Hongyu Zhang. Log parsing with prompt-based few-shot learning. In Proceedings of the 45th International Conference on Software Engineering, ICSE '23, page 2438-2449. IEEE Press, 2023. ISBN 9781665457019. doi: 10.1109/ICSE48619.2023.00204. URL https://doi.org/10.1109/ICSE48619.2023.00204. · DOI · Paper page
  44. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436, 2015.
  45. Xiaoyun Li, Pengfei Chen, Linxiao Jing, Zilong He, and Guangba Yu. Swisslog: Robust and unified deep learning based log anomaly detection for diverse faults. Proceedings -International Symposium on Software Reliability Engineering, ISSRE, 2020-Octob:92-103, 2020. ISSN 10719458. doi: 10.1109/ISSRE5003.2020.00018. · DOI
  46. Qingwei Lin, Hongyu Zhang, Jian-Guang Lou, Yu Zhang, and Xuewei Chen. Log Clustering Based Problem Iden- tification for Online Service Systems. In Proceedings of the 38th International Conference on Software Engineering, ICSE -Companion Volume, pages 102-111, Austin, TX, USA, 2016. ACM. doi: 10.1145/2889160.2889232. URL https://doi.org/10.1145/2889160.2889232. · DOI
  47. Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 Eighth IEEE International Conference on Data Mining, pages 413-422, 2008. doi: 10.1109/ICDM.2008.17. · DOI · Paper page
  48. Yilun Liu, Shimin Tao, Weibin Meng, Feiyu Yao, Xiaofeng Zhao, and Hao Yang. Logprompt: Prompt engineering towards zero-shot and interpretable log analysis. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, ICSE-Companion '24, page 364-365, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400705021. doi: 10.1145/3639478.3643108. URL https://doi.org/10. 1145/3639478.3643108. · DOI · Paper page
  49. Steven Locke, Heng Li, Tse-Hsun Peter Chen, Weiyi Shang, and Wei Liu. Logassist: Assisting log analysis through log summarization. IEEE Transactions on Software Engineering, 48(9):3227-3241, 2022. doi: 10.1109/TSE.2021.3083715. · DOI · Paper page
  50. Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright Pamela Mishkin Paul Christiano Jan Leike Ryan Lowe et al. Gpt-4 technical report. 2023. URL https://api.semanticscholar.org/CorpusID:257532815.
  51. Siyang Lu, Xiang Wei, Yandong Li, and Liqiang Wang. Detecting anomaly in big data system logs using convolutional neural network. IEEE Access, 6:21929-21940, 2018. doi: 10.1109/ACCESS.2018.2811530. · DOI
  52. Zeyang Ma, An Ran Chen, Dong Jae Kim, Tse-Hsun Chen, and Shaowei Wang. Llmparser: An exploratory study on using large language models for log parsing. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400702174. doi: 10.1145/3597503.3639150. URL https://doi.org/10.1145/3597503.3639150. · DOI
  53. Leland McInnes, John Healy, and Steve Astels. hdbscan: Hierarchical density based clustering. J. Open Source Softw., 2(11):205, 2017.
  54. Weibin Meng, Ying Liu, Yichen Zhu, Shenglin Zhang, Dan Pei, Yuqing Liu, Yihao Chen, Ruizhi Zhang, Shimin Tao, Pei Sun, et al. Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In IJCAI, volume 19, pages 4739-4745, 2019.
  55. Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner, 2018. URL https://arxiv.org/abs/1707.03141. · arXiv
  56. Roweida Mohammed, Jumanah Rawashdeh, and Malak Abdullah. Machine learning with oversampling and under- sampling techniques: overview study and experimental results. In 2020 11th international conference on information and communication systems (ICICS), pages 243-248. IEEE, 2020.
  57. Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, and Yanai Elazar. Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, pages 12284-12314, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.779. URL https://aclanthology. org/2023.findings-acl.779. · DOI
  58. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155, 2022. URL https://api.semanticscholar.org/CorpusID: 246426909. · arXiv
  59. C. Pei, Z.Liu, Jianhui Li, E. Zhang, L. Zhang, H. Zhang, W. Chen, D. Pei, and G. Xie. Self-evolutionary group-wise log parsing based on large language model. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE), Tsukuba, Japan, 2024. IEEE Computer Society.
  60. Robi Polikar. Ensemble learning. Ensemble machine learning: Methods and applications, pages 1-34, 2012.
  61. M. Schuster and K.K. Paliwal. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11): 2673-2681, 1997. doi: 10.1109/78.650093. · DOI · Paper page
  62. James Turnbull. Monitoring with Prometheus. Turnbull Press, 2018.
  63. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 6000-6010, Red Hook, NY, USA, 2017.
  64. Vinod Kumar Vavilapalli, Arun C Murthy, Chris Douglas, Sharad Agarwal, Mahadev Konar, Robert Evans, Thomas Graves, Jason Lowe, Hitesh Shah, Siddharth Seth, et al. Apache hadoop yarn: Yet another resource negotiator. In Proceedings of the 4th annual Symposium on Cloud Computing, pages 1-16, 2013.
  65. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6): 186345, 2024.
  66. Xiaolei Wang, Lin Yang, Dongyang Li, Linru Ma, Yongzhong He, Junchao Xiao, Jiyuan Liu, and Yuexiang Yang. Maddc: Multi-scale anomaly detection, diagnosis and correction for discrete event logs. In Proceedings of the 38th Annual Computer Security Applications Conference, ACSAC '22, page 769-784, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450397599. doi: 10.1145/3564625.3567972. URL https://doi.org/10.1145/3564625. 3567972. · DOI
  67. Yi Xiao, Van-Hoang Le, and Hongyu Zhang. Demonstration-free: Towards more practical log parsing with large language models. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE '24, page 153-165, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400712487. doi: 10.1145/3691620.3694994. URL https://doi.org/10.1145/3691620.3694994. · DOI
  68. Junjielong Xu, Ruichun Yang, Yintong Huo, Chengyu Zhang, and Pinjia He. Divlog: Log parsing with prompt enhanced in-context learning. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400702174. doi: 10.1145/3597503.3639155. URL https://doi.org/10.1145/3597503.3639155. · DOI
  69. Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael I. Jordan. Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP '09, page 117-132, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605587523. doi: 10.1145/1629575.1629587. URL https://doi.org/10.1145/1629575.1629587. · DOI · Paper page
  70. An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang Ren, Xinlong Yang, Yong Li, Zhiying Xu, and Zipeng Zhang. Qwen2.5-1m technical report. arXiv preprint arXiv:2501.15383, 2025. · arXiv
  71. Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, and Mor Geva. Do large language models perform latent multi-hop reasoning without exploiting shortcuts?, 2024. URL https://arxiv.org/abs/2411.16679. · arXiv
  72. Boxi Yu, Jiayi Yao, Qiuai Fu, Zhiqing Zhong, Haotian Xie, Yaoliang Wu, Yuchi Ma, and Pinjia He. Deep learning or classical machine learning? an empirical study on log-based anomaly detection. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400702174. doi: 10.1145/3597503.3623308. URL https://doi.org/10.1145/3597503.3623308. · DOI
  73. Boxi Yu, Jiayi Yao, Qiuai Fu, Zhiqing Zhong, Haotian Xie, Yaoliang Wu, Yuchi Ma, and Pinjia He. Deep learning or classical machine learning? an empirical study on log-based anomaly detection. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400702174. doi: 10.1145/3597503.3623308. URL https://doi.org/10.1145/3597503.3623308. · DOI
  74. Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan. Metaformer is actually what you need for vision, 2022. URL https://arxiv.org/abs/2111.11418. · arXiv
  75. Shichao Zhang. Challenges in knn classification. IEEE Transactions on Knowledge and Data Engineering, 34(10): 4663-4675, 2021.
  76. T. Zhang, H. Qiu, G. Castellano, M. Rifai, C. Chen, and F. Pianese. System log parsing: A survey. IEEE Transactions on Knowledge & Data Engineering, 35(08):8596-8614, aug 2023. ISSN 1558-2191. doi: 10.1109/TKDE.2022.3222417. · DOI
  77. W. Zhang, T. Jia, C. Duan, H Cai, Y. Li, , and G. Huang. Leveraging rag-enhanced large language model for semi- supervised log anomaly detection. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE), Tsukuba, Japan, 2024. IEEE Computer Society.
  78. Xu Zhang, Yong Xu, Qingwei Lin, Bo Qiao, Hongyu Zhang, Yingnong Dang, Chunyu Xie, Xinsheng Yang, Qian Cheng, Ze Li, Junjie Chen, Xiaoting He, Randolph Yao, Jian-Guang Lou, Murali Chintalapati, Furao Shen, and Dongmei Zhang. Robust log-based anomaly detection on unstable log data. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2019, page 807-817, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450355728. doi: 10.1145/3338906.3338931. URL https://doi.org/10.1145/3338906.3338931. · DOI · Paper page
  79. Yuzhen Zhang, Jingjing Liu, and Wenjuan Shen. A review of ensemble learning algorithms used in remote sensing applications. Applied Sciences, 12(17):8654, 2022.
  80. Chen Zhi, Liye Cheng, Meilin Liu, Xinkui Zhao, Yueshen Xu, and Shuiguang Deng. Llm-powered zero-shot online log parsing. In 2024 IEEE International Conference on Web Services (ICWS), pages 877-887, 2024. doi: 10.1109/ICWS62655. 2024.00106. · DOI
  81. Aoxiao Zhong, Dengyao Mo, Guiyang Liu, Jinbu Liu, Qingda Lu, Qi Zhou, Jiesheng Wu, Quanzheng Li, and Qingsong Wen. Logparser-llm: Advancing efficient log parsing with large language models, 2024. URL https://arxiv.org/abs/ 2408.13727.
  82. Zhi-Hua Zhou and Zhi-Hua Zhou. Ensemble learning. Springer, 2021.

Cited by

Sorted by citations

Authors
Hadadi, F., Xu, Q., Bianculli, D., & Briand, L.
Affiliations
  • University of Ottawa (CA)
  • University of Limerick (IE)
  • University of Luxembourg (LU)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
35
Issue
8
Pages
1-48
ISSN
1049-331X (ISSN-L)
DOI
10.48550/arxiv.2406.07467
arXiv
2406.07467
OpenAlex
W4414993542
Cited by
7
References
0
Citations by year
2026: 2 · 2025: 5
FWCI
2.96
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Transactions on Software Engineering and Methodology (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Open Repository and Bibliography (University of Luxembourg) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Canada Research Chairs
  • Fonds National de la Recherche Luxembourg (C22/IS/17373407/LOGODOR)
  • Alliance de recherche numérique du Canada
  • Natural Sciences and Engineering Research Council of Canada (13/RC/2094-2)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
Keywords
log anomaly detection, large language models, retrieval-augmented generation, ensemble learning, data-efficient learning, hybrid machine learning, Mistral, decision tree, k-nearest neighbors, feedforward neural networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review