Transformer-based LLMs in Cybersecurity: An in-depth Study on Log Anomaly Detection and Conversational Defense Mechanisms
The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Abstract
With the advancement of conversational AI and Large Language Models (LLMs), interactive chatbots are emerging as pivotal assets for connecting with users across various sectors, enabling various capabilities and functions. However, their potential in the cybersecurity domain remains largely untapped. This article introduces a novel method to enhance chatbot performance by incorporating anomaly detection features. Our chatbot uses advanced GPT-3 models and rule-based logic to identify and extract unusual patterns and deviations within logs, making it more proficient in detecting anomalies. We present the architecture and methodology behind our anomaly detection system, showcasing its effectiveness in real-world scenarios. Combining machine learning and domain expertise, our chatbot sets a new standard in interactive, anomaly-aware conversational agents. Our anomaly detection classifier was able to achieve more than 99% of accuracy by illustrating its robust performance in accurately identifying and flagging outliers or unusual patterns in log file data. We also compared the performance of GPT-3 models with other LLMs: BERT, DistilBERT, and ALBERT. Our findings concluded that GPT-3 models consistently outperform all the other LLM models and exhibit significantly higher performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The first is the emergence of large language models (LLMs) capable of zero-shot reasoning over heterogeneous textual artifacts-packet captures rendered as text, system logs, threat-intelligence reports, MITRE ATT&CK technique descriptions, CVE narratives-without bespoke feature engineering”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [16]
In the citing paper
“Hybrid IDS architectures that combine line-rate ML detectors with gateway-resident LLMs for triage are gaining traction because they balance latency (tens of milliseconds) with reasoning capacity [16,68,74,87].”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [16]
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“The BERT model, developed by Google AI [5], stands as a state-of-the-art transformer-based model that is pre-trained bidirectionally on an extensive textual dataset.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 1×
“The BERT model, developed by Google AI [5], stands as a state-of-the-art transformer-based model that is pre-trained bidirectionally on an extensive textual dataset.”
MethodDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 1×
“2) DistilBERT DistilBERT is perceived as a streamlined and swifter counterpart to BERT, birthed by Hugging Face [11].”
BackgroundALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 1×
“3) ALBERT (A Lite BERT) ALBERT, developed by Google AI [9], emerged as a solution to address BERT’s limitations, specifically targeting a diminished model size and augmented training efficacy.”
MethodExperience Report: System Log Analysis for Anomaly Detection
He, S., Zhu, J., He, P., & Lyu, M. R. (2016) · 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2016.21 · Cited by 593 · cited 1×
“[7].”
Loghub: a large collection of system log datasets for AI-driven log analytics
Zhu, J., He, S., He, P., Liu, J., & Lyu, M. R. (2023) · 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre59848.2023.00071 · Cited by 302 · cited 1×
“To support this training, we utilize a comprehensive and annotated open-source dataset [17] from prior research on anomaly detection [18], establishing a robust groundwork for detecting anomalies.”
Evaluation of Machine Learning Algorithms for Anomaly Detection
Elmrabit, N., Zhou, F., Li, F., & Zhou, H. (2020) · 2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security) · doi:10.1109/cybersecurity49315.2020.9138871 · Cited by 212 · cited 1×
“Elmrabit et al. [6] conducted a comprehensive comparison of various machine learning algorithms, including prominent algorithms like Long Short-Term Memory (LSTM), Decision Tree (DT), Random Forest (RF), Deep Neural Network (DNN), etc.”
Experience Report: Log Mining Using Natural Language Processing and Application to Anomaly Detection
Bertero, C., Roy, M., Sauvanaud, C., & Tredan, G. (2017) · 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre.2017.43 · Cited by 139 · cited 1×
“In [2], the authors employed the word2vec model to produce word embeddings for log data.”
Dynamic log file analysis: An unsupervised cluster evolution approach for anomaly detection
Landauer, M., Wurzenberger, M., Skopik, F., Settanni, G., & Filzmoser, P. (2018) · Computers & Security · doi:10.1016/j.cose.2018.08.009 · Cited by 89 · cited 2×
“The importance of identifying anomalies within logs has been underscored in various studies [10, 13, 4].”
BackgroundMachine learning to detect anomalies in web log analysis
Cao, Q., Qiao, Y., & Lyu, Z. (2017) · 2017 3rd IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/compcomm.2017.8322600 · Cited by 55 · cited 2×
“The importance of identifying anomalies within logs has been underscored in various studies [10, 13, 4].”
GPT-2C: a parser for honeypot logs using large pre-trained language models
Setianto, F., Tsani, E., Sadiq, F., Domalis, G., Tsakalidis, D., & Kostakos, P. (2021) · Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining · doi:10.1145/3487351.3492723 · Cited by 35
Self-citationOn the Power of Special-purpose GPT Models to Create and Evaluate New Poetry in Old Styles
Sawicki, P., Grześ, M., Góes, L. F. W., (5017016), D. B., Peeperkorn, M., Khatun, A., & Paraskevopoulou, S. (2023) · Kent Academic Repository (University of Kent) · Cited by 27
Log Files Analysis For Network Intrusion Detection
Brandão, A., & Georgieva, P. (2020) · IEEE International Conference on Intelligent Systems · doi:10.1109/is48319.2020.9199976 · Cited by 19 · cited 1×
“In their study, Brandao and Georgieva [3] evaluated various machine learning techniques for analyzing log files from a Network Intrusion detection system, and as per their final results, the Decision tree model has achieved better accuracy compared to K-Nearest Neighbours and the basic neural…”
Log File Analysis Based on Machine Learning: A Survey
Abdalla, R. R., & Jumaa, A. K. (2022) · UHD Journal of Science and Technology · doi:10.21928/uhdjst.v6n2y2022.pp77-84 · Cited by 17
Event log analysis with the LogCluster tool
Vaarandi, R., Kont, M., & Pihelgas, M. (2016) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom.2016.7795458 · Cited by 16 · cited 2×
“As such, log analysis has attracted significant attention, as evidenced by numerous works in the field [15, 1].”
Trace Bench: An Open Data Set for Trace-Oriented Monitoring
Zhou, J., Chen, Z., Wang, J., Zheng, Z., & Lyu, M. R.-T. (2014) · IEEE International Conference on Cloud Computing Technology and Science · doi:10.1109/cloudcom.2014.79 · Cited by 10 · cited 1×
“To support this training, we utilize a comprehensive and annotated open-source dataset [17] from prior research on anomaly detection [18], establishing a robust groundwork for detecting anomalies.”
Research works printed but not linked: 3
- He, 2020, Loghub: A large collection of system log datasets towards automated log analytics, arXiv preprint arXiv:2008.06448 · arXiv
- Saxena, 2021, IoT Dynamic Log File Analysis: Security Approach for Anomaly Detection In Multi Sensor Environment, International research journal of modernization in engineering technology & Science, 3, 2582
- 10.7717/peerj.12726/table-1 · doi:10.7717/peerj.12726/table-1
Cited by
Kheddar, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103347 · Cited by 161
A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions
Senevirathna, T., La, V. H., Marchal, S., Siniarski, B., Liyanage, M., & Wang, S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3437248 · Cited by 152
“Furthermore, LLM-based penetrating testing tools (e.g.: PentestGPT [169]), anomaly detection tools (e.g., PAC-GPT [170], LogBERT [171], LogBot [172]), and threat detection tools (e.g., TSTEM [173], Cyber Sentinel [174]) The rise of this new addition to the security stack of 5G and beyond begs the…”
Balasubramanian, P., Liyana, S., Sankaran, H., Sivaramakrishnan, S., Pusuluri, S., Pirttikangas, S., & Peltonen, E. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11338-z · Cited by 81
Self-citationLLMs are one-shot URL classifiers and explainers
Rashid, F., Ranaweera, N., Doyle, B., & Seneviratne, S. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.111004 · Cited by 40
CYGENT: A cybersecurity conversational agent with log summarization powered by GPT-3
Balasubramanian, P., Seby, J., & Kostakos, P. (2024) · International Conference on Artificial Intelligence for Internet of Things (AIIoT) · doi:10.1109/aiiot58432.2024.10574658 · Cited by 26
“As part of feature development, in our previous paper [11] we developed an anomaly detection feature for the CYGENT framework which uses fine tuned GPT3 models for detection of anomalies in the log data.”
MethodSelf-citation
Metadata
- Authors
- Balasubramanian, P., Seby, J., & Kostakos, P.
- Affiliations
- University of Oulu (FI)
- Venue
- 2023 IEEE International Conference on Big Data (BigData)
- Publisher
- IEEE
- Published
- 2023-12-15
- Pages
- 3590-3599
- OpenAlex
- W4391094158
- Cited by
- 22
- References
- 17
- Citations by year
- 2026: 4 · 2025: 12 · 2024: 6
- FWCI
- 6.65
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Big Data (BigData) (publishedVersion)
- Funders
- European Commission
- Academy of Finland
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Keywords
- log anomaly detection, chatbots, large language models, GPT-3, cybersecurity, transformer models, BERT, DistilBERT, ALBERT, model comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.