A Systematic Comparison of Large Language Models Performance for Intrusion Detection
Open accessAbstract
We explore the capabilities of Large Language Models (LLMs) to assist or substitute devices (i.e., firewalls) and humans (i.e., security experts) respectively in the detection and analysis of security incidents. We leverage transformer-based technologies, from relatively small to foundational sizes, to address the problem of correctly identifying the attack severity (and accessorily identifying and explaining the attack type). We contrast a broad range of LLM techniques (prompting, retrieval augmented generation, and fine-tuning of several models) using state-of-the-art machine learning models as a baseline. Using proprietary data from commercial deployment, our study provides an unbiased picture of the strengths and weaknesses of LLM for intrusion detection.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Out-of-the-box LLMs may lack cybersecurity knowledge, requiring careful prompt design through few-shot exemplars, chain-of-thought prompting, and retrieval augmentation.
“Out-of-the-box LLMs may lack cybersecurity knowledge, requiring careful prompt design through few-shot exemplars, chain-ofthought prompting, and retrieval augmentation [17], [22].”
From Can Large Language Models Be Trusted for AI-Based Intrusion Detection in Cybersecurity? An Evidence-Grounded Log Analysis (2026) · cites this paper as [22]
The citation lists haven’t been fetched yet
References 16 · Cited by 19
- Authors
- Bui, M.-T., Boffa, M., Valentim, R. V., Navarro, J. M., Chen, F., Bao, X., Houidi, Z. B., & Rossi, D.
- Affiliations
- Huawei Technologies (France) (FR)
- Politecnico di Torino (IT)
- University of Turin (IT)
- ORCID
- Venue
- Proceedings of the ACM on Networking
- Published
- 2024-11-25
- Volume
- 2
- Issue
- CoNEXT4
- Pages
- 1-23
- ISSN
- 2834-5509 (ISSN-L)
- OpenAlex
- W4404691174
- Cited by
- 19
- References
- 16
- Citations by year
- 2026: 10 · 2025: 9
- FWCI
- 3.17
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Proceedings of the ACM on Networking (publishedVersion)
- PORTO@Iris (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Network Packet Processing and Optimization
- Keywords
- large language models, intrusion detection, security incident analysis, retrieval-augmented generation, prompting, LLM fine-tuning, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
