The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index; 46 of the 73 sources its claims cite were checked for support.
The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines
Open accessAbstract
Cybersecurity operations are increasingly adopting agentic AI solutions due to the time-critical and complex decision-making in security operations centers (SOCs). While large language models (LLMs) are good with summarization tasks or interpreting structured and unstructured reports, real-world SOC workflows have additional requirements such as access to original logs, reproducibility and accountability to triage security incidents. For example, analysts routinely correlate alerts to understand the kill-chain of the cyber-attack and analyze the event telemetries to identify the root cause event which may not have triggered an alert. Incorrect and incomplete automations in such settings can directly impact production systems and business operations.In this survey, we examine the architectural shifts from single-model assistants to tool-augmented agents, distributed multiagent systems, and schema-constrained investigation pipelines. We introduce a five-generation taxonomy that represents the evolution of agentic AI systems, their limitations and risks across different parameters, such as reasoning depth, tool interaction, memory, reproducibility and safety. We also review the emerging benchmarks to evaluate cyber-oriented agents and identify open challenges including response validation, tool-use correctness, multi-agent coordination, long-horizon reasoning and safeguards for high-impact actions. Finally, we discuss how these challenges influence deployment decisions in operational SOC environments. Our analysis provides a structured perspective on the current state of agentic AI in cybersecurity and highlights the technical and governance considerations necessary for its deployment.
Overview
CoChat Summary
TL;DR
This survey traces the architectural evolution of agentic AI in cybersecurity through a five-generation taxonomy, from stateless single-LLM reasoners to schema-constrained tool ecosystems and fully autonomous SOC pipelines. It argues that each capability gain brings new failure modes—hallucination, error cascades, inter-agent misalignment, and unsafe autonomous action—making verification, governance, and human oversight increasingly necessary.
Key findings
- The paper proposes a five-generation taxonomy: Gen-1 single LLM reasoners (stateless, text-only, no tool access), Gen-2 tool-augmented agents (ReAct/Planner-Executor loops, but single point of failure), Gen-3 multi-agent systems (triage/investigator/verifier roles, e.g., CORTEX and IRCopilot), Gen-4 MCP-based standardized tool ecosystems (schema-bound, auditable, reproducible), and Gen-5 autonomous pipelines (goal-driven, minimal human intervention).
- Capability gains track oversight demands: Gen-1 offers only static recall and single-step reasoning; Gen-2 adds multi-step planning but lacks long-term memory; Gen-3 adds shared long-horizon context and cross-agent verification; Gen-4 adds schema-validated, logged, replayable interactions; Gen-5 offers full autonomy "but risk-sensitive."
- IRCopilot reports up to ~150% more tasks completed compared to a traditional single monolithic LLM, cited as evidence for semi-automated orchestration with constrained autonomy.
- Evaluation gaps are substantial: no benchmark measures end-to-end SOC workflows (detection through reporting), multi-agent collaboration, tool-use correctness, or verifier/critic effectiveness; existing benchmarks (AgentBench, DefenderBench, CyBench, SecEval, AttackSeqBench, AutoPenBench) lack realistic SOC noise and ambiguity.
Methods
- Design: a survey with two research questions—(1) the architectural evolution of cybersecurity agents over time and (2) the capabilities and risks emerging as evolution progresses.
- Scope: agentic AI for SOC workflows, threat-intelligence processing, detection engineering, and incident-response automation; temporal boundary 2020–2025. Excludes classic ML classifiers, LLM red-team automation, synthetic phishing generation, and generic chat assistants.
- Analysis: cross-generational comparative analysis across reasoning depth, tool interaction, memory, reproducibility, and safety (Table 1), plus a synthesis of cyber-agent benchmarks (Table 2) and a structured research agenda.
Limitations
- Stated: the survey focuses on architectural evolution rather than product comparison or vendor assessment; the authors note that vendor and open-source platforms often do not sufficiently describe their agentic system architectural underpinnings, and that academic literature on this topic is disjointed.
- Evident from the text: the taxonomy is presented without a systematic literature-search protocol (no databases, inclusion criteria counts, or PRISMA-style flow reported), so coverage and selection bias cannot be assessed.
- Evident from the text: quantitative claims rest on secondary citation rather than the authors' own experiments—the ~150% IRCopilot figure is reported from reference [34], and the benchmark comparison table is a qualitative synthesis rather than an empirical evaluation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
32 Valid · 4 Not valid · 18 Not checked · 46 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“but static LLM-only systems still don't suffice in realistic defensive environments [3].”
What the sources say
- Partly supportedcites [3] Wu, 2024, A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems sourceExperimental — accuracy not yet measured
The Introduction (p. 1) says that studying LLMs as stand-alone models misses security issues at the level of the full system, and Related Work (p. 3) says real-world systems involve more complex interactions with tools. That supports the need to look beyond an LLM alone, but the paper does not assess whether static LLM-only systems suffice for defensive work in realistic environments. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“They remain suitable for summarizing, explaining, and threat-intel digestion [7]”
What the sources say
- Partly supportedcites [7] sourceExperimental — accuracy not yet measured
The Threat intelligence section describes LLMs summarizing threat information and explaining cyberattack techniques. It does not identify these capabilities as specific to “Gen-1” LLMs or assess whether that generation remains suitable for them. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“It allows the multi-step multi-task complexity (i.e., associating multiple log sources) to be passed between agents, leading to more reliable reasoning [33], [34].”
What the sources say
- Supportedcites [33] Wei, 2025, CORTEX: Collaborative LLM Agents for High-Stakes Alert Triage source
The Introduction (p. 2) describes specialized agents querying SIEM logs and threat intelligence, passing structured messages, and cross-checking claims before a coordination agent synthesizes a decision. The Conclusion (p. 6) reports improved decision quality relative to single-agent baselines. This supports the claim’s account of distributed investigative work leading to more reliable reasoning. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [34] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This scope accords with new studies suggesting that cyber-defense tasks increasingly involve multi-hop reasoning in communication with operational sources of data [7], [8].”
What the sources say
- Partly supportedcites [7] sourceExperimental — accuracy not yet measured
The Further research section (page unavailable) says cybersecurity challenges call for planning, reasoning and tool use, and describes defense agents drawing on troubleshooting guides, incident histories and user instructions. It does not specifically establish multi-hop reasoning in communication with operational data sources, or show that such tasks are increasingly common. (LLM, conf=0.82)
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
Section 3.2 says the defense method can explore multi-hop attack paths, and Section 3.1 describes network-state information and detection results from security devices. The paper does not establish that cyber-defense tasks are increasingly conducted through communication with operational data sources; its method is tested in a simulated environment (Section 5.1). (LLM, conf=0.82)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“To deploy in a SOC context with regulatory/compliance requirements, auditability, and verifiability are critical”
What the sources say
- Supportedcites [6] Bollikonda, 2025, Secure Pipelines, Smarter AI: LLM-Powered Data Engineering for Threat Detection and Compliance source
Section 8 says the ability to audit and reconstruct AI decisions is critical in regulated environments. It also describes audit records feeding SOC alerts and compliance systems, and says auditors can validate that regulatory boundaries were not crossed. This supports the claim’s auditability and verifiability requirements in a SOC context. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
45 more claims · 15 with their cited sources checked
Create a free account to see all 100 claims and to verify the 4 references still open.
- Probably not a substantial claim. 20
- It may state two separate claims as one. 37
Citations
Cites
AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671 · cited 1×
“Rather than having one LLM plan, execute, and evaluate tasks, the systems now orchestrate a series of specialized agents sharing context, critiquing one another, and modularizing the incident-response pipeline [30], [31], [32].”
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language ModelsPreprint
Zhang, A. K., Perry, N., Dulepet, R., Ji, J., Menders, C., Lin, J. W., Jones, E. K., Hussein, G., Liu, S., Jasper, D. J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.08926 · Cited by 266
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207 · cited 1×
“Rather than having one LLM plan, execute, and evaluate tasks, the systems now orchestrate a series of specialized agents sharing context, critiquing one another, and modularizing the incident-response pipeline [30], [31], [32].”
Large Language Models for Cyber Security: A Systematic Literature Review
Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3769676 · Cited by 189
Raza, S., Sapkota, R., Karkee, M., & Emmanouilidis, C. (2025) · AI Open · doi:10.48550/arxiv.2506.04133 · Cited by 134
LLMs Will Always Hallucinate, and We Need to Live with This
Banerjee, S., Agarwal, A., & Singla, S. (2025) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-031-99965-9_39 · Cited by 93
Large Language Models in Cybersecurity: State-of-the-Art
Motlagh, F. N., Hajizadeh, M., Majd, M., Najafi, P., Cheng, F., & Meinel, C. (2024) · arXiv · doi:10.48550/arxiv.2402.00891 · Cited by 92 · cited 1×
“Recent state-of-the-art reviews in the area of cybersecurity argue that LLMs facilitate innovative defensive capabilities; however, their reliability, controllability, and auditability are still key open challenges to mission-critical deployment [20].”
Large Language Models in Cybersecurity: State-of-the-Art
Motlagh, F., Hajizadeh, M., Majd, M., Najafi, P., Cheng, F., & Meinel, C. (2025) · Proceedings of the 11th International Conference on Information Systems Security and Privacy · doi:10.5220/0013377600003899 · Cited by 92
Research works printed but not linked: 2
- Bandara, 2024, Hallucination as Disinformation: The Role of LLMs in Amplifying Conspiracy Theories and Fake News, 14, 65
- Liu, CyberBench: A Multi-Task Benchmark for Evaluating Large Language Models in Cybersecurity
Cited by
Chen, Z., Zhen, H., Zhu, Z., Jiang, J., Kong, M., & Gu, X. (2026) · International Conference on Communications, Information System and Computer Engineering (CISCE) · doi:10.1109/cisce69494.2026.11504945 · Cited by 3
Kizhakel, S. P., & Pande, S. D. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02058-6
“Tools and frameworks that are built with multi agent frameworks and are specifically configured for a specific set of tasks perform better than a single agent based system [10,11].”
A Threat Modeling Prioritization and Automation Framework for Composable Architectures
Popescu, L.-M., & Brad, R. (2026) · Future Internet · doi:10.3390/fi18100499
Metadata
- Authors
- Vinay, V.
- Affiliations
- Microsoft (United States) (US)
- Corresponding author
- Vinay, Vaishali
- Venue
- 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC)
- Publisher
- IEEE
- Published
- 2026-02-18
- Pages
- 1-8
- OpenAlex
- W7131102438
- Semantic Scholar
- 6417c2e047247e4e844c8ab390fb3812fb4f80e1
- Cited by
- 15
- References
- 27
- Citations by year
- 2026: 15
- FWCI
- 108.27
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE International Conference on AI in Cybersecurity (ICAIC) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- agentic AI, large language models, multi-agent systems, tool-augmented agents, alert correlation, agent taxonomy, agent evaluation benchmarks, multi-agent coordination, long-horizon reasoning, reproducibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines. https://app.cochat.ai/papers/10.1109/icaic67076.2026.11395809
- The paper
- Vinay, V. (2026). The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines. 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC). https://doi.org/10.1109/icaic67076.2026.11395809
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.