Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index; 46 of the 73 sources its claims cite were checked for support.

The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines

Open access
Vinay, Vaishali · 2026 · 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC)

Abstract

Cybersecurity operations are increasingly adopting agentic AI solutions due to the time-critical and complex decision-making in security operations centers (SOCs). While large language models (LLMs) are good with summarization tasks or interpreting structured and unstructured reports, real-world SOC workflows have additional requirements such as access to original logs, reproducibility and accountability to triage security incidents. For example, analysts routinely correlate alerts to understand the kill-chain of the cyber-attack and analyze the event telemetries to identify the root cause event which may not have triggered an alert. Incorrect and incomplete automations in such settings can directly impact production systems and business operations.In this survey, we examine the architectural shifts from single-model assistants to tool-augmented agents, distributed multiagent systems, and schema-constrained investigation pipelines. We introduce a five-generation taxonomy that represents the evolution of agentic AI systems, their limitations and risks across different parameters, such as reasoning depth, tool interaction, memory, reproducibility and safety. We also review the emerging benchmarks to evaluate cyber-oriented agents and identify open challenges including response validation, tool-use correctness, multi-agent coordination, long-horizon reasoning and safeguards for high-impact actions. Finally, we discuss how these challenges influence deployment decisions in operational SOC environments. Our analysis provides a structured perspective on the current state of agentic AI in cybersecurity and highlights the technical and governance considerations necessary for its deployment.

Overview

CoChat Summary

TL;DR

This survey traces the architectural evolution of agentic AI in cybersecurity through a five-generation taxonomy, from stateless single-LLM reasoners to schema-constrained tool ecosystems and fully autonomous SOC pipelines. It argues that each capability gain brings new failure modes—hallucination, error cascades, inter-agent misalignment, and unsafe autonomous action—making verification, governance, and human oversight increasingly necessary.

Key findings

  • The paper proposes a five-generation taxonomy: Gen-1 single LLM reasoners (stateless, text-only, no tool access), Gen-2 tool-augmented agents (ReAct/Planner-Executor loops, but single point of failure), Gen-3 multi-agent systems (triage/investigator/verifier roles, e.g., CORTEX and IRCopilot), Gen-4 MCP-based standardized tool ecosystems (schema-bound, auditable, reproducible), and Gen-5 autonomous pipelines (goal-driven, minimal human intervention).
  • Capability gains track oversight demands: Gen-1 offers only static recall and single-step reasoning; Gen-2 adds multi-step planning but lacks long-term memory; Gen-3 adds shared long-horizon context and cross-agent verification; Gen-4 adds schema-validated, logged, replayable interactions; Gen-5 offers full autonomy "but risk-sensitive."
  • IRCopilot reports up to ~150% more tasks completed compared to a traditional single monolithic LLM, cited as evidence for semi-automated orchestration with constrained autonomy.
  • Evaluation gaps are substantial: no benchmark measures end-to-end SOC workflows (detection through reporting), multi-agent collaboration, tool-use correctness, or verifier/critic effectiveness; existing benchmarks (AgentBench, DefenderBench, CyBench, SecEval, AttackSeqBench, AutoPenBench) lack realistic SOC noise and ambiguity.

Methods

  • Design: a survey with two research questions—(1) the architectural evolution of cybersecurity agents over time and (2) the capabilities and risks emerging as evolution progresses.
  • Scope: agentic AI for SOC workflows, threat-intelligence processing, detection engineering, and incident-response automation; temporal boundary 2020–2025. Excludes classic ML classifiers, LLM red-team automation, synthetic phishing generation, and generic chat assistants.
  • Analysis: cross-generational comparative analysis across reasoning depth, tool interaction, memory, reproducibility, and safety (Table 1), plus a synthesis of cyber-agent benchmarks (Table 2) and a structured research agenda.

Limitations

  • Stated: the survey focuses on architectural evolution rather than product comparison or vendor assessment; the authors note that vendor and open-source platforms often do not sufficiently describe their agentic system architectural underpinnings, and that academic literature on this topic is disjointed.
  • Evident from the text: the taxonomy is presented without a systematic literature-search protocol (no databases, inclusion criteria counts, or PRISMA-style flow reported), so coverage and selection bias cannot be assessed.
  • Evident from the text: quantitative claims rest on secondary citation rather than the authors' own experiments—the ~150% IRCopilot figure is reported from reference [34], and the benchmark comparison table is a qualitative synthesis rather than an empirical evaluation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/icaic67076.2026.11395809

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

32 Valid · 4 Not valid · 18 Not checked · 46 Own results and interpretations

45 more claims · 15 with their cited sources checked

Create a free account to see all 100 claims and to verify the 4 references still open.

See all claims — free account
  • Probably not a substantial claim. 20
  • It may state two separate claims as one. 37

Create a free account to see these claims.

Citations

Cites

Show all 27 references

Research works printed but not linked: 2

  1. Bandara, 2024, Hallucination as Disinformation: The Role of LLMs in Amplifying Conspiracy Theories and Fake News, 14, 65
  2. Liu, CyberBench: A Multi-Task Benchmark for Evaluating Large Language Models in Cybersecurity

Cited by

See all 15

Metadata

Authors
Vinay, V.
Affiliations
Microsoft (United States) (US)
Corresponding author
Vinay, Vaishali
Venue
2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC)
Publisher
IEEE
Published
2026-02-18
Pages
1-8
OpenAlex
W7131102438
Cited by
15
References
27
Citations by year
2026: 15
FWCI
108.27
Type
conference paper
Language
en
Open access
Locations
  • IEEE International Conference on AI in Cybersecurity (ICAIC) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
  • Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
agentic AI, large language models, multi-agent systems, tool-augmented agents, alert correlation, agent taxonomy, agent evaluation benchmarks, multi-agent coordination, long-horizon reasoning, reproducibility

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines. https://app.cochat.ai/papers/10.1109/icaic67076.2026.11395809
The paper
Vinay, V. (2026). The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines. 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC). https://doi.org/10.1109/icaic67076.2026.11395809

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.