A Novel Framework for Enhancing Decision-Making in Autonomous Cyber Defense Through Graph Embedding
Open accessAbstract
Faced with challenges posed by sophisticated cyber attacks and dynamic characteristics of cyberspace, the autonomous cyber defense (ACD) technology has shown its effectiveness. However, traditional decision-making methods for ACD are unable to effectively characterize the network topology and internode dependencies, which makes it difficult for defenders to identify key nodes and critical attack paths. Therefore, this paper proposes an enhanced decision-making method combining graph embedding with reinforcement learning algorithms. By constructing a game model for cyber confrontations, this paper models important elements of the network topology for decision-making, which guide the defender to dynamically optimize its strategy based on topology awareness. We improve the reinforcement learning with the Node2vec algorithm to characterize information for the defender from the network. And, node attributes and network structural features are embedded into low-dimensional vectors instead of using traditional one-hot encoding, which can address the perceptual bottleneck in high-dimensional sparse environments. Meanwhile, the algorithm training environment Cyberwheel is extended by adding new fine-grained defense mechanisms to enhance the utility and portability of ACD. In experiments, our decision-making method based on graph embedding is compared and analyzed with traditional perception methods. The results show and verify the superior performance of our approach in the strategy selection of defensive decision-making. Also, diverse parameters of the graph representation model Node2vec are analyzed and compared to find the impact on the enhancement of the embedding effectiveness for the decision-making of ACD.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Cyber-defense tasks increasingly involve multi-hop reasoning in communication with operational sources of data.
“This scope accords with new studies suggesting that cyber-defense tasks increasingly involve multi-hop reasoning in communication with operational sources of data [7], [8].”
From The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines (2026) · cites this paper as [8]
Section 3.2 says the defense method can explore multi-hop attack paths, and Section 3.1 describes network-state information and detection results from security devices. The paper does not establish that cyber-defense tasks are increasingly conducted through communication with operational data sources; its method is tested in a simulated environment (Section 5.1). (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
node2vec: Scalable Feature Learning for Networks
Grover, A., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939754 · Cited by 10,926 · see citing works
Outside the Closed World: On Using Machine Learning for Network Intrusion Detection
Sommer, R., & Paxson, V. (2010) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2010.25 · Cited by 2,316 · see citing works
A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks
Yin, C., Zhu, Y., Fei, J., & He, X. (2017) · IEEE Access · doi:10.1109/access.2017.2762418 · Cited by 2,100 · see citing works
Toward developing a systematic approach to generate benchmark datasets for intrusion detection
Shiravi, A., Shiravi, H., Tavallaee, M., & Ghorbani, A. A. (2012) · Computers & Security · doi:10.1016/j.cose.2011.12.012 · Cited by 1,345 · see citing works
Tambe, M. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511973031 · Cited by 691 · see citing works
Deep reinforcement learning for cyber security
Nguyen, T. T., & Reddi, V. J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3121870 · Cited by 620 · see citing works
Game strategies in network security
Lye, K.-W., & Wing, J. M. (2005) · International Journal of Information Security · doi:10.1007/s10207-004-0060-x · Cited by 445 · see citing works
PettingZoo: Gym for Multi-Agent Reinforcement LearningPreprint
Terry, J. K., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R., Santos, L., Perez, R., Horsch, C., Dieffendahl, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14471 · Cited by 193 · see citing works
Cyber Resilience of Systems and Networks
Kott, A., & Linkov, I. (2018) · Springer eBooks · doi:10.1007/978-3-319-77492-3 · Cited by 187 · see citing works
Cyber Deception: Overview and the Road Ahead
Wang, C. X., & Lu, Z. (2018) · IEEE Security & Privacy · doi:10.1109/msp.2018.1870866 · Cited by 111 · see citing works
A Survey on the Moving Target Defense Strategies: An Architectural Perspective
Zheng, J., & Namin, A. S. (2019) · Journal of Computer Science and Technology · doi:10.1007/s11390-019-1906-z · Cited by 94 · see citing works
Finding Effective Security Strategies through Reinforcement Learning and Self-Play
Hammar, K., & Stadler, R. (2020) · International Conference on Network and Service Management (CNSM) · doi:10.23919/cnsm50824.2020.9269092 · Cited by 70 · see citing works
CybORG: A Gym for the Development of Autonomous Cyber AgentsPreprint
Standen, M., Lucas, M., Bowman, D. B., Richer, T. J., Kim, J., & Marriott, D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09118 · Cited by 56 · see citing works
Applying Reinforcement Learning for Enhanced Cybersecurity against Adversarial Simulation
Oh, S. H., Jeong, M. K., Kim, H. C., & Park, J. (2023) · Sensors · doi:10.3390/s23063000 · Cited by 38 · see citing works
Tang, Y., Sun, J., Wang, H., Deng, J., Liang, T., & Xu, W. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103871 · Cited by 36 · see citing works
Yao, Q., Wang, Y., Xiong, X., Wang, P., & Li, Y. (2023) · Entropy · doi:10.3390/e25040605 · Cited by 34 · see citing works
Self-citationDecision and Game Theory for Security
Khouzani, M. H. R., Panaousis, E., & Theodorakopoulos, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25594-1 · Cited by 33 · see citing works
On Autonomous Agents in a Cyber Defence EnvironmentPreprint
Kiely, M., Bowman, D. B., Standen, M., & Moir, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.07388 · Cited by 33 · see citing works
CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network SystemsPreprint
Li, L., Fayad, R., & Taylor, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03331 · Cited by 24 · see citing works
Deep Reinforcement Learning for Cyber System Defense under Dynamic Adversarial UncertaintiesPreprint
Dutta, A., Chatterjee, S., Bhattacharya, A., & Halappanavar, M. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.01595 · Cited by 23 · see citing works
Towards a High Fidelity Training Environment for Autonomous Cyber Defense Agents
Oesch, S., Chaulagain, A., Weber, B., Dixson, M., Sadovnik, A., Roberson, B., Watson, C., & Austria, P. (2024) · Cyber Security Experimentation and Test Workshop (CSET) · doi:10.1145/3675741.3675752 · Cited by 22 · see citing works
Autonomous Intelligent Cyber Defense Agent (AICA)
Kott, A. (2023) · Advances in information security · doi:10.1007/978-3-031-29269-9 · Cited by 19 · see citing works
PenGym: Realistic training environment for reinforcement learning pentesting agents
Nguyen, H. P. T., Hasegawa, K., Fukushima, K., & Beuran, R. (2024) · Computers & Security · doi:10.1016/j.cose.2024.104140 · Cited by 16 · see citing works
Learning automated defense strategies using graph-based cyber attack simulations
Nyberg, J., & Johnson, P. (2023) · Workshop on Security Operation Center Operations and Construction (WOSOC) · doi:10.14722/wosoc.2023.23006 · Cited by 11 · see citing works
Autonomous Intelligent Cyber-defense Agent: Introduction and Overview
Kott, A. (2023) · Advances in information security · doi:10.1007/978-3-031-29269-9_1 · Cited by 11 · see citing works
Tsang, H., Akbari, I., Salahuddin, M. A., Limam, N., & Boutaba, R. (2021) · IEEE Communications Magazine · doi:10.1109/mcom.009.2100389 · Cited by 10 · see citing works
The Path to Autonomous Cyberdefense
Oesch, S., Austria, P., Chaulagain, A., Weber, B., Watson, C., Dixson, M., & Sadovnik, A. (2024) · IEEE Security & Privacy · doi:10.1109/msec.2024.3427640 · Cited by 10 · see citing works
Wang, H., Tang, Y., Wang, Y., Wei, N., Deng, J., Bin, Z., & Li, W. (2023) · High-Confidence Computing · doi:10.1016/j.hcc.2023.100145 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 5
- [8]Sengupta S. Kambhampati S. Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense arXiv 2020 cs.GT/2007.10457
- [22]Brockman G. Cheung V. Pettersson L. Schneider J. Schulman J. Tang J. Zaremba W. OpenAI Gym arXiv 2016 1606.01540
- [24]Janisch J. Pevný T. Lisý V. NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios Proceedings of the Computer Security. ESORICS 2023 International Workshops Hague, The Netherlands 25–29 September 2023 Katsikas S. Abie H. Ranise S. Verderame L. Cambiaso E. Ugarelli R. Praça I. Li W. Meng W. Furnell S. Springer Cham, Switzerland 2024 589 608
- [31]Vyas S. Hannay J. Bolton A. Burnap P.P. Automated Cyber Defence: A Review arXiv 2023 cs.CR/2303.04926
- [34]Schulman J. Wolski F. Dhariwal P. Radford A. Klimov O. Proximal Policy Optimization Algorithms arXiv 2017 1707.06347
Cited by
Sorted by citations
Vinay, V. (2026) · 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC) · doi:10.1109/icaic67076.2026.11395809 · Cited by 15 · see citing works
Autonomous Cyber Defense Agents: A Reinforcement Learning Approach to Real-Time Threat Mitigation
Girei, A. A., Abraham, F., Majekodunmi, A. O., & ALEBIOSU, J. (2025) · International Journal of Computer Applications · doi:10.5120/ijca2025925775 · Cited by 2 · see citing works
Makarem, M. A., Razaz, M. A., & Saleh, Z. (2026) · Future Internet · doi:10.3390/fi18050239 · Cited by 2 · see citing works
Machine Learning-Assisted Secure Random Communication System
Ahmed, A., & Bosnić, Z. (2025) · Entropy · doi:10.3390/e27080815 · Cited by 1 · see citing works
Adaptive Honeypot Systems via RAG: Enhancing Threat Intelligence with Generative AI
Sezgin, A. (2025) · International Journal of Innovative Engineering Applications · doi:10.46460/ijiea.1797978 · Cited by 1 · see citing works
Ahmed, A., & Bosnić, Z. (2025) · Mathematics · doi:10.3390/math13223590
Nair, M. C., R, S., Sayshan, G., Velvizhi, R., & Kabitha, P. (2026) · International Conference on Artificial Intelligence and Machine Learning Applications Theme: Healthcare and Internet of Things (AIMLA) · doi:10.1109/aimla67915.2026.11522326
AURA: Agent-based unified response architecture for retrieval-augmented cyber deception
Sezgin, A. (2026) · Journal of Innovative Engineering and Natural Science · doi:10.61112/jiens.1856246
- Authors
- Wang, Z., Wang, Y., Xiong, X., Ren, Q., & Huang, J.
- Affiliations
- National University of Defense Technology (CN)
- Corresponding author
- Wang, Yongjie
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2025-06-11
- Volume
- 27
- Issue
- 6
- Pages
- 622
- ISSN
- 1099-4300 (ISSN-L)
- PMC
- PMC12192219
- OpenAlex
- W4411208483
- Cited by
- 7
- References
- 23
- Citations by year
- 2026: 4 · 2025: 3
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Entropy (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- autonomous cyber defense, decision-making, graph embedding, node2vec, reinforcement learning, network topology, attack path identification, critical nodes, high-dimensional sparse data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
