CoChat
Sign Up
PreprintCited by 1Auction Theory and ApplicationsInformation and Cyber SecurityNetwork Security and Intrusion Detection

Strategic Cyber Defense via Reinforcement Learning-Guided Combinatorial Auctions

Open access
Pham, Mai, Vaze, Vikrant, Chin, Peter·2025·arXiv
DOI 10.48550/arxiv.2509.10983arXiv 2509.10983
Open this paper in CoChat — free accountOpen PDF

Abstract

Cyber defense operations increasingly require long-term strategic planning under uncertainty and resource constraints. We propose a new use of combinatorial auctions for allocating defensive action bundles in a realistic cyber environment, using host-specific valuations derived from reinforcement learning (RL) Q-values. These Q-values encode long-term expected utility, allowing upstream planning. We train CAFormer, a differentiable Transformer-based auction mechanism, to produce allocations that are approximately incentive-compatible under misreporting. Rather than benchmarking against existing agents, we explore the qualitative and strategic properties of the learned mechanisms. Compared to oracle and heuristic allocations, our method achieves competitive revenue while offering robustness to misreporting. In addition, we find that allocation patterns correlate with adversarial and defensive activity, suggesting implicit alignment with operational priorities. Our results demonstrate the viability of auction-based planning in cyber defense and highlight the interpretability benefits of RL-derived value structures.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2509.10983

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In adversarial settings, decentralization enhances fault tolerance and responsiveness but also demands robust coordination mechanisms to ensure alignment in the presence of strategic manipulation or partial compromise.

    “In adversarial settings, decentralisation enhances fault tolerance and responsiveness but also demands robust coordination mechanisms to ensure alignment in the presence of strategic manipulation or partial compromise (Pham et al., 2025).”

    From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Pham et al., 2025

    The Introduction (pp. 1–2) describes decentralized cyber-defense planning and an incentive-compatible auction intended to discourage agents from misreporting valuations. The paper therefore supports the need for coordination under strategic manipulation, but the cited passages do not establish that decentralization enhances fault tolerance or responsiveness, or address partial compromise. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 24

  1. [1][1] Q. Zhu and T. Basar, 'Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems: Games-in-games principle for optimal cross-layer resilient control systems,' IEEE Control Systems Magazine , vol. 35, no. 1, pp. 46-65, 2015. -
  2. [2][2] A. Schlenker, H. Xu, M. Guirguis, C. Kiekintveld, A. Sinha, M. Tambe, S. Sonya, D. Balderas, and N. Dunstatter, 'Don't bury your head in warnings: A game-theoretic approach for intelligent allocation of cybersecurity alerts,' in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17 , 2017, pp. 381-387. [Online]. Available: https://doi.org/10.24963/ijcai.2017/54 - · DOI
  3. [3][3] M. Kiely, D. Bowman, M. Standen, and C. Moir, 'On autonomous agents in a cyber defence environment,' arXiv preprint arXiv:2309.07388 , 2023. - · arXiv
  4. [4][4] A. D. Kent, 'Comprehensive, multi-source cyber-security events data set,' 05 2015. [Online]. Available: https://www.osti.gov/biblio/1179829 -
  5. [5][5] R. Lippmann and K. Ingols, 'An annotated review of past papers on attack graphs,' MIT Lincoln Laboratory Technical Report , 2005. -
  6. [6][6] Y. Yun, X. Xi-shan, and Q. Zhi-chang, 'A probabilistic computing approach of attack graph-based nodes in large-scale network,' Procedia Environmental Sciences , vol. 10, pp. 3-8, 2011, 2011 3rd International Conference on Environmental Science and Information Application Technology ESIAT 2011. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1878029611001976 -
  7. [7][7] J. Zhang, N. Xie, X. Zhang, and W. Li, 'An online auction mechanism for cloud computing resource allocation and pricing based on user evaluation and cost,' Future Generation Computer Systems , vol. 89, pp. 286-299, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X17317983 -
  8. [8][8] T. Alpcan and T. Bas ¸ar, Network Security: A Decision and GameTheoretic Approach . Cambridge University Press, 2010. -
  9. [9][9] O. Sheyner, J. Haines, S. Jha, R. Lippmann, and J. M. Wing, 'Automated generation and analysis of attack graphs,' in Proceedings 2002 IEEE Symposium on Security and Privacy . IEEE, 2002, pp. 273-284. -
  10. [10][10] K. Durkota, V. Lisy, B. Boˇ sansky, and C. Kiekintveld, 'Optimal network security hardening using attack graph games,' in Proceedings of the 24th International Conference on Artificial Intelligence , ser. IJCAI'15. AAAI Press, 2015, p. 526-532. -
  11. [11][11] T. T. Nguyen and V. J. Reddi, 'Deep reinforcement learning for cyber security,' IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 8, pp. 3779-3795, 2023. -
  12. [12][12] G. Li, P. Sharma, L. Pan, S. Rajasegarar, C. Karmakar, and N. Patterson, 'Deep learning algorithms for cyber security applications: A survey,' J. Comput. Secur. , vol. 29, no. 5, p. 447-471, Jan. 2021. [Online]. Available: https://doi.org/10.3233/JCS-200095 - · DOI
  13. [13][13] M. Pham, V. Vaze, and P. Chin, 'Advancing differentiable economics: A neural network framework for revenue-maximizing combinatorial auction mechanisms,' 2025. [Online]. Available: https://arxiv.org/abs/2501.19219 - · arXiv
  14. [14][14] S. Vyas, J. Hannay, A. Bolton, and P. Burnap, 'Automated cyber defence: A review,' arXiv preprint arXiv:2303.04926 , 2023. - · arXiv
  15. [17][17] A. Molina-Markham, C. Miniter, B. Powell, and A. Ridley, 'Network environment design for autonomous cyberdefense,' arXiv preprint arXiv:2103.07583 , 2021. [Online]. Available: https://arxiv.org/abs/2103.07583 - · arXiv
  16. [18][18] S. Cheung, J. Claypoole, P. Sharma, V. Yegneswaran, A. Gehani, S. Jha, J. Emanuello, and A. Ridley, 'Eireland: Evaluating and interpreting reinforcement-learning-based network defenses,' https://sites.google.com/go.spawar.navy.mil/acd-2023/proceedings, 2023, accessed: 2025-06-23. -
  17. [19][19] M. Foley, C. Hicks, K. Highnam, and V. Mavroudis, 'Autonomous network defence using reinforcement learning,' Proceedings of the 2022 ACM Workshop on Artificial Intelligence and Security , pp. 79-90, 2022. -
  18. [20][20] M. Wolk, A. Applebaum, C. Dennler, P. Dwyer, M. Moskowitz, H. Nguyen, N. Nichols, N. Park, P. Rachwalski, F. Rau, and A. Webster, 'Beyond cage: Investigating generalization of learned autonomous network defense policies,' arXiv preprint arXiv:2211.15557 , 2022. - · arXiv
  19. [21][21] B. B. Gupta and A. Dahiya, 'An auction-based incentivized solution against ddos attacks,' J. Inf. Technol. Res. , vol. 14, no. 1, p. 1-19, Jan. 2021. [Online]. Available: https://doi.org/10.4018/JITR.2021010101 - · DOI
  20. [22][22] P. Duetting, Z. Feng, H. Narasimhan, D. Parkes, and S. Ravindranath, 'Optimal auctions through deep learning,' in International Conference on Machine Learning (ICML) , 2019, pp. 1706-1715. -
  21. [23][23] J. Rahme, S. Jelassi, J. Bruna, and S. M. Weinberg, 'A permutationequivariant neural network architecture for auction design,' 2021. [Online]. Available: https://arxiv.org/abs/2003.01497 - · arXiv
  22. [24][24] V. Ivanov, R. Rahme, A. Bian, and D. C. Parkes, 'Optimal-er auctions through attention,' in Advances in Neural Information Processing Systems (NeurIPS) , 2022. -
  23. [25][25] R. Duan, R. Rahme, and D. C. Parkes, 'A scalable neural network for dsic affine maximizer auction design,' Journal of Artificial Intelligence Research (JAIR) , vol. 79, pp. 543-576, 2024. -
  24. [27][27] J. Claypoole, S. Cheung, A. Gehani, V. Yegneswaran, and A. Ridley, 'Interpreting agent behaviors in reinforcement-learningbased cyber-battle simulation platforms,' 2025. [Online]. Available: https://arxiv.org/abs/2506.08192 · arXiv

Cited by

Sorted by citations

Authors
Pham, M., Vaze, V., & Chin, P.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2509.10983
arXiv
2509.10983
OpenAlex
W4415087607
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Auction Theory and Applications
  • Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
Keywords
cyber defense, combinatorial auctions, reinforcement learning, Q-value, incentive compatibility, misreporting, resource allocation under uncertainty, revenue maximization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review