Escalation Risks from Language Models in Military and Diplomatic Decision-Making
Open accessAbstract
Governments are increasingly considering integrating autonomous AI agents in high-stakes military and foreign-policy decision-making, especially with the emergence of advanced generative AI models like GPT-4. Our work aims to scrutinize the behavior of multiple AI agents in simulated wargames, specifically focusing on their predilection to take escalatory actions that may exacerbate multilateral conflicts. Drawing on political science and international relations literature about escalation dynamics, we design a novel wargame simulation and scoring framework to assess the escalation risks of actions taken by these agents in different scenarios. Contrary to prior studies, our research provides both qualitative and quantitative insights and focuses on large language models (LLMs). We find that all five studied off-the-shelf LLMs show forms of escalation and difficult-to-predict escalation patterns. We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons. Qualitatively, we also collect the models’ reported reasoning for chosen actions and observe worrying justifications based on deterrence and first-strike tactics. Given the high stakes of military and foreign-policy contexts, we recommend further examination and cautious consideration before deploying autonomous language model agents for strategic military or diplomatic decision-making.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Automated, sophisticated offensive and defensive strategies heightens the risk of an AI-driven arms race, reduce decision-cycle times to fractions of a second (the 'flash war' scenario), lowering the threshold for conflict and increasing instability among major powers.
“Automated, sophisticated offensive and defensive strategies heightens the risk of an AIdriven arms race, reduce decision-cycle times to fractions of a second (the”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Rivera et al. (2024)
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredIn experiments with AI agents controlling eight distinct nation-states, even neutral starting conditions did not prevent the rapid emergence of arms race dynamics and aggressive strategies.
“In experiments with AI agents controlling eight distinct nation-states, even neutral starting conditions did not prevent the rapid emergence of arms race dynamics and aggressive strategies.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Rivera et al., 2024
All five off-the-shelf LLMs studied showed forms of escalation, even when peaceful alternatives were available.
“Strikingly, all five off-the-shelf LLMs studied showed forms of escalation, even when peaceful alternatives were available.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Rivera et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., & Kaplan, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08073 · Cited by 326 · see citing works
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 145 · see citing works
Flagged first, then by citations
Research works printed but not linked: 46
- Alex Braithwaite and Douglas Lemke. 2011. Unpacking escalation. Conflict Management and Peace Science , 28(2):111-123.
- Michael Brecher. 1996. Crisis escalation: Model and findings. International Political Science Review , 17(2):215-230.
- Garry D Brewer and Bruce G Blair. 1979. War games and national security with a grain of salt. Bulletin of the Atomic Scientists , 35(6):18-26.
- Charles N Brower and Jeremy K Sharpe. 2003. International arbitration and the islamic world: The third phase. American Journal of International Law , 97(3):643-656.
- Todd Burkhardt. 2017. Just war and human rights: fighting with right intention . State University of New York Press.
- Jason Davidson. 2016. The origins of revisionist and status-quo states . Springer.
- John R Emery. 2021. Moral choices without moral language: 1950s political-military wargaming at the rand corporation (fall 2021). Texas National Security Review .
- FAIR, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. 2022. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science , 378(6624):1067-1074.
- Jacques Fontanel and Michael D Ward. 1993. Military expenditures, armament, and disarmament. Defence and Peace economics , 4(1):63-78.
- Moshe Gat. 2016. Military power and foreign policy inaction: Israel, 1967-1973. Israel Affairs , 22(1):6995.
- Charles S Gochman and Zeev Maoz. 1984. Militarized interstate disputes, 1816-1976: Procedures, patterns, and insights. Journal of Conflict Resolution , 28(4):585-616.
- Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. arXiv preprint arXiv:2302.12173 . · arXiv
- Amélie Guillin. 2013. Trade in services and regional trade agreements: Do negotiations on services have to be specific? The World Economy , 36(11):1406-1423.
- Natali Helberger, Theo Araujo, and Claes H de Vreese. 2020. Who is the fairest of them all? Public attitudes and expectations regarding automated decision-making. Computer Law & Security Review , 39:105456.
- Rebecca Hersman. 2020. Wormhole escalation in the new nuclear age (summer 2020). Texas National Security Review .
- Emilio Iasiello. 2013. Cyber attack: A dull tool to shape foreign policy. In 2013 5th International Conference on Cyber Conflict (CYCON 2013) , pages 1-18. IEEE.
- Jesse C Johnson. 2015. The cost of security: Foreign policy concessions and military alliances. Journal of Peace Research , 52(5):665-679.
- Herman Kahn. 1970. The concept of escalation. In Theories of Peace and Security: A Reader in Contemporary Strategic Thought , pages 248-258. Springer.
- Herman Kahn. 2010. On escalation: Metaphors and scenarios . Routledge.
- Brandon J Kinne. 2020. The defense cooperation agreement dataset (dcad). Journal of conflict resolution , 64(4):729-755.
- Henry A Kissinger. 1969. The vietnam negotiations: Foreign affairs january 1969. Survival , 11(2):38-50.
- Erik Lin-Greenberg. 2023. Evaluating escalation: Conceptualizing escalation in an era of emerging military technologies. The Journal of Politics , 85(3):000-000.
- Erik Lin-Greenberg, Reid BC Pauly, and Jacquelyn G Schneider. 2022. Wargaming for international relations research. European Journal of International Relations , 28(1):83-109.
- James M Lindsay. 1986. Trade sanctions as policy instruments: A re-examination. International Studies Quarterly , 30(2):153-173.
- Nunzio Lorè and Babak Heydari. 2023. Strategic behavior of large language models: Game structure vs. contextual framing. arXiv preprint arXiv:2309.05898 . · arXiv
- Tshilidzi Marwala and Monica Lagazio. 2011. Support Vector Machines for Modeling Interstate Conflict. Militarized Conflict Modeling Using Computational Intelligence , pages 89-105.
- Dmitry I Mikhailov. 2023. Optimizing national security strategies through llm-driven artificial intelligence integration. arXiv preprint arXiv:2305.13927 . · arXiv
- Andrew P Owsiak. 2015. Forecasting conflict management in militarized interstate disputes. Conflict Management and Peace Science , 32(1):50-75.
- Martin Patchen. 1987. The escalation of inter-nation conflicts. Sociological Focus , 20(2):95-110.
- Duncan Purves and Ryan Jenkins. 2016. Right intention and the ends of war. Journal of Military Ethics , 15(1):18-35.
- Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Finetuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693 . · arXiv
- Derek S Reveron. 2006. Old allies, new friends: intelligence-sharing in the war on terror. Orbis , 50(3):453-468.
- Scott D Sagan. 1996. Why do states build nuclear weapons?: Three models in search of a bomb. International security , 21(3):54-86.
- Thomas C Schelling. 2006. An astonishing sixty years: The legacy of hiroshima. American Economic Review , 96(4):929-937.
- Jacquelyn Schneider, Benjamin Schechter, and Rachael Shaffer. 2022. A lot of cyber fizzle but not a lot of bang: Evidence about the use of cyber operations from wargames. Journal of Global Security Studies , 7(2):ogac005.
- Jacquelyn Schneider, Benjamin Schechter, and Rachael Shaffer. 2023. Hacking nuclear stability: Wargaming technology, uncertainty, and escalation. International Organization , 77(3):633-667.
- István Szabadföldi. 2021. Artificial intelligence in military application-opportunities and challenges. Land Forces Academy Review , 26(2):157-165.
- Hugo Touvron, Louis Martin, and Kevin Stone. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 . · arXiv
- Charles Turnitsa, Curtis Blais, and Andreas Tolk. 2022. Simulation and wargaming . Wiley Online Library.
- Brian Wade. 2018. The four critical elements of analytic wargame design. Phalanx , 51(4):18-23.
- Raymond Williams. 1980. The politics of nuclear disarmament. New Left Review , 1(124):25-42.
- Yotam Wolf, Noam Wies, Yoav Levine, and Amnon Shashua. 2023. Fundamental limitations of alignment in large language models. arXiv preprint arXiv:2304.11082 . · arXiv
- Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. 2023. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949 . · arXiv
- Yining Ye, Xin Cong, Yujia Qin, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2023. Large language model as autonomous decision maker. arXiv preprint arXiv:2308.12519 . · arXiv
- Baohui Zhang. 2010. Chinese foreign policy in transition: Trends and implications. Journal of Current Chinese Affairs , 39(2):39-68.
- Hangfan Zhang, Zhimeng Guo, Huaisheng Zhu, Bochuan Cao, Lu Lin, Jinyuan Jia, Jinghui Chen, and Dinghao Wu. 2023. On the safety of open-sourced large language models: Does alignment really prevent them from being misused? arXiv preprint arXiv:2310.01581 . · arXiv
Cited by
Sorted by citations
Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation
Grabb, D., Lamparth, M., & Vasan, N. (2024) · doi:10.1101/2024.04.07.24305462 · Cited by 40 · see citing works
Self-citationFeedback Loops With Language Models Drive In-Context Reward Hacking
Pan, A., Jones, E., Jagadeesan, M., & Steinhardt, J. (2024) · arXiv · doi:10.48550/arxiv.2402.06627 · Cited by 3 · see citing works
“…to produce toxic text (Gehman et al., 2020; Perez et al., 2022a), re-inforce and amplify biases in the pretraining data (Blodgett et al., 2020b;a; Rivera et al., 2024; Xu et al., 2024), output misinformation (Lin et al., 2021) and hallucinations (Ji et al., 2023), expose private training data…”
BackgroundOpen Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 2 · see citing works
“Recent research by Rivera et al. (2024) raises critical concerns about the emergence of escalatory behaviours when AI tools or agents inform military decision-making.”
Self-citationGupta, P., Zhong, Q., Yakura, H., Eisenmann, T., & Rahwan, I. (2026) · Proceedings of the 25th International Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/czdc3237 · Cited by 2 · see citing works
“To understand cooperation in such settings, researchers have explored both complex, high-context scenarios, such as LLM agents in historical diplomacy [17, 30] or virtual societies [26, 38], and simplified, game-theoretic environments that serve as testbeds for cooperative mechanisms [27, 31, 37].”
- Authors
- Rivera, J.-P., Mukobi, G., Reuel, A., Lamparth, M., Smith, C., & Schneider, J.
- Affiliations
- Georgia Institute of Technology (US)
- Stanford University (US)
- Northeastern University (US)
- Hoover Wargaming and Crisis Simulation Initiative, USA
- Venue
- The 2024 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2024-06-03
- Pages
- 836-898
- OpenAlex
- W4399364678
- Cited by
- 77
- References
- 3
- Citations by year
- 2026: 46 · 2025: 18 · 2024: 13
- FWCI
- 196.72
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Center for International Security and Cooperation, Stanford University
- Topics
- Social Sciences › Political Science and International Relations › International Relations and Foreign Policy
- Physical Sciences › Engineering › Control and Systems Engineering › Military Strategy and Technology
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- large language models, AI agents, military decision-making, diplomatic decision-making, escalation risk, arms race, nuclear weapons deployment, deterrence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
