CoChat
Sign Up
PreprintCited by 1Multi-Agent Systems and NegotiationDeception detection and forensic psychologyEthics and Social Impacts of AI

The Traitors: Deception and Trust in Multi-Agent Language Model Simulations

Open access
Curvo, Pedro M. P.·2025·arXiv
DOI 10.48550/arxiv.2505.12923arXiv 2505.12923
Open this paper in CoChat — free accountOpen PDF

Abstract

As AI systems increasingly assume roles where trust and alignment with human values are essential, understanding when and why they engage in deception has become a critical research priority. We introduce The Traitors, a multi-agent simulation framework inspired by social deduction games, designed to probe deception, trust formation, and strategic communication among large language model (LLM) agents under asymmetric information. A minority of agents the traitors seek to mislead the majority, while the faithful must infer hidden identities through dialogue and reasoning. Our contributions are: (1) we ground the environment in formal frameworks from game theory, behavioral economics, and social cognition; (2) we develop a suite of evaluation metrics capturing deception success, trust dynamics, and collective inference quality; (3) we implement a fully autonomous simulation platform where LLMs reason over persistent memory and evolving social dynamics, with support for heterogeneous agent populations, specialized traits, and adaptive behaviors. Our initial experiments across DeepSeek-V3, GPT-4o-mini, and GPT-4o (10 runs per model) reveal a notable asymmetry: advanced models like GPT-4o demonstrate superior deceptive capabilities yet exhibit disproportionate vulnerability to others' falsehoods. This suggests deception skills may scale faster than detection abilities. Overall, The Traitors provides a focused, configurable testbed for investigating LLM behavior in socially nuanced interactions. We position this work as a contribution toward more rigorous research on deception mechanisms, alignment challenges, and the broader social reliability of AI systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.12923

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Adversarial debate is vulnerable to spurious categorizations.

    “However, adversarial debate is vulnerable to two varieties of sophistry.39 The first is when the model makes spurious categorizations.”

    From When a Human‐in‐the‐Loop Amplifies the Risk of Misalignment: Considerations for Research Ethics Oversight (2026) · cites this paper as [39]

    Section C.1.2 describes adversarial dialogue in which agents try to identify deceptive peers, and “Information Cascade Phenomena” describes incorrect accusations spreading through a group. These passages support a general vulnerability to deception and mistaken judgments, but they do not identify spurious categorizations as a form of sophistry in adversarial debate or establish the claimed two-variety taxonomy. No page numbers were supplied for these passages. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 62

  1. G. A. Akerlof. The market for "lemons": Quality uncertainty and the market mechanism. Quarterly Journal of Economics, 84(3):488-500, 1970.
  2. D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016. · arXiv
  3. J. Andreas. Language models as agent models. arXiv preprint arXiv:2212.01681, 2022. · arXiv
  4. S. Athey and J. Levin. Information and competition in us forest service timber auctions. Journal of Political Economy, 109(2):375-417, 2001.
  5. C. L. Baker, J. Jara-Ettinger, R. Saxe, and J. B. Tenenbaum. Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nature Human Behaviour, 1(4):1-10, 2017.
  6. A. Bakhtin, N. Brown, E. Dinan, G. Farina, C. Flaherty, D. Fried, A. Goff, J. Gray, H. Hu, A. P. Jacob, et al. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 378(6624):1067-1074, 2022.
  7. S. Baronett. Logic. Pearson Prentice Hall, 2008.
  8. S. Bikhchandani, D. Hirshleifer, and I. Welch. A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of political Economy, 100(5):992-1026, 1992.
  9. C. F. Bond Jr and B. M. DePaulo. Accuracy of deception judgments. Personality and Social Psychology Review, 10(3):214-234, 2006.
  10. S. P. Borgatti, A. Mehra, D. J. Brass, and G. Labianca. Network analysis in the social sciences. Science, 323(5916):892-895, 2009.
  11. N. Bostrom. Superintelligence: Paths, dangers, strategies. Oxford University Press, 2014.
  12. D. B. Buller and J. K. Burgoon. Interpersonal deception theory. Communication Theory, 6(3): 203-242, 1996.
  13. C. F. Camerer. Behavioral game theory: Experiments in strategic interaction. Princeton University Press, 2011.
  14. K. Cao, A. Lazaridou, M. Lanctot, J. Z. Leibo, K. Tuyls, and S. Clark. Emergent communication through negotiation. 2018. URL https://arxiv.org/abs/1804.03980. · arXiv
  15. M. Carroll, R. Shah, M. K. Ho, T. Griffiths, S. Seshia, P. Abbeel, and A. Dragan. On the utility of learning about humans for human-ai coordination. Advances in Neural Information Processing Systems, 32, 2019.
  16. G. Charness, D. Masclet, and M. C. Villeval. Social and moral norms in allocation choices in the laboratory. Journal of Economic Behavior & Organization, 94:192-206, 2013.
  17. W.-L. Chiang, L. Zheng, Y. Sheng, A. N. Angelopoulos, T. Li, D. Li, H. Zhang, B. Zhu, M. Jordan, J. E. Gonzalez, and I. Stoica. Chatbot arena: An open platform for evaluating llms by human preference, 2024. URL https://arxiv.org/abs/2403.04132. · arXiv
  18. V. P. Crawford and J. Sobel. Strategic information transmission. Econometrica: Journal of the Econometric Society, pages 1431-1451, 1982.
  19. A. Dafoe, Y. Bachrach, G. Hadfield, E. Horvitz, K. Larson, and T. Graepel. Cooperative ai: machines must learn to find common ground. Nature, 593(7857):33-36, 2021.
  20. R. Dawkins. The selfish gene. Oxford University Press, 1976.
  21. B. M. DePaulo, J. I. Stone, and G. D. Lassiter. Telling ingratiating lies: Effects of target sex and target attractiveness on verbal and nonverbal deceptive success. Journal of Personality and Social Psychology, 48(5):1191-1203, 1985. doi: 10.1037/0022-3514.48.5.1191. · DOI
  22. O. Evans, O. Cotton-Barratt, L. Finnveden, A. Bales, A. Balwit, P. Wills, L. Righetti, and W. Saunders. Truthful ai: Developing and governing ai that does not lie. arXiv preprint arXiv:2110.06674, 2021. · arXiv
  23. T. Everitt, V. Krakovna, L. Orseau, and S. Legg. Reinforcement learning with a corrupted reward channel. arXiv preprint arXiv:1705.08417, 2017. · arXiv
  24. J. Farrell and M. Rabin. Cheap talk. Journal of Economic Perspectives, 10(3):103-118, 1996.
  25. J. H. Flavell. Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10):906, 1979.
  26. U. Gneezy. Deception: The role of consequences. American Economic Review, 95(1):384-394, 2005.
  27. D. Hendrycks, N. Carlini, J. Schulman, and J. Steinhardt. X-risk analysis for ai research. arXiv preprint arXiv:2206.05862, 2022. · arXiv
  28. B. Holmstrom. Managerial incentive problems: A dynamic perspective. The Review of Economic Studies, 66(1):169-182, 1999.
  29. E. Hubinger, C. van Merwijk, V. Mikulik, J. Skalse, and S. Garrabrant. Risks from learned optimization in advanced machine learning systems. arXiv preprint arXiv:1906.01820, 2019. · arXiv
  30. M. Idziejczak, V. Korzavatykh, M. Stawicki, A. Chmutov, M. Korcz, I. Bł ądek, and D. Brzezin- ski. Among them: A game-based framework for assessing persuasion capabilities of llms. 2025. URL https://arxiv.org/abs/2502.20426. · arXiv
  31. G. Irving, P. Christiano, and D. Amodei. Ai safety via debate. arXiv preprint arXiv:1805.00899, 2018. · arXiv
  32. S. A. M. Jack, K. Morrison, J. E. Paik, A. M. Wisner, and X. Zhu. Information manipulation theory 2: A propositional theory of deceptive discourse production. Journal of Language and Social Psychology, 29(2):139-156, 2010.
  33. A. Jøsang, R. Ismail, and C. Boyd. A survey of trust and reputation systems for online service provision. Decision Support Systems, 43(2):618-644, 2007.
  34. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99-134, 1998.
  35. P. Kollock. Social dilemmas: The anatomy of cooperation. Annual review of sociology, 24(1): 183-214, 1998.
  36. R. M. Kramer. Paranoid cognition in social systems: thinking and acting in the shadow of doubt. Personality and Social Psychology Review, 2(4):251-275, 1998.
  37. D. Krueger, T. Maharaj, and J. Leike. Hidden incentives for auto-induced distributional shift. 2020. URL https://arxiv.org/abs/2009.09153. · arXiv
  38. J. Leike, D. Krueger, T. Everitt, M. Martic, V. Maini, and S. Legg. Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv:1811.07871, 2018. · arXiv
  39. T. R. Levine. Truth-default theory (tdt): A theory of human deception and deception detection. Journal of Language and Social Psychology, 33(4):378-392, 2014.
  40. R. J. Lewicki, E. C. Tomlinson, and N. Gillespie. Models of interpersonal trust development: Theoretical approaches, empirical evidence, and future directions. Journal of management, 32 (6):991-1022, 2006.
  41. J. Light, M. Cai, S. Shen, and Z. Hu. Avalonbench: Evaluating llms playing the game of avalon, 2023. URL https://arxiv.org/abs/2310.05036. · arXiv
  42. S. Lin, J. Hilton, and O. Evans. Truthfulqa: Measuring how models mimic human falsehoods. 2022. URL https://arxiv.org/abs/2109.07958. · arXiv
  43. J. Menick, M. Trebacz, V. Mikulik, J. Aslanides, F. Song, M. Chadwick, M. Glaese, S. Young, L. Campbell-Gillingham, G. Irving, and N. McAleese. Teaching language models to support answers with verified quotes. 2022. URL https://arxiv.org/abs/2203.11147. · arXiv
  44. H. Mercier and D. Sperber. Why do humans reason? arguments for an argumentative theory. Behavioral and brain sciences, 34(2):57-74, 2011.
  45. Meta Fundamental AI Research Diplomacy Team (FAIR), A. Bakhtin, N. Brown, E. Dinan, G. Farina, C. Flaherty, D. Fried, A. Goff, J. Gray, H. Hu, A. P. Jacob, M. Komeili, K. Konath, M. Kwon, A. Lerer, M. Lewis, A. H. Miller, S. Mitts, A. Renduchintala, S. Roller, D. Rowe, W. Shi, J. Spisak, A. Wei, D. Wu, H. Zhang, and M. Zijlstra. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 378(6624): 1067-1074, 2022.
  46. D. Mguni, J. Jennings, and E. M. de Cote. Decentralised learning in systems with many, many strategic agents. 2018. URL https://arxiv.org/abs/1803.05028. · arXiv
  47. R. B. Myerson. Game theory: analysis of conflict. Harvard University Press, 1991.
  48. S. M. Omohundro. The basic ai drives. Proceedings of the 2008 Conference on Artificial General Intelligence, pages 483-492, 2008.
  49. J. Pearl. Causality. Cambridge University Press, 2009.
  50. G. Piatti, Z. Jin, M. Kleiman-Weiner, B. Schölkopf, M. Sachan, and R. Mihalcea. Cooperate or collapse: Emergence of sustainable cooperation in a society of llm agents. 2024. URL https://arxiv.org/abs/2404.16698. · arXiv
  51. G. P. Ramani, S. Karande, S. V, and Y. Bhatia. Persuasion games using large language models, 2024. URL https://arxiv.org/abs/2408.15879. · arXiv
  52. L. Ross, D. Greene, and P. House. The false consensus effect: An egocentric bias in social perception and attribution processes. Journal of Experimental Social Psychology, 13(3):279- 301, 1977.
  53. A. Rubinstein. The electronic mail game: Strategic behavior under almost common knowledge. The American Economic Review, pages 385-391, 1989.
  54. R. Selten. Reexamination of the perfectness concept for equilibrium points in extensive games. International journal of game theory, 4(1):25-55, 1975.
  55. J. M. Smith. Evolution and the theory of games. American Scientist, 64(1):41-45, 1982.
  56. M. Spence. Job market signaling. Quarterly Journal of Economics, 87(3):355-374, 1973.
  57. A. Vrij, K. Edward, K. Roberts, and R. Bull. Detecting deceit via analysis of verbal and nonverbal behavior. Journal of Nonverbal Behavior, 24:239-263, 01 2000. doi: 10.1023/A: 1006610329284. · DOI
  58. S. Wang, C. Liu, Z. Zheng, S. Qi, S. Chen, Q. Yang, A. Zhao, C. Wang, S. Song, and G. Huang. Avalon's game of thoughts: Battle against deception through recursive contemplation. 2023. URL https://arxiv.org/abs/2310.01320. · arXiv
  59. A. W. Woolley, C. F. Chabris, A. Pentland, N. Hashmi, and T. W. Malone. Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004):686-688, 2010.
  60. H. Xu, R. Zhao, L. Zhu, J. Du, and Y. He. Opentom: A comprehensive benchmark for evaluating theory-of-mind reasoning capabilities of large language models. 2024. URL https: //arxiv.org/abs/2402.06044.
  61. Y. Xu, S. Wang, P. Li, F. Luo, X. Wang, W. Liu, and Y. Liu. Exploring large language models for communication games: An empirical study on werewolf. 2024. URL https: //arxiv.org/abs/2309.04658.
  62. A. Zahavi. Mate selection-a selection for a handicap. Journal of theoretical Biology, 53(1): 205-214, 1975.

Cited by

Sorted by citations

Authors
Curvo, P. M. P.
Affiliations
Amsterdam University of Applied Sciences (NL)
Corresponding author
Curvo, Pedro M. P.
ORCID
Curvo, Pedro M. P.
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2505.12923
arXiv
2505.12923
OpenAlex
W4417302326
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Social Sciences › Psychology › Social Psychology › Deception detection and forensic psychology
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
AI deception, LLM agents, trust formation, multi-agent simulation, social deduction games, information asymmetry, strategic communication, deception detection, AI alignment, GPT-4o, DeepSeek-V3, GPT-4o-mini

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review