Generating Personas for Games with Multimodal Adversarial Imitation Learning
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Reinforcement learning has been widely successful in producing agents capable of playing games at a human level. However, this requires complex reward engineering, and the agent’s resulting policy is often unpredictable. Going beyond reinforcement learning is necessary to model a wide range of human playstyles, which can be difficult to represent with a reward function. This paper presents a novel imitation learning approach to generate multiple persona policies for playtesting. Multimodal Generative Adversarial Imitation Learning (Multi-GAIL) uses an auxiliary input parameter to learn distinct personas using a single-agent model. MultiGAIL is based on generative adversarial imitation learning and uses multiple dis-criminators as reward models, inferring the environment reward by comparing the agent and distinct expert policies. The reward from each discriminator is weighted according to the auxiliary input. Our experimental analysis demonstrates the effectiveness of our technique in two environments with continuous and discrete action spaces.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies, such as [11], [20], [37], [43], address this issue by utilizing human gameplay data to define feasible behavior targets.”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [20]
Citations
Cites
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 1×
“Policy learning is done through Proximal Policy Optimization (PPO) [29] with the reward signal given by Equation (2).”
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.01815 · Cited by 1,102 · cited 1×
“Monte-Carlo Tree Search (MCTS) has been shown to be able to master games such as Chess, Shogi and Go [12], but has also been applied to mimic human playstyles in Match-3 games to analyze the performance range of human players [13].”
BackgroundDota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065 · cited 1×
“Self-learning Reinforcement Learning (RL) agents have been successful in creating agents able to play complex games at a human, or even beyond human, level of performance [1, 2, 3].”
BackgroundChallenges of real-world reinforcement learning: definitions, benchmarks and analysis
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2021) · Machine Learning · doi:10.1007/s10994-021-05961-4 · Cited by 645
AMP: adversarial motion priors for stylized physics-based character control
Peng, X. B., Ma, Z., Abbeel, P., Levine, S., & Kanazawa, A. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459670 · Cited by 396
A framework for behavioural cloning
Bain, M., & Sammut, C. (2000) · Machine Intelligence 15 · doi:10.1093/oso/9780198538677.003.0006 · Cited by 360
Research works printed but not linked: 4
- Cho, Online game testing using scenario-based control of massive virtual users
- Aytemiz, Acting with style: Towards designer-centred reinforcement learning for the video games industry
- Ross, A reduction of imitation learning and structured prediction to no-regret online learning
- Ho, 2016, Generative adversarial imitation learning, Advances in neural information processing systems, 29
Cited by
UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors
Cilan, A., & Ozgovde, A. (2026) · IEEE Transactions on Games · doi:10.1109/tg.2026.3703355
“Collectively, these approaches demonstrate high fidelity to observed human trajectories but share a key limitation: they depend heavily on large-scale demonstration datasets and struggle to generalize to unseen or intentionally designed behaviors [11], [19], [20], [21].”
Influential
Metadata
- Authors
- Ahlberg, W., Sestini, A., Tollmar, K., & Gisslén, L.
- Affiliations
- Electronic Arts (United States) (US)
- Venue
- 2023 IEEE Conference on Games (CoG)
- Publisher
- IEEE
- Published
- 2023-08-21
- Pages
- 1-8
- OpenAlex
- W4389315142
- Cited by
- 5
- References
- 28
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2
- FWCI
- 0.89
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Conference on Games (CoG) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Keywords
- generative adversarial imitation learning, reward modeling, auxiliary input, game-playing agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generating Personas for Games with Multimodal Adversarial Imitation Learning. https://app.cochat.ai/papers/10.1109/cog57401.2023.10333167
- The paper
- Ahlberg, W., Sestini, A., Tollmar, K., & Gisslén, L. (2023). Generating Personas for Games with Multimodal Adversarial Imitation Learning. 2023 IEEE Conference on Games (CoG). https://doi.org/10.1109/cog57401.2023.10333167
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.