Learning Policies for First Person Shooter Games Using Inverse Reinforcement Learning
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
The creation of effective autonomous agents (bots) for combat scenarios has long been a goal of the gaming industry. However, a secondary consideration is whether the autonomous bots behave like human players; this is especially important for simulation/training applications which aim to instruct participants in real-world tasks. Bots often compensate for a lack of combat acumen with advantages such as accurate targeting, predefined navigational networks, and perfect world knowledge, which makes them challenging but often predictable opponents. In this paper, we examine the problem of teaching a bot to play like a human in first-person shooter game combat scenarios. Our bot learns attack, exploration and targeting policies from data collected from expert human player demonstrations in Unreal Tournament. We hypothesize that one key difference between human players and autonomous bots lies in the relative valuation of game states. To capture the internal model used by expert human players to evaluate the benefits of different actions, we use inverse reinforcement learning to learn rewards for different game states. We report the results of a human subjects' study evaluating the performance of bot policies learned from human demonstration against a set of standard bot policies. Our study reveals that human players found our bots to be significantly more human-like than the standard bots during play. Our technique represents a promising stepping-stone toward addressing challenges such as the Bot Turing Test (the CIG Bot 2K Competition).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621
Cited by
Playing FPS Games with Deep Reinforcement Learning
Lample, G., & Chaplot, D. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10827 · Cited by 452
Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics
Holmgard, C., Green, M. C., Liapis, A., & Togelius, J. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2018.2808198 · Cited by 125
“In contrast, procedural personas are driven by utility functions that can be either specified in a top–down manner by game or persona designers or formulated from play data through methods like inverse reinforcement learning [21] or evolution [14].”
BackgroundAutomated Playtesting of Matching Tile Games
Mugrai, L., Silva, F., Holmgard, C., & Togelius, J. (2019) · 2019 IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848057 · Cited by 28
Playtesting: What is Beyond Personas
Ariyurek, S., Surer, E., & Betin-Can, A. (2023) · IEEE Transactions on Games · doi:10.1109/tg.2022.3165882 · Cited by 7
“This alteration might help the RL agent to generate a playtest that is more human-like [8][31].”
Background
Metadata
- Authors
- Tastan, B., & Sukthankar, G.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
- Published
- 2011-10-09
- Volume
- 7
- Issue
- 1
- Pages
- 85-90
- ISSN
- 2326-909X (ISSN-L)
- 2334-0924
- OpenAlex
- W2399570347
- Cited by
- 44
- References
- 1
- FWCI
- 3.27
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.