CoChat

HRLB⌃2: A Reinforcement Learning Based Framework for Believable Bots

Open access
Arzate Cruz, Christian, Ramirez Uresti, Jorge Adolfo · 2018 · Applied Sciences

The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.

Abstract

The creation of believable behaviors for Non-Player Characters (NPCs) is key to improve the players’ experience while playing a game. To achieve this objective, we need to design NPCs that appear to be controlled by a human player. In this paper, we propose a hierarchical reinforcement learning framework for believable bots (HRLB⌃2). This novel approach has been designed so it can overcome two main challenges currently faced in the creation of human-like NPCs. The first difficulty is exploring domains with high-dimensional state–action spaces, while satisfying constraints imposed by traits that characterize human-like behavior. The second problem is generating behavior diversity, by also adapting to the opponent’s playing style. We evaluated the effectiveness of our framework in the domain of the 2D fighting game named Street Fighter IV. The results of our tests demonstrate that our bot behaves in a human-like manner.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/app8122453

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 An algebraic decision diagram.
Figure 1 An algebraic decision diagram.CC-BY · From the PDF
Figure 2 An example of a HRLB^2 task graph.
Figure 2 An example of a HRLB^2 task graph.CC-BY · From the PDF
Figure 3 Asnapshot of Street Fighter IV .
Figure 3 Asnapshot of Street Fighter IV .CC-BY · From the PDF
Figure 4 The task graph for our HRLB^2-based agent that plays Street Fighter IV .
Figure 4 The task graph for our HRLB^2-based agent that plays Street Fighter IV .CC-BY · From the PDF
Figure 5 Box plots of the learning rates of three different bots.
Figure 5 Box plots of the learning rates of three different bots.CC-BY · From the PDF
Figure 6 Computed human-likeness ratios for the third-person Turing test.
Figure 6 Computed human-likeness ratios for the third-person Turing test.CC-BY · From the PDF
Image 7
CC-BY · From the PDF
Figure 8 Computed human-likeness ratios for the third-person Turing test for HRLB^2.
Figure 8 Computed human-likeness ratios for the third-person Turing test for HRLB^2.CC-BY · From the PDF
Figure 9 Evaluation of HRLB^2 violin plot.
Figure 9 Evaluation of HRLB^2 violin plot.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 42 references

Research works printed but not linked: 6

  1. Laird, J.E., and Duchi, J.C. (2000). Creating Human-Like Synthetic Characters With Multiple Skill-Levels: A Case Study Using the Soar Quakebot, AAAI.
  2. Ng, A.Y., Harada, D., and Russell, S. (1999). Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping, ICML.
  3. Hingston, P. (2012). Assessing Believability. Believable Bots, Springer. · DOI
  4. Usunier, N., Synnaeve, G., Lin, Z., and Chintala, S. (arXiv, 2016). Episodic Exploration for Deep Deterministic Policies for StarCraft Micromanagement, arXiv.
  5. Puterman, M.L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons.
  6. Coe, R. (2002, January 12–14). It’s the effect size, stupid: What effect size is and why it is important. Presented at the Annual Conference of the British Educational Research Association, University of Exeter, Exeter, UK.

Cited by

See all 21

Metadata

Authors
Arzate Cruz, C., & Ramirez Uresti, J. A.
Affiliations
School of Engineering and Science, Tecnologico de Monterrey, CP 64849 Monterrey, Mexico
Venue
Applied Sciences
Publisher
MDPI AG
Published
2018-12-01
Volume
8
Issue
12
Pages
2453
ISSN
2076-3417 (ISSN-L)
OpenAlex
W2902782170
Cited by
21
References
39
Citations by year
2026: 1 · 2025: 2 · 2023: 7 · 2022: 4 · 2021: 2 · 2020: 3 · 2019: 2
FWCI
0.99
Type
journal article
Language
en
Open access
Locations
  • Applied Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Social Sciences › Sociology and Political Science › Digital Games and Media
Keywords
non-player characters, hierarchical reinforcement learning, human-like behavior, behavioral diversity, game AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: HRLB⌃2: A Reinforcement Learning Based Framework for Believable Bots. https://app.cochat.ai/papers/10.3390/app8122453
The paper
Arzate Cruz, C., & Ramirez Uresti, J. A. (2018). HRLB⌃2: A Reinforcement Learning Based Framework for Believable Bots. Applied Sciences. https://doi.org/10.3390/app8122453

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.