Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

A Q-Learning Novelty Search Strategy for Evaluating Robustness of Deep Reinforcement Learning in Open-World Environments

Islam, Shafkat, Chiu, Min-Hsueh, Bonjour, Trevor, de Oliveira, Ruy, Bhargava, Bharat, Kejriwal, Mayank · 2025 · IEEE Intelligent Systems

Abstract

Despite substantial progress in deep reinforcement learning (DRL), a systematic characterization of DRL agents’ robustness to unexpected events in the environment is relatively understudied. Such unexpected events (“novelties”), especially those that are more structural than parametric, may significantly deteriorate the performance of DRL agents, leading them to be unfit for open-world environments and applications. However, not all novelties affect an agent’s performance equally. Unfortunately, even with reasonable and constrained definitions of the problem, the space of all novelties can be (at least) exponential. Hence, an effective search strategy is required to find novelties that can adversely affect the agent. This article presents a formalism for this problem and proposes a deep Q-learning-based novelty search strategy that efficiently and systematically finds candidate (potentially complex) novelties with significant negative impact on a DRL agent. We conduct a detailed set of experiments in a stochastic multiagent game environment (Monopoly) with complex decision-making properties.

Overview

Integrity signals

Registry

Crossref · 10.1109/mis.2024.3469574

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 10 references

Research works printed but not linked: 5

  1. 10.1609/aaai.v35i17.17766 · DOI
  2. Siddiqui, 2023, 17 fatalities, 736 crashes: The shocking toll of teslas autopilot
  3. Kejriwal, 2022, Designing artificial intelligence for open worlds
  4. Mirzaei, 2022, Fake it until you make it: Towards accurate near-distribution novelty detection, 11th Int. Conf. Learn. Representations
  5. Bailis, 2014, Learning to play monopoly: A reinforcement learning approach

Cited by

See all 13

Metadata

Authors
Islam, S., Chiu, M.-H., Bonjour, T., de Oliveira, R., Bhargava, B., & Kejriwal, M.
Affiliations
  • Purdue University West Lafayette (US)
  • University of Southern California (US)
  • Purdue University, West Lafayette, IN, USA
Venue
IEEE Intelligent Systems
Published
2025-09
Volume
40
Issue
5
Pages
5-15
ISSN
  • 1541-1672 (ISSN-L)
  • 1941-1294
OpenAlex
W4402915306
Cited by
13
References
10
Citations by year
2026: 13
FWCI
1.93
Type
journal article
Language
en
Open access
closed
Locations
IEEE Intelligent Systems (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords
novelty search, deep reinforcement learning, robustness, open-world environments, Q-learning, structural novelty, monopoly

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Q-Learning Novelty Search Strategy for Evaluating Robustness of Deep Reinforcement Learning in Open-World Environments. https://app.cochat.ai/papers/10.1109/mis.2024.3469574
The paper
Islam, S., Chiu, M.-H., Bonjour, T., de Oliveira, R., Bhargava, B., & Kejriwal, M. (2025). A Q-Learning Novelty Search Strategy for Evaluating Robustness of Deep Reinforcement Learning in Open-World Environments. IEEE Intelligent Systems. https://doi.org/10.1109/mis.2024.3469574

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.