The DOI matches its DataCite record.
OpenAI Gym
Open accessAbstract
OpenAI Gym is a toolkit for reinforcement learning research. It includes a growing collection of benchmark problems that expose a common interface, and a website where people can share their results and compare the performance of algorithms. This whitepaper discusses the components of OpenAI Gym and the design decisions that went into the software.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“SmartPlay takes a suite of benchmarks (Brockman et al., 2016; Hafner, 2021; Fan et al., 2022) developed over different times to best represent a broad range of difficulties and skills.”
From SmartPlay: A Benchmark for LLMs as Intelligent Agents (2023) · cites this paper as Brockman et al., 2016
In the citing paper
“SmartPlay follows a unified OpenAI Gym interface (Brockman et al., 2016) for all games, with text-based observations, text-based manuals with content as described in Table 1, text describing historical actions and observations covering past steps of length 'history length', and flat categorical actions.”
From SmartPlay: A Benchmark for LLMs as Intelligent Agents (2023) · cites this paper as Brockman et al., 2016
In the citing paper
“For instance, the Open AI Gym [77], one of the most widely used frameworks for reinforcement learning, provides multiple environments for dexterous manipulation.”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [77]
In the citing paper
“They are reconfigured into standard OpenAI gym-style environments [5].”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [5]
In the citing paper
“Inspired by OpenAI Gym [5], FinRL provides strategy builders with a collection of universal training environments for various trading tasks.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [5]
In the citing paper
“However, much current RL research works on benchmarks such as Atari (Bellemare et al., 2013) and MuJoCo (E. Todorov et al., 2012; Brockman et al., 2016), which do not have the attributes described above: they evaluate the policy on exactly the same environment it was trained on, which often does not match with real-world scenarios (Fig. 1 left column).”
From A Survey of Zero-shot Generalisation in Deep Reinforcement Learning (2023) · cites this paper as Brockman et al., 2016
In the citing paper
“Inspired by OpenAI Gym [5], FinRL provides strategy builders with a collection of universal training environments for various trading tasks.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [5]
In the citing paper
“They are reconfigured into standard OpenAI gym-style environments [5].”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [5]
In the citing paper
“Our environment implementation is based on PettingZoo [24] and OpenAI Gym [5].”
From NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds (2024) · cites this paper as [5]
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Self-citationDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
AutoML: A survey of the state-of-the-art
He, X. J., Zhao, K., & Chu, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,912
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Metadata
- Authors
- Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.01540
- OpenAlex
- W4394666657
- Cited by
- 582
- References
- 0
- Citations by year
- 2026: 16 · 2025: 25 · 2024: 93 · 2023: 149 · 2022: 119 · 2021: 82 · 2020: 37 · 2019: 31 · 2018: 24 · 2017: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- OpenAI Gym, reinforcement learning, benchmark environments, software design, algorithm comparison, research toolkit
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
