The DOI matches its DataCite record; 3 of the 3 references it cites were found in a registry or an index.
CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms
Open accessAbstract
CleanRL is an open-source library that provides high-quality single-file implementations of Deep Reinforcement Learning algorithms. It provides a simpler yet scalable developing experience by having a straightforward codebase and integrating production tools to help interact and scale experiments. In CleanRL, we put all details of an algorithm into a single file, making these performance-relevant details easier to recognize. Additionally, an experiment tracking feature is available to help log metrics, hyperparameters, videos of an agent's gameplay, dependencies, and more to the cloud. Despite succinct implementations, we have also designed tools to help scale, at one point orchestrating experiments on more than 2000 machines simultaneously via Docker and cloud providers. Finally, we have ensured the quality of the implementations by benchmarking against a variety of environments. The source code of CleanRL can be found at https://github.com/vwxyzjn/cleanrl
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Our baselines utilize CleanRL's Proximal Policy Optimization (PPO) implementation.
“Our baselines utilize CleanRL's [Huang et al., 2021] Proximal Policy Optimization (PPO) [Schulman et al., 2017] implementation”
From Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learning (2023) · cites this paper as Huang et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
A Closer Look at Invalid Action Masking in Policy Gradient Algorithms
Huang, S., & Ontañón, S. (2022) · The International FLAIRS Conference Proceedings · doi:10.32473/flairs.v35i.130584 · Cited by 435
Self-citation
Flagged first, then by citations
Research works printed but not linked: 14
- M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47: 253-279, jun 2013.
- Marc G Bellemare, Will Dabney, and Rémi Munos. A distributional perspective on rein- forcement learning. In International Conference on Machine Learning, pages 449-458.
- PMLR, 2017.
- Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural gen- eration to benchmark reinforcement learning. In International conference on machine learning, pages 2048-2056. PMLR, 2020.
- Carlo D'Eramo, Davide Tateo, Andrea Bonarini, Marcello Restelli, and Jan Peters. Mush- roomrl: Simplifying reinforcement learning research. Journal of Machine Learning Re- search, 2020.
- Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. Implementation matters in deep rl: A case study on ppo and trpo. In International Conference on Learning Representations, 2019.
- Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approximation error in actor-critic methods. In International Conference on Machine Learning, pages 1587-1596.
- PMLR, 2018.
- Yasuhiro Fujita, Prabhat Nagarajan, Toshiki Kataoka, and Takahiro Ishikawa. Chainerrl: A deep reinforcement learning library. Journal of Machine Learning Research, 22(77): 1-14, 2021.
- Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. In Inter- national conference on machine learning, pages 1861-1870. PMLR, 2018.
- Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado Van Hasselt, and David Silver. Distributed prioritized experience replay. arXiv preprint arXiv:1803.00933, 2018. · arXiv
- Shengyi Huang, Santiago Ontañón, Chris Bamford, and Lukasz Grela. Gym-µrts: Toward affordable full game real-time strategy games research with deep reinforcement learning. arXiv preprint arXiv:2105.13807, 2021. · arXiv
- Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica. Rllib: Abstractions for distributed reinforcement learning. In International Conference on Machine Learning, pages 3053- 3062. PMLR, 2018.
- Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous Control with Deep Reinforcement Learning. arXiv preprint arXiv:1509.02971, 2015. · arXiv
Cited by
Sorted by citations
Design and evaluation of a global workspace agent embodied in a realistic multimodal environment
Dossa, R. F. J., Arulkumaran, K., Juliani, A., Sasai, S., & Kanai, R. (2024) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2024.1352685 · Cited by 5
Self-citationXLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX
Nikulin, A., Kurenkov, V., Zisman, I., Agarkov, A., Sinii, V., & Kolesnikov, S. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1390 · Cited by 3
Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learning
Suarez, J., Bloomin, D., Choe, K. W., Li, H. X., Sullivan, R., Kanna, N., Scott, D., Shuman, R., Bradley, H., Castricato, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2178 · Cited by 1
Metadata
- Authors
- Huang, S., Dossa, R. F. J., Ye, C., & Braga, J.
- Affiliations
- Drexel University (US)
- New York University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2111.08819
- OpenAlex
- W3213774989
- Cited by
- 116
- References
- 3
- Citations by year
- 2026: 6 · 2025: 39 · 2024: 40 · 2023: 27 · 2022: 3 · 2021: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Keywords
- deep reinforcement learning, open-source library, experiment tracking, distributed training, Docker, cloud computing, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
