ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning
Open accessAbstract
Deep reinforcement learning (DRL) has revolutionized learning and actuation in applications such as game playing and robotic control. The cost of data collection, i.e., generating transitions from agent-environment interactions, remains a major challenge for wider DRL adoption in complex real-world problems. Following a cloud-native paradigm to train DRL agents on a GPU cloud platform is a promising solution. In this paper, we present a scalable and elastic library ElegantRL-podracer for cloud-native deep reinforcement learning, which efficiently supports millions of GPU cores to carry out massively parallel training at multiple levels. At a high-level, ElegantRL-podracer employs a tournament-based ensemble scheme to orchestrate the training process on hundreds or even thousands of GPUs, scheduling the interactions between a leaderboard and a training pool with hundreds of pods. At a low-level, each pod simulates agent-environment interactions in parallel by fully utilizing nearly 7,000 GPU CUDA cores in a single GPU. Our ElegantRL-podracer library features high scalability, elasticity and accessibility by following the development principles of containerization, microservices and MLOps. Using an NVIDIA DGX SuperPOD cloud, we conduct extensive experiments on various tasks in locomotion and stock trading and show that ElegantRL-podracer substantially outperforms RLlib. Our codes are available on GitHub.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FinRL fine-tunes three representative open-source DRL libraries: Stable Baseline 3, RLlib, and ElegantRL.
“As a backbone, we fine-tune three representative open-source DRL libraries, namely Stable Baseline 3 [37], RLlib [25] and ElegantRL [28].”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [28]
FinRL delivers FinRL-Podracer, a cloud-native solution for FinRL with high performance and high scalability training.
“FinRL delivers FinRL-Podracer [24, 29], a cloud-native solution for FinRL with high performance and high scalability training.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [29]
FinRL fine-tunes three representative open-source DRL libraries: Stable Baseline 3, RLlib and ElegantRL.
“As a backbone, we fine-tune three representative open-source DRL libraries, namely Stable Baseline 3 [37], RLlib [25] and ElegantRL [28].”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [28]
Cites
Still being checked.
Cited by
Sorted by citations
Ensemble reinforcement learning: A survey
Song, Y., Suganthan, P. N., Pedrycz, W., Ou, J., He, Y., Chen, Y., & Wu, Y. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110975 · Cited by 64 · see citing works
FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021) · doi:10.2139/ssrn.3955949 · Cited by 46 · see citing works
Self-citationDong, X., Hong, H., Deng, X., Zhong, W., & Hu, G. (2023) · Physics of Fluids · doi:10.1063/5.0170316 · Cited by 30 · see citing works
David, H. A., Bahrpeyma, F., & Reichelt, D. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.09.019 · Cited by 33 · see citing works
Noise Optimization in Artificial Neural Networks
Xiao, L., Zhang, Z., Huang, K., Jiang, J., & Peng, Y. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3384409 · Cited by 13 · see citing works
Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical ApproachPreprint
Guan, M., & Liu, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4061958 · Cited by 8 · see citing works
Self-citationPOE: A General Portfolio Optimization Environment for FinRL
Costa, C. D. S. B., & Costa, A. H. R. (2023) · Anais do II Brazilian Workshop on Artificial Intelligence in Finance (BWAIF 2023) · doi:10.5753/bwaif.2023.231144 · Cited by 4 · see citing works
Risk Sensitive Distributional Soft Actor Critic for Portfolio Management
Liu, J., Wang, Y.-L., Huang, S.-L., & Sun, D. (2024) · Journées Bases de Données Avancées · doi:10.1109/icbda61153.2024.10607182 · Cited by 4 · see citing works
Multi-Constraint Deep Reinforcement Learning for Smooth Action Control
Zou, G., He, Y., Yu, F. R., Chen, L., Pan, W., & Ming, Z. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/528 · Cited by 2 · see citing works
Wen, Z., Dong, M., & Chen, X. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801893 · Cited by 2 · see citing works
Hou, Z., Liu, Y., & Fan, Q. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3560879 · Cited by 1 · see citing works
SLAMuZero: Plan and Learn to Map for Joint SLAM and Navigation
Fang, B., Chen, X., Pan, Z., & Di, X. S. (2024) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v34i1.31476 · Cited by 1 · see citing works
Demo:SCDRL: Scalable and Customized Distributed Reinforcement Learning System
Sun, C., Qiang, W., & Li, J. (2024) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs60910.2024.00132 · Cited by 1 · see citing works
High-throughput Sampling, Communicating and Training for Reinforcement Learning Systems
Zhao, L., Dai, X., Zhao, Z., Xin, Y., Hu, Y., Qian, J., Yao, J. P., & Li, K. (2023) · International Workshop on Quality of Service · doi:10.1109/iwqos57198.2023.10188703
RLPortfolio: Reinforcement Learning for Financial Portfolio Optimization
Costa, C. D. S. B., & Costa, A. H. R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79035-5_29
- Authors
- Liu, X.-Y., Li, Z., Yang, Z., Zheng, J., Wang, Z., Walid, A., Guo, J., & Jordan, M. I.
- Affiliations
- Columbia University (US)
- University of California, Berkeley (US)
- Shenzhen Institutes of Advanced Technology (CN)
- Northwestern University (US)
- Amazon (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2112.05923
- OpenAlex
- W4200630651
- Cited by
- 15
- References
- 0
- Citations by year
- 2025: 2 · 2024: 6 · 2023: 4 · 2022: 2 · 2021: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- deep reinforcement learning, cloud-native computing, containerization, microservices, MLOps, locomotion control, stock trading
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
