RLlib: Abstractions for Distributed Reinforcement Learning
Open accessAbstract
Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL components in a composable way by adapting algorithms for top-down hierarchical control, thereby encapsulating parallelism and resource requirements within short-running compute tasks. We demonstrate the benefits of this principle through RLlib: a library that provides scalable software primitives for RL. These primitives enable a broad range of algorithms to be implemented with high performance, scalability, and substantial code reuse. RLlib is available at https://rllib.io/.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FinRL fine-tunes three representative open-source DRL libraries: Stable Baseline 3, RLlib and ElegantRL.
“As a backbone, we fine-tune three representative open-source DRL libraries, namely Stable Baseline 3 [37], RLlib [25] and ElegantRL [28].”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [25]
Cited by
Sorted by citations
From cloud computing to sky computing
Stoica, I., & Shenker, S. (2021) · Proceedings of the Workshop on Hot Topics in Operating Systems · doi:10.1145/3458336.3465301 · Cited by 89
Self-citationFinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3737257 · Cited by 90 · see citing works
Huang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88 · see citing works
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Chen, Q., Zhang, R., Yang, L., Xiao, B., & Wang, C. (2025) · doi:10.2139/ssrn.3737859 · Cited by 51 · see citing works
FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021) · doi:10.2139/ssrn.3955949 · Cited by 46 · see citing works
FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance
Li, Z., Liu, X.-Y., Zheng, J., Wang, Z., Walid, A., & Guo, J. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494413 · Cited by 28 · see citing works
Mobile-kube: mobility-aware and energy-efficient service orchestration on kubernetes edge servers
Ghafouri, S., Karami, A., Bakhtiarvan, D. B., Bigdeli, A. S., Gill, S. S., & Doyle, J. (2022) · 2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc56403.2022.00019 · Cited by 20 · see citing works
Formal contracts mitigate social dilemmas in multi-agent reinforcement learning
Haupt, A., Christoffersen, P., Damani, M., & Hadfield-Menell, D. (2024) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-024-09682-5 · Cited by 5 · see citing works
Dizon-Paradis, O., Wormald, S., Capecci, D., Bhandarkar, A., & Woodard, D. (2023) · doi:10.36227/techrxiv.23739099 · Cited by 1 · see citing works
Environment Complexity and Nash Equilibria in a Sequential Social Dilemma
Yasir, M., Howes, A., Mavroudis, V., & Hicks, C. (2024) · arXiv · doi:10.48550/arxiv.2408.02148 · Cited by 1 · see citing works
- Authors
- Liang, E., Liaw, R., Moritz, P., Nishihara, R., Fox, R., Goldberg, K., Gonzalez, J. E., Jordan, M. I., & Stoica, I.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1712.09381
- OpenAlex
- W2811374584
- Cited by
- 473
- References
- 0
- Citations by year
- 2026: 2 · 2025: 47 · 2024: 108 · 2023: 113 · 2022: 85 · 2021: 63 · 2020: 42 · 2019: 13
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Simulation Techniques and Applications
- Keywords
- distributed reinforcement learning, hierarchical control, parallelism, resource requirements, code reuse, scalability, deep reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
