The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index; 4 of the 25 sources its claims cite were checked for support.
FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Open accessAbstract
Deep reinforcement learning (DRL) has been envisioned to have a competitive edge in quantitative finance. However, there is a steep development curve for quantitative traders to obtain an agent that automatically positions to win in the market, namely to decide where to trade, at what price and what quantity, due to the error-prone programming and arduous debugging. In this paper, we present the first open-source framework FinRL as a full pipeline to help quantitative traders overcome the steep learning curve. FinRL is featured with simplicity, applicability and extensibility under the key principles, full-stack framework, customization, reproducibility and hands-on tutoring. Embodied as a three-layer architecture with modular structures, FinRL implements fine-tuned state-of-the-art DRL algorithms and common reward functions, while alleviating the debugging work- loads. Thus, we help users pipeline the strategy design at a high turnover rate. At multiple levels of time granularity, FinRL simu- lates various markets as training environments using historical data and live trading APIs. Being highly extensible, FinRL reserves a set of user-import interfaces and incorporates trading constraints such as market friction, market liquidity and investor’s risk-aversion. Moreover, serving as practitioners’ stepping stones, typical trad- ing tasks are provided as step-by-step tutorials, e.g., stock trading, portfolio allocation, cryptocurrency trading, etc.
Overview
CoChat Summary
## TL;DR
FinRL is an open-source, three-layer deep reinforcement learning framework for automated trading in quantitative finance. The paper presents the framework, its training-testing-trading pipeline, and benchmark/tutorial use cases in stocks, portfolio allocation, and cryptocurrencies.
## Key findings
- FinRL is described as the first open-source framework for end-to-end DRL-based trading in quantitative finance, with a three-layer architecture: application, agent, and environment.
- The framework supports fine-tuned DRL libraries including Stable Baselines 3, RLlib, and ElegantRL, and integrates historical data plus live trading APIs into Gym-style environments.
- In the stock-trading use case on Dow 30 constituent stocks, the ensemble strategy achieved an annualized return of 52.61%, Sharpe ratio of 2.81, and max drawdown of -7.09%; the DJIA benchmark had 32.84% annualized return and 2.02 Sharpe ratio.
- In the portfolio-allocation use case on the same Dow 30 universe, A2C had the best reported performance with Sharpe ratio 2.36 and annual return 42.57%; TD3, PPO, and DDPG also outperformed the DJIA index and min-variance strategy.
- In the cryptocurrency use case on the top 10 market-cap cryptocurrencies, PPO from ElegantRL reached the highest cumulative return of 103%, ahead of equally weighted (99%) and BTC buy-and-hold (93%).
## Methods
- Framework/design paper introducing a modular DRL trading system with three layers: application layer for state-action-reward specification, agent layer for DRL algorithms, and environment layer for market simulators.
- Uses historical market data and live trading APIs to build training/testing/trading workflows; supported tasks include stock trading, portfolio allocation, and cryptocurrency trading.
- Evaluates the framework through backtesting on benchmark use cases, comparing DRL agents against baselines such as mean-variance, min-variance, passive buy-and-hold, and equally weighted portfolios, using metrics including final portfolio value, cumulative return, annualized return, annualized standard deviation, Sharpe ratio, and maximum drawdown.
## Limitations
- The paper states that the development pipeline is error-prone and debugging is arduous for quantitative traders; FinRL is presented as a way to alleviate this, not as evidence of solving all implementation difficulties.
- The text makes evident that the benchmark evidence is limited to reproduced examples and backtests on selected markets/tasks; this suggests limited external validity beyond the showcased use cases (text-evident).
- The paper does not report a controlled ablation or statistical significance analysis of the framework components; this is not stated in the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 8 Not valid · 12 Not checked · 56 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Inspired by OpenAI Gym [5], FinRL provides strategy builders with a collection of universal training environments for various trading tasks.”
What the sources say
- Supportedcites [5] Greg Brockman, 2016, OpenAI gym source
The Abstract (p. 1) describes OpenAI Gym as 'a toolkit for reinforcement learning research' containing 'a growing collection of benchmark problems that expose a common interface,' and the Introduction (p. 1) states it 'includes a diverse collection of tasks (called environments) with a common interface.' Section 4 Environments (p. 3) enumerates those task families. The citing sentence attributes to Gym only the design idea of a shared collection of environments with a common interface (the trading-task part is the FinRL authors' own contribution), which our copy supports directly. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Therefore, the backtesting performance shows that FinRL successfully reproduce [20] with completeness and simplicity.”
What the sources say
- Supportedcites [20] Zhengyao Jiang, 2017, Cryptocurrency portfolio management with deep reinforcement learning, Intelligent Systems Conference (IntelliSys source
The citation is offered as an identification: [20] is the study whose backtesting performance FinRL claims to reproduce. Our copy of Jiang & Liang is exactly that work — a deep-RL (deterministic-policy-gradient CNN) cryptocurrency portfolio-management study evaluated by simulated back-tests (A. Results: three test windows 2016/03/14-2016/05/03, 2016/04/14-2016/06/03, 2016/05/14-2016/07/03 against Uniform Buy and Hold, Best Single Asset, Uniform Constant Rebalanced Portfolio, Universal Portfolio, Online Newton Step and PAMR), with reported performance measures (TABLE III / Fig. 2: Our CNN Agent final portfolio value 16.305332, Sharpe 0.036886, max drawdown 0.296082). The remainder of the sentence — that FinRL reproduces it 'with completeness and simplicity' — is the citing authors' own conclusion about their library's experiment, not a statement attributed to this source, so nothing in the sentence is left unsupported by our copy. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“They are reconfigured into standard OpenAI gym-style environments [5].”
What the sources say
- Supportedcites [5] Greg Brockman, 2016, OpenAI gym source
The citation is offered only to identify what a 'standard OpenAI gym-style environment' is, and our copy of OpenAI Gym is exactly that reference: the Abstract (p. 1) describes a toolkit whose 'benchmark problems ... expose a common interface,' and the Introduction (p. 1) states it 'includes a diverse collection of tasks (called environments) with a common interface.' Section 2 Background (p. 1) specifies the standard environment API (env.reset() returning the first observation; env.step(action) returning observation, reward, done, info), which is the interface a custom task would be written against. The citing paper's own act of reconfiguring its historical-data and live-trading APIs into such environments is the authors' contribution, not something attributed to this source, so nothing here is unsupported. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The ensemble strategy [51] combines three DRL algorithms (PPO [42], A2C [32] and DDPG [26]) to improve the robustness.”
What the sources say
- Not checked — no full text of the sourcecites [51] Hongyang Yang, 2020, Deep reinforcement learning for automated stock trading: An ensemble strategy, ACM International Conference on AI in Finance (ICAIF) source
- Supportedcites [42] John Schulman, 2017, Proximal policy optimization algorithms source
The citation to this source is attached to PPO, not to the ensemble strategy or its robustness claim. The Abstract (p. 1) identifies the methods introduced in this paper as proximal policy optimization (PPO). (LLM, conf=0.94)
- Not checked — no full text of the sourcecites [32] Volodymyr Mnih, 2016, Asynchronous methods for deep reinforcement learning, International Conference on Machine Learning source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“BTC buy and hold strategy with a cumulative return of 93% .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 80 claims and every source verdict.
- It may state two separate claims as one. 25
- The quote only partly states this claim. 2
- Probably not a substantial claim. 50
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We evaluate the training time of three popular DRL libraries, FinRL [25, 26], RLlib [19] and Stable Baseline3 [9], on NASDAQ-100 constituent stocks with minutelevel data (described in Section 5.2) in Table 1, which shows that it is difficult for them to effectively train a profitable trading agent in a short cycle time.”
From FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance (2021) · cites this paper as [26]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“FinRL-Meta [30] adds financial data engineering for FinRL with unified data processor and hundreds of market environments.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [30]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Most existing libraries [13, 19, 25, 26] send the gradients of learners by following a traditional synchronization approach on supervised learning.”
From FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance (2021) · cites this paper as [26]
Experimental — accuracy not yet measuredIn the citing paper
“For DRL strategy design, existing works studied algorithmic trading on a daily time-frame [2, 25, 26, 42-45] or hourly time-frame [14], which is hard to fully capture the dynamics of a highly volatile market.”
From FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance (2021) · cites this paper as [26]
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
(1999) · Cited by 5,147
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650
Portfolio theory and capital markets
Breen, W. J., & Sharpe, W. F. (1972) · The Journal of Finance · doi:10.2307/2978700 · Cited by 1,221
Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522401 · Cited by 822
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582
Research works printed but not linked: 13
- Joshua Achiam, 2018, Spinning up in deep reinforcement learning
- Greg Brockman, 2016, OpenAI gym
- 2017, CSI, 300
- Prafulla Dhariwal, 2017, OpenAI baselines
- 2018, Shanghai Stock Exchange, SSE 180 Index Methodology
- G Thomas, 2018, Reinforcement learning in financial markets -a survey. FAU Discussion Papers in Economics
- 2020, Hang Seng index and sub-indexes
- Xiao-Yang Liu, 2021, ElegantRL: A scalable and elastic deep reinforcement learning library
- Xiao-Yang Liu, 2021, Data-driven deep reinforcement learning in quantitative finance
- Quantopian, 2019, Pyfolio: A toolkit for portfolio and risk analytics in Python
- Antonin Raffin, 2019, Stable Baselines3
- Jonathan Sadighian, 2019, Deep reinforcement learning in cryptocurrency market making. arXiv: Trading and Market Microstructure
- Nelson Vadori, 2020, Risksensitive reinforcement learning: a martingale approach to reward uncertainty. International Conference on AI in Finance (ICAIF)
Cited by
Sahu, S. K., Mokhade, A., & Bokde, N. D. (2023) · Applied Sciences · doi:10.3390/app13031956 · Cited by 208
Recent Advances in Reinforcement Learning in FinancePreprint
Hambly, B. M., Xu, R., & Yang, H. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3971071 · Cited by 104
Self-citationStockFormer: Learning Hybrid Trading Machines with Predictive Coding
Gao, S., Wang, Y., & Yang, X. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/530 · Cited by 66
A Review of Reinforcement Learning in Financial Applications
Bai, Y., Gao, Y., Wan, R., Zhang, S., & Song, R. (2024) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-112723-034423 · Cited by 83
FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance
Li, Z., Liu, X.-Y., Zheng, J., Wang, Z., Walid, A., & Guo, J. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494413 · Cited by 28
Self-citation
Metadata
- Authors
- Liu, X.-Y., Yang, H., Gao, J., & Wang, C.
- Affiliations
- Columbia University (US)
- University of Virginia (US)
- New York University (US)
- Corresponding author
- Yang, Hongyang
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2021
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W3213398673
- Cited by
- 46
- References
- 43
- Citations by year
- 2026: 16 · 2025: 7 · 2024: 6 · 2023: 9 · 2022: 3 · 2021: 3
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Social Sciences › Business, Management and Accounting › Management Information Systems › FinTech, Crowdfunding, Digital Finance
- Keywords
- deep reinforcement learning, algorithmic trading, quantitative trading, open source software framework, reward function, market simulation, trading constraints, asset allocation, cryptocurrency trading, stock trading, market frictions, liquidity, risk aversion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance. https://app.cochat.ai/papers/10.2139/ssrn.3955949
- The paper
- Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021). FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3955949
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.