FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Open accessThe DOI matches its Crossref record; 51 of the 54 references it cites were found in a registry or an index; 1 of the 14 sources its claims cite were checked for support.
Overview
CoChat Summary
## TL;DR
FinRL is presented as an open-source, three-layer deep reinforcement learning library for automated stock trading, aimed at making quantitative finance workflows beginner-friendly, reproducible, and extensible. The paper demonstrates the library on three use cases and reports that several DRL strategies outperform benchmark indices or traditional allocation strategies on its test examples.
## Key findings
- FinRL organizes automated stock trading into a three-layer architecture: market environment, DRL agent, and trading applications, with modular components and reserved user-import interfaces.
- The library provides six standard trading environments based on representative listings: NASDAQ-100, DJIA, S&P 500, SSE 50, CSI 300, and HSI, plus one user-defined environment.
- In the single-stock example using PPO over 2019/01/01–2020/09/23, final portfolio values ranged from 127,044 to 192,031 versus 133,402 for the S&P 500; annualized returns ranged from 14.89% to 44.94% versus 17.81% for the S&P 500, and Sharpe ratios ranged from 1.10 to 1.49 versus 0.74 for the S&P 500.
- In the multiple-stock / portfolio-allocation example over DJIA constituents, TD3 and DDPG achieved Sharpe ratios above the DJIA and min-variance baseline: TD3 1.38 and 1.03, DDPG 1.28 and 0.98, versus 0.48 for DJIA and 0.44 for min-variance.
## Methods
- The paper introduces FinRL as a three-layer library: an environment layer built on OpenAI Gym time-driven trading simulators, an agent layer with fine-tuned DRL algorithms, and an application layer with trading use cases.
- The environment models stock trading as a Markov Decision Process and supports state variables such as balance, shares held, prices, volume, and technical indicators; actions include sell/hold/buy and can be defined over multiple shares.
- Evaluation follows a training-validation-testing flow, uses automated backtesting with Quantopian pyfolio, and reports standard metrics including final portfolio value, annualized return, annualized standard deviation, maximum drawdown, and Sharpe ratio.
## Limitations
- The paper does not report a formal experimental design with statistical significance testing, confidence intervals, or repeated-run variability for the reported performance numbers.
- It is not stated in the paper whether the reported case-study results generalize beyond the shown datasets, exchanges, time period, or chosen algorithms.
- Evident from the text: the paper is primarily a library/system description and demonstration paper, so the evaluation is limited to illustrative use cases rather than a comprehensive head-to-head benchmark across all methods and markets.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 5 Not valid · 4 Not checked · 17 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“FinRL uses five conventional strategies, namely passive buy-and-hold trading strategy [32], mean-variance strategy [1], and min-variance strategy [1], momentum trading strategy [15], and equal-weighted strategy”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we demonstrate easily reproducible profitable strategies under different scenarios using FinRL”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“FinRL is an open source library specifically designed and implemented for quantitative finance.”
- Also stated in: § Abstract As deep reinforcement learning (DRL) has been recognized as an effective approach in quantitative finance, getting hands-on experiences is attractive to beginners. However, to train a practical DRL trading agent that decides where to trade, at what price, and what quantity involves error-prone and arduous development and debugging. In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies. Along with easily-reproducible tutorials, FinRL library allows users to streamline their own developments and to compare with existing schemes easily. Within FinRL, virtual environments are configured with stock market datasets, trading agents are trained with neural networks, and extensive backtesting is analyzed via trading performance. Moreover, it incorporates important trading constraints such as transaction cost, market liquidity and the investor's degree of risk-aversion. FinRL is featured with completeness, hands-on tutorial and reproducibility that favors beginners: (i) at multiple levels of time granularity, FinRL simulates trading environments across various stock markets, including NASDAQ-100, DJIA, S&P 500, HSI, SSE 50, and CSI 300; (ii) organized in a layered architecture with modular structure, FinRL provides fine-tuned state-of-theart DRL algorithms (DQN, DDPG, PPO, SAC, A2C, TD3, etc.), commonly-used reward functions and standard evaluation baselines to alleviate the debugging workloads and promote the reproducibility, and (iii) being highly extendable, FinRL reserves a complete set of user-import interfaces. Furthermore, we incorporated three application demonstrations, namely single stock trading, multiple stock trading, and portfolio allocation. The FinRL library will be available on Github at link https://github.com/AI4Finance-LLC/FinRL-Library. · plain
“In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
- Also stated in: § Abstract As deep reinforcement learning (DRL) has been recognized as an effective approach in quantitative finance, getting hands-on experiences is attractive to beginners. However, to train a practical DRL trading agent that decides where to trade, at what price, and what quantity involves error-prone and arduous development and debugging. In this paper, we introduce a DRL library FinRL that facilitates beginners to expose themselves to quantitative finance and to develop their own stock trading strategies. Along with easily-reproducible tutorials, FinRL library allows users to streamline their own developments and to compare with existing schemes easily. Within FinRL, virtual environments are configured with stock market datasets, trading agents are trained with neural networks, and extensive backtesting is analyzed via trading performance. Moreover, it incorporates important trading constraints such as transaction cost, market liquidity and the investor's degree of risk-aversion. FinRL is featured with completeness, hands-on tutorial and reproducibility that favors beginners: (i) at multiple levels of time granularity, FinRL simulates trading environments across various stock markets, including NASDAQ-100, DJIA, S&P 500, HSI, SSE 50, and CSI 300; (ii) organized in a layered architecture with modular structure, FinRL provides fine-tuned state-of-theart DRL algorithms (DQN, DDPG, PPO, SAC, A2C, TD3, etc.), commonly-used reward functions and standard evaluation baselines to alleviate the debugging workloads and promote the reproducibility, and (iii) being highly extendable, FinRL reserves a complete set of user-import interfaces. Furthermore, we incorporated three application demonstrations, namely single stock trading, multiple stock trading, and portfolio allocation. The FinRL library will be available on Github at link https://github.com/AI4Finance-LLC/FinRL-Library. · plain
In the paper
“To control the risk in a worst-case scenario, such as financial crisis of 2007-2008, FinRL employs the financial turbulence index turbulence t that measures extreme asset price fluctuation [25]:”
What the sources say
- Not checked — no full text of the sourcecites [25] Mark Kritzman, 2010, Skulls, financial turbulence, and risk management, Financial Analysts Journal source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, it incorporates important trading constraints such as transaction cost, market liquidity and the investor's degree of risk-aversion.”
- Also stated in: § 3.1 Architecture of the FinRL Library · plain
“We build a trading simulator that replicates live stock market and provides backtesting support that incorporates important market frictions such as transaction cost, market liquidity and the investor's degree of risk-aversion.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 3 assumptions not shown here.
- Also stated in: § 3.1 Architecture of the FinRL Library · plain
3 more claims
Create a free account to see all 27 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 10
- Probably not a substantial claim. 14
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5 Conclusions
“As a walk-through tutorial in Jupyter notebook format, we demonstrate easily reproducible profitable strategies under different scenarios using FinRL: (i) single stock trading; (ii) multiple stock trading; (iii) incorporating the mechanism of stock information penetration.”
§ 1 Introduction
“The top layer includes applications in automated stock trading, where we demonstrate three use cases, namely single stock trading, multiple stock trading and portfolio allocation.”
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
“Options, Futures, and Other Derivatives”
(2016) · AMBER – ABBS Management Business and Entrepreneurship Review · doi:10.23874/amber/2016/v7/i1/121351 · Cited by 5,007
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377
Research works printed but not linked: 6
- G Thomas, 2018, Reinforcement learning in financial markets -a survey. Fau discussion papers in economics
- Olivier Jin, 2016, Portfolio management using reinforcement learning
- Ryan Lowe, 2017, Multi-agent actorcritic for mixed cooperative-competitive environments, Advances in Neural Information Processing Systems, 6379
- David G Luenberger, 1997, Investment science
- Pg Nechchi, 2016, Reinforcement learning for automated trading, Mathematical EngineeringPolitecnico di Milano
- Quantopian, 2019, A toolkit for portfolio and risk analytics in python
Cited by
A survey on model-based reinforcement learning
Luo, F.-M., Xu, T., Lai, H., Chen, X., Zhang, W., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3696-5 · Cited by 259
A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist
Zhang, W., Zhao, L., Xia, H., Sun, S., Sun, J., Qin, M., Li, X., Zhao, Y., Zhao, Y., Cai, X., et al. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671801 · Cited by 74
Dynamic datasets and market environments for financial reinforcement learning
Liu, X.-Y., Xia, Z., Yang, H., Gao, J., Zha, D., Zhu, M., Wang, C. D., Wang, Z., & Guo, J. (2024) · Machine Learning · doi:10.1007/s10994-023-06511-w · Cited by 70
Self-citationDynamic stock-decision ensemble strategy based on deep reinforcement learning
Yu, X., Wu, W., Liao, X., & Han, Y. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03606-0 · Cited by 42
A Review of Reinforcement Learning in Financial Applications
Bai, Y., Gao, Y., Wan, R., Zhang, S., & Song, R. (2024) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-112723-034423 · Cited by 83
Metadata
- Authors
- Liu, X.-Y.
- Affiliations
- Columbia University (US)
- Corresponding author
- Liu, Xiao-Yang
- ORCID
- Liu, Xiao-Yang
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2020
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W3106281374
- Cited by
- 90
- References
- 54
- Citations by year
- 2026: 19 · 2025: 25 · 2024: 21 · 2023: 13 · 2022: 7 · 2021: 5
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Social Sciences › Business, Management and Accounting › Management Information Systems › FinTech, Crowdfunding, Digital Finance
- Keywords
- deep reinforcement learning, automated stock trading, quantitative trading, reward function, market frictions, asset allocation, cryptocurrency trading, trading constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.