A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
Open accessAbstract
Financial portfolio management is the process of constant redistribution of a fund into different financial products. This paper presents a financial-model-free Reinforcement Learning framework to provide a deep machine learning solution to the portfolio management problem. The framework consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. This framework is realized in three instants in this work with a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). They are, along with a number of recently reviewed or published portfolio-selection strategies, examined in three back-test experiments with a trading period of 30 minutes in a cryptocurrency market. Cryptocurrencies are electronic and decentralized alternatives to government-issued money, with Bitcoin as the best-known example of a cryptocurrency. All three instances of the framework monopolize the top three positions in all experiments, outdistancing other compared trading algorithms. Although with a high commission rate of 0.25% in the backtests, the framework is able to achieve at least 4-fold returns in 50 days.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FinRL reproduces a portfolio allocation strategy [21] that uses a DRL agent to allocate capital to a set of stocks and reallocate periodically.
“We reproduce a portfolio allocation strategy [21] that uses a DRL agent to allocate capital to a set of stocks and reallocate periodically.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [21]
The Introduction (p. 2) describes an RL agent that sets new portfolio weights for each trading period, supporting periodic reallocation. But the experiments described in Deep Portfolio Management (p. 3) allocate among cryptocurrencies, not stocks; the source does not establish a stock-allocation strategy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredDRL trading agents build a multi-factor model and provide algorithmic trading strategies, which are difficult for human traders.
“Taking many complex financial factors into account, DRL trading agents build a multi-factor model and provide algorithmic trading strategies, which are difficult for human traders [3, 47, 24, 22].”
From FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2020) · cites this paper as [22]
The Introduction (p. 2) and Deep Portfolio Management (p. 3) describe an RL trading agent that uses asset histories and prior portfolio weights to choose trading actions while accounting for transaction costs. The Abstract (p. 1) calls the framework financial-model-free; the paper does not establish that it builds a multi-factor model or that its strategies are difficult for human traders. (LLM, conf=0.85)
Experimental — accuracy not yet measuredMachine learning or deep learning models such as Random Forest, LightGBM, LSTM, and BERT are used as baselines.
“Given that classification models can be used for news sentiment prediction, machine learning or deep learning models such as Random Forest[30], LightGBM[17], LSTM[13], and BERT[7] are also used as baselines.”
From Large Language Model Agent in Financial Trading: A Survey (2026) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 32 · Cited by 321
Research works printed but not linked: 11
- Amit Agarwal, Elad Hazan, Satyen Kale, and Robert E Schapire. Algorithms for portfolio management based on the newton method. In Proceedings of the 23rd international conference on Machine learning , pages 9-16. ACM, 2006.
- Iddo Bentov, Charles Lee, Alex Mizrahi, and Meni Rosenfeld. Proof of activity: Extending bitcoin's proof of work via proof of stake [extended abstract] y. ACM SIGMETRICS Performance Evaluation Review , 42(3):34-37, 2014.
- Joseph Bonneau, Andrew Miller, Jeremy Clark, Arvind Narayanan, Joshua A Kroll, and Edward W Felten. Sok: Research perspectives and challenges for bitcoin and cryptocurrencies. In 2015 IEEE Symposium on Security and Privacy , pages 104-121. IEEE, 2015.
- Allan Borodin, Ran El-Yaniv, and Vincent Gogan. On the competitive theory and practice of portfolio selection. In Latin American Symposium on Theoretical Informatics , pages 173-196. Springer, 2000.
- DCharles, II Kirkpatrick, and Julie R Dahlquist. Technical analysis: The complete resource for financial market technician. ISBN-13 , pages 978-0137059447, 2006.
- Thomas M Cover. Universal portfolios. Mathematical finance , 1(1):1-29, 1991.
- Thomas M Cover. Universal data compression and portfolio selection. In Foundations of Computer Science, 1996. Proceedings., 37th Annual Symposium on , pages 534-538. IEEE, 1996.
- James Cumming. An investigation into the use of reinforcement learning techniques within the algorithmic trading domain. Master's thesis, Imperial College London, United Kiongdoms, 2015. URL http://www.doc.ic.ac.uk/teaching/distinguished-projects/2015/j.cumming.pdf .
- Puja Das and Arindam Banerjee. Meta optimization and its application to portfolio selection. Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '11 , page 1163, 2011. doi: 10.1145/2020408.2020588. URL http://dl.acm.org/citation.cfm?doid=2020408.2020588 . · DOI
- M.A.H. Dempster and V. Leemans. An automated fx trading system using adaptive reinforcement learning. Expert Systems with Applications , 30(3):543 -552, 2006. ISSN 0957-4174. doi: http://dx.doi.org/10.1016/j.eswa.2005.10.012. URL http://www.sciencedirect.com/science/article/pii/S0957417405003015 . Intelligent Information Systems for Financial Engineering. · DOI
- Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. Deep direct reinforcement learning for financial signal representation and trading. IEEE transactions on neural networks and learning systems , 28(3):653-664, 2017.
- Authors
- Jiang, Z., Xu, D., & Liang, J.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- arXiv
- 1706.10059
- OpenAlex
- W2731083990
- Cited by
- 321
- References
- 32
- Citations by year
- 2026: 6 · 2025: 34 · 2024: 48 · 2023: 53 · 2022: 44 · 2021: 56 · 2020: 50 · 2019: 20 · 2018: 9 · 2017: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Decision Sciences › Management Science and Operations Research › Risk and Portfolio Optimization
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Keywords
- financial portfolio management, deep reinforcement learning, model-free reinforcement learning, reward function design, cryptocurrency trading, trading algorithms, convolutional neural networks, recurrent neural networks, long short-term memory, backtesting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
