The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Learning to trade via direct reinforcement
Abstract
We present methods for optimizing portfolios, asset allocations, and trading systems based on direct reinforcement (DR). In this approach, investment decision-making is viewed as a stochastic control problem, and strategies are discovered directly. We present an adaptive algorithm called recurrent reinforcement learning (RRL) for discovering investment policies. The need to build forecasting models is eliminated, and better trading performance is obtained. The direct reinforcement approach differs from dynamic programming and reinforcement algorithms such as TD-learning and Q-learning, which attempt to estimate a value function for the control problem. We find that the RRL direct reinforcement framework enables a simpler problem representation, avoids Bellman's curse of dimensionality and offers compelling advantages in efficiency. We demonstrate how direct reinforcement can be used to optimize risk-adjusted investment returns (including the differential Sharpe ratio), while accounting for the effects of transaction costs. In extensive simulation work using real financial data, we find that our approach based on RRL produces better trading strategies than systems utilizing Q-learning (a value function method). Real-world applications include an intra-daily currency trader and a monthly asset allocation system for the S&P 500 Stock Index and T-Bills.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Identification and control of dynamical systems using neural networks
Narendra, K. S., & Parthasarathy, K. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80202 · Cited by 8,014
Option pricing: A simplified approach
Cox, J. C., Ross, S. A., & Rubinstein, M. (1979) · Journal of Financial Economics · doi:10.1016/0304-405x(79)90015-1 · Cited by 6,083
Optimum consumption and portfolio rules in a continuous-time model
Merton, R. C. (1971) · Journal of Economic Theory · doi:10.1016/0022-0531(71)90038-x · Cited by 5,958
Portfolio Selection: Efficient Diversification of Investments
Stuart, A., & Markowitz, H. M. (1959) · OR · doi:10.2307/3006625 · Cited by 5,534
Cited by
Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522401 · Cited by 822
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation
Tesauro, G., Jong, N. K., Das, R., & Bennani, M. N. (2006) · 2006 IEEE International Conference on Autonomic Computing · doi:10.1109/icac.2006.1662383 · Cited by 365
A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
Jiang, Z., Xu, D., & Liang, J. (2017) · arXiv · doi:10.48550/arxiv.1706.10059 · Cited by 321
(2017) · doi:10.1109/intellisys.2017.8324237 · Cited by 298
Cryptocurrency Portfolio Management with Deep Reinforcement Learning
Jiang, Z., & Liang, J. (2016) · arXiv · doi:10.48550/arxiv.1612.01277 · Cited by 298
DeepLOB: Deep Convolutional Neural Networks for Limit Order Books
Zhang, Z., Zohren, S., & Roberts, S. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2907260 · Cited by 285
Kernel-Based Least Squares Policy Iteration for Reinforcement Learning
Xu, X., Hu, D., & Lu, X. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.899161 · Cited by 268
Deep Reinforcement Learning for Trading
Zhang, Z., Zohren, S., & Roberts, S. (2020) · The Journal of Financial Data Science · doi:10.3905/jfds.2020.1.030 · Cited by 227
Deep reinforcement learning for automated stock trading: an ensemble strategy
Yang, H., Liu, X.-Y., Zhong, S., & Walid, A. (2020) · Proceedings of the First ACM International Conference on AI in Finance · doi:10.1145/3383455.3422540 · Cited by 196
“[24].”
Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies
Tesauro, G. (2007) · IEEE Internet Computing · doi:10.1109/mic.2007.21 · Cited by 143
Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach
Liu, Y., Liu, Q., Zhao, H., Pan, Z., & Liu, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5587 · Cited by 141
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3737257 · Cited by 90
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. D. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494366 · Cited by 87
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Chen, Q., Zhang, R., Yang, L., Xiao, B., & Wang, C. (2025) · doi:10.2139/ssrn.3737859 · Cited by 53
“Moody and Saffell [36] utilized policy search and direct RL for stock trading.”
FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021) · doi:10.2139/ssrn.3955949 · Cited by 46
StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?
Chen, Y., Yao, Z., Liu, Y., Xin, A., Ye, J., Yu, J., Hou, L., & Li, J. (2025) · arXiv · Cited by 62
“In particular, we follow previous frameworks (Zhang et al., 2020; Tsantekidis et al., 2017; Moody and Saffell, 2001; Deng et al., 2016) and organize the stock-trading workflow into four essential stages: portfolio overview, in-depth stock analysis, decision generation, and execution and validation.”
Metadata
- Authors
- Moody, J., & Saffell, M.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 2001-07
- Volume
- 12
- Issue
- 4
- Pages
- 875-889
- ISSN
- 1045-9227 (ISSN-L)
- OpenAlex
- W2169015875
- Cited by
- 535
- References
- 62
- FWCI
- 5.54
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.