Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
Abstract
Can we train the computer to beat experienced traders for financial assert trading? In this paper, we try to address this challenge by introducing a recurrent deep neural network (NN) for real-time financial signal representation and trading. Our model is inspired by two biological-related learning concepts of deep learning (DL) and reinforcement learning (RL). In the framework, the DL part automatically senses the dynamic market condition for informative feature learning. Then, the RL module interacts with deep representations and makes trading decisions to accumulate the ultimate rewards in an unknown environment. The learning system is implemented in a complex NN that exhibits both the deep and recurrent structures. Hence, we propose a task-aware backpropagation through time method to cope with the gradient vanishing issue in deep training. The robustness of the neural system is verified on both the stock and the commodity future markets under broad testing conditions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In machinelearning systems, humans shape and reshape algorithms through their daily actions and interactions (Deng, Bao, Kong, Ren, & Dai, 2017).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Deng, Bao, Kong, Ren, & Dai, 2017
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1999) · Cited by 12,812
(1995) · Cited by 10,558
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“DL techniques have been witnessed in image categorization [26] and speech recognition [29].”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“When directly applying the BPTT, the gradient vanish on deep layers is not avoided in the fine-tuning step [41].”
MethodMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Research works printed but not linked: 1
- prokhorov, 2001, Dynamical neural networks for control, A Field Guide to Dynamical Recurrent Networks
Cited by
Items 801–822 of 822
Huang, Y., Yan, J., & Chen, Z. (2026) · doi:10.1145/3810280.3810368
“Early implementations employed RNNs or Long Short-Term Memory (LSTM) units to capture temporal sequence features [18], while later approaches integrated CNNs for spatial feature extraction [1].”
Adaptive Asset Allocation: A Multi-Asset Rotation Strategy Based on Deep Reinforcement Learning
Gao, M.-D. (2026) · Proceedings of the 2026 5th International Conference on Cyber Security, Artificial Intelligence and Digital Economy · doi:10.1145/3819836.3819888
“With the rise of deep learning, Deng et al. (2016) [5] proposed and carried out research on the combination of deep neural networks and reinforcement learning, marking an important starting point for the application of DRL in the financial field [6].”
Yin, B., Liu, Y., & Zhang, B. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00073
“Deep RL meth-ods were applied to the representation of financial signals and single-asset trading [21], and later extended to the management of multiple-asset portfolios.”
Jain, A., Saxena, S., & Singhal, N. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-29319-0_16
Vishwakarma, M., Chakole, J., & Kurhekar, M. P. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03298-9
“In order to mitigate these limitations, the article by [40] uses stacked autoencoders and fuzzy deep neural networks to generate compact and denoised features on raw time series before feeding them to a Q-learning agent, thus resulting in better profitability and stability.”
Research on reinforcement learning stock trading strategy based on cost-aware information selection
Hu, M.-Q., & Zhang, Z. (2026) · Journal of Asset Management · doi:10.1057/s41260-026-00476-w
“The end-to-end model of Deng et al. (2017 ) avoids artificial feature engineering, but its scalability is insufficient in large-scale factor pool scenarios.”
Ensemble Deep Reinforcement Learning (DRL) for Adaptive Crypto Trading with Risk Control
Lekha, J., & Salim, M. (2026) · International Conference on Trends in Quantum Computing and Emerging Business Technologies (TQCEBT) · doi:10.1109/tqcebt67648.2026.11681775
“Deng et al. [11] utilized deep direct RL with recurrent networks for trading in stocks and identified temporal patterns without ensemble strategies.”
Uncertainty-Aware Forecast-Conditioned Reinforcement Learning for Multi-Asset Algorithmic Trading
Verma, A. A., Arti, M. K., & Kumar, S. (2026) · International Conference Computational Vision and Bio Inspired Computing · doi:10.1109/iccvbic71195.2026.11688423
“While learning-based approaches have improved predictive accuracy in some cases, they still struggle under strong non-stationarity and sudden regime changes [3], [4].”
Oh, J. C. (2026) · IEEE Conference on Computational Intelligence for Financial Engineering & Economics · doi:10.1109/cifer67845.2026.11692424
“Deep learning approaches to financial prediction include LSTM networks [16], reinforcement learning [17], and neural ensemble methods [18].”
Tatia, A., Agarwal, S., Limbad, R., Chachad, S. P., & Agarwal, D. (2026) · IGI Global eBooks · doi:10.4018/979-8-2600-4285-4.ch012
Online self-knowledge distillation for financial trading agents using deep learning
Floratos, P., Tsantekidis, A., Passalis, N., & Tefas, A. (2026) · Soft Computing · doi:10.1007/s00500-025-10801-x
“…range from supervised stock price prediction using Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) Mehtab and Sen ( 2020 ); Mehtab et al. ( 2020 ) to Deep Reinforcement Learning (DRL) methods Tsantekidis et al. ( 2020 ); Deng et al. ( 2016 ).”
Synaptic Bridges: Connecting Internal Neural Network Layers for Multimodal Information Fusion
Vilão, C. O., & Bianchi, R. A. C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39895-6_15
Praveen, S., Yeswanth, M., & Priyanka Pramila, R. (2026) · International Conference on Smart Communication and Sustainable Technologies (ICSCST) · doi:10.1109/icscst70031.2026.11709517
“Third, deep learning models such as LSTM and Transformer model complex temporal dependencies but require large datasets and significant computational resources [11], [12].”
Mora, F., & Kristjanpoller, W. (2026) · Computational Economics · doi:10.1007/s10614-026-11433-5
Sutar, S. G., Jacob, D. G., & Jeyasudha, J. (2026) · International Conference on Natural Language Processing and Computer Vision (ICNPCV) · doi:10.1109/icnpcv69434.2026.11708808
“Recent studies suggest that RL-driven trading systems are capable of adapting to dynamic market conditions, thereby improving long-term objectives like risk-adjusted returns [5], [6].”
InfluentialEnhancing Multi Agent Large Language Models for Stock Trading through Quantitative Alpha IntegrationPreprint
Hong-Viet, T., Tang, N. V., & Phan, H.-D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7629484
A Market-Aligned Reinforcement Learning Framework for Financial Sentiment Analysis
Liu, Z., Sham, C., Li, J., Ma, S. L., Yue, Z., & Fu, C. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729208
“In the financial domain, RL has primarily been applied to trading strategy optimization, where reward signals are naturally defined by realized returns [27], [28].”
Direct Log-Utility Optimization for Portfolio Allocation under Exogenous Market Dynamics
Kurnosau, I., & Altahhan, A. (2026) · IEEE International Joint Conference on Neural Network · doi:10.1109/ijcnn69854.2026.11729518
“Under this standard assumption [3, 12, 24, 25, 21], market dynamics are exogenously generated and the transition distribution simplifies to indicating conditional independence of S t from A t − 1 given S t − 1 .”
AI-Native Asset Management: A Product Strategy Framework for Intelligent Investment Platforms
Bandhuvula, A. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.5520
“Deng et al. [14] investigated this in the contexts of representation learning and decision control, directly learning financial signals and trading using deep direct reinforcement learning.”
Huang, Z., Yao, Y., Mao, Y., Tang, R., & 谭其震 (2026) · International Conference on Artificial Intelligence and Digital Finance (AIDF) · doi:10.1145/3844275.3844290
“The global financial market is a complex dynamical system characterized by dynamic interconnectivity, asymmetric information flows, and continuous structural evolution [2, 36].”
Automated Market Trend Analysis Using Deep Learning and Big Data Analytics
Chellamuthu, S., Nagarathinam, A., Palanikumar, M., Ramachandran, V., Nayeem, A. F. A., & Kumar, V. P. (2026) · International Conference on Next-Gen Intelligent Computing and Applications (ICONICA) · doi:10.1109/iconica70605.2026.11736578
“Direct reinforcement approaches that learn a position function without an intermediate price forecast predate the deep era [38] and were later combined with deep representation learning and fuzzy input reduction, reporting improved profit and stability on index and commodity futures [39].”
Avoiding Zero-Trade Policies in RL with a Decoupled MLOps Architecture
Tanaka, S. (2026) · Proceedings of the Python in Science Conferences · doi:10.25080/fdgu8736
Metadata
- Authors
- Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q.
- Affiliations
- University of California, San Francisco (US)
- Tsinghua University (CN)
- Southeast University (CN)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2017-03
- Volume
- 28
- Issue
- 3
- Pages
- 653-664
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2344786740
- Cited by
- 822
- References
- 51
- Citations by year
- 2026: 90 · 2025: 90 · 2024: 95 · 2023: 110 · 2022: 109 · 2021: 100 · 2020: 98 · 2019: 76 · 2018: 40 · 2017: 10
- FWCI
- 47.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Project of the National Natural Science Foundation of China (61327902, 61120106003)
- National Science Foundation of Jiangsu Province, China (BK20150650)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- deep reinforcement learning, financial trading, recurrent neural networks, backpropagation through time, gradient vanishing, stock market, commodity futures, trading decision-making, representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.