Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
Abstract
Can we train the computer to beat experienced traders for financial assert trading? In this paper, we try to address this challenge by introducing a recurrent deep neural network (NN) for real-time financial signal representation and trading. Our model is inspired by two biological-related learning concepts of deep learning (DL) and reinforcement learning (RL). In the framework, the DL part automatically senses the dynamic market condition for informative feature learning. Then, the RL module interacts with deep representations and makes trading decisions to accumulate the ultimate rewards in an unknown environment. The learning system is implemented in a complex NN that exhibits both the deep and recurrent structures. Hence, we propose a task-aware backpropagation through time method to cope with the gradient vanishing issue in deep training. The robustness of the neural system is verified on both the stock and the commodity future markets under broad testing conditions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In machinelearning systems, humans shape and reshape algorithms through their daily actions and interactions (Deng, Bao, Kong, Ren, & Dai, 2017).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Deng, Bao, Kong, Ren, & Dai, 2017
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1999) · Cited by 12,812
(1995) · Cited by 10,558
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“DL techniques have been witnessed in image categorization [26] and speech recognition [29].”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“When directly applying the BPTT, the gradient vanish on deep layers is not avoided in the fine-tuning step [41].”
MethodMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Research works printed but not linked: 1
- prokhorov, 2001, Dynamical neural networks for control, A Field Guide to Dynamical Recurrent Networks
Cited by
Items 301–400 of 822
Financial Time Series Forecasting Using Prophet
Yusof, U. K., Khalid, M. N. A., Hussain, A. J., & Shamsudin, H. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-70713-2_45 · Cited by 14
Reinforcement Learning with Self-Attention Networks for Cryptocurrency Trading
Betancourt, C., & Chen, W.-H. (2021) · Applied Sciences · doi:10.3390/app11167377 · Cited by 24
“[10] 2016 Combined fuzzy logic and RNNs for stock trading.”
BackgroundLouati, F., Ktata, F. B., & Amous, I. (2024) · SN Computer Science · doi:10.1007/s42979-024-03001-1 · Cited by 23
S&P 500 Stock Price Prediction Using Technical, Fundamental and Text Data
Zhong, S., & Hitchcock, D. B. (2021) · Statistics Optimization & Information Computing · doi:10.19139/soic-2310-5070-1362 · Cited by 24
“This is a self-taught learning method to solve MDP problems when the environment is partly unobservable [10].”
MethodDeepMark: Embedding Watermarks into Deep Neural Network Using Pruning
Xie, C., Yi, P., Zhang, B., & Zou, F. (2021) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai52525.2021.00031 · Cited by 13
A Novel Convolutional Neural Networks for Stock Trading Based on DDQN Algorithm
Cui, K., Hao, R., Huang, Y., Li, J., & Song, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3259424 · Cited by 19
“They can also be classified as value-based method [38], [39], [40], [41], [42], [43], [44], [45], [46] and policy-based method [47], [48], respectively.”
SAAS parallel task scheduling based on cloud service flow load algorithm
Zhu, J. F., Li, Q., & Shi, Y. (2021) · Computer Communications · doi:10.1016/j.comcom.2021.10.037 · Cited by 13
Market Making Strategy Optimization via Deep Reinforcement Learning
Sun, T., Huang, D., & Yu, J. (2022) · IEEE Access · doi:10.1109/access.2022.3143653 · Cited by 21
“[3] used direct deep reinforcement learning method to represent real-time financial signals, and trained an agent for financial asset trading, and few work has been done on the buy-side market making strategies.”
MethodA new representation learning approach for credit data analysis
Li, T., Kou, G., & Peng, Y. (2023) · Information Sciences · doi:10.1016/j.ins.2023.01.068 · Cited by 18
Buah, E., Linnanen, L., & Wu, H. (2019) · Energy · doi:10.1016/j.energy.2019.116210 · Cited by 14
TradeBot: Bandit learning for hyper-parameters optimization of high frequency trading strategy
Zhang, W., Wang, L., Xie, L., Feng, K., & Liu, X. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108490 · Cited by 13
An improved all-optical diffractive deep neural network with less parameters for gesture recognition
Zhou, Y., Shui, S., Cai, Y., Chen, C., Chen, Y., & Abdi–Ghaleh, R. (2022) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2022.103688 · Cited by 18
Mercury: A Deep Reinforcement Learning-Based Investment Portfolio Strategy for Risk-Return Balance
Bai, Z.-L., Zhao, Y.-N., Zhou, Z., Li, W.-Q., Gao, Y.-Y., Tang, Y. G., Dai, L.-Z., & Dong, Y.-Y. (2023) · IEEE Access · doi:10.1109/access.2023.3298562 · Cited by 13
“Due to the development of reinforcement learning technology, the model combines deep neural networks with reinforcement learning for strategy transactions [27], [28], [29],”
MethodMethod of Motion Planning for Digital Twin Navigation and Cutting of Shearer
Miao, B., Ge, S., Li, Y., & Guo, Y. (2024) · Sensors · doi:10.3390/s24185878 · Cited by 12
Yin, J.-L., Chen, B., Peng, Y., & Hwang, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3088907 · Cited by 12
“Instead of mechanically using the intensity mapping function in [8] and [12], we turn this automatic intermediate image generation into an interactive sequential decision-making problem and train an intermediate image generation agent with deep reinforcement learning [13]–[15].”
MethodAn efficient isomorphic CNN-based prediction and decision framework for financial time series
Liu, Z., Luo, H., Chen, P., Xia, Q., Gan, Z., & Shan, W. (2022) · Intelligent Data Analysis · doi:10.3233/ida-216142 · Cited by 14
“[ 9,21] exploited deep reinforcement learning approaches in stock and commodity future markets using RNN and LSTM for feature engineering.”
Intelligent trading systems: a sentiment-aware reinforcement learning approach
Paiva, F. C. L., Felizardo, L. K., Bianchi, R. A. C., & Costa, A. H. R. (2021) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3490354.3494445 · Cited by 12
“When designing RL systems, researchers typically adopt feature representation that relates exclusively to the time series of asset prices [1, 4, 17].”
BackgroundMethodInfluentialA Novel End-to-End Turkish Text-to-Speech (TTS) System via Deep Learning
Oyucu, S. (2023) · Electronics · doi:10.3390/electronics12081900 · Cited by 16
Delta Hedging of Derivatives using Deep Reinforcement LearningPreprint
Giurca, A., & Borovkova, S. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3847272 · Cited by 21
Machine Behaviour (Originally Published 2019 by Springer Nature)
Rahwan, I., Cebrián, M., Obradovich, N., Bongard, J., Bonnefon, J., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2022) · Wiley eBooks · doi:10.1002/9781119815075.ch10 · Cited by 13
Moreira, T. M., de Faria, J. G., de Melo, P. O. S. V., & Medeiros‐Ribeiro, G. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120506 · Cited by 11
Du, X.-X., Tang, Z., & Chen, K. (2023) · Energy · doi:10.1016/j.energy.2023.129394 · Cited by 11
Twin-system recurrent reinforcement learning for optimizing portfolio strategy
Park, H., Sim, M. K., & Choi, D. G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124193 · Cited by 17
Portfolio construction using explainable reinforcement learning
Cortés, D. G., Onieva, E., Pastor-López, I., Trinchera, L., & Wu, J. L. (2024) · Expert Systems · doi:10.1111/exsy.13667 · Cited by 16
“…et al., 1999; Watkins & Dayan, 1992) have emerged as important advances in the discovery of investment policies in the stock and futures markets (Deng et al., 2017; Dixon et al., 2020; Moody & Saffell, 2001; Pendharkar & Cusatis, 2018) as well as in the cryptocurrency market (Sattarov et al.,…”
Explainable machine learning for high frequency trading dynamics discovery
Han, H., Forrest, J. Y., Wang, J., yuan, S., Han, F., & Li, D. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121286 · Cited by 16
Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning
Cai, H., Xu, P., Tang, X., & Gan, L. (2024) · Electronics · doi:10.3390/electronics13163242 · Cited by 17
“These algorithms find application across numerous domains, including robotics [16], finance [17], recommendation systems [18], and various decision-making contexts.”
Tu, R., Luo, Z., Pan, C., Wang, Z., Su, J., Zhang, Y., & Wang, Y. (2025) · Human-Centric Intelligent Systems · doi:10.1007/s44230-025-00093-7 · Cited by 24
“Other areas using offline RL methods include autonomous driving [37, 38], finance [39], etc.”
A Holistic Auto-Configurable Ensemble Machine Learning Strategy for Financial Trading
Carta, S., Corriga, A., Ferreira, A., Recupero, D. R., & Saia, R. (2019) · Computation · doi:10.3390/computation7040067 · Cited by 12
“Several other interesting studies have been carried out in the literature, such as a comparison of deep learning technologies to price prediction [43], the use of deep learning and statistical approaches to forecast crisis in the stock market [44], and the use of reward-based classifiers such as Deep Reinforcement Lea…”
BackgroundBitcoin Price Prediction Using Deep Learning and Real Time Deployment
Mahendra, E., Madan, H., Gupta, S., & Singh, S. V. (2020) · International Conference on Advances in Computing, Communication Control and Networking (ICAC3N) · doi:10.1109/icacccn51052.2020.9362735 · Cited by 11
LSTM-DDPG for Trading with Variable Positions
Jia, Z., Gao, Q., & Peng, X. (2021) · Sensors · doi:10.3390/s21196571 · Cited by 12
“Deng et al. [11] introduce a Recurrent Deep Neural Network (RDNN) for simultaneous environment sensing and recurrent decision making for online financial assert trading.”
BackgroundHan, Y., Liu, J., & Zhang, Q. (2022) · Systems · doi:10.3390/systems10050146 · Cited by 15
“The policy-based approach has been implemented in [6,41,42].”
MethodA Generative Adversarial Network for Financial Advisor Recruitment in Smart Crowdsourcing Platforms
Hamadi, R., Ghazzai, H., & Massoud, Y. (2022) · Applied Sciences · doi:10.3390/app12199830 · Cited by 11
Smart Robotic Strategies and Advice for Stock Trading Using Deep Transformer Reinforcement Learning
Malibari, N., Katib, I. A., & Mehmood, R. (2022) · Applied Sciences · doi:10.3390/app122412526 · Cited by 12
Lagging problem in financial time series forecasting
Li, J., Song, L., Wu, D., Shui, J., & Wang, T. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08879-1 · Cited by 14
“In recent years, deep learning technology has achieved widespread applications in financial time series forecasting [6, 7], including stock market forecasting [8–10], stock index forecasting [11, 12], futures price forecasting [13], and exchange rate forecasting [14].”
BackgroundDeep Reinforcement Learning Monitor for Snapshot Recording
Dao, G., Mishra, I., & Lee, M. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00095 · Cited by 14
“cloud computing resource [8], advertisement technology [9], and finance [10].”
BackgroundRobust Deep Neural Network Using Fuzzy Denoising Autoencoder
Han, H., Zhang, H., & Qiao, J. (2020) · International Journal of Fuzzy Systems · doi:10.1007/s40815-020-00845-6 · Cited by 12
Deep Reinforcement Learning Algorithm Based on Optimal Energy Dispatching for Microgrid
Bian, H. W. Z. Y., Tian, X., Zhang, J., & Han, X. (2020) · Asia Conference on Power and Electrical Engineering (ACPEE) · doi:10.1109/acpee48638.2020.9136221 · Cited by 11
PAGAN: Portfolio Analysis with Generative Adversarial NetworksPreprint
Mariani, G., Zhu, Y., Li, J., Scheidegger, F., Istrate, R., Bekas, C., & Malossi, A. C. I. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1909.10578 · Cited by 38
“learning (RL) frameworks to trading strategy [8]–[10].”
Deep Reinforcement Learning for TradingPreprint
Zhang, Z., Zohren, S., & Roberts, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10107 · Cited by 11
Neural Forecasting of the Italian Sovereign Bond Market with Economic News
Consoli, S., Pezzoli, L. T., & Tosetti, E. (2022) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12813 · Cited by 10
“, 2018; Singh & Srivastava, 2017), stock trading (Deng et al., 2017; Jeong & Kim, 2019), and portfolio management (Almahdi & Yang, 2017).”
Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance
Fang, Y., Tang, Z., Ren, K., Liu, W., Zhao, L., Bian, J., Li, D., Zhang, W., Yu, Y. H., & Liu, T. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599856 · Cited by 29
“Several following works [4, 8, 9, 26, 27] adopt a data-driven mindset and utilized model-free RLmethods to learn optimal trading strategies.”
MethodFusion of standard and ordinal dropout techniques to regularise deep models
Bérchez-Moreno, F., Fernández, J. C. F., Hervás‐Martínez, C., & Gutiérrez, P. A. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102299 · Cited by 15
Evolution-guided value iteration for optimal tracking control
Huang, H., Wang, D., Zhao, M., & Hu, Q. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127835 · Cited by 10
Jeong, D. W., & Gu, Y. H. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124465 · Cited by 18
Deep recurrent Q-network algorithm for carbon emission allowance trading strategy
Wu, C., Bi, W., & Liu, H. (2024) · Journal of Environmental Management · doi:10.1016/j.jenvman.2024.123308 · Cited by 11
RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learning
Wu, Y., Pan, L., Wu, W., Wang, G., Miao, Y., Xu, F., & Wang, H. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128103 · Cited by 47
Study on Singular Spectrum Analysis as a New Technical Oscillator for Trading Rules Design
Leles, M. C. R., Mozelli, L. A., Nascimento, C. L., Sbruzzi, E. F., & Guimarães, H. N. (2018) · Fluctuation and Noise Letters · doi:10.1142/s0219477518500347 · Cited by 10
Only frequency domain diffractive deep neural networks
Song, M., Li, R., & Wang, J. (2023) · Applied Optics · doi:10.1364/ao.480640 · Cited by 10
Exploring Deep Reinforcement Learning for Autonomous Powerline Tracking
Pienroj, P., Schönborn, S., & Birke, R. (2019) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infcomw.2019.8845212 · Cited by 12
“Since then the potential of DRL has been demonstrated across different domains such as fi-nance [21], robotics [22]–[24], self-driving cars [25] and smart grids [26] via numerous algorithm evolutions [2], [8]–[10], [12], [15], [16].”
BackgroundBuah, E., Linnanen, L., Wu, H., & Kesse, M. A. (2020) · Energies · doi:10.3390/en13236259 · Cited by 13
“Using the CO 2 storage example as shown in Table 1, PA indicates positive a ff ective feelings toward the CO 2 storage.”
BackgroundMethodFair classification via Monte Carlo policy gradient method
Petrović, A., Nikolić, M., Jovanović, M., Bijanić, M., & Delibašić, B. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104398 · Cited by 14
Conditional Value-at-Risk for Quantitative Trading: A Direct Reinforcement Learning Approach
Al-Ameer, A., & Alshehri, K. A. (2022) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta49430.2022.9966017 · Cited by 13
“Here we follow the approach in [2] since it is consistent with the pioneering DRL trading systems [5], [16].”
MethodResultA tabular sarsa-based stock market agent
de Oliveira, R. A., Ramos, H. S., Dalip, D. H., & Pereira, A. C. M. (2020) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3383455.3422559 · Cited by 11
“More recently, RL has been used with success in the context of trading agents [6, 9, 12, 14, 22, 31].”
MethodEmbedded draw-down constraint reward function for deep reinforcement learning
Wu, J. M., Lin, S.-H., Syu, J.-H., & Wu, M. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109150 · Cited by 10
Application of deep reinforcement learning for Indian stock trading automationPreprint
Bajpai, S. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2106.16088 · Cited by 17
“Recently, people investigated DRL method to solve the algorithmic trading problem [20, 21, 22, 23, 24, 20, 25].”
MethodReinforcement Learning in Quantitative Trading: A SurveyPreprint
Alameer, A., Saleh, H., & Alshehri, K. A. (2022) · doi:10.36227/techrxiv.19303853 · Cited by 14
A Novel Stock Trading Model based on Reinforcement Learning and Technical Analysis
Pourahmadi, Z., Fareed, D., & Mirzaei, H. R. (2023) · Annals of Data Science · doi:10.1007/s40745-023-00469-1 · Cited by 12
“A combination of deep learning and reinforcement learning (DRL) was used to present the model in [33, 34].”
Cardinality and Bounding Constrained Portfolio Optimization Using Safe Reinforcement Learning
Li, Y., Du, N., Song, X., Yang, X., Cui, T., Xue, N., Farjudian, A., Ren, J., & Cheah, W. P. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651491 · Cited by 12
“…value V t , and time-series signal X t − 1 , which is represented by: Note that the input signal is formulated as X t = { X t 0 , X t 1 , ..., X tn } , where n represents the number of assets considered in the portfolio optimization problem, building upon the approach proposed in [16], [42].”
MethodA Stock Prediction Method Based on Deep Reinforcement Learning and Sentiment Analysis
Du, S., & Shen, H. (2024) · Applied Sciences · doi:10.3390/app14198747 · Cited by 15
Optimization-based spectral end-to-end deep reinforcement learning for equity portfolio management
Yu, P., Liu, S., Jin, C., Gu, R., & Gong, X. (2025) · Pacific-Basin Finance Journal · doi:10.1016/j.pacfin.2025.102746 · Cited by 9
Pankwaen, K., Thongkairat, S., & Saijai, W. (2025) · Mathematics · doi:10.3390/math13081317 · Cited by 13
Deep Reinforcement Learning for irrigation optimization: Advantages, opportunities, and challenges
Liu, J., Chang, F., Yang, J., Jie, X., Lu, C., Wang, C., Xie, L., Ma, L., & Su, H. (2025) · Agricultural Water Management · doi:10.1016/j.agwat.2025.110030 · Cited by 12
Deep reinforcement learning with positional context for intraday trading
Goluža, S., Kovačević, T., Bauman, T., & Kostanjčar, Z. (2024) · Evolving Systems · doi:10.1007/s12530-024-09593-6 · Cited by 9
“In (Deng et al. 2016), the authors introduced a model called fuzzy deep direct reinforcement learning .”
An Asymptotic Statistical Learning Algorithm for Prediction of Key Trading Events
Yin, J., Wang, R., Ju, S., Bai, Y., & Huang, J. Z. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2977590 · Cited by 9
“6,9,14,15 Trading actions are operations such as buying, selling, and holding.”
BackgroundMethodInfluentialLee, J., & Kang, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0230635 · Cited by 11
“However, for stock index prediction, the majority of the recent works used historical price and volume data [9, 10] as input features because such data are publicly free and easy to obtain.”
BackgroundAugmenting the communication and engagement toolkit for CO2 capture and storage projects
Buah, E., Linnanen, L., & Wu, H. (2021) · International journal of greenhouse gas control · doi:10.1016/j.ijggc.2021.103269 · Cited by 10
Optimal Action Space Search: An Effective Deep Reinforcement Learning Method for Algorithmic Trading
Duan, Z., Chen, C., Cheng, D., Liang, Y., & Qian, W. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557412 · Cited by 14
“[8] proposed a method using differential profit function as loss function.”
MethodHuang, Y., Cui, K., Song, Y., & Chen, Z. (2023) · Mathematics · doi:10.3390/math11112467 · Cited by 10
Profitability of alternative methods of combining the signals from technical trading systems
Banga, J. S., & Brorsen, B. W. (2019) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1442 · Cited by 11
Shin, W., Bu, S.-J., & Cho, S. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1909.03278 · Cited by 18
“In addition, successful deep RL-based studies that solved portfolio management problem using the Deep Q-Network(DQN) method [19], which has recently achieved tremendous success in video games, further highlight the importance of RL-based research [20], [21].”
Deep Reinforcement Learning for Asset Allocation in US EquitiesPreprint
Alonso, M. N. I., & Srivastava, S. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3711487 · Cited by 17
“• [Deng et al., 2016] proposes a task-aware backpropagation through time method to cope with the gradient vanishing issue in in-depth training.”
Self organizing optimization and phase transition in reinforcement learning minority game system
Zhang, S., Dong, J., Zhang, H., Lu, Y., Wang, J., & Huang, Z. (2024) · Frontiers of Physics · doi:10.1007/s11467-023-1378-z · Cited by 9
A Deep Reinforcement Learning Approach for Portfolio Management in Non‐Short‐Selling Market
Su, R., Chi, C., Tu, S., & Xu, L. (2024) · IET Signal Processing · doi:10.1049/2024/5399392 · Cited by 8
A Sharpe Ratio Based Reward Scheme in Deep Reinforcement Learning for Financial Trading
Rodinos, G., Nousi, P., Passalis, N., & Tefas, A. (2023) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-34111-3_2 · Cited by 8
Forecasting Turning Points in Stock Price by Integrating Chart Similarity and Multipersistence
Li, S., Liu, Y., Chen, X., Wu, J., & Xu, K. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3444814 · Cited by 8
“(cid:2) FDDR [15]: fuzzy deep direct reinforcement (FDDR), the fourth baseline, combines a recurrent deep neural network and reinforcement learning for real-time financial signal representation and trading.”
BackgroundGenerating trading rules on US Stock Market using strongly typed genetic programming
Michell, K., & Kristjanpoller, W. (2019) · Soft Computing · doi:10.1007/s00500-019-04085-1 · Cited by 8
Automatic Extraction of Blur Regions on a Single Image Based on Semantic Segmentation
Shen, A., Dong, H., Wang, K., Kong, Y., Wu, J., & Shu, H. (2020) · IEEE Access · doi:10.1109/access.2020.2978084 · Cited by 8
“In recent years, deep learning has become a promising framework, which has been widely applied in the fields of computer vision, pattern recognition and signal processing [10]–[12].”
BackgroundSelf-citationQiu, Y., Qiu, Y., Yuan, Y., Chen, Z., & Lee, R. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.749878 · Cited by 10
“In the T + n strategy, RL agents are assigned a long, neutral, or short position in each trading day, including the Fuzzy Deep Recurrent Neural Networks (FDRNN) (Deng et al., 2016) and Direct Reinforcement Learning (DRL) (Moody and Saffell, 2001).”
BackgroundMethodInfluentialA Simple Reinforcement Learning Algorithm for Stock Trading
Fiorini, P. M., & Fiorini, P.-G. (2021) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs53288.2021.9660900 · Cited by 12
Stock Price Formation: Precepts from a Multi-Agent Reinforcement Learning Model
Lussange, J., Vrizzi, S., Bourgeois‐Gironde, S., Palminteri, S., & Gutkin, B. S. (2022) · Computational Economics · doi:10.1007/s10614-022-10249-3 · Cited by 16
“Moreover, there are further machine learning applications to finance (Ganesh et al. 2019; Hu and Lin 2019; Neuneier 1997; Deng et al. 2017) and order book models have become an active area of research (Spooner et al. 2018; Biondo et al. 2019; Sirignano and Cont 2019).”
BackgroundPROBLEMS OF THE DEVELOPMENT OF SCIENCE AND THE VIEW OF SOCIETY
International Science Group, & Vladlenov, D. (2023) · doi:10.46299/isg.p.2023.1.11 · Cited by 9
Learning to Manipulate a Financial Benchmark
Shearer, M., Rauterberg, G. V., & Wellman, M. P. (2023) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3604237.3626847 · Cited by 14
“Most employ DRL with a discrete or continuous observation space and discrete action space [10, 20, 27– 29, 36, 45, 49], but some recent work considers continuous observation and action spaces [22, 30, 45, 46, 48].”
MethodMulti-Slot Spectrum Auction in Heterogeneous Networks Based on Deep Feedforward Network
Zhao, F., Zhang, Y., & Wang, Q. (2018) · IEEE Access · doi:10.1109/access.2018.2865437 · Cited by 9
“[24] proposed a task-aware back propagation through timemethod to cope with the gradient vanishing issue in deep training.”
BackgroundLeles, M. C. R., Sbruzzi, E. F., de Oliveira, J. M. P., & Nascimento, C. L. (2019) · IEEE International Systems Conference (SysCon) · doi:10.1109/syscon.2019.8836887 · Cited by 8
“are based on this approach [12–14].”
CROP: Certifying Robust Policies for Reinforcement Learning through Functional SmoothingPreprint
Wu, F., Li, L., Huang, Z., Vorobeychik, Y., Zhao, D. X., & Li, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09292 · Cited by 44
“Reinforcement learning (RL) has been widely applied to different applications, such as robotics [37, 15, 51], autonomous driving vehicles [57, 54], and trading [16, 3, 71].”
BackgroundMethodThongkairat, S., & Yamaka, W. (2025) · Mathematics · doi:10.3390/math13030461 · Cited by 11
A new hybrid method of recurrent reinforcement learning and BiLSTM for algorithmic trading
Huang, Y., & Song, Y. (2023) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-223101 · Cited by 9
“Deng et al. introduced a recurrent deep neural network for real-time financial signal representation and proposed a task-aware backpropagation method to alleviate the gradient vanishing problem in DRL [33].”
Market Making with Deep Reinforcement Learning from Limit Order Books
Guo, H. M., Lin, J., & Huang, F. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191123 · Cited by 7
Reinforced Labels: Multi-Agent Deep Reinforcement Learning for Point-Feature Label Placement
Bobák, P., Čmolík, L., & Čadík, M. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3313729 · Cited by 11
“RecentresearchhassuccessfullyproventhatDRLcan solvecomplexproblemswithinseveraldomainsevenwith enormousstatespaces,e.g.,robotics[14],[15],self-driving cars[16],industrialdesign[17],andfinance[18].”
Improved pairs trading strategy using two-level reinforcement learning framework
Xu, Z., & Luo, C. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107148 · Cited by 9
Predicting price trends combining kinetic energy and deep reinforcement learning
Ghotbi, M., & Zahedi, M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122994 · Cited by 7
Reinforcement Learning Review: Past Acts, Present Facts and Future Prospects
Kommey, B., Isaac, O. J., Tamakloe, E., & Opoku, D. (2024) · IT JOURNAL RESEARCH AND DEVELOPMENT · doi:10.25299/itjrd.2023.13474 · Cited by 10
Model-Based Off-Policy Deep Reinforcement Learning With Model-Embedding
Tan, X., Qu, C., Xiong, J., Zhang, J. Y., Qiu, X., & Jin, Y. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3369636 · Cited by 11
Off-Policy Meta-Reinforcement Learning With Belief-Based Task Inference
Imagawa, T., Hiraoka, T., & Tsuruoka, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3170582 · Cited by 11
Deep Reinforcement Learning for Automated Stock Trading: Inclusion of Short Selling
Asodekar, E., Nookala, A., Ayre, S., & Nimkar, A. V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16564-1_18 · Cited by 8
Kurumatani, K. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00020 · Cited by 7
“Recently, improved versions of LSTM, such as Phased LSTM [8], Augmented LSTM [9], a CNN-based algorithm called WaveNet [10], and a new reinforcement learningbased prediction algorithm [11] have been proposed and showed good performances in prediction at a specific point in the future.”
BackgroundShih, W.-Y., Lu, Y.-S., Tsai, H., & Huang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3016824 · Cited by 7
Transferring trading strategy knowledge to deep learning models
Tsantekidis, A., & Tefas, A. (2020) · Knowledge and Information Systems · doi:10.1007/s10115-020-01510-y · Cited by 8
“In [2], DRL is applied to financial trading, with an agent that is trained to automatically perform trading decisions on several types of financial products.”
MethodImproving Deep Reinforcement Learning for Financial Trading Using Neural Network Distillation
Tsantekidis, A., Passalis, N., & Tefas, A. (2020) · Machine learning for signal processing ... · doi:10.1109/mlsp49062.2020.9231849 · Cited by 8
“Also, trading in the context of RL has been previously implemented in works such as [19, 20] where the profit or metrics relating to returns have been defined as the objective of an RL agent.”
BackgroundPassalis, N., Seficha, S., Tsantekidis, A., & Tefas, A. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-79150-6_59 · Cited by 7
Metadata
- Authors
- Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q.
- Affiliations
- University of California, San Francisco (US)
- Tsinghua University (CN)
- Southeast University (CN)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2017-03
- Volume
- 28
- Issue
- 3
- Pages
- 653-664
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2344786740
- Cited by
- 822
- References
- 51
- Citations by year
- 2026: 90 · 2025: 90 · 2024: 95 · 2023: 110 · 2022: 109 · 2021: 100 · 2020: 98 · 2019: 76 · 2018: 40 · 2017: 10
- FWCI
- 47.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Project of the National Natural Science Foundation of China (61327902, 61120106003)
- National Science Foundation of Jiangsu Province, China (BK20150650)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- deep reinforcement learning, financial trading, recurrent neural networks, backpropagation through time, gradient vanishing, stock market, commodity futures, trading decision-making, representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.