Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
Abstract
Can we train the computer to beat experienced traders for financial assert trading? In this paper, we try to address this challenge by introducing a recurrent deep neural network (NN) for real-time financial signal representation and trading. Our model is inspired by two biological-related learning concepts of deep learning (DL) and reinforcement learning (RL). In the framework, the DL part automatically senses the dynamic market condition for informative feature learning. Then, the RL module interacts with deep representations and makes trading decisions to accumulate the ultimate rewards in an unknown environment. The learning system is implemented in a complex NN that exhibits both the deep and recurrent structures. Hence, we propose a task-aware backpropagation through time method to cope with the gradient vanishing issue in deep training. The robustness of the neural system is verified on both the stock and the commodity future markets under broad testing conditions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In machinelearning systems, humans shape and reshape algorithms through their daily actions and interactions (Deng, Bao, Kong, Ren, & Dai, 2017).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Deng, Bao, Kong, Ren, & Dai, 2017
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1999) · Cited by 12,812
(1995) · Cited by 10,558
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“DL techniques have been witnessed in image categorization [26] and speech recognition [29].”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“When directly applying the BPTT, the gradient vanish on deep layers is not avoided in the fine-tuning step [41].”
MethodMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Research works printed but not linked: 1
- prokhorov, 2001, Dynamical neural networks for control, A Field Guide to Dynamical Recurrent Networks
Cited by
State-of-the-art in artificial neural network applications: A survey
Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., & Arshad, H. (2018) · Heliyon · doi:10.1016/j.heliyon.2018.e00938 · Cited by 3,054
“The comparison was made based on (i) author(s)/year of publication (ii) ANN modeling (iii) ANN area of application (iii) studied contribution to human challenges (with references to Supplementary Table 2) [11, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,…”
MethodArtificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Villarraga Ossa, E. F. (2021) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 1,694
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364
“More sophisticated agents may compute strategies based on adaptive heuristics or explicit maximization of expected utility 86 .”
MethodTransfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
A Review of Machine Learning and Deep Learning Applications
Shinde, P. P., & Shah, S. (2018) · 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA) · doi:10.1109/iccubea.2018.8697857 · Cited by 773
“A prediction domain topic such as Financial Signal Representation and Trading can be very well be implemented usingDirect Deep Reinforcement Learning technique in this paper [19].”
MethodComprehensive Review of Artificial Neural Network Applications to Pattern Recognition
Abiodun, O. I., Kiru, M. U., Jantan, A., Omolara, A. E., Dada, K. V., Umar, A. M., Linus, O. U., Arshad, H., Kazaure, A. A., & Gana, U. (2019) · IEEE Access · doi:10.1109/access.2019.2945545 · Cited by 750
“[153] proposed a deep direct reinforcement learning (DDRL) to financial sign depiction and trading, and the result proves effective to stock market business.”
BackgroundApplications of deep learning in stock market prediction: Recent progress
Jiang, W. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115537 · Cited by 714
“Reinforcement learning is also used for building algorithmic trading systems (Deng et al., 2016).”
MethodDeep learning for financial applications : A survey
Ozbayoglu, A. M., Gudelek, M. U., & Sezer, O. B. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106384 · Cited by 545
Deep learning-based feature engineering for stock price movement prediction
Long, W., Lu, Z., & Cui, L. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.10.034 · Cited by 483
Deep learning in microscopy image analysis: a survey
Xing, F., Xie, Y., Su, H., Liu, F., & Yang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766168 · Cited by 460
Sezer, O. B., & Ozbayoglu, A. M. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.04.024 · Cited by 432
AI in Finance: Challenges, Techniques, and Opportunities
Cao, L. (2023) · ACM Computing Surveys · doi:10.1145/3502289 · Cited by 366
“, prices) and trends of securities, foreign currencies, and commodities; modeling the supply-demand relations and stakeholder relationships in property markets, energy markets and capital markets; optimizing order routing in foreign exchange and features markets; and optimizing trading agent actions, option valuation,…”
BackgroundMethodInfluentialHerman, D., Googin, C., Liu, X., Sun, Y., Galda, A., Safro, I., Pistoia, M., & Alexeev, Y. (2023) · Nature Reviews Physics · doi:10.1038/s42254-023-00603-1 · Cited by 284
Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning
Li, Y., Wen, Y., Tao, D., & Guan, K. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2927410 · Cited by 274
“propose an off-policy RL method to solve nonlinear nonzero-sum games; in [37], Deng et al.”
MethodAdaptive stock trading strategies with deep reinforcement learning methods
Wu, X., Chen, H., Wang, J., Troiano, L., Loia, V., & Fujita, H. (2020) · Information Sciences · doi:10.1016/j.ins.2020.05.066 · Cited by 263
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 257
Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Ashmore, R., Calinescu, R., & Paterson, C. (2022) · ACM Computing Surveys · doi:10.1145/3453444 · Cited by 256
“Stakeholders as diverse as researchers, industrialists, policy makers and the general public envisage that ML will soon be at the core of novel applications and services used in healthcare, transportation, defence and other key areas of economy and society [10, 41, 74, 89, 109].”
BackgroundDeep learning in finance and banking: A literature review and classification
Huang, J., Chai, J., & Cho, S. (2020) · Frontiers of Business Research in China · doi:10.1186/s11782-020-00082-6 · Cited by 248
“It is a widely used technique for filtering and mining single-dimensional signals (Bao et al. 2017).”
BackgroundMethodInfluentialLi, C. (2019) · Reinforcement Learning for Cyber-Physical Systems · doi:10.1201/9781351006620-6 · Cited by 188
Temporal Attention-Augmented Bilinear Network for Financial Time-Series Data Analysis
Tran, D. T., Iosifidis, A., Kanniainen, J., & Gabbouj, M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869225 · Cited by 238
“In addition, deep reinforcement learning methods are also popular among the class of financial asset trading models [57], [58].”
BackgroundCooperative Deep Reinforcement Learning for Large-Scale Traffic Grid Signal Control
Tan, T., Bao, F., Deng, Y., Jin, A., Dai, Q., & Wang, J. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2904742 · Cited by 234
“First of all, Coder is implemented in a deep learning framework [20], [21] so it naturally exhibits the desired properties of deep feature learning and nonlinear Qfunction learning as in [22] and [23].”
MethodSelf-citationDeep Reinforcement Learning for Trading
Zhang, Z., Zohren, S., & Roberts, S. (2020) · The Journal of Financial Data Science · doi:10.3905/jfds.2020.1.030 · Cited by 227
Multi-DQN: An ensemble of Deep Q-learning agents for stock market forecasting
Carta, S., Ferreira, A., Podda, A. S., Reforgiato Recupero, D., & Sanna, A. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113820 · Cited by 226
“Deng, Bao, Kong, Ren, and Dai (2017) proposed to use deep neural networks with a novel back-propagation approach in RRL to commodity and stock markets forecasting.”
BackgroundMethodAn application of deep reinforcement learning to algorithmic trading
Théate, T., & Ernst, D. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114632 · Cited by 224
“Following this approach, DL techniques have already been investigated with good results, see e.g. Ar´evalo et al. (2016) introducing a trading strategy based on a DNN, and especially Bao et al. (2017) using wavelet transforms, stacked autoencoders and long short-term memory (LSTM).”
BackgroundMethodExperienced Gray Wolf Optimization Through Reinforcement Learning and Neural Networks
Emary, E., Zawbaa, H. M., & Grosan, C. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2634548 · Cited by 222
“The goal is to determine the actions that tend to increase the long-run sum of values of the reinforcement signal [5].”
BackgroundAn Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning
Guo, S., Zhang, X., Zheng, Y., & Du, Y. (2020) · Sensors · doi:10.3390/s20020426 · Cited by 215
Sahu, S. K., Mokhade, A., & Bokde, N. D. (2023) · Applied Sciences · doi:10.3390/app13031956 · Cited by 208
Deep Adaptive Input Normalization for Time Series Forecasting
Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., & Iosifidis, A. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944933 · Cited by 203
Recent advances in reinforcement learning in finance
Hambly, B., Xu, R., & Yang, H. (2023) · Mathematical Finance · doi:10.1111/mafi.12382 · Cited by 205
“…functions include cash inflow or outflow (depending on whether we sell or buy) (Nevmyvaka et al., 2006; Shen et al., 2014), implementation shortfall (Hendricks &Wilcox, 2014), profit (Deng et al., 2017), Sharpe ratio (Deng et al., 2017), return (Jeong . . . Kim, 2019), and PnL (Wei et al., 2019).”
BackgroundMethodInfluentialEinola, K., & Khoreva, V. (2023) · Human Resource Management · doi:10.1002/hrm.22147 · Cited by 205
Jeong, G., & Kim, H. Y. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.09.036 · Cited by 204
Hierarchical Deep Reinforcement Learning for Continuous Action Control
Yang, Z., Merrick, K., Jin, L., & Abbass, H. A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805379 · Cited by 197
“Many works [21]–[24] emerged after DQN’s success in achieving human level performance in playing Atari games.”
BackgroundDeep reinforcement learning for automated stock trading: an ensemble strategy
Yang, H., Liu, X.-Y., Zhong, S., & Walid, A. (2020) · Proceedings of the First ACM International Conference on AI in Finance · doi:10.1145/3383455.3422540 · Cited by 196
“The actor-only approach has been used in [24], [25], [26].”
A new hybrid ensemble deep reinforcement learning model for wind speed short term forecasting
Liu, H., Yu, C., Wu, H., Duan, Z., & Yan, G. (2020) · Energy · doi:10.1016/j.energy.2020.117794 · Cited by 186
Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning
Ilahi, I., Usama, M., Qadir, J., Janjua, M. U., Al-Fuqaha, A., Hoang, D. T., & Niyato, D. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3111139 · Cited by 170
Reinforcement Learning in Economics and Finance
Charpentier, A., Élie, R., & Remlinger, C. (2021) · Computational Economics · doi:10.1007/s10614-021-10119-4 · Cited by 1,307
“Such observation goes back to Moody and Saffell (2001) and has developed a growing interest in the recent literature (Deng et al. 2016; Almahdi and Yang 2017): Classical Meanvariance trade-off in a continuous time setting is for example revisited in Wang and Zhou (2019) using such viewpoint.”
BackgroundDeep Robust Reinforcement Learning for Practical Algorithmic Trading
Li, Y., Zheng, W., & Zheng, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2932789 · Cited by 195
“[22] trades one share per time, which has the action space”
BackgroundPerception and Navigation in Autonomous Systems in the Era of Learning: A Survey
Tang, Y. S., Zhao, C., Wang, J., Zhang, C., Sun, Q., Zheng, W. X., Du, W., Qian, F., & Kurths, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3167688 · Cited by 199
“With the development in learning framework [20], deep learning and reinforcement learning have demonstrated outstanding performance in image processing [21], [22], natural language processing [23], [24], motion estimation [25], game theory [26], biology [2], finance [3], and control [27], etc.”
BackgroundHe, H., Meng, X., Wang, Y., Khajepour, A., An, X., Wang, R., & Sun, F. (2023) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2023.114248 · Cited by 175
Yu, C., Yan, G., Yu, C., Zhang, Y., & Mi, X. (2022) · Energy · doi:10.1016/j.energy.2022.126034 · Cited by 147
Wang, J., Zhang, Y., Tang, K., Wu, J., & Xiong, Z. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330647 · Cited by 187
“) t . 3.3 Winners and Losers Selection In the traditional RL-based strategy models, the investment portfolio is often directly generated from the stock representations through a softmax normalization [3, 6, 16]. The drawback of this type of methods is that it does not fully exploit the interrelationships among stocks,…”
BackgroundMethodResultInfluentialAdaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach
Liu, Y., Liu, Q., Zhao, H., Pan, Z., & Liu, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5587 · Cited by 141
“However, these predefined and handcraft technical indicators are frequently criticized for their poor generalization capacity (Deng et al. 2016).”
BackgroundInfluentialWang, Z., Huang, B., Tu, S., Zhang, K., & Xu, L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16144 · Cited by 152
“Unlike other types of signals, financial data contains large amount of unpredictable uncertainties due to the random gambling and emergencies (Deng et al. 2016).”
BackgroundMethodInfluentialStockNet—GRU based stock index prediction
Gupta, U., Bhattacharjee, V., & Bishnu, P. S. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117986 · Cited by 156
An intelligent financial portfolio trading strategy using deep Q-learning
Park, H., Sim, M. K., & Choi, D. G. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113573 · Cited by 167
Lei, K., Zhang, B., Yu, L., Yang, M., & Shen, Y. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112872 · Cited by 150
Kim, A., Yang, Y., Lessmann, S., Ma, T., Sung, M., & JOHNSON, J. E. V. (2019) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.11.007 · Cited by 147
A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets
Shavandi, A., & Khedmati, M. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118124 · Cited by 176
Large data sets and machine learning: Applications to statistical arbitrage
Huck, N. (2019) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.04.013 · Cited by 138
Reinforcement learning applied to Forex trading
Carapuço, J., Neves, R. F., & Horta, N. C. G. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.09.017 · Cited by 136
Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning
Zhang, Y., Zhao, P., Li, B., Wu, Q., Huang, J., & Tan, M. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2979700 · Cited by 158
Global Stock Market Prediction Based on Stock Chart Images Using Deep Q-Network
Lee, J., Kim, R., Koh, Y., & Kang, J. (2019) · IEEE Access · doi:10.1109/access.2019.2953542 · Cited by 134
“Many studies have implemented NN based architectures [12]–[15] or RL [16] techniques.”
BackgroundFinancial time series forecasting with deep learning : A systematic literature review: 2005–2019
Sezer, Ö. B., Gudelek, M. U., & Özbayoğlu, A. M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106181 · Cited by 794
Yu, H., Tan, Z.-H., Ma, Z., Martin, R., & Guo, J. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2771947 · Cited by 111
Aboussalah, A. M., & Lee, C.-G. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112891 · Cited by 144
An Efficient Hardware Implementation of Reinforcement Learning: The Q-Learning Algorithm
Spanò, S., Cardarilli, G. C., Nunzio, L. D., Fazzolari, R., Giardino, D., Matta, M., Nannarelli, A., & Re, M. (2019) · IEEE Access · doi:10.1109/access.2019.2961174 · Cited by 115
“Recently, the applications of RL have become increasingly popular in various fields such as robotics [3]–[5], Internet of Things (IoT) [6], power management [7], financial trading [8] and telecommunications [9], [10].”
BackgroundDeep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 142
Multisource Transfer Double DQN Based on Actor Learning
Pan, J., Wang, X., Cheng, Y., & Yu, Q. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2806087 · Cited by 107
“Inspired by the advantages of DQN, researchers have developed various applications, such as optimization hyper-parameters [6], character segmentation of license plate images [7], traffic signal timing [8], automatic bridge bidding [9], financial signal processing [10], autonomous vehicle control [11], language underst…”
A Survey on Fuzzy Deep Neural Networks
Das, R., Sen, S., & Maulik, U. (2020) · ACM Computing Surveys · doi:10.1145/3369798 · Cited by 114
“Deng et al. proposed a deep learning architecture has deep learning layers and fuzzy membership functions running in parallel [Deng et al. 2017].”
BackgroundMovement forecasting of financial time series based on adaptive LSTM-BN network
Fang, Z. Y., Ma, X., Pan, H., Yang, G., & Arce, G. R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119207 · Cited by 99
Manis, K. T., & Madhavaram, S. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2022.113485 · Cited by 136
An adaptive deep Q-learning strategy for handwritten digit recognition
Qiao, J., Wang, G., Li, W., & Chen, M. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.02.010 · Cited by 97
Residual D2NN: training diffractive deep neural networks via learnable light shortcuts
Dou, H., Deng, Y. C., Yan, T., Wu, H., Lin, X., & Dai, Q. (2020) · Optics Letters · doi:10.1364/ol.389696 · Cited by 102
Self-citationFinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3737257 · Cited by 90
Zou, J., Lou, J., Wang, B., & Liu, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122801 · Cited by 89
Deep Reinforcement Learning Approach for Trading Automation in the Stock Market
Kabbani, T., & Duman, E. (2022) · IEEE Access · doi:10.1109/access.2022.3203697 · Cited by 88
“Most works that studied the RL’s applications in financial markets and particularly in trading stocks considered discrete action spaces [9], [10], [11], [12], i.”
BackgroundLiu, X.-Y., Yang, H., Gao, J., & Wang, C. D. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494366 · Cited by 87
“e deep neural networks are significantly powerful at estimating the expected return of taking a certain action at a state. Moody and Saffell [33] utilized a policy search for stock trading; Deng etal.[9] showed that DRL can obtain more profits than conventional methods. More applications include stock trading [35, 47,…”
BackgroundReinforcement Learning on Graphs: A Survey
Nie, M., Chen, D., & Wang, D. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3222545 · Cited by 113
“The rapid development of RL in cross-disciplinary domains has motivated scholars to explore novel RL models to address real-world applications, e.g., financial and economic [5]–[7], biomedical [8], [9], and transportation [10], [11].”
BackgroundAssuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Ashmore, R., Călinescu, R., & Paterson, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04223 · Cited by 83
“Stakeholders as diverse as researchers, industrialists, policy makers and the general public envisage that ML will soon be at the core of novel applications and services used in healthcare, transportation, defence and other key areas of economy and society [10, 41, 74, 89, 109].”
BackgroundDemand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach
Huang, X., Hong, S. H., Yu, M., Ding, Y., & Jiang, J. (2019) · IEEE Access · doi:10.1109/access.2019.2924030 · Cited by 86
“In practical terms, deep RL has found applications in robotics [15], financial trading [16], allocation of communication resources [17], and vehicular networking [18].”
Biju, A. V. N., Thomas, A. S., & Thasneem, J. (2023) · Quality & Quantity · doi:10.1007/s11135-023-01673-0 · Cited by 140
“Further scrutiny of the top-cited publications revealed the increasing usage of deep learning frameworks for financial time series and financial signal representation (Bao et al. 2017; Deng et al. 2017).”
BackgroundDetection of driver drowsiness using wearable devices: A feasibility study of the proximity sensor
Hu, J., Choi, W., Yang, Y., Lu, J., Wu, X., & Peng, K. (2017) · Applied Ergonomics · doi:10.1016/j.apergo.2017.02.016 · Cited by 94
Cao, D., Hu, W., Xu, X., Dragičević, T., Huang, Q., Liu, Z., Chen, Z., & Blaabjerg, F. (2019) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2019.105648 · Cited by 83
Application of deep reinforcement learning on automated stock trading
Chen, L., & Gao, Q. (2019) · 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS) · doi:10.1109/icsess47205.2019.9040728 · Cited by 78
“However, unlike the aforementioned deterministic and noise-free environment, the stock prices are known as full of noises and trading environment is changing over time[3].”
BackgroundKurumatani, K. (2020) · SN Applied Sciences · doi:10.1007/s42452-020-03225-9 · Cited by 93
“Improved versions of LSTM such as Phased LSTM [11], Augmented LSTM [12], a temporal convolution-based algorithm called WaveNet [13], and a new reinforcement learning-based prediction algorithm [14], have also been proposed.”
MethodReinforcement Learning for Quantitative Trading
Sun, S., Wang, R., & An, B. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3582560 · Cited by 148
Adaptive learning rule for hardware-based deep neural networks using electronic synapse devices
Lim, S., Bae, J.-H., Eum, J.-H., Lee, S., Kim, C.-H., Kwon, D., Park, B., & Lee, J.-H. (2018) · Neural Computing and Applications · doi:10.1007/s00521-018-3659-y · Cited by 87
Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows
Ghraieb, H., Viquerat, J., Larcher, A., Méliga, P., & Hachem, E. (2021) · Physical Review Fluids · doi:10.1103/physrevfluids.6.053902 · Cited by 91
Leow, E. K. W., Nguyen, B. P., & Chua, M. C. H. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115060 · Cited by 91
A New CNN-Based Model for Financial Time Series: TAIEX and FTSE Stocks Forecasting
Kirisci, M., & Yolcu, Ö. C. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-10767-z · Cited by 90
Multi-period portfolio optimization using a deep reinforcement learning hyper-heuristic approach
Cui, T., Du, N., Yang, X., & Ding, S. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122944 · Cited by 104
A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist
Zhang, W., Zhao, L., Xia, H., Sun, S., Sun, J., Qin, M., Li, X., Zhao, Y., Zhao, Y., Cai, X., et al. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671801 · Cited by 74
“Later on, we provide the formal formulation of FinAgent , which integrates LLMs into the RL pipeline to enable flexible reasoning and decision-making in financial trading.”
Price Trailing for Financial Trading Using Deep Reinforcement Learning
Tsantekidis, A., Passalis, N., Toufa, A.-S., Saitas-Zarkias, K., Chairistanidis, S., & Tefas, A. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2997523 · Cited by 89
Artificial intelligence techniques in financial trading: A systematic literature review
Dakalbab, F. M., Talib, M. W. A., Nasir, Q., & Saroufil, T. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102015 · Cited by 137
“…behavior, as financial markets include a few or dozens of variables; typically, a tiny change in a variable may have a devastating influence on performance; so, making a trading decision is a systematic operation that should take a lot of practical considerations into account (Deng et al., 2017).”
A deep Q-learning portfolio management framework for the cryptocurrency market
Lucarelli, G., & Borrotti, M. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05359-8 · Cited by 99
“Dynamic decision making requires the agent to explore an unknown environment independently from human interactions and, at the same time, perform correct online activities [7].”
BackgroundvrAIn: A Deep Learning Approach Tailoring Computing and Radio Resources in Virtualized RANs
Ayala‐Romero, J. A., García‐Saavedra, A., Gramaglia, M., Costa‐Pérez, X., Banchs, A., & Alcaraz, J. J. (2019) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3300061.3345431 · Cited by 76
“This falls naturally into the realm of reinforcement learning (RL) [35], an area of machine learning applied in humanlevel control (mastering games such as Go [32] or StarCraft II [41]), health-care [25] or finances [6].”
BackgroundUnderstanding Human-AI Augmentation in the Workplace: A Review and a Future Research Agenda
Trinh, N. T. T., & Elbanna, A. R. (2025) · Information Systems Frontiers · doi:10.1007/s10796-025-10591-5 · Cited by 127
Deep Reinforcement Learning for Optimizing Finance Portfolio Management
Hu, Y.-J., & Lin, S.-J. (2019) · Amity International Conference on Artificial Intelligence (AICAI) · doi:10.1109/aicai.2019.8701368 · Cited by 77
“RL has been developed for several decades [13].”
BackgroundMethodA novel CNN-DDPG based AI-trader: Performance and roles in business operations
Luo, S., Lin, X., & Zheng, Z. (2019) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2019.09.013 · Cited by 74
Application of Deep Q-Network in Portfolio Management
Gao, Z., Gao, Y., Hu, Y., Jiang, Z., & Su, J. (2020) · IEEE International Conference on Big Data Analytics (ICBDA) · doi:10.1109/icbda49040.2020.9101333 · Cited by 95
“Therefore, many studies [24],[25] solve the problem by Reinforcement Learning without predicting future prices.”
BackgroundCui, Y., Han, X., Chen, J., Zhang, X., Yang, J., & Zhang, X. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3543450 · Cited by 84
“In the context of algorithmic trading, Deng et al. [24] demonstrated how deep reinforcement learning can be used to learn trading strategies directly from market data.”
Benchmark Analysis for Robustness of Multi-Scale Urban Road Networks Under Global Disruptions
Shang, W., Gao, Z., Daina, N., Zhang, H., Long, Y., Guo, Z., & Ochieng, W. Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3149969 · Cited by 69
“H EALTH examination (HE) of urban transportation is a relatively new concept along with the establishment of urban physical examination, which is first proposed in Beijing urban master plan (2016-2035) [1].”
BackgroundProtein design via deep learning
Ding, W., Nakai, K., & Gong, H. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac102 · Cited by 66
Learning financial asset-specific trading rules via deep reinforcement learning
Taghian, M., Asadi, A., & Safabakhsh, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116523 · Cited by 82
A New Hybrid VMD-ICSS-BiGRU Approach for Gold Futures Price Forecasting and Algorithmic Trading
Li, Y., Wang, S., Wei, Y., & Zhu, Q. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3084847 · Cited by 72
Zheng, Q., Tian, X., Jiang, N., & Yang, M. (2019) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-190861 · Cited by 73
Dessain, J. (2021) · SSRN Electronic Journal · doi:10.1016/j.eswa.2022.116970 · Cited by 72
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators
Song, L., Chen, F., Zhuo, Y., Qian, X., Li, H. H., & Chen, Y. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00036 · Cited by 91
“Advances in deep learning (DL) have become the main drivers of revolutions in various commercial and enterprise applications, such as computer vision [1–3], social network [4– 6], financial data analysis [7–9], healthcare [10–12] and sci-entific computing [13–15].”
BackgroundModelling Stock Markets by Multi-agent Reinforcement Learning
Lussange, J., Lazarevich, I. A., Bourgeois‐Gironde, S., Palminteri, S., & Gutkin, B. S. (2020) · Computational Economics · doi:10.1007/s10614-020-10038-w · Cited by 90
Metadata
- Authors
- Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q.
- Affiliations
- University of California, San Francisco (US)
- Tsinghua University (CN)
- Southeast University (CN)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2017-03
- Volume
- 28
- Issue
- 3
- Pages
- 653-664
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2344786740
- Cited by
- 822
- References
- 51
- Citations by year
- 2026: 90 · 2025: 90 · 2024: 95 · 2023: 110 · 2022: 109 · 2021: 100 · 2020: 98 · 2019: 76 · 2018: 40 · 2017: 10
- FWCI
- 47.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Project of the National Natural Science Foundation of China (61327902, 61120106003)
- National Science Foundation of Jiangsu Province, China (BK20150650)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- deep reinforcement learning, financial trading, recurrent neural networks, backpropagation through time, gradient vanishing, stock market, commodity futures, trading decision-making, representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.