The DOI matches its Crossref record; 30 of the 33 references it cites were found in a registry or an index.
Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach
Open accessAbstract
In recent years, considerable efforts have been devoted to developing AI techniques for finance research and applications. For instance, AI techniques (e.g., machine learning) can help traders in quantitative trading (QT) by automating two tasks: market condition recognition and trading strategies execution. However, existing methods in QT face challenges such as representing noisy high-frequent financial data and finding the balance between exploration and exploitation of the trading agent with AI techniques. To address the challenges, we propose an adaptive trading model, namely iRDPG, to automatically develop QT strategies by an intelligent trading agent. Our model is enhanced by deep reinforcement learning (DRL) and imitation learning techniques. Specifically, considering the noisy financial data, we formulate the QT process as a Partially Observable Markov Decision Process (POMDP). Also, we introduce imitation learning to leverage classical trading strategies useful to balance between exploration and exploitation. For better simulation, we train our trading agent in the real financial market using minute-frequent data. Experimental results demonstrate that our model can extract robust market features and be adaptive in different markets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
EFFICIENT CAPITAL MARKETS: A REVIEW OF THEORY AND EMPIRICAL WORK*
Malkiel, B. G., & Fama, E. F. (1970) · The Journal of Finance · doi:10.1111/j.1540-6261.1970.tb00518.x · Cited by 24,478
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
(1999) · Cited by 5,147
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149
Sharpe, W. F. (1966) · The Journal of Business · doi:10.1086/294846 · Cited by 3,937
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
An Introduction to Deep Reinforcement Learning
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000071 · Cited by 1,512
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889
Deep Direct Reinforcement Learning for Financial Signal Representation and Trading
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522401 · Cited by 822 · cited 4×
“However, these predefined and handcraft technical indicators are frequently criticized for their poor generalization capacity (Deng et al. 2016).”
BackgroundInfluential(1999) · Choice Reviews Online · doi:10.5860/choice.36-4016 · Cited by 773
Overcoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712
Learning to trade via direct reinforcement
Moody, J. E., & Saffell, M. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935097 · Cited by 586
Gated Feedback Recurrent Neural Networks
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.02367 · Cited by 583
Temporal Relational Ranking for Stock Prediction
Feng, F., He, X., Wang, X., Luo, C., Liu, Y., & Chua, T.-S. (2019) · ACM Transactions on Information Systems · doi:10.1145/3309547 · Cited by 553
Self-citationLeveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Rothörl, T., Lampe, T., & Riedmiller, M. (2017) · arXiv · doi:10.48550/arxiv.1707.08817 · Cited by 478
Efficient Reductions for Imitation Learning
Ross, S., & Bagnell, D. (2018) · doi:10.1184/r1/6554699 · Cited by 366
A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
Jiang, Z., Xu, D., & Liang, J. (2017) · arXiv · doi:10.48550/arxiv.1706.10059 · Cited by 321 · cited 1×
“For instance, Jiang, Xu, and Liang (2017) uses the model-free Deep Deterministic Policy Gradient (DDPG) (Lillicrap et al. 2015) to dynamically optimize cryptocurrency portfolios.”
MethodKim, H.-J., & Shin, K.-S. (2006) · Applied Soft Computing · doi:10.1016/j.asoc.2006.03.004 · Cited by 223
Memory-based control with recurrent neural networks
Heess, N., Hunt, J. J., Lillicrap, T. P., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1512.04455 · Cited by 211
Model-based Deep Reinforcement Learning for Dynamic Portfolio OptimizationPreprint
Yu, P., Sern, L. J., Kulyatin, I., Shi, Z., & Dasgupta, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08740 · Cited by 106
Optimal Asset Allocation using Adaptive Dynamic Programming
Neuneier, R. (1995) · Cited by 74
Reinforcement learning for trading
Moody, J. E., & Saffell, M. (1999) · Cited by 54
Tensor-Based Learning for Predicting Stock Movements
Li, Q., Jiang, L.-L., Li, P., & Chen, H. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9452 · Cited by 52
A Sequential Approach to Market State Modeling and Analysis in Online P2P Lending
Zhao, H., Liu, Q., Zhu, H., Ge, Y., Chen, E., Zhu, Y., & Du, J. (2017) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2017.2665038 · Cited by 47
Self-citationTracking the Dynamics in Crowdfunding
Zhao, H., Zhang, H., Ge, Y., Liu, Q., Chen, E., Li, H., & Wu, L. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098030 · Cited by 30
Self-citationEstimating the Days to Success of Campaigns in Crowdfunding: A Deep Survival Perspective
Jin, B., Zhao, H., Chen, E., Liu, Q., & Ge, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014023 · Cited by 21
Self-citation
Flagged first, then by citations
3 of the 33 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Magdon-Ismail, M., and Atiya, A. F. 2004. Maximum drawdown. Risk Magazine 17(10):99-102.
- Pruitt, G., and Hill, J. R. 2012. Building Winning Trading Systems with Tradestation,+ Website , volume 542. John Wiley & Sons.
Cited by
Sorted by citations
AI in Finance: Challenges, Techniques, and Opportunities
Cao, L. (2022) · ACM Computing Surveys · doi:10.1145/3502289 · Cited by 567
Sahu, S. K., Mokhade, A., & Bokde, N. D. (2023) · Applied Sciences · doi:10.3390/app13031956 · Cited by 238
Deep Reinforcement Learning for Smart Grid Operations: Algorithms, Applications, and Prospects
Li, Y., Yu, C., Shahidehpour, M., Yang, T., Zeng, Z., & Chai, T. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3303358 · Cited by 223
Stock Selection via Spatiotemporal Hypergraph Attention Network: A Learning to Rank Approach
Sawhney, R., Agarwal, S., Wadhwa, A., Derr, T., & Shah, R. R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16127 · Cited by 171
Wang, Z., Huang, B., Tu, S., Zhang, K., & Xu, L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16144 · Cited by 150
Metadata
- Authors
- Liu, Y., Liu, Q., Zhao, H., Pan, Z., & Liu, C.
- Affiliations
- University of Science and Technology of China (CN)
- Tianjin University (CN)
- University of Tennessee at Knoxville (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 02
- Pages
- 2128-2135
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2997874851
- Cited by
- 141
- References
- 33
- Citations by year
- 2026: 10 · 2025: 21 · 2024: 39 · 2023: 26 · 2022: 25 · 2021: 19 · 2020: 1
- FWCI
- 24.10
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- National Natural Science Foundation of China (61922073, 71790594, U1605251, 61672483)
- Youth Innovation Promotion Association
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Social Sciences › Business, Management and Accounting › Management Information Systems › FinTech, Crowdfunding, Digital Finance
- Keywords
- quantitative trading, deep reinforcement learning, imitation learning, partially observable Markov decision processes, exploration-exploitation balance, high-frequency financial data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
