Explainable deep reinforcement learning for portfolio management: an empirical approach
Abstract
Deep reinforcement learning (DRL) has been widely studied in the portfolio management task. However, it is challenging to understand a DRL-based trading strategy because of the black-box nature of deep neural networks. In this paper, we propose an empirical approach to explain the strategies of DRL agents for the portfolio management task. First, we use a linear model in hindsight as the reference model, which finds the best portfolio weights by assuming knowing actual stock returns in foresight. In particular, we use the coefficients of a linear model in hindsight as the reference feature weights. Secondly, for DRL agents, we use integrated gradients to define the feature weights, which are the coefficients between reward and features under a linear regression model. Thirdly, we study the prediction power in two cases, single-step prediction and multi-step prediction. In particular, we quantify the prediction power by calculating the linear correlations between the feature weights of a DRL agent and the reference feature weights, and similarly for machine learning methods. Finally, we evaluate a portfolio management task on Dow Jones 30 constituent stocks during 01/01/2009 to 09/01/2021. Our approach empirically reveals that a DRL agent exhibits a stronger multi-step prediction power than machine learning methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Explainable DRL for portfolio management and DRL ensemble strategy for stock trading are implemented in FinRL.
“Explainable DRL for portfolio management [17] and DRL ensemble strategy for stock trading [50, 51] are also implemented.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [17]
Cites
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
The Capital Asset Pricing Model: Theory and Evidence
Fama, E. F., & French, K. R. (2004) · The Journal of Economic Perspectives · doi:10.1257/0895330042162430 · Cited by 2,080 · see citing works
Explainability in deep reinforcement learning
Heuillet, A., Couthouis, F., & Díaz-Rodríguez, N. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106685 · Cited by 337 · see citing works
Multi-Period Trading via Convex Optimization
Boyd, S., DIAMOND, S. E., Koh, K., et al. (2017) · Foundations and Trends® in Optimization · doi:10.1561/2400000023 · Cited by 100 · see citing works
Explainable Reinforcement Learning through a Causal Lens
Madumal, P., Miller, T., Sonenberg, L., & Vetere, F. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5631 · Cited by 58 · see citing works
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Chen, Q., Zhang, R., Yang, L., Xiao, B., & Wang, C. (2025) · doi:10.2139/ssrn.3737859 · Cited by 51 · cited 2× · see citing works
“We use a similar setting as in the open-source FinRL library [12][13].”
MethodSelf-citationFinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021) · doi:10.2139/ssrn.3955949 · Cited by 46 · see citing works
Self-citationFinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance
Li, Z., Liu, X.-Y., Zheng, J., Wang, Z., Walid, A., & Guo, J. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494413 · Cited by 28 · see citing works
Self-citationSanity Checks for Saliency Metrics
Tomsett, R., Harborne, D., Chakraborty, S., Gurram, P., & Preece, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6064 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. D. (2021) · Proceedings of the Second ACM International Conference on AI in Finance · doi:10.1145/3490354.3494366 · Cited by 87 · see citing works
“Explainable DRL for portfolio management [17] and DRL ensemble strategy for stock trading [50, 51] are also implemented.”
MethodSelf-citationFinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021) · doi:10.2139/ssrn.3955949 · Cited by 46 · see citing works
Self-citation
- Authors
- Guan, M., & Liu, X.-Y.
- Affiliations
- Columbia University (US)
- ORCID
- Liu, Xiao-Yang
- Venue
- Proceedings of the Second ACM International Conference on AI in Finance
- Publisher
- ACM
- Published
- 2021-11-03
- Pages
- 1-9
- OpenAlex
- W4285333950
- Semantic Scholar
- 467ff83987087e6abc1696720d03df9c75d6ea6e
- Cited by
- 28
- References
- 12
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 11 · 2023: 2 · 2022: 1 · 2021: 1
- FWCI
- 1.72
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on AI in Finance (ICAIF) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Decision Sciences › Management Science and Operations Research › Risk and Portfolio Optimization
- Keywords
- portfolio management, integrated gradients, feature attribution, multi-step forecasting, black-box interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
