CoChat
Sign Up
Cited by 28Stock Market Forecasting MethodsFinancial Markets and Investment StrategiesFinTech, Crowdfunding, Digital Finance

FinRL-podracer: high performance and scalable deep reinforcement learning for quantitative finance

Open access
Li, Zechu, Liu, Xiao-Yang, Zheng, Jiahao, Wang, Zhaoran, Walid, Anwar, Guo, Jian·2021·Proceedings of the Second ACM International Conference on AI in Finance
DOI 10.1145/3490354.3494413
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning techniques are playing more and more important roles in finance market investment. However, finance quantitative modeling with conventional supervised learning approaches has a number of limitations, including the difficulty in defining appropriate labels, lack of consistency in modeling and trading execution, and lack of modeling the dynamic nature of the finance market. The development of deep reinforcement learning techniques is partially addressing these issues. Unfortunately, the steep learning curve and the difficulty in quick modeling and agile development are impeding finance researchers from using deep reinforcement learning in quantitative trading. In this paper, we propose an RLOps in finance paradigm and present a FinRL-Podracer framework to accelerate the development pipeline of deep reinforcement learning (DRL)-driven trading strategy and to improve both trading performance and training efficiency. FinRL-Podracer is a cloud solution that features high performance and high scalability and promises continuous training, continuous integration, and continuous delivery of DRL-driven trading strategies, facilitating a rapid transformation from algorithmic innovations into a profitable trading strategy. First, we propose a generational evolution mechanism with an ensemble strategy to improve the trading performance of a DRL agent, and schedule the training of a DRL algorithm onto a GPU cloud via multi-level mapping. Then, we carry out the training of DRL components with high-performance optimizations on GPUs. Finally, we evaluate the FinRL-Podracer framework for a stock trend prediction task on an NVIDIA DGX SuperPOD cloud. FinRL-Podracer outperforms three popular DRL libraries Ray RLlib, Stable Baseline 3 and FinRL, i.e., 12% ~ 35% improvements in annual return, 0.1 ~ 0.6 improvements in Sharpe ratio and 3× ~ 7× speed-up in training time. We show the high scalability by training a trading agent in 10 minutes with 80 A100 GPUs, on NASDAQ-100 constituent stocks with minute-level data over 10 years.

AI snapshot

Written from the paper’s full text.

TL;DR

FinRL-Podracer is a cloud-based RLOps framework for deep reinforcement learning trading strategies, combining a generational evolution mechanism with GPU-optimized training. It reports better trading performance and faster training than RLlib, Stable Baseline3, and FinRL on a NASDAQ-100 stock trend prediction task.

Key findings

  • FinRL-Podracer outperforms Ray RLlib, Stable Baseline3, and FinRL by 12%–35% in annual return, 0.1–0.6 in Sharpe ratio, and 3×–7× speed-up in training time.
  • On NASDAQ-100 constituent stocks, FinRL-Podracer achieves cumulative returns of 149.553% (daily) and 362.408% (minute-level), annual returns of 56.431% and 111.549%, Sharpe ratios of 2.12 and 2.42, and max drawdowns of -13.834% and -15.874%.
  • Vanilla FinRL-Podracer, without agent evolution, reaches cumulative returns of 73.546% (daily) and 231.747% (minute-level) and Sharpe ratios of 1.27 and 2.05, supporting the reported benefit of the generational evolution mechanism.
  • The abstract reports training a trading agent in 10 minutes with 80 A100 GPUs on NASDAQ-100 constituent stocks with minute-level data over 10 years.

Methods

  • The stock trend prediction task is modeled as a Markov Decision Process with state features including balance, shares, closing prices, and technical indicators; actions are buy/sell/hold; reward is the change in account value minus transaction cost. Proximal Policy Optimization (PPO) is used in reported results.
  • Data: NASDAQ-100 constituent stocks; daily data from 01/01/2009 to 05/12/2019 for training; minute-level data from 01/01/2016 to 05/12/2019 for training; backtesting from 05/13/2019 to 05/26/2021; initial capital $1,000,000 and transaction cost 0.2%.
  • The framework uses a three-layer architecture—trading task, evolution, and training—with a generational evolution mechanism, evaluator/selector, ensemble training across K pods per agent, multi-level mapping to GPU pods, and hardware-oriented optimizations on an NVIDIA DGX SuperPOD cloud with up to 80 A100 GPUs.

Limitations

  • The paper does not state a dedicated limitations section; it discusses limitations of existing works, such as backtesting having no market impact, limited actions beyond buying/holding/selling, and only one stock type per agent.
  • Evident from the text: evaluation is confined to a single stock trend prediction task on NASDAQ-100 constituent stocks, uses PPO for reported results, and relies on historical backtesting rather than live trading or explicit market-impact simulation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3490354.3494413

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § ABSTRACT · p. 1

    FinRL-Podracer trains a trading agent in 10 minutes with 80 A100 GPUs on NASDAQ-100 constituent stocks with minute-level data over 10 years.

    “We show the high scalability by training a trading agent in 10 minutes with 80 A100 GPUs, on NASDAQ-100 constituent stocks with minute-level data over 10 years.”
    • Also stated in: § 1 INTRODUCTION · p. 2 · boosted
      “We show the training efficiency by obtaining a trading agent in 10 minutes on an NVIDIA DGX SuperPOD cloud [38] with 80 A100 GPUs, for a stock trend prediction task on NASDAQ100 constituent stocks with minute-level data over 10 years.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [38] [38] NVIDIA DGX A100 system reference architecture. 2020. NVIDIA DGX SuperPOD: Scalable infrastructure for AI leadership . NVIDIA Corporation. -

      Not checked in the free run.

  • Findingnone given
    § 4.1 Overview · p. 4

    Existing DRL libraries/packages have no criteria to determine overfitting or underfitting of models during the training process.

    “There is no criteria to determine overfitting or underfitting of models (trading strategies) during the training process.”
    Quote in the paperNot a substantial claim
  • Findingreasoning
    § 4.4 High Performance Training Layer · p. 6

    In our experiments, some financial environments run much slower on GPUs than CPUs.

    “In our experiments, some environments run much slower on GPUs than CPUs.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 1 INTRODUCTION · p. 1

    It is difficult for existing DRL libraries to effectively train a profitable trading agent in a short cycle time.

    “We evaluate the training time of three popular DRL libraries, FinRL [25, 26], RLlib [19] and Stable Baseline3 [9], on NASDAQ-100 constituent stocks with minutelevel data (described in Section 5.2) in Table 1, which shows that it is difficult for them to effectively train a profitable trading agent in a short cycle time.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [25] [25] Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. 2020. FinRL: a deep reinforcement learning library for automated stock trading in quantitative finance. Deep Reinforcement Learning Workshop at NeurIPS (2020). - source

      Not checked in the free run.

    • cites [26] [26] Xiao-Yang Liu, Hongyang Yang, Jiechao Gao, and Christina Dan Wang. 2021. FinRL: Deep reinforcement learning framework to automate trading in quantitative finance. ACM International Conference on AI in Finance (ICAIF) (2021). - source

      Not checked in the free run.

    • cites [19] [19] Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Joseph Gonzalez, Ken Goldberg, and Ion Stoica. 2017. Ray RLLib: a composable and scalable reinforcement learning library. ArXiv abs/1712.09381 (2017). -

      Not checked in the free run.

    • cites [9] [9] [DLR-RM. 2021. Stable-baseline 3. https://github.com/DLR-RM/stable-baselines3.](https://github.com/DLR-RM/stable-baselines3) -

      Not checked in the free run.

  • Findingown data
    § 5.3 Trading Performance · p. 7

    All methods have much better performance on the minute-level dataset than on the daily dataset.

    “all methods have a much better performance on the minute-level dataset than that on the daily dataset.”
    Quote in the paperSubstantial claim
  • May combine two claims. 27
  • Classified as probably not a substantial claim. 18

  • May combine two claims.

    Findingnone given
    § 4.1 Overview · p. 4

    The training process of a trading strategy is sensitive to hyperparameters, which may result in unstable trading performance in backtesting and trading.

    “The training process of a trading strategy is sensitive to hyperparameters , which may result in unstable trading performance in backtesting and trading.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § ABSTRACT · p. 1

    FinRL-Podracer outperforms three popular DRL libraries (Ray RLlib, Stable Baseline 3, and FinRL) with 12%-35% improvements in annual return, 0.1-0.6 improvements in Sharpe ratio, and 3×-7× speed-up in training time.

    “FinRL-Podracer outperforms three popular DRL libraries Ray RLlib , Stable Baseline 3 and FinRL , i.e., 12% ∼ 35% improvements in annual return, 0 . 1 ∼ 0 . 6 improvements in Sharpe ratio and 3 × ∼ 7 × speed-up in training time.”
    • Also stated in: § 1 INTRODUCTION · p. 2 · plain
      “We evaluate the trained agent for one year and show its trading performance outperforms three DRL libraries, FinRL [25, 26], Ray RLlib [19] and Stable Baseline3 [13], i.e., 12% ∼ 35% improvements in annual return, 0 . 1 ∼ 0 . 6 improvements in Sharpe ratio and 3 × ∼ 7 × speed-up in training time.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Trading Performance · p. 7

    FinRL-Podracer achieves the highest cumulative returns of 149.533% and 362.408%, annual returns of 56.431% and 111.549%, and Sharpe ratios of 2.12 and 2.42.

    “our FinRL-Podracer achieves the highest cumulative returns of 149.533% and 362.408%, annual returns of 56.431% and 111.549%, and Sharpe ratios of 2.12 and 2.42”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Trading Performance · p. 7

    FinRL-Podracer achieves Max drawdown -13.834% and -15.874%, which is much lower than other methods.

    “it achieves Max drawdown -13.834% and -15.874%, which is much lower than other methods.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Trading Performance · p. 7

    All DRL agents achieve a better or equal cumulative return than the market on both daily and minute-level datasets.

    “all DRL agents are able to achieve a better or equal performance than the market in cumulative return”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Trading Performance · p. 7

    FinRL-Podracer outperforms other baselines on both datasets in expected return, stability, and Sharpe ratio.

    “our FinRLPodracer outperforms other baselines on both datasets, in terms of expected return, stability, and Sharpe ratio.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3 Trading Performance · p. 7

    Agents achieve Sharpe ratios of 2.42, 2.05, 2.33, 1.82, 1.35 on the minute-level dataset, which are 0.3, 0.78, 1.42, 0.7, and 0.41 higher than on the daily dataset.

    “agents achieve Sharpe ratios of 2.42, 2.05, 2.33, 1.82, 1.35 on the minute-level dataset, which are 0.3, 0.78, 1.42, 0.7, and 0.41 higher than those on the daily dataset.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation
    § 5.4 Training Efficiency · p. 8

    The increase of GPU resources not only makes FinRL-Podracer have fast training but also improves its trading performance over RLlib.

    “the increase of GPU resources not only makes FinRL-Podracer have a fast training, but also improves the trading performance over RLlib [19] .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] [19] Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Joseph Gonzalez, Ken Goldberg, and Ion Stoica. 2017. Ray RLLib: a composable and scalable reinforcement learning library. ArXiv abs/1712.09381 (2017). -

      Not checked in the free run.

  • May combine two claims.

    Findingown data
    § 5.4 Training Efficiency · p. 8

    FinRL-Podracer needs approximately 300s to achieve a cumulative return of 3.5, while RLlib needs 2,200s to achieve the same cumulative return.

    “FinRL-Podracer needs approximately 300s to achieve a cumulative return of 3 . 5, however, RLlib needs 2 , 200s to achieve the same cumulative return.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.4 Training Efficiency · p. 8

    The generalization curves of RLlib with different numbers of GPUs are relatively similar, with no much speed-up.

    “The generalization curves of RLlib with different numbers of GPUs are relatively similar, and we do not observe much speed-up.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 4.4 High Performance Training Layer · p. 5

    A batch mode with multiple independent sub-environments maximizes hardware utilization (either CPUs or GPUs).

    “We propose a batch mode to perform massively parallel simulations, which maximizes the hardware utilization (either CPUs or GPUs).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Market simulation is both computing- and communication-intensive.
    • Rests on an assumption: The optimization of each component is critical to the overall performance.

  • May combine two claims.

    Premisecitation
    § 5.2 Performance Metrics · p. 6

    FinRL and Stable Baseline 3 do not support training on multiple GPUs.

    “both FinRL [25] and Stable Baseline 3 [9] do not support the training on multiple GPUs”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [25] [25] Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. 2020. FinRL: a deep reinforcement learning library for automated stock trading in quantitative finance. Deep Reinforcement Learning Workshop at NeurIPS (2020). - source

      Not checked in the free run.

    • cites [9] [9] [DLR-RM. 2021. Stable-baseline 3. https://github.com/DLR-RM/stable-baselines3.](https://github.com/DLR-RM/stable-baselines3) -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § 4.2 Scalable Evolution Layer · p. 5

    The ensemble training design (concurrently running K pods with same hyperparameters but different random seeds and fusing models) guarantees randomness and stabilizes the learning process of the agent.

    “Such a design, as shown in the ensemble stage of the evolution layer in Fig. 2, guarantees randomness and stabilizes the learning process of the agent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § 1 INTRODUCTION · p. 2

    FinRL-Podracer schedules the training process through a multi-level mapping and employs a generational evolution mechanism with an ensemble strategy; such a design guarantees scalability on a cloud platform.

    “FinRL-Podracer schedules the training process through a multi-level mapping and employs a generational evolution mechanism with an ensemble strategy. Such a design guarantees scalability on a cloud platform.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 INTRODUCTION · p. 2

    An NVIDIA DGX SuperPOD cloud is the most powerful AI infrastructure for enterprise deployments.

    “an NVIDIA DGX SuperPOD cloud [38] that is the most powerful AI infrastructure for enterprise deployments.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38] [38] NVIDIA DGX A100 system reference architecture. 2020. NVIDIA DGX SuperPOD: Scalable infrastructure for AI leadership . NVIDIA Corporation. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning
    § 4.1 Overview · p. 4

    Computing power is critical to effectively explore and exploit large-scale financial data.

    “Computing power is critical to effectively explore and exploit large-scale financial data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.4 High Performance Training Layer · p. 6

    Communication happens once at the end of each epoch, which is a significantly lower frequency of communication.

    “Here, communication happens once at the end of each epoch, which is a significantly lower frequency of communication.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 INTRODUCTION · p. 2

    Distributed DRL frameworks and massively parallel simulations have been recognized as the critical software development for the RLOps paradigm.

    “In recent years, distributed DRL frameworks and massively parallel simulations have been recognized as the critical software development for the RLOps paradigm [12, 19, 20, 27].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [12] [12] Matteo Hessel, Manuel Kroiss, Aidan Clark, Iurii Kemaev, John Quan, Thomas Keck, Fabio Viola, and Hado van Hasselt. 2021. Podracer architectures for scalable reinforcement learning. ArXiv abs/2104.06272 (2021). -

      Not checked in the free run.

    • cites [19] [19] Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Joseph Gonzalez, Ken Goldberg, and Ion Stoica. 2017. Ray RLLib: a composable and scalable reinforcement learning library. ArXiv abs/1712.09381 (2017). -

      Not checked in the free run.

    • cites [20] [20] Jacky Liang, Viktor Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox. 2018. GPU-accelerated robotic simulation for distributed reinforcement learning. In Conf. on Robot Learning (CoRL) . -

      Not checked in the free run.

    • cites [27] [27] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. 2021. Isaac Gym: High performance GPU-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470 (2021). -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Premisereasoning
    § 3 STOCK TREND PREDICTION TASK · p. 3

    In practice, the market simulation, alternative data processing and index analyzing are computationally expensive, therefore a cloud-level solution is critical to a fast iteration of a trading strategy.

    “In practice, the market simulation, alternative data processing and index analyzing are computationally expensive, therefore, a cloud-level solution is critical to a fast iteration of a trading strategy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 4.1 Overview · p. 4

    An effective cloud-level evolution requires a high-quality and scalable scheduling.

    “An effective cloud-level evolution requires a high-quality and scalable scheduling, therefore we schedule a population of parallel agents through a multi-level mapping.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.4 High Performance Training Layer · p. 6

    Sending network parameters rather than gradients is a novel and effective way for learners of each pod to communicate.

    “we propose a novel and effective way for learners of each pod to communicate, i.e., sending the network parameters rather than the gradients.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § 4.4 High Performance Training Layer · p. 6

    Allocating the replay buffer on contiguous GPU memory increases addressing speed and bypasses CPUs for faster data transfer.

    “We allocate the replay buffer on the contiguous memory of GPUs, which increases the addressing speed and bypasses CPUs for faster data transfer.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given
    § 1 INTRODUCTION · p. 1

    The large-scale financial data and fast iteration of trading strategies bring imperative challenges in terms of computing power.

    “However, the large-scale financial data and fast iteration of trading strategies bring imperative challenges in terms of computing power.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 5.2 Performance Metrics · p. 6

    Hyperparameters and computing resources were kept the same to guarantee fair comparisons.

    “We keep hyperparameters and computing resources the same to guarantee fair comparisons”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.1 Overview · p. 4

    Such an evolution-and-training workflow pipelines the development of a trading strategy on a GPU cloud; it enjoys great performance and scalability, promoting fast and flexible development, deployment and production of profitable DRL trading strategies.

    “Such an evolution-and-training workflow pipelines the development of a trading strategy on a GPU cloud. It enjoys great performance and scalability, which promotes fast and flexible development, deployment and production of profitable DRL trading strategies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 1

    For DRL strategy design, existing works studied algorithmic trading on a daily time-frame or hourly time-frame, which is hard to fully capture the dynamics of a highly volatile market.

    “For DRL strategy design, existing works studied algorithmic trading on a daily time-frame [2, 25, 26, 42-45] or hourly time-frame [14], which is hard to fully capture the dynamics of a highly volatile market.”
    Quote in the paperSubstantial claim0 of 8 sources checked
    • cites [2] [2] H. Buehler, L. Gonon, J. Teichmann, and B. Wood. 2019. Deep hedging. Quantitative Finance 19 (2019), 1271 - 1291. - source

      Not checked in the free run.

    • cites [25] [25] Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. 2020. FinRL: a deep reinforcement learning library for automated stock trading in quantitative finance. Deep Reinforcement Learning Workshop at NeurIPS (2020). - source

      Not checked in the free run.

    • cites [26] [26] Xiao-Yang Liu, Hongyang Yang, Jiechao Gao, and Christina Dan Wang. 2021. FinRL: Deep reinforcement learning framework to automate trading in quantitative finance. ACM International Conference on AI in Finance (ICAIF) (2021). - source

      Not checked in the free run.

    • cites [42] [42] Jia Wu, Chen Wang, Lidong Xiong, and Hongyong Sun. 2019. Quantitative trading on stock market based on deep reinforcement learning. In 2019 International Joint Conference on Neural Networks (IJCNN) . 1-8. - source

      Not checked in the free run.

    • cites [43] [43] Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. 2018. Practical deep reinforcement learning approach for stock trading. NeurIPS Workshop (2018). -

      Not checked in the free run.

    • cites [44] [44] Hongyang Yang, Xiao-Yang Liu, Shan Zhong, and Anwar Walid. 2020. Deep reinforcement learning for automated stock trading: An ensemble strategy. ACM International Conference on AI in Finance (ICAIF) (2020). - source

      Not checked in the free run.

    • cites [45] [45] Zihao Zhang, Stefan Zohren, and Stephen Roberts. 2020. Deep reinforcement learning for trading. The Journal of Financial Data Science 2(2) (2020), 25-40.

      Not checked in the free run.

    • cites [14] [14] Zhengyao Jiang, Dixing Xu, and Jinjun Liang. 2017. A deep reinforcement learning framework for the financial portfolio management problem. ArXiv abs/1706.10059 (2017). -

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § ABSTRACT · p. 1

    FinRL-Podracer is a cloud solution that features high performance and high scalability and promises continuous training, continuous integration, and continuous delivery of DRL-driven trading strategies.

    “FinRL-Podracer is a cloud solution that features high performance and high scalability and promises continuous training , continuous integration , and continuous delivery of DRL-driven trading strategies, facilitating a rapid transformation from algorithmic innovations into a profitable trading strategy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § 1 INTRODUCTION · p. 2

    FinRL-Podracer realizes hardware-oriented optimizations (parallelism encapsulation, GPU acceleration, storage optimization), thus achieving high performance.

    “At a low level, FinRL-Podracer realizes hardware-oriented optimizations, including parallelism encapsulation, GPU acceleration, and storage optimization, thus achieving high performance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning
    § 1 INTRODUCTION · p. 1

    The RLOps in finance paradigm removes the latency between stages and results in a compact software stack.

    “Such a paradigm aims to help quantitative traders develop an end-to-end trading strategy with a high degree of automation, which removes the latency between stages and results in a compact software stack.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.3 Packaging Worker-Learner into Pod · p. 5

    The pod replication reflects strong parallelism and is highly scalable since a GPU cloud can support a large number of pods.

    “The pod replication reflects strong parallelism, and it is highly scalable since a GPU cloud can support a large number of pods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given
    § 4.4 High Performance Training Layer · p. 6

    Model size in DRL is not comparable to that in other deep learning fields, so model parameters are amenable to communication.

    “The parameter of the models is amenable to communication because model size in DRL is not comparable to that in other deep learning fields.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 4.4 High Performance Training Layer · p. 5

    Financial tasks are highly suitable to GPUs because financial simulations involve simple arithmetics.

    “Environments of financial tasks are highly suitable to GPUs because financial simulations involve”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.2 Scalable Evolution Layer · p. 5

    The synergy of the evaluator and selector enables FinRL-Podracer to naturally select the best agent for the future generation and eliminates the potential negative impact from poorly evolved agents, effectively improving the stability and efficiency of the training.

    “The synergy of the evaluator and selector enables FinRL-Podracer to naturally select the best agent for the future generation and eliminates the potential negative impact from poorly evolved agents, which effectively improves the stability and efficiency of the training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 3 STOCK TREND PREDICTION TASK · p. 3

    The supercomputing power is necessary to achieve the massively parallel simulations for an STP task.

    “The supercomputing power is necessary to achieve the massively parallel simulations for an STP task.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.1 Overview · p. 4

    We locally optimize each component (explorer, replay buffer, learner) through parallelism encapsulation, GPU acceleration, efficient parameter fusion, and storage optimization; thus we maximize hardware usage and minimize communication overhead, allowing each component to be efficiently executed on a GPU cloud.

    “We locally optimize each component (a container within a pod), namely explorer, replay buffer, and learner, through parallelism encapsulation, GPU acceleration, efficient parameter fusion, and storage optimization. Thus, we maximize the hardware usage and minimize the communication overhead, which allows each component to be efficiently executed on a GPU cloud.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 6 DISCUSSION AND CONCLUSION · p. 8

    FinRL-Podracer provides a highly automated development pipeline of DRL-driven trading strategies on a GPU cloud.

    “FinRL-Podracer provides a highly automated development pipeline of DRL-driven trading strategies on a GPU cloud”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 6 DISCUSSION AND CONCLUSION · p. 8

    Training on the cloud makes DRL agents adapt to much more complex financial simulations and neural networks, thus achieving wider DRL applications to various financial tasks.

    “training on the cloud makes DRL agents adapt to much more complex financial simulations and neural networks, thus achieving wider DRL applications to various financial tasks”
    Quote in the paperNot a substantial claim
37 of 37

  • The paper relies on this without support.

    Market simulation is both computing- and communication-intensive.

    “The market simulation is both computing- and communication-intensive.”

  • The paper relies on this without support.

    The optimization of each component is critical to the overall performance.

    “The optimization of each component is critical to the overall performance.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Minute-level training periods conflict

    Major

    Table 1 says its minute-level data were used for training from 01/01/2009, while Section 5.2 says minute-level training began on 01/01/2016.

  • Several daily-data agents underperform the market

    Major

    The text says all DRL agents match or exceed the market in cumulative return, but Table 3 gives FinRL a daily-data return of 78.255%, below the QQQ benchmark's 89.614%.

  • Daily cumulative return differs from Table 3

    Minor

    The prose reports a daily cumulative return of 149.533% for FinRL-Podracer, whereas Table 3 reports 149.553%.

3 of 3

Cites

Flagged first, then by citations

4 of the 34 references couldn’t be matched to a known paper.

Research works printed but not linked: 11

  1. Sridhar Alla and Suman Kalyan Adari . 2021. What Is MLOps? In Beginning MLOps with MLFlow . Springer , 79--124. Sridhar Alla and Suman Kalyan Adari. 2021. What Is MLOps? In Beginning MLOps with MLFlow. Springer, 79--124.
  2. Google Cloud . 2020. MLOps: Continuous delivery and automation pipelines in machine learning. https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning#mlops_level_0_manual_process. Google Cloud , Jan. 07 , 2020 . Google Cloud. 2020. MLOps: Continuous delivery and automation pipelines in machine learning. https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning#mlops_level_0_manual_process. Google Cloud, Jan. 07, 2020.
  3. Jacomo Corbo , Oliver Flemin , and Nicolas Hohn . 2021 . It's time for businesses to chart a course for reinforcement learning. https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/its-time-for-businesses-to-chart-a-course-for-reinforcement-learning. McKinsey Analytics , Apirl. 01 , 2021 . Jacomo Corbo, Oliver Flemin, and Nicolas Hohn. 2021. It's time for businesses to chart a course for reinforcement learning. https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/its-time-for-businesses-to-chart-a-course-for-reinforcement-learning. McKinsey Analytics, Apirl. 01, 2021.
  4. Lasse Espeholt , Raphaël Marinier , Piotr Stanczyk , Ke Wang , and Marcin Michalski . 2020 . SEED RL: scalable and efficient deep-RL with accelerated central inference . In Proceedings of the International Conference on Learning Representations (ICLR). Lasse Espeholt, Raphaël Marinier, Piotr Stanczyk, Ke Wang, and Marcin Michalski. 2020. SEED RL: scalable and efficient deep-RL with accelerated central inference. In Proceedings of the International Conference on Learning Representations (ICLR).
  5. Lasse Espeholt , Hubert Soyer , Remi Munos , Karen Simonyan , Volodymir Mnih , Tom Ward , Yotam Doron , Vlad Firoiu , Tim Harley , Iain Dunning , Shane Legg , and Koray Kavukcuoglu . 2018 . IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures . In Proceedings of the International Conference on Machine Learning (ICML). Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Volodymir Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu. 2018. IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures. In Proceedings of the International Conference on Machine Learning (ICML).
  6. Marko Kolanovic and Rajesh T. Krishnamachari . 2017. Big data and AI strategies: machine learning and alternative data approach to investing. https://www.cognitivefinance.ai/single-post/big-data-and-ai-strategies. J.P . Morgan Securities LLC , May . 18, 2017 . Marko Kolanovic and Rajesh T. Krishnamachari. 2017. Big data and AI strategies: machine learning and alternative data approach to investing. https://www.cognitivefinance.ai/single-post/big-data-and-ai-strategies. J.P. Morgan Securities LLC, May. 18, 2017.
  7. Xinyi Li , Yinchuan Li , Yuancheng Zhan , and Xiao-Yang Liu . 2019 . Optimistic bull or pessimistic bear: Adaptive deep reinforcement learning for stock portfolio allocation . ICML Workshop on Applications and Infrastructure for Multi-Agent Learning (2019). Xinyi Li, Yinchuan Li, Yuancheng Zhan, and Xiao-Yang Liu. 2019. Optimistic bull or pessimistic bear: Adaptive deep reinforcement learning for stock portfolio allocation. ICML Workshop on Applications and Infrastructure for Multi-Agent Learning (2019).
  8. Jacky Liang , Viktor Makoviychuk , A. Handa , N. Chentanez , M. Macklin , and D. Fox . 2018. GPU-accelerated robotic simulation for distributed reinforcement learning . In Conf. on Robot Learning (CoRL). Jacky Liang, Viktor Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox. 2018. GPU-accelerated robotic simulation for distributed reinforcement learning. In Conf. on Robot Learning (CoRL).
  9. Xiao-Yang Liu , Zechu Li , Zhuoran Yang , Jiahao Zheng , Zhaoran Wang , Anwar Walid , Jiang Guo , and Michael Jordan . 2021 . ElegantRL-Podracer: Scalable and elastic library for cloud-native deep reinforcement learning . Deep Reinforcement Learning Workshop at NeurIPS (2021). Xiao-Yang Liu, Zechu Li, Zhuoran Yang, Jiahao Zheng, Zhaoran Wang, Anwar Walid, Jiang Guo, and Michael Jordan. 2021. ElegantRL-Podracer: Scalable and elastic library for cloud-native deep reinforcement learning. Deep Reinforcement Learning Workshop at NeurIPS (2021).
  10. David Silver , Aja Huang , Chris J. Maddison , Arthur Guez , Laurent Sifre , George van den Driessche , Julian Schrittwieser , Ioannis Antonoglou , Veda Panneershelvam , Marc Lanctot , Sander Dieleman , Dominik Grewe , John Nham , Nal Kalchbrenner , Ilya Sutskever , Timothy Lillicrap , Madeleine Leach , Koray Kavukcuoglu , Thore Graepel , and Demis Hassabis . 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529(7587) ( 2016 ), 484--489. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529(7587) (2016), 484--489.
  11. NVIDIA DGX A100 system reference architecture. 2020. NVIDIA DGX SuperPOD: Scalable infrastructure for AI leadership . NVIDIA Corporation . NVIDIA DGX A100 system reference architecture. 2020. NVIDIA DGX SuperPOD: Scalable infrastructure for AI leadership. NVIDIA Corporation.

Cited by

Sorted by citations

Related

Authors
Li, Z., Liu, X.-Y., Zheng, J., Wang, Z., Walid, A., & Guo, J.
Affiliations
  • Columbia University (US)
  • Shenzhen Institutes of Advanced Technology (CN)
  • Northwestern University (US)
  • IDEA Research
ORCID
Venue
Proceedings of the Second ACM International Conference on AI in Finance
Publisher
ACM
Published
2021-11-03
Pages
1-9
DOI
10.1145/3490354.3494413
OpenAlex
W3213740880
Semantic Scholar
17390797566679c8df66a77b9ca30737f0f1a117
Cited by
28
References
34
Citations by year
2026: 3 · 2025: 9 · 2024: 5 · 2023: 5 · 2022: 3 · 2021: 3
FWCI
5.82
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM International Conference on AI in Finance (ICAIF) (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
  • Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
  • Social Sciences › Business, Management and Accounting › Management Information Systems › FinTech, Crowdfunding, Digital Finance
Keywords
deep reinforcement learning, quantitative trading, ensemble methods, stock trend prediction, continuous integration and delivery, training efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review