Cited by 176Reinforcement Learning in RoboticsAI-based Problem Solving and PlanningArtificial Intelligence Applications

Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods

Open access
Cao, Yuji, Zhao, Huan, Cheng, Yuheng, Shu, Ting, Chen, Yue, Liu, Guolong, Liang, Gaoqi, Zhao, Junhua +2 more·2025·IEEE Transactions on Neural Networks and Learning Systems

Abstract

With extensive pretrained knowledge and high-level general capabilities, large language models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in aspects, such as multitask learning, sample efficiency, and high-level task planning. In this survey, we provide a comprehensive review of the existing literature in LLM-enhanced RL and summarize its characteristics compared with conventional RL methods, aiming to clarify the research scope and directions for future studies. Utilizing the classical agent-environment interaction paradigm, we propose a structured taxonomy to systematically categorize LLMs' functionalities in RL, including four roles: information processor, reward designer, decision-maker, and generator. For each role, we summarize the methodologies, analyze the specific RL challenges that are mitigated and provide insights into future directions. Finally, the comparative analysis of each role, potential applications, prospective opportunities, and challenges of the LLM-enhanced RL are discussed. By proposing this taxonomy, we aim to provide a framework for researchers to effectively leverage LLMs in the RL field, potentially accelerating RL applications in complex applications, such as robotics, autonomous driving, and energy systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tnnls.2024.3497992

Cites

175 of 175 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Framework implementations support comprehensive ecosystems: AutoGen enables dynamic response generation, MetaGPT provides shared message pools, CAMEL offers integrated orchestration, CrewAI facilitates adaptation, with reinforcement learning integration enhancing reward redesign, action selection, and policy interpretation.

    “Framework implementations support comprehensive ecosystems: AutoGen enables dynamic response generation, MetaGPT provides shared message pools, CAMEL offers integrated orchestration, CrewAI facilitates adaptation, with reinforcement learning integration enhancing reward redesign, action selection, and policy interpretation [188, 38, 119, 1004, 228, 871, 935, 958, 1273].”

    From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [119]

    Split into 2 atoms — 1 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 176 references couldn’t be matched to a known paper.

Research works printed but not linked: 17

  1. R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018.
  2. N. B. Schmid, A. Botev, A. Hennig, A. Lerer, Q. Wu, D. Yarats, J. Foerster, T. Rocktäschel et al., "Rebel: A general game playing ai," Science, vol. 373, no. 6556, pp. 664-670, 2021.
  3. J. Luketina, N. Nardelli, G. Farquhar, J. Foerster, J. Andreas, E. Grefenstette, S. Whiteson, and T. Rocktäschel, "A survey of re- inforcement learning informed by natural language," arXiv preprint arXiv:1906.03926, 2019. · arXiv
  4. A. Radford and K. Narasimhan, "Improving language understanding by generative pre-training," 2018. [Online]. Available: https://api. semanticscholar.org/CorpusID:49313245
  5. J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, "Code as policies: Language model programs for em- bodied control," in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 9493-9500.
  6. R. Devidze, P. Kamalaruban, and A. Singla, "Exploration-guided reward shaping for reinforcement learning under sparse rewards," Advances in Neural Information Processing Systems, vol. 35, pp. 5829- 5842, 2022.
  7. M. Cho, J. Park, S. Lee, and Y. Sung, "Hard tasks first: Multi- task reinforcement learning through task scheduling," in Forty-first International Conference on Machine Learning, 2024. [Online]. Available: https://openreview.net/forum?id=haUOhXo70o
  8. G. Zhang, A. Jain, I. Hwang, S.-H. Sun, and J. J. Lim, "Efficient multi-task reinforcement learning via selective behavior sharing," 2024. [Online]. Available: https://openreview.net/forum?id=LYGHdwyXUb
  9. H. Yuan, C. Zhang, H. Wang, F. Xie, P. Cai, H. Dong, and Z. Lu, "Plan4mc: Skill reinforcement learning and planning for open-world minecraft tasks," arXiv preprint arXiv:2303.16563, 2023. · arXiv
  10. A. Vaswani, N. M. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention is all you need," in Neural Information Processing Systems, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:13756489
  11. B. A. Spiegel, Z. Yang, W. Jurayj, B. Bachmann, S. Tellex, and G. Konidaris, "Informing reinforcement learning agents by grounding language to markov decision processes," in Workshop on Training Agents with Foundation Models at RLC 2024, 2024. [Online]. Available: https://openreview.net/forum?id=uFm9e4Ly26
  12. S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta, "R3m: A universal visual representation for robot manipulation," 2022. [Online]. Available: https://arxiv.org/abs/2203.12601 · arXiv
  13. T. Xie, S. Zhao, C. H. Wu, Y. Liu, Q. Luo, V. Zhong, Y. Yang, and T. Yu, "Text2reward: Reward shaping with language models for reinforcement learning," in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=tUM39YTRxH
  14. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Eds., vol. 202. PMLR, 23-29 Jul 2023, pp. 10 835-10 866.
  15. S. Chakraborty, A. Bhaskar, A. Singh, P. Tokekar, D. Manocha, and A. S. Bedi, "Rebel: A regularization-based solution for reward overoptimization in reinforcement learning from human feedback," arXiv preprint arXiv:2312.14436, 2023. · arXiv
  16. J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine, "D4rl: Datasets for deep data-driven reinforcement learning," arXiv preprint arXiv:2004.07219, 2020. · arXiv
  17. D. Das, S. Chernova, and B. Kim, "State2explanation: Concept-based explanations to benefit agent learning and user understanding," in Thirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id= xGz0wAIJrS

Cited by

Sorted by citations

Authors
Cao, Y., Zhao, H., Cheng, Y., Shu, T., Chen, Y., Liu, G., Liang, G., Zhao, J., Yan, J., & Li, Y.
Affiliations
  • Chinese University of Hong Kong (HK)
  • Hong Kong Polytechnic University (HK)
  • Chinese University of Hong Kong, Shenzhen (CN)
  • Shenzhen University (CN)
  • Nanyang Technological University (SG)
  • Harbin Institute of Technology (CN)
  • University of Electronic Science and Technology of China (CN)
Venue
IEEE Transactions on Neural Networks and Learning Systems
Published
2025-06
Volume
36
Issue
6
Pages
9737-9757
ISSN
  • 2162-237X (ISSN-L)
  • 2162-2388
OpenAlex
W4393763811
Cited by
174
References
176
Citations by year
2026: 87 · 2025: 79 · 2024: 8
FWCI
25.68
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DR-NTU (Nanyang Technological University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (72331009, 72171206, 92270105)
  • Guangdong Power Grid Company (GDKJXM20231024)
  • Shenzhen Key Lab of Crowd Intelligence Empowered Low-Carbon Energy Network (ZDSYS20220606100601002)
  • Shenzhen Natural Science Fund (GXWD20231128112434001)
  • Shenzhen Institute of Artificial Intelligence and Robotics for Society
  • PolyU Direct Grant (P0047700, P0043885, P0051105)
  • CUHK Strategic Partnership Award for Research Collaboration (4750467)
  • CUHK Direct Grant for Research (4055228)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
large language models, reinforcement learning, reward function design, multi-task learning, sample efficiency, task planning, agent-environment interaction, robotics, autonomous driving, energy systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review