The DOI matches its Crossref record; 179 of the 179 references it cites were found in a registry or an index.
Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
Open accessAbstract
With extensive pretrained knowledge and high-level general capabilities, large language models (LLMs) emerge as a promising avenue to augment reinforcement learning (RL) in aspects, such as multitask learning, sample efficiency, and high-level task planning. In this survey, we provide a comprehensive review of the existing literature in LLM-enhanced RL and summarize its characteristics compared with conventional RL methods, aiming to clarify the research scope and directions for future studies. Utilizing the classical agent-environment interaction paradigm, we propose a structured taxonomy to systematically categorize LLMs' functionalities in RL, including four roles: information processor, reward designer, decision-maker, and generator. For each role, we summarize the methodologies, analyze the specific RL challenges that are mitigated and provide insights into future directions. Finally, the comparative analysis of each role, potential applications, prospective opportunities, and challenges of the LLM-enhanced RL are discussed. By proposing this taxonomy, we aim to provide a framework for researchers to effectively leverage LLMs in the RL field, potentially accelerating RL applications in complex applications, such as robotics, autonomous driving, and energy systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Framework implementations support comprehensive ecosystems: AutoGen enables dynamic response generation, MetaGPT provides shared message pools, CAMEL offers integrated orchestration, CrewAI facilitates adaptation, with reinforcement learning integration enhancing reward redesign, action selection, and policy interpretation [188, 38, 119, 1004, 228, 871, 935, 958, 1273].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [119]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Self-citationProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Research works printed but not linked: 18
- Sutton, 2018, Reinforcement Learning: An Introduction
- Schmid, 2021, Rebel: A general game playing AI, Science, 373, 664
- Mandlekar, 2021, What matters in learning from offline human demonstrations for robot manipulation, arXiv:2108.03298 · arXiv
- Radford, 2018, Improving Language Understanding By Generative Pre-Training
- Devidze, Exploration-guided reward shaping for reinforcement learning under sparse rewards, 35, 5829
- Cho, Hard tasks first: Multi-task reinforcement learning through task scheduling, 1
- Zhang, Efficient multi-task reinforcement learning via selective behavior sharing, 1
- Yuan, 2023, Plan4mc: Skill reinforcement learning and planning for open-world minecraft tasks, arXiv:2303.16563 · arXiv
- Paischer, 2023, History compression via language models in reinforcement learning, arXiv:2205.12258 · arXiv
- 10.1109/ICRA48891.2023.10160591 · DOI · Paper page
- Spiegel, Informing reinforcement learning agents by grounding language to Markov decision processes, 1
- Nair, 2022, R3M: A universal visual representation for robot manipulation, arXiv:2203.12601 · arXiv
- Xie, Text2reward: Reward shaping with language models for reinforcement learning, 1
- Chakraborty, 2023, REBEL: A regularization-based solution for reward overoptimization in robotic reinforcement learning from human feedback, arXiv:2312.14436 · arXiv
- Fu, 2020, D4RL: Datasets for deep data-driven reinforcement learning, arXiv:2004.07219 · arXiv
- Robine, 2023, Transformer-based world models are happy with 100k interactions, arXiv:2303.07109 · arXiv
- Das, State2explanation: Concept-based explanations to benefit agent learning and user understanding, 1
- Slumbers, Leveraging large language models for optimised coordination in textual multi-agent reinforcement learning, 1
Cited by
Sorted by citations
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
LLM-based framework for bearing fault diagnosis
Tao, L., Liu, H., Ning, G., Cao, W., Huang, B., & Lu, C. (2024) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2024.112127 · Cited by 211
Chen, C., Zhao, K., Leng, J., Liu, C., Fan, J., & Zheng, P. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.102982 · Cited by 118
Survey of Different Large Language Model Architectures: Trends, Benchmarks, and Challenges
Shao, M., Basit, A., Karri, R., & Shafique, M. (2024) · IEEE Access · doi:10.1109/access.2024.3482107 · Cited by 130
A Comprehensive Survey of Foundation Models in Medicine
Khan, W., Leem, S., See, K. B., Wong, J. K., Zhang, S., & Fang, R. (2025) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2025.3531360 · Cited by 137
Metadata
- Authors
- Cao, Y., Zhao, H., Cheng, Y., Shu, T., Chen, Y., Liu, G., Liang, G., Zhao, J., Yan, J., & Li, Y.
- Affiliations
- Chinese University of Hong Kong (HK)
- Hong Kong Polytechnic University (HK)
- Chinese University of Hong Kong, Shenzhen (CN)
- Shenzhen University (CN)
- Nanyang Technological University (SG)
- Harbin Institute of Technology (CN)
- University of Electronic Science and Technology of China (CN)
- ORCID
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2025-06
- Volume
- 36
- Issue
- 6
- Pages
- 9737-9757
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W4393763811
- Cited by
- 174
- References
- 176
- Citations by year
- 2026: 87 · 2025: 79
- FWCI
- 25.68
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (72331009, 72171206, 92270105)
- Guangdong Power Grid Company (GDKJXM20231024)
- Shenzhen Key Lab of Crowd Intelligence Empowered Low-Carbon Energy Network (ZDSYS20220606100601002)
- Shenzhen Natural Science Fund (GXWD20231128112434001)
- Shenzhen Institute of Artificial Intelligence and Robotics for Society
- PolyU Direct Grant (P0047700, P0043885, P0051105)
- CUHK Strategic Partnership Award for Research Collaboration (4750467)
- CUHK Direct Grant for Research (4055228)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- large language models, reinforcement learning, reward function design, multi-task learning, sample efficiency, task planning, agent-environment interaction, robotics, autonomous driving, energy systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
