The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Goal-Conditioned Reinforcement Learning: Problems and Solutions
Open accessAbstract
Goal-conditioned reinforcement learning (GCRL), related to a set of complex RL problems, trains an agent to achieve different goals under particular scenarios. Compared to the standard RL solutions that learn a policy solely depending on the states or observations, GCRL additionally requires the agent to make decisions according to different goals. In this survey, we provide a comprehensive overview of the challenges and algorithms for GCRL. Firstly, we answer what the basic problems are studied in this field. Then, we explain how goals are represented and present how existing solutions are designed from different points of view. Finally, we make the conclusion and discuss potential future prospects that recent researches focus on.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“UBCL is most accurately described as a form of Goal-Conditioned RL (GCRL) [42]: the agent receives a target point in a bounded, interpretable behavior space and is rewarded for reducing its distance to that point (Eq. 1).”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355 · cited 10×
“Different from standard RL, GCRL augments the observation with an additional goal that the agent is required to achieve when making a decision in an episode [Schaul et al., 2015; Andrychowicz et al., 2017].”
BackgroundMethodInfluentialDecision Transformer: Reinforcement Learning via Sequence ModelingPreprint
Chen, L.-L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01345 · Cited by 849 · cited 1×
“Recently, Chen et al. [ 2021 ] proposed decision transformer (DT) to solve offline RL problems, which treats RL as a sequence modeling problem by generating actions from offline datasets.”
BackgroundOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · cited 3×
“Researchers used to evaluate their image algorithms on simulation environments, as in Warde-Farley et al. ; Lee et al. ; Chane-Sane et al. [2019; 2020; 2021], and also in real-world robotics applications [Nair et al. , 2018b; Nair et al. , 2020b].”
BackgroundMethodUniversal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · cited 1×
“…the robotics benchmarking tasks discussed in Section 3.1 with image state / goal space instead; moreover, DeepMind Lab [Beattie et al. , 2016], Deepmind Control Suite (DMC) [Tassa et al. , 2020], Atari games [Brockman et al. , 2016] and Minecraft [Johnson et al. , 2016] are also popular testbeds.”
MethodInfluentialDiversity is All You Need: Learning Skills without a Reward Function
Eysenbach, B., Gupta, A., Ibarz, J., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1802.06070 · Cited by 480 · cited 2×
“4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [Eysenbach et al., 2019a; Sharma et al., 2020].”
BackgroundInfluentialVisual Reinforcement Learning with Imagined GoalsPreprint
Nair, A., Pong, V. H., Dalal, M., Bahl, S., Lin, S., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04742 · Cited by 380 · cited 4×
“[2019; 2020; 2021], and also real-world robotics applications [Nair et al., 2018b; Nair et al., 2020b].”
BackgroundMethodInfluentialMulti-Goal Reinforcement Learning: Challenging Robotics Environments and Request for ResearchPreprint
Plappert, M., Andrychowicz, M., Ray, A., McGrew, B., Baker, B., Powell, G., Schneider, J., Tobin, J., Chociej, M., Welinder, P., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09464 · Cited by 372
Research works printed but not linked: 5
- [21]21. Yevgen Chebotar, Karol Hausman, et al. Actionable models: Unsupervised offline reinforce- ment learning of robotic skills. ICML, 2021.
- [42]42. Dibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu, Coline Devin, Benjamin Eysenbach, and Sergey Levine. Learning to reach goals via iterated super- vised learning. ICLR, 2021.
- [76]76. Vitchyr H Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. Skew- fit: State-covering self-supervised reinforcement learning. ICML, 2020.
- [83]83. Schaul et al., 2015] Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function ap- proximators. ICML, 2015.
- [91]91. Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Universal plan- ning networks: Learning generalizable representations for visuomotor control. ICML, 2018.
Cited by
Sorted by citations
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
“Each individual goal g ∼ G can be differentiated by its reward function r ( s t , a t , g ) , hence the objective for GCRL becomes maximizing the expected return over the distribution of goals: J ( π ) := IE ( s t ,a t ) ∼ μ π ,g ∼G [ (cid:5) t γ t r ( s, a, g )] [15].”
BackgroundA survey on model-based reinforcement learning
Luo, F.-M., Xu, T., Lai, H., Chen, X., Zhang, W., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3696-5 · Cited by 205
“Goal-conditioned reinforcement learning (GCRL), also named as goal-oriented RL [Liu et al., 2022], deals with the tasks where agents are expected to achieve different goals [Pitis et al.”
BackgroundSelf-citationWang, J., Qin, Y., Wu, Q., Zeng, D., Gao, X., Wang, Q., Li, Z., Ni, Y., Li, H., Zhang, P., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03724-5 · Cited by 56
Large sequence models for sequential decision-making: a survey
Wen, M., Lin, R., Wang, H., Yang, Y., Wen, Y., Mai, L., Wang, Y.-X., Zhang, H., & Zhang, W. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2689-5 · Cited by 50
“Goal-condition RL [84–86] learns a general policy function to finish a series of simple tasks, for instance, to reach di ff erent goal states.”
BackgroundSelf-citationGrasp Multiple Objects With One Hand
Li, Y., Liu, B., Geng, Y., Li, P., Yang, Y., Zhu, Y., Liu, T., & Huang, S. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3374190 · Cited by 38
“Instead, the entire length and sides of the fingers must be engaged [3], [9], necessitating a carefully configured grasping pose for effective force closure on each object while avoiding collisions.”
Metadata
- Authors
- Liu, M., Zhu, M., & Zhang, W.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
- Published
- 2022-07
- Pages
- 5502-5511
- OpenAlex
- W4285601030
- Cited by
- 115
- References
- 63
- Citations by year
- 2026: 14 · 2025: 47 · 2024: 28 · 2023: 24 · 2022: 2
- FWCI
- 13.96
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076161, 2021SHZDZX0102)
- Shanghai Jiao Tong University (2021SHZDZX0102)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- goal-conditioned reinforcement learning, goal representation, goal-conditioned policies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
