The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Goal-Conditioned Reinforcement Learning: Problems and Solutions
Open accessAbstract
Goal-conditioned reinforcement learning (GCRL), related to a set of complex RL problems, trains an agent to achieve different goals under particular scenarios. Compared to the standard RL solutions that learn a policy solely depending on the states or observations, GCRL additionally requires the agent to make decisions according to different goals. In this survey, we provide a comprehensive overview of the challenges and algorithms for GCRL. Firstly, we answer what the basic problems are studied in this field. Then, we explain how goals are represented and present how existing solutions are designed from different points of view. Finally, we make the conclusion and discuss potential future prospects that recent researches focus on.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“UBCL is most accurately described as a form of Goal-Conditioned RL (GCRL) [42]: the agent receives a target point in a bounded, interpretable behavior space and is rewarded for reducing its distance to that point (Eq. 1).”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355 · cited 10×
“Different from standard RL, GCRL augments the observation with an additional goal that the agent is required to achieve when making a decision in an episode [Schaul et al., 2015; Andrychowicz et al., 2017].”
BackgroundMethodInfluentialDecision Transformer: Reinforcement Learning via Sequence ModelingPreprint
Chen, L.-L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01345 · Cited by 849 · cited 1×
“Recently, Chen et al. [ 2021 ] proposed decision transformer (DT) to solve offline RL problems, which treats RL as a sequence modeling problem by generating actions from offline datasets.”
BackgroundOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · cited 3×
“Researchers used to evaluate their image algorithms on simulation environments, as in Warde-Farley et al. ; Lee et al. ; Chane-Sane et al. [2019; 2020; 2021], and also in real-world robotics applications [Nair et al. , 2018b; Nair et al. , 2020b].”
BackgroundMethodUniversal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · cited 1×
“…the robotics benchmarking tasks discussed in Section 3.1 with image state / goal space instead; moreover, DeepMind Lab [Beattie et al. , 2016], Deepmind Control Suite (DMC) [Tassa et al. , 2020], Atari games [Brockman et al. , 2016] and Minecraft [Johnson et al. , 2016] are also popular testbeds.”
MethodInfluentialDiversity is All You Need: Learning Skills without a Reward Function
Eysenbach, B., Gupta, A., Ibarz, J., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1802.06070 · Cited by 480 · cited 2×
“4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [Eysenbach et al., 2019a; Sharma et al., 2020].”
BackgroundInfluentialVisual Reinforcement Learning with Imagined GoalsPreprint
Nair, A., Pong, V. H., Dalal, M., Bahl, S., Lin, S., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04742 · Cited by 380 · cited 4×
“[2019; 2020; 2021], and also real-world robotics applications [Nair et al., 2018b; Nair et al., 2020b].”
BackgroundMethodInfluentialMulti-Goal Reinforcement Learning: Challenging Robotics Environments and Request for ResearchPreprint
Plappert, M., Andrychowicz, M., Ray, A., McGrew, B., Baker, B., Powell, G., Schneider, J., Tobin, J., Chociej, M., Welinder, P., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09464 · Cited by 372
Research works printed but not linked: 5
- [21]21. Yevgen Chebotar, Karol Hausman, et al. Actionable models: Unsupervised offline reinforce- ment learning of robotic skills. ICML, 2021.
- [42]42. Dibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu, Coline Devin, Benjamin Eysenbach, and Sergey Levine. Learning to reach goals via iterated super- vised learning. ICLR, 2021.
- [76]76. Vitchyr H Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. Skew- fit: State-covering self-supervised reinforcement learning. ICML, 2020.
- [83]83. Schaul et al., 2015] Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function ap- proximators. ICML, 2015.
- [91]91. Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Universal plan- ning networks: Learning generalizable representations for visuomotor control. ICML, 2018.
Cited by
Sorted by citations
Items 101–116 of 116
Learning goal-oriented agents with limited supervision
Mezghani, L. (2023) · theses.fr (ABES) · doi:10.70675/81301ccfz41d1z457cz8afcz7049fd4699c5
Towards better efficiency and generalization in imitation learning : a causal perspective
Jabri, M. K. (2024) · theses.fr (ABES) · doi:10.70675/be781be3zd6ebz4272zadf1zd906cde4da09
Learning to Grasp in Robotics with Quality-Diversity Generated Data
Huber, J. B. (2024) · theses.fr (ABES) · doi:10.70675/b98f392dz4df9z4646z8b34z59f447564543
Automatic state representation and goal selection in unsupervised reinforcement learning
Castanet, N. (2025) · theses.fr (ABES) · doi:10.70675/a7e98d33z1351z4cfczb78fz8814c0d72ffe
Simulation Of Diverse Human Play-Styles In Video Games : A Reinforcement Learning Approach
de Woillemont, P. L. P. (2023) · theses.fr (ABES) · doi:10.70675/51b3c6d7z2e51z47fcz8d7azf1106953ba36
From learning-based identification to model-based control of robotic systems
Israilov, S. (2024) · theses.fr (ABES) · doi:10.70675/283cad75z522dz4a01z86b5za3f3b57587c1
Automatic Symbolic Goal Abstraction via Reachability Analysis in Hierarchical Reinforcement Learning
Zadem, M. (2024) · theses.fr (ABES) · doi:10.70675/f273f52az0906z443cza443z172023a62cf5
Task Specification and Generalization in Multi-Task Reinforcement Learning
Cachet, T. (2025) · theses.fr (ABES) · doi:10.70675/08eadb30z8466z4763zbba6za6a4dab9a29e
AdaTracker: Learning Adaptive In-Context Policy for Cross-Embodiment Active Visual Tracking
Wu, K., Chen, H., Han, J., Liu, H., Wang, C., Wang, Y., Li, Z., Liu, S., & Zhong, F. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3692075 · Cited by 2
Preference‐Based Representations for Collective Agency
Amir, N., van Vugt, M. K., & Fraser, M. (2026) · Topics in Cognitive Science · doi:10.1111/tops.70055
Incremental Reinforcement Learning with Temporally Dependent Goals
Yang, Y., Zhu, S., Giacomo, G. D., Zhao, Q., Li, X., & An, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30693-7_27
Abellán-Galiana, P., Pérez-Grau, F. J., Viguria, A., & Ollero, A. (2026) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas69441.2026.11598742
“Liu et al. [9] formalize the GCRL framework and discuss challenges such as sparse rewards and generalization across goal distributions.”
Dual-distance-based Goal Representation Learning
Kwon, N., & Kim, J. (2026) · International Conference on Ubiquitous and Future Networks · doi:10.1109/icufn69619.2026.11628551
“Goal-conditioned reinforcement learning (GCRL) is a reinforcement learning paradigm that aims to learn policies that enable an agent to reach specified goals [1].”
Serçe, A. B., & Aksoy, N. (2026) · Artificial Intelligence for Engineering · doi:10.1049/aie2.70025
Improving Policy Learning via Language-Guided State Representation in World Models
Zhang, L., Kan, M., Shan, S., & Chen, X. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3728343
“Restrictions apply. process (GC-POMDP) [40], defined as ( S , O , A , T , ρ 0 , L , φ ) , where S and O denote the set of states and observations, respectively; A is the action space; T is the environment transition dynamics; ρ 0 is the initial state distribution; L represents the set of free-form…”
Zhu, M., Chen, F., Yang, M., Xu, G., Wu, Y., Ma, G., Cheng, G., & Zheng, X. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00064
“For this purpose, we reframe ER as a goal-reaching RL [18], formulated as goal-augmented MDP by ( G , S , A , T , ϕ, R ) , and map ER concepts into it: • State s consists of the personalized prompt and previously generated tokens { ˆ e u,v, 1 , ..., ˆ e u,v,i } .”
Self-citation
Metadata
- Authors
- Liu, M., Zhu, M., & Zhang, W.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
- Published
- 2022-07
- Pages
- 5502-5511
- OpenAlex
- W4285601030
- Cited by
- 115
- References
- 63
- Citations by year
- 2026: 14 · 2025: 47 · 2024: 28 · 2023: 24 · 2022: 2
- FWCI
- 13.96
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076161, 2021SHZDZX0102)
- Shanghai Jiao Tong University (2021SHZDZX0102)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- goal-conditioned reinforcement learning, goal representation, goal-conditioned policies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
