Goal-Conditioned Reinforcement Learning: Problems and Solutions
Open accessAbstract
Goal-conditioned reinforcement learning (GCRL), related to a set of complex RL problems, trains an agent to achieve different goals under particular scenarios. Compared to the standard RL solutions that learn a policy solely depending on the states or observations, GCRL additionally requires the agent to make decisions according to different goals. In this survey, we provide a comprehensive overview of the challenges and algorithms for GCRL. Firstly, we answer what the basic problems are studied in this field. Then, we explain how goals are represented and present how existing solutions are designed from different points of view. Finally, we make the conclusion and discuss potential future prospects that recent researches focus on.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
UBCL is most accurately described as a form of Goal-Conditioned RL, where the agent receives a target point in a bounded, interpretable behavior space and is rewarded for reducing its distance to that point.
“UBCL is most accurately described as a form of Goal-Conditioned RL (GCRL) [42]: the agent receives a target point in a bounded, interpretable behavior space and is rewarded for reducing its distance to that point (Eq. 1).”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,821 · see citing works
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355 · cited 10× · see citing works
“Different from standard RL, GCRL augments the observation with an additional goal that the agent is required to achieve when making a decision in an episode [Schaul et al., 2015; Andrychowicz et al., 2017].”
BackgroundMethodInfluentialOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · cited 3× · see citing works
“Researchers used to evaluate their image algorithms on simulation environments, as in Warde-Farley et al. ; Lee et al. ; Chane-Sane et al. [2019; 2020; 2021], and also in real-world robotics applications [Nair et al. , 2018b; Nair et al. , 2020b].”
BackgroundMethodUniversal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 677 · see citing works
Decision Transformer: Reinforcement Learning via Sequence ModelingPreprint
Chen, L.-L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01345 · Cited by 677 · cited 1× · see citing works
“Recently, Chen et al. [ 2021 ] proposed decision transformer (DT) to solve offline RL problems, which treats RL as a sequence modeling problem by generating actions from offline datasets.”
BackgroundDo As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624 · see citing works
Visual Reinforcement Learning with Imagined GoalsPreprint
Nair, A., Pong, V. H., Dalal, M., Bahl, S., Lin, S., & Levine, S. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1807.04742 · Cited by 623 · cited 4× · see citing works
“[2019; 2020; 2021], and also real-world robotics applications [Nair et al., 2018b; Nair et al., 2020b].”
BackgroundMethodInfluentialBrockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · cited 1× · see citing works
“…the robotics benchmarking tasks discussed in Section 3.1 with image state / goal space instead; moreover, DeepMind Lab [Beattie et al. , 2016], Deepmind Control Suite (DMC) [Tassa et al. , 2020], Atari games [Brockman et al. , 2016] and Minecraft [Johnson et al. , 2016] are also popular testbeds.”
MethodInfluentialDiversity is All You Need: Learning Skills without a Reward Function
Eysenbach, B., Gupta, A., Ibarz, J., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1802.06070 · Cited by 480 · cited 2× · see citing works
“4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [Eysenbach et al., 2019a; Sharma et al., 2020].”
BackgroundInfluentialMulti-Goal Reinforcement Learning: Challenging Robotics Environments and Request for ResearchPreprint
Plappert, M., Andrychowicz, M., Ray, A., McGrew, B., Baker, B., Powell, G., Schneider, J., Tobin, J., Chociej, M., Welinder, P., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09464 · Cited by 372 · see citing works
A Survey of Reinforcement Learning Informed by Natural Language
Luketina, J., Nardelli, N., Farquhar, G., Foerster, J., Andreas, J., Grefenstette, E., Whiteson, S., & Rocktäschel, T. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/880 · Cited by 208 · cited 1× · see citing works
“Since our work focus on the essential of GCRL, we will only cover how language forms the goals, and readers may refer to Luketina et al. [2019] for a detailed review of language usage in RL.”
BackgroundThe Malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T. J., & Bignell, D. E. (2016) · International Joint Conference on Artificial Intelligence · Cited by 205 · see citing works
dm_control: Software and tasks for continuous control
Tunyasuvunakool, S., Muldal, A., Doron, Y., Liu, S., Bohez, S., Merel, J., Erez, T., Lillicrap, T., Heess, N., & Tassa, Y. (2020) · Software Impacts · doi:10.1016/j.simpa.2020.100022 · Cited by 200 · cited 1× · see citing works
“…the robotics benchmarking tasks discussed in Section 3.1 with image state / goal space instead; moreover, DeepMind Lab [Beattie et al. , 2016], Deepmind Control Suite (DMC) [Tassa et al. , 2020], Atari games [Brockman et al. , 2016] and Minecraft [Johnson et al. , 2016] are also popular testbeds.”
MethodInfluentialAutomatic Goal Generation for Reinforcement Learning AgentsPreprint
Florensa, C., Held, D., Geng, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06366 · Cited by 193 · cited 5× · see citing works
“Specifically, during a training loop, the agent first receives a desired goal from the environment or it should choose its own goal that is easier to achieve at the current stage [Ren et al., 2019; Florensa et al., 2018].”
BackgroundMethodInfluentialDynamics-Aware Unsupervised Discovery of SkillsPreprint
Sharma, A., Gu, S., Levine, S., Kumar, V., & Hausman, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01657 · Cited by 187 · cited 2× · see citing works
“4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [Eysenbach et al., 2019a; Sharma et al., 2020].”
BackgroundInfluentialEfficient Exploratory Learning of Inverse Kinematics on a Bionic Elephant Trunk
Rolf, M., & Steil, J. J. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2287890 · Cited by 165 · see citing works
Universal Planning NetworksPreprint
Srinivas, A., Jabri, A., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00645 · Cited by 163 · see citing works
Learning with AMIGo: Adversarially Motivated Intrinsic GoalsPreprint
Campero, A., Raileanu, R., Küttler, H., Tenenbaum, J., Rocktäschel, T., & Grefenstette, E. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2006.12122 · Cited by 150 · cited 1× · see citing works
“, 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al., 2021] Image X GAP [Nair et al.”
BackgroundAutomatic Curriculum Learning through Value DisagreementPreprint
Zhang, Y.-Z., Abbeel, P., & Pinto, L. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2006.09641 · Cited by 134 · cited 1× · see citing works
“Moreover, Zhang et al. [2020] treated the epistemic uncertainty of the Q-function as a measurement of the difficulty for reached goals, which is further used for constructing a distribution to sample goals.”
MethodSearch on the Replay Buffer: Bridging Planning and Reinforcement Learning
Eysenbach, B., Salakhutdinov, R., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.05253 · Cited by 123 · cited 3× · see citing works
“…the circle of reaching a meaningful goal state (we have discussed some examples like reward and commands in Section 3.4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [ Eysenbach et al. , 2019a; Sharma et al. , 2020 ] .”
BackgroundMethodInfluentialGRAIL: A Goal-Discovering Robotic Architecture for Intrinsically-Motivated Learning
Santucci, V., Baldassarre, G., & Mirolli, M. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2538961 · Cited by 113 · cited 1× · see citing works
“Technically, goal is a core data structure widely used in many re-lated domains, for describing either achieving or maintaining some condition, e.g., goal reasoning [Roberts et al. , 2018; Niemueller et al. , 2019] and robotics [Rolf and Steil, 2013; Santucci et al. , 2016].”
BackgroundLanguage as an Abstraction for Hierarchical Deep Reinforcement Learning
Jiang, Y., Gu, S., Murphy, K., & Finn, C. (2019) · arXiv · doi:10.48550/arxiv.1906.07343 · Cited by 109 · see citing works
Curriculum-guided Hindsight Experience Replay
Fang, M., Zhou, T., Du, Y., Han, L., & Zhang, Z. (2019) · neural information processing systems · Cited by 102 · see citing works
Contextual Imagined Goals for Self-Supervised Robotic LearningPreprint
Nair, A., Bahl, S., Khazatsky, A., Pong, V. H., Berseth, G., & Levine, S. (2019) · Conference on Robot Learning · doi:10.48550/arxiv.1910.11670 · Cited by 77 · see citing works
Beattie, C., Leibo, J. Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., Sadik, A., et al. (2016) · arXiv · doi:10.48550/arxiv.1612.03801 · Cited by 68 · see citing works
Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
Colas, C., Karch, T., Lair, N., Dussoux, J.-M., Moulin-Frier, C., Dominey, P. F., & Oudeyer, P.-Y. (2020) · arXiv · doi:10.48550/arxiv.2002.09253 · Cited by 64 · cited 2× · see citing works
“[Jiang et et al. , 2019 ] , or taken into a recurrent neural network (RNN) as learning the embedding jointly with the RL agent [ Colas et al. , 2020a ] .”
BackgroundMethodLTLf/LDLf Non-Markovian Rewards
Brafman, R. I., Giacomo, G. D., & Patrizi, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11572 · Cited by 62 · cited 2× · see citing works
“Apart from directions mentioned above, we note that there are more intriguing future directions worth explored in the formulation of GCRL, such as time-extended goals [Forestier and Oudeyer, 2016; Brafman et al., 2018], constraint-based goals [Colas et al.”
BackgroundUnsupervised Control Through Non-Parametric Discriminative RewardsPreprint
Warde-Farley, D., Van de Wiele, T., Kulkarni, T. D., Ionescu, C., Hansen, S., & Mnih, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11359 · Cited by 62 · cited 1× · see citing works
“, 2018b] Image X DISCERN [Warde-Farley et al., 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al.”
BackgroundModular active curiosity-driven discovery of tool use
Forestier, S., & Oudeyer, P. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759584 · Cited by 59 · cited 2× · see citing works
“Apart from directions mentioned above, we note that there are more intriguing future directions worth explored in the formulation of GCRL, such as time-extended goals [Forestier and Oudeyer, 2016; Brafman et al. , 2018],”
BackgroundExploration via Hindsight Goal GenerationPreprint
Ren, Z., Dong, K., Zhou, Y., Liu, Q., & Peng, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04279 · Cited by 57 · cited 4× · see citing works
“Specifically, during a training loop, the agent first receives a desired goal from the environment or it should choose its own goal that is easier to achieve at the current stage [Ren et al., 2019; Florensa et al., 2018].”
BackgroundMethodInfluentialHierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal GenerationPreprint
Nair, S., & Finn, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05829 · Cited by 56 · cited 1× · see citing works
“, 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al.”
BackgroundMaximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement LearningPreprint
Pitis, S., Chan, H., Zhao, S., Stadie, B. C., & Ba, J. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02832 · Cited by 54 · see citing works
Training Agents using Upside-Down Reinforcement LearningPreprint
Srivastava, R. K., Shyam, P., Mutz, F. W., Jaśkowski, W., & Schmidhuber, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02877 · Cited by 52 · cited 1× · see citing works
“Concurrently, Schmidhuber [2019] and Srivastava et al. [2019] also exploited reward-form goals on standard RL problems.”
BackgroundKeeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped RewardsPreprint
Trott, A., Zheng, S., Xiong, C., & Socher, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01417 · Cited by 50 · cited 5× · see citing works
“A straightforward idea to prevent the problem is to reshape the reward with a distance measure d between the achieved goal and the desired goal without any domain knowledge [Trott et al. , 2019], i.e., ˜ r g ( s t , a t , g ) = − d ( φ ( s t +1 ) , g ) .”
BackgroundInfluentialReinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to ActionsPreprint
Schmidhuber, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02875 · Cited by 46 · cited 2× · see citing works
“To solve image tasks, the state and goal images can be directly fed into the policy and value networks and trained in an end-to-end manner, as in Chane-Sane et al. ; Eysen-bach et al. ; Trott et al. [ 2021; 2019b; 2019 ] .”
BackgroundMethodRewriting History with Inverse RL: Hindsight Inference for Policy ImprovementPreprint
Eysenbach, B., Geng, X., Levine, S., & Salakhutdinov, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11089 · Cited by 38 · cited 1× · see citing works
“Recently, Eysenbach et al. [ 2020 ] revealed the relationship between HER and inverse RL which studies how to recover the reward function on a set of offline expert trajectories.”
BackgroundRapid Exploration for Open-World Navigation with Latent Goal ModelsPreprint
Shah, D., Eysenbach, B., Kahn, G., Rhinehart, N., & Levine, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.05859 · Cited by 30 · cited 2× · see citing works
“…be used to command the robots to handle a variety of tasks with kinds of goal elements, such as a sentence of text description [Ahn et al. , 2022], a goal image of the order of object placement [Chebotar et al. , 2021], a goal image of the targeted area in the open world [Shah et al. , 2021], etc.”
BackgroundMethodThe Laplacian in RL: Learning Representations with Efficient ApproximationsPreprint
Wu, Y., Tucker, G., & Nachum, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04586 · Cited by 29 · cited 1× · see citing works
“, 2017] Vector/Image X Laplacian Representation Learning [Wu et al., 2018] Vector/Image X SR [Trott et al.”
BackgroundReward-Conditioned PoliciesPreprint
Kumar, A., Peng, X. B., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13465 · Cited by 28 · cited 1× · see citing works
“Kumar et al. [2019] proposed to learn a return-conditioned policy π ( a | s, R ) where R is an estimated return of the whole trajectory.”
BackgroundWorld Model as a Graph: Learning Latent Landmarks for PlanningPreprint
Zhang, L., Yang, G., & Stadie, B. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.12491 · Cited by 28 · see citing works
Grounding Language to Autonomously-Acquired Skills via Goal GenerationPreprint
Akakzia, A., Colas, C., Oudeyer, P., Chétouani, M., & Sigaud, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07185 · Cited by 27 · see citing works
EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Models
Colas, C., Hejblum, B. P., Rouillon, S., Thiébaut, R., Oudeyer, P.-Y., Moulin-Frier, C., & Prague, M. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12588 · Cited by 26 · see citing works
Model-Based Visual Planning with Self-Supervised Functional DistancesPreprint
Tian, S., Nair, S., Ebert, F., Dasari, S., Eysenbach, B., Finn, C., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15373 · Cited by 25 · see citing works
Learning from Trajectories via Subgoal DiscoveryPreprint
Paul, S., Vanbaar, J., & Roy–Chowdhury, A. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07224 · Cited by 25 · cited 1× · see citing works
“Such an intuition motivates the work of Paul et al. [2019], which made use of the order of sub-goal indices along the expert trajectory as the evidence of difficulty, and learned to predict the sub-goals in the equipartitions from the expert trajectory conditioned on the given state.”
MethodGoal-Conditioned Reinforcement Learning with Imagined SubgoalsPreprint
Chane-Sane, E., Schmid, C., & Laptev, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00541 · Cited by 24 · see citing works
What Can I Do Here? Learning New Skills by Imagining Visual Affordances
Khazatsky, A., Nair, A., Jing, D., & Levine, S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561692 · Cited by 24 · cited 2× · see citing works
“A common choice of the encoder is variational auto-encoder (VAE) [ Nair et al. , 2018b; Khazatsky et al. , 2021 ] ; or a context-conditioned VAE [ Nair et al. , 2020a ] for generalizing to different environments.”
BackgroundDiscovering and Achieving Goals via World ModelsPreprint
Mendonca, R., Rybkin, O., Daniilidis, K., Hafner, D., & Pathak, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09514 · Cited by 24 · see citing works
Goal-Aware Prediction: Learning to Model What MattersPreprint
Nair, S., Savarese, S., & Finn, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07170 · Cited by 23 · cited 3× · see citing works
“[2019; 2020; 2021], and also real-world robotics applications [Nair et al., 2018b; Nair et al., 2020b].”
BackgroundMethodPlanGAN: Model-based Planning With Sparse Rewards and Multiple GoalsPreprint
Charlesworth, H. J., & Montana, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.00900 · Cited by 23 · cited 1× · see citing works
“, 2020] Vector X X PlanGAN [Charlesworth and Montana, 2020] Vector X AIM [Durugkar et al.”
BackgroundACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement LearningPreprint
Chan, H., Wu, Y., Kiros, J., Fidler, S., & Ba, J. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04546 · Cited by 20 · cited 3× · see citing works
“To train RL algorithms, those words must be either translated into embeddings via pre-training models [Chan et al., 2019], or taken into a recurrent neural network (RNN) as learning the embedding jointly with the RL agent [Colas et al.”
BackgroundMethodInfluentialGuided goal generation for hindsight multi-goal reinforcement learning
Bai, C., Liu, P., Zhao, W., & Tang, X. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.022 · Cited by 20 · cited 1× · see citing works
“, 2019] Vector X G-HER [Bai et al., 2019] Vector X Hindsight Planner [Lai et al.”
BackgroundMapGo: Model-Assisted Policy Optimization for Goal-Oriented Tasks
Zhu, M., Liu, M., Shen, J. G., Zhang, Z., Chen, S., Zhang, W., Ye, D., Yu, Y. H., Fu, Q., & Yang, W. P. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/480 · Cited by 19 · cited 3× · see citing works
“On the other hand, setting the goal as a desired moving speed and ask the cheetah agent to run up at the speed (HalfCheetah) [Zhu et al., 2021]; or a vector of the target position to reach, as is used in AntLocomotion [Florensa et al.”
MethodInfluentialSelf-citationGoal Reasoning in the CLIPS Executive for Integrated Planning and Execution
Niemueller, T., Hofmann, T., & Lakemeyer, G. (2019) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v29i1.3544 · Cited by 18 · see citing works
Chitnis, R., Silver, T., Tenenbaum, J. B., Kaelbling, L. P., & Lozano‐Pérez, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17400 · Cited by 18 · cited 1× · see citing works
“, 2021], composition of goals [Chitnis et al., 2021] etc.”
BackgroundDynamical Distance Learning for Semi-Supervised and Unsupervised Skill DiscoveryPreprint
Hartikainen, K., Geng, X., Haarnoja, T., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.08225 · Cited by 17 · cited 1× · see citing works
“, 2019] Vector/Image X DDL [Hartikainen et al., 2019] Vector/Image X X SoRB [Eysenbach et al.”
BackgroundIntrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short SurveyPreprint
Colas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2021) · HAL · Cited by 16 · see citing works
Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward TasksPreprint
McCarthy, R., Wang, Q., & Redmond, S. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02414 · Cited by 16 · see citing works
Adversarial Intrinsic Motivation for Reinforcement LearningPreprint
Durugkar, I., Tec, M., Niekum, S., & Stone, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13345 · Cited by 14 · cited 2× · see citing works
“HER is widely used as the backbone in many related works [Ren et al., 2019; Pitis et al., 2020; Durugkar et al., 2021], and researchers also take efforts on extending the idea of HER on designing advanced relabeling strategies.”
MethodKuang, Y., Weinberg, A. I., Vogiatzis, G., & Faria, D. R. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223473 · Cited by 13 · cited 1× · see citing works
“, 2020] Vector X GDP [Kuang et al., 2020] Vector X VDS [Zhang et al.”
BackgroundSpecial issue on goal reasoning
Roberts, M., Borrajo, D., Cox, M. T., & Yorke‐Smith, N. (2018) · AI Communications · doi:10.3233/aic-180760 · Cited by 9 · cited 1× · see citing works
“Technically, goal is a core data structure widely used in many re-lated domains, for describing either achieving or maintaining some condition, e.g., goal reasoning [Roberts et al. , 2018; Niemueller et al. , 2019] and robotics [Rolf and Steil, 2013; Santucci et al. , 2016].”
BackgroundColas, C., Karch, T., Sigaud, O., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09830 · Cited by 8 · see citing works
Weakly-Supervised Reinforcement Learning for Controllable BehaviorPreprint
Lee, L., Eysenbach, B., Salakhutdinov, R., Gu, S., & Finn, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.02860 · Cited by 7 · cited 2× · see citing works
“The encoding can also be derived via training a weak supervised model for learning disentangled representation [Lee et al., 2020]; and even an end-to-end supervised imitation learning objective [Srinivas et al.”
MethodHindsight Expectation Maximization for Goal-conditioned Reinforcement LearningPreprint
Tang, Y., & Kucukelbir, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07549 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [21]21. Yevgen Chebotar, Karol Hausman, et al. Actionable models: Unsupervised offline reinforce- ment learning of robotic skills. ICML, 2021.
- [42]42. Dibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu, Coline Devin, Benjamin Eysenbach, and Sergey Levine. Learning to reach goals via iterated super- vised learning. ICLR, 2021.
- [76]76. Vitchyr H Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. Skew- fit: State-covering self-supervised reinforcement learning. ICML, 2020.
- [83]83. Schaul et al., 2015] Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function ap- proximators. ICML, 2015.
- [91]91. Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Universal plan- ning networks: Learning generalizable representations for visuomotor control. ICML, 2018.
Cited by
Sorted by citations
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861 · see citing works
“Each individual goal g ∼ G can be differentiated by its reward function r ( s t , a t , g ) , hence the objective for GCRL becomes maximizing the expected return over the distribution of goals: J ( π ) := IE ( s t ,a t ) ∼ μ π ,g ∼G [ (cid:5) t γ t r ( s, a, g )] [15].”
BackgroundA survey on model-based reinforcement learning
Luo, F.-M., Xu, T., Lai, H., Chen, X., Zhang, W., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3696-5 · Cited by 173 · see citing works
“Goal-conditioned reinforcement learning (GCRL), also named as goal-oriented RL [Liu et al., 2022], deals with the tasks where agents are expected to achieve different goals [Pitis et al.”
BackgroundSelf-citationWang, J., Qin, Y., Wu, Q., Zeng, D., Gao, X., Wang, Q., Li, Z., Ni, Y., Li, H., Zhang, P., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03724-5 · Cited by 56 · see citing works
Large sequence models for sequential decision-making: a survey
Wen, M., Lin, R., Wang, H., Yang, Y., Wen, Y., Mai, L., Wang, Y.-X., Zhang, H., & Zhang, W. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2689-5 · Cited by 50 · see citing works
“Goal-condition RL [84–86] learns a general policy function to finish a series of simple tasks, for instance, to reach di ff erent goal states.”
BackgroundSelf-citationGrasp Multiple Objects With One Hand
Li, Y., Liu, B., Geng, Y., Li, P., Yang, Y., Zhu, Y., Liu, T., & Huang, S. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3374190 · Cited by 34 · see citing works
“Instead, the entire length and sides of the fingers must be engaged [3], [9], necessitating a carefully configured grasping pose for effective force closure on each object while avoiding collisions.”
Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation
Yan, J., Luo, B., & Xu, X. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10794-3 · Cited by 19 · see citing works
“Thus, GoRL is by default accompanied by the problem of reward sparsity (Liu et al. 2022; Plappert et al. 2018).”
Goal-Conditioned Reinforcement Learning within a Human-Robot Disassembly Environment
Elguea-Aguinaco, Í., Serrano-Muñoz, A., Chrysostomou, D., Inziarte-Hidalgo, I., Bøgh, S., & Arana-Arexolaleiba, N. (2022) · Applied Sciences · doi:10.3390/app122211610 · Cited by 18 · see citing works
Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learning
Ding, H., Tang, Y., Wu, Q. H., Wang, B., Chen, C., & Wang, Z. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123477 · Cited by 13 · see citing works
“Goal-conditioned RL aims to learn a universal policy that can master reaching multiple goals simultaneously [37].”
BackgroundInteractive Policy Shaping for Human-Robot Collaboration with Transparent Matrix Overlays
Brawer, J., Ghose, D., Candon, K., Qin, M., Roncone, A., Vázquez, M., & Scassellati, B. M. (2023) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3568162.3576983 · Cited by 13 · see citing works
“We also assume at some future point the exis� tence of new user preferences for the task, � ′, which induce a new MDP parameterized by � ′ and optimal policy �∗′ .”
MethodAccelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping*
Pizzuto, G., Wang, H., Fakhruldeen, H., Peng, B., Luck, K. S., & Cooper, A. I. (2024) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case59546.2024.10711291 · Cited by 14 · see citing works
“We aim to contribute to this area by proposing a novel way of laboratory skill learning using goal-conditioned reinforcement learning [22] for a task that has not been addressed before, while demonstrating the important role that DRL solutions can have when adopted in real-world laboratories to…”
BackgroundGoals as reward-producing programs
Davidson, G., Todd, G., Togelius, J., Gureckis, T. M., & Lake, B. M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-00981-4 · Cited by 14 · see citing works
Sun, Z., Pang, B. J., Yuan, X., Xu, X., Song, Y. H., Song, R., & Li, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110866 · Cited by 11 · see citing works
Jang, K., Lichtlé, N., Vinitsky, E., Shah, A., Bunting, M., Nice, M. W., Piccoli, B., Seibold, B., Work, D. B., Monache, M. L. D., et al. (2025) · IEEE Control Systems · doi:10.1109/mcs.2024.3503372 · Cited by 15 · see citing works
Lyapunov-Informed Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks
Feng, P., Shi, R., Wang, S., Wu, Q., Yu, X., & Wu, W. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3603024 · Cited by 10 · see citing works
“In LMG, typical goal states are usually represented as vectors of desired properties or features of multi-agent systems, such as a combination of desired locations of multiple agents, moving speeds or rotation poses of multiple robots (for more examples of goal states, please refer to [7]).”
Zhao, Z., Li, Y., Liu, C., Liu, X., & Gao, J. X. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3453431 · Cited by 9 · see citing works
Bridging language and action: A survey of language-conditioned robot manipulation
Yao, X., Zhou, H., Mees, O., Meng, Y., Xiao, T. G., Bisk, Y., Oh, J., Johns, E., Shridhar, M., Shah, D., et al. (2026) · The International Journal of Robotics Research · doi:10.1177/02783649261468360 · Cited by 23 · see citing works
“6: Typical representations of goals in goal-conditioned learning: vectors, images and languages (illustration from 139 ). with only less than 1% demonstration data labeled with language instructions, the trained agent can successfully solve 18 distinct tasks with a high success rate in the tabletop…”
Goal-Oriented Multi-Modal Interactive Recommendation with Verbal and Non-Verbal Relevance Feedback
Wu, Y., Macdonald, C., & Ounis, I. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3608775 · Cited by 9 · see citing works
“reinforcement learning (GORL) [11, 33] can be easily adapted to the MMIR task with a goal-oriented policy optimisation function that allows the agents to pursue their own goals (i.”
BackgroundMethodInfluentialLearning Control Policies for Variable Objectives from Offline Data
Weber, M., Swazinna, P., Hein, D. M., Udluft, S., & Sterzing, V. (2023) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci52147.2023.10371978 · Cited by 8 · see citing works
“This scheme was used in a similar form in [25]–[32] where the parameters are instead input to the Q-function.”
MethodZou, Q., & Suzuki, E. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3344880 · Cited by 7 · see citing works
“Goal-conditioned RL [7], [8], [9] maximizes an accumulated reward in a trajectory for learning diverse goal-reaching behaviors without making assumptions about the form of state and goal or the dynamic model of the environment.”
Curriculum-guided skill learning for long-horizon robot manipulation tasks
Alves, J., Lau, N., & da Silva, F. M. T. P. (2025) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2025.105032 · Cited by 7 · see citing works
Stitching Sub-trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL
Kim, S., Choi, Y., Matsunaga, D. E., & Kim, K.-E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29215 · Cited by 7 · see citing works
A Controllable Agent by Subgoals in Path Planning Using Goal-Conditioned Reinforcement Learning
Lee, G. T., & Kim, K. (2023) · IEEE Access · doi:10.1109/access.2023.3264264 · Cited by 7 · see citing works
“One similar paper with our method introduces variational empowerment, a method to learn a low-dimensional state representation that reflects the agent’s capability to achieve various goals [69].”
MethodGoal-Conditioned Reinforcement Learning With Disentanglement-Based Reachability Planning
Qian, Z., You, M., Zhou, H., Xu, X., & He, B. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3287362 · Cited by 7 · see citing works
Huang, X., Hu, J., Liu, Q., Zhao, G., Deng, W., & Liu, W. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113738 · Cited by 6 · see citing works
Rank2Reward: Learning Shaped Reward Functions from Passive Video
Yang, D. Z., Tjia, D., Berg, J., Damen, D., Agrawal, P., & Gupta, A. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610873 · Cited by 5 · see citing works
“This suggests that the true reward for the task (which is unknown) is positive and non-zero for all states, a common occurrence in a huge variety of problems especially goal-reaching tasks [48]–[50].”
BackgroundExplicit-Implicit Subgoal Planning for Long-Horizon Tasks With Sparse Rewards
Wang, F., Duan, A., Zhou, P., Huo, S., Guo, G., Yang, C., & Navarro-Alarcón, D. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3574162 · Cited by 5 · see citing works
“Current approaches include imitating expert trajectories [3], [9], [10], increasing goal-oriented exploration [4], [11], [12], [13], and employing divide-and-conquer strategies [2], [14], [15].”
Language-Conditioned Offline RL for Multi-Robot Navigation
Morad, S., Shankar, A., Blumenkamp, J., & Prorok, A. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127288 · Cited by 5 · see citing works
“It is sometimes called goal-conditioned RL, multi-task RL, or universal value function approximation [38], [39], [40].”
Zhang, W. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/808 · Cited by 5 · see citing works
“Note that the return-to-go in DT acts like the command used in upside-down RL, which is analogous to the goal in goal-conditional RL [ Liu et al. , 2022 ] .”
MethodSelf-citationDGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data
Chen, G., Qi, J., Gao, Y. Q., Zhu, X., Dong, Z., & Sun, Y. (2023) · Information Sciences · doi:10.1016/j.ins.2023.120019 · Cited by 5 · see citing works
Wu, L., & Chen, K. (2024) · Machine Learning · doi:10.1007/s10994-023-06503-w · Cited by 5 · see citing works
“Several strategies have been proposed to explore new sub-goals efficiently (Florensa et al., 2018; Pong et al., 2020; Pitis et al., 2020; Mendonca et al., 2021; Liu et al., 2022).”
InfluentialLong-Term Feature Extraction via Frequency Prediction for Efficient Reinforcement Learning
Wang, J., Ye, M., Kuang, Y., Yang, R., Zhou, W., Li, H., & Wu, F. H. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3529264 · Cited by 5 · see citing works
Interactive Navigation for Legged Manipulators with Learned Arm-Pushing Controller
Bi, Z., Chen, K., Zheng, C., Li, Y., Li, H., & Ma, J. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246770 · Cited by 4 · see citing works
“Motivated by goal-conditioned RL [21], we formulate the object pushing problem as a finite-horizon Partially Observable Markov Decision Process.”
Generative subgoal oriented multi-agent reinforcement learning through potential field
Li, S., Jiang, H., Liu, Y., Zhang, J., Xu, X., & Liu, D. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106552 · Cited by 4 · see citing works
Goal-Conditioned Q-learning as Knowledge Distillation
Levine, A., & Feizi, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26024 · Cited by 5 · see citing works
A Definition of Open-Ended Learning Problems for Goal-Conditioned AgentsPreprint
Sigaud, O., Baldassarre, G., Colas, C., Doncieux, S., Duro, R. J., Oudeyer, P.-Y., Perrin, N., & Santucci, V. G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.00344 · Cited by 4 · see citing works
“In addition, goals have emerged as a key means to support open-ended learning.”
BackgroundAutonomous Wheel Loader Navigation Using Goal-Conditioned Actor-Critic MPC
Mäki-Penttilä, A., Toulkani, N. E., & Ghabcheloo, R. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127375 · Cited by 3 · see citing works
Synergizing habits and goals with variational Bayes
Han, D., Doya, K., Li, D., & Tani, J. (2024) · Nature Communications · doi:10.1038/s41467-024-48577-7 · Cited by 3 · see citing works
“In machine learning, one practical reason for such a separation isthat the inputs are different — in goal-conditioned control or decision-Received making 16 , the goalisusually an input to the model.”
BackgroundSun, Z., Yuan, X., Xu, Q., Pang, B. J., Song, Y. H., Song, R., & Li, Y. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3495775 · Cited by 3 · see citing works
A view on learning robust goal-conditioned value functions: Interplay between RL and MPC
Lawrence, N. P., Loewen, P. D., Forbes, M. G., Gopaluni, R. B., & Mesbah, A. (2025) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2025.101027 · Cited by 3 · see citing works
Zhang, K., Lee, C. K. M., & Tsang, Y. P. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3565554 · Cited by 3 · see citing works
“For the proposed OR-BIDT that involves more general and complex scenarios, we adopt the GA-MDP framework to define the problem and address it with GCRL [43].”
Task-Agnostic Learning to Accomplish New Tasks
Zhang, X., Wang, X., Liu, X., Wang, W., Xiaopeng Fan, B., & Zhao, D. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3565515 · Cited by 2 · see citing works
“Recently, goal-conditioned RL is proposed to enable an agent to be able to perform multiple tasks [35], [36], [37], [38], [39], [40], [41].”
Takeda, S., Yamamori, S., Yagi, S., & Morimoto, J. (2025) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case58245.2025.11163857 · Cited by 2 · see citing works
“For lower-level policy learning, goal-conditioned reinforcement learning [4] is employed, with an encoder network providing task representations.”
Goal-Conditioned Reinforcement Learning for Data-Driven Maritime Navigation
Vaidheeswaran, V., Jayakody, D., Mulay, S., Lo, A., Alam, M. M., & Spadon, G. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402252 · Cited by 2 · see citing works
“Goal-Conditioned Reinforcement Learning (GCRL) is emerging as a promising approach for learning policies that can adapt and generalize across multiple goals [6], [7].”
Nguyen, T. M., & Truong, T. N. (2025) · npj Computational Materials · doi:10.1038/s41524-025-01852-7 · Cited by 2 · see citing works
Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning
Lu, Y., Sinha, A., & Varakantham, P. R. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30132 · Cited by 2 · see citing works
WayEx: Waypoint Exploration using a Single Demonstration
Levy, M., Saini, N., & Shrivastava, A. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611088 · Cited by 2 · see citing works
“Prior works have studied how to use RL in many different ways in order to solve these tasks [14, 15, 16, 17, 18, 19].”
A new paradigm to study social and physical affordances as model-based reinforcement learning
Chartouny, A., Amini, K., Khamassi, M., & Girard, B. (2024) · Cognitive Robotics · doi:10.1016/j.cogr.2024.08.001 · Cited by 2 · see citing works
“Thus, goal-conditioned reinforcement learning and reward-free exploration approaches could be other interesting means of learning affordances in reinforcement learning [23] .”
Foundational Policy Acquisition via Multitask Learning for Motor Skill Generation
Yamamori, S., & Morimoto, J. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3543350 · Cited by 2 · see citing works
“To acquire a policy that can cope with multiple task goals, goal-conditioned RL methods have been developed [9], [10], [11], [12], [13].”
Goal-Conditioned Reinforcement Learning Approach for Autonomous Parking in Complex Environments
Kim, T. Y., Kang, T., Son, S., Ko, K. W., & Har, D. (2025) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic64266.2025.10920799 · Cited by 2 · see citing works
“To address these challenges, a goal-oriented learning strategy through Goal-conditioned RL (GCRL) [13] offers a promising solution.”
State Representation Learning for Goal-Conditioned Reinforcement Learning
Steccanella, L., & Jönsson, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26412-2_6 · Cited by 2 · see citing works
Invariant Representations Learning with Future Dynamics
Hu, W., He, M., Chen, X., & Wang, N. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107496 · Cited by 1 · see citing works
A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks
Kobanda, A., Maillard, O.-A., & Portelas, R. (2025) · IEEE Conference on Games (CoG) · doi:10.1109/cog64752.2025.11114167 · Cited by 1 · see citing works
“An interesting subset of tasks within the CRL framework is Goal-Conditioned RL (GCRL) (5; 6) allowing policies to be conditioned to specific target states, which is particularly valuable in navigation tasks.”
Li, X., Wang, S., Chen, C., Wei, H., Shi, Y., & Mo, H. (2025) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar65431.2025.11139480 · Cited by 1 · see citing works
“The problem of language-conditioned robot control can be modeled as a goal-conditioned partially observable Markov decision process [26]: ℳ = 〈𝒮, 𝒪, 𝒜, 𝒯, 𝜌 0 , ℒ, ∅, 𝑓〉 , where 𝒮 and 𝒪 are the set of states and observations separately, 𝒜 is the action space, 𝒯: 𝒮 × 𝒜 → 𝒮 is the…”
Hierarchical Reinforcement Learning for Articulated Tool Manipulation with Multifingered Hand
Xu, W., Zhao, Y., Guo, W., & Sheng, X. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246691 · Cited by 1 · see citing works
“We employ a goal-conditioned RL (GCRL) framework with two hierarchical policy layers to decompose a high-DoF system into manageable subproblems [14]–[17].”
A Unified Framework to Learn Collision-Free Loco-Manipulation via Adversarial Motion Priors
Yin, H., Qian, T., Li, M., Lu, G., Cai, M., & Kan, Z. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11245884 · Cited by 1 · see citing works
“By incorporating the loco-manipulation motion prior into the Goal-Conditioned RL (GCRL) [7] paradigm, we achieve both efficient and effective skill learning.”
Kobayashi, T. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1649239 · Cited by 1 · see citing works
“Next, O2S, a modified RS, was evaluated using a goal-conditioned RL (Liu et al., 2022) problem, which demand both memory consolidation and plasticity.”
The Cultural Evolution of Human Goals
Perez, J., Colas, C., Molinaro, G., Oudeyer, P., Derex, M., & Moulin-Frier, C. (2026) · Topics in Cognitive Science · doi:10.1111/tops.70078 · Cited by 1 · see citing works
MURM: Utilization of Multi-Views for Goal-Conditioned Reinforcement Learning in Robotic Manipulation
Jang, S., Jeong, H., & Yang, H. (2023) · Robotics · doi:10.3390/robotics12040119 · Cited by 1 · see citing works
Jabri, M. K., Papadakis, P., Abbasnejad, E., Coppin, G., & Shi, Q. (2023) · International Conference on Informatics in Control, Automation and Robotics (ICINCO) · doi:10.5220/0012268200003543 · Cited by 1 · see citing works
“In goal-conditioned imitation learning (Liu et al., 2022), each task or trajectory is uniquely characterized by a distinct goal state.”
BackgroundEwerton, M., Villamizar, M., Jankowski, J., Calinon, S., & Odobez, J. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341729 · Cited by 1 · see citing works
“This relates to the Goal-Conditioned RL problem [13], [14], [15] in which the Q (or Value) function is parametrized by a goal.”
Ricci Planner: Zero-Shot Transfer for Goal-Conditioned Reinforcement Learning via Geometric Flow
Song, W., & Lee, J. (2024) · IEEE Access · doi:10.1109/access.2024.3361478 · Cited by 1 · see citing works
Goal-conditioned offline reinforcement learning through state space partitioning
Wang, M., Jin, Y., & Montana, G. (2024) · Machine Learning · doi:10.1007/s10994-023-06500-z · Cited by 1 · see citing works
“Goal-conditioned reinforcement learning (GCRL) aims to learn policies capable of reaching a wide range of distinct goals, effectively creating a vast repertoire of skills (Liu et al., 2022; Plappert et al., 2018; Andrychowicz et al., 2017).”
Generalization of temporal logic tasks via future dependent options
Xu, D., & Fekri, F. (2024) · Machine Learning · doi:10.1007/s10994-024-06614-y · Cited by 1 · see citing works
“Some works (Araki et al., 2021; Hengst et al., 2022; Liu et al., 2022) solve new TL tasks by leveraging the learned reusable skills or options, but produce sub-optimal or even infeasible solutions, especially when the symbolic observation is lossy, i.e., one symbolic state can correspond to…”
Robust contact-rich manipulation through implicit motor adaptation
Xue, T., Razmjoo, A., Shetty, S., & Calinon, S. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251344638 · Cited by 1 · see citing works
“Many approaches have been proposed to learn manipulation policies for contact-rich manipulation, including Behavior Cloning (BC) (Florence et al. 2022; Torabi et al. 2018), Deep Reinforcement Learning (DRL) (Pertsch et al. 2021; Qi et al. 2023; Lee et al. 2020a), and Approximate Dynamic Programming (ADP) (Powell 2007;…”
Advancing Self-Determination Theory With Computational Intrinsic Motivation: The Case of CompetencePreprint
Lintunen, E. M., Ady, N. M., Christian Guckelsberger, B., & Deterding, S. (2024) · doi:10.31234/osf.io/7qy35 · Cited by 1 · see citing works
Unifying Modern AI with Robotics: Survey on MDPs with Diffusion and Foundation ModelsPreprint
Zhang, Z., Chen, R., Yang, Z., Xie, S., Chen, H., & Xiong, H. (2025) · doi:10.36227/techrxiv.174494994.46313754/v1 · Cited by 1 · see citing works
UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors
Cilan, A., & Ozgovde, A. (2026) · IEEE Transactions on Games · doi:10.1109/tg.2026.3703355
“Generating new or targeted play styles often requires retraining from scratch, limiting flexi-bility in design-time personalization.”
Synergizing habits and goals with variational BayesPreprint
Han, D., Doya, K., Li, D., & Tani, J. (2023) · doi:10.31234/osf.io/v63yj
Habits and goals in synergy: a variational Bayesian framework for behaviorPreprint
Han, D., Doya, K., Li, D., & Tani, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.05008
“In machine learning, one practical reason for such a separation isthat the inputs are different — in goal-conditioned control or decision-Received making 16 , the goalisusually an input to the model.”
BackgroundScaling Goal-based Exploration via Pruning Proto-goals
Bagaria, A., & Schaul, T. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/384
Liu, W., Liu, M., Jin, B., Zhu, Y., Gao, Q., & Sun, J. (2023) · IEEE Access · doi:10.1109/access.2023.3307573
Self-citationA Goal-Conditioned Reinforcement Learning Algorithm with Environment Modeling
Yu, Z., Sun, K., Li, C., Zhong, D., Yang, Y., & Zhao, Q. (2023) · Chinese Control Conference (CCC) · doi:10.23919/ccc58697.2023.10240963
“In model-based GcRL method, [13] proposes using a task-agnostic dynamics predictor to estimate the motion pattern of dynamic objects and utilizing it to augment the state input.”
BackgroundMethodInfluentialOptimal bipartite graph matching-based goal selection for policy-based hindsight learning
Sun, S., Zhang, H., Liu, Z., Chen, X., & Lan, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127734
Learning Efficient Representations for Goal-conditioned Reinforcement Learning via Tabu Search
Liang, T., Chen, T., Chen, X., & Ren, Q. (2024) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/cis-ram61939.2024.10672981
“In the literature, goal-conditioned tasks are a set of complex RL problems that train an agent to achieve different goals under different scenarios [10].”
BackgroundLatent State Space Quantization for Learning and Exploring Goals
Latyshev, A., & Panov, A. I. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-75540-8_3
Feng, D., Gong, X., Zhang, X., & Fu, J. (2024) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa63168.2024.00013
CraftFactory: A Conditioned Control Policy Benchmark for Compositional Generalization
Hou, J., Zhao, Y., & Zhao, J. F. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i2.32121
Inductive biases for zero-shot systematic generalization in language-informed reinforcement learning
Dijujin, N. H., Rohani, S. R. R., Samiei, M. M., & Baghshah, M. S. (2025) · Machine Learning · doi:10.1007/s10994-025-06764-7
“In the goal-conditioned settings, it is common to measure the performance of the agent using Success Rate (SR) (Liu et al., 2022).”
Skill Learning Using a Single Demonstration for Hierarchical Robotic Navigation
Alves, J. B., Lau, N., & da Silva, F. M. T. P. (2025) · IEEE International Conference on Autonomous Robot Systems and Competitions · doi:10.1109/icarsc65809.2025.10970170
“These segmented sub-tasks are then learned independently using a curriculum-based approach with goal-conditioned reinforcement learning (GCRL) [3].”
A Physics-Guided Neural Network Framework for Prediction and Control of Legged LocomotionPreprint
Öztürk, A. S., Uyanık, İ., & Morgül, Ö. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5269334
Adaptformer: Sequence Models as Adaptive Iterative Planners
Karthikeyan, A., & Pant, Y. V. (2025) · doi:10.21428/d82e957c.be853a08
Hashimoto, T., Isokawa, T., & Kamiura, N. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6579-2_11
Enerdast: a Free Energy Based Mechanism for Strategy Tuning in Sparse Reward Scenarios
Li, Z., Lang, Y., Wang, S., & Ren, Q. (2025) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc65474.2025.11090349
V-Pilot: A Velocity Vector Control Agent for Fixed-Wing UAVs from Imperfect Demonstrations
Gong, X., Feng, D., Xu, K., Zhou, X., Zheng, S. Q., Ding, B., & Wang, H. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128340
“In GCRL, there are three types of goals [14]: The desired goal represents the requirements of the task and its distribution is denoted as p dg .”
Neural Lyapunov Function Approximation with Self-Supervised Reinforcement Learning
McCutcheon, L., Gharesifard, B., & Fallah, S. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128069
“…| x t , u t ) at time t ∈ T denotes the stochastic transition dynamics from the current state x t ∈ X to x t +1 ∈ X under a policy π ϕ parameterized by ϕ , from which actions u t are sampled Building upon the standard MDP framework, the notation is extended to accommodate goal-conditioned RL [23].”
Target-oriented exploration guided by anticipated returns
Tsuboya, A., Kono, Y., & Takahashi, T. (2025) · Artificial Life and Robotics · doi:10.1007/s10015-025-01068-4
Cheng, H., & Brown, J. W. (2024) · neural information processing systems · doi:10.52202/079017-0104
Complex Robotic Manipulation via Hindsight Goal Diffusion and Graph-based Experience Replay
Sun, Z., Li, Z., He, J., Song, Y., Liu, P., Xu, Q., Yuan, X., & Song, R. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247131
“GCRL is a goal-extended reinforcement learning approach, that drives agents to learn policies that can adapt to different goals [10].”
Robots that Suggest Safe Alternatives
Jeong, H. J., Chen, R., & Bajcsy, A. V. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247186
“While this paradigm has enabled robots to learn complex behaviors and adapt to specified goals online [3], these learned policies can degrade when faced with out-of-distribution goal requests or states [4].”
An Enhanced Adviser-Actor-Critic Framework for High-Precision Reinforcement Learning Control
Yue, J., Chen, D., Cheng, L., & Gong, S. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.11.243
Generalization of Compositional Tasks with Logical Specification via Implicit Planning
Xu, D., & Fekri, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06106-5_12
Goal-conditioned Trajectory Generation for Planning with Flow Matching in Offline RL
Diao, H., Zhang, J., Zhang, P., & Peng, B. (2025) · IEEE International Conference on Unmanned Systems (ICUS) · doi:10.1109/icus66297.2025.11294744
“These types of tasks are referred to as Goal-conditioned planning, characterized by the agent receiving positive feedback only upon achieving a specific goal [5] [6].”
A DRL Approach for Mapless Planar Pushing of Arbitrary Objects in Cluttered Environments
Luz, G. S., & Macharet, D. G. (2025) · Anais Estendidos do XVII Simpósio Brasileiro de Robótica (SBR 2025) e do XVI Workshop de Robótica na Educação (WRE 2025) · doi:10.5753/sbrlars_estendido.2025.248248
“To support generalization across varying capsule widths W , which are randomly sampled, the observation includes W as part of a goal-conditioned setup [Liu et al. 2022].”
Shi, Z., Pan, H., Zhu, R., Li, C., Wu, S., & Wu, E. Q. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3648590
“In Goal-conditioned Reinforcement Learning (GCRL) [9], we define an infinite-horizon, discounted, goal-augmented Markov Decision Process (GMDP) using the tuple M G = S , A , G , P , R , γ, d 0 (cid:105) , where S , A , P , γ follow the standard MDP formulation and d 0 represents the initial state…”
Yu, H., Yang, H.-C., & Huang, Y. (2026) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-026-09735-x
Noh, S., Kim, S., & Jang, I. (2025) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc66702.2025.11388104
“A variety of methods have been proposed to address this issue, including reward shaping [7], intrinsic motivation [8], and goal-conditioned RL [23], [24].”
Cui, R., & Yan, Y. (2025) · IEEE Industrial Electronics Society Annual On-Line Conference (ONCON) · doi:10.1109/oncon68412.2025.11384494
“Through continuous interaction with the environment, DRL-based methods can autonomously learn obstacle avoidance policies without relying on predefined maps or handcrafted control rules, enabling flexible deployment and strong adaptability across diverse environments [8]–[10].”
Teaching Predicate-based Autotelic Agents
Akakzia, A. (2022) · theses.fr (ABES) · doi:10.70675/9b85df43z32c4z4101z8095z2f0bec7cdd3e
Learning goal-oriented agents with limited supervision
Mezghani, L. (2023) · theses.fr (ABES) · doi:10.70675/81301ccfz41d1z457cz8afcz7049fd4699c5
- Authors
- Liu, M., Zhu, M., & Zhang, W.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
- Published
- 2022-07
- Pages
- 5502-5511
- OpenAlex
- W4285601030
- Cited by
- 115
- References
- 63
- Citations by year
- 2026: 14 · 2025: 47 · 2024: 28 · 2023: 24 · 2022: 2
- FWCI
- 13.96
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076161, 2021SHZDZX0102)
- Shanghai Jiao Tong University (2021SHZDZX0102)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- goal-conditioned reinforcement learning, goal representation, goal-conditioned policies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
