The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Goal-Conditioned Reinforcement Learning: Problems and Solutions
Open accessAbstract
Goal-conditioned reinforcement learning (GCRL), related to a set of complex RL problems, trains an agent to achieve different goals under particular scenarios. Compared to the standard RL solutions that learn a policy solely depending on the states or observations, GCRL additionally requires the agent to make decisions according to different goals. In this survey, we provide a comprehensive overview of the challenges and algorithms for GCRL. Firstly, we answer what the basic problems are studied in this field. Then, we explain how goals are represented and present how existing solutions are designed from different points of view. Finally, we make the conclusion and discuss potential future prospects that recent researches focus on.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“UBCL is most accurately described as a form of Goal-Conditioned RL (GCRL) [42]: the agent receives a target point in a bounded, interpretable behavior space and is rewarded for reducing its distance to that point (Eq. 1).”
From UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors (2026) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355 · cited 10×
“Different from standard RL, GCRL augments the observation with an additional goal that the agent is required to achieve when making a decision in an episode [Schaul et al., 2015; Andrychowicz et al., 2017].”
BackgroundMethodInfluentialDecision Transformer: Reinforcement Learning via Sequence ModelingPreprint
Chen, L.-L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01345 · Cited by 849 · cited 1×
“Recently, Chen et al. [ 2021 ] proposed decision transformer (DT) to solve offline RL problems, which treats RL as a sequence modeling problem by generating actions from offline datasets.”
BackgroundOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · cited 3×
“Researchers used to evaluate their image algorithms on simulation environments, as in Warde-Farley et al. ; Lee et al. ; Chane-Sane et al. [2019; 2020; 2021], and also in real-world robotics applications [Nair et al. , 2018b; Nair et al. , 2020b].”
BackgroundMethodUniversal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · cited 1×
“…the robotics benchmarking tasks discussed in Section 3.1 with image state / goal space instead; moreover, DeepMind Lab [Beattie et al. , 2016], Deepmind Control Suite (DMC) [Tassa et al. , 2020], Atari games [Brockman et al. , 2016] and Minecraft [Johnson et al. , 2016] are also popular testbeds.”
MethodInfluentialDiversity is All You Need: Learning Skills without a Reward Function
Eysenbach, B., Gupta, A., Ibarz, J., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1802.06070 · Cited by 480 · cited 2×
“4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [Eysenbach et al., 2019a; Sharma et al., 2020].”
BackgroundInfluentialVisual Reinforcement Learning with Imagined GoalsPreprint
Nair, A., Pong, V. H., Dalal, M., Bahl, S., Lin, S., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04742 · Cited by 380 · cited 4×
“[2019; 2020; 2021], and also real-world robotics applications [Nair et al., 2018b; Nair et al., 2020b].”
BackgroundMethodInfluentialMulti-Goal Reinforcement Learning: Challenging Robotics Environments and Request for ResearchPreprint
Plappert, M., Andrychowicz, M., Ray, A., McGrew, B., Baker, B., Powell, G., Schneider, J., Tobin, J., Chociej, M., Welinder, P., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09464 · Cited by 372
dm_control: Software and tasks for continuous control
Tunyasuvunakool, S., Muldal, A., Doron, Y., Liu, S., Bohez, S., Merel, J., Erez, T., Lillicrap, T., Heess, N., & Tassa, Y. (2020) · Software Impacts · doi:10.1016/j.simpa.2020.100022 · Cited by 330 · cited 1×
“…the robotics benchmarking tasks discussed in Section 3.1 with image state / goal space instead; moreover, DeepMind Lab [Beattie et al. , 2016], Deepmind Control Suite (DMC) [Tassa et al. , 2020], Atari games [Brockman et al. , 2016] and Minecraft [Johnson et al. , 2016] are also popular testbeds.”
MethodInfluentialDynamics-Aware Unsupervised Discovery of SkillsPreprint
Sharma, A., Gu, S., Levine, S., Kumar, V., & Hausman, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01657 · Cited by 301 · cited 2×
“4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [Eysenbach et al., 2019a; Sharma et al., 2020].”
BackgroundInfluentialA Survey of Reinforcement Learning Informed by Natural Language
Luketina, J., Nardelli, N., Farquhar, G., Foerster, J., Andreas, J., Grefenstette, E., Whiteson, S., & Rocktäschel, T. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/880 · Cited by 279 · cited 1×
“Since our work focus on the essential of GCRL, we will only cover how language forms the goals, and readers may refer to Luketina et al. [2019] for a detailed review of language usage in RL.”
BackgroundAutomatic Goal Generation for Reinforcement Learning AgentsPreprint
Florensa, C., Held, D., Geng, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06366 · Cited by 221 · cited 5×
“Specifically, during a training loop, the agent first receives a desired goal from the environment or it should choose its own goal that is easier to achieve at the current stage [Ren et al., 2019; Florensa et al., 2018].”
BackgroundMethodInfluentialThe Malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T. J., & Bignell, D. E. (2016) · International Joint Conference on Artificial Intelligence · Cited by 205
Efficient Exploratory Learning of Inverse Kinematics on a Bionic Elephant Trunk
Rolf, M., & Steil, J. J. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2287890 · Cited by 170
Universal Planning NetworksPreprint
Srinivas, A., Jabri, A., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00645 · Cited by 163
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal GenerationPreprint
Nair, S., & Finn, C. (2019) · International Conference on Learning Representations · doi:10.48550/arxiv.1909.05829 · Cited by 163 · cited 1×
“, 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al.”
BackgroundAutomatic Curriculum Learning through Value DisagreementPreprint
Zhang, Y.-Z., Abbeel, P., & Pinto, L. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2006.09641 · Cited by 134 · cited 1×
“Moreover, Zhang et al. [2020] treated the epistemic uncertainty of the Q-function as a measurement of the difficulty for reached goals, which is further used for constructing a distribution to sample goals.”
MethodTraining Agents using Upside-Down Reinforcement LearningPreprint
Srivastava, R. K., Shyam, P., Mutz, F. W., Jaśkowski, W., & Schmidhuber, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02877 · Cited by 128 · cited 1×
“Concurrently, Schmidhuber [2019] and Srivastava et al. [2019] also exploited reward-form goals on standard RL problems.”
BackgroundSearch on the Replay Buffer: Bridging Planning and Reinforcement Learning
Eysenbach, B., Salakhutdinov, R., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.05253 · Cited by 123 · cited 3×
“…the circle of reaching a meaningful goal state (we have discussed some examples like reward and commands in Section 3.4) and ask the agents to explore all possible goals through itself, as a trend of developing totally intrinsic skill learning [ Eysenbach et al. , 2019a; Sharma et al. , 2020 ] .”
BackgroundMethodInfluentialUnsupervised Control Through Non-Parametric Discriminative RewardsPreprint
Warde-Farley, D., Van de Wiele, T., Kulkarni, T. D., Ionescu, C., Hansen, S., & Mnih, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11359 · Cited by 122 · cited 1×
“, 2018b] Image X DISCERN [Warde-Farley et al., 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al.”
BackgroundLanguage as an Abstraction for Hierarchical Deep Reinforcement Learning
Jiang, Y., Gu, S., Murphy, K., & Finn, C. (2019) · arXiv · doi:10.48550/arxiv.1906.07343 · Cited by 109
Curriculum-guided Hindsight Experience Replay
Fang, M., Zhou, T., Du, Y., Han, L., & Zhang, Z. (2019) · neural information processing systems · Cited by 102
Learning with AMIGo: Adversarially Motivated Intrinsic GoalsPreprint
Campero, A., Răileanu, R., Küttler, H., Tenenbaum, J. B., Rocktäschel, T., & Grefenstette, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12122 · Cited by 102 · cited 1×
“, 2019] Image X HVF [Nair and Finn, 2020] Image X AMIGo [Campero et al., 2021] Image X GAP [Nair et al.”
BackgroundGRAIL: A Goal-Discovering Robotic Architecture for Intrinsically-Motivated Learning
Santucci, V. G., Baldassarre, G., & Mirolli, M. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2538961 · Cited by 99 · cited 1×
“Technically, goal is a core data structure widely used in many re-lated domains, for describing either achieving or maintaining some condition, e.g., goal reasoning [Roberts et al. , 2018; Niemueller et al. , 2019] and robotics [Rolf and Steil, 2013; Santucci et al. , 2016].”
BackgroundBeattie, C., Leibo, J. Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., Sadik, A., et al. (2016) · arXiv · doi:10.48550/arxiv.1612.03801 · Cited by 68
Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
Colas, C., Karch, T., Lair, N., Dussoux, J.-M., Moulin-Frier, C., Dominey, P. F., & Oudeyer, P.-Y. (2020) · arXiv · doi:10.48550/arxiv.2002.09253 · Cited by 64 · cited 2×
“[Jiang et et al. , 2019 ] , or taken into a recurrent neural network (RNN) as learning the embedding jointly with the RL agent [ Colas et al. , 2020a ] .”
BackgroundMethodLTLf/LDLf Non-Markovian Rewards
Brafman, R. I., Giacomo, G. D., & Patrizi, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11572 · Cited by 62 · cited 2×
“Apart from directions mentioned above, we note that there are more intriguing future directions worth explored in the formulation of GCRL, such as time-extended goals [Forestier and Oudeyer, 2016; Brafman et al., 2018], constraint-based goals [Colas et al.”
BackgroundExploration via Hindsight Goal GenerationPreprint
Ren, Z., Dong, K., Zhou, Y., Liu, Q., & Peng, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04279 · Cited by 61 · cited 4×
“Specifically, during a training loop, the agent first receives a desired goal from the environment or it should choose its own goal that is easier to achieve at the current stage [Ren et al., 2019; Florensa et al., 2018].”
BackgroundMethodInfluentialModular active curiosity-driven discovery of tool use
Forestier, S., & Oudeyer, P. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759584 · Cited by 60 · cited 2×
“Apart from directions mentioned above, we note that there are more intriguing future directions worth explored in the formulation of GCRL, such as time-extended goals [Forestier and Oudeyer, 2016; Brafman et al. , 2018],”
BackgroundMaximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement LearningPreprint
Pitis, S., Chan, H., Zhao, S., Stadie, B. C., & Ba, J. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02832 · Cited by 56
Contextual Imagined Goals for Self-Supervised Robotic LearningPreprint
Nair, A., Bahl, S., Khazatsky, A., Pong, V. H., Berseth, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11670 · Cited by 54
Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped RewardsPreprint
Trott, A., Zheng, S., Xiong, C., & Socher, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01417 · Cited by 50 · cited 5×
“A straightforward idea to prevent the problem is to reshape the reward with a distance measure d between the achieved goal and the desired goal without any domain knowledge [Trott et al. , 2019], i.e., ˜ r g ( s t , a t , g ) = − d ( φ ( s t +1 ) , g ) .”
BackgroundInfluentialReinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to ActionsPreprint
Schmidhuber, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02875 · Cited by 46 · cited 2×
“To solve image tasks, the state and goal images can be directly fed into the policy and value networks and trained in an end-to-end manner, as in Chane-Sane et al. ; Eysen-bach et al. ; Trott et al. [ 2021; 2019b; 2019 ] .”
BackgroundMethodRewriting History with Inverse RL: Hindsight Inference for Policy ImprovementPreprint
Eysenbach, B., Geng, X., Levine, S., & Salakhutdinov, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11089 · Cited by 38 · cited 1×
“Recently, Eysenbach et al. [ 2020 ] revealed the relationship between HER and inverse RL which studies how to recover the reward function on a set of offline expert trajectories.”
BackgroundModel-Based Visual Planning with Self-Supervised Functional DistancesPreprint
Tian, S., Nair, S., Ebert, F., Dasari, S., Eysenbach, B., Finn, C., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15373 · Cited by 37
World Model as a Graph: Learning Latent Landmarks for PlanningPreprint
Zhang, L., Yang, G., & Stadie, B. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.12491 · Cited by 32
Rapid Exploration for Open-World Navigation with Latent Goal ModelsPreprint
Shah, D., Eysenbach, B., Kahn, G., Rhinehart, N., & Levine, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.05859 · Cited by 30 · cited 2×
“…be used to command the robots to handle a variety of tasks with kinds of goal elements, such as a sentence of text description [Ahn et al. , 2022], a goal image of the order of object placement [Chebotar et al. , 2021], a goal image of the targeted area in the open world [Shah et al. , 2021], etc.”
BackgroundMethodACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement LearningPreprint
Chan, H., Wu, Y., Kiros, J., Fidler, S., & Ba, J. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04546 · Cited by 30 · cited 3×
“To train RL algorithms, those words must be either translated into embeddings via pre-training models [Chan et al., 2019], or taken into a recurrent neural network (RNN) as learning the embedding jointly with the RL agent [Colas et al.”
BackgroundMethodInfluentialThe Laplacian in RL: Learning Representations with Efficient ApproximationsPreprint
Wu, Y., Tucker, G., & Nachum, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04586 · Cited by 29 · cited 1×
“, 2017] Vector/Image X Laplacian Representation Learning [Wu et al., 2018] Vector/Image X SR [Trott et al.”
BackgroundReward-Conditioned PoliciesPreprint
Kumar, A., Peng, X. B., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13465 · Cited by 28 · cited 1×
“Kumar et al. [2019] proposed to learn a return-conditioned policy π ( a | s, R ) where R is an estimated return of the whole trajectory.”
BackgroundWeakly-Supervised Reinforcement Learning for Controllable BehaviorPreprint
Lee, L., Eysenbach, B., Salakhutdinov, R., Gu, S., & Finn, C. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2004.02860 · Cited by 28 · cited 2×
“The encoding can also be derived via training a weak supervised model for learning disentangled representation [Lee et al., 2020]; and even an end-to-end supervised imitation learning objective [Srinivas et al.”
MethodGrounding Language to Autonomously-Acquired Skills via Goal GenerationPreprint
Akakzia, A., Colas, C., Oudeyer, P., Chétouani, M., & Sigaud, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07185 · Cited by 27
EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Models
Colas, C., Hejblum, B. P., Rouillon, S., Thiébaut, R., Oudeyer, P.-Y., Moulin-Frier, C., & Prague, M. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12588 · Cited by 26
What Can I Do Here? Learning New Skills by Imagining Visual Affordances
Khazatsky, A., Nair, A., Jing, D., & Levine, S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561692 · Cited by 26 · cited 2×
“A common choice of the encoder is variational auto-encoder (VAE) [ Nair et al. , 2018b; Khazatsky et al. , 2021 ] ; or a context-conditioned VAE [ Nair et al. , 2020a ] for generalizing to different environments.”
BackgroundLearning from Trajectories via Subgoal DiscoveryPreprint
Paul, S., Vanbaar, J., & Roy–Chowdhury, A. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07224 · Cited by 25 · cited 1×
“Such an intuition motivates the work of Paul et al. [2019], which made use of the order of sub-goal indices along the expert trajectory as the evidence of difficulty, and learned to predict the sub-goals in the equipartitions from the expert trajectory conditioned on the given state.”
MethodGoal-Conditioned Reinforcement Learning with Imagined SubgoalsPreprint
Chane-Sane, E., Schmid, C., & Laptev, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00541 · Cited by 24
Discovering and Achieving Goals via World ModelsPreprint
Mendonca, R., Rybkin, O., Daniilidis, K., Hafner, D., & Pathak, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09514 · Cited by 24
Goal-Aware Prediction: Learning to Model What MattersPreprint
Nair, S., Savarese, S., & Finn, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07170 · Cited by 23 · cited 3×
“[2019; 2020; 2021], and also real-world robotics applications [Nair et al., 2018b; Nair et al., 2020b].”
BackgroundMethodPlanGAN: Model-based Planning With Sparse Rewards and Multiple GoalsPreprint
Charlesworth, H. J., & Montana, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.00900 · Cited by 23 · cited 1×
“, 2020] Vector X X PlanGAN [Charlesworth and Montana, 2020] Vector X AIM [Durugkar et al.”
BackgroundGuided goal generation for hindsight multi-goal reinforcement learning
Bai, C., Liu, P., Zhao, W., & Tang, X. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.022 · Cited by 20 · cited 1×
“, 2019] Vector X G-HER [Bai et al., 2019] Vector X Hindsight Planner [Lai et al.”
BackgroundMapGo: Model-Assisted Policy Optimization for Goal-Oriented Tasks
Zhu, M., Liu, M., Shen, J. G., Zhang, Z., Chen, S., Zhang, W., Ye, D., Yu, Y. H., Fu, Q., & Yang, W. P. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/480 · Cited by 19 · cited 3×
“On the other hand, setting the goal as a desired moving speed and ask the cheetah agent to run up at the speed (HalfCheetah) [Zhu et al., 2021]; or a vector of the target position to reach, as is used in AntLocomotion [Florensa et al.”
MethodInfluentialSelf-citationImaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward TasksPreprint
McCarthy, R., Wang, Q., & Redmond, S. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02414 · Cited by 19
Goal Reasoning in the CLIPS Executive for Integrated Planning and Execution
Niemueller, T., Hofmann, T., & Lakemeyer, G. (2019) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v29i1.3544 · Cited by 18
Chitnis, R., Silver, T., Tenenbaum, J. B., Kaelbling, L. P., & Lozano‐Pérez, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17400 · Cited by 18 · cited 1×
“, 2021], composition of goals [Chitnis et al., 2021] etc.”
BackgroundDynamical Distance Learning for Semi-Supervised and Unsupervised Skill DiscoveryPreprint
Hartikainen, K., Geng, X., Haarnoja, T., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.08225 · Cited by 17 · cited 1×
“, 2019] Vector/Image X DDL [Hartikainen et al., 2019] Vector/Image X X SoRB [Eysenbach et al.”
BackgroundIntrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short SurveyPreprint
Colas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2021) · HAL · Cited by 16
Adversarial Intrinsic Motivation for Reinforcement LearningPreprint
Durugkar, I., Tec, M., Niekum, S., & Stone, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13345 · Cited by 14 · cited 2×
“HER is widely used as the backbone in many related works [Ren et al., 2019; Pitis et al., 2020; Durugkar et al., 2021], and researchers also take efforts on extending the idea of HER on designing advanced relabeling strategies.”
MethodKuang, Y., Weinberg, A. I., Vogiatzis, G., & Faria, D. R. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223473 · Cited by 13 · cited 1×
“, 2020] Vector X GDP [Kuang et al., 2020] Vector X VDS [Zhang et al.”
BackgroundSpecial issue on goal reasoning
Roberts, M., Borrajo, D., Cox, M. T., & Yorke‐Smith, N. (2018) · AI Communications · doi:10.3233/aic-180760 · Cited by 9 · cited 1×
“Technically, goal is a core data structure widely used in many re-lated domains, for describing either achieving or maintaining some condition, e.g., goal reasoning [Roberts et al. , 2018; Niemueller et al. , 2019] and robotics [Rolf and Steil, 2013; Santucci et al. , 2016].”
BackgroundColas, C., Karch, T., Sigaud, O., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09830 · Cited by 8
Hindsight Expectation Maximization for Goal-conditioned Reinforcement LearningPreprint
Tang, Y., & Kucukelbir, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07549 · Cited by 5
Research works printed but not linked: 5
- [21]21. Yevgen Chebotar, Karol Hausman, et al. Actionable models: Unsupervised offline reinforce- ment learning of robotic skills. ICML, 2021.
- [42]42. Dibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu, Coline Devin, Benjamin Eysenbach, and Sergey Levine. Learning to reach goals via iterated super- vised learning. ICLR, 2021.
- [76]76. Vitchyr H Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. Skew- fit: State-covering self-supervised reinforcement learning. ICML, 2020.
- [83]83. Schaul et al., 2015] Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function ap- proximators. ICML, 2015.
- [91]91. Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Universal plan- ning networks: Learning generalizable representations for visuomotor control. ICML, 2018.
Cited by
Sorted by citations
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
“Each individual goal g ∼ G can be differentiated by its reward function r ( s t , a t , g ) , hence the objective for GCRL becomes maximizing the expected return over the distribution of goals: J ( π ) := IE ( s t ,a t ) ∼ μ π ,g ∼G [ (cid:5) t γ t r ( s, a, g )] [15].”
BackgroundA survey on model-based reinforcement learning
Luo, F.-M., Xu, T., Lai, H., Chen, X., Zhang, W., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3696-5 · Cited by 205
“Goal-conditioned reinforcement learning (GCRL), also named as goal-oriented RL [Liu et al., 2022], deals with the tasks where agents are expected to achieve different goals [Pitis et al.”
BackgroundSelf-citationWang, J., Qin, Y., Wu, Q., Zeng, D., Gao, X., Wang, Q., Li, Z., Ni, Y., Li, H., Zhang, P., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03724-5 · Cited by 56
Large sequence models for sequential decision-making: a survey
Wen, M., Lin, R., Wang, H., Yang, Y., Wen, Y., Mai, L., Wang, Y.-X., Zhang, H., & Zhang, W. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2689-5 · Cited by 50
“Goal-condition RL [84–86] learns a general policy function to finish a series of simple tasks, for instance, to reach di ff erent goal states.”
BackgroundSelf-citationGrasp Multiple Objects With One Hand
Li, Y., Liu, B., Geng, Y., Li, P., Yang, Y., Zhu, Y., Liu, T., & Huang, S. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3374190 · Cited by 38
“Instead, the entire length and sides of the fingers must be engaged [3], [9], necessitating a carefully configured grasping pose for effective force closure on each object while avoiding collisions.”
Metadata
- Authors
- Liu, M., Zhu, M., & Zhang, W.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
- Published
- 2022-07
- Pages
- 5502-5511
- OpenAlex
- W4285601030
- Cited by
- 115
- References
- 63
- Citations by year
- 2026: 14 · 2025: 47 · 2024: 28 · 2023: 24 · 2022: 2
- FWCI
- 13.96
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076161, 2021SHZDZX0102)
- Shanghai Jiao Tong University (2021SHZDZX0102)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- goal-conditioned reinforcement learning, goal representation, goal-conditioned policies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
