The DOI matches its DataCite record; 44 of the 48 references it cites were found in a registry or an index.
Planning to Explore via Self-Supervised World Models
Open accessAbstract
Reinforcement learning allows solving complex tasks, however, the learning tends to be task-specific and the sample efficiency remains a challenge. We present Plan2Explore, a self-supervised reinforcement learning agent that tackles both these challenges through a new approach to self-supervised exploration and fast adaptation to new tasks, which need not be known during exploration. During exploration, unlike prior methods which retrospectively compute the novelty of observations after the agent has already reached them, our agent acts efficiently by leveraging planning to seek out expected future novelty. After exploration, the agent quickly adapts to multiple downstream tasks in a zero or a few-shot manner. We evaluate on challenging control tasks from high-dimensional image inputs. Without any training supervision or task-specific interaction, Plan2Explore outperforms prior self-supervised exploration methods, and in fact, almost matches the performances oracle which has access to rewards. Videos and code at https://ramanans1.github.io/plan2explore/
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, world models summarize general dynamics knowledge about the environment that, once learned, could be reused for a wide range of downstream tasks (Sekar et al., 2020).”
From DayDreamer: World Models for Physical Robot Learning (2022) · cites this paper as Sekar et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Stochastic Backpropagation and Approximate Inference in Deep Generative ModelsPreprint
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 3,921
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666
On a measure of the information provided by an experiment
Lindley, D. V. (1956) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728069 · Cited by 1,509
Information-based objective functions for active data selection
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.590 · Cited by 1,221
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
Deisenroth, M. P., & Rasmussen, C. E. (2011) · Cambridge University Engineering Department Publications Database · Cited by 1,136
Intrinsic Motivation Systems for Autonomous Mental Development
Oudeyer, P.-Y., Kaplan, F., & Hafner, V. V. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2006.890271 · Cited by 1,126
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837
What is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 1×
“World models summarize past experience into a representation of the environment that enables predicting imagined future sequences (Sutton, 1991; Watter et al., 2015; Ha & Schmidhuber, 2018).”
Background
Research works printed but not linked: 11
- Amos, B., Dinh, L., Cabi, S., Rothrl, T., Muldal, A., Erez, T., Tassa, Y., de Freitas, N., and Denil, M. Learning awareness models. In ICLR , 2018. 8
- Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., and Zaremba, W. Hindsight experience replay. In NIPS , 2017. 8
- Breiman, L. Bagging predictors. Machine learning , 24(2): 123-140, 1996. 4
- Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., and Efros, A. A. Large-scale study of curiosity-driven learning. ICLR , 2019. 8
- Gal, Y. Uncertainty in deep learning. University of Cambridge , 1:3, 2016. 4
- Haber, N., Mrowca, D., Wang, S., Fei-Fei, L. F., and Yamins, D. L. Learning to play with intrinsically-motivated, selfaware agents. In NeurIPS , 2018. 8
- Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013. 3 · arXiv
- Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T. Curiosity-driven exploration by self-supervised prediction. In ICML , 2017. 1, 2, 6, 8, 11
- Pathak, D., Mahmoudieh, P., Luo, G., Agrawal, P., Chen, D., Shentu, Y., Shelhamer, E., Malik, J., Efros, A. A., and Darrell, T. Zero-shot visual imitation. In ICLR , 2018. 8
- Seung, H., Opper, M., and Sompolinsky, H. Query by committee. COLT , 1992. 8
- Watter, M., Springenberg, J., Boedecker, J., and Riedmiller, M. Embed to control: A locally linear latent dynamics model for control from raw images. In NIPS , 2015. 3
Cited by
Sorted by citations
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 871
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3236361 · Cited by 245
Reasoning with Language Model is Planning with World Model
Hao, S., Gu, Y., Ma, H., Hong, J., Wang, Z., Wang, D., & Hu, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.507 · Cited by 168
(2021) · UCL Discovery (University College London) · Cited by 141
Self-Supervised Policy Adaptation during Deployment
Hansen, N., Jangir, R., Sun, Y., Alenyà, G., Abbeel, P., Efros, A. A., Pinto, L., & Wang, X.-L. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2007.04309 · Cited by 212
Self-citationColas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13554 · Cited by 137
Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective
Thiam, P., Hihn, H., Braun, D. A., Kestler, H. A., & Schwenker, F. (2021) · Frontiers in Physiology · doi:10.3389/fphys.2021.720464 · Cited by 75
Reinforcement Learning with Prototypical RepresentationsPreprint
Yarats, D., Fergus, R., Lazaric, A., & Pinto, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.11271 · Cited by 86
Behavior From the Void: Unsupervised Active Pre-TrainingPreprint
Liu, H., & Abbeel, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04551 · Cited by 86
Self-citationSmooth Model Predictive Path Integral Control Without Smoothing
Kim, T., Park, G., Kwak, K., Bae, J., & Lee, W. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3192800 · Cited by 61
Foundation models and intelligent decision-making: Progress, challenges, and perspectives
Huang, J., Xu, Y., Wang, Q., Wang, Q., Liang, X., Wang, F., Zhang, Z., Wei, W., Zhang, B., Huang, L., et al. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100948 · Cited by 87
Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation
Al-Halah, Z., Ramakrishnan, S. K., & Grauman, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01652 · Cited by 58
An Information-Theoretic Perspective on Intrinsic Motivation in Reinforcement Learning: A Survey
Aubret, A., Matignon, L., & Hassas, S. (2023) · Entropy · doi:10.3390/e25020327 · Cited by 52
High-accuracy model-based reinforcement learning, a survey
Plaat, A., Kosters, W. A., & Preuß, M. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10335-w · Cited by 63
A Survey of Traversability Estimation for Mobile Robots
Sevastopoulos, C., & Konstantopoulos, S. (2022) · IEEE Access · doi:10.1109/access.2022.3202545 · Cited by 55
Dreamwalker: Mental Planning for Continuous Vision-Language Navigation
Wang, H., Liang, W. G., Gool, L. V., & Wang, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00998 · Cited by 64
Curiosity-driven exploration: foundations in neuroscience and computational modeling
Modirshanechi, A., Kondrakiewicz, K., Wulfram Gerstner, B., & Haesler, S. (2023) · Trends in Neurosciences · doi:10.1016/j.tins.2023.10.002 · Cited by 50
Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
Okada, M., & Taniguchi, T. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560734 · Cited by 49
Model-based variable impedance learning control for robotic manipulation
Anand, A. S., Gravdahl, J. T., & Abu‐Dakka, F. J. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104531 · Cited by 47
DayDreamer: World Models for Physical Robot Learning
Wu, P., Escontrela, A., Hafner, D., Goldberg, K., & Abbeel, P. (2022) · arXiv · doi:10.48550/arxiv.2206.14176 · Cited by 76
“By predicting future outcomes, world models allow for planning and behavior learning given only small amounts of real world interaction (Gal et al., 2016; Ebert et al., 2018).”
BackgroundSelf-citationThe Free Energy Principle for Perception and Action: A Deep Learning Perspective
Mazzaglia, P., Verbelen, T., Çatal, O., & Dhoedt, B. (2022) · Entropy · doi:10.3390/e24020301 · Cited by 57
Machine learning meets advanced robotic manipulation
Nahavandi, S., Alizadehsani, R., Nahavandi, D., Lim, C. P., Kelly, K., & Bello, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102221 · Cited by 46
Curiosity-Driven Exploration via Latent Bayesian Surprise
Mazzaglia, P., Çatal, O., Verbelen, T., & Dhoedt, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20743 · Cited by 36
Action and Perception as Divergence MinimizationPreprint
Hafner, D., Ortega, P. A., Ba, J. L., Parr, T., Friston, K. J., & Heess, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01791 · Cited by 52
Self-citationDreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction
Okada, M., & Taniguchi, T. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981405 · Cited by 26
Human’s Intuitive Mental Models as a Source of Realistic Artificial Intelligence and Engineering
Suomala, J., & Kauttonen, J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.873289 · Cited by 32
Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learning
Ma, Y. J., Shen, A., Bastani, O., & Dinesh, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20478 · Cited by 24
Learning Periodic Tasks from Human Demonstrations
Yang, J., Zhang, J., Settle, C., Rai, A., Antonova, R., & Bohg, J. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812402 · Cited by 21
Curiosity and Interactive Learning in Artificial Systems
Haber, N. (2022) · Springer eBooks · doi:10.1007/978-3-031-09687-7_3 · Cited by 22
Ocana, J. M. C., Capobianco, R., & Nardi, D. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14390 · Cited by 21
Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learning
Bai, C., Liu, P., Liu, K., Wang, L., Zhao, Y., Han, L., & Wang, Z. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3129160 · Cited by 21
Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitation
Yao, Y., Li, X., An, Z., Zhang, W., & Luo, D. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561842 · Cited by 20
Batch Exploration With Examples for Scalable Robotic Reinforcement Learning
Chen, A. S., Nam, H., Nair, S., & Finn, C. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068655 · Cited by 22
Chitnis, R., Silver, T., Tenenbaum, J. B., Kaelbling, L. P., & Lozano‐Pérez, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17400 · Cited by 18
Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations
Zhao, X., Weber, C., Hafez, M. B., & Wermter, S. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981510 · Cited by 17
Continual Robot Learning Using Self-Supervised Task Inference
Hafez, M. B., & Wermter, S. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3315513 · Cited by 20
Chen, B., Kang, J., Zhong, P., Cui, Y., Lu, S., Liang, Y., & Wang, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3324380 · Cited by 17
Airline dynamic pricing with patient customers using deep exploration-based reinforcement learning
Jo, S., Lee, G. M., & Moon, I. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108073 · Cited by 17
A Unifying Framework for Reinforcement Learning and Planning
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.908353 · Cited by 38
Benchmarking the Spectrum of Agent Capabilities
Hafner, D. (2021) · arXiv · doi:10.48550/arxiv.2109.06780 · Cited by 16
Self-citationSTAP: Sequencing Task-Agnostic Policies
Agia, C., Migimatsu, T., Wu, J., & Bohg, J. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160220 · Cited by 16
Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires
Lim, B., Grillotti, L., Bernasconi, L., & Cully, A. (2021) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra46639.2022.9811559 · Cited by 29
Shielded Planning Guided Data-Efficient and Safe Reinforcement Learning
Wang, H., Qin, J., & Kan, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3359031 · Cited by 16
Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short SurveyPreprint
Colas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2021) · HAL · Cited by 16
Kim, T., Mun, J., Seo, J., Kim, B., & Hong, S. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.086 · Cited by 16
On Optimism in Model-Based Reinforcement Learning.Preprint
Pacchiano, A., Ball, P., Parker-Holder, J., Choromański, K., & Roberts, S. (2020) · arXiv (Cornell University) · Cited by 14
Novelty as a drive of human exploration in complex stochastic environments
Modirshanechi, A., Lin, W.-H., He, X., Herzog, M. H., & Wulfram Gerstner, B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2502193122 · Cited by 19
Deep Model-Based Reinforcement Learning for High-Dimensional Problems, a SurveyPreprint
Plaat, A., Kosters, W. A., & Preuß, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05598 · Cited by 36
LEAF: Latent Exploration Along the Frontier
Bharadhwaj, H., Garg, A., & Shkurti, F. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560922 · Cited by 13
Backprop-Free Reinforcement Learning with Active Neural Generative Coding
Ororbia, A. G., & Mali, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19876 · Cited by 17
Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey.Preprint
Plaat, A., Kosters, W. A., & Preuß, M. (2020) · arXiv (Cornell University) · Cited by 12
Weibull-Open-World (WOW) Multi-Type Novelty Detection in CartPole3D
Boult, T. E., Windesheim, N. M., Zhou, S., Pereyda, C., & Holder, L. B. (2022) · Algorithms · doi:10.3390/a15100381 · Cited by 10
Dropout’s Dream Land: Generalization from Learned Simulators to Reality
Wellmer, Z., & Kwok, J. T.-Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86486-6_16 · Cited by 10
Data-Efficient Task Generalization via Probabilistic Model-Based Meta Reinforcement Learning
Bhardwaj, A., Rothfuss, J., Sukhija, B., As, Y., Hutter, M., Coros, S., & Krause, A. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3371260 · Cited by 9
CarDreamer: Open-Source Learning Platform for World-Model-Based Autonomous Driving
Gao, D., Cai, S., Zhou, H., Wang, H., Soltani, I., & Zhang, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3479088 · Cited by 9
QT-TDM: Planning With Transformer Dynamics Model and Autoregressive Q-Learning
Kotb, M., Weber, C., Hafez, M. B., & Wermter, S. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3504341 · Cited by 10
FOCUS: object-centric world models for robotic manipulation
Ferraro, S., Mazzaglia, P., Verbelen, T., & Dhoedt, B. (2025) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2025.1585386 · Cited by 9
Sampling diversity driven exploration with state difference guidance
Lu, J., Han, S. D., Lü, S., Kang, M., & Zhang, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117418 · Cited by 9
Dreaming: Model-based Reinforcement Learning by Latent Imagination without ReconstructionPreprint
Okada, M., & Taniguchi, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14535 · Cited by 9
EXCALIBUR: Encouraging and Evaluating Embodied Exploration
Zhu, H., Kapoor, R., Min, S. Y., Han, W., Li, J., Geng, K., Neubig, G., Bisk, Y., Kembhavi, A., & Weihs, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01434 · Cited by 9
DynaSTI: Dynamics modeling with sequential temporal information for reinforcement learning in Atari
Kim, J., Lee, Y. J., Kwak, M., Park, Y. J., & Kim, S. B. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112103 · Cited by 8
Dissociable mechanisms of information sampling in prefrontal cortex and the dopaminergic system
Kaanders, P., Juechems, K., O’Reilly, J. X., & Hunt, L. T. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.04.005 · Cited by 8
Active sensing with artificial neural networks
Solopchuk, O., & Zénon, A. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.007 · Cited by 8
Evaluating Agents without RewardsPreprint
Matusch, B., Ba, J. L., & Hafner, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.11538 · Cited by 14
Self-citationA Survey of Demonstration LearningPreprint
Correia, A. R. D. S. P., & Alexandre, L. A. (2023) · Robotics Auton. Syst. · doi:10.2139/ssrn.4390650 · Cited by 56
Exploratory State Representation Learning
Merckling, A., Perrin, N., Coninx, A., & Doncieux, S. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.762051 · Cited by 7
Cartoni, E., Montella, D., Triesch, J., & Baldassarre, G. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3270081 · Cited by 7
Pessimistic value iteration for multi-task data sharing in Offline Reinforcement Learning
Bai, C., Wang, L., Hao, J., Yang, Z., Zhao, B., Wang, Z., & Li, X. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.104048 · Cited by 7
TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer
Yamada, J., Rigter, M. P., Collins, J., & Posner, I. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610450 · Cited by 14
Bayesian Distributional Policy Gradients
Li, L., & Faisal, A. A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17024 · Cited by 7
Addressing Sample Efficiency and Model-bias in Model-based Reinforcement Learning
Anand, A. S., Kveen, J. E., Abu‐Dakka, F. J., Grøtli, E. I., & Gravdahl, J. T. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00009 · Cited by 6
Measuring Progress in Deep Reinforcement Learning Sample Efficiency
Dorner, F. E. (2021) · arXiv · doi:10.48550/arxiv.2102.04881 · Cited by 6
Lu, Y., Hegyi, A., Salomons, A. M., & Wang, H. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121485 · Cited by 8
Model-Based Reinforcement Learning With Isolated Imaginations
Pan, M., Zhu, X., Zheng, Y., Wang, Y., & Yang, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3335263 · Cited by 5
Pre-Grasp Approaching on Mobile Robots: A Pre-Active Layered Approach
Naik, L., Kalkan, S., & Krüger, N. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3358077 · Cited by 5
An Adversarial Objective for Scalable Exploration
Bucher, B., Schmeckpeper, K., Matni, N., & Daniilidis, K. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636298 · Cited by 7
Self-citationBabaeizadeh, M., Saffar, M., Hafner, D., Kannan, H., Finn, C., Levine, S., & Erhan, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.04603 · Cited by 12
Self-citationCuriosity-driven learning in artificial intelligence and its applications
Sun, C. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/172831 · Cited by 6
Neighborhood-Curiosity-Based Exploration in Multiagent Reinforcement Learning
Yang, S., He, Z., Li, J., Shi, H., Ji, Q., Hwang, K., & Li, X. (2024) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2024.3460368 · Cited by 4
ALAN: Autonomously Exploring Robotic Agents in the Real World
Mendonca, R., Bahl, S., & Pathak, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161016 · Cited by 19
Self-citationImproving the Diversity of Bootstrapped DQN by Replacing Priors With Noise
Meng, L., Goodwin, M., Yazidi, A., & Engelstad, P. E. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2022.3185330 · Cited by 4
Active Exploration for Robotic Manipulation
Schneider, T., Belousov, B., Chalvatzaki, G., Romeres, D., Jha, D. K., & Peters, J. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982061 · Cited by 5
Sample-Efficient Real-Time Planning with Curiosity Cross-Entropy Method and Contrastive Learning
Kotb, M., Weber, C., & Wermter, S. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342018 · Cited by 6
SEQUEL: Semi-Supervised Preference-based RL with Query Synthesis via Latent Interpolation
Marta, D., Holk, S., Pek, C., & Leite, I. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610534 · Cited by 4
Skill Transfer via Partially Amortized Hierarchical PlanningPreprint
Xie, K., Bharadhwaj, H., Hafner, D., Garg, A., & Shkurti, F. (2020) · arXiv (Cornell University) · Cited by 4
Self-citationTask-Prompt Generalised World Model in Multi-Environment Offline Reinforcement Learning
Xiong, X., Meng, L., Ruan, J., Zhang, Q., Li, G., Xing, D., & Xu, B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230588 · Cited by 3
Sun, Z., Yuan, X., Xu, Q., Pang, B. J., Song, Y. H., Song, R., & Li, Y. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3495775 · Cited by 3
Within the scope of prediction: Shaping intrinsic rewards via evaluating uncertainty
Zhou, X., Zhu, F., & Zhao, P. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117775 · Cited by 3
Model-Based Reinforcement Learning
Plaat, A. (2022) · Springer eBooks · doi:10.1007/978-981-19-0638-1_5 · Cited by 3
Stabilizing Visual Reinforcement Learning via Asymmetric Interactive Cooperation
Zhai, Y., Peng, P., Zhao, Y., Huang, Y., & Tian, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00026 · Cited by 3
Haptics-based Curiosity for Sparse-reward Tasks
Rajeswar, S., Ibrahim, C., Surya, N., Golemo, F., Vázquez, D., Courville, A. C., & Pinheiro, P. O. (2021) · Conference on Robot Learning · Cited by 3
Advances in Optimizing Physically Simulated Movements
Babadi, A. (2022) · Aaltodoc (Aalto University) · Cited by 3
Huang, Y., Peng, P., Zhao, Y., Zhai, Y., Xu, H., & Tian, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00023 · Cited by 2
OPEn: An Open-ended Physics Environment for Learning Without a Task
Gan, C., Bhandwaldar, A., Torralba, A. B., Tenenbaum, J. B., & Isola, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636830 · Cited by 3
Predicting before acting: improving policy quality by taking a vision of consequence
Zhou, X., Zhu, F., & Zhao, P. (2022) · Connection Science · doi:10.1080/09540091.2022.2025765 · Cited by 2
Learning Compositional Neural Programs for Continuous ControlPreprint
Pierrot, T., Perrin, N., Behbahani, F., Laterre, A., Sigaud, O., Beguir, K., & de Freitas, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13363 · Cited by 4
Online Safety Assurance for Deep Reinforcement LearningPreprint
Rotman, N. H., Schapira, M., & Tamar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03625 · Cited by 2
Deep Model Predictive Variable Impedance ControlPreprint
Anand, A. S., Abu‐Dakka, F. J., & Gravdahl, J. T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.09614 · Cited by 2
Metadata
- Authors
- Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., & Pathak, D.
- Affiliations
- University of Pennsylvania (US)
- University of California, Berkeley (US)
- Google (United States) (US)
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- Pages
- 8583-8592
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2005.05960
- OpenAlex
- W3025660841
- Cited by
- 129
- References
- 48
- Citations by year
- 2025: 7 · 2024: 25 · 2023: 29 · 2022: 29 · 2021: 24 · 2020: 15
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Funders
- National Science Foundation (1703319)
- Defense Advanced Research Projects Agency (1703319)
- Office of Naval Research (ONR-N00014, N00014-17-1, W911NF-17-2-0181, N00014-17-1-2093, ONR N00014-17-1-2093, N00014, N00014-17)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- world models, planning, novelty seeking, few-shot adaptation, zero-shot transfer, sample efficiency, visual servoing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
