Habitat 2.0: Training home assistants to rearrange their habitat
Open accessAbstract
We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions to all levels of the embodied AI stack - data, simulation, and benchmark tasks. Specifically, we present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850x real-time) on an 8-GPU node, representing 100x speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, prepare groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from 'hand-off problems', and (3) SPA pipelines are more brittle than RL policies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Shifting the responsibility of mobility into the subpolicies ensures that the subpolicies can start from a large set of initial positions, resolving the 'hand-off' problem and simplifying the learning process for the high-level policy.
“This ensures that the subpolicies can start from a large set of initial positions, resolving the”
From Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation (2023) · cites this paper as [9]
Appendix F.2 (pp. 25–27) describes a separate Navigate skill that brings the robot to the starting position of other skills; Pick and Place instead start facing their targets with only small position and orientation variations. Section 6.2 (around p. 12) says learning navigation to chain skills remains challenging. The paper therefore does not describe shifting mobility into the manipulation subpolicies to resolve the hand-off problem. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredPrevious work mainly relies on 3D artists to design environments, which is hard to scale up, or constructs scenes from 3D scans to reduce human labor, but scenes are less interactive.
“Previous work mainly relies on 3D artists to design the environments [6, 13, 2527, 40, 56], which is hard to scale up or construct scenes from 3D scans [43, 45, 48] to reduce human labor, but scenes are less interactive.”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [43]
Section 3 (p. 4) says static 3D scans cannot be moved or manipulated, supporting the claim that scanned scenes are less interactive. The paper also reports 900+ person-hours of artist effort for ReplicaCAD, but it does not establish that previous work mainly relies on artists or that scanning is used specifically to reduce human labor. (LLM, conf=0.82)
Experimental — accuracy not yet measuredNaive skill-chaining of arbitrary skills often results in 'hand-off' failures in which the subsequent skill cannot start from the current state.
“While naive skill-chaining of arbitrary skills often results in”
From Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation (2023) · cites this paper as [9]
Appendix F.2 (p. 25) says skills must return the arm to a resting position to facilitate transitions, and its Navigate description (p. 27) says navigation brings the robot to the start of other skills. These passages support the concern that a subsequent skill needs a suitable starting state, but they do not establish that naive chaining of arbitrary skills often produces hand-off failures. (LLM, conf=0.79)
Experimental — accuracy not yet measuredHousehold rearrangement benchmarks such as Habitat 2.0 HAB, TDW Transport, and SAPIEN ManiSkill support realistic action execution and accurate physics simulation for rigid bodies, but only include a handful of tasks.
“On the other hand, household rearrangement benchmarks such as Habitat 2.0 HAB [26], TDW Transport [19], and SAPIEN ManiSkill [54, 55] support realistic action execution and accurate physics simulation for rigid bodies, but only include a handful of tasks.”
From BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation (2024) · cites this paper as [26]
The H2.0 description and simulator comparison table support the rigid-body-physics claim, and Section 6 (p. 9) describes three HAB task families. The action-space description supports motor-controlled arm execution, but it also says grasping is abstracted: nearby objects snap into the gripper and cannot slip. Section 4.1 further says the robot base moves kinematically rather than through simulated wheel-ground contact, so the unqualified claim of realistic action execution overstates this source. (LLM, conf=0.83)
Experimental — accuracy not yet measuredThe tasks and activities in existing simulation benchmarks are designed by researchers, and it remains unclear if they address actual human needs.
“Inspiring as they are, the tasks and activities in those benchmarks are designed by researchers; it remains unclear if they are addressing the actual needs of humans.”
From BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation (2024) · cites this paper as [26]
In the conclusion (p. 12), the authors say they defined the benchmark’s rearrangement tasks, supporting the point that researchers designed these tasks. Section 6 (p. 9) calls them common household assistive tasks, but the paper does not say it is unclear whether they address actual human needs or make that claim about existing benchmarks generally. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Some form of action-space abstraction must be necessary to solve long-horizon activities of BEHAVIOR-1K.
“This supports the idea that some form of action-space abstraction must be necessary to solve long-horizon activities of BEHA VIOR-1K, as others reported [27, 26, 69].”
From BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation (2024) · cites this paper as [26]
In §6.2 (p. 11), the paper reports that its monolithic RL approach performed poorly on long-horizon Home Assistant Benchmark tasks and that chaining skills without an architectural bias remained out of reach. It studies a hierarchical planner with parameterized skills, but does not establish that action-space abstraction is necessary or test BEHAVIOR-1K activities. (LLM, conf=0.86)
Experimental — accuracy not yet measuredExisting objects in HM3D scene are not interactive in Habitat-sim.
“However, existing objects in HM3D scene are not interactive in Habitat-sim (Szot et al., 2021).”
From 3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model (2025) · cites this paper as Szot et al., 2021
The simulator comparison table describes earlier Habitat as using 3D scans with no physics support, consistent with scanned objects not being interactive. But the paper does not identify HM3D scenes or specifically state that their existing objects are non-interactive in Habitat-sim. (LLM, conf=0.82)
Experimental — accuracy not yet measuredRoboCasa supports photorealistic rendering and multiple robot embodiments.
“RoboCasa is one of the few frameworks to feature photorealistic rendering [21, 40, 27, 10, 6] and multiple robot embodiments [21, 26, 27, 6, 34].”
From RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots (2024) · cites this paper as [40]
The introduction describes Habitat 2.0 as a photorealistic 3D simulator, and Table 1 lists its rendering as “3D scans + PBR shading.” Section 5.1 (p. 7) describes a Fetch robot as the agent embodiment, but the passages found do not establish support for multiple robot embodiments. The source therefore supports the rendering part of the claim, not the multiple-embodiments part. (LLM, conf=0.83)
Experimental — accuracy not yet measuredExisting Embodied AI environments are typically crafted through manual design, 3D scanning, or procedurally generated with hard-coded rules.
“Existing Embodied AI environments are typically crafted through manual design [6, 13, 26, 27], 3D scanning [8, 43, 45], or procedurally generated with hard-coded rules [7].”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [43]
The efficacy of the proposed agent is thoroughly evaluated across various dimensions on two widely-used platforms, ShapeNet and Habitat.
“The efficacy of our agent is thoroughly evaluated across various dimensions on two widely-used platforms, namely ShapeNet [5] and Habitat [40].”
From Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations (2024) · cites this paper as [40]
Existing ObjectNav benchmarks consider only household environments and support a very limited collection of object types (16 object types in total).
“As existing benchmarks [6, 7, 43] for ObjectNav consider only household environments and support a very limited collection of object types (16 object types in total combining the above benchmarks), we introduce NOVELTYTHOR, an artist-designed benchmark to evaluate embodied agents in diverse environments.”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [43]
Section 6 (p. 9) describes a Home Assistant Benchmark of household manipulation tasks, not an ObjectNav benchmark. Searches for ObjectNav and the claimed total of 16 object types found no passage assessing the environments or combined object types of existing ObjectNav benchmarks; the paper’s 92 ReplicaCAD objects refer to a different dataset. (LLM, conf=0.73)
Experimental — accuracy not yet measuredFor handle detection and grasp-pose detection, a retrained YOLO model on a public handle-specific dataset achieves accurate results.
“Finally, for handle detection and grasp-pose detection, [9] achieve accurate results with a retrained YOLO model on a public handle-specific dataset.”
From Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation (2023) · cites this paper as [9]
Section 5.1 (p. 7) describes a robot picking up objects, and the paper describes an abstracted grasp action rather than results for handle detection or grasp-pose detection. Searches for YOLO, handle detection, grasp-pose detection, and a public handle-specific dataset found no passage supporting the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 83 · Cited by 143
Research works printed but not linked: 82
- Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797, 2019. · arXiv
- [3]Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A Platform for Embodied AI Research. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9339–9347, 2019.
- [7]Chuang Gan, Jeremy Schwartz, Seth Alter, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, et al. ThreeDWorld: A platform for interactive multi-modal physical simulation. arXiv preprint arXiv:2007.04954, 2020. · arXiv
- [10]Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757, 2018. · arXiv
- [11]Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. DD-PPO: Learning near-perfect pointgoal navigators from 2.5 billion frames. In International Conference on Learning Representations (ICLR), 2020.
- [12]Erik Wijmans, Irfan Essa, and Dhruv Batra. How to train pointgoal navigation agents on a (sample and compute) budget. arXiv preprint arXiv:2012.06117, 2020. · arXiv
- [13]Joel Ye, Dhruv Batra, Erik Wijmans, and Abhishek Das. Auxiliary tasks speed up learning pointgoal navigation. arXiv preprint arXiv:2007.04561, 2020. · arXiv
- [14]Yilun Du, Chuang Gan, and Phillip Isola. Curious representation learning for embodied intelligence. arXiv preprint arXiv:2105.01060, 2021. · arXiv
- [15]Peter Karkus, Shaojun Cai, and David Hsu. Differentiable slam-net: Learning particle slam for visual navigation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
- [16]Claudia Pérez-D'Arpino, Can Liu, Patrick Goebel, Roberto Martín-Martín, and Silvio Savarese. Robot navigation in constrained pedestrian environments using reinforcement learning. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2021.
- [17]Santhosh K. Ramakrishnan, Ziad Al-Halah, and Kristen Grauman. Occupancy anticipation for efficient exploration and navigation. In ECCV, 2020.
- [18]Somil Bansal, Varun Tolani, Saurabh Gupta, Jitendra Malik, and Claire Tomlin. Combining optimal control and learning for visual navigation in novel environments. In Conference on Robot Learning (CoRL), 2019.
- [19]Fei Xia, Chengshu Li, Roberto Martín-Martín, Or Litany, Alexander Toshev, and Silvio Savarese. Relmo- gen: Leveraging motion generation in reinforcement learning for mobile manipulation. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2021.
- [20]Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.13171, 2020. · arXiv
- [21]Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding. arXiv preprint arXiv:2010.07954, 2020. · arXiv
- [22]Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3674–3683, 2018.
- [23]Adithyavairavan Murali, Arsalan Mousavian, Clemens Eppner, Chris Paxton, and Dieter Fox. 6-dof grasping for target-driven object manipulation in clutter. In 2020 IEEE International Conference on http://fetchrobotics.com/ https://www.franka.de https://www.unitree.com
- Robotics and Automation (ICRA), pages 6232–6238. IEEE, 2020.
- [24]Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic. Data-driven grasp synthesis—a survey. IEEE Transactions on Robotics, 30(2):289–309, 2013.
- Danica Kragic. Hierarchical fingertip space: A unified framework for grasp planning and in-hand grasp adaptation. IEEE Transactions on robotics, 32(4):960–972, 2016.
- [26]Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kem- bhavi, and Roozbeh Mottaghi. ManipulaTHOR: A framework for visual object manipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
- [27]Robin R Murphy. Introduction to AI robotics. MIT press, 2019.
- 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012.
- Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016. · arXiv
- An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, 2013.
- Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018. · arXiv
- [35]Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019–3026, 2020.
- [36]Bokui Shen, Fei Xia, Chengshu Li, Roberto Martın-Martın, Linxi Fan, Guanzhi Wang, Shyamal Buch, Claudia D'Arpino, Sanjana Srivastava, Lyne P Tchapmi, Kent Vainio, Li Fei-Fei, and Silvio Savarese. iGibson, a simulation environment for interactive tasks in large realistic scenes. arXiv preprint, 2020.
- [38]Jeongseok Lee, Michael X Grey, Sehoon Ha, Tobias Kunz, Sumit Jain, Yuting Ye, Siddhartha S Srinivasa, Mike Stilman, and C Karen Liu. Dart: Dynamic animation and robotics toolkit. Journal of Open Source Software, 3(22):500, 2018.
- [42]Hammad Mazhar, Toby Heyn, Arman Pazouki, Dan Melanz, Andrew Seidl, Aaron Bartholomew, Alessan- dro Tasora, and Dan Negrut. CHRONO: A parallel multi-physics library for rigid-body, flexible-body, and fluid dynamics. Mechanical Sciences, 4:49–64, 02 2013. doi: 10.5194/ms-4-49-2013. URL https://projectchrono.org/. · DOI
- 2019. URL https://arxiv.org/abs/1906.11633. · arXiv
- Multimodal indoor simulator for navigation in complex environments. arXiv:1712.03931, 2017. · arXiv
- [49]Yi Wu, Yuxin Wu, Georgia Gkioxari, and Yuandong Tian. Building generalizable agents with a realistic and rich 3d environment. arXiv preprint arXiv:1801.02209, 2018. · arXiv
- Cornell house agent learning environment. arXiv preprint arXiv:1801.07357, 2018. · arXiv
- VirtualHome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8494–8502, 2018.
- [52]Fei Xia, William B Shen, Chengshu Li, Priya Kasimbeg, Micael Edmond Tchapmi, Alexander Toshev, Roberto Martín-Martín, and Silvio Savarese. Interactive gibson benchmark: A benchmark for interactive navigation in cluttered environments. IEEE Robotics and Automation Letters, 5(2):713–720, 2020.
- [54]Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Integrated task and motion planning. arXiv preprint arXiv:2010.01083, 2020. · arXiv
- [55]Caelan Reed Garrett, Chris Paxton, Tomás Lozano-Pérez, Leslie Pack Kaelbling, and Dieter Fox. Online replanning in belief space for partially observable task and motion problems. In IEEE International Conference on Robotics and Automation (ICRA), 2020.
- [56]Dipendra Misra, Andrew Bennett, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi. Mapping instructions to actions in 3d environments with visual goal prediction. arXiv preprint arXiv:1809.00786, 2018. · arXiv
- [57]Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740–10749, 2020.
- [58]Luca Weihs, Matt Deitke, Aniruddha Kembhavi, and Roozbeh Mottaghi. Visual room rearrangement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
- [60]Abhishek Kadian, Joanne Truong, Aaron Gokaslan, Alexander Clegg, Erik Wijmans, Stefan Lee, Manolis Savva, Sonia Chernova, and Dhruv Batra. Sim2real predictivity: Does evaluation in simulation predict real-world performance? IEEE Robotics and Automation Letters, 5(4):6670–6677, 2020.
- [61]Simon Thorpe, Denis Fize, and Catherine Marlot. Speed of processing in the human visual system. nature, 381(6582):520–522, 1996.
- [62]Sandeep Singh Sandha, Luis Garcia, Bharathan Balaji, Fatima M Anwar, and Mani Srivastava. Sim2real transfer for deep reinforcement learning with stochastic state transition delays. CoRL 2020, 2020.
- [63]Gabriel Dulac-Arnold, Nir Levine, Daniel J. Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. An empirical investigation of the challenges of real-world reinforcement learning. arXiv preprint, 2020.
- [66]Ioan A Sucan, Mark Moll, and Lydia E Kavraki. The open motion planning library. IEEE Robotics & Automation Magazine, 19(4):72–82, 2012.
- [67]Steven M LaValle. Planning algorithms. Cambridge university press, 2006.
- [69]Richard Sahala Hartanto, Ryoichi Ishikawa, Menandro Roxas, and Takeshi Oishi. Hand-motion-guided articulation and segmentation estimation. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 807–813. IEEE, 2020.
- [70]Dmitry Berenson, Siddhartha Srinivasa, and James Kuffner. Task space regions: A framework for pose- constrained manipulation planning. The International Journal of Robotics Research, 30(12):1435–1460, 2011.
- [71]Felix Burget, Armin Hornung, and Maren Bennewitz. Whole-body motion planning for manipulation of articulated objects. In 2013 IEEE International Conference on Robotics and Automation, pages 1656–1662. IEEE, 2013.
- [72]Zachary Kingston, Mark Moll, and Lydia E Kavraki. Sampling-based methods for motion planning with constraints. Annual review of control, robotics, and autonomous systems, 1:159–185, 2018.
- [73]Wim Meeussen, Melonee Wise, Stuart Glaser, Sachin Chitta, Conor McGann, Patrick Mihelich, Eitan Marder-Eppstein, Marius Muja, Victor Eruhimov, Tully Foote, et al. Autonomous door opening and plugging in with a personal robot. In 2010 IEEE International Conference on Robotics and Automation, pages 729–736. IEEE, 2010.
- [74]Advait Jain and Charles C Kemp. Pulling open doors and drawers: Coordinating an omni-directional base and a compliant arm with equilibrium point control. In 2010 IEEE International Conference on Robotics and Automation, pages 1807–1814. IEEE, 2010.
- [75]Steven Dalton, Iuri Frosio, and Michael Garland. Accelerating reinforcement learning through gpu atari emulation. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
- [76]Adam Stooke and Pieter Abbeel. rlpyt: A research code base for deep reinforcement learning in pytorch. arXiv preprint arXiv:1909.01500, 2019. · arXiv
- [77]Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, et al. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. In International Conference on Machine Learning, pages 1407–1416. PMLR, 2018. https://www.pnas.org/content/118/1/e1907856118 https://www.pnas.org/content/118/1/e1907856118 https://github.com/BinomialLLC/basis_universal
- [78]Lasse Espeholt, Raphaël Marinier, Piotr Stanczyk, Ke Wang, and Marcin Michalski. Seed rl: Scalable and efficient deep-rl with accelerated central inference. arXiv preprint arXiv:1910.06591, 2019. · arXiv
- [79]Aleksei Petrenko, Zhehui Huang, Tushar Kumar, Gaurav Sukhatme, and Vladlen Koltun. Sample factory: Egocentric 3D control from pixels at 100000 FPS with asynchronous reinforcement learning. In International Conference on Machine Learning, pages 7652–7662. PMLR, 2020.
- [80]Brennan Shacklett, Erik Wijmans, Aleksei Petrenko, Manolis Savva, Dhruv Batra, Vladlen Koltun, and Kayvon Fatahalian. Large batch simulation for deep reinforcement learning. In International Conference on Learning Representations (ICLR), 2021. URL https://openreview.net/forum? id=cP5IcoAkfKa.
- [82]Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. · arXiv
- [83]David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the barrier to entry of complex robotic software: a moveit! case study. arXiv preprint arXiv:1404.3785, 2014. · arXiv
- [84]James J Kuffner and Steven M LaValle. Rrt-connect: An efficient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 2, pages 995–1001. IEEE, 2000.
- [85]Yoshiaki Kuwata, Gaston A Fiore, Justin Teo, Emilio Frazzoli, and Jonathan P How. Motion planning for urban driving using rrt. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1681–1686. IEEE, 2008.
- [86]Nathan Ratliff, Matt Zucker, J Andrew Bagnell, and Siddhartha Srinivasa. Chomp: Gradient optimization techniques for efficient motion planning. In 2009 IEEE International Conference on Robotics and Automation, pages 489–494. IEEE, 2009.
- [87]John Schulman, Yan Duan, Jonathan Ho, Alex Lee, Ibrahim Awwal, Henry Bradlow, Jia Pan, Sachin Patil, Ken Goldberg, and Pieter Abbeel. Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research, 33(9):1251–1270, 2014.
- [88]Carlos Hernandez, Mukunda Bharatheesha, Wilson Ko, Hans Gaiser, Jethro Tan, Kanter van Deurzen, Maarten de Vries, Bas Van Mil, Jeff van Egmond, Ruben Burger, et al. Team delft's robot winner of the amazon picking challenge 2016. In Robot World Cup, pages 613–624. Springer, 2016.
- [89]Mustafa Mukadam, Jing Dong, Xinyan Yan, Frank Dellaert, and Byron Boots. Continuous-time gaussian process motion planning via probabilistic inference. The International Journal of Robotics Research, 37 (11):1319–1340, 2018.
- [90]Brian Ichter, James Harrison, and Marco Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7087–7094. IEEE, 2018.
- [91]Brian Hou, Sanjiban Choudhury, Gilwoo Lee, Aditya Mandalika, and Siddhartha S Srinivasa. Posterior sampling for anytime motion planning on graphs with expensive-to-evaluate edges. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4266–4272. IEEE, 2020.
- [92]Fahad Islam, Chris Paxton, Clemens Eppner, Bryan Peele, Maxim Likhachev, and Dieter Fox. Alternative paths planner (app) for provably fixed-time manipulation planning in semi-structured environments. arXiv preprint arXiv:2012.14970, 2020. · arXiv
- [93]Michael Pantic, Lionel Ott, Cesar Cadena, Roland Siegwart, and Juan Nieto. Mesh manifold based riemannian motion planning for omnidirectional micro aerial vehicles. arXiv preprint arXiv:2102.10313, 2021. · arXiv
- [94]Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Informed rrt*: Optimal sampling- based path planning focused via direct sampling of an admissible ellipsoidal heuristic. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2997–3004. IEEE, 2014.
- [95]Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Batch informed trees (bit*): Sampling-based optimal planning via the heuristically guided search of implicit random geometric graphs. In 2015 IEEE international conference on robotics and automation (ICRA), pages 3067–3074. IEEE, 2015.
- [96]Daniel Kappler, Franziska Meier, Jan Issac, Jim Mainprice, Cristina Garcia Cifuentes, Manuel Wüthrich, Vincent Berenz, Stefan Schaal, Nathan Ratliff, and Jeannette Bohg. Real-time perception meets reactive motion generation. IEEE Robotics and Automation Letters, 3(3):1864–1871, 2018.
- [97]Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. Ros: an open-source robot operating system. In ICRA workshop on open source software, volume 3, page 5. Kobe, Japan, 2009.
- [98]Aleksandra Faust, Kenneth Oslund, Oscar Ramirez, Anthony Francis, Lydia Tapia, Marek Fiser, and James Davidson. Prm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 5113–5120. IEEE, 2018.
- [99]Mohak Bhardwaj, Byron Boots, and Mustafa Mukadam. Differentiable gaussian process motion planning. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 10598–10604. IEEE, 2020. https://openreview.net/forum?id=cP5IcoAkfKa https://openreview.net/forum?id=cP5IcoAkfKa
- [100]Naoki Yokoyama, Sehoon Ha, and Dhruv Batra. Success weighted by completion time: A dynamics-aware evaluation criteria for embodied navigation. arXiv preprint arXiv:2103.08022, 2021. · arXiv
- [101]Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. In International Conference on Learning Representations (ICLR), 2021.
- [102]Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
- [103]Akanksha Atrey, Kaleigh Clary, and David Jensen. Exploratory not explanatory: Counterfactual analysis of saliency maps for deep reinforcement learning. In International Conference on Learning Representa- tions (ICLR), 2020. URL https://openreview.net/forum?id=rkl3m1BFDB.
- [104]Julius Adebayo, Justin Gilmer, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Conference on Neural Information Processing Systems (NeurIPS), 2018. https://openreview.net/forum?id=rkl3m1BFDB
- Authors
- Szot, A., Clegg, A., Undersander, E., Wijmans, E., Zhao, Y., Turner, J., Maestre, N., Mukadam, M., Chaplot, D., Maksymets, O., Gokaslan, A., Vondrus, V., Dharur, S., Meier, F., Galuba, W., Chang, A., Kira, Z., Koltun, V., Malik, J., Savva, M., & Batra, D.
- Affiliations
- University of Southern California (US)
- Georgia Institute of Technology (US)
- Meta (Israel) (IL)
- Brown University (US)
- University of Illinois Urbana-Champaign (US)
- Intel (United States) (US)
- Simon Fraser University (CA)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Volume
- 34
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2106.14405
- OpenAlex
- W3175074116
- Cited by
- 143
- References
- 83
- Citations by year
- 2025: 5 · 2024: 34 · 2023: 59 · 2022: 38 · 2021: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Engineering › Control and Systems Engineering › Robotics and Automated Systems
- Keywords
- assistive robots, embodied AI, three-dimensional simulation, physics simulation, mobile manipulation, deep reinforcement learning, hierarchical reinforcement learning, long-horizon tasks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review