Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
A kitchen assistant needs to operate human-scale objects, such as cabinets and ovens, in unmapped environments with dynamic obstacles. Autonomous interactions in such environments require integrating dexterous manipulation and fluid mobility. While mobile manipulators in different form factors provide an extended workspace, their real-world adoption has been limited. Executing a high-level task for general objects requires a perceptual understanding of the object as well as adaptive whole-body control among dynamic obstacles. In this paper, we propose a two-stage architecture for autonomous interaction with large articulated objects in unknown environments. The first stage, object-centric planner, only focuses on the object to provide an action-conditional sequence of states for manipulation using RGB-D data. The second stage, agent-centric planner, formulates the whole-body motion control as an optimal control problem that ensures safe tracking of the generated plan, even in scenes with moving obstacles. We show that the proposed pipeline can handle complex static and dynamic kitchen settings for both wheel-based and legged mobile manipulators. Compared to other agent-centric planners, our proposed planner achieves a higher success rate and a lower execution time. We also perform hardware tests on a legged mobile manipulator to interact with various articulated objects in a kitchen. For additional material, please check: www.pair.toronto.edularticulated-mm/.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SAPIEN: A SimulAted Part-Based Interactive ENvironment
Xiang, F., Qin, Y., Mo, K., Xia, Y., Zhu, H., Liu, F., Liu, M., Jiang, H., Yuan, Y., Wang, H., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01111 · Cited by 406
Task Space Regions: A framework for pose-constrained manipulation planning
Berenson, D., Srinivasa, S., & Kuffner, J. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364910396389 · Cited by 368
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis
Seichter, D., Kohler, M., Lewandowski, B., Wengefeld, T., & Gross, H.-M. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561675 · Cited by 330 · cited 1×
“to recent advances in computer vision, progress in semantic scene segmentation [31], [32] and grasp identification [21], [33] can be leveraged to lift some of these assumptions.”
BackgroundCategory-Level Articulated Object Pose Estimation
Li, X., Wang, H., Yi, L., Guibas, L. J., Abbott, A. L., & Song, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00376 · Cited by 196
Where2Act: From Pixels to Actions for Articulated 3D Objects
Mo, K., Guibas, L., Mukadam, M., Gupta, A., & Tulsiani, S. (2021) · arXiv · doi:10.48550/arxiv.2101.02692 · Cited by 132
Shape2Motion: Joint Analysis of Motion Parts and Attributes From 3D Shapes
Wang, X., Zhou, B., Shi, Y., Chen, X., Zhao, Q., & Xu, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00908 · Cited by 104 · cited 2×
“For the three categories considered in this work, the authors used CAD models from Shape2Motion [18] and PartNet [19] datasets to generate synthetic datasets from PyBullet rendering.”
MethodActive articulation model estimation through interactive perception
Hausman, K., Niekum, S., Osentoski, S., & Sukhatme, G. S. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139655 · Cited by 77
ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory
Jain, A., Lioutikov, R., Chuck, C., & Niekum, S. (2020) · arXiv · doi:10.48550/arxiv.2008.10518 · Cited by 63
An integrated approach to visual perception of articulated objects
Martin-Martin, R., Hofer, S., & Brock, O. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487714 · Cited by 41
Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation
Wong, J., Tung, A., Kurenkov, A., Mandlekar, A., Fei-Fei, L., Savarese, S., & Martín-Martín, R. (2021) · arXiv · doi:10.48550/arxiv.2112.05251 · Cited by 19 · cited 1×
“Imitation learning leverages demonstrations from an expert or a teacher [2], [26].”
Background
Cited by
Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments
Mittal, M., Yu, C., Yu, Q., Liu, J., Rudin, N., Hoeller, D., Yuan, J. L., Singh, R., Guo, Y., Mazhar, H., et al. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3270034 · Cited by 260
Self-citationManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
Mu, T., Ling, Z., Xiang, F., Yang, D., Li, X., Tao, S., Huang, Z., Jia, Z., & Su, H. (2021) · arXiv · doi:10.48550/arxiv.2107.14483 · Cited by 34
“For example, works such as [57, 58, 59] identify articulated objects’ kinematic constraints, parts, and poses through neural networks.”
BackgroundHonerkamp, D., Welschehold, T., & Valada, A. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3284346 · Cited by 30
Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation
Wong, J., Tung, A., Kurenkov, A., Mandlekar, A., Fei-Fei, L., Savarese, S., & Martín-Martín, R. (2021) · arXiv · doi:10.48550/arxiv.2112.05251 · Cited by 19
“Autonomous Mobile Manipulation: Algorithms for autonomous MM have been studied for decades [36, 37], and have primarily been addressed with either control [38, 39, 40, 41] or Task and motion planning (TaMP) [3, 42, 43, 44] approaches.”
Method
Metadata
- Authors
- Mittal, M., Hoeller, D., Farshidian, F., Hutter, M., & Garg, A.
- Venue
- 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2022-10-23
- Pages
- 1647-1654
- OpenAlex
- W3138392552
- Cited by
- 86
- References
- 10
- FWCI
- 28.71
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation. https://app.cochat.ai/papers/10.1109/iros47612.2022.9981779
- The paper
- Mittal, M., Hoeller, D., Farshidian, F., Hutter, M., & Garg, A. (2022). Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). https://doi.org/10.1109/iros47612.2022.9981779
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.