The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects
Open accessAbstract
We demonstrate model-based, visual robot manipulation of deformable linear objects. Our approach is based on a state-space representation of the physical system that the robot aims to control. This choice has multiple advantages, including the ease of incorporating physics priors in the dynamics model and perception model, and the ease of planning manipulation actions. In addition, physical states can naturally represent object instances of different appearances. Therefore, dynamics in the state space can be learned in one setting and directly used in other visually different settings. This is in contrast to dynamics learned in pixel space or latent space, where generalization to visual differences are not guaranteed. Challenges in taking the state-space approach are the estimation of the high-dimensional state of a deformable object from raw images, where annotations are very expensive on real data, and finding a dynamics model that is both accurate, generalizable, and efficient to compute. We are the first to demonstrate self-supervised training of rope state estimation on real images, without requiring expensive annotations. This is achieved by our novel self-supervising learning objective, which is generalizable across a wide range of visual appearances. With estimated rope states, we train a fast and differentiable neural network dynamics model that encodes the physics of mass-spring systems. Our method has a higher accuracy in predicting future states compared to models that do not involve explicit state estimation and do not use any physics prior, while only using 3% of training data. We also show that our approach achieves more efficient manipulation, both in simulation and on a real robot, when used within a model predictive controller.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Interaction Networks for Learning about Objects, Relations and Physics
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1612.00222 · Cited by 969 · cited 2×
“We also experimented with the recently proposed graph network [23] but found it less effective in propagating along a long chain of nodes.”
BackgroundMethodFolding clothes autonomously: A complete pipeline
Doumanoglou, A., Stria, J., Peleka, G., Mariolis, I., Petrik, V., Kargakos, A., Wagner, L., Hlavac, V., Kim, T.-K., & Malassiotis, S. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2602376 · Cited by 122
Learning to dress: Synthesizing human dressing motion via deep reinforcement learning
Clegg, A., Yu, W., Tan, J., Liu, C. K., & Turk, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275048 · Cited by 89
(2003) · doi:10.1109/robot.2003.1242193 · Cited by 73 · cited 1×
“This visual perception task has previously been tackled by hand-engineered image processing algorithms, e.g. in [7].”
MethodLee, A. X., Huang, S. H., Hadfield-Menell, D., Tzeng, E., & Abbeel, P. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6943185 · Cited by 32 · cited 1×
“Lee et al. [16] and Tang et al. [17] use spatial warping to transfer demonstrated manipulation skills to new but similar initial conditions.”
Background
Cited by
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122807 · Cited by 410
Modeling, learning, perception, and control methods for deformable object manipulation
Yin, H., Varava, A., & Kragic, D. (2021) · Science Robotics · doi:10.1126/scirobotics.abd8803 · Cited by 299
Challenges and Outlook in Robotic Manipulation of Deformable Objects
Zhu, J., Cherubini, A., Dune, C., Navarro-Alarcon, D., Alambeigi, F., Berenson, D., Ficuciello, F., Harada, K., Kober, J., Li, X., et al. (2022) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2022.3147415 · Cited by 265
Object detection recognition and robot grasping based on machine learning: a survey
Bai, Q., Li, S., Yang, J., Song, Q., Li, Z., & Zhang, X. (2020) · IEEE Access · doi:10.1109/access.2020.3028740 · Cited by 154
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Lin, X., Wang, Y., Olkin, J., & Held, D. (2020) · arXiv · doi:10.48550/arxiv.2011.07215 · Cited by 140
Metadata
- Authors
- Yan, M., Zhu, Y., Jin, N., & Bohg, J.
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2020-04
- Volume
- 5
- Issue
- 2
- Pages
- 2372-2379
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W3003169307
- Cited by
- 165
- References
- 6
- FWCI
- 12.42
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.