The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Learning to Manipulate Deformable Objects without Demonstrations
Open accessAbstract
In this paper we tackle the problem of deformable object manipulation through model-free visual reinforcement learning (RL).In order to circumvent the sample inefficiency of RL, we propose two key ideas that accelerate learning.First, we propose an iterative pick-place action space that encodes the conditional relationship between picking and placing on deformable objects.The explicit structural encoding enables faster learning under complex object dynamics.Second, instead of jointly learning both the pick and the place locations, we only explicitly learn the placing policy conditioned on random pick points.Then, by selecting the pick point that has Maximal Value under Placing (MVP), we obtain our picking policy.This provides us with an informed picking policy during testing, while using only random pick points during training.Experimentally, this learning framework obtains an order of magnitude faster learning compared to independent action-spaces on our suite of deformable object manipulation tasks with visual RGB observations.Finally, using domain randomization, we transfer our policies to a real PR2 robot for challenging cloth and rope coverage tasks, and demonstrate significant improvements over standard RL techniques on average coverage.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
Self-citationAddressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650
Soft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994
Self-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114
Self-citationLearning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., & Levine, S. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.049 · Cited by 847
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618
Cited by
Deep Learning applications for COVID-19
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal of Big Data · doi:10.1186/s40537-020-00392-9 · Cited by 350
Challenges and Outlook in Robotic Manipulation of Deformable Objects
Zhu, J., Cherubini, A., Dune, C., Navarro-Alarcon, D., Alambeigi, F., Berenson, D., Ficuciello, F., Harada, K., Kober, J., Li, X., et al. (2022) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2022.3147415 · Cited by 265
CLIPort: What and Where Pathways for Robotic Manipulation
Shridhar, M., Manuelli, L., & Fox, D. (2021) · arXiv · doi:10.48550/arxiv.2109.12098 · Cited by 223
Transporter Networks: Rearranging the Visual World for Robotic Manipulation
Zeng, A., Florence, P., Tompson, J., Welker, S., Chien, J., Attarian, M., Armstrong, T., Krasin, I., Duong, D., Wahid, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.14406 · Cited by 200
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Lin, X., Wang, Y., Olkin, J., & Held, D. (2020) · arXiv · doi:10.48550/arxiv.2011.07215 · Cited by 140
Seita, D., Florence, P., Tompson, J., Coumans, E., Sindhwani, V., Goldberg, K., & Zeng, A. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561391 · Cited by 130
Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
Yan, W., Vangipuram, A., Abbeel, P., & Pinto, L. (2020) · arXiv · doi:10.48550/arxiv.2003.05436 · Cited by 122
Self-citationFlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding
Ha, H., & Song, S. (2021) · arXiv · doi:10.48550/arxiv.2105.03655 · Cited by 93
VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.034 · Cited by 74
Learning Visible Connectivity Dynamics for Cloth Smoothing
Lin, X., Wang, Y., Huang, Z., & Held, D. (2021) · arXiv · doi:10.48550/arxiv.2105.10389 · Cited by 62
Srivastava, S., Li, C., Lingelbach, M., Martín-Martín, R., Xia, F., Vainio, K., Lian, Z., Gokmen, C., Buch, S., Liu, C. K., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.03332 · Cited by 55
FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy
Weng, T., Bajracharya, S., Wang, Y., Agrawal, K., & Held, D. (2021) · arXiv · doi:10.48550/arxiv.2111.05623 · Cited by 48
Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience
Lee, R., Ward, D., Cosgun, A., Dasagi, V., Corke, P., & Leitner, J. (2020) · arXiv · doi:10.48550/arxiv.2010.03209 · Cited by 47
Visuospatial foresight for physical sequential fabric manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A. K., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2022) · Autonomous Robots · doi:10.1007/s10514-021-10001-0 · Cited by 45
DextAIRity: Deformable Manipulation Can be a Breeze
Xu, Z., Chi, C., Burchfiel, B., Cousineau, E., Feng, S., & Song, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.017 · Cited by 45
Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation
Huang, Z., Lin, X., & Held, D. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.011 · Cited by 39
Learning Latent Graph Dynamics for Visual Manipulation of Deformable Objects
Ma, X., Hsu, D., & Lee, W. S. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9811597 · Cited by 39
In-air Knotting of Rope using Dual-Arm Robot based on Deep Learning
Suzuki, K., Kanamura, M., Suga, Y., Mori, H., & Ogata, T. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635954 · Cited by 31
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
“Additionally, policy noise [156], demonstration data [20,271], specially engineered data [237,272], HER [182], MaxEnt-RL objective [29,225,228] and advantage-weighted loss exploration term [271] are leveraged to improve the sampling efficiency of DRL applications in cloth shaping.”
MethodArnold, S., Tanaka, D., & Yamazaki, K. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.1045747 · Cited by 6
Metadata
- Authors
- Wu, Y., Yan, W., Kurutach, T., Pinto, L., & Abbeel, P.
- Venue
- Robotics: Science and Systems XVI
- Published
- 2020-07-12
- OpenAlex
- W3040477237
- Cited by
- 173
- References
- 14
- FWCI
- 11.69
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.