Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies
Open accessAbstract
Robot manipulation for untangling 1D deformable structures such as ropes, cables, and wires is challenging due to their infinite dimensional configuration space, complex dynamics, and tendency to self-occlude.Analytical controllers often fail in the presence of dense configurations, due to the difficulty of grasping between adjacent cable segments.We present two algorithms that enhance robust cable untangling, LOKI and SPi-DERMan, which operate alongside HULK, a high-level planner from prior work.LOKI uses a learned model of manipulation features to refine a coarse grasp keypoint prediction to a precise, optimized location and orientation, while SPiDERMan uses a learned model to sense task progress and apply recovery actions.We evaluate these algorithms in physical cable untangling experiments with 336 knots and over 1500 actions on real cables using the da Vinci surgical robot.We find that the combination of HULK, LOKI, and SPiDERMan is able to untangle dense overhand, figure-eight, double-overhand, square, bowline, granny, stevedore, and triple-overhand knots.The composition of these methods successfully untangles a cable from a dense initial configuration in 68.3% of 60 physical experiments and achieves 50% higher success rates than baselines from prior work.Supplementary material, code, and videos can be found at https://tinyurl.com/rssuntangling.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A robust RMS should be able to recognise the failing states and recover from these failures.
“A robust RMS should be able to recognise the failing states and recover from these failures [9,36].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741 · see citing works
Learning openCV: computer vision with the openCV library
Bradski, G. R., & Kaehler, A. (2008) · CERN Document Server (European Organization for Nuclear Research) · Cited by 2,468 · see citing works
Residual Reinforcement Learning for Robot Control
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., & Levine, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794127 · Cited by 480 · see citing works
Cable manipulation with a tactile-reactive gripper
She, Y., Wang, S., Dong, S., Sunil, N., Rodríguez, A., & Adelson, E. H. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211027233 · Cited by 267 · see citing works
Combining self-supervised learning and imitation for vision-based rope manipulation
Nair, A., Chen, D., Agrawal, P., Isola, P., Abbeel, P., Malik, J., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989247 · Cited by 260 · see citing works
Topological structural analysis of digitized binary images by border following
Suzuki, S., & Abe, K. (1985) · Computer Vision Graphics and Image Processing · doi:10.1016/0734-189x(85)90136-7 · Cited by 252 · see citing works
Sim-to-real reinforcement learning for deformable object manipulation
Matas, J., James, S., & Davison, A. J. (2018) · arXiv · doi:10.48550/arxiv.1806.07851 · Cited by 241 · see citing works
van den Berg, J., Miller, S., Duckworth, D., Hu, H., Wan, A., Fu, X., Goldberg, K., & Abbeel, P. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509621 · Cited by 233 · see citing works
Self-citationRecovery RL: Safe Reinforcement Learning With Learned Recovery Zones
Thananjeyan, B., Balakrishna, A., Nair, S., Luo, M., Srinivasan, K., Hwang, M., Gonzalez, J. E., Ibarz, J., Finn, C., & Goldberg, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3070252 · Cited by 218 · see citing works
Self-citationDense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic ManipulationPreprint
Florence, P., Manuelli, L., & Tedrake, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08756 · Cited by 181 · see citing works
Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects
Yan, M., Zhu, Y., Jin, N., & Bohg, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2969931 · Cited by 166 · see citing works
Self-Supervised Visual Descriptor Learning for Dense Correspondence
Schmidt, T., Newcombe, R., & Fox, D. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2016.2634089 · Cited by 146 · see citing works
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Lin, X., Wang, Y., Olkin, J., & Held, D. (2020) · arXiv · doi:10.48550/arxiv.2011.07215 · Cited by 140 · see citing works
Seita, D., Florence, P., Tompson, J., Coumans, E., Sindhwani, V., Goldberg, K., & Zeng, A. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561391 · Cited by 130 · see citing works
Self-citationLearning Predictive Representations for Deformable Objects Using Contrastive Estimation
Yan, W., Vangipuram, A., Abbeel, P., & Pinto, L. (2020) · arXiv · doi:10.48550/arxiv.2003.05436 · Cited by 122 · see citing works
Error Detection and Recovery in Robotics
(1989) · Springer eBooks · doi:10.1007/bfb0039640 · Cited by 116 · see citing works
Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor
Seita, D., Ganapathi, A., Hoque, R., Hwang, M., Cen, E., Tanwani, A. K., Balakrishna, A., Thananjeyan, B., Ichnowski, J., Jamali, N., et al. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341608 · Cited by 113 · see citing works
Self-citationLearning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data
Sundaresan, P., Grannen, J., Thananjeyan, B., Balakrishna, A., Laskey, M., Stone, K., Gonzalez, J. E., & Goldberg, K. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197121 · Cited by 107 · see citing works
Self-citationRobot Manipulation of Deformable Objects
Henrich, D., & Wörn, H. (2000) · Advanced manufacturing series · doi:10.1007/978-1-4471-0749-1 · Cited by 96 · see citing works
VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.034 · Cited by 74 · see citing works
Self-citationTangled: Learning to untangle ropes with RGB-D perception
Lui, W. H., & Saxena, A. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696448 · Cited by 71 · see citing works
Lee, M. A., Florensa, C., Tremblay, J., Ratliff, N., Garg, A., Ramos, F. T., & Fox, D. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197125 · Cited by 57 · see citing works
Occlusion-robust Deformable Object Tracking without Physics Simulation
Chi, C., & Berenson, D. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967827 · Cited by 55 · see citing works
Hwang, M., Thananjeyan, B., Paradis, S., Seita, D., Ichnowski, J., Fer, D., Low, T., & Goldberg, K. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3010746 · Cited by 51 · see citing works
Self-citationLearning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience
Lee, R., Ward, D., Cosgun, A., Dasagi, V., Corke, P., & Leitner, J. (2020) · arXiv · doi:10.48550/arxiv.2010.03209 · Cited by 47 · see citing works
Robotized assembly of a wire harness in car production line
Jiang, X., Koo, K., Kikuchi, K., Konno, A., & Uchiyama, M. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5653133 · Cited by 44 · see citing works
Paradis, S., Hwang, M., Thananjeyan, B., Ichnowski, J., Seita, D., Fer, D., Low, T., Gonzalez, J. E., & Goldberg, K. Y. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561070 · Cited by 34 · see citing works
Self-citationUntangling Dense Knots by Learning Task-Relevant KeypointsPreprint
Grannen, J., Sundaresan, P., Thananjeyan, B., Ichnowski, J., Balakrishna, A., Hwang, M., Viswanath, V., Laskey, M., Gonzalez, J. E., & Goldberg, K. Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04999 · Cited by 30 · see citing works
Self-citationWang, A. S., & Kroemer, O. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793623 · Cited by 29 · see citing works
Tethered Tool Manipulation Planning With Cable Maneuvering
Sánchez, D., Wan, W., & Harada, K. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974675 · Cited by 20 · see citing works
VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric ManipulationPreprint
Hoque, R., Seita, D., Balakrishna, A., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09044 · Cited by 20 · see citing works
Self-citationRecovery RL: Safe Reinforcement Learning with Learned Recovery ZonesPreprint
Thananjeyan, B., Balakrishna, A., Nair, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15920 · Cited by 20 · see citing works
Self-citationGanapathi, A., Sundaresan, P., Thananjeyan, B., Balakrishna, A., Seita, D., Grannen, J., Hwang, M., Hoque, R., Gonzalez, J. E., Jamali, N., et al. (2020) · Cited by 19 · see citing works
Self-citationCappell, M. S. (2009) · Digestive Diseases and Sciences · doi:10.1007/s10620-009-0923-0 · Cited by 17 · see citing works
Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth DataPreprint
Sundaresan, P., Grannen, J., Thananjeyan, B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01835 · Cited by 16 · see citing works
Self-citationCable Manipulation with a Tactile-Reactive GripperPreprint
She, Y., Wang, S., Dong, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02860 · Cited by 11 · see citing works
Seita, D., Florence, P., Tompson, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03385 · Cited by 11 · see citing works
Self-Supervised Learning of State Estimation for Manipulating Deformable Linear ObjectsPreprint
Yan, M., Zhu, Y., Jin, N., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06283 · Cited by 10 · see citing works
Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic SupervisorPreprint
Seita, D., Ganapathi, A., Hoque, R., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04854 · Cited by 9 · see citing works
Robots of the Lost Arc: Learning to Dynamically Manipulate Fixed-Endpoint Ropes and Cables.Preprint
Zhang, H., Ichnowski, J., Seita, D., Wang, J., & Goldberg, K. Y. (2020) · arXiv (Cornell University) · Cited by 4 · see citing works
Self-citationTethered Tool Manipulation Planning with Cable ManeuveringPreprint
Sánchez, D., Wan, W., & Harada, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10686 · Cited by 3 · see citing works
Paradis, S., Hwang, M., Thananjeyan, B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06163 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Peter Kazanzides, Zihan Chen, Anton Deguet, Gregory S. Fischer, Russell H. Taylor, and Simon P. DiMaio. An open-source research kit for the da Vinci surgical system. In Proc. IEEE Int. Conf. Robotics and Automation (ICRA), 2014.
Cited by
Sorted by citations
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T., Kumar, V., Levine, S., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.016 · Cited by 674 · see citing works
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“Sundaresan et al. (2021b) [36] developed Sensing Progress in Dense Entanglements for Recovery Manipulation (SPiDERMan) to recover from the error caused by HULK .”
BackgroundMethodInfluentialDisentangling Dense Multi-Cable Knots
Viswanath, V., Grannen, J., Sundaresan, P., Thananjeyan, B., Balakrishna, A., Novoseller, E., Ichnowski, J., Laskey, M., Gonzalez, J. E., & Goldberg, K. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636397 · Cited by 21 · see citing works
Self-citation
- Authors
- Sundaresan*, P., Grannen*, J., Thananjeyan, B., Balakrishna, A., Ichnowski, J., Novoseller, E., Hwang, M., Laskey, M., Gonzalez, J., & Goldberg, K.
- Affiliations
- University of California, Berkeley (US)
- Toyota Research Institute (US)
- ORCID
- Venue
- Robotics: Science and Systems XVII
- Published
- 2021-07-12
- OpenAlex
- W3174282793
- Cited by
- 26
- References
- 42
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 4 · 2023: 8 · 2022: 3 · 2021: 2
- FWCI
- 6.34
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Robotics: Science and Systems (publishedVersion)
- Funders
- National Science Foundation (1637789, 1637759, 1637444)
- University of Washington (IIS 1637444, 1637444)
- Johns Hopkins University (IIS 1637789)
- Intuitive Surgical
- Toyota Research Institute
- Worcester Polytechnic Institute (IIS 1637759)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Engineering › Biomedical Engineering › Soft Robotics and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- deformable structures, robotic manipulation, recovery policy, da Vinci surgical robot, figure-eight knot, self-occlusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
