Learning to dress: Synthesizing human dressing motion via deep reinforcement learning
Open accessAbstract
Creating animation of a character putting on clothing is challenging due to the complex interactions between the character and the simulated garment. We take a model-free deep reinforcement learning (deepRL) approach to automatically discovering robust dressing control policies represented by neural networks. While deepRL has demonstrated several successes in learning complex motor skills, the data-demanding nature of the learning algorithms is at odds with the computationally costly cloth simulation required by the dressing task. This paper is the first to demonstrate that, with an appropriately designed input state space and a reward function, it is possible to incorporate cloth simulation in the deepRL framework to learn a robust dressing control policy. We introduce a salient representation of haptic information to guide the dressing process and utilize it in the reward function to provide learning signals during training. In order to learn a prolonged sequence of motion involving a diverse set of manipulation skills, such as grasping the edge of the shirt or pulling on a sleeve, we find it necessary to separate the dressing task into several subtasks and learn a control policy for each subtask. We introduce a policy sequencing algorithm that matches the distribution of output states from one task to the input distribution for the next task in the sequence. We have used this approach to produce character controllers for several dressing tasks: putting on a t-shirt, putting on a jacket, and robot-assisted dressing of a sleeve.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reliable force estimation is needed for safe and comfortable reactive manipulation.
“We still need reliable force estimation [32,55] for safe and comfortable reactive manipulation.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [32]
Section 6.3 (Haptics; page unavailable) says contact-force information helps avoid garment damage and minimize discomfort, supporting the need for force information in dressing. But Section 8.3 (Assistive Hospital Gown; page unavailable) says the assistive gripper policy receives pose inputs only. The paper does not establish that reliable force estimation is needed for reactive manipulation generally. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIn cloth-like deformable object manipulation, policy-gradient methods VPG and TRPO are applied in dressing in simulation.
“In CDO manipulation, policy-gradient methods VPG [1] and TRPO [31,32] are applied in dressing in simulation (Section 6.3.2).”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [32]
Dressing tasks heavily depend on tactile and haptic sensors to infer the progress of the task.
“Third, dressing tasks generally deviate from common manipulation tasks. They heavily depend on tactile and haptic sensors to infer the progress of the task [32]:”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [32]
In self-dressing, it is specifically challenging for an agent with a tactile sensor to formalise and integrate such perception ability into the control procedure.
“In self-dressing, it is specifically challenging for an agent with a tactile sensor to formalise and integrate such perception ability into the control procedure [32].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · see citing works
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103 · see citing works
He, S., & Lau, R. W. H. (2021) · Computer Graphics Forum · doi:10.1111/(issn)1467-8659 · Cited by 1,236 · see citing works
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · see citing works
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 946 · see citing works
Emergence of Locomotion Behaviours in Rich Environments
Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., et al. (2017) · arXiv · doi:10.48550/arxiv.1707.02286 · Cited by 657 · see citing works
Hodgins, J. K., Wooten, W. L., Brogan, D. C., et al. (1995) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/218380.218414 · Cited by 627 · see citing works
Unified particle physics for real-time applications
Macklin, M., Müller, M., Chentanez, N., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601152 · Cited by 452 · see citing works
Yin, K., Loken, K., & van de Panne, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276509 · Cited by 340 · see citing works
Yin, K., Loken, K., & van de Panne, M. (2007) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1275808.1276509 · Cited by 334 · see citing works
Simulation tools for model-based robotics: Comparison of Bullet, Havok, MuJoCo, ODE and PhysX
Erez, T., Tassa, Y., & Todorov, E. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139807 · Cited by 319 · see citing works
Flexible muscle-based locomotion for bipedal creatures
Geijtenbeek, T., van de Panne, M., & van der Stappen, A. F. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508399 · Cited by 265 · see citing works
DART: Dynamic Animation and Robotics Toolkit
Lee, J., Grey, M. X., Ha, S., Kunz, T., Jain, S., Ye, Y., Srinivasa, S. S., Stilman, M., & Liu, C. K. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00500 · Cited by 261 · see citing works
Self-citationFeature-based locomotion controllers
de Lasa, M., Mordatch, I., & Hertzmann, A. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1781157 · Cited by 243 · see citing works
A geometric approach to robotic laundry folding
Miller, S., van den Berg, J., Fritz, M., Darrell, T., Goldberg, K., & Abbeel, P. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911430417 · Cited by 238 · see citing works
Learning symmetric and low-energy locomotion
Yu, W., Turk, G., & Liu, C. K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201397 · Cited by 208 · see citing works
Self-citationManipulation of deformable objects without modeling and simulating deformation
Berenson, D. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6697007 · Cited by 162 · see citing works
Guided Learning of Control Graphs for Physics-Based Characters
Liu, L., van de Panne, M., & Yin, K. (2016) · ACM Transactions on Graphics · doi:10.1145/2893476 · Cited by 112 · see citing works
Optimization-based interactive motion synthesis
Jain, S., Ye, Y., & Liu, C. K. (2009) · ACM Transactions on Graphics · doi:10.1145/1477926.1477936 · Cited by 105 · see citing works
Self-citationReinforcement learning of clothing assistance with a dual-arm robot
Tamei, T., Matsubara, T., Rai, A., & Shibata, T. (2011) · 2011 11th IEEE-RAS International Conference on Humanoid Robots · doi:10.1109/humanoids.2011.6100915 · Cited by 103 · see citing works
Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning
Liu, L., & Hodgins, J. K. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3083723 · Cited by 80 · see citing works
Gravity-Based Robotic Cloth Folding
van den Berg, J., Miller, S., Goldberg, K., & Abbeel, P. (2010) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-17452-0_24 · Cited by 75 · see citing works
Character Motion Synthesis by Topology Coordinates
Ho, E. S. L., & Komura, T. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01369.x · Cited by 65 · see citing works
Harmonic parameterization by electrostatics
Wang, H., Sidorov, K. A., Sandilands, P., & Komura, T. (2013) · ACM Transactions on Graphics · doi:10.1145/2503177 · Cited by 53 · see citing works
Learning to schedule control fragments for physics-based characters using deep q-learning
Liu, L., & Hodgins, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3083723 · Cited by 49 · see citing works
Dexterous Manipulation of Cloth
Bai, Y., Yu, W., & Liu, C. K. (2016) · Computer graphics forum (Print) · doi:10.1111/cgf.12852 · Cited by 45 · see citing works
Self-citationDiscovering and synthesizing humanoid climbing movements
Naderi, K., Rajamäki, J., & Hämäläinen, P. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073707 · Cited by 41 · see citing works
(2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-63426-1 · Cited by 36 · see citing works
Clegg, A., Tan, J., Turk, G., & Liu, C. K. (2015) · ACM Transactions on Graphics · doi:10.1145/2766986 · Cited by 34 · see citing works
Self-citationMotion planning for cooperative manipulators folding flexible planar objects
Balaguer, B., & Carpin, S. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5649031 · Cited by 5 · see citing works
Towards Cloth-Manipulating Characters
Miguel, E., Feng, A., Xu, Y., & Shapiro, A. (2014) · Cited by 3 · see citing works
Data-Driven Approach to Simulating Realistic Human Joint ConstraintsPreprint
Jiang, Y., & Liu, C. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.08685 · Cited by 2 · see citing works
Self-citationLearning to Navigate Cloth using HapticsPreprint
Clegg, A., Yu, W., Erickson, Z., Tan, J., Liu, C. K., & Turk, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.06905 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Transactions on Graphics (Proc. SIGGRAPH 2018 -to appear) 37, 4 (2018).
- David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of Go with deep neural networks and tree search. nature 529, 7587 (2016), 484-489.
- KangKang Yin, Kevin Loken, and Michiel van de Panne. 2007. SIMBICON: simple biped locomotion control. ACM Trans. on Graphics (SIGGRAPH) 26, 3 (2007), 105.
Cited by
Sorted by citations
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570 · see citing works
Self-citationModeling, learning, perception, and control methods for deformable object manipulation
Yin, H., Varava, A., & Kragic, D. (2021) · Science Robotics · doi:10.1126/scirobotics.abd8803 · Cited by 299 · see citing works
Learning to collaborate from simulation for robot-assisted dressing
Clegg, A., Erickson, Z., Grady, P., Turk, G., Kemp, C. C., & Liu, C. K. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2972852 · Cited by 62 · see citing works
Self-citationKadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning
Kim, S., Sorokin, M., Lee, J., & Ha, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.021 · Cited by 9 · see citing works
- Authors
- Clegg, A., Yu, W., Tan, J., Liu, C. K., & Turk, G.
- Affiliations
- Georgia Institute of Technology (US)
- Google (United States) (US)
- Venue
- ACM Transactions on Graphics
- Published
- 2018-12-04
- Volume
- 37
- Issue
- 6
- Pages
- 1-10
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2902791462
- Cited by
- 89
- References
- 36
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 12 · 2023: 14 · 2022: 15 · 2021: 15 · 2020: 12 · 2019: 8 · 2018: 1
- FWCI
- 5.04
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Graphics (publishedVersion)
- Funders
- National Science Foundation (DGE-1650044,IIS-1514258)
- Amazon Web Services (Cloud Credits for Research)
- Samsung (Global Research Outreach grant)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- character animation, dressing, garment simulation, deep reinforcement learning, cloth simulation, haptic information, reward function design, policy sequencing, subtask decomposition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
