Reinforcement learning of a motor skill for wearing a T-shirt using topology coordinates
Abstract
This article focuses on learning motor skills for anthropomorphic robots that must interact with non-rigid materials to perform such tasks as wearing clothes, turning socks inside out, and applying bandages. We propose a novel reinforcement learning framework for learning motor skills that interact with non-rigid materials. Our learning framework focuses on the topological relationship between the configuration of the robot and the non-rigid material based on the consideration that most details of the material (e.g. wrinkles) are not important for performing the motor tasks. This focus allows us to define the task performance and provide reward signals based on a low-dimensional variable, i.e. topology coordinates, in a real environment using reliable sensors. We constructed an experimental setting with an anthropomorphic dual-arm robot and a tailor-made T-shirt for it. To demonstrate the feasibility of our framework, a robot performed a T-shirt wearing task, whose goal was to put both of its arms into the corresponding sleeves of the T-shirt. The robot acquired sequential movements that put both of its arms into the T-shirt.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reliably finding the topological correspondence between the different parts of the body and the garment is a challenge for perception.
“Reliably finding the topological correspondence between the different parts of the body and the garment is a challenge for perception [246,312].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [246]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
(1999) · Cited by 5,147 · see citing works
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · see citing works
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 721 · see citing works
Maitin-Shepard, J., Cusumano-Towner, M., Lei, J., & Abbeel, P. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509439 · Cited by 418 · see citing works
Stochastic policy gradient reinforcement learning on a simple 3D biped
Tedrake, R., Zhang, T., & Seung, H. S. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2004.1389841 · Cited by 273 · see citing works
Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning
Morimoto, J., & Doya, K. (2001) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(01)00113-0 · Cited by 262 · see citing works
Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot
Endo, G., Morimoto, J., Matsubara, T., Nakanishi, J., & Cheng, G. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907084980 · Cited by 206 · see citing works
A Kendama Learning Robot Based on Bi-directional Theory
Miyamoto, H., Schaal, S., Gandolfo, F., Gomi, H., Koike, Y., Osu, R., Nakano, E., Wada, Y., & Kawato, M. (1996) · Neural Networks · doi:10.1016/s0893-6080(96)00043-3 · Cited by 171 · see citing works
Learning tasks from observation and practice
Bentivegna, D. C., Atkeson, C. G., & Cheng, G. (2004) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2004.03.010 · Cited by 102 · see citing works
Learning CPG-based biped locomotion with a policy gradient method
Matsubara, T., Morimoto, J., Nakanishi, J., Sato, M.-A., & Doya, K. (2006) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2006.05.012 · Cited by 98 · see citing works
A theory for cursive handwriting based on the minimization principle
Wada, Y., & Kawato, M. (1995) · Biological Cybernetics · doi:10.1007/bf00199051 · Cited by 89 · see citing works
Leg motion primitives for a dancing humanoid robot
Nakaoka, S., Nakazawa, A., Yokoi, K., & Ikeuchi, K. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1307216 · Cited by 81 · see citing works
Clothes state recognition using 3D observed data
Kita, Y., Ueshiba, T., Neo, E., & Kita, N. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152741 · Cited by 73 · see citing works
Character Motion Synthesis by Topology Coordinates
Ho, E. S. L., & Komura, T. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01369.x · Cited by 65 · see citing works
Combining imitation and reinforcement learning to fold deformable planar objects
Balaguer, B., & Carpin, S. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6094992 · Cited by 59 · see citing works
A Cloth Detection Method Based on Image Wrinkle Feature for Daily Assistive Robots.
Yamazaki, K. (2009) · Cited by 50 · see citing works
Learning to acquire whole-body humanoid CoM movements to achieve dynamic tasks
Matsubara, T., Morimoto, J., Nakanishi, J., Hyon, S.-H., Hale, J. G., & Cheng, G. (2007) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2007.363871 · Cited by 13 · see citing works
Learning to Acquire Whole-Body Humanoid Center of Mass Movements to Achieve Dynamic Tasks
Matsubara, T., Morimoto, J., Nakanishi, J., Hyon, S.-H., Hale, J. G., & Cheng, G. (2008) · Advanced Robotics · doi:10.1163/156855308x324785 · Cited by 7 · see citing works
Acquisition of a page turning skill for a multifingered hand using reinforcement learning
Ueda, J., Negi, R., & Yoshikawa, T. (2004) · Advanced Robotics · doi:10.1163/156855304322753326 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep reinforcement learning with smooth policy update: Application to robotic cloth manipulation
Tsurumine, Y., Cui, Y., Uchibe, E., & Matsubara, T. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.004 · Cited by 171 · see citing works
Learning garment manipulation policies toward robot-assisted dressing
Zhang, F., & Demiris, Y. (2022) · Science Robotics · doi:10.1126/scirobotics.abm6010 · Cited by 83 · see citing works
Probabilistic real-time user posture tracking for personalized robot-assisted dressing
Zhang, F., Cully, A., & Demiris, Y. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2904461 · Cited by 70 · see citing works
“errors in [5], [6], [10], and [11] that may put the users at risk.”
BackgroundPersonalized Robot Assistant for Support in Dressing
Jevtić, A., Flores-Valle, A., Alenyà, G., Chance, G., Caleb-Solly, P., Dogramadzi, S., & Carme Torras, A. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2817283 · Cited by 60 · see citing works
“The robot was able to pull a T-shirt over the mannequin’s head while tracking the position of the collar and sleeves.”
BackgroundEMD Net: An Encode–Manipulate–Decode Network for Cloth Manipulation
Tanaka, D., Arnold, S., & Yamazaki, K. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2800122 · Cited by 60 · see citing works
“Tanaka et al. [13] proposed a method of learning motor skills.”
MethodLearning Grasping Points for Garment Manipulation in Robot-Assisted Dressing
Zhang, F., & Demiris, Y. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196994 · Cited by 55 · see citing works
“reinforcement learning has been successfully used to enable a humanoid robot to wrap a scarf [14] and to put on a Tshirt for mannequins with different postures [15].”
MethodPersonalized Assistance for Dressing Users
Klee, S. D., Ferreira, B. Q., Silva, R., Costeira, J. P., Melo, F. S., & Veloso, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25554-5_36 · Cited by 51 · see citing works
“Researchers have used reinforcement learning to teach robots to put on a t-shirt and wrap a scarf around a mannequin’s neck [10,5].”
BackgroundBottom dressing by a life-sized humanoid robot provided failure detection and recovery functions
Yamazaki, K., Oya, R., Nagahama, K., Okada, K., & Inaba, M. (2014) · 2014 IEEE/SICE International Symposium on System Integration · doi:10.1109/sii.2014.7028101 · Cited by 42 · see citing works
“[13] proposed the reinforcement learning of t-shirt dressing.”
BackgroundPersonalized robot-assisted dressing using user modeling in latent spaces
Zhang, F., Cully, A., & Demiris, Y. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206206 · Cited by 37 · see citing works
“For example, reinforcement learning has been successfully used to enable a robot to wrap a scarf [2] and to dress a Tshirt on human mannequins with different head and shoulder inclinations [3, 4, 8].”
MethodBottom dressing by a dual-arm robot using a clothing state estimation based on dynamic shape changes
Yamazaki, K., Oya, R., Nagahama, K., Okada, K., & Inaba, M. (2016) · International Journal of Advanced Robotic Systems · doi:10.5772/61930 · Cited by 34 · see citing works
A quantitative analysis of dressing dynamics for robotic dressing assistance
Chance, G., Jevtić, A., Caleb-Solly, P., & Dogramadzi, S. (2017) · Frontiers in Robotics and AI · doi:10.3389/frobt.2017.00013 · Cited by 34 · see citing works
In-air Knotting of Rope using Dual-Arm Robot based on Deep Learning
Suzuki, K., Kanamura, M., Suga, Y., Mori, H., & Ogata, T. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635954 · Cited by 31 · see citing works
“On the contrary, in machine learning-based approaches, especially in those based on deep neural networks (DNNs), the sensorimotor experience of the robot can be learned [6][11][12] or reinforcement learning using real robots or in a sim-to-real manner can be used [13][14][15][16][17].”
BackgroundVisual-Tactile Learning of Garment Unfolding for Robot-Assisted Dressing
Zhang, F., & Demiris, Y. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3296371 · Cited by 30 · see citing works
“…target users (e. g., healthy users [4], impaired users [6], paralyzed patients [10]), personalized assistive dressing (e. g., taking user capabilities [25] and preferences [26] into account), and learning dressing with a variety of sensing modalities (e. g., vision [3], force [6], capacity [5]).”
BackgroundMethodGenerative Adversarial Imitation Learning with Deep P-Network for Robotic Cloth Manipulation
Tsurumine, Y., Cui, Y., Yamazaki, K., & Matsubara, T. (2019) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids43949.2019.9034991 · Cited by 27 · see citing works
“With several successful applications in robot control [9]– [11], Reinforcement learning (RL) [12] that searches for optimal policies by interacting with the environment becomes one potential solution to learn robotic cloth manipulation [13, 14].”
BackgroundKoganti, N., Tamei, T., Matsubara, T., & Shibata, T. (2014) · The 23rd IEEE International Symposium on Robot and Human Interactive Communication · doi:10.1109/roman.2014.6926241 · Cited by 24 · see citing works
“[3] proposed a Reinforcement Learning scheme for acquiring motor skills in the handling of clothes.”
BackgroundSelf-citation“Elbows Out”—Predictive Tracking of Partially Occluded Pose for Robot-Assisted Dressing
Chance, G., Jevtić, A., Caleb-Solly, P., Alenyà, G., Carme Torras, A., & Dogramadzi, S. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2854926 · Cited by 23 · see citing works
Koganti, N., Tamei, T., Matsubara, T., & Shibata, T. (2013) · Proceedings of Conference on Advances In Robotics · doi:10.1145/2506095.2506146 · Cited by 21 · see citing works
“…Vision and Scene Understanding Representations, Data structures and transforms ; J.3 [Computer Applications]: Life and Medical Sciences Health General Terms Algorithms, Human Factors, Measurement Keywords Robotic Clothing Assistance, Depth Sensor, Human-Cloth relationship, Topology…”
BackgroundSelf-citationTsurumine, Y., & Matsubara, T. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2022.104264 · Cited by 20 · see citing works
“On the other hand, compared to the previous methods using RL [2, 14, 15, 16], our method does not require the design of a complex reward function for each cloth-manipulation task.”
BackgroundMethodKadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“[246].”
BackgroundMethodInfluentialA method of state recognition of dressing clothes based on dynamic state matching
Yamazaki, K., Oya, R., Nagahama, K., & Inaba, M. (2013) · Proceedings of the 2013 IEEE/SICE International Symposium on System Integration · doi:10.1109/sii.2013.6776728 · Cited by 20 · see citing works
Computer Vision – ECCV 2022 Workshops
Karlinsky, L., Michaeli, T., & Nishino, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25075-0 · Cited by 19 · see citing works
Bodies Uncovered: Learning to Manipulate Real Blankets Around People via Physics Simulations
Puthuveetil, K. P., Kemp, C. C., & Erickson, Z. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3142732 · Cited by 18 · see citing works
“[12] proposed a reinforcement learning framework that allows a dualarm robot to learn motor skills for a self-dressing task based on the topological relationship between the robot and the cloth.”
BackgroundUser Modelling Using Multimodal Information for Personalised Dressing Assistance
Gao, Y., Chang, H. J., & Demiris, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2978207 · Cited by 14 · see citing works
“[22] proposed to use reinforcement learning to teach a dual-arm robot to learn the motions of dressing T-shirts.”
BackgroundNocentini, O., Kim, J., Bashir, Z. M., & Cavallo, F. (2022) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-022-01078-4 · Cited by 15 · see citing works
“Several works consider only a single dressing task, or a single object grasped, or specific configurations [55], and these limitations should be overcome in the future to have as many generic models as possible.”
BackgroundFlattening and folding towels with a single-arm robot based on reinforcement learning
Shehawy, H., Pareyson, D., Caruso, V., Bernardi, S. D., Zanchettin, A. M., & Rocco, P. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104506 · Cited by 14 · see citing works
Kotsovolis, S., & Demiris, Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611478 · Cited by 10 · see citing works
“A few prior studies involved the insertion of the opening, for instance, when putting a T-shirt’s neck around the head [11, 12] or during upper body dressing, with a T-shirt or a sleeveless jacket [13, 33].”
BackgroundChance, G., Caleb-Solly, P., Jevtić, A., & Dogramadzi, S. (2017) · IEEE RO-MAN · doi:10.1109/roman.2017.8172315 · Cited by 8 · see citing works
“Similarly, stereo cameras used in combination with garment markers have been used for dressing a mannequin with a t-shirt [11] and the group have also achieved similar results using depth (RGB-D) cameras [12].”
MethodBi-Manual Manipulation of Multi-Component Garments towards Robot-Assisted Dressing
Kotsovolis, S., & Demiris, Y. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161335 · Cited by 7 · see citing works
“For example, robot self-dressing is achieved in [8] by learning sequential control policies with reinforcement learning.”
BackgroundMethodDevelopment of a compliant gripper for safe robot-assisted trouser dressing-undressing
Unde, J., Inden, T., Wakayama, Y., Colan, J., Zhu, Y., Aoyama, T., & Hasegawa, Y. (2024) · Advanced Robotics · doi:10.1080/01691864.2024.2376024 · Cited by 6 · see citing works
Garment Diffusion Models for Robot-Assisted Dressing
Kotsovolis, S., & Demiris, Y. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3518104 · Cited by 5 · see citing works
Cross-Domain Representation Learning for Clothes Unfolding in Robot-Assisted Dressing
Qie, J., Gao, Y., Feng, R., Wang, X., Yang, J., Dasgupta, E., Chang, H. J., & Chang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25075-0_44 · Cited by 5 · see citing works
Matsubara, T., Hasegawa, T., & Sugimoto, K. (2015) · IEEE RO-MAN · doi:10.1109/roman.2015.7333587 · Cited by 4 · see citing works
“[6] achieved wearing a T-shirt by a dual-arm robot.”
MethodYamasaki, K., Shibata, T., & Hénaff, P. (2024) · Advanced Robotics · doi:10.1080/01691864.2024.2399728 · Cited by 3 · see citing works
“Matsubara [7], Zhang [8], and Gao [9,10] observed and modeled the human condition by using cameras and tactile sensors to understand the human body during the dressing process.”
A framework for robot-assisted doffing of personal protective equipment
Umali, A., & Berenson, D. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989333 · Cited by 3 · see citing works
“proposed using reinforcement learning to learn motor skills necessary to interact with non-rigid materials [9][10].”
BackgroundYe, Y., Mao, Y., Qiu, S., Tang, C., Pan, Z., Wan, W., Cai, S., & Bao, G. (2025) · Chinese Journal of Mechanical Engineering · doi:10.1186/s10033-025-01204-y · Cited by 2 · see citing works
Kullback Leibler control approach to rubber band manipulation
Murase, M., Yamazaki, K., & Matsubara, T. (2017) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii.2017.8279300 · Cited by 2 · see citing works
“for robot control in simulation [13] and real robots [14]– [16] subsequently.”
BackgroundFlattening Clothes with a Single-Arm Robot Based on Reinforcement Learning
Shehawy, H., Pareyson, D., Caruso, V., Zanchettin, A. M., & Rocco, P. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-22216-0_39 · Cited by 1 · see citing works
Kadi, H. A., & Terzić, K. (2022) · Preprints.org · doi:10.20944/preprints202212.0305.v1 · Cited by 1 · see citing works
Cloth manipulation based on motion generation and shape prediction by EM*D-net
Tanaka, D., Arnold, S., & Yamazaki, K. (2018) · Transactions of the JSME (in Japanese) · doi:10.1299/transjsme.18-00069
Learning Control Policies by Reinforcement Learning
Matsubara, T. (2018) · Journal of the Robotics Society of Japan · doi:10.7210/jrsj.36.597
Designing of low-DoF robot system from low-dimensional latent space
Yamasaki, K., & Shibata, T. (2026) · ROBOMECH Journal · doi:10.1186/s40648-026-00336-7
- Authors
- Matsubara, T., Shinohara, D., & Kidode, M.
- Affiliations
- Nara Institute of Science and Technology (JP)
- Corresponding author
- Matsubara, Takamitsu
- ORCID
- Matsubara, Takamitsu
- Venue
- Advanced Robotics
- Publisher
- Informa UK Limited
- Published
- 2013-05
- Volume
- 27
- Issue
- 7
- Pages
- 513-524
- ISSN
- 0169-1864 (ISSN-L)
- 1568-5535
- OpenAlex
- W2001328977
- Cited by
- 41
- References
- 21
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 4 · 2023: 7 · 2022: 5 · 2021: 1 · 2020: 2 · 2019: 2 · 2018: 6 · 2017: 5 · 2016: 1 · 2015: 2 · 2014: 2 · 2013: 2
- FWCI
- 2.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Advanced Robotics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Engineering › Biomedical Engineering › Soft Robotics and Applications
- Keywords
- reinforcement learning, motor skill learning, humanoid robots, dual-arm robot, fabric manipulation, reward signals, sequential movements
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
