ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
This paper introduces a novel deep-learning approach for human-to-robot motion retargeting, enabling robots to mimic human poses accurately. Contrary to prior deep-learning-based works, our method does not require paired human-to-robot data, which facilitates its translation to new robots. First, we construct a shared latent space between humans and robots via adaptive contrastive learning that takes advantage of a proposed cross-domain similarity metric between the human and robot poses. Additionally, we propose a consistency term to build a common latent space that captures the similarity of the poses with precision while allowing direct robot motion control from the latent space. For instance, we can generate in-between motion through simple linear interpolation between two projected human poses. We conduct a comprehensive evaluation of robot control from diverse modalities (i.e., texts, RGB videos, and key poses), which facilitates robot control for non-expert users. Our model outperforms existing works regarding human-to-robot retargeting in terms of efficiency and precision. Finally, we implemented our method in a real robot with self-collision avoidance through a whole-body controller to showcase the effectiveness of our approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While these studies have successfully demonstrated motion retargeting from humanoid to non-humanoid morphologies, they often rely on either embodiment-specific or task-specific models.”
From LEGATO: Cross-Embodiment Imitation Using a Grasping Tool (2025) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Nevertheless, these approaches often encounter challenges in achieving broad generalization across different scenarios.”
From LEGATO: Cross-Embodiment Imitation Using a Grasping Tool (2025) · cites this paper as [34]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Deep Metric Learning Using Triplet Network
Hoffer, E., & Ailon, N. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24261-3_7 · Cited by 2,134
Retargetting motion to new characters
Gleicher, M. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280820 · Cited by 854
Generating Diverse and Natural 3D Human Motions from Text
Guo, C., Zou, S., Zuo, X., Wang, S., Ji, W., Li, X., & Cheng, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00509 · Cited by 575
A hierarchical approach to interactive motion editing for human-like figures
Lee, J., & Shin, S. Y. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311539 · Cited by 566
Li, J., Xu, C., Chen, Z., Bian, S., Yang, L., & Lu, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00339 · Cited by 402
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00870 · Cited by 299
Tevet, G., Raab, S., Gordon, B., Shafir, Y., Cohen-Or, D., & Bermano, A. H. (2022) · arXiv · doi:10.48550/arxiv.2209.14916 · Cited by 290
Teleoperation of Humanoid Robots: A Survey
Darvish, K., Penco, L., Ramos, J., Cisneros, R., Pratt, J., Yoshida, E., Ivaldi, S., & Pucci, D. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3236952 · Cited by 278
Neural Kinematic Networks for Unsupervised Motion Retargetting
Villegas, R., Yang, J., Ceylan, D., & Lee, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00901 · Cited by 240
Skeleton-aware networks for deep motion retargeting
Aberman, K., Li, P., Lischinski, D., Sorkine-Hornung, O., Cohen-Or, D., & Chen, B. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392462 · Cited by 230 · cited 4×
“Contrary to prior works that use the local quaternions [16] or the relative XYZ position of the end effector [18], we consider the global rotation of body limbs as the similarity metric that better preserves the skeleton visual appearance.”
BackgroundMethodResultInfluentialListen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Alexanderson, S., Nagy, R., Beskow, J., & Henter, G. E. (2023) · ACM Transactions on Graphics · doi:10.1145/3592458 · Cited by 194
Mansard, N., Stasse, O., Evrard, P., & Kheddar, A. (2009) · HAL · Cited by 126
Motion capture based human motion recognition and imitation by direct marker control
Ott, C., Lee, D., & Nakamura, Y. (2008) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2008.4755984 · Cited by 125
Self-citationTowards Diverse and Natural Scene-aware 3D Human Motion Synthesis
Wang, J., Rong, Y., Liu, J., Yan, S., Lin, D., & Dai, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01981 · Cited by 117
Robust Real-Time Whole-Body Motion Retargeting from Human to Humanoid
Penco, L., Clement, B., Modugno, V., Hoffman, E. M., Nava, G., Pucci, D., Tsagarakis, N. G., Mouret, J.-B., & Ivaldi, S. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8624943 · Cited by 113
Whole-Body Geometric Retargeting for Humanoid Robots
Darvish, K., Tirupachuri, Y., Romualdi, G., Rapetti, L., Ferigo, D., Chavez, F. J. A., & Pucci, D. (2019) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids43949.2019.9035059 · Cited by 109
Mimetic Communication Model with Compliant Physical Contact in Human—Humanoid Interaction
Lee, D., Ott, C., & Nakamura, Y. (2010) · The International Journal of Robotics Research · doi:10.1177/0278364910364164 · Cited by 94
Self-citationPMnet: learning of disentangled pose and movement for unsupervised motion retargeting
Lim, J., Chang, H. J., & Choi, J. Y. (2019) · University of Birmingham Research Portal (University of Birmingham) · Cited by 78
Contact-Aware Retargeting of Skinned Motion
Villegas, R., Ceylan, D., Hertzmann, A., Yang, J., & Saito, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00958 · Cited by 48
Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry
Zhang, J., Weng, J., Kang, D., Zhao, F., Huang, S., Zhe, X., Bao, L., Shan, Y., Wang, J., & Tu, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01332 · Cited by 48
A variational U-Net for motion retargeting
Jang, H., Kwon, B., Yu, M., Kim, S. U., & Kim, J. (2018) · SIGGRAPH Asia Posters · doi:10.1145/3283289.3283316 · Cited by 46
Online human walking imitation in task and joint space based on quadratic programming
Hu, K., Ott, C., & Lee, D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907357 · Cited by 45
Self-citationTowards a Natural Motion Generator: a Pipeline to Control a Humanoid based on Motion Data
Choi, S., & Kim, J. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967941 · Cited by 45
Transfer learning of shared latent spaces between robots with similar kinematic structure
Delhaisse, B., Esteban, D., Rozo, L. D., & Caldwell, D. G. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966379 · Cited by 36
ADHERENT: Learning Human-like Trajectory Generators for Whole-body Control of Humanoid Robots
Viceconte, P. M., Camoriano, R., Romualdi, G., Ferigo, D., Dafarra, S., Traversaro, S., Oriolo, G., Rosasco, L., & Pucci, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3141658 · Cited by 23
Nonparametric Motion Retargeting for Humanoid Robots on Shared Latent Space
Choi, S., Pan, M. K. X. J., & Kim, J. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.071 · Cited by 22
Self-Supervised Motion Retargeting with Safety Guarantee
Choi, S., Song, M. J., Ahn, H., & Kim, J. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560860 · Cited by 17
Robust Human Motion Forecasting using Transformer-based Model
Mascaro, E. V., Ma, S., Ahn, H., & Lee, D. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981877 · Cited by 14
Self-citationHumanoid Whole-Body Movement Optimization from Retargeted Human Motions
Gomes, W., Radhakrishnan, V., Penco, L., Modugno, V., Mouret, J.-B., & Ivaldi, S. (2019) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids43949.2019.9035070 · Cited by 13
Research works printed but not linked: 1
- [32] Glenn Jocher, Ayush Chaurasia, and Jing Qiu. YOLO by Ultralytics, January 2023. -
Cited by
Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking
Araújo, J. P., Ze, Y., Xu, P., Wu, J., & Liu, C. K. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11697263 · Cited by 259
EMMA: Scaling Mobile Manipulation via Egocentric Human Data
Zhu, L., Kuppili, P., Punamiya, R., Aphiwetsa, P., Patel, D., Kareer, S., Ha, S., & Xu, D. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3653320 · Cited by 44
I-CTRL: Imitation to Control Humanoid Robots Through Bounded Residual Reinforcement Learning
Yan, Y., Mascaro, E. V., Egle, T., & Lee, D. (2025) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2025.3527284 · Cited by 26
Self-citationBerliana, C. I., Nurkhasanah, S., Purnomo, P., & Rohmah, D. (2025) · Profetika Jurnal Studi Islam · doi:10.23917/profetika.v26i01.10632 · Cited by 22
ReConForM : Real‐time Contact‐aware Motion Retargeting for more Diverse Character Morphologies
Cheynel, T., Rossi, T., Bellot-Gurlet, B., Rohmer, D., & Cani, M. (2025) · Computer Graphics Forum · doi:10.1111/cgf.70028 · Cited by 20
Metadata
- Authors
- Yan, Y., Mascaro, E. V., & Lee, D.
- Affiliations
- TU Wien (AT)
- Venue
- 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids)
- Publisher
- IEEE
- Published
- 2023-12-12
- Pages
- 1-8
- OpenAlex
- W4390480930
- Cited by
- 27
- References
- 32
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 7
- FWCI
- 11.55
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE-RAS International Conference on Humanoid Robots (Humanoids) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- elib (German Aerospace Center) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- unsupervised learning, shared latent space, consistency regularization, robot motion control, keyframe interpolation, whole-body control, collision avoidance, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.