CoChat
Sign Up
PreprintCited by 0Robot Manipulation and LearningReinforcement Learning in RoboticsSoft Robotics and Applications

DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation

Open access
Mandi, Zhao, Hou, Yifan, Fox, Dieter, Narang, Yashraj, Mandlekar, Ajay, Song, Shuran·2025·arXiv
DOI 10.48550/arxiv.2505.24853arXiv 2505.24853
Open this paper in CoChat — free accountOpen PDF

Abstract

We study the problem of functional retargeting: learning dexterous manipulation policies to track object states from human hand-object demonstrations. We focus on long-horizon, bimanual tasks with articulated objects, which is challenging due to large action space, spatiotemporal discontinuities, and embodiment gap between human and robot hands. We propose DexMachina, a novel curriculum-based algorithm: the key idea is to use virtual object controllers with decaying strength: an object is first driven automatically towards its target states, such that the policy can gradually learn to take over under motion and contact guidance. We release a simulation benchmark with a diverse set of tasks and dexterous hands, and show that DexMachina significantly outperforms baseline methods. Our algorithm and benchmark enable a functional comparison for hardware designs, and we present key findings informed by quantitative and qualitative results. With the recent surge in dexterous hand development, we hope this work will provide a useful platform for identifying desirable hardware capabilities and lower the barrier for contributing to future research. Videos and more at https://project-dexmachina.github.io/

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.24853

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing real2sim2real pipelines treat reconstruction as task-specific engineering, requiring a separate policy, reward curriculum, and training run for each new skill.

    “existing real-to-sim-to-real (real2sim2real) pipelines treat reconstruction as per-task engineering, with a separate policy, reward curriculum, and training run for every new skill (Qin et al., 2022b;Zhao et al., 2025;Xu et al., 2026).”

    From Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy Learning (2026) · cites this paper as Zhao et al., 2025

    Section 4.1 (p. 3) describes training an RL policy in an environment paired with a particular demonstration and robot-hand setup, which supports the narrower point that policy training is task-specific. But the paper does not say that existing real2sim2real pipelines treat reconstruction as per-task engineering or require a separately designed reward curriculum for every new skill. Section 7 (p. 9) instead discusses reconstruction as a requirement for obtaining high-quality demonstration data. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Research works printed but not linked: 46

  1. [1][1] O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research , 39(1):3-20, 2020. 1, 2 -
  2. [2][2] T. G. W. Lum, M. Matak, V. Makoviychuk, A. Handa, A. Allshire, T. Hermans, N. D. Ratliff, and K. V. Wyk. Dextrah-g: Pixels-to-action dexterous arm-hand grasping with geometric fabrics, 2024. URL https://arxiv.org/abs/2407.02274 . 1, 2, 9 - · arXiv
  3. [4][4] M. Raki´ c. Paper 11: The 'belgrade hand prosthesis'. Proceedings of the Institution of Mechanical Engineers, Conference Proceedings , 183(10):60-67, 1968. doi:10.1243/PIME\_CONF\_ 1968\_183\_179\_02. URL https://doi.org/10.1243/PIME\_CONF\_1968\_183\_179\_02 . 2 - · DOI
  4. [5][5] C. Loucks, V. Johnson, P. Boissiere, G. Starr, and J. Steele. Modeling and control of the stanford/jpl hand. In Proceedings. 1987 IEEE International Conference on Robotics and Automation , volume 4, pages 573-578, 1987. doi:10.1109/ROBOT.1987.1088031. 2 - · DOI
  5. [6][6] S. Jacobsen, E. Iversen, D. Knutti, R. Johnson, and K. Biggers. Design of the utah/m.i.t. dextrous hand. In Proceedings. 1986 IEEE International Conference on Robotics and Automation , volume 3, pages 1520-1532, 1986. doi:10.1109/ROBOT.1986.1087395. 2 - · DOI
  6. [7][7] J. Butterfass, G. Hirzinger, S. Knoch, and H. Liu. Dlr's multisensory articulated hand. i. hard- and software architecture. In Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146) , volume 3, pages 2081-2086 vol.3, 1998. doi:10.1109/ROBOT.1998.680625. 2 - · DOI
  7. [8][8] D. Shiokata, A. Namiki, and M. Ishikawa. Robot dribbling using a high-speed multifingered hand and a high-speed vision system. In 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 2097-2102. IEEE, 2005. 2 -
  8. [9][9] R. Higo, Y. Yamakawa, T. Senoo, and M. Ishikawa. Rubik's cube handling using a highspeed multi-fingered hand and a high-speed vision system. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 6609-6614. IEEE, 2018. 2 -
  9. [10][10] Z. Fan, O. Taheri, D. Tzionas, M. Kocabas, M. Kaufmann, M. J. Black, and O. Hilliges. ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2023. 2, 6, 9, 14, 15 -
  10. [11][11] A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, et al. Dextreme: Transfer of agile in-hand manipulation from simulation to reality. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 5977-5984. IEEE, 2023. 2 -
  11. [12][12] H. Qi, A. Kumar, R. Calandra, Y. Ma, and J. Malik. In-hand object rotation via rapid motor adaptation. In Conference on Robot Learning , pages 1722-1732. PMLR, 2023. 2 -
  12. [13][13] Z.-H. Yin, B. Huang, Y. Qin, Q. Chen, and X. Wang. Rotating without seeing: Towards in-hand dexterity through touch. arXiv preprint arXiv:2303.10880 , 2023. 2 - · arXiv
  13. [14][14] T. Chen, M. Tippur, S. Wu, V. Kumar, E. Adelson, and P. Agrawal. Visual dexterity: In-hand reorientation of novel and complex object shapes. Science Robotics , 8(84):eadc9244, 2023. doi:10.1126/scirobotics.adc9244. URL https://www.science.org/doi/abs/10.1126/ scirobotics.adc9244 . 2 - · DOI
  14. [15][15] V. Caggiano, S. Dasari, and V. Kumar. Myodex: A generalizable prior for dexterous manipulation. ArXiv , abs/2309.03130, 2023. URL https://api.semanticscholar.org/CorpusID: 260927595 . 2 - · arXiv
  15. [16][16] Z. Luo, J. Cao, S. J. Christen, A. Winkler, K. Kitani, and W. Xu. Grasping diverse objects with simulated humanoids. ArXiv , abs/2407.11385, 2024. URL https://api.semanticscholar. org/CorpusID:271217823 . 2 - · arXiv
  16. [17][17] P. Mandikal and K. Grauman. Dexterous robotic grasping with object-centric visual affordances. In International Conference on Robotics and Automation (ICRA) , 2021. 2 -
  17. [18][18] T. Zhu, R. Wu, J. Hang, X. Lin, and Y. Sun. Toward human-like grasp: Functional grasp by dexterous robotic hand via object-hand semantic representation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45:12521-12534, 2023. URL https://api. semanticscholar.org/CorpusID:258462924 . 2 -
  18. [19][19] H. Yuan, B. Zhou, Y. Fu, and Z. Lu. Cross-embodiment dexterous grasping with reinforcement learning. ArXiv , abs/2410.02479, 2024. URL https://api.semanticscholar.org/ CorpusID:273098035 . 2 - · arXiv
  19. [20][20] A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations, 2018. URL https://arxiv.org/abs/1709.10087 . 3 - · arXiv
  20. [21][21] K. Xu, Z. Hu, R. Doshi, A. Rovinsky, V. Kumar, A. Gupta, and S. Levine. Dexterous manipulation from images: Autonomous real-world rl via substep guidance. 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 5938-5945, 2022. URL https://api.semanticscholar.org/CorpusID:254877506 . 3 -
  21. [22][22] C. Bao, H. Xu, Y. Qin, and X. Wang. Dexart: Benchmarking generalizable dexterous manipulation with articulated objects. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 21190-21200, 2023. doi:10.1109/CVPR52729.2023.02030. 3 - · DOI
  22. [24][24] C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and C. K. Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation. arXiv preprint arXiv:2403.07788 , 2024. 3 - · arXiv
  23. [25][25] S. Yang, M. Liu, Y. Qin, D. Runyu, L. Jialong, X. Cheng, R. Yang, S. Yi, and X. Wang. Ace: A cross-platfrom visual-exoskeletons for low-cost dexterous teleoperation. arXiv preprint arXiv:240 , 2024. 3 -
  24. [26][26] X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang. Open-television: Teleoperation with immersive active visual feedback. arXiv preprint arXiv:2407.01512 , 2024. 3, 14 - · arXiv
  25. [27][27] K. Shaw, Y. Li, J. Yang, M. K. Srirama, R. Liu, H. Xiong, R. Mendonca, and D. Pathak. Bimanual dexterity for complex tasks. In 8th Annual Conference on Robot Learning , 2024. 3 -
  26. [28][28] H. Zhang, S. Hu, Z. Yuan, and H. Xu. Doglove: Dexterous manipulation with a low-cost open-source haptic force feedback glove. arXiv preprint arXiv:2502.07730 , 2025. 3 - · arXiv
  27. [29][29] P. Mandikal and K. Grauman. Learning dexterous grasping with object-centric visual affordances. 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 6169-6176, 2020. URL https://api.semanticscholar.org/CorpusID:233439776 . 3 -
  28. [30][30] S. Park, S. Lee, M. Choi, J. Lee, J. Kim, J. Kim, and H. Joo. Learning to transfer human hand skills for robot manipulations, 2025. URL https://arxiv.org/abs/2501.04169 . 3 - · arXiv
  29. [31][31] K. Shaw, S. Bahl, and D. Pathak. Videodex: Learning dexterity from internet videos, 2022. URL https://arxiv.org/abs/2212.04498 . 3 - · arXiv
  30. [32][32] M. Xu, Z. Xu, C. Chi, M. Veloso, and S. Song. Xskill: Cross embodiment skill discovery. In Conference on robot learning , pages 3536-3555. PMLR, 2023. 3 -
  31. [33][33] X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph. , 37(4):143:1143:14, July 2018. ISSN 0730-0301. doi:10.1145/3197517.3201311. URL http://doi.acm. org/10.1145/3197517.3201311 . 3 - · DOI
  32. [34][34] Y. Wang, J. Lin, A. Zeng, Z. Luo, J. Zhang, and L. Zhang. Physhoi: Physics-based imitation of dynamic human-object interaction, 2023. URL https://arxiv.org/abs/2312.04393 . 3 - · arXiv
  33. [35][35] Y. Chen, C. Wang, Y. Yang, and C. K. Liu. Object-centric dexterous manipulation from human motion data. ArXiv , abs/2411.04005, 2024. URL https://api.semanticscholar.org/ CorpusID:273850263 . 3, 6, 7, 9, 16, 17 - · arXiv
  34. [36][36] K. Li, P. Li, T. Liu, Y . Li, and S. Huang. Maniptrans: Efficient dexterous bimanual manipulation transfer via residual learning. arXiv preprint arXiv:2503.21860 , 2025. 3, 6, 8, 9, 16, 17 - · arXiv
  35. [37][37] I. Mordatch, E. Todorov, and Z. Popovi´ c. Discovery of complex behaviors through contactinvariant optimization. ACM Transactions on Graphics (ToG) , 31(4):1-8, 2012. 3 -
  36. [38][38] T. Pang and R. Tedrake. A convex quasistatic time-stepping scheme for rigid multibody systems with contact and friction. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 6614-6620. IEEE, 2021. 3 -
  37. [39][39] T. Pang, H. T. Suh, L. Yang, and R. Tedrake. Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models. IEEE Transactions on robotics , 39(6): 4691-4711, 2023. 3 -
  38. [40][40] A. S. Chiappa, P. Tano, N. Patel, A. Ingster, A. Pouget, and A. Mathis. Acquiring musculoskeletal skills with curriculum-based reinforcement learning. bioRxiv , 2024. doi: 10.1101/2024.01.24.577123. URL https://www.biorxiv.org/content/early/2024/ 01/25/2024.01.24.577123 . 3 - · DOI
  39. [41][41] H. Zhang, S. Christen, Z. Fan, L. Zheng, J. Hwangbo, J. Song, and O. Hilliges. ArtiGrasp: Physically plausible synthesis of bi-manual dexterous grasping and articulation. In International Conference on 3D Vision (3DV) , 2024. 3 -
  40. [42][42] X. Mao, Y. Xu, Z. Sun, E. Miller, D. Layeghi, and M. Mistry. Learning long-horizon robot manipulation skills via privileged action. arXiv preprint arXiv:2502.15442 , 2025. 3 - · arXiv
  41. [44][44] X. Xu, D. Bauer, and S. Song. Robopanoptes: The all-seeing robot with whole-body dexterity. arXiv preprint arXiv:2501.05420 , 2025. 6 - · arXiv
  42. [46][46] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347 . 6 - · arXiv
  43. [53][53] J. Romero, D. Tzionas, and M. J. Black. Embodied hands: Modeling and capturing hands and bodies together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) , 36(6), Nov. 2017. 14 -
  44. [55][55] B. Wen, W. Yang, J. Kautz, and S. Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. In CVPR , 2024. 17 -
  45. [56][56] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes, 2018. URL https://arxiv.org/abs/1711. 00199 . 17 -
  46. [57][57] S. Hinterstoisser, V . Lepetit, S. Ilic, S. Holzer, G. Bradski, K. Konolige, and N. Navab. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In Computer Vision, ACCV 2012 - 11th Asian Conference on Computer Vision, Revised Selected Papers , number PART 1 in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pages 548-562, 2013. ISBN 9783642373305. doi:10.1007/978-3-642-37331-2\_42. 11th Asian Conference on Computer Vision, ACCV 2012 ; Conference date: 05-11-2012 Through 09-11-2012. 17 [Please see our submission website: dexmachina-submission.github.io](dexmachina-submission.github.io) · DOI · Paper page

Authors
Mandi, Z., Hou, Y., Fox, D., Narang, Y., Mandlekar, A., & Song, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2505.24853
arXiv
2505.24853
OpenAlex
W4414858769
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Biomedical Engineering › Soft Robotics and Applications
Keywords
dexterous manipulation, bimanual manipulation, articulated objects, curriculum learning, robotic hand design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review