LEGATO: Cross-Embodiment Imitation Using a Grasping Tool

May combine two claims.

Learning from videos of direct human demonstrations limits robot deployment because it relies on third-person visual observations and lacks real-world physical interaction data.

“However, robot deployment is limited by its reliance on third-person visual observations and the absence of real-world physical interaction data.”

• cites [16] sourceDisputed

  The Introduction (p. 2) says the robot uses translated human videos to guide model-based reinforcement learning, then practices the task to learn its physical execution. The Experimental Setup (p. 6) also reports collecting robot trajectories. Although the paper recognizes a challenge in learning from human visual observations, it does not describe an absence of physical interaction data as limiting this approach. (LLM, conf=0.86)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [17] Zakka, 2022, XIRL: Cross-embodiment inverse reinforcement learning sourcePartly supported

  The Introduction (p. 1) describes the difficulty of learning robot policies from third-person demonstrations, including the gap between human demonstrators and robot hardware. The Conclusion (p. 8) says policy learning was demonstrated in simulation, not on a real robot. It does not identify an absence of real-world physical interaction data as the reason robot deployment is limited. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [18] Shaw, 2023, VideoDex: Learning dexterity from internet videos sourcePartly supported

  The Introduction (pp. 1–2) says prior work often uses human videos only for visual pretraining and that collecting real-world robot interaction data is difficult. It does not characterize those videos as third-person observations; the paper cites egocentric datasets and says VideoDex supplements videos with real-world demonstrations. Thus, the claimed limitations are only partly supported. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [19] Wang, 2023, MimicPlay: Long-horizon imitation learning by watching human play sourcePartly supported

  Section 3.1 (p. 4) says common human-video datasets provide single-view, 2D hand trajectories that suffer from depth ambiguity and occlusion, supporting a limitation of visual observations but not specifically a reliance on third-person views. The claimed absence of real-world physical interaction data is not supported: Section 3.1 (p. 3) describes humans physically interacting with the task environment, and the Introduction (p. 2) says the method also uses robot teleoperation demonstrations. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites [20] Xu, 2023, XSkill: Cross embodiment skill discovery sourcePartly supported

  The Abstract says extracting robot skills directly from human videos is challenging because of embodiment differences and unobserved action parameters. The Additional information section describes human videos as RGB observations only, but also describes real-world robot teleoperation data with recorded actions. The paper does not attribute limits on robot deployment to an absence of real-world physical interaction data or specifically to third-person views. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [21] Li, 2024, Okami: Teaching humanoid robots manipulation skills through single video imitation sourcePartly supported

  The Introduction (p. 2) says learning from a human video is challenging because the video lacks action labels, supporting the claim that visual observation has limitations. It does not identify a lack of real-world physical interaction data as a limit on deployment; Section 4.3 (p. 8) describes collecting robot rollout trajectories to train visuomotor policies. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

Regularizing the gripper's trajectories in a motion-invariant space during training prevents bias toward the demonstration embodiment and effectively learns motor skills.

“To address this, we incorporate regularization on the gripper's trajectories in a motion-invariant space during training, preventing bias toward the demonstration embodiment and effectively learning motor skills.”

May combine two claims.

Regularization in the motion-invariant space using the DHB invariant representation ensures robust alignment with demonstrated trajectories and improves policy generalizability across embodiments.

“The regularization in this motion-invariant space ensures robust alignment with demonstrated trajectories, enhancing policy generalizability across various embodiments.”

May combine two claims.

LEGATO is feasible and effective for cross-embodiment transfer of visuomotor policies in MuJoCo simulation and real-world settings.

“In this section, we demonstrate the feasibility and effectiveness of LEGATO for cross-embodiment transfer of visuomotor policies, both in MuJoCo simulation [49] and real-world settings.”

• cites [49] sourceNot checked — no full text of the source

• AssumptionRests on an assumption: The three manipulation tasks chosen for simulation and real-world experiments are representative of complex manipulation tasks.
• AssumptionRests on an assumption: Simulation environments are a valid proxy for investigating cross-embodiment transfer to real robots.

May combine two claims.

Simulation and real-robot evaluations show that the framework is effective at learning and transferring visuomotor skills across multiple robots.

“Our evaluations in simulation and real-robot experiments highlight the framework's effectiveness in learning and transferring visuomotor skills across various robots.”

• AssumptionRests on an assumption: The three manipulation tasks chosen for simulation and real-world experiments are representative of complex manipulation tasks.
• AssumptionRests on an assumption: Simulation environments are a valid proxy for investigating cross-embodiment transfer to real robots.

May combine two claims.

LEGATO (SE3) and Diffusion Policy have higher success rates in the Abstract domain but show a notable drop when deployed on other robots.

“However, the higher success rates of LEGATO (SE3) and Diffusion Policy, which uses the state-of-the-art diffusion model and performs best in the Abstract domain, are followed by a notable decline when applied to other robots.”

Classified as probably not a substantial claim.

The LEGATO Gripper is significantly lighter than typical commercial grippers.

“Made from 3D-printed parts, with TPU 95A for the fingertips and PLA for other components, the LEGATO Gripper is significantly lighter than typical commercial grippers.”

• AssumptionRests on an assumption: The LEGATO Gripper's design features are sufficient to support generalizable grasping actions across tasks.

May combine two claims.

LEGATO outperformed Diffusion Policy (Relative Trajectory) in cross-embodiment settings for the Closing the lid and Ladle reorganization tasks, but not in deployment within the same domain.

“Nevertheless, LEGATO achieved higher performance than Diffusion Policy (Relative Trajectory) in cross-embodiment settings for the Closing the lid and Ladle reorganization tasks, though not in deployment within the same domain.”

The quote only partly states this claim.

The effect of motion-invariant regularization on success rates holds regardless of the neural network architectures used.

“Our findings indicate that incorporating the motion-invariant regularization during training generally reduces success rates when deploying policies on the Abstract embodiment but enhances performance in cross-embodiment settings with different robot embodiments, regardless of the neural network architectures.”

May combine two claims.

In real-robot experiments on the Panda robot, LEGATO succeeded in 16 of 20 trials for Closing the lid, 13 of 20 for Cup shelving, and 14 of 20 for Ladle reorganization.

“Our method succeeded in 16 trials of the Closing the lid task, 13 trials of the Cup shelving task, and 14 trials of the Ladle reorganization task, respectively (see Figure 8).”

May combine two claims.

LEGATO policies achieved the highest success rates across most cross-embodiment settings, except for deployment on the Google Robot for the Cup shelving task and the Spot robot for the Ladle reorganization task.

“Although Diffusion Policy (Relative Trajectory) outperformed Diffusion Policy (Velocity) in cross-embodiment settings, LEGATO policies achieved the highest success rates across most cross-embodiment settings, except for deployment on the Goodle Robot for the Cup shelving task and the Spot robot for the Ladle reorganization task.”

May combine two claims.

Among policies trained on robot teleoperation, demonstrations from the Spot robot produce the highest success rates, while those from the Panda robot produce the lowest.

“Among the policies trained on robot teleoperation, demonstrations on the Spot robot yield the highest success rates, whereas those on the Panda robot yield the lowest.”

Probably not a substantial claim: may not pass S4 (based).

The handheld grasping tool ensures that physical interactions remain consistent across different embodiments.

“We present a cross-embodiment imitation learning framework using a versatile handheld grasping tool that ensures consistent physical interactions across different embodiments.”

Probably not a substantial claim: may not pass S4 (based).

Motion retargeting allows trajectories to be executed on different robots without needing robot-specific training data.

“Motion retargeting enables the execution of trajectories on different robots without requiring robot-specific training data.”

May combine two claims.

Sharing the gripper across robots and having humans directly manipulate it produces consistent, actively actuated grasping actions.

“Sharing the gripper across robots and directly manipulating human demonstrators ensures consistent, actively actuated grasping actions.”

May combine two claims.

The framework ensures that robot movements follow the prescribed hierarchy and satisfy constraints, which guarantees flexible task execution and robust motion generalization.

“Our framework ensures that robot movements not only follow the prescribed hierarchy but also adhere to the constraints, guaranteeing flexible task execution and robust motion generalization.”

Classified as probably not a substantial claim.

Leader-follower systems require leader hardware whose kinematic structure matches the target robot so that joint states can be mapped directly.

“However, this method requires designing leader hardware with a kinematic structure identical to a target robot, enabling its joint states to be directly mapped onto the robot.”

• cites [22] sourceNot checked — no full text of the source
• cites [23] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S4 (based).

The consistency of the action space for handheld gripper motions and ego-centric visual observations across robots enables deployment to various embodiments.

“The consistency of the action space of handheld gripper motions and ego-centric visual observations across different robots enables deployment to various embodiments.”

Probably not a substantial claim: may not pass S4 (based).

Translating human movements into equivalent motions for different target morphologies is difficult because the task is inherently ambiguous.

“In contrast, translating human movements into equivalent motions for varying target morphologies presents a challenge due to the inherently ambiguous nature of the task.”

May combine two claims.

Only the handle parts need to be replaced for different robots, so all other core components can be shared across robots.

“Thus, only the handle parts need replacement for different robots, allowing all other core components to be shared across robots.”

May combine two claims.

The LEGATO Gripper serves as a versatile and adaptive tool that represents tasks via its trajectories and grasping actions, ensuring consistency across embodiments during demonstration and deployment.

“In our method, the LEGATO Gripper, a custom-designed handheld gripper, acts as a versatile and adaptive tool, representing tasks through its trajectories and grasping actions to ensure consistency across embodiments during both demonstration and deploy-ment.”

May combine two claims.

The handheld gripper keeps a consistent shape and viewpoint, which unifies visual observation and action spaces across embodiments.

“The handheld gripper maintains a consistent shape and viewpoint, unifying visual observation and action spaces across embodiments.”

Probably not a substantial claim: may not pass S4 (based).

Differences in hardware and kinematics lead to variations in control latency and errors.

“While the IK optimization adapts gripper motions to each robot, variations in hardware and kinematics introduce differences in control latency and errors.”

Probably not a substantial claim: may not pass S4 (based).

Other data collection methods often cause discrepancies in grasping actions because their demonstrations use passive actuation by humans, whereas robots need active actuation during deployment.

“In contrast, other data collection methods often create discrepancies in grasping actions because demonstrations rely on passive actuation by demonstrators, while robots require active actuation during deployment.”

The quote only partly states this claim.

Using a lower-dimensional action space based on the handheld gripper's motions enables flexible motion retargeting across diverse robot embodiments with different morphologies.

“This approach enables flexible motion retargeting across diverse robot embodiments with varying morphologies.”

May combine two claims.

IK converts the visuomotor policy's commands for the handheld gripper into whole-body motions, enabling adaptation across robotic systems using only kinematic information and avoiding extensive robot-specific training.

“Commands from the visuomotor policy for the handheld gripper are converted into whole-body motions using IK, enabling adaptation across robotic systems with only kinematic information, thereby avoiding extensive robotspecific training.”

May combine two claims.

The handheld gripper unifies the action and observation spaces, which lets tasks be specified consistently across different robots.

“We introduce a handheld gripper that unifies action and observation spaces, allowing tasks to be defined consistently across robots.”

May combine two claims.

Joint controller latency and IK motion retargeting affect the quality of demonstrations.

“The policies trained on the Abstract embodiment performed best, indicating that joint controller latency and the IK motion retargeting affect demonstration quality.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Across all baselines and tasks, the initial robot states are consistent, and the initial object poses, except for the table, are uniformly randomized.

“Across all baselines and tasks, the initial states of robots are consistent, and the initial poses of objects, other than the table, are uniformly randomized.”

May combine two claims.

Even when trained on specific embodiments, motion-invariant regularization enables policies to adapt to different robots and handle variations in control latency and tracking errors.

“Although trained on specific embodiments, regularization in a motion-invariant space allows these policies to adapt easily to different robots, managing variations in control latency and tracking errors.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Task demonstrations in simulation use the same tracking camera setup as in real-robot evaluations.

“Task demonstrations in simulation use the same tracking camera setup as in real-robot evaluations-a Realsense T265 [50] camera.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The same IK optimizer with consistent parameters is used within a single robot.

“Additionally, the same IK optimizer with consistent parameters is used within a single robot.”

May combine two claims.

Leveraging motion invariance is applicable across various neural network architectures and is effective for cross-embodiment learning.

“This highlights the applicability of leveraging motion invariance across various neural network architectures and its effectiveness for cross-embodiment learning.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The motion-invariant transform used to compute L invar follows the DHB motion-invariant framework.

“The motion-invariant transform ϕ(•), used to compute L invar in Equation 5, follows the DHB motion-invariant framework [42].”

• cites [42] sourceNot checked — no full text of the source

May combine two claims.

The reduced performance in those tasks points to limitations of fixed-base robots for tasks requiring extensive motion and highlights the need for whole-body manipulation.

“This underscores the limitations of fixed-base robots for tasks requiring extensive motion, highlighting the need for whole-body manipulation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The button interface for triggering grasp actions and recording data is kept consistent with the real-world setup.

“The button interface for triggering grasp actions and recording data is kept consistent with the realworld setup.”

Probably not a substantial claim: may not pass S4 (based).

The action and observation spaces, defined in the handheld-gripper frame, remain consistent across various robot platforms.

“These action and observation spaces, defined in the handheld-gripper frame, remain consistent across various robot platforms.”

Probably not a substantial claim: may not pass S4 (based).

Generalizing latency compensation across various robot embodiments is challenging because it requires finetuning for each target robot system.

“As described in Section I, generalizing latency compensation across various robot embodiments is challenging because it requires finetuning for each target robot system.”

May combine two claims.

Whole-body motion capability and redundancy influence the quality of task demonstrations, as the Spot robot has full 6 DOFs in its body while the Panda robot has less redundancy.

“This suggests that whole-body motion capability and redundancy influence task demonstration quality, as the Spot robot has full 6 DOFs in its body, while the Panda robot has less redundancy.”

Classified as probably not a substantial claim.

In the DHB transformation, the motion of a rigid body is separated into position and orientation components using two frames.

“In the DHB transformation, the motion of a rigid body is separated into position and orientation components using two frames.”

• cites [42] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S4 (based).

Training with the action space of relative trajectories requires additional geometric information, such as the initial gripper pose for a sequence of receding-horizon actions, to generate the training dataset.

“Unlike our setting, training with the action space of relative trajectories requires additional geometric information, such as the initial gripper pose for a sequence of receding-horizon actions, to generate the training dataset.”