Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy Learning

May combine two claims.

A single residual-RL policy trained jointly on all demonstrations repairs kinematics instead of learning tasks, so it needs no per-task training and can generate additional physically consistent variants of each demonstration.

“the key design is a single residual-RL policy (Johannink et al., 2019;Silver et al., 2018) trained jointly across all demonstrations, which repairs kinematics rather than learning tasks, and therefore needs no per-task training and can synthesize additional physically consistent variants of each demonstration.”

• cites Johannink et al., 2019 sourceDisputed

  The Method Summary describes a learned residual policy added to a fixed controller, but not a policy trained jointly on demonstrations. The Related Work passage distinguishes this method from learning through demonstrations, while the Environment Variation results report training corrective behavior for a specific block-insertion task. Those passages contradict the claim that the policy repairs demonstrations rather than learning tasks and needs no per-task training; our copy does not describe generating variants of demonstrations. (LLM, conf=0.85)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites Silver et al., 2018 sourcePartly supported

  Section IV (p. 3) supports the general idea of learning a residual policy that corrects an initial policy. But the paper describes training residuals for manipulation tasks, not one policy trained jointly on demonstrations. It does not establish kinematics-only repair, freedom from per-task training, or generation of physically consistent demonstration variants. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

May combine two claims.

Residual policy learning restores dynamic feasibility on the robot by learning small corrections to the retargeted reference, using the same residual formulation as physics refinement, so the policy transfers directly to hardware.

“Residual policy learning restores dynamic feasibility on the robot by learning small corrections to the retargeted reference, reusing the same residual formulation as physics refinement, so that the resulting policy transfers directly to hardware (Peng et al., 2018).”

• cites Peng et al., 2018 sourcePartly supported

  The Introduction (p. 1) reports that a policy trained in simulation transferred directly to a real robot. The Method (p. 3) describes training across randomized dynamics, not residual policy learning. Our copy does not support the claims about correcting a retargeted reference, restoring dynamic feasibility, or reusing a physics-refinement formulation. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

For multifingered hands, human control uses low-dimensional synergies rather than independent joints, so operators cannot intentionally control more than 20 coupled degrees of freedom through a retargeting layer they cannot perceive.

“human hand control is organized in low-dimensional synergies rather than independent joints, so operators cannot deliberately steer 20+ coupled degrees of freedom through a retargeting layer they cannot perceive (Santello et al., 1998;Li et al., 2026)”

• cites Santello et al., 1998 sourcePartly supported

  In “How many effective degrees of freedom are there?” (page unavailable), the paper reports that two principal components explain over 80% of hand-posture variance, supporting the claim about coordinated, low-dimensional control. But it also finds that higher-order components contribute to hand shape, and the Figure 11 discussion allows for finer control of individual fingers or joints. It does not establish that operators cannot deliberately steer 20-plus degrees of freedom through an imperceptible retargeting layer. (LLM, conf=0.90)

  Experimental — accuracy not yet measured
• cites Li et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Video by itself provides neither accurate hand-object pose nor contact force.

“Video alone, however, recovers neither precise hand-object pose nor contact force; EgoScale's action labels themselves come from instrumented gloves rather than pixels (Zheng et al., 2026).”

• cites Zheng et al., 2026 sourcePartly supported

  The Method (p. 3) and Human Data Sources and Processing (p. 4) describe hand-motion labels estimated from egocentric video; those estimates are noisy, which lends some support to the concern about pose precision. The p. 4 description says Manus gloves supply hand-pose data for the smaller aligned mid-training dataset, not EgoScale’s action labels generally. The cited passages do not establish a claim about recovering contact force from video. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Motion capture records hand and object kinematics with millimeter accuracy and without a robot in the loop.

“Motion capture occupies the precision end, recording millimeter-accurate hand and object kinematics without a robot in the loop (Wang et al., 2024).”

• cites Wang et al., 2024 sourcePartly supported

  The Introduction (p. 1) says mocap collects accurate 3D information without robot hardware, and the DexCap description (pp. 3–4) says it tracks finger motion and wrist poses. The Results (p. 8) describe object point clouds, not recorded object kinematics. The tracking-accuracy section (p. 20) reports 0.8 cm average hand-pose tracking error; it does not establish millimeter-accurate hand and object kinematics. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

For each setting, one policy is trained on all retargeted trajectories in Isaac Gym using PPO, with conditioning on task identifier, object geometry, and reference motion.

“As with physics refinement, a single policy per setting is trained across all retargeted trajectories in Isaac Gym (Makoviychuk et al., 2021) with PPO (Schulman et al., 2017), conditioned on task identifier, object geometry, and reference motion.”

• cites Makoviychuk et al., 2021 sourceSupported
• cites Schulman et al., 2017 sourceSupported

May combine two claims.

In the retargeting comparison, Vector achieved the best raw fidelity at 10.6 mm but had 29.9% joint-limit violations and 3.46 rad/s jitter.

“Vector achieves the best raw fidelity (10.6 mm) but incurs 29.9% joint-limit violations and 3.46 rad/s jitter.”

• cites Qin et al., 2023 sourceCould not verify

  The Hand Pose Retargeting section describes an optimization with joint-limit constraints and a temporal-smoothness penalty, but does not report a comparison involving Vector. Searches for Vector and the figures 10.6 mm, 29.9%, and 3.46 rad/s found no passage in our copy supporting the attributed results; Table II instead reports module runtimes. (LLM, conf=0.74)

  Experimental — accuracy not yet measured

May combine two claims.

Physics refinement uses a simulated actuated MANO hand in Isaac Gym to replay each demonstration, so the simulator's contact model supplies the contact forces absent from the raw kinematic data.

“Physics refinement recovers the contact forces missing from raw kinematic recordings by reexecuting each demonstration in Isaac Gym (Makoviychuk et al., 2021) with a simulated actuated MANO hand model, letting the simulator's contact model produce physically consistent forces.”

• cites Makoviychuk et al., 2021 sourceSupported

• AssumptionRests on an assumption: The physics simulator's contact model produces physically consistent forces that can substitute for unrecorded human contact forces.

May combine two claims.

Contact conditioning increased the average success rate from 34.7% to 46.5%, a gain of 11.8 percentage points, with the largest improvement on Cube 2 at 19.2 percentage points.

“Contact conditioning raises the average from 34.7% to 46.5% (+11.8 pp), with the largest gain on Cube 2 (+19.2pp), confirming the value of contact information for dexterous manipulation.”

• cites Chi et al., 2023 sourceCould not verify

  The Introduction and Key Findings report a 46.9% average improvement for Diffusion Policy across benchmarks, not a contact-conditioning comparison. Searches for contact conditioning, Cube 2, and the stated success rates found no passage in our copy reporting the claimed 34.7%-to-46.5% change or 19.2-percentage-point task gain. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

May combine two claims.

The pipeline reconstructs 25,454 single-hand trajectories, raising success from 7.3% to 59.3%, and 25 dual-hand tasks, raising success from 16.0% to 62.4%, with one shared policy per setting.

“The pipeline reconstructs 25,454 single-hand trajectories (success 7.3%→59.3%)and 25 dual-hand tasks (16.0%→62.4%)with one shared policy per setting”

May combine two claims.

For the dual-hand dataset, residual correction increases task-level success from 16.0% to 62.4% and attempt-level success from 8.4% to 43.9%.

“residual correction raises task-level success from 16.0% to 62.4% and attempt-level from 8.4% to 43.9%”

May combine two claims.

The single-hand dataset has 25,454 mocap trajectories across 14 objects and 85 object-gesture combinations; the dual-hand dataset has 250 trajectories across 25 tasks and 12 objects.

“The single-hand dataset contains 25,454 mocap trajectories (14 objects, 85 object-gesture combinations, with objects and gestures as defined in DexCanvas (Xu et al., 2025)); the dual-hand dataset contains 250 trajectories (25 tasks, 12 objects).”

• cites Xu et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The proposed retargeting method achieved 14.7 mm fidelity, 0.53 rad/s jitter, and 3.2% joint-limit violations, and was the only compared method to jointly achieve high fidelity, high retention, low violations, and low jitter.

“Ours is a close second in fidelity (14.7 mm) with the lowest jitter (0.53 rad/s) and only 3.2% violations-the only method that jointly achieves high fidelity, high retention, low violations, and low jitter.”

May combine two claims.

Without residual correction, replay succeeds on average only 7.26% of the time; with residual correction, the average rises to 59.31%.

“Replay without residual correction averages only 7.26%, whereas residual correction raises the average to 59.31%”

May combine two claims. The quote only partly states this claim.

On physical hardware, per-task success rates were 7/10 for stacking cups, 5/10 for passing and standing, 6/10 for pouring, and 3/10 for in-hand reorientation.

“Bimanual coordination tasks (passing & standing, pouring) show stable performance at 5/10 and 6/10 respectively; in-hand reorientation remains hardest (3/10) due to demanding continuous re-grasping where small timing errors compound.”

• Also stated in: § 4.3 From Data to Deployment · plain

  “with stacking cups performing best (7/10)”

May combine two claims.

Without residual correction, success rates ranged from 0.5% for DexHand 021 to 50.0% for Allegro Hand; with it, all four hands reached 76.5-82.5%.

“Without it, success varies from 0.5% (DexHand 021) to 50.0% (Allegro Hand); with it, all four hands converge to 76.5-82.5% (Fig. 6), demonstrating that a single training recipe generalizes across morphologically distinct embodiments.”

May combine two claims.

The shared policy, without fine-tuning, achieved a pooled success rate of 72.4% on six unseen YCB objects, with per-object rates ranging from 46.5% to 88.7%.

“is 72.4%, with per-object rates spanning 46.5%-88.7%, confirming generalization to novel object”

• Also stated in: § Toward in-context policy learning. · plain

  “Pooled | 7,683 | 72.4%”

• cites Calli et al., 2015 sourceNot checked — no full text of the source

May combine two claims.

Real-robot execution of four contact-rich bimanual dexterous tasks is achieved with a single residual policy shared across tasks, zero real-robot training data, and no per-task reward design or retraining.

“real-robot execution of four contact-rich bimanual dexterous tasks with a single residual policy shared across tasks, zero real-robot training data, and no per-task reward design or retraining (Sec.4.3).”

May combine two claims.

A single residual-RL policy, trained once across all tasks, converts raw kinematic recordings into physically consistent, contact-annotated data, removing per-task reward engineering from the real2sim2real loop.

“We show that a single residual-RL policy, trained once across all tasks, converts raw kinematic recordings into physically consistent, contact-annotated data, removing per-task reward engineering from the real2sim2real loop”

May combine two claims.

A kinematic recording has no forces, so simulation-based reconstruction is necessary.

“A kinematic recording contains no forces, so reconstruction in simulation is unavoidable”

May combine two claims.

Contact-anchored retargeting transfers the demonstrated contact structure to a target robot hand, and its objective depends only on surface geometry, not on the hand's kinematics.

“Contact-anchored retargeting transfers the demonstrated contact structure to a target robot hand: the optimization asks that designated finger surface regions reach the recorded contact locations on the object, and its objective depends only on surface geometry, not on the hand's kinematics.”

May combine two claims.

Learning from demonstrations avoids per-task reward engineering but makes data the bottleneck, since contact-rich demonstrations are difficult to collect at scale.

“Learning from demonstrations is the prevailing route to such skills, avoiding the per-task reward engineering of reinforcement learning (Andrychowicz et al., 2020;Chen et al., 2022), but it shifts the bottleneck to data: contact-rich demonstrations are precisely what is hard to collect at scale.”

• cites Andrychowicz et al., 2020 sourceNot checked — no full text of the source
• cites Chen et al., 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

A single residual reinforcement learning policy, trained once on many demonstrations, can turn kinematic recordings into physically consistent trajectories with contact annotations, and the same residual approach can restore dynamic feasibility after retargeting.

“a single residual reinforcement learning (RL) policy, trained once across diverse demonstrations, can repair kinematic recordings into physically consistent, contact-annotated trajectories, and the same residual formulation restores dynamic feasibility after retargeting.”

May combine two claims.

The paper's three-stage pipeline addresses both gaps without using teleoperated data at any stage.

“This paper presents a three-stage pipeline that closes both gaps with no teleoperated data at any stage (Fig. 1).”

May combine two claims.

Turning such demonstrations into deployable robot skills requires addressing two gaps: embodiment and physics.

“Converting such demonstrations into deployable robot skills therefore requires closing two gaps.”

May combine two claims.

The retargeting optimization uses forward-kinematic surface points, embodiment geometry, joint limits, and an active control vector; switching the target hand alters the optimization dimension and geometric queries but leaves the mathematical loss unchanged.

“The optimization operates on forward-kinematic surface points, embodiment geometry, joint limits, and an active control vector.Changing the target hand changes the optimization dimension and geometric queries but not the mathematical loss.”

May combine two claims.

In-hand reorientation was the hardest task because it requires continuous re-grasping where small timing errors compound.

“in-hand reorientation remains hardest (3/10) due to demanding continuous re-grasping where small timing errors compound.”

May combine two claims.

The real-world deployment used frozen policies and zero real-robot training data.

“All four tasks are executed with frozen policies and zero real-robot training data, confirming that the pipeline transfers human demonstrations to physical hardware.”

Classified as probably not a substantial claim.

A single shared policy is trained for each setting without any per-task tuning.

“One shared policy is trained per setting with no per-task tuning.”

May combine two claims.

The retargeting objective depends only on surface geometry, which suffices for transfer across hands.

“The retargeting objective depends only on surface geometry, and the experiments confirm that this suffices: a single human dataset reconstructs successfully on four morphologically distinct hands with identical configuration, and the formulation is the only one among those compared that jointly achieves high contact fidelity, low joint-limit violations, and low temporal jitter (Sec.4.2).”

Classified as probably not a substantial claim. May combine two claims.

The deployment performs no dynamics identification, policy fine-tuning, or online adaptation.

“No dynamics identification, policy fine-tuning, or online adaptation is performed.”

May combine two claims.

The retargeted trajectory puts the robot's fingers at the right contact locations in the right sequence, but it remains kinematic and ignores actuator torque limits, tendon coupling, and object dynamics under the robot's contact geometry.

“The retargeted trajectory places the robot's fingers at the correct contact locations in the correct order, but it is a kinematic solution: it does not account for actuator torque limits, tendon coupling, or object dynamics under the robot's contact geometry.”

Classified as probably not a substantial claim. The quote only partly states this claim.

The Diffusion Policy comparison shared architecture, data, and hyperparameters between the kinematic baseline and the contact-conditioned variant.

“Architecture, data, and hyperparameters are shared.”

May combine two claims.

Every variant keeps the task's contact pattern while exploring nearby dynamically feasible paths.

“Each variant preserves the task's contact structure while exploring neighboring dynamically feasible trajectories.”

May combine two claims.

The cross-embodiment evaluation used identical training configuration and no per-hand tuning.

“200 single-arm trajectories (3 objects, 7 gestures, 14 object-gesture combinations) are retargeted to four robot hands-DexHand 021 (Yuan et al., 2025), DexHand 021Pro, Shadow Hand, and Allegro Hand-with identical training configuration and no per-hand tuning.”

May combine two claims.

The success rates also indicate the quality of data synthesis: the trained policy acts as a data augmentation engine, perturbing initial object and hand poses within limits and filtering generated trajectories using the same success criteria.

“These success rates also reflect data synthesis quality-the trained policy serves as a data augmentation engine, randomly perturbing initial object and hand poses within a bounded range and filtering synthesized trajectories by the same success criteria.”

May combine two claims.

A second residual policy learns minor corrections to wrist translation and finger controls, while wrist orientation tracks the reference without change.

“A second residual policy (Eq.( 1)) learns small corrections to wrist translation and finger controls; wrist orientation follows the reference unchanged.”

Classified as probably not a substantial claim. The quote only partly states this claim.

The policy's weights remain frozen when deployed on hardware.

“Policy weights are frozen.”

May combine two claims.

The method is not specific to the DexHand 021 robot; changing the target hand requires only an updated embodiment profile and retraining the robot-side residual policy.

“The method itself is not specific to this robot: changing the target hand requires only an updated embodiment profile (Sec.3.3) and retraining the robot-side residual policy.”