CoChat
Sign Up
PreprintCited by 673Robot Manipulation and LearningReinforcement Learning in RoboticsTeleoperation and Haptic Systems

Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Open access
Zhao, Tony Z., Kumar, Vikash, Levine, Sergey, Finn, Chelsea·2023·arXiv
DOI 10.48550/arxiv.2304.13705arXiv 2304.13705
Open this paper in CoChat — free account

Abstract

Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback. Performing these tasks typically requires high-end robots, accurate sensors, or careful calibration, which can be expensive and difficult to set up. Can learning enable low-cost and imprecise hardware to perform these fine manipulation tasks? We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations, collected with a custom teleoperation interface. Imitation learning, however, presents its own challenges, particularly in high-precision domains: errors in the policy can compound over time, and human demonstrations can be non-stationary. To address these challenges, we develop a simple yet novel algorithm, Action Chunking with Transformers (ACT), which learns a generative model over action sequences. ACT allows the robot to learn 6 difficult tasks in the real world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, with only 10 minutes worth of demonstrations. Project website: https://tonyzhaozh.github.io/aloha/

Integrity signals

Registry

DataCite · 10.48550/arxiv.2304.13705

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 40 cited sources checked · 0 supported · 0 flagged

  • Findingnone given

    ALOHA enables teleoperation of precise tasks such as threading zip cable ties, picking credit cards out of wallets, and opening or closing ziploc bags.

    “• Precise tasks such as threading zip cable ties, picking credit cards out of wallets, and opening or closing ziploc bags.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    Unlike Kim et al., the system does not use special encoders, sensors, or machined components.

    “Unlike this previous system, we do not make use of special encoders, sensors, or machined components.”
    Quote in the paperSubstantial claim
  • Findingnone given

    Using delta joint positions as actions instead of target joint positions leads to degraded performance.

    “We also noted degraded performance when using delta joint positions as actions instead of target joint positions.”
    Quote in the paperNot a substantial claim
  • Findingcitation

    ALOHA enables teleoperation of contact-rich tasks such as inserting 288-pin RAM into a computer motherboard, turning pages of a book, and assembling chains and belts in NIST board #2.

    “• Contact-rich tasks such as inserting 288-pin RAM into a computer motherboard, turning pages of a book, and assembling the chains and belts in the NIST board #2 [4]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] 4. Assembly performance metrics and test methods, Apr 2022. URL https://www. nist.gov/el/intelligent-systems-division-73500/ robotic-grasping-and-manipulation-assembly/assembly.

      Not checked in the free run.

  • Findingown data

    Using L1 loss for reconstruction leads to more precise modeling of the action sequence than L2 loss.

    “We use L1 loss for reconstruction instead of the more common L2 loss: we noted that L1 loss leads to more precise modeling of the action sequence.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 47
  • Classified as probably not a substantial claim. 23
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingreasoning

    The system learns 6 fine manipulation skills directly in the real world, such as opening a translucent condiment cup and slotting a battery, with 80-90% success from only 10 minutes or 50 demonstration trajectories.

    “ACT allows the robot to learn 6 difficult tasks in the real world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, with only 10 minutes worth of demonstrations.”
    • Also stated in: § I. INTRODUCTION · plain
      “The synergy between these two parts allows learning of 6 fine manipulation skills directly in the real-world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, from only 10 minutes or 50 demonstration trajectories.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: High-quality human demonstrations can provide value by allowing the system to learn from human dexterity.

  • May combine two claims.

    Findingnone given

    ACT can learn precise, closed-loop behavior and drastically outperforms previous methods.

    “The imitation learning algorithm, Action Chunking with Transformers (ACT), is capable of learning precise, close-loop behavior and drastically outperforms previous methods.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    The custom 3D printed see-through fingers with gripping tape allow good visibility during delicate operations and robust grip even with thin plastic films.

    “This allows for good visibility when performing delicate operations, and robust grip even with thin plastic films.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    ALOHA was built within a 20k USD budget, comparable to a single research arm such as Franka Emika Panda.

    “With the design considerations above, we build the bimanual teleoperation setup ALOHA within a 20k USD budget, compa-”
    • Also stated in: § III. ALOHA: A LOW-COST OPEN-SOURCE HARDWARE SYSTEM FOR BIMANUAL TELEOPERATION · plain
      “rable to a single research arm such as Franka Emika Panda.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On Slide Ziploc and Slot Battery, ACT reaches 88% and 96% final success rates, while other methods make no progress beyond the first stage.

    “For the two real-world tasks Slide Ziploc and Slot Battery, ACT achieves 88% and 96% final success rates respectively, with other methods making no progress past the first stage.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Training ACT takes around 5 hours on a single 11G RTX 2080 Ti GPU, and inference takes around 0.01 seconds on the same machine.

    “The training takes around 5 hours on a single 11G RTX 2080 Ti GPU, and the inference time is around 0.01 seconds on the same machine.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    The OEM fingers are not versatile enough to handle fine manipulation tasks.

    “The OEM fingers, however, are not versatile enough to handle fine manipulation tasks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The rubber band load balancing mechanism partially counteracts gravity on the leader side, reduces operator effort, and makes longer teleoperation sessions (e.g., >30 minutes) possible.

    “We also design a rubber band load balancing mechanism that partially counteracts the gravity on the leader side.It reduces the effort needed from the operator and makes longer teleoperation sessions (e.g.>30 minutes) possible.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    ACT has around 80M parameters and is trained from scratch for each task.

    “The model has around 80M parameters, and we train from scratch for each task.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Prior methods can progress through the first two subtasks, but their final success rate stays below 30%.

    “While previous methods are able to make progress in the first two subtasks, the final success rate remains low, below 30%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    The weight of the leader robot prevents the user from moving too fast and dampens small vibrations.

    “The weight of the leader robot prevents the user from moving too fast, and also dampens small vibrations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    The system is built with only off-the-shelf robots and a handful of 3D printed parts, allowing non-experts to assemble it in less than 2 hours.

    “We build our system with only off-the-shelf robots and a handful of 3D printed parts, allowing non-experts to assemble it in less than 2 hours.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    Temporal ensembling incurs no additional training cost, only extra inference-time computation.

    “This procedure also incurs no additional training cost, only extra inference-time computation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    On the two simulated tasks with scripted or human data, ACT improves success rate over the best previous method by 59%, 49%, 29%, and 20%.

    “For the two simulated tasks with scripted or human data, ACT outperforms the best previous method in success rate by 59%, 49%, 29%, and 20%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both action chunking and temporal ensembling are important for the success of ACT, which produces precise and smooth motion.

    “In practice, we find both action chunking and temporal ensembling to be important for the success of ACT, which produces precise and smooth motion.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Existing imitation learning algorithms perform poorly on fine-grained tasks that require high-frequency control and closed-loop feedback.

    “As we will see in Section V, existing imitation learning algorithms perform poorly on fine-grained tasks that require high-frequency control and closed-loop feedback.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    ACT attains the highest success rate among all compared prior methods, beating the second-best method by a large margin on every task.

    “ACT achieves the highest success rate compared to all prior methods, outperforming the second best algorithm by a large margin on each task.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Some tasks remain beyond the capability of the robots or the learning algorithm, such as buttoning a dress shirt.

    “While the system is quite capable, there exist tasks that are beyond the capability of either the robots or the learning algorithm, such as buttoning up a dress shirt.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Temporal Ensemble improves our method and BC-ConvMLP but harms VINN.

    “(b) Temporal Ensemble (TE) improves our method and BC-ConvMLP, while hurting VINN.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    In another evaluation with 10 trials per candy across 5 candies, the policy successfully unwraps 3 of 5 candies.

    “For this protocol, our policy successfully unwraps 3/5 candies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Performance rises sharply from 1% at k = 1 to 44% at k = 100, then declines slightly as k increases further.

    “We observe that performance improves drastically from 1% at k = 1 to 44% at k = 100, then slightly tapers down with higher k.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Temporal ensembling gives BC-ConvMLP its largest gain at 4%, followed by a 3.3% gain for ACT.

    “In Figure 8 (b), we show that BC-ConvMLP benefits from temporal ensembling the most with a 4% gain, followed by a 3.3% gain for our method.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For opening a small ziploc bag, a policy trained with 50 demonstrations consistently picks up the bag but struggles with the following three mid-air manipulation steps.

    “Our policy trained with 50 demonstrations can consistently pick up the bag, while having difficulties performing the following 3 mid-air manipulation steps.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    A large hidden dimension generally helps performance.

    “Large hidden dimension also generally helps.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    For BeT, increasing the history length from 10 to 100 greatly improves performance.

    “For BeT, we found that increasing history length from 10 (as in original paper) to 100 greatly improves the performance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Three demonstration videos include fifteen example uses of the Shadow Teleoperation System.

    “Specifically, we found three demonstration videos [56,57,58] that contain 15 example use cases of the Shadow Teleoperation System, and seek to recreate them using ALOHA.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [56] 56. srcteam. Shadow teleoperation system plays jenga, Mar 2021. URL https://www.youtube.com/watch?v= 7K9brH27jvM.

      Not checked in the free run.

    • cites [57] 57. srcteam. How researchers are using shadow robot's technology, Jun 2022. URL https://www.youtube.com/ watch?v=p36fYIoTD8M.

      Not checked in the free run.

    • cites [58] 58. srcteam. Shadow teleoperation system, Jun 2022. URL https://www.youtube.com/watch?v=cx8eznfDUJA.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    For the remaining three real-world tasks, the comparison is only against BeT, which had the highest task success rate so far.

    “For these tasks, we only compare with BeT, which has the highest task success rate so far.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    ALOHA is a bimanual setup that costs $18k, or $20k with optional add-ons such as cameras.

    “ALOHA, on the other hand, is a bimanual setup that costs $18k ($20k after adding optional add-ons such as cameras).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The observed Thread Velcro failures are that the right arm closes its gripper too early in stage 2 and misses the cable tie tail mid-air, and that stage 3 insertion is too imprecise and misses the loop.

    “The failure modes we observe are 1) at stage 2, the right arm closes its gripper too early and fails to grasp the tail of the cable tie mid-air, and 2) in stage 3, the insertion is not precise enough and misses the loop.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Adding action chunking to BC-ConvMLP and VINN produces trends similar to ACT, with more chunking improving performance.

    “We visualize their performance in Figure 8 (a) with different k, showing trends consistent with ACT, where more action chunking improves performance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    ACT reaches 84% success on Cup Open, 20% on Thread Velcro, 64% on Prep Tape, and 92% on Put On Shoe, while BeT achieves zero final success on these tasks.

    “Our method ACT reaches 84% success for Cup Open, 20% for Thread Velcro, 64% for Prep Tape and 92% for Put On Shoe, again outperforming BeT, which achieve zero final success on these challenging tasks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The paper reports all two tasks it attempted where ACT failed to learn the behavior.

    “On the software front, we report all 2 tasks that we attempted where ACT failed to learn the behavior.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants took an average of 33 seconds to thread the zip tie at 5 Hz, which fell to 20 seconds at 50 Hz.

    “On average, it took 33s for participants to thread the zip tie at 5Hz, which is lowered to 20s at 50Hz.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In a preliminary 10-trial evaluation on unwrapping candies, the policy picked up the candy 10/10, pulled both ends 8/10, and unwrapped the candy 0/10.

    “In our preliminary evaluation with 10 trials, the policy picks up the candy 10/10, pulls on both ends 8/10, while unwraps the candy 0/10.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    The ViperX arm has a working payload of 750g, 1.5m span, and accuracy of 5-8mm.

    “The ViperX arm used has a working payload of 750g and 1.5m span, with an accuracy of 5-8mm.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] 1. Viperx 300 robot arm 6dof. URL https://www. trossenrobotics.com/viperx-300-robot-arm-6dof.aspx.

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    A naive implementation of action chunking can be suboptimal because a new observation is incorporated abruptly every k steps and can result in jerky robot motion.

    “A naïve implementation of action chunking can be suboptimal: a new environment observation is incorporated abruptly every k steps and can result in jerky robot motion.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    An end-to-end policy that maps RGB images from commodity web cameras directly to actions is especially suitable for fine manipulation because learning the policy is simpler than modeling the environment.

    “This pixel-to-action formulation is particularly suitable for fine manipulation, because fine manipulation often involves objects with complex physical properties, such that learning the manipulation policy is much simpler than modeling the whole environment.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Fine manipulation tasks require precision, coordinated contact forces, and closed-loop visual feedback.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Higher β results in less information transmitted in z.

    “Intuitively, higher β will result in less information transmitted in z [62].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] 62. Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. 2015 IEEE Information Theory Workshop (ITW), pages 1-5, 2015.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Behavioral cloning is one of the simplest imitation learning algorithms, casting imitation as supervised learning from observations to actions.

    “Behavioral cloning (BC) [44] is one of the simplest imitation learning algorithms, casting imitation as supervised learning from observations to actions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] 44. Dean A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In NIPS, 1988.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The teleoperation setup is most similar to Kim et al. [32], which also uses joint-space mapping between leader and follower robots.

    “Our teleoperation setup is most similar to Kim et al. [32], which also uses joint-space mapping between the leader and follower robots.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] 32. Heecheol Kim, Yoshiyuki Ohmura, and Yasuo Kuniyoshi. Robot peels banana with goal-conditioned dual-action deep imitation learning. ArXiv, abs/2203.09749, 2022.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Expert annotation for on-policy corrections can be time-consuming and unnatural with a teleoperation interface.

    “However, expert annotation can be time-consuming and unnatural with a teleoperation interface [29].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] 29. Liyiming Ke, Jingqiang Wang, Tapomayukh Bhattachar- jee, Byron Boots, and Siddhartha Srinivasa. Grasping with chopsticks: Combating covariate shift in model-free imitation learning for fine manipulation. In International Conference on Robotics and Automation (ICRA), 2021.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Early classical control bimanual manipulation works require known environment dynamics, but designing such models can be time-consuming and may not be accurate for objects with complex physical properties.

    “Early works tackle bimanual manipulation from a classical control perspective, with known environment dynamics [54,48], but designing such models can be timeconsuming, and they may not be accurate for objects with complex physical properties.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [54] 54. Christian Smith, Yiannis Karayiannidis, Lazaros Nal- pantidis, Xavi Gratal, Peng Qi, Dimos V. Dimarogonas, and Danica Kragic. Dual arm manipulation -a survey. Robotics Auton. Syst., 60:1340-1353, 2012.

      Not checked in the free run.

    • cites [48] 48. Seyed Sina Mirrazavi Salehian, Nadia Figueroa, and Aude Billard. A unified framework for coordinated multi-arm motion planning. The International Journal of Robotics Research, 37:1205 -1232, 2018.

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Action chunking can mitigate temporally correlated confounders when the confounder is within a chunk, without introducing causal confusion for history-conditioned policies.

    “Action chunking can mitigate this issue when the confounder is within a chunk, without introducing the causal confusion issue for history-conditioned policies [12].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] 12. Pim de Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. In Neural Information Processing Systems, 2019.

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Joint-space mapping from a smaller leader robot guarantees high-bandwidth control within joint limits, requires less computation, and reduces latency compared to task-space mapping.

    “Joint space mapping, on the other hand, guarantees high-bandwidth control within the joint limits, while also requiring less computation and reducing latency.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    People lack industrial-grade proprioception but still perform delicate tasks by using closed-loop visual feedback and compensating for errors.

    “Humans also do not have industrial-grade proprioception [71], and yet we are able to perform delicate tasks by learning from closed-loop visual feedback and actively compensating for errors.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [71] 71. Áron Horváth, Eszter Ferentzi, Kristóf Schwartz, Nina Jacobs, Pieter Meyns, and Ferenc Köteles. The measure- ment of proprioceptive accuracy: A systematic literature review. Journal of Sport and Health Science, 2022. ISSN 2095-2546. doi: https://doi.org/10.1016/j.jshs.2022.04. 001. URL https://www.sciencedirect.com/science/article/ pii/S2095254622000473.

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Modeling contact and deformation for tasks like nudging a cup or prying a lid involves complex physics with many degrees of freedom, so an accurate planning model would require substantial research and task-specific engineering.

    “Take the condiment cup example: modeling the contact when nudging the cup, and also the deformation when prying open the lid involves complex physics on a large number of degrees of freedom.Designing a model accurate enough for planning would require significant research and task specific engineering efforts.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Temporal ensembling aggregates actions predicted for the same timestep, unlike typical smoothing that aggregates current action with adjacent timesteps and leads to bias.

    “We note that unlike typical smoothing, where the current action is aggregated with actions in adjacent timesteps, which leads to bias, we aggregate actions predicted for the same timestep.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The weak performance of prior methods is attributed to compounding errors and non-Markovian behavior in the data, with behavior degrading near the end of episodes and the robot sometimes pausing indefinitely.

    “We attribute the poor performance of prior methods to compounding errors and non-Markovian behavior in the data: the behavior degrades significantly towards the end of an episode, and the robot can pause indefinitely for certain states.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Thread Velcro requires visual feedback to correct perturbations at each grasp, because a few millimeters of error in the first grasp compounds in the mid-air second grasp and produces more than 10 mm of deviation during insertion.

    “For this task to be successful, the robot must use visual feedback to correct for perturbations with each grasp, as even a few millimeters of error during the first grasp will compound in the second grasp mid-air, giving more than a 10mm deviation in the insertion phase.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    In the simulated Transfer Cube task, the roughly 1 cm clearance between the cube and the left gripper means small errors can cause collisions and task failure.

    “Due to the small clearance between the cube and the left gripper (around 1cm), small errors could result in collisions and task failure.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    To succeed, the policy must learn that the two grippers should never collide during handover and that the left gripper should move to a graspable position rather than memorize the exact handover location.

    “Thus to successfully perform the task, the policy will need to learn that the two grippers should never collide with each other during the handover, and the left gripper should always move to a position that can grasp the tape, instead of trying to memorize where exactly the handover happens, which can vary across demonstrations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    More action chunking and a shorter effective horizon generally improve performance.

    “This illustrates that more chunking and a lower effective horizon generally improve performance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S4 (based). May combine two claims.

    Interpretationnone given

    The ziploc bag is hard to perceive, and small differences in pickup position can change how the bag deforms, causing large differences in the pulling region's location.

    “We hypothesize that the bag is hard to perceive, and in addition, small differences in the pick up position can affect how the bag deforms, and result in large differences in where the pulling region ends up.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    ACT reduces both compounding errors and non-Markovian behavior through action chunking.

    “ACT mitigates both issues with action chunking.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    The Shadow Teleoperation System is the most capable among the compared prior systems, due to its bimanual design.

    “This system is the most capable among all aforementioned works, benefitted from its bimanual design.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] 5. Teleoperated robots -shadow teleoperation system, Nov 2022. URL https://www.shadowrobot.com/teleoperation/.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    More recently, RE2 and other companies also built highly dexterous teleoperation systems using joint-space mapping.

    “More recently, companies like RE2 [3] also built highly dexterous teleoperation systems with joint-space mapping.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] 3. Highly dexterous manipulation system -capabilities -part 1, Nov 2014. URL https://www.youtube.com/watch?v= TearcKVj0iY.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Reducing dexterous hands to parallel jaw grippers allows use of light-weight, low-cost robots that can be more nimble and require less service.

    “Reducing dexterous hands to parallel jaw grippers allows us to use light-weight and low-cost robots, which can be more nimble and require less service.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    ALOHA is similar to previous teleoperation systems that use joint-space mapping, but benefits from recent low-cost actuators and robot arms.

    “ALOHA is similar to these previous systems, while benefiting significantly from recent advances of low-cost actuators and robot arms.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The objects used in the tasks create a significant perception challenge, with transparent or translucent items being hard to perceive precisely and poorly suited to depth cameras, and the black tabletop giving low contrast.

    “In addition to the delicate bimanual control required to solve these tasks, the objects we use also present a significant perception challenge.”
    • Also stated in: § A. Tasks · plain
      “Other transparent or translucent objects include the tape and both the lid and body of the condiment cup, making them hard to perceive precisely and ill-suited for depth cameras.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A leader-follower design dates back to at least 1953, when Central Research Laboratories built teleoperation systems for hazardous material.

    “A leader-follower design choice dates back to at least as far as 1953, when Central Research Laboratories built teleoperation systems for handling hazardous material [27].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] 27. R G Jenness and C D Wicker. Master-slave manipulators and remote maintenance at the oak ridge national labora- tory, Jan 1975. URL https://www.osti.gov/biblio/4179544.

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    ACT uses action chunking and temporal ensembling to reduce compounding errors and to better handle non-Markovian demonstrations.

    “ACT employs action chunking and temporal ensembling to mitigate compounding errors and better handle non-Markovian demonstrations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    ALOHA achieves similar levels of dexterity to prior teleoperation systems at much lower cost and without specialized hardware or expert assembly.

    “It allows us to achieve similar levels of dexterity with much lower cost, and also without specialized hardware or expert assembly.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The 15 example use cases from the Shadow Teleoperation System videos are representative of that system's capabilities.
    • Rests on an assumption: The tasks recreated with similar objects and comparable time are a valid measure of dexterity.

  • May combine two claims.

    Interpretationown data

    The candy-unwrapping failure is attributed to perception difficulty and insufficient data.

    “We attribute the failure to the difficulty of perception and lack of data.”
    Quote in the paperNot a substantial claim
63 of 63

  • The paper relies on this without support.

    Fine manipulation tasks require precision, coordinated contact forces, and closed-loop visual feedback.

    “Fine manipulation tasks involve precise, closed-loop feedback and require high degrees of hand-eye coordination to adjust and re-plan in response to changes in the environment.”

  • The paper relies on this without support.

    Low-cost hardware is inevitably less precise than high-end platforms.

    “However, low-cost hardware is inevitably less precise than high-end platforms, making the sensing and planning challenge more pronounced.”

  • The paper relies on this without support.

    High-quality human demonstrations can provide value by allowing the system to learn from human dexterity.

    “in the case of fine manipulation, high-quality human demonstrations can provide tremendous value by allowing the system to learn from human dexterity.”

  • The paper relies on this without support.

    Given the same observation, a human can use different trajectories to solve the task.

    “Given the same observation, a human can use different trajectories to solve the task.”

  • The paper relies on this without support.

    The amount of force applied is implicitly defined by the difference between leader and follower joint positions through the low-level PID controller.

    “because the amount of force applied is implicitly defined by the difference between them, through the low-level PID controller.”

  • The paper relies on this without support.

    The eight selected tasks are representative of fine-grained bimanual manipulation.

    “All 8 tasks require fine-grained, bimanual manipulation”
    “We present experiments to evaluate ACT's performance on fine manipulation tasks.”

  • The paper relies on this without support.

    The 15 example use cases from the Shadow Teleoperation System videos are representative of that system's capabilities.

    “Specifically, we found three demonstration videos [56,57,58] that contain 15 example use cases of the Shadow Teleoperation System, and seek to recreate them using ALOHA.”

  • The paper relies on this without support.

    The two baseline methods used to test action chunking are representative of imitation learning methods.

    “Both methods significantly benefit from action chunking, suggesting that it is a generally useful technique.”

  • The paper relies on this without support.

    The tasks recreated with similar objects and comparable time are a valid measure of dexterity.

    “We are able to recreate 14 out of the 15 tasks with similar objects and comparable amount of time.”
    “It allows us to achieve similar levels of dexterity with much lower cost, and also without specialized hardware or expert assembly.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior transformer-based policy designs typically encode input images with large ResNet-style encoders and fuse the resulting image features with a comparatively small transformer.

    “Prior transformer-based policy designs typically encode input images with large ResNetstyle [32] encoders and fuse the resulting image features with a comparatively small transformer [10, 67, 81, 17, 98, 61, 83].”

    From Octo: An Open-Source Generalist Robot Policy (2024) · cites this paper as [98]

    In “C. Implementing ACT” (page unavailable), the paper says its policy processes images with ResNet18 backbones and passes the resulting features to a transformer encoder and decoder. The Figure 4 description shows four encoder blocks and seven decoder blocks. These passages support the architecture described, but do not establish that its ResNet18 encoders are large relative to its transformer or that this design is typical of prior policies. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Action chunking has recently emerged as the de facto standard in imitation learning for visuomotor control.

    “Inspired in part by human motor control [33], action chunking has recently emerged as the de facto standard in imitation learning for visuomotor control [68, 11].”

    From Real-Time Execution of Action Chunking Flow Policies (2025) · cites this paper as [68]

    The Introduction and the Action Chunking and Temporal Ensembling section describe an action-chunking imitation-learning method for image-based robot control and report benefits in the tasks tested. They do not establish that action chunking has become the de facto standard across imitation learning for visuomotor control; the Experiment Results section instead describes a widely used baseline that predicts one action at a time. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Action chunking has enabled temporal consistency in high-frequency control tasks.

    “While action chunking has enabled temporal consistency in high-frequency control tasks, it does not fully address the latency problem, leading to pauses or out-of-distribution jerky movements at chunk boundaries.”

    From Real-Time Execution of Action Chunking Flow Policies (2025) · cites this paper as [68]

    In “Action Chunking and Temporal Ensemble” (page not given), the paper says chunking helps model temporally correlated behavior in high-frequency trajectories. It also says chunking alone can produce jerky motion at chunk boundaries and credits temporal ensembling with improving smoothness, so it does not establish that chunking by itself enables temporal consistency. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Octo's action head predicts a chunk of several consecutive actions.

    “This action head predicts a 'chunk”

    From Octo: An Open-Source Generalist Robot Policy (2024) · cites this paper as [98]

    The Introduction and Section IV say that ACT predicts a sequence of future actions, and the Action Chunking and Temporal Ensemble section specifies the next k actions. These passages support the action-chunking mechanism, but they describe ACT, not Octo’s action head; the cited paper does not establish that Octo uses it. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Improving the inference throughput of OpenVLA is critical to enable VLA control for high-frequency control setups such as ALOHA, which runs at 50Hz.

    “Secondly, improving the inference throughput of OpenVLA is critical to enable VLA control for high-frequency control setups such as ALOHA [90], which runs at 50Hz.”

    From OpenVLA: An Open-Source Vision-Language-Action Model (2024) · cites this paper as [90]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The stated reconstruction loss differs from the algorithm

    Major

    The architecture section says ACT uses L1 reconstruction loss, but Algorithm 1 specifies mean squared error for that loss.

  • Four real-world tasks remain, not three

    Minor

    The paper specifies six real-world tasks and discusses two in the detailed comparison, leaving four. The following paragraph also names four tasks.

    Reported 3 · recomputed 4

  • Figure 10's task count disagrees with its description

    Minor

    The text says Figure 10 provides observations for each of the six real-world tasks, whereas its caption says it shows five.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 39

  1. [8]8. Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nico- las Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. ArXiv, abs/2005.12872, 2020. · arXiv
  2. [10]10. Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, and Abhinav Kumar Gupta. Efficient bimanual manipulation using learned task schemas. 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1149-1155, 2019.
  3. [11]11. Sudeep Dasari and Abhinav Kumar Gupta. Transformers for one-shot visual imitation. In Conference on Robot Learning, 2020.
  4. [12]12. Pim de Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. In Neural Information Processing Systems, 2019.
  5. [16]16. Peter R. Florence, Lucas Manuelli, and Russ Tedrake. Self- supervised correspondence in visuomotor policy learning. IEEE Robotics and Automation Letters, 5:492-499, 2019.
  6. [18]18. Aditya Ganapathi, Priya Sundaresan, Brijen Thananjeyan, Ashwin Balakrishna, Daniel Seita, Jennifer Grannen, Minho Hwang, Ryan Hoque, Joseph Gonzalez, Nawid Jamali, Katsu Yamane, Soshi Iba, and Ken Goldberg. Learning dense visual correspondences in simulation to smooth and fold real fabrics. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 11515-11522, 2020.
  7. [19]19. Jennifer Grannen, Priya Sundaresan, Brijen Thananjeyan, Jeffrey Ichnowski, Ashwin Balakrishna, Minho Hwang, Vainavi Viswanath, Michael Laskey, Joseph Gonzalez, and Ken Goldberg. Untangling dense knots by learning task-relevant keypoints. In Conference on Robot Learning, 2020.
  8. [21]21. Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu- Wei Chao, Qian Wan, Stan Birchfield, Nathan D. Ratliff, and Dieter Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9164-9170, 2019.
  9. [22]22. Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2015.
  10. [23]23. Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2016.
  11. [24]24. Ryan Hoque, Ashwin Balakrishna, Ellen R. Novoseller, Albert Wilcox, Daniel S. Brown, and Ken Goldberg. Thriftydagger: Budget-aware novelty and risk gating for interactive imitation learning. In Conference on Robot Learning, 2021.
  12. [26]26. Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning, 2022.
  13. [27]27. R G Jenness and C D Wicker. Master-slave manipulators and remote maintenance at the oak ridge national labora- tory, Jan 1975. URL https://www.osti.gov/biblio/4179544.
  14. [28]28. Edward Johns. Coarse-to-fine imitation learning: Robot manipulation from a single demonstration. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 4613-4619, 2021.
  15. [29]29. Liyiming Ke, Jingqiang Wang, Tapomayukh Bhattachar- jee, Byron Boots, and Siddhartha Srinivasa. Grasping with chopsticks: Combating covariate shift in model-free imitation learning for fine manipulation. In International Conference on Robotics and Automation (ICRA), 2021.
  16. [30]30. Michael Kelly, Chelsea Sidrane, K. Driggs-Campbell, and Mykel J. Kochenderfer. Hg-dagger: Interactive imitation learning with human experts. 2019 International Conference on Robotics and Automation (ICRA), pages 8077-8083, 2018.
  17. [31]31. Heecheol Kim, Yoshiyuki Ohmura, and Yasuo Kuniyoshi. Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation. IEEE Robotics and Automation Letters, 6:1630-1637, 2021.
  18. [34]34. Oliver Kroemer, Christian Daniel, Gerhard Neumann, Herke van Hoof, and Jan Peters. Towards learning hierarchical skills for multi-phase manipulation tasks. 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 1503-1510, 2015.
  19. [35]35. Lucy Lai, Ann Z Huang, and Samuel J Gershman. Action chunking as policy compression, Sep 2022. URL psyarxiv. com/z8yrv.
  20. [36]36. Michael Laskey, Jonathan Lee, Roy Fox, Anca D. Dragan, and Ken Goldberg. Dart: Noise injection for robust imitation learning. In Conference on Robot Learning, 2017.
  21. [37]37. Alex X. Lee, Henry Lu, Abhishek Gupta, Sergey Levine, and P. Abbeel. Learning force-based manipulation of deformable objects from multiple demonstrations. 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 177-184, 2015.
  22. [39]39. Ajay Mandlekar, Danfei Xu, J. Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Mart'in-Mart'in. What matters in learning from offline human demonstrations for robot manipulation. In Conference on Robot Learning, 2021.
  23. [40]40. Kunal Menda, K. Driggs-Campbell, and Mykel J. Kochen- derfer. Ensembledagger: A bayesian approach to safe imitation learning. 2019 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), pages 5041-5048, 2018.
  24. [41]41. Samuel Paradis, Minho Hwang, Brijen Thananjeyan, Jeffrey Ichnowski, Daniel Seita, Danyal Fer, Thomas Low, Joseph Gonzalez, and Ken Goldberg. Intermittent visual servoing: Efficiently learning policies robust to instrument changes for high-precision surgical manipula- tion. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 7166-7173, 2020.
  25. [43]43. Peter Pastor, Heiko Hoffmann, Tamim Asfour, and Stefan Schaal. Learning and generalization of motor skills by learning from demonstration. 2009 IEEE International Conference on Robotics and Automation, pages 763-768, 2009.
  26. [44]44. Dean A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In NIPS, 1988.
  27. [45]45. Yuzhe Qin, Hao Su, and Xiaolong Wang. From one hand to multiple hands: Imitation learning for dexterous manipulation from single-camera teleoperation. IEEE Robotics and Automation Letters, 7:10873-10881, 2022.
  28. [46]46. Rouhollah Rahmatizadeh, Pooya Abolghasemi, Ladislau Bölöni, and Sergey Levine. Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration. 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 3758-3765, 2017.
  29. [47]47. Stéphane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics, 2010.
  30. [48]48. Seyed Sina Mirrazavi Salehian, Nadia Figueroa, and Aude Billard. A unified framework for coordinated multi-arm motion planning. The International Journal of Robotics Research, 37:1205 -1232, 2018.
  31. [55]55. Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In NIPS, 2015.
  32. [59]59. Simon Stepputtis, Maryam Bandari, Stefan Schaal, and Heni Ben Amor. A system for imitation learning of contact-rich bimanual manipulation policies. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11810-11817, 2022.
  33. [61]61. Gokul Swamy, Sanjiban Choudhury, J. Andrew Bagnell, and Zhiwei Steven Wu. Causal imitation learning under temporally correlated noise. In International Conference on Machine Learning, 2022.
  34. [62]62. Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. 2015 IEEE Information Theory Workshop (ITW), pages 1-5, 2015.
  35. [63]63. Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. 2012 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems, pages 5026-5033, 2012.
  36. [64]64. Stephen Tu, Alexander Robey, Tingnan Zhang, and N. Matni. On the sample complexity of stability con- strained imitation learning. In Conference on Learning for Dynamics & Control, 2021.
  37. [68]68. Andy Zeng, Peter R. Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, and Johnny Lee. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning, 2020.
  38. [69]69. Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Ken Goldberg, and P. Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1-8, 2017.
  39. [71]71. Áron Horváth, Eszter Ferentzi, Kristóf Schwartz, Nina Jacobs, Pieter Meyns, and Ferenc Köteles. The measure- ment of proprioceptive accuracy: A systematic literature review. Journal of Sport and Health Science, 2022. ISSN 2095-2546. doi: https://doi.org/10.1016/j.jshs.2022.04. 001. URL https://www.sciencedirect.com/science/article/ pii/S2095254622000473. · DOI

Cited by

Sorted by citations

Related

Authors
Zhao, T. Z., Kumar, V., Levine, S., & Finn, C.
Affiliations
  • Stanford University (US)
  • University of California, Berkeley (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2304.13705
arXiv
2304.13705
OpenAlex
W4385430674
Cited by
673
References
20
Citations by year
2026: 276 · 2025: 310 · 2024: 86 · 2023: 1
FWCI
287.82
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Robotics: Science and Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Office of Naval Research (N00014-21-1-2685, N00014)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems
Keywords
bimanual manipulation, fine manipulation, low-cost robotics, open-source hardware, teleoperation, imitation learning, contact-rich tasks, robotic grasping, ALOHA

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review