CoChat
Sign Up
PreprintCited by 16Teleoperation and Haptic SystemsRobot Manipulation and LearningReinforcement Learning in Robotics

PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection

Open access
Dass, Shivin, Pertsch, Karl, Zhang, Hejia, Lee, Youngwoon, Lim, Joseph J., Nikolaidis, Stefanos·2022·arXiv
DOI 10.48550/arxiv.2212.04708arXiv 2212.04708
Open this paper in CoChat — free account

Abstract

Large-scale data is an essential component of machine learning as demonstrated in recent advances in natural language processing and computer vision research. However, collecting large-scale robotic data is much more expensive and slower as each operator can control only a single robot at a time. To make this costly data collection process efficient and scalable, we propose Policy Assisted TeleOperation (PATO), a system which automates part of the demonstration collection process using a learned assistive policy. PATO autonomously executes repetitive behaviors in data collection and asks for human input only when it is uncertain about which subtask or behavior to execute. We conduct teleoperation user studies both with a real robot and a simulated robot fleet and demonstrate that our assisted teleoperation system reduces human operators' mental load while improving data collection efficiency. Further, it enables a single operator to control multiple robots in parallel, which is a first step towards scalable robotic data collection. For code and video results, see https://clvrai.com/pato

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2212.04708

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior work has attempted to develop systems that can selectively query humans for demonstrations when they are needed, in order to reduce human operator time and burden.

    “Prior work has also attempted to develop systems that can selectively query humans for demonstrations when they are needed, in order to reduce human operator time and burden [67-70].”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [70]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 43

  1. David Acuna, Huan Ling, Amlan Kar, and Sanja Fidler. Efficient interactive annotation of segmentation datasets with polygon-rnn++. In CVPR, pages 859-868, 2018.
  2. Heni Ben Amor, Erik Berger, David Vogt, and Bernhard Jung. Kinesthetic bootstrapping: Teaching motor skills to humanoid robots through physical interaction. In Annual conference on artificial intelligence, 2009.
  3. Brenna D Argall. Autonomy in rehabilitation robotics: An intersection. Annual Review of Control, Robotics, and Autonomous Systems, 1:441, 2018.
  4. Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. A survey of robot learning from demonstration. Robotics and autonomous systems, 57(5): 469-483, 2009.
  5. Pierre Berthet-Rayne, Maura Power, Hawkeye King, and Guang-Zhong Yang. Hubot: A three state human-robot collaborative framework for bimanual surgical tasks based on learned models. In ICRA, pages 715-722. IEEE, 2016.
  6. Anthony Brohan, Noah Brown, Justice Carbajal, Yev- gen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalash- nikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Van- houcke, Steve Vega, Quan Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-1: Robotics transformer for real-world control at scale. In arXiv preprint arXiv:2212.06817, 2022. · arXiv
  7. Lluis Castrejon, Kaustav Kundu, Raquel Urtasun, and Sanja Fidler. Annotating object instances with a polygon- rnn. In CVPR, 2017.
  8. Henry M Clever, Ankur Handa, Hammad Mazhar, Kevin Parker, Omer Shapira, Qian Wan, Yashraj Narang, Iretiayo Akinola, Maya Cakmak, and Dieter Fox. Assistive tele-op: Leveraging transformers to collect robotic task demonstrations. arXiv preprint arXiv:2112.05129, 2021. · arXiv
  9. Jacob W Crandall and Michael A Goodrich. Characteriz- ing efficiency of human robot interaction: A case study of shared-control teleoperation. In IEEE/RSJ international conference on intelligent robots and systems, volume 2, pages 1290-1295. IEEE, 2002.
  10. Anca D Dragan and Siddhartha S Srinivasa. A policy- blending formalism for shared control. The International Journal of Robotics Research, 32(7):790-805, 2013.
  11. Matthew Fontaine and Stefanos Nikolaidis. A quality diversity approach to automatically generating human- robot interaction scenarios in shared autonomy. arXiv preprint arXiv:2012.04283, 2020. · arXiv
  12. Matthew C Fontaine and Stefanos Nikolaidis. Evaluating human-robot interaction algorithms in shared autonomy via quality diversity scenario generation. ACM Transac- tions on Human-Robot Interaction (THRI), 11(3):1-30, 2022.
  13. Yang Gao and Steve Chien. Review on space robotics: To- ward top-level science through space exploration. Science Robotics, 2(7):eaan5074, 2017.
  14. Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solv- ing long-horizon tasks via imitation and reinforcement learning. CoRL, 2019.
  15. Sandra G Hart and Lowell E Staveland. Development of nasa-tlx (task load index): Results of empirical and theo- retical research. In Advances in psychology, volume 52, pages 139-183. Elsevier, 1988.
  16. Sepp Hochreiter and Jürgen Schmidhuber. Long short- term memory. Neural computation, 9(8):1735-1780, 1997.
  17. Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S Brown, and Ken Goldberg. Thriftydagger: Budget-aware novelty and risk gating for interactive imitation learning. CoRL, 2021.
  18. Ryan Hoque, Ashwin Balakrishna, Carl Putterman, Michael Luo, Daniel S Brown, Daniel Seita, Bri- jen Thananjeyan, Ellen Novoseller, and Ken Goldberg. Lazydagger: Reducing context switching in interactive imitation learning. In 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), 2021.
  19. Mishel Johns, Brian Mok, David Sirkin, Nikhil Gowda, Catherine Smith, Walter Talamonti, and Wendy Ju. Ex- ploring shared control in automated driving. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 91-98. IEEE, 2016.
  20. Michael Kelly, Chelsea Sidrane, Katherine Driggs- Campbell, and Mykel J Kochenderfer. Hg-dagger: In- teractive imitation learning with human experts. In ICRA, 2019.
  21. David Kortenkamp, R Peter Bonasso, Dan Ryan, and Debbie Schreckenghost. Traded control with autonomous robots as mixed initiative interaction. In AAAI Symposium on Mixed Initiative Interaction, volume 97, pages 89-94, 1997.
  22. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In NIPS, 2017.
  23. Youngwoon Lee, Edward S Hu, Zhengyu Yang, Alex Yin, and Joseph J Lim. IKEA furniture assembly environment for long-horizon complex manipulation tasks. ICRA, 2021. URL https://clvrai.com/furniture.
  24. Dylan P Losey, Krishnan Srinivasan, Ajay Mandlekar, Animesh Garg, and Dorsa Sadigh. Controlling assistive robots with learned latent actions. In ICRA, 2020.
  25. Yao Lu, Karol Hausman, Yevgen Chebotar, Mengyuan Yan, Eric Jang, Alexander Herzog, Ted Xiao, Alex Irpan, Mohi Khansari, Dmitry Kalashnikov, and Sergey Levine. Aw-opt: Learning robotic skills with imitation and reinforcement at scale. In CoRL, 2021.
  26. Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play. In CoRL, 2020.
  27. Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, and Li Fei-Fei. Roboturk: A crowdsourcing platform for robotic skill learning through imitation. In CoRL, 2018.
  28. Ajay Mandlekar, Fabio Ramos, Byron Boots, Li Fei- Fei, Animesh Garg, and Dieter Fox. Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data. ICRA, 2020.
  29. Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martín. What matters in learning from offline human demonstrations for robot manipulation. CoRL, 2021.
  30. Kunal Menda, Katherine Driggs-Campbell, and Mykel J Kochenderfer. Ensembledagger: A bayesian approach to safe imitation learning. In IROS, 2019.
  31. Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. In CoRL, 2022.
  32. Stefanos Nikolaidis, Yu Xiang Zhu, David Hsu, and Siddhartha Srinivasa. Human-robot mutual adaptation in shared autonomy. In 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 294-302. IEEE, 2017.
  33. Yoojin Oh, Tim Schäfer, Benedikt Rüther, Marc Toussaint, and Jim Mainprice. A system for traded control teleoper- ation of manipulation tasks using intent prediction from hand gestures. In 2021 30th IEEE International Con- ference on Robot & Human Interactive Communication (RO-MAN), pages 503-508. IEEE, 2021.
  34. Calder Phillips-Grafflin, Halit Bener Suay, Jim Mainprice, Nicholas Alunni, Daniel Lofaro, Dmitry Berenson, Sonia Chernova, Robert W Lindeman, and Paul Oh. From autonomy to cooperative traded control of humanoid manipulation tasks with unreliable communication: Ap- plications to the valve-turning task of the darpa robotics challenge and lessons learned. Journal of Intelligent & Robotic Systems, 82:341-361, 2016.
  35. Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In NIPS, pages 305-313, 1989.
  36. Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In AISTATS, pages 627-635, 2011.
  37. Mariah L Schrum, Michael Johnson, Muyleng Ghuy, and Matthew C Gombolay. Four years in review: Statistical practices of likert scales in human-robot interaction stud- ies. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pages 43-52, 2020.
  38. Mario Selvaggio, Marco Cognetti, Stefanos Nikolaidis, Serena Ivaldi, and Bruno Siciliano. Autonomy in physical human-robot interaction: A brief survey. IEEE Robotics and Automation Letters, 2021.
  39. Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In NIPS, 2015.
  40. Justin G Storms and Dawn M Tilbury. Blending of human and obstacle avoidance control for a high speed mobile robot. In 2014 American Control Conference, pages 3488- 3493. IEEE, 2014.
  41. Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IROS, 2012.
  42. Jiakai Zhang and Kyunghyun Cho. Query-efficient imitation learning for end-to-end autonomous driving. AAAI, 2017.
  43. Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, and Pieter Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In ICRA, 2018.

Cited by

Sorted by citations

Authors
Dass, S., Pertsch, K., Zhang, H., Lee, Y., Lim, J. J., & Nikolaidis, S.
Affiliations
  • University of California, Berkeley (US)
  • Korea Advanced Institute of Science and Technology (KR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2212.04708
arXiv
2212.04708
OpenAlex
W4385403839
Cited by
16
References
0
Citations by year
2026: 3 · 2025: 6 · 2024: 7
FWCI
14.14
Type
conference paper
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • Robotics: Science and Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Research Foundation
  • National Research Foundation of Korea (2019-0-00075)
  • Korea Advanced Institute of Science and Technology (2019-0-00075)
  • Ministry of Science and ICT, South Korea (2019-0-00075, No.2019-0-00075)
  • Institute for Information and Communications Technology Promotion (2019-0-00075)
Topics
  • Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
teleoperation, human-robot interaction, mental workload, operator workload, robot fleet

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review