CoChat
Sign Up
PreprintCited by 66Reinforcement Learning in RoboticsRobot Manipulation and LearningProsthetics and Rehabilitation Robotics

What Matters in Learning from Offline Human Demonstrations for Robot Manipulation

Open access
Mandlekar, Ajay, Xu, Danfei, Wong, Josiah, Nasiriany, Soroush, Wang, Chen, Kulkarni, Rohun, Fei-Fei, Li, Savarese, Silvio +2 more·2021·arXiv
DOI 10.48550/arxiv.2108.03298arXiv 2108.03298
Open this paper in CoChat — free accountOpen PDF

Abstract

Imitating human demonstrations is a promising approach to endow robots with various manipulation capabilities. While recent advances have been made in imitation learning and batch (offline) reinforcement learning, a lack of open-source human datasets and reproducible learning methods make assessing the state of the field difficult. In this paper, we conduct an extensive study of six offline learning algorithms for robot manipulation on five simulated and three real-world multi-stage manipulation tasks of varying complexity, and with datasets of varying quality. Our study analyzes the most critical challenges when learning from offline human data for manipulation. Based on the study, we derive a series of lessons including the sensitivity to different algorithmic design choices, the dependence on the quality of the demonstrations, and the variability based on the stopping criteria due to the different objectives in training and evaluation. We also highlight opportunities for learning from human datasets, such as the ability to learn proficient policies on challenging, multi-stage tasks beyond the scope of current reinforcement learning methods, and the ability to easily scale to natural, real-world manipulation scenarios where only raw sensory signals are available. We have open-sourced our datasets and all algorithm implementations to facilitate future research and fair comparisons in learning from human demonstration data. Codebase, datasets, trained models, and more available at https://arise-initiative.github.io/robomimic-web/

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2108.03298

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A relatively-large dataset of 200 demonstrations was required to achieve a modest success rate of 73.3% on the robomimic coke can task.

    “Consider a case study from robomimic [7], in which the agent is tasked with moving a coke can from one bin into another. This is a simple task involving a single scene, single object, and single robot; however, a relatively-large dataset of 200 demonstrations was required to achieve a modest success rate of 73.3%.”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [7]

    Section 3.1 identifies the Can task, and Appendix B.1 describes a 200-demonstration Can-Paired dataset. But Appendix B.1 says BC-RNN can achieve near-perfect performance on just 100 good-quality Can demonstrations, contrary to the claim that 200 were required for a 73.3% success rate. Section 4.2 associates 73.3% with a Square dataset, not the Can task; page numbers are unavailable in our copy. (LLM, conf=0.87)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • LSTM-GMM and IBC are biased toward one mode in the Push-T task.

    “In contrast, both LSTM-GMM [29] and IBC [12] are biased toward one mode”

    From Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (2023) · cites this paper as [29]

    Section D.2 describes a BC-RNN policy with an LSTM hidden state and GMM action output, supporting the identification of the cited method as LSTM-GMM. Section J says rollouts sample one of the GMM modes, but it does not say the policy is biased toward a single mode in Push-T. The IBC citation is to a different source and is not assessed here. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Offline data augmentation can be effective for generating larger dataset for robot manipulation.

    “Offline data augmentation can be effective for generating larger dataset for robot manipulation [7, 35-45]”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [7]

    Section 4.3 reports that pixel-shift image randomization improves robot-manipulation policy performance, and Appendix E.3 describes applying random crops when images are fed to the network. But the paper does not show offline augmentation generating a larger dataset; Section 4.6 studies dataset size using subsets of collected demonstrations instead. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Several works have used offline data augmentation to increase the dataset size for learning policies.

    “Several works have used offline data augmentation to increase the dataset size for learning policies [7, 35-45].”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [7]

    Section E.3 describes random crops applied to image observations when they are fed to the network, and Section 4.3 reports that this randomization improves learned policy performance. The paper therefore supports using augmentation in offline policy learning, but it does not say the augmentation was performed offline to increase the dataset size; Section 4.6 studies dataset size by sampling subsets of collected demonstrations. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • LSTM-GMM and IBC are biased toward one mode in the Push T task.

    “In contrast, both LSTM-GMM Mandlekar et al. (2021) and IBC Florence et al. (2021) are biased toward one mode, while BET Shafiullah et al. (2022) fails to commit to a single mode due to its lack of temporal action consistency.”

    From Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (2023) · cites this paper as Mandlekar et al. (2021)

    Section D.2 describes a BC-RNN policy with an LSTM and GMM action output, supporting the identification of the cited method. Section J discusses sampling GMM modes during rollouts, but neither section reports a Push T experiment or a bias toward one mode; searches for Push T and mode bias found no such result. (LLM, conf=0.81)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 78

  1. [1][1] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009. -
  2. [5][5] P. Rajpurkar, R. Jia, and P. Liang. Know What You Don't Know: Unanswerable Questions for SQuAD. arXiv preprint arXiv:1806.03822 , 2018. - · arXiv
  3. [6][6] L. Floridi and M. Chiriatti. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines , 30(4):681-694, 2020. -
  4. [8][8] D. A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In Advances in neural information processing systems , pages 305-313, 1989. -
  5. [9][9] T. Zhang, Z. McCarthy, O. Jow, D. Lee, K. Goldberg, and P. Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. arXiv preprint arXiv:1710.04615 , 2017. - · arXiv
  6. [10][10] A. Mandlekar, D. Xu, R. Martín-Martín, S. Savarese, and L. Fei-Fei. Learning to generalize across long-horizon tasks from human demonstrations. arXiv preprint arXiv:2003.06085 , 2020. - · arXiv
  7. [11][11] S. Lange, T. Gabel, and M. Riedmiller. Batch reinforcement learning. In Reinforcement learning , pages 45-73. Springer, 2012. -
  8. [12][12] S. Levine, A. Kumar, G. Tucker, and J. Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 , 2020. - · arXiv
  9. [13][13] S. Cabi, S. Gómez Colmenarejo, A. Novikov, K. Konyushkova, S. Reed, R. Jeong, K. Zolna, Y. Aytar, D. Budden, M. Vecerik, et al. Scaling data-driven robotics with reward sketching and batch reinforcement learning. arXiv , pages arXiv-1909, 2019. -
  10. [14][14] P. Florence, L. Manuelli, and R. Tedrake. Self-supervised correspondence in visuomotor policy learning. IEEE Robotics and Automation Letters , 5(2):492-499, 2019. -
  11. [15][15] A. Mandlekar, Y. Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, S. Savarese, and L. Fei-Fei. RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation. In Conference on Robot Learning , 2018. -
  12. [16][16] P. Sharma, L. Mohan, L. Pinto, and A. Gupta. Multiple interactions made easy (mime): Large scale demonstrations data for imitation. In Conference on robot learning , pages 906-915. PMLR, 2018. -
  13. [18][18] J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219 , 2020. - · arXiv
  14. [19][19] C. Gulcehre, Z. Wang, A. Novikov, T. L. Paine, S. G. Colmenarejo, K. Zolna, R. Agarwal, J. Merel, D. Mankowitz, C. Paduraru, et al. Rl unplugged: Benchmarks for offline reinforcement learning. arXiv preprint arXiv:2006.13888 , 2020. - · arXiv
  15. [20][20] A. Mandlekar, F. Ramos, B. Boots, S. Savarese, L. Fei-Fei, A. Garg, and D. Fox. Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data. In IEEE International Conference on Robotics and Automation (ICRA) , pages 4414-4420. IEEE, 2020. -
  16. [21][21] S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages 627-635, 2011. -
  17. [23][23] S. Toyer, R. Shah, A. Critch, and S. Russell. The magical benchmark for robust imitation. arXiv preprint arXiv:2011.00401 , 2020. - · arXiv
  18. [24][24] S. James, Z. Ma, D. R. Arrojo, and A. J. Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters , 5(2):3019-3026, 2020. -
  19. [25][25] A. Zeng, P. Florence, J. Tompson, S. Welker, J. Chien, M. Attarian, T. Armstrong, I. Krasin, D. Duong, V. Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. arXiv preprint arXiv:2010.14406 , 2020. - · arXiv
  20. [26][26] S. Fujimoto, D. Meger, and D. Precup. Off-policy deep reinforcement learning without exploration. In International Conference on Machine Learning , pages 2052-2062. PMLR, 2019. -
  21. [27][27] A. Kumar, A. Zhou, G. Tucker, and S. Levine. Conservative q-learning for offline reinforcement learning. arXiv preprint arXiv:2006.04779 , 2020. - · arXiv
  22. [28][28] T. L. Paine, C. Paduraru, A. Michi, C. Gulcehre, K. Zolna, A. Novikov, Z. Wang, and N. de Freitas. Hyperparameter selection for offline reinforcement learning. arXiv preprint arXiv:2007.09055 , 2020. - · arXiv
  23. [29][29] J. Fu, M. Norouzi, O. Nachum, G. Tucker, Z. Wang, A. Novikov, M. Yang, M. R. Zhang, Y. Chen, A. Kumar, et al. Benchmarks for deep off-policy evaluation. arXiv preprint arXiv:2103.16596 , 2021. - · arXiv
  24. [30][30] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning , pages 1861-1870. PMLR, 2018. -
  25. [31][31] A. J. Ijspeert, J. Nakanishi, and S. Schaal. Movement imitation with nonlinear dynamical systems in humanoid robots. Proceedings 2002 IEEE International Conference on Robotics and Automation , 2:1398-1403 vol.2, 2002. -
  26. [32][32] C. Finn, T. Yu, T. Zhang, P. Abbeel, and S. Levine. One-shot visual imitation learning via meta-learning. In Conference in Robot Learning , volume abs/1709.04905, 2017. - · arXiv
  27. [33][33] A. Billard, S. Calinon, R. Dillmann, and S. Schaal. Robot programming by demonstration. In Springer Handbook of Robotics , 2008. -
  28. [34][34] S. Calinon, F. D'halluin, E. L. Sauser, D. G. Caldwell, and A. Billard. Learning and reproduction of gestures by imitation. IEEE Robotics and Automation Magazine , 17:44-54, 2010. -
  29. [35][35] C. Wang, R. Wang, D. Xu, A. Mandlekar, L. Fei-Fei, and S. Savarese. Generalization through hand-eye coordination: An action space for learning spatially-invariant visuomotor control. arXiv preprint arXiv:2103.00375 , 2021. - · arXiv
  30. [36][36] A. Tung, J. Wong, A. Mandlekar, R. Martín-Martín, Y. Zhu, L. Fei-Fei, and S. Savarese. Learning multi-arm manipulation through collaborative teleoperation. arXiv preprint arXiv:2012.06738 , 2020. - · arXiv
  31. [37][37] Y. Wu, G. Tucker, and O. Nachum. Behavior regularized offline reinforcement learning. arXiv preprint arXiv:1911.11361 , 2019. - · arXiv
  32. [38][38] Z. Wang, A. Novikov, K. ˙ Zołna, J. T. Springenberg, S. Reed, B. Shahriari, N. Siegel, J. Merel, C. Gulcehre, N. Heess, et al. Critic regularized regression. arXiv preprint arXiv:2006.15134 , 2020. - · arXiv
  33. [39][39] N. Y. Siegel, J. T. Springenberg, F. Berkenkamp, A. Abdolmaleki, M. Neunert, T. Lampe, R. Hafner, N. Heess, and M. Riedmiller. Keep doing what worked: Behavioral modelling priors for offline reinforcement learning. arXiv preprint arXiv:2002.08396 , 2020. - · arXiv
  34. [40][40] R. Kidambi, A. Rajeswaran, P. Netrapalli, and T. Joachims. Morel: Model-based offline reinforcement learning. arXiv preprint arXiv:2005.05951 , 2020. - · arXiv
  35. [41][41] T. Yu, G. Thomas, L. Yu, S. Ermon, J. Zou, S. Levine, C. Finn, and T. Ma. Mopo: Model-based offline policy optimization. arXiv preprint arXiv:2005.13239 , 2020. - · arXiv
  36. [42][42] S. K. S. Ghasemipour, D. Schuurmans, and S. S. Gu. Emaq: Expected-max q-learning operator for simple yet effective offline and online rl. arXiv preprint arXiv:2007.11091 , 2020. - · arXiv
  37. [43][43] T. Yu, A. Kumar, R. Rafailov, A. Rajeswaran, S. Levine, and C. Finn. Combo: Conservative offline model-based policy optimization. arXiv preprint arXiv:2102.08363 , 2021. - · arXiv
  38. [45][45] C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel. Deep spatial autoencoders for visuomotor learning. In 2016 IEEE International Conference on Robotics and Automation (ICRA) , pages 512-519. IEEE, 2016. -
  39. [47][47] R. Agarwal, D. Schuurmans, and M. Norouzi. An optimistic perspective on offline reinforcement learning. In International Conference on Machine Learning , pages 104-114. PMLR, 2020. -
  40. [48][48] A. Ajay, A. Kumar, P. Agrawal, S. Levine, and O. Nachum. Opal: Offline primitive discovery for accelerating offline reinforcement learning. arXiv preprint arXiv:2010.13611 , 2020. - · arXiv
  41. [49][49] K. Pertsch, Y. Lee, and J. J. Lim. Accelerating reinforcement learning with learned skill priors. arXiv preprint arXiv:2010.11944 , 2020. - · arXiv
  42. [50][50] L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch. Decision transformer: Reinforcement learning via sequence modeling. arXiv preprint arXiv:2106.01345 , 2021. - · arXiv
  43. [51][51] M. Janner, Q. Li, and S. Levine. Reinforcement learning as one big sequence modeling problem. arXiv preprint arXiv:2106.02039 , 2021. - · arXiv
  44. [52][52] M. Yang and O. Nachum. Representation matters: Offline pretraining for sequential decision making. arXiv preprint arXiv:2102.05815 , 2021. - · arXiv
  45. [53][53] O. Nachum and M. Yang. Provable representation learning for imitation with contrastive fourier features. arXiv preprint arXiv:2105.12272 , 2021. - · arXiv
  46. [54][54] M. Hersch, F. Guenter, S. Calinon, and A. Billard. Dynamical system modulation for robot learning via kinesthetic demonstrations. IEEE Transactions on Robotics , 24(6):1463-1467, 2008. -
  47. [55][55] P. Kormushev, S. Calinon, and D. G. Caldwell. Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Advanced Robotics , 25(5):581-603, 2011. -
  48. [56][56] B. Akgun, M. Cakmak, J. W. Yoo, and A. L. Thomaz. Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction , pages 391-398, 2012. -
  49. [57][57] A. Bajcsy, D. P. Losey, M. K. O'Malley, and A. D. Dragan. Learning robot objectives from physical human interaction. Proceedings of Machine Learning Research , 78:217-226, 2017. -
  50. [58][58] A. Bajcsy, D. P. Losey, M. K. O'Malley, and A. D. Dragan. Learning from physical human corrections, one feature at a time. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction , pages 141-149, 2018. -
  51. [60][60] R. Zhang, F. Torabi, L. Guan, D. H. Ballard, and P. Stone. Leveraging human guidance for deep reinforcement learning tasks. arXiv preprint arXiv:1909.09906 , 2019. - · arXiv
  52. [61][61] R. Loftin, B. Peng, J. MacGlashan, M. L. Littman, M. E. Taylor, J. Huang, and D. L. Roberts. Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning. Autonomous agents and multi-agent systems , 30(1):30-59, 2016. -
  53. [62][62] J. MacGlashan, M. K. Ho, R. Loftin, B. Peng, D. Roberts, M. E. Taylor, and M. L. Littman. Interactive learning from policy-dependent human feedback. arXiv preprint arXiv:1701.06049 , 2017. - · arXiv
  54. [63][63] P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems , pages 4299-4307, 2017. -
  55. [64][64] A. Singh, L. Yang, K. Hartikainen, C. Finn, and S. Levine. End-to-end robotic reinforcement learning without reward engineering. arXiv preprint arXiv:1904.07854 , 2019. - · arXiv
  56. [65][65] Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel. Benchmarking deep reinforcement learning for continuous control. In International conference on machine learning , pages 13291338. PMLR, 2016. -
  57. [66][66] T. Wang, X. Bao, I. Clavera, J. Hoang, Y. Wen, E. Langlois, S. Zhang, G. Zhang, P. Abbeel, and J. Ba. Benchmarking model-based reinforcement learning. arXiv preprint arXiv:1907.02057 , 2019. - · arXiv
  58. [67][67] M. Andrychowicz, A. Raichuk, P. Sta´ nczyk, M. Orsini, S. Girgin, R. Marinier, L. Hussenot, M. Geist, O. Pietquin, M. Michalski, et al. What matters in on-policy reinforcement learning? a large-scale empirical study. arXiv preprint arXiv:2006.05990 , 2020. - · arXiv
  59. [68][68] P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger. Deep reinforcement learning that matters. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018. -
  60. [69][69] R. Memmesheimer, I. Mykhalchyshyna, V. Seib, and D. Paulus. Simitate: A hybrid imitation learning benchmark. arXiv preprint arXiv:1905.06002 , 2019. - · arXiv
  61. [70][70] L. Hussenot, M. Andrychowicz, D. Vincent, R. Dadashi, A. Raichuk, L. Stafiniak, S. Girgin, R. Marinier, N. Momchev, S. Ramos, et al. Hyperparameter selection for imitation learning. arXiv preprint arXiv:2105.12034 , 2021. - · arXiv
  62. [71][71] M. A. Rana, D. Chen, J. Williams, V. Chu, S. R. Ahmadzadeh, and S. Chernova. Benchmark for skill learning from demonstration: Impact of user experience, task complexity, and start configuration on performance. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 7561-7567. IEEE, 2020. -
  63. [72][72] A. Lemme, Y. Meirovitch, M. Khansari-Zadeh, T. Flash, A. Billard, and J. J. Steil. Opensource benchmarking for learned reaching motion generation in robotics. Paladyn, Journal of Behavioral Robotics , 6(1), 2015. -
  64. [73][73] L. Fan, Y. Zhu, J. Zhu, Z. Liu, O. Zeng, A. Gupta, J. Creus-Costa, S. Savarese, and L. Fei-Fei. Surreal: Open-source reinforcement learning framework and robot manipulation benchmark. In Conference on Robot Learning , pages 767-782. PMLR, 2018. -
  65. [74][74] X. Lin, Y. Wang, J. Olkin, and D. Held. Softgym: Benchmarking deep reinforcement learning for deformable object manipulation. arXiv preprint arXiv:2011.07215 , 2020. - · arXiv
  66. [75][75] T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on Robot Learning , pages 1094-1100. PMLR, 2020. -
  67. [76][76] Y. Zhu, J. Wong, A. Mandlekar, and R. Martín-Martín. robosuite: A modular simulation framework and benchmark for robot learning. In arXiv preprint arXiv:2009.12293 , 2020. - · arXiv
  68. [77][77] O. Ahmed, F. Träuble, A. Goyal, A. Neitz, Y. Bengio, B. Schölkopf, M. Wüthrich, and S. Bauer. Causalworld: A robotic manipulation benchmark for causal structure and transfer learning. arXiv preprint arXiv:2010.04296 , 2020. - · arXiv
  69. [78][78] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014. - · arXiv
  70. [79][79] D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013. - · arXiv
  71. [80][80] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 , 2015. - · arXiv
  72. [82][82] E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026-5033, 2012. -
  73. [83][83] O. Khatib. A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal on Robotics and Automation , 3(1):43-53, 1987. -
  74. [84][84] M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas. Reinforcement learning with augmented data. arXiv preprint arXiv:2004.14990 , 2020. - · arXiv
  75. [85][85] I. Kostrikov, D. Yarats, and R. Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. arXiv preprint arXiv:2004.13649 , 2020. - · arXiv
  76. [86][86] S. Young, D. Gandhi, S. Tulsiani, A. Gupta, P. Abbeel, and L. Pinto. Visual imitation made easy. arXiv e-prints , pages arXiv-2008, 2020. -
  77. [87][87] A. Zhan, P. Zhao, L. Pinto, P. Abbeel, and M. Laskin. A framework for efficient robotic manipulation. arXiv preprint arXiv:2012.07975 , 2020. - · arXiv
  78. [88][88] M. Hoffman, B. Shahriari, J. Aslanides, G. Barth-Maron, F. Behbahani, T. Norman, A. Abdolmaleki, A. Cassirer, F. Yang, K. Baumli, S. Henderson, A. Novikov, S. G. Colmenarejo, S. Cabi, C. Gulcehre, T. L. Paine, A. Cowie, Z. Wang, B. Piot, and N. de Freitas. Acme: A research framework for distributed reinforcement learning. arXiv preprint arXiv:2006.00979 , 2020. URL https://arxiv.org/abs/2006.00979 . · arXiv

Cited by

Sorted by citations

Authors
Mandlekar, A., Xu, D., Wong, J., Nasiriany, S., Wang, C., Kulkarni, R., Fei-Fei, L., Savarese, S., Zhu, Y., & Martín-Martín, R.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2108.03298
arXiv
2108.03298
OpenAlex
W3187722890
Cited by
66
References
8
Citations by year
2026: 1 · 2025: 12 · 2024: 18 · 2023: 29 · 2022: 4 · 2021: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Engineering › Biomedical Engineering › Prosthetics and Rehabilitation Robotics
Keywords
robotic manipulation, offline reinforcement learning, imitation learning, stopping rules, open-source dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review