The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Open accessAbstract
Current model free learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, there are two problems with such a methodology: (a) since each object can be grasped in multiple ways, manually labeling grasp locations is not a trivial task; (b) human labeling is biased by semantics. While there have been attempts to train robots using trial-and-error experiments, the amount of data used in such experiments remains substantially low and hence makes the learner prone to over-fitting. In this paper, we take the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts. This allows us to train a Convolutional Neural Network (CNN) for the task of predicting grasp locations without severe overfitting. In our formulation, we recast the regression problem to an 18-way binary classification over image patches. We also present a multi-stage learning approach where a CNN trained in one stage is used to collect hard negatives in subsequent stages. Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping. We also compare to several baselines and show state-of-the-art performance on generalization to unseen objects for grasping.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
ROS: an open-source Robot Operating System
Quigley, M. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · Cited by 8,729
(1989) · neural information processing systems · Cited by 3,736
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Deep learning for detecting robotic grasps
Lenz, I., Lee, H., & Saxena, A. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914549607 · Cited by 1,634
Cited by
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Self-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.058 · Cited by 1,268
Data-Efficient Image Recognition with Contrastive Predictive Coding
Hénaff, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., & Oord, A. V. D. (2019) · arXiv · doi:10.48550/arxiv.1905.09272 · Cited by 1,014
(2022) · doi:10.15607/rss.2023.xix.025 · Cited by 916
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020) · arXiv · doi:10.48550/arxiv.2005.01643 · Cited by 824
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
Overcoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping
Fang, H.-S., Wang, C., Gou, M., & Lu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01146 · Cited by 683
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., et al. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460875 · Cited by 636
Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
Morrison, D., Leitner, J., & Corke, P. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.021 · Cited by 630
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599
Multi-task Self-Supervised Visual Learning
Doersch, C., & Zisserman, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.226 · Cited by 598
Grasp Pose Detection in Point Clouds
ten Pas, A., Gualtieri, M., Saenko, K., & Platt, R. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917735594 · Cited by 594
Generative Image Modeling Using Style and Structure Adversarial Networks
Wang, X., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_20 · Cited by 574
Self-citation6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
Mousavian, A., Eppner, C., & Fox, D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00299 · Cited by 571
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Learning ambidextrous robot grasping policies
Mahler, J., Matl, M., Satish, V., Danielczuk, M., DeRose, B., McKinley, S., & Goldberg, K. (2019) · Science Robotics · doi:10.1126/scirobotics.aau4984 · Cited by 563
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 552
Robotic grasp detection using deep convolutional neural networks
Kumra, S., & Kanan, C. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202237 · Cited by 536
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 506
Learning robust, real-time, reactive robotic grasping
Morrison, D., Corke, P., & Leitner, J. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364919859066 · Cited by 489
Du, G., Wang, K., Lian, S., & Zhao, K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09888-5 · Cited by 475
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., & Fox, D. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793789 · Cited by 457
Real-World Multiobject, Multigrasp Detection
Chu, F.-J., Xu, R., & Vela, P. A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2852777 · Cited by 453
Zeng, A., Song, S., Yu, K.-T., Donlon, E., Hogan, F. R., Bauza, M., Ma, D., Taylor, O., Liu, M., Romo, E., et al. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461044 · Cited by 441
Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network
Kumra, S., Joshi, S., & Sahin, F. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340777 · Cited by 437
James, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., & Bousmalis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01291 · Cited by 420
“limitation, since real-robot data collection is expensive and cumbersome, often requiring days or even months for a single task [34, 44].”
BackgroundMethodJacquard: A Large Scale Dataset for Robotic Grasp Detection
Depierre, A., Dellandrea, E., & Chen, L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593950 · Cited by 372
Mahler, J., Pokorny, F. T., Hou, B., Roderick, M., Laskey, M., Aubry, M., Kohlhoff, K., Kroger, T., Kuffner, J., & Goldberg, K. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487342 · Cited by 359
More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch
Calandra, R., Owens, A., Jayaraman, D., Lin, J., Yuan, W., Malik, J., Adelson, E. H., & Levine, S. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2852779 · Cited by 359
Owens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E. H., & Freeman, W. T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.264 · Cited by 346
Asymmetric Actor Critic for Image-Based Robot Learning
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., & Abbeel, P. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.008 · Cited by 343
Self-citationDeep learning in robotics: A review of recent research
Pierson, H. A., & Gashler, M. S. (2017) · Advanced Robotics · doi:10.1080/01691864.2017.1365009 · Cited by 331
Self-Supervised GANs via Auxiliary Rotation Loss
Chen, T., Zhai, X., Ritter, M., Lucic, M., & Houlsby, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01243 · Cited by 328
“It has been widely used in the video domain [39, 40], the robotics domain [41, 42] and the image domain [24, 35].”
MethodMahler, J., Matl, M., Liu, X., Li, A., Gealy, D., & Goldberg, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460887 · Cited by 323
Deep reinforcement learning for high precision assembly tasks
Inoue, T., De Magistris, G., Munawar, A., Yokoya, T., & Tachibana, R. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202244 · Cited by 309
High precision grasp pose detection in dense clutter
Gualtieri, M., ten Pas, A., Saenko, K., & Platt, R. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759114 · Cited by 303
A Survey on Learning-Based Robotic Grasping
Kleeberger, K., Bormann, R., Kraus, W., & Huber, M. F. (2020) · Current Robotics Reports · doi:10.1007/s43154-020-00021-6 · Cited by 290
Visual Reinforcement Learning with Imagined Goals
Nair, A., Pong, V. H., Dalal, M., Bahl, S., Lin, S., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04742 · Cited by 288
AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains
Fang, H.-S., Wang, C., Fang, H., Gou, M., Liu, J., Yan, H., Liu, W., Xie, Y., & Lu, C. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3281153 · Cited by 276
Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection
Levine, S., Pastor, P., Krizhevsky, A., & Quillen, D. (2017) · Springer Proceedings in Advanced Robotics · doi:10.1007/978-3-319-50115-4_16 · Cited by 274
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15818 · Cited by 276
“A promising approach for enabling generalization in robotic manipulation is by learning from large and diverse datasets [31, 32, 33].”
BackgroundReinforcement and Imitation Learning for Diverse Visuomotor Skills
Zhu, Y., Wang, Z., Merel, J., Rusu, A., Erez, T., Cabi, S., Tunyasuvunakool, S., Kramár, J., Hadsell, R., de Freitas, N., et al. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.009 · Cited by 260
Combining self-supervised learning and imitation for vision-based rope manipulation
Nair, A., Chen, D., Agrawal, P., Isola, P., Abbeel, P., Malik, J., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989247 · Cited by 255
Gandhi, D., Pinto, L., & Gupta, A. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206247 · Cited by 253
Self-citationPreparing for the unknown: Learning a universal policy with online system identification
Yu, W., Tan, J., Karen Liu, C., & Turk, G. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.048 · Cited by 237
Preparing for the Unknown: Learning a Universal Policy with Online System Identification
Yu, W., Tan, J., Liu, C. K., & Turk, G. (2017) · arXiv · doi:10.48550/arxiv.1702.02453 · Cited by 237
Interactive perception: Leveraging action in perception and perception in action
Bohg, J., Hausman, K., Sankaran, B., Brock, O., Kragic, D., Schaal, S., & Sukhatme, G. S. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2721939 · Cited by 227
“1) Commonalities and differences between CNS and APR: Approaches that exploit the novel sensory signal (CNS) also rely on regularities in the sensory response to an interaction.”
BackgroundInfluentialOpen X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration 0
O’Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611477 · Cited by 210
Self-citationTransferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
James, S., Davison, A. J., & Johns, E. (2017) · arXiv · doi:10.48550/arxiv.1707.02267 · Cited by 208
“An alternative approach for getting data from simulation is to instead use a large number of real world robots [11, 2, 16].”
BackgroundGrasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations
Song, S., Zeng, A., Lee, J., & Funkhouser, T. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3004787 · Cited by 185
“in simple tabletop settings [23, 37, 36], or detect grasps in 6DoF but with open-loop execution [17, 21, 7].”
BackgroundInfluentialBC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., & Finn, C. (2022) · arXiv · doi:10.48550/arxiv.2202.02005 · Cited by 167
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
Mandlekar, A., Zhu, Y., Garg, A., Booher, J., Spero, M., Tung, A., Gao, J., Emmons, J., Gupta, A., Orbay, E., et al. (2018) · arXiv · doi:10.48550/arxiv.1811.02790 · Cited by 134
“SSL has been used successfully to collect and learn from large quantities of data in both simulated [7, 8, 9] and physical settings [5, 10, 11] for tasks such as grasping.”
MethodSelf-citationHigh-Fidelity Image Generation With Fewer Labels
Lucic, M., Tschannen, M., Ritter, M., Zhai, X., Bachem, O., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1903.02271 · Cited by 134
“This approach was successfully applied to different data modalities, such as images (Doersch et al., 2015; Caron et al., 2018), video (Agrawal et al., 2015; Lee et al., 2017), and robotics (Jang et al., 2018; Pinto & Gupta, 2016).”
MethodWhere2Act: From Pixels to Actions for Articulated 3D Objects
Mo, K., Guibas, L., Mukadam, M., Gupta, A., & Tulsiani, S. (2021) · arXiv · doi:10.48550/arxiv.2101.02692 · Cited by 132
Self-citationNavigating to objects in the real world
Gervet, T., Chintala, S., Batra, D., Malik, J., & Chaplot, D. S. (2023) · Science Robotics · doi:10.1126/scirobotics.adf6991 · Cited by 125
“It is the option of choice for perception stacks across robotics and has been applied to end-to-end control in domains where sub-optimal policy operation is safe, like grasping and static manipulation [97, 98].”
BackgroundContactGrasp: Functional Multi-finger Grasp Synthesis from Contact
Brahmbhatt, S., Handa, A., Hays, J., & Fox, D. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967960 · Cited by 125
Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations
Jiang, Z., Zhu, Y., Svetlik, M., Fang, K., & Zhu, Y. (2021) · Robotics: Science and Systems XVII · doi:10.15607/rss.2021.xvii.024 · Cited by 122
Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations
Jiang, Z., Zhu, Y., Svetlik, M., Fang, K., & Zhu, Y. (2021) · arXiv · doi:10.48550/arxiv.2104.01542 · Cited by 122
“Supervision from grasp, in turn, produce better 3D reconstruction in graspable regions. end-to-end on large-scale grasping datasets, either through manual labeling [19] or self-exploration [20, 43].”
BackgroundLearning Predictive Representations for Deformable Objects Using Contrastive Estimation
Yan, W., Vangipuram, A., Abbeel, P., & Pinto, L. (2020) · arXiv · doi:10.48550/arxiv.2003.05436 · Cited by 122
Self-citationAffordances from Human Videos as a Versatile Representation for Robotics
Bahl, S., Mendonca, R., Chen, L., Jain, U., & Pathak, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01324 · Cited by 119
“Previous works have leveraged spatial structures of convolutional networks to directly output locations for grasping and pushing from just an image of the scene [92, 130, 131], which can limit the type of tasks possible.”
BackgroundTowards Generalization and Simplicity in Continuous Control
Rajeswaran, A., Lowrey, K., Todorov, E., & Kakade, S. (2017) · arXiv · doi:10.48550/arxiv.1703.02660 · Cited by 94
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.063 · Cited by 94
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2109.13396 · Cited by 94
Yuan, W., Hang, K., Kragic, D., Wang, M. Y., & Stork, J. A. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.06.007 · Cited by 54
Mandlekar, A., Booher, J., Spero, M., Tung, A., Gupta, A., Zhu, Y., Garg, A., Savarese, S., & Fei-Fei, L. (2019) · arXiv · doi:10.48550/arxiv.1911.04052 · Cited by 48
Self-citationOpenVLA: An Open-Source Vision-Language-Action Model
Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.09246 · Cited by 62
DayDreamer: World Models for Physical Robot Learning
Wu, P., Escontrela, A., Hafner, D., Goldberg, K., & Abbeel, P. (2022) · arXiv · doi:10.48550/arxiv.2206.14176 · Cited by 76
Language-driven Grasp Detection
Vuong, A. D., Vu, M. N., Huang, B., Nguyen, N., Le, H., Vo, T., & Nguyen, A. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01695 · Cited by 39
Learning Affordance Landscapes for Interaction Exploration in 3D Environments
Nagarajan, T., & Grauman, K. (2020) · arXiv · doi:10.48550/arxiv.2008.09241 · Cited by 38
“Self-supervised interaction learning Prior work studies actively learning manipulation policies through self-supervised training for grasping [44, 42, 33, 35, 61], pushing/poking [3, 41] and drone control [20].”
BackgroundLearning robots to grasp by demonstration
De Coninck, E., Verbelen, T., Van Molle, P., Simoens, P., & Dhoedt, B. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103474 · Cited by 34
Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
Yan, M., Frosio, I., Tyree, S., & Kautz, J. (2017) · arXiv · doi:10.48550/arxiv.1712.03303 · Cited by 31
Human Hands as Probes for Interactive Object Understanding
Goyal, M., Modi, S., Goyal, R., & Gupta, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00329 · Cited by 31
“This can be in the form of learning a) where and how to grasp [9, 20, 25, 26, 28, 33, 34, 37, 41, 49], b) state classifiers [24], c) interaction hotspots [14, 40, 42, 59], d) spatial priors for action sites [44], e) object articulation modes [11,36], f) reward functions [27,29,48,50], g) functional…”
BackgroundSee to touch: Learning tactile dexterity through visual incentives
Guzey, I., Dai, Y., Evans, B., Chintala, S., & Pinto, L. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611407 · Cited by 26
Self-citationAction Image Representation: Learning Scalable Deep Grasping Policies with Zero Real World Data
Khansari, M., Kappler, D., Luo, J., Bingham, J., & Kalakrishnan, M. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197415 · Cited by 23
SEAL: Self-supervised embodied active learning using exploration and 3D consistency
Chaplot, D. S., Dalal, M., Gupta, S., Malik, J., & Salakhutdinov, R. (2021) · arXiv · doi:10.48550/arxiv.2112.01001 · Cited by 22
“collisions, graspability) can be done through the use of appropriate sensors [17, 36].”
Background(2018) · Cited by 20
I Find Your Lack of Uncertainty in Computer Vision Disturbing
Valdenegro-Toro, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00139 · Cited by 15
$π_0$: A Vision-Language-Action Flow Model for General Robot Control
Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.24164 · Cited by 13
“Early work in this area often utilized self-supervised or autonomous data collection [26, 22, 8], providing a tractable data source for simple tasks such as grasping [18, 37] or pushing [56], but without the complexity of more dexterous behaviors.”
MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation
Lancaster, P., Hansen, N., Rajeswaran, A., & Kumar, V. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611121 · Cited by 8
“…learning on real hardware, most of which fall into one of three categories: learning from human demonstrations [36], [37], [38], [39], learning from large uncurated datasets [40], [41], [20], learning from online interaction via RL [42], [43], or any combination of them [17], [44], [45], [46].”
BackgroundMimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Mandlekar, A., Nasiriany, S., Wen, B., Akinola, I., Narang, Y., Fan, L., Zhu, Y., & Fox, D. (2023) · arXiv · doi:10.48550/arxiv.2310.17596 · Cited by 226
RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
Nasiriany, S., Maddukuri, A., Zhang, L., Parikh, A., Lo, A., Joshi, A., Mandlekar, A., & Zhu, Y. (2024) · arXiv · doi:10.48550/arxiv.2406.02523 · Cited by 3
“One approach is self-supervised learning, where trial-and-error is used to collect data for tasks like grasping and pushing [24, 36, 18, 19, 46, 7].”
BackgroundComparing Task Simplifications to Learn Closed-Loop Object Picking Using Deep Reinforcement Learning
Breyer, M., Furrer, F., Novkovic, T., Siegwart, R., & Nieto, J. (2018) · arXiv · doi:10.48550/arxiv.1803.04996 · Cited by 1
Metadata
- Authors
- Pinto, L., & Gupta, A.
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 3406-3413
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2201912979
- Cited by
- 1,047
- References
- 34
- FWCI
- 210.90
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.