The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Open accessAbstract
Current model free learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, there are two problems with such a methodology: (a) since each object can be grasped in multiple ways, manually labeling grasp locations is not a trivial task; (b) human labeling is biased by semantics. While there have been attempts to train robots using trial-and-error experiments, the amount of data used in such experiments remains substantially low and hence makes the learner prone to over-fitting. In this paper, we take the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts. This allows us to train a Convolutional Neural Network (CNN) for the task of predicting grasp locations without severe overfitting. In our formulation, we recast the regression problem to an 18-way binary classification over image patches. We also present a multi-stage learning approach where a CNN trained in one stage is used to collect hard negatives in subsequent stages. Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping. We also compare to several baselines and show state-of-the-art performance on generalization to unseen objects for grasping.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
ROS: an open-source Robot Operating System
Quigley, M. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · Cited by 8,729
(1989) · neural information processing systems · Cited by 3,736
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Deep learning for detecting robotic grasps
Lenz, I., Lee, H., & Saxena, A. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914549607 · Cited by 1,634
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Robotic grasping and contact: a review
Bicchi, A., & Kumar, V. (2002) · Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065) · doi:10.1109/robot.2000.844081 · Cited by 1,069
Miller, A. T., & Allen, P. K. (2004) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2004.1371616 · Cited by 1,023
Data-Driven Grasp Synthesis—A Survey
Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2289018 · Cited by 929
Robotic Grasping of Novel Objects using Vision
Saxena, A., Driemeyer, J., & Ng, A. Y. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907087172 · Cited by 923
Constructing Force- Closure Grasps
Nguyen, V.-D. (1988) · The International Journal of Robotics Research · doi:10.1177/027836498800700301 · Cited by 902
Real-time grasp detection using convolutional neural networks
Redmon, J., & Angelova, A. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139361 · Cited by 893
Automatic grasp planning using shape primitives
Miller, A. T., Knoop, S., Christensen, H. I., & Allen, P. K. (2004) · 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422) · doi:10.1109/robot.2003.1241860 · Cited by 705
Robot Grasp Synthesis Algorithms: A Survey
Shimoga, K. B. (1996) · The International Journal of Robotics Research · doi:10.1177/027836499601500302 · Cited by 666
Efficient grasping from RGBD images: Learning using a new rectangle representation
Jiang, Y., Moseson, S., & Saxena, A. (2011) · 2011 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2011.5980145 · Cited by 657
1Data-Driven Grasp Synthesis- A SurveyPreprint
Bohg, J., Morales, A., Asfour, T., & Kragić, D. (2014) · Cited by 532
Automated construction of robotic manipulation programs
Takeo Kanade, A., Kuffner, J., & Diankov, R. (2010) · Cited by 458
Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152709 · Cited by 309
Leveraging big data for grasp planning
Kappler, D., Bohg, J., & Schaal, S. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139793 · Cited by 268
Pose error robust grasping from contact wrench space metrics
Weisz, J., & Allen, P. K. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6224697 · Cited by 171
Task-level planning of pick-and-place robot motions
Lozano-Perez, T., Jones, J. L., Mazer, E., & O'Donnell, P. A. (1989) · Computer · doi:10.1109/2.16222 · Cited by 161
Using depth and appearance features for informed robot grasping of highly wrinkled clothes
Arnau Ramisa, A., Alenyà, G., Moreno-Noguer, F., & Carme Torras, A. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225045 · Cited by 122
Learning to Manipulate Unknown Objects in Clutter by Reinforcement
Boularias, A., Bagnell, J. A., & Stentz, A. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9378 · Cited by 103
Learning object-specific grasp affordance densities
Detry, R. J., Başeski, E., Popović, M., Touati, Y., Krüger, N., Kroemer, O., Peters, J., & Piater, J. (2009) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2009.5175520 · Cited by 71
USING EXPERIENCE FOR ASSESSING GRASP RELIABILITY
Morales, A., Chinellato, E., Fagg, A. H., & del Pobil, A. P. (2004) · International Journal of Humanoid Robotics · doi:10.1142/s0219843604000290 · Cited by 70
A data-driven statistical framework for post-grasp manipulation
Paolini, R., Rodríguez, A., Srinivasa, S. S., & Mason, M. T. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364913507756 · Cited by 47
Active learning of visual descriptors for grasping using non-parametric smoothed beta distributions
Montesano, L., & Lopes, M. C. (2011) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2011.07.013 · Cited by 40
Perceiving, Learning, and Exploiting Object Affordances for Autonomous Pile Manipulation
Katz, D., Venkatraman, A., Kazemi, M., Bagnell, D., & Stentz, A. (2013) · Robotics: Science and Systems · doi:10.15607/rss.2013.ix.039 · Cited by 32
VisGraB: A Benchmark for Vision-Based Grasping
Kootstra, G., Popović, M., Jørgensen, J. A., Kragić, D., Petersen, H. G., & Krüger, N. (2012) · Paladyn Journal of Behavioral Robotics · doi:10.2478/s13230-012-0020-5 · Cited by 31
Cited by
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Self-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Metadata
- Authors
- Pinto, L., & Gupta, A.
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 3406-3413
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2201912979
- Cited by
- 1,047
- References
- 34
- FWCI
- 210.90
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.