The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Deep learning for detecting robotic grasps
Abstract
We consider the problem of detecting robotic grasps in an RGB-D view of a scene containing objects. In this work, we apply a deep learning approach to solve this problem, which avoids time-consuming hand-design of features. This presents two main challenges. First, we need to evaluate a huge number of candidate grasps. In order to make detection fast and robust, we present a two-step cascaded system with two deep networks, where the top detections from the first are re-evaluated by the second. The first network has fewer features, is faster to run, and can effectively prune out unlikely candidate grasps. The second, with more features, is slower but has to run only on the top few detections. Second, we need to handle multimodal inputs effectively, for which we present a method that applies structured regularization on the weights based on multimodal group regularization. We show that our method improves performance on an RGBD robotic grasping dataset, and can be used to successfully execute grasps on two different robotic platforms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Independent Component Analysis
Hyvärinen, A., Karhunen, J., & Oja, E. (2001) · Wiley eBooks · doi:10.1002/0471221317 · Cited by 5,007
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Fast Point Feature Histograms (FPFH) for 3D registration
Rusu, R. B., Blodow, N., & Beetz, M. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152473 · Cited by 3,878
(2011) · Cited by 2,752
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Self-citationAcoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Deep Neural Networks for Object Detection
Szegedy, C., Toshev, A., & Erhan, D. (2013) · neural information processing systems · Cited by 1,663
Learning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F. J., & Bottou, L. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315150 · Cited by 1,324
A large-scale hierarchical multi-view RGB-D object dataset
Lai, K., Bo, L., Ren, X., & Fox, D. (2011) · 2011 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2011.5980382 · Cited by 1,316
Robotic grasping and contact: a review
Bicchi, A., & Kumar, V. (2002) · Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065) · doi:10.1109/robot.2000.844081 · Cited by 1,069
(2009) · neural information processing systems · Cited by 962
Data-Driven Grasp Synthesis—A Survey
Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2289018 · Cited by 929
Robotic Grasping of Novel Objects using Vision
Saxena, A., Driemeyer, J., & Ng, A. Y. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907087172 · Cited by 923
Self-citationFerrari, C., & Canny, J. (2003) · Proceedings 1992 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.1992.219918 · Cited by 893
Fast 3D recognition and pose using the Viewpoint Feature Histogram
Rusu, R. B., Bradski, G., Thibaux, R., & Hsu, J. (2010) · 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2010.5651280 · Cited by 847
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 823
3-D Mapping With an RGB-D Camera
Endres, F., Hess, J., Sturm, J., Cremers, D., & Burgard, W. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2279412 · Cited by 820
Pedestrian Detection with Unsupervised Multi-stage Feature Learning
Sermanet, P., Kavukcuoglu, K., Chintala, S., & Lecun, Y. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.465 · Cited by 742
Automatic grasp planning using shape primitives
Miller, A. T., Knoop, S., Christensen, H. I., & Allen, P. K. (2004) · 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422) · doi:10.1109/robot.2003.1241860 · Cited by 705
Learning human activities and object affordances from RGB-D videos
Koppula, H. S., Gupta, R., & Saxena, A. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913478446 · Cited by 683
Self-citationRobot Grasp Synthesis Algorithms: A Survey
Shimoga, K. B. (1996) · The International Journal of Robotics Research · doi:10.1177/027836499601500302 · Cited by 666
Efficient grasping from RGBD images: Learning using a new rectangle representation
Jiang, Y., Moseson, S., & Saxena, A. (2011) · 2011 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2011.5980145 · Cited by 657
Self-citation1Data-Driven Grasp Synthesis- A Survey
Bohg, J., Morales, A., Asfour, T., & Kragić, D. (2014) · MPG.PuRe (Max Planck Society) · Cited by 532
Convolutional-Recursive Deep Learning for 3D Object Classification
Socher, R., Huval, B., Bath, B., Manning, C. D., & Ng, A. Y. (2012) · neural information processing systems · Cited by 509
An overview of 3D object grasp synthesis algorithms
Sahbani, A., El-Khoury, S., & Bidaud, P. (2012) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2011.07.016 · Cited by 478
Measuring Invariances in Deep Networks
Goodfellow, I., Lee, H., Le, Q. V., Saxe, A., & Ng, A. Y. (2009) · neural information processing systems · Cited by 461
Self-citationTopographic Independent Component Analysis
Hyvärinen, A., Hoyer, P. O., & Inki, M. (2001) · Neural Computation · doi:10.1162/089976601750264992 · Cited by 459
The MOPED framework: Object recognition and pose estimation for manipulation
Collet, A., Martinez, M., & Srinivasa, S. S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911401765 · Cited by 421
Maitin-Shepard, J., Cusumano-Towner, M., Lei, J., & Abbeel, P. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509439 · Cited by 418
Semantic Labeling of 3D Point Clouds for Indoor Scenes
Koppula, H. S., Anand, A., Joachims, T., & Saxena, A. (2011) · neural information processing systems · Cited by 385
Self-citationText Detection and Character Recognition in Scene Images with Unsupervised Feature Learning
Coates, A., Carpenter, B., Case, C., Satheesh, S., Suresh, B., Wang, T., Wu, D. J., & Ng, A. Y. (2011) · 2011 International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2011.95 · Cited by 378
A Dirty Model for Multi-task Learning
Jalali, A., Sanghavi, S., Ruan, C., & Ravikumar, P. (2010) · Cited by 368
Unsupervised Feature Learning for RGB-D Based Object Recognition
Bo, L., Ren, X., & Fox, D. (2013) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-00065-7_27 · Cited by 360
Robust real-time visual odometry for dense RGB-D mapping
Whelan, T., Johannsson, H., Kaess, M., Leonard, J. J., & McDonald, J. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631400 · Cited by 344
Goldfeder, C., Ciocarlie, M., Dang, H., & Allen, P. K. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152709 · Cited by 309
Object recognition and full pose registration from a single image for robotic manipulation
Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152739 · Cited by 275
Pose error robust grasping from contact wrench space metrics
Weisz, J., & Allen, P. K. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6224697 · Cited by 171
Rigid 3D geometry matching for grasping of known objects in cluttered scenes
Papazov, C., Haddadin, S., Parusel, S., Krieger, K., & Burschka, D. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911436019 · Cited by 160
Robotic Grasping of Novel Objects
Saxena, A., Driemeyer, J., Kearns, J., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0156 · Cited by 155
Self-citationSelecting Receptive Fields in Deep Networks
Coates, A. (2024) · neural information processing systems · doi:10.57702/4h3zav33 · Cited by 153
Learning grasp strategies with partial shape information
Saxena, A., Wong, L. L. S., & Ng, A. Y. (2008) · Cited by 149
Self-citationLearning to place new objects in a scene
Jiang, Y. P., Lim, M., Zheng, C., & Saxena, A. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364912438781 · Cited by 139
Self-citationLearning to grasp objects with multiple contact points
Le, Q. V., Kamm, D., Kara, A. F., & Ng, A. Y. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509508 · Cited by 129
Sohn, K., Jung, D. Y., Lee, H., & Hero, A. O. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126554 · Cited by 129
Self-citationGrasping novel objects with depth segmentation
Rao, D., Le, Q. V., Phoka, T., Quigley, M., Sudsang, A., & Ng, A. Y. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5650493 · Cited by 125
Learning to grasp using visual information
Kamon, I., Flash, T., & Edelman, S. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1996.506534 · Cited by 119
On Computing Two-Finger Force-Closure Grasps of Curved 2D Objects
Ponce, J., Stam, D., & Faverjon, B. (1993) · The International Journal of Robotics Research · doi:10.1177/027836499301200305 · Cited by 118
Learning and Evaluation of the Approach Vector for Automatic Grasp Generation and Planning
Ekvall, S., & Kragić, D. (2007) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2007.364205 · Cited by 109
Selection of robot pre-grasps using box-based shape approximation
Huebner, K., & Kragić, D. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4650722 · Cited by 109
Kragić, D., & Christensen, H. I. (2003) · The International Journal of Robotics Research · doi:10.1177/027836490302210009 · Cited by 76
Learning object-specific grasp affordance densities
Detry, R. J., Başeski, E., Popović, M., Touati, Y., Krüger, N., Kroemer, O., Peters, J., & Piater, J. (2009) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2009.5175520 · Cited by 71
Manipulation of unmodeled objects using intelligent grasping schemes
Bowers, D. L., & Lumia, R. (2003) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2003.812689 · Cited by 55
Bakebot: Baking Cookies with the PR2
Bollini, M. A., Barry, J. D., & Rus, D. L. (2011) · Cited by 48
Vision-based computation of three-finger grasps on unknown planar objects
Morales, A., Sanz, P. J., & del Pobil, A. P. (2003) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/irds.2002.1044002 · Cited by 45
Probabilistic Models of Object Geometry for Grasp Planning
Glover, J., Rus, D. L., & Roy, N. (2008) · Robotics: Science and Systems · doi:10.15607/rss.2008.iv.036 · Cited by 43
Learning hardware agnostic grasps for a universal jamming gripper
Jiang, Y. P., Amend, J. R., Lipson, H., & Saxena, A. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6225049 · Cited by 36
Self-citationSemantic parsing for priming object detection in indoors RGB-D scenes
Cadena, C., & Košecká, J. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364914549488 · Cited by 34
Learning Visual Features to Predict Hand Orientations
Piater, J. (2002) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 21
Global Optimization of Robotic Grasps
Gallegos, C. R., Porta, J. M., & Ros, L. (2011) · Robotics: Science and Systems · doi:10.15607/rss.2011.vii.012 · Cited by 18
Visual grasp affordances from appearance-based cues
Song, H. O., Fritz, M., Gu, C., & Darrell, T. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130360 · Cited by 17
Direct 3D servoing using dense depth maps
Teulière, C., & Marchand, É. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385630 · Cited by 16
Global Optimization of Robotic Grasps
Rosales, C., Porta, J. M., & Ros, L. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9481.003.0042 · Cited by 6
Cited by
VoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270
“There are many applications such as synthetic object insertion in computer graphics [29], synthetic depth of field in computational photography [3], grasping in robotics [34], using depth as a cue in human body pose estimation [48], robot assisted surgery [49], and automatic 2D to 3D conversion in film [53].”
MethodDeep Neural Networks Based Recognition of Plant Diseases by Leaf Image Classification
Sladojevic, S., Arsenovic, M., Anderla, A., Culibrk, D., & Stefanovic, D. (2016) · Computational Intelligence and Neuroscience · doi:10.1155/2016/3289801 · Cited by 2,029
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.058 · Cited by 1,268
Metadata
- Authors
- Lenz, I., Lee, H., & Saxena, A.
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2015-03-16
- Volume
- 34
- Issue
- 4-5
- Pages
- 705-724
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W1999156278
- Cited by
- 1,634
- References
- 67
- FWCI
- 109.50
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.