The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching
Abstract
This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses a category-agnostic affordance prediction algorithm to select and execute among four different grasping primitive behaviors. It then recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional training data. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT-Princeton Team system that took 1st place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.edu.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Adapting visual category models to new domains
Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_16 · Cited by 2,847
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Aparicio, J., & Goldberg, K. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.058 · Cited by 1,268
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Pinto, L., & Gupta, A. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487517 · Cited by 1,047
Real-time grasp detection using convolutional neural networks
Redmon, J., & Angelova, A. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139361 · Cited by 893
Automatic grasp planning using shape primitives
Miller, A. T., Knoop, S., Christensen, H. I., & Allen, P. K. (2004) · 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422) · doi:10.1109/robot.2003.1241860 · Cited by 705
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606
Self-citationMulti-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge
Zeng, A., Yu, K.-T., Song, S., Suo, D., Walker, E., Rodriguez, A., & Xiao, J. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989165 · Cited by 457
Self-citationAnalysis and Observations From the First Amazon Picking Challenge
Correll, N., Bekris, K. E., Berenson, D., Brock, O., Causo, A., Hauser, K., Okada, K., Rodriguez, A., Romano, J. M., & Wurman, P. R. (2018) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2600527 · Cited by 425
Learning visual similarity for product design with convolutional neural networks
Bell, S., & Bala, K. (2015) · ACM Transactions on Graphics · doi:10.1145/2766959 · Cited by 408
Active vision in robotic systems: A survey of recent developments
Chen, S., Li, Y., & Kwok, N. M. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911410755 · Cited by 402
High precision grasp pose detection in dense clutter
Gualtieri, M., ten Pas, A., Saenko, K., & Platt, R. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759114 · Cited by 303
Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection
Levine, S., Pastor, P., Krizhevsky, A., & Quillen, D. (2017) · Springer Proceedings in Advanced Robotics · doi:10.1007/978-3-319-50115-4_16 · Cited by 274
Fast object localization and pose estimation in heavy clutter for robotic bin picking
Liu, M.-Y., Tuzel, O., Veeraraghavan, A., Taguchi, Y., Marks, T. K., & Chellappa, R. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911436018 · Cited by 189
Self-citationSegICP: Integrated deep semantic segmentation and pose estimation
Wong, J. M., Kee, V., Le, T., Wagner, S., Mariottini, G.-L., Schneider, A. R., Hamilton, L., Chipalkatty, R., Hebert, M. R., Johnson, D. M. S., et al. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206470 · Cited by 174
Using Geometry to Detect Grasp Poses in 3D Point Clouds
ten Pas, A., & Platt, R. (2017) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-51532-8_19 · Cited by 171
Data-driven visual similarity for cross-domain image matching
Shrivastava, A., Malisiewicz, T., Gupta, A. K., & Efros, A. A. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024188 · Cited by 155
Deep Learning for Detecting Robotic Grasps
Lenz, I., Lee, H., & Saxena, A. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.012 · Cited by 126
Team Delft’s Robot Winner of the Amazon Picking Challenge 2016
Hernández, C., Bharatheesha, M., Ko, W., Gaiser, H., Tan, J., van Deurzen, K., de Vries, M., Mil, B. V., van Egmond, J., Burger, R., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68792-6_51 · Cited by 122
Mobile bin picking with an anthropomorphic service robot
Nieuwenhuisen, M., Droeschel, D., Holz, D., Stückler, J., Berner, A., Li, J., Klein, R., & Behnke, S. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630892 · Cited by 111
NimbRo picking: Versatile part handling for warehouse automation
Schwarz, M., Milan, A., Lenz, C., Munoz, A., Periyasamy, A. S., Schreiber, M., Schuller, S., & Behnke, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989348 · Cited by 95
Probabilistic multi-class segmentation for the Amazon Picking Challenge
Jonschkowski, R., Eppner, C., Höfer, S., Martín-Martín, R., & Brock, O. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7758087 · Cited by 80
USING EXPERIENCE FOR ASSESSING GRASP RELIABILITY
Morales, A., Chinellato, E., Fagg, A. H., & del Pobil, A. P. (2004) · International Journal of Humanoid Robotics · doi:10.1142/s0219843604000290 · Cited by 77
Active and exploratory perception
Bajcsy, R. K., & Campos, M. F. M. (1992) · CVGIP Image Understanding · doi:10.1016/1049-9660(92)90083-f · Cited by 76
Look-Ahead Before You Leap: End-to-End Active Recognition by Forecasting the Effect of Motion
Jayaraman, D., & Grauman, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_30 · Cited by 75
Supervision via competition: Robot adversaries for learning tasks
Pinto, L., Davidson, J., & Gupta, A. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989190 · Cited by 68
Active recognition and pose estimation of household objects in clutter
Wu, K., Ranasinghe, R., & Dissanayake, G. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139782 · Cited by 52
End-to-End Learning of Object Grasp Poses in the Amazon Robotics Challenge
Matsumoto, E., Saito, M., Kume, A., & Tan, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-35679-8_6 · Cited by 22
A novel occlusion-free active recognition algorithm for objects in clutter
Jiang, D., Wang, H., Chen, W., & Wu, R. (2016) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2016.7866521 · Cited by 10
Cited by
Siamese Neural Networks: An Overview
Chicco, D. (2021) · Methods in Molecular Biology · doi:10.1007/978-1-0716-0826-5_3 · Cited by 663
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606
Self-citationLearning ambidextrous robot grasping policies
Mahler, J., Matl, M., Satish, V., Danielczuk, M., DeRose, B., McKinley, S., & Goldberg, K. (2019) · Science Robotics · doi:10.1126/scirobotics.aau4984 · Cited by 563
Learning robust, real-time, reactive robotic grasping
Morrison, D., Corke, P., & Leitner, J. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364919859066 · Cited by 489
Du, G., Wang, K., Lian, S., & Zhao, K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09888-5 · Cited by 475
Metadata
- Authors
- Zeng, A., Song, S., Yu, K.-T., Donlon, E., Hogan, F. R., Bauza, M., Ma, D., Taylor, O., Liu, M., Romo, E., Fazeli, N., Alet, F., Dafle, N. C., Holladay, R., Morena, I., Qu Nair, P., Green, D., Taylor, I., Liu, W., Funkhouser, T., & Rodriguez, A.
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 3750-3757
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2962746398
- Cited by
- 441
- References
- 33
- FWCI
- 101.87
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.