Object recognition and full pose registration from a single image for robotic manipulation
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Robust perception is a vital capability for robotic manipulation in unstructured scenes. In this context, full pose estimation of relevant objects in a scene is a critical step towards the introduction of robots into household environments. In this paper, we present an approach for building metric 3D models of objects using local descriptors from several images. Each model is optimized to fit a set of calibrated training images, thus obtaining the best possible alignment between the 3D model and the real object. Given a new test image, we match the local descriptors to our stored models online, using a novel combination of the RANSAC and Mean Shift algorithms to register multiple instances of each object. A robust initialization step allows for arbitrary rotation, translation and scaling of objects in the test images. The resulting system provides markerless 6-DOF pose estimation for complex objects in cluttered scenes. We provide experimental results demonstrating orientation and translation accuracy, as well a physical implementation of the pose output being used by an autonomous robot to perform grasping in highly cluttered scenes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Compared to our method, traditional approaches require less extensive training and take advantage of richer sensory data, allowing them to detect the full 3D pose of objects (position and orientation) without any assumptions about the location or size of the surface on which the objects are placed.”
From Domain randomization for transferring deep neural networks from simulation to the real world (2017) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893 · cited 9×
“Finally, the combination of RANSAC and Mean Shift clustering allows for efficient search of multiple instances of the same object with minimal extra overhead.”
MethodInfluentialMultiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
A performance evaluation of local descriptors
Mikolajczyk, K., & Schmid, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.188 · Cited by 6,723 · cited 1×
“distinctive and robust local descriptors across a wide range of transformations [17].”
BackgroundGeneralizing the Hough transform to detect arbitrary shapes
Ballard, D. H. (1981) · Pattern Recognition · doi:10.1016/0031-3203(81)90009-1 · Cited by 4,371
Mean shift, mode seeking, and clustering
Cheng, Y. (1995) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.400568 · Cited by 3,856
EPnP: An Accurate O(n) Solution to the PnP Problem
Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0152-6 · Cited by 3,298
Multiple View Geometry in Computer Vision (2nd ed)
Hartley, R. I., & Zisserman, A. (2003) · Cambridge University Press eBooks · Cited by 2,761
Marker tracking and HMD calibration for a video-based augmented reality conferencing system
Kato, H., & Billinghurst, M. (2003) · Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99) · doi:10.1109/iwar.1999.803809 · Cited by 2,194
Cited by
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124
“There has also been some success with only monocular camera images [5].”
BackgroundMethodDeep learning for detecting robotic grasps
Lenz, I., Lee, H., & Saxena, A. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914549607 · Cited by 1,634
Data-Driven Grasp Synthesis—A Survey
Bohg, J., Morales, A., Asfour, T., & Kragic, D. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2289018 · Cited by 929
Multimodal deep learning for robust RGB-D object recognition
Eitel, A., Springenberg, J. T., Spinello, L., Riedmiller, M., & Burgard, W. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353446 · Cited by 631
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599
“Learning and optimization-based methods have been applied to various parts of the state-estimation problem, such as object recognition [11], pose registration [12], and dynamics learning [13].”
MethodThe MOPED framework: Object recognition and pose estimation for manipulation
Collet, A., Martinez, M., & Srinivasa, S. S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911401765 · Cited by 421
Self-citationTask Space Regions: A framework for pose-constrained manipulation planning
Berenson, D., Srinivasa, S., & Kuffner, J. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364910396389 · Cited by 368
Self-citation6-DoF object pose from semantic keypoints
Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989233 · Cited by 366
Odessa: enabling interactive perception applications on mobile devices
Ra, M.-R., Sheth, A., Mummert, L., Pillai, P., Wetherall, D., & Govindan, R. (2011) · Proceedings of the 9th international conference on Mobile systems, applications, and services · doi:10.1145/1999995.2000000 · Cited by 361
Gu, C., Lim, J. J., Arbelaez, P., & Malik, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206727 · Cited by 347
Unsupervised feature learning for 3D scene labeling
Lai, K., Bo, L., & Fox, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907298 · Cited by 317
“There is a large body of literature on 3D scene labeling in indoor environments ([1], [2], [3], [4], [5]).”
BackgroundHERB: a home exploring robotic butler
Srinivasa, S. S., Ferguson, D., Helfrich, C. J., Berenson, D., Collet, A., Diankov, R., Gallagher, G., Hollinger, G., Kuffner, J., & Weghe, M. V. (2010) · Autonomous Robots · doi:10.1007/s10514-009-9160-9 · Cited by 306
Self-citationRF-compass: robot object manipulation using RFIDs
Wang, J., Adib, F., Knepper, R., Katabi, D., & Rus, D. (2013) · Proceedings of the 19th annual international conference on Mobile computing & networking - MobiCom '13 · doi:10.1145/2500423.2500451 · Cited by 186
“When successful at detecting an object, they can locate it with an error of a few centimeters to tens of centimeters, depending on the amount of clutter and the system capability [28, 15].”
BackgroundA dataset for improved RGBD-based object detection and pose estimation for warehouse pick-and-place
Rennie, C., Shome, R., Bekris, K. E., & De Souza, A. F. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2016.2532924 · Cited by 182
“of different algorithms for this problem, given the rich literature on the subject [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29].”
BackgroundPush-grasping with dexterous hands: Mechanics and a method
Dogar, M. R., & Srinivasa, S. S. (2010) · 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2010.5652970 · Cited by 180
“[15].”
Self-citationLearning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data
Sundaresan, P., Grannen, J., Thananjeyan, B., Balakrishna, A., Laskey, M., Stone, K., Gonzalez, J. E., & Goldberg, K. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197121 · Cited by 107
“the rope to generate descriptors and correspondences [4, 11, 16, 34].”
BackgroundReal-Time Data Processing Architecture for Multi-Robots Based on Differential Federated Learning
Zhou, W., Li, Y., Chen, S., & Ding, B. (2018) · 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/smartworld.2018.00106 · Cited by 33
“We take the robotic recognition [10] as a study case to talk about the business values on such big data.”
Background6-DoF Object Pose from Semantic Keypoints
Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K. (2017) · arXiv · doi:10.48550/arxiv.1703.04670 · Cited by 26
Metadata
- Authors
- Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D.
- Venue
- 2009 IEEE International Conference on Robotics and Automation
- Publisher
- IEEE
- Published
- 2009-05
- Pages
- 48-55
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2146447720
- Cited by
- 275
- References
- 28
- FWCI
- 21.26
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.