The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
6-DoF object pose from semantic keypoints
Abstract
This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior work, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Empirically, we show that the proposed approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios with a cluttered background. For class-based object pose estimation, state-of-the-art accuracy is shown on the large-scale PASCAL3D+ dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 4×
“, [22], [23], [24], [25]) has attracted considerable study, while limited attention has been given to their application with generic object categories [26], [20].”
BackgroundMethodInfluentialKinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 2×
“More specifically, a complete 3D model of the gas canister was reconstructed using KinectFusion [40] and the groundtruth object pose for each image was calculated by ICP with careful manual initialization.”
MethodEPnP: An Accurate O(n) Solution to the PnP Problem
Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0152-6 · Cited by 3,298
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.214 · Cited by 3,185
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
Instance-aware Semantic Segmentation via Multi-task Network Cascades
Dai, J., He, K., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.04412 · Cited by 1,273
Instance-Aware Semantic Segmentation via Multi-task Network Cascades
Dai, J., He, K., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.343 · Cited by 1,244
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Zhuowen Tu, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 993
Deeplysupervised netsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015) · Cited by 849
Human Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805 · cited 1×
“Moreover, the stacking of the individual hourglass modules provides a form of iterative processing that has been show to be effective with several other recent network designs [35], [25] and offers additional refinement of the network estimates.”
BackgroundLearning 6D Object Pose Estimation Using 3D Object Coordinates
Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., & Rother, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_35 · Cited by 782
Manopt, a Matlab toolbox for optimization on manifolds
Boumal, N., Mishra, B., Absil, P.-A., & Sepulchre, R. J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.5200 · Cited by 737
Beyond PASCAL: A benchmark for 3D object detection in the wild
Xiang, Y., Mottaghi, R., & Savarese, S. (2014) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2014.6836101 · Cited by 727
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Su, H., Qi, C. R., Li, Y., & Guibas, L. J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.308 · Cited by 680
Gradient Response Maps for Real-Time Detection of Textureless Objects
Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab, N., Fua, P., & Lepetit, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.206 · Cited by 610
Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image
Brachmann, E., Michel, F., Krull, A., Yang, M. Y., Gumhold, S., & Rother, C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.366 · Cited by 530
The MOPED framework: Object recognition and pose estimation for manipulation
Collet, A., Martinez, M., & Srinivasa, S. S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911401765 · Cited by 421
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
Self-citation3D shape regression for real-time facial animation
Cao, C., Weng, Y., Lin, S., & Zhou, K. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462012 · Cited by 348
Tulsiani, S., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298758 · Cited by 346
Reconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
Do Convnets Learn Correspondence?Preprint
Long, J., Zhang, N., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.1091 · Cited by 275
Object recognition and full pose registration from a single image for robotic manipulation
Collet, A., Berenson, D., Srinivasa, S. S., & Ferguson, D. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152739 · Cited by 275
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
Doumanoglou, A., Kouskouridas, R., Malassiotis, S., & Kim, T.-K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.390 · Cited by 242
Single image 3D object detection and pose estimation for grasping
Zhu, M., Derpanis, K. G., Yang, Y., Brahmbhatt, S., Zhang, M., Phillips, C., Lecce, M., & Daniilidis, K. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907430 · Cited by 218
Self-citationTeaching 3D geometry to deformable part models
Pepik, B., Stark, M., Gehler, P., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248075 · Cited by 213
Discriminative Mixture-of-Templates for Viewpoint Classification
Gu, C., & Ren, X. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_30 · Cited by 187
Discriminatively Trained Templates for 3D Object Detection: A Real Time Scalable Approach
Rios-Cabrera, R., & Tuytelaars, T. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.256 · Cited by 169
Detailed 3D Representations for Object Recognition and Modeling
Zia, M. Z., Stark, M., Schiele, B., & Schindler, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.87 · Cited by 165
3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model
Fidler, S., Dickinson, S., & Urtasun, R. (2012) · doi:10.13140/2.1.2839.7440 · Cited by 149
3D shape estimation from 2D landmarks: A convex relaxation approach
Zhou, X., Leonardos, S., Hu, X., & Daniilidis, K. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299074 · Cited by 139
Self-citationGlobal Hypothesis Generation for 6D Object Pose Estimation
Michel, F., Kirillov, A., Brachmann, E., Krull, A., Gumhold, S., Savchynskyy, B., & Rother, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.20 · Cited by 139
Real-time scalable 6DOF pose estimation for textureless objects
Cao, Z., Sheikh, Y. A., & Banerjee, N. K. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487396 · Cited by 79
Reconstructing vehicles from a single image: Shape priors for road scene understanding
Jatavallabhula, K. M., Krishna, G. V. S., Chhaya, F., & Krishna, K. M. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989089 · Cited by 76
REIN - A fast, robust, scalable REcognition INfrastructure
Muja, M., Rusu, R. B., Bradski, G. R., & Lowe, D. G. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5980153 · Cited by 70
Multimodal blending for high-accuracy instance recognition
Xie, Z., Singh, A., Uang, J., Narayan, K. S., & Abbeel, P. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696666 · Cited by 54
Convolutional Neural Networks for joint object detection and pose estimation: A comparative studyPreprint
Massa, F., Aubry, M., & Marlet, R. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.7190 · Cited by 20
Single Image Pop-Up from Discriminatively Learned Parts
Zhu, M., Zhou, X., & Daniilidis, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.112 · Cited by 16
Self-citationReconstructing Vechicles from a Single Image: Shape Priors for Road Scene UnderstandingPreprint
Jatavallabhula, K. M., Krishna, G. V. S., Chhaya, F., & Krishna, K. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.09468 · Cited by 8
Cited by
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
Wang, C., Xu, D., Zhu, Y., Martín-Martín, R., Lu, C., Fei-Fei, L., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00346 · Cited by 1,154
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129
“Table 2 shows that MultiBin modules are more effective than discretized classification [21] and also keypoint based method of [14] which is based on localizing keypoints and solving a sophisticated optimization to recover the pose.”
MethodBottom-Up Object Detection by Grouping Extreme and Center Points
Zhou, X., Zhuo, J., & Krahenbuhl, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00094 · Cited by 1,088
Self-citationPVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation
Peng, S., Liu, Y., Huang, Q., Zhou, X., & Bao, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00469 · Cited by 1,061
Self-citation
Metadata
- Authors
- Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K.
- Venue
- 2017 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2017-05
- Pages
- 2011-2018
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2600447016
- Cited by
- 366
- References
- 47
- FWCI
- 11.89
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.