The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation
Open accessAbstract
In this work, we present a novel data-driven method for robust 6DoF object pose estimation from a single RGBD image. Unlike previous methods that directly regressing pose parameters, we tackle this challenging task with a keypoint-based approach. Specifically, we propose a deep Hough voting network to detect 3D keypoints of objects and then estimate the 6D pose parameters within a least-squares fitting manner. Our method is a natural extension of 2D-keypoint approaches that successfully work on RGB based 6DoF estimation. It allows us to fully utilize the geometric constraint of rigid objects with the extra depth information and is easy for a network to learn and optimize. Extensive experiments were conducted to demonstrate the effectiveness of 3D-keypoint detection in the 6D pose estimation task. Experimental results also show our method outperforms the state-of-the-art methods by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/PVN3D.git.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472 · cited 2×
“Classical methods [30, 42, 2] are able to detect 2D keypoint of objects with rich texture efficiently.”
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 2×
“[41, 47, 20] directly regress the 2D coordinate of the keypoints, while [33, 24, 34] use heatmaps to locate the 2D keypoints.”
BackgroundMethodComparing images using the Hausdorff distance
Huttenlocher, D. P., Klanderman, G. A., & Rucklidge, W. J. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.232073 · Cited by 4,545
CenterNet: Keypoint Triplets for Object Detection
Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., & Tian, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00667 · Cited by 4,079
Least-Squares Fitting of Two 3-D Point Sets
Arun, K. S., Huang, T. S., & Blostein, S. D. (1987) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1987.4767965 · Cited by 3,952 · cited 1×
“Given two point sets of an object, one from the M detected keypoints {kpj}j=1 in the camera coordinate system, and another from their corresponding points {kp′j}Mj=1 in the object coordinate system, the 6D pose estimation module computes the pose parameters (R, t) with a least-squares fitting algorithm [1], which find…”
MethodMulti-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139
Deep Hough Voting for 3D Object Detection in Point Clouds
Qi, C. R., Litany, O., He, K., & Guibas, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00937 · Cited by 1,244
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37331-2_42 · Cited by 1,161
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
Wang, C., Xu, D., Zhu, Y., Martín-Martín, R., Lu, C., Fei-Fei, L., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00346 · Cited by 1,154
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation
Peng, S., Liu, Y., Huang, Q., Zhou, X., & Bao, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00469 · Cited by 1,061
SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again
Kehl, W., Manhardt, F., Tombari, F., Ilic, S., & Navab, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.169 · Cited by 1,047 · cited 1×
“On the ADD(S) metric, our model outperforms PoseCNN+ICP [52] by 6.4% RGB RGBD PoseCNNDeepIM [26,52] PVNet[37] CDPN[27] ImplicitICP[45] SSD-6DICP[22] Point-Fusion[50] DF(per-pixel)[50] DF(iterative)[50 and exceeds DF(iterative) [50] by 5.7%.”
MethodInfluentialReal-Time Seamless Single Shot 6D Object Pose Prediction
Tekin, B., Sinha, S. N., & Fua, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00038 · Cited by 900
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856
Rad, M., & Lepetit, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.413 · Cited by 843
The YCB object and model set: Towards common benchmarks for manipulation research
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015) · 2015 International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2015.7251504 · Cited by 838 · cited 1×
“YCB-Video Dataset contains 21 YCB [4] objects of varying shape and texture.”
Learning 6D Object Pose Estimation Using 3D Object Coordinates
Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., & Rother, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_35 · Cited by 782
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
Wang, H., Sridhar, S., Huang, J., Valentin, J., Song, S., & Guibas, L. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00275 · Cited by 757
Pose estimation for augmented reality: a hands-on survey
Marchand, E., Uchiyama, H., & Spindler, F. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2513408 · Cited by 746 · cited 1×
“It is an important component in many real-world applications, such as robotic grasping and manipulation [6, 48, 55], autonomous driving [11, 5, 53], augmented reality [31] and so on.”
BackgroundRender for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Su, H., Qi, C. R., Li, Y., & Guibas, L. J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.308 · Cited by 680
Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes
Hinterstoisser, S., Holzer, S., Cagniart, C., Ilic, S., Konolige, K., Navab, N., & Lepetit, V. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126326 · Cited by 643
Gradient Response Maps for Real-Time Detection of Textureless Objects
Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab, N., Fua, P., & Lepetit, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.206 · Cited by 610
DeepIM: Deep Iterative Matching for 6D Pose Estimation
Li, Y., Wang, G., Ji, X., Xiang, Y., & Fox, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_42 · Cited by 576
Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
Sundermeyer, M., Marton, Z.-C., Durner, M., Brucker, M., & Triebel, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_43 · Cited by 564
Li, Z., Wang, G., & Ji, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00777 · Cited by 485
Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation
Park, K., Patten, T., & Vincze, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00776 · Cited by 455
The MOPED framework: Object recognition and pose estimation for manipulation
Collet, A., Martinez, M., & Srinivasa, S. S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911401765 · Cited by 421
Rothganger, F., Lazebnik, S., Schmid, C., & Ponce, J. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-005-3674-1 · Cited by 416
6-DoF object pose from semantic keypoints
Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989233 · Cited by 366
Tulsiani, S., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298758 · Cited by 346
Segmentation-Driven 6D Object Pose Estimation
Hu, Y., Hugonot, J., Fua, P., & Salzmann, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00350 · Cited by 333
In Defense of Classical Image Processing: Fast Depth Completion on the CPU
Ku, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 15th Conference on Computer and Robot Vision (CRV) · doi:10.1109/crv.2018.00013 · Cited by 325
Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation
Kehl, W., Milletari, F., Tombari, F., Ilic, S., & Navab, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_13 · Cited by 283
Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation
Oberweger, M., Rad, M., & Lepetit, V. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_8 · Cited by 253
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
Tremblay, J., To, T., Sundaralingam, B., Yu, X., Fox, D., & Birchfield, S. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10790 · Cited by 244
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
Doumanoglou, A., Kouskouridas, R., Malassiotis, S., & Kim, T.-K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.390 · Cited by 242
Single image 3D object detection and pose estimation for grasping
Zhu, M., Derpanis, K. G., Yang, Y., Brahmbhatt, S., Zhang, M., Phillips, C., Lecce, M., & Daniilidis, K. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907430 · Cited by 218
Viewpoint-independent object class detection using 3D Feature Maps
Liebelt, J., Schmid, C., & Schertler, K. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587614 · Cited by 200
Discovery of Latent 3D Keypoints via End-to-end Geometric ReasoningPreprint
Suwajanakorn, S., Snavely, N., Tompson, J., & Norouzi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03146 · Cited by 190
Discriminative Mixture-of-Templates for Viewpoint Classification
Gu, C., & Ren, X. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_30 · Cited by 187
Depth-Encoded Hough Voting for Joint Object Detection and Shape Recovery
Sun, M., Bradski, G. R., Xu, B., & Savarese, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_48 · Cited by 141
Global Hypothesis Generation for 6D Object Pose Estimation
Michel, F., Kirillov, A., Brachmann, E., Krull, A., Gumhold, S., Savchynskyy, B., & Rother, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.20 · Cited by 126
Viewpoint-aware object detection and pose estimation
Gläsner, D., Galun, M., Alpert, S., Basri, R., & Shakhnarovich, G. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126379 · Cited by 95
CenterNet: Object Detection with Keypoint TripletsPreprint
Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., & Tian, Q. C. (2019) · arXiv (Cornell University) · Cited by 61
CullNet: Calibrated and Pose Aware Confidence Scores for Object Pose Estimation
Gupta, K., Petersson, L., & Hartley, R. I. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00337 · Cited by 38
Cited by
Du, G., Wang, K., Lian, S., & Zhao, K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09888-5 · Cited by 475
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation
He, Y., Huang, H., Fan, H., Chen, Q., & Sun, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00302 · Cited by 376
Self-citationFoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
Wen, B., Yang, W., Kautz, J., & Birchfield, S. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01692 · Cited by 353
Metadata
- Authors
- He, Y., Sun, W., Huang, H., Liu, J., Fan, H., & Sun, J.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 11629-11638
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034986117
- Cited by
- 613
- References
- 55
- FWCI
- 127.67
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.