The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Abstract
We address the problem of inferring the pose of an RGB-D camera relative to a known 3D scene, given only a single acquired image. Our approach employs a regression forest that is capable of inferring an estimate of each pixel's correspondence to 3D points in the scene's world coordinate frame. The forest uses only simple depth and RGB pixel comparison features, and does not require the computation of feature descriptors. The forest is trained to be capable of predicting correspondences at any pixel, so no interest point detectors are required. The camera pose is inferred using a robust optimization scheme. This starts with an initial set of hypothesized camera poses, constructed by applying the forest at a small fraction of image pixels. Preemptive RANSAC then iterates sampling more pixels at which to evaluate the forest, counting inliers, and refining the hypothesized poses. We evaluate on several varied scenes captured with an RGB-D camera and observe that the proposed technique achieves highly accurate relocalization and substantially out-performs two state of the art baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
ORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008
Self-citationBRIEF: Binary Robust Independent Elementary Features
Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_56 · Cited by 3,597
Scalable Recognition with a Vocabulary Tree
Nister, D., & Stewenius, H. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.264 · Cited by 3,579
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995316 · Cited by 3,491
Self-citationA solution for the best rotation to relate two sets of vectors
Kabsch, W. (1976) · Acta Crystallographica Section A · doi:10.1107/s0567739476001873 · Cited by 3,187
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110
Self-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 1×
“This data could be captured in several ways, for example by tracking from depth camera input [15, 21], or by using dense reconstruction and tracking from RGB input [22].”
Method
Cited by
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858
Enhanced Computer Vision With Microsoft Kinect Sensor: A Review
Han, J., Shao, L., Xu, D., & Shotton, J. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2265378 · Cited by 1,367
Self-citationGo-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration
Yang, J., Li, H., Campbell, D., & Jia, Y. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2513405 · Cited by 1,170
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions
Zeng, A., Song, S., Niessner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.29 · Cited by 1,128
Rad, M., & Lepetit, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.413 · Cited by 843
Geometric Loss Functions for Camera Pose Regression with Deep Learning
Kendall, A., & Cipolla, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.694 · Cited by 803
Learning 6D Object Pose Estimation Using 3D Object Coordinates
Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., & Rother, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_35 · Cited by 782
Self-citationNormalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
Wang, H., Sridhar, S., Huang, J., Valentin, J., Song, S., & Guibas, L. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00275 · Cited by 757
Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions
Sattler, T., Maddern, W., Toft, C., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00897 · Cited by 699
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 643
PPFNet: Global Context Aware Local Features for Robust 3D Point Matching
Deng, H., Birdal, T., & Ilic, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00028 · Cited by 631
DSAC — Differentiable RANSAC for Camera Localization
Brachmann, E., Krull, A., Nowozin, S., Shotton, J., Michel, F., Gumhold, S., & Rother, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.267 · Cited by 607
Self-citationDeepIM: Deep Iterative Matching for 6D Pose Estimation
Li, Y., Wang, G., Ji, X., Xiang, Y., & Fox, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_42 · Cited by 576
Modelling uncertainty in deep learning for camera relocalization
Kendall, A., & Cipolla, R. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487679 · Cited by 548
“A learning approach to relocalization has also been proposed previously using random forests to regress Scene Coordinate labels [16].”
MethodInfluentialUncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image
Brachmann, E., Michel, F., Krull, A., Yang, M. Y., Gumhold, S., & Rother, C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.366 · Cited by 530
The Perfect Match: 3D Point Cloud Matching With Smoothed Densities
Gojcic, Z., Zhou, C., Wegner, J. D., & Wieser, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00569 · Cited by 530
Image-Based Localization Using LSTMs for Structured Feature Correlation
Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.75 · Cited by 521
Fusion4D: real-time performance capture of challenging scenes
Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925969 · Cited by 500
Self-citationGDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation
Wang, G., Manhardt, F., Tombari, F., & Ji, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01634 · Cited by 462
InLoc: Indoor Visual Localization with Dense Matching and View Synthesis
Taira, H., Okutomi, M., Sattler, T., Cimpoi, M., Pollefeys, M., Sivic, J., Pajdla, T., & Torii, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00752 · Cited by 449
Learning Less is More - 6D Camera Localization via 3D Surface Regression
Brachmann, E., & Rother, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00489 · Cited by 399
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression
Sattler, T., Zhou, Q., Pollefeys, M., & Leal-Taixe, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00342 · Cited by 418
Geometry-Aware Learning of Maps for Camera Localization
Brahmbhatt, S., Gu, J., Kim, K., Hays, J., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00277 · Cited by 417
iNeRF: Inverting Neural Radiance Fields for Pose Estimation
Yen-Chen, L., Florence, P., Barron, J. T., Rodriguez, A., Isola, P., & Lin, T.-Y. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636708 · Cited by 404
PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors
Deng, H., Birdal, T., & Ilic, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_37 · Cited by 400
Li, J., Skinner, K. A., Eustice, R. M., & Johnson-Roberson, M. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2730363 · Cited by 397
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration
Ao, S., Hu, Q., Yang, B., Markham, A., & Guo, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01158 · Cited by 397
Review: Deep Learning on 3D Point Clouds
Bello, S. A., Yu, S., Wang, C., Adam, J. M., & Li, J. (2020) · Remote Sensing · doi:10.3390/rs12111729 · Cited by 387
State of the Art on 3D Reconstruction with RGB‐D Cameras
Zollhöfer, M., Stotko, P., Görlitz, A., Theobalt, C., Nießner, M., Klein, R., & Kolb, A. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13386 · Cited by 339
SceneNN: A Scene Meshes Dataset with aNNotations
Hua, B.-S., Pham, Q.-H., Nguyen, D. T., Tran, M.-K., Yu, L.-F., & Yeung, S.-K. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.18 · Cited by 333
Diving deeper into underwater image enhancement: A survey
Anwar, S., & Li, C. (2020) · Signal Processing: Image Communication · doi:10.1016/j.image.2020.115978 · Cited by 317
NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video
Sun, J., Xie, Y., Chen, L., Zhou, X., & Bao, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01534 · Cited by 313
Back to the Feature: Learning Robust Camera Localization from Pixels to Pose
Sarlin, P.-E., Unagar, A., Larsson, M., Germain, H., Toft, C., Larsson, V., Pollefeys, M., Lepetit, V., Hammarstrand, L., Kahl, F., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00326 · Cited by 297
Gong, Z., Zhong, P., & Hu, W. (2019) · IEEE Access · doi:10.1109/access.2019.2917620 · Cited by 277
Data-driven fluid simulations using regression forests
Ladický, L., Jeong, S., Solenthaler, B., Pollefeys, M., & Gross, M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818129 · Cited by 275
Schonberger, J. L., Pollefeys, M., Geiger, A., & Sattler, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00721 · Cited by 267
Visual place recognition: A survey from deep learning perspective
Zhang, X., Wang, L., & Su, Y. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2020.107760 · Cited by 261
Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses
Brachmann, E., & Rother, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00442 · Cited by 260
VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.284 · Cited by 249
VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
Radwan, N., Valada, A., & Burgard, W. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2869640 · Cited by 237
Exploiting uncertainty in regression forests for accurate camera relocalization
Valentin, J., Niebner, M., Shotton, J., Fitzgibbon, A., Izadi, S., & Torr, P. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299069 · Cited by 156
Self-citationDeep Learning for Visual Localization and Mapping: A Survey
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3309809 · Cited by 97
Is Geometry Enough for Matching in Visual Localization?
Zhou, Q., Agostinho, S., Osep, A., & Leal-Taixé, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12979 · Cited by 52
Leveraging Equivariant Features for Absolute Pose Regression
Musallam, M. A., Gaudilliere, V., Del Castillo, M. O., Al Ismaeil, K., & Aouada, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00675 · Cited by 27
X-Ray PoseNet: 6 DoF Pose Estimation for Mobile X-Ray Devices
Bui, M., Albarqouni, S., Schrapp, M., Navab, N., & Ilic, S. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.120 · Cited by 27
X-Ray to CT Rigid Registration Using Scene Coordinate Regression
Shrestha, P., Xie, C., Shishido, H., Yoshii, Y., & Kitahara, I. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-43999-5_74 · Cited by 12
Metadata
- Authors
- Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A.
- Venue
- 2013 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2013-06
- Pages
- 2930-2937
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1989476314
- Cited by
- 930
- References
- 38
- FWCI
- 34.81
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.