Pixel-Perfect Structure-from-Motion with Featuremetric Refinement
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Finding local features that are repeatable across multiple views is a cornerstone of sparse 3D reconstruction. The classical image matching paradigm detects keypoints per-image once and for all, which can yield poorly-localized features and propagate large errors to the final geometry. In this paper, we refine two key steps of structure-from-motion by a direct alignment of low-level image information from multiple views: we first adjust the initial keypoint locations prior to any geometric estimation, and subsequently refine points and camera poses as a post-processing. This refinement is robust to large detection noise and appearance changes, as it optimizes a featuremetric error based on dense features predicted by a neural network. This significantly improves the accuracy of camera poses and scene geometry for a wide range of keypoint detectors, challenging viewing conditions, and off-the-shelf deep features. Our system easily scales to large image collections, enabling pixel-perfect crowd-sourced localization at scale. Our code is publicly available at github.com/cvg/pixel-perfect-sfm as an add-on to the popular SfM software COLMAP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Machine Learning for High-Speed Corner Detection
Rosten, E., & Drummond, T. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_34 · Cited by 4,555
Bundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988
Robust Statistics: The Approach Based on Influence Functions
Ruppert, D. (1987) · Technometrics · doi:10.2307/1269782 · Cited by 3,880
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · cited 1×
“Direct photometric optimization has been successfully applied to optical flow [8,49], visual odometry [18,25,26,42], SLAM [5, 69], multi-view stereo (MVS) [19, 21, 87], and pose refinement [70].”
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
Robust regression using iteratively reweighted least-squares.
Holland, P. W., & Welsch, R. E. (1977) · Communications in Statistics - Theory and Methods · doi:10.1080/03610927708827533 · Cited by 2,115 · cited 1×
“The robust mean in (7) is computed with iteratively reweighted least squares [87] initialized with the non-robust mean.”
DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo
Tola, E., Lepetit, V., & Fua, P. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.77 · Cited by 1,405 · cited 1×
“Differently,densematching[36],[37],[38],[39],[40],[41], [42]considersallpixelsineachimage,resultingindenserand moreaccuratecorrespondences.”
A Multi-view Stereo Benchmark with High-Resolution Images and Multi-camera Videos
Schops, T., Schonberger, J. L., Galliani, S., Sattler, T., Schindler, K., Pollefeys, M., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.272 · Cited by 915
Self-citationBenchmarking 6DOF Outdoor Visual Localization in Changing Conditions
Sattler, T., Maddern, W., Toft, C., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00897 · Cited by 699 · cited 2×
“We believe that our featuremetric refinement can significantly improve the accuracy of existing datasets [34] and push the community towards sub-pixel accurate localization at large scale.”
Self-citationImage matching across wide baselines: From paper to practice
Jin, Y., Mishkin, D., Mishchuk, A., Matas, J., Fua, P., Yi, K. M., & Trulls, E. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01385-0 · Cited by 388
Cited by
LightGlue: Local Feature Matching at Light Speed
Lindenberger, P., Sarlin, P.-E., & Pollefeys, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01616 · Cited by 943
Self-citationMega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly- Throughs
Turki, H., Ramanan, D., & Satyanarayanan, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01258 · Cited by 423
Grounding Image Matching in 3D with MASt3R
Leroy, V., Cabon, Y., & Revaud, J. (2024) · arXiv · doi:10.48550/arxiv.2406.09756 · Cited by 309
“We also report results for more traditional SfM methods like PixSFM [50] and COLMAP [76] extended with Su-perPoint [21] and SuperGlue [72] (COLMAP+SPSG).”
MethodGrounding Image Matching in 3D with MASt3R
Leroy, V., Cabon, Y., & Revaud, J. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73220-1_5 · Cited by 305
Metadata
- Authors
- Lindenberger, P., Sarlin, P.-E., Larsson, V., & Pollefeys, M.
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 5967-5977
- OpenAlex
- W3194532238
- Cited by
- 172
- References
- 16
- FWCI
- 9.01
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.