Image Matching Using Generalized Scale-Space Interest Points
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian $$\nabla ^2_{norm} L$$ ∇ n o r m 2 L , the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$ det H n o r m L and four new unsigned or signed Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$ D 1 , n o r m L , $$\tilde{\mathcal {D}}_{1,norm} L$$ D ~ 1 , n o r m L , $${\mathcal {D}}_{2,norm} L$$ D 2 , n o r m L and $$\tilde{\mathcal {D}}_{2,norm} L$$ D ~ 2 , n o r m L , which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$ det H n o r m L is a truly affine covariant differential entity and the Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$ D 1 , n o r m L and $$\tilde{\mathcal {D}}_{1,norm} L$$ D ~ 1 , n o r m L have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differential entities will be more robust under affine image deformations than local extrema of the Laplacian operator or the Hessian feature strength measures $${\mathcal {D}}_{2,norm} L$$ D 2 , n o r m L , $$\tilde{\mathcal {D}}_{2,norm} L$$ D ~ 2 , n o r m L . It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$ D 1 , n o r m L , $$\tilde{\mathcal {D}}_{1,norm} L$$ D ~ 1 , n o r m L and the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$ det H n o r m L being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris–Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557 · cited 5×
“…) Matching relations between the interest points drawn as black lines on top of a superposition of the original grey-level images ator (Bay et al. [7]) have been demonstrated to be highly useful for this purpose with many successful applications, including multi-view image matching, object…”
BackgroundMethodInfluentialA Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901 · cited 3×
“The Harris corner detector [55] implies that corners are detected from positive spatial maxima of the entity where k is a constant required to be in the interval k ∈ ] 0 , 0 .”
BackgroundMethodBeyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Shi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724 · cited 1×
“Shi and Tomasi [142] proposed to instead use local maxima of the minimum eigenvalue ν 1 of μ as image features Figure 2 shows an example of computing Harris corners and Shi-and-Tomasi corners at a fixed scale.”
A performance evaluation of local descriptors
Mikolajczyk, K., & Schmid, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.188 · Cited by 6,723 · cited 2×
“Different generalizations of the SIFT operator in terms of the image descriptor have been presented by Ke and Sukthankar [66], Mikolajczyk and Schmid [125], Burghouts and Geusebroek [24], Toews and Wells [149], van de Sande et al. [138], Tola et al. [150] and Larsen et al. [81].”
MethodMarr, D., & Hildreth, E. (1980) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1980.0020 · Cited by 6,510
Scale & Affine Invariant Interest Point Detectors
Mikolajczyk, K., & Schmid, C. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000027790.02288.f2 · Cited by 3,436
PCA-SIFT: a more distinctive representation for local image descriptors
Ke, Y., & Sukthankar, R. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315206 · Cited by 3,237
A comparison of affine region detectors
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Gool, L. V. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-3848-x · Cited by 2,939
Feature detection with automatic scale selection
Lindeberg, T. (1998) · International Journal of Computer Vision · doi:10.1023/a:1008045108935 · Cited by 2,541 · cited 7×
“In Lindeberg [93,95,100,101] a general framework for automatic scale selection was proposed based on the idea of detecting local extrema over scale of γ -normalized derivatives according to (5).”
BackgroundMethodResultInfluentialSelf-citationKoenderink, J. J. (1984) · Biological Cybernetics · doi:10.1007/bf00336961 · Cited by 2,452 · cited 2×
“…for the development of a well-founded operational theory for detecting robust image features from real-world image data was provided by the framework of representing image data at multiple scales using scale-space representation, as originally proposed by Witkin [157] and Koenderink [69].”
BackgroundMethodContent-based multimedia information retrieval: State of the art and challenges
Lew, M. S., Sebe, N., Djeraba, C., & Jain, R. (2006) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/1126004.1126005 · Cited by 1,528
DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo
Tola, E., Lepetit, V., & Fua, P. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.77 · Cited by 1,405 · cited 2×
“Different generalizations of the SIFT operator in terms of the image descriptor have been presented by Ke and Sukthankar [66], Mikolajczyk and Schmid [125], Burghouts and Geusebroek [24], Toews and Wells [149], van de Sande et al. [138], Tola et al. [150] and Larsen et al. [81].”
BackgroundMethodRobust Wide Baseline Stereo from Maximally Stable Extremal Regions
Matas, J., Chum, O., Urban, M., & Pajdla, T. (2002) · Procedings of the British Machine Vision Conference 2002 · doi:10.5244/c.16.36 · Cited by 1,350
Real-Time Scale Selection in Hybrid Multi-scale Representations
Lindeberg, T., & Bretzner, L. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-44935-3_11 · Cited by 85
Self-citation
Cited by
Scale Invariant Feature Transform
Lindeberg, T. (2012) · Scholarpedia · doi:10.4249/scholarpedia.10491 · Cited by 621
Self-citationHoffman, D. P., Shtengel, G., Xu, C. S., Campbell, K. R., Freeman, M., Wang, L., Milkie, D. E., Pasolli, H. A., Iyer, N., Bogovic, J. A., et al. (2020) · Science · doi:10.1126/science.aaz5357 · Cited by 349
Metadata
- Authors
- Lindeberg, T.
- Venue
- Journal of Mathematical Imaging and Vision
- Published
- 2014-10-24
- Volume
- 52
- Issue
- 1
- Pages
- 3-36
- ISSN
- 0924-9907 (ISSN-L)
- 1573-7683
- OpenAlex
- W2023200581
- Cited by
- 208
- References
- 19
- FWCI
- 9.50
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.