The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Matching Local Self-Similarities across Images and Videos
Abstract
We present an approach for measuring similarity between visual entities (images or videos) based on matching internal self-similarities. What is correlated across images (or across video sequences) is the internal layout of local self-similarities (up to some distortions), even though the patterns generating those local self-similarities are quite different in each of the images/videos. These internal self-similarities are efficiently captured by a compact local "self-similarity descriptor"', measured densely throughout the image/video, at multiple scales, while accounting for local and global geometric distortions. This gives rise to matching capabilities of complex visual data, including detection of objects in real cluttered images using only rough hand-sketches, handling textured objects with no clear boundaries, and detecting complex actions in cluttered video data with no prior learning. We compare our measure to commonly used image-based and video-based similarity measures, and demonstrate its applicability to object detection, retrieval, and action detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Textural Features for Image Classification
Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1973.4309314 · Cited by 22,886
A performance evaluation of local descriptors
Mikolajczyk, K., & Schmid, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.188 · Cited by 6,723
Shape matching and object recognition using shape contexts
Belongie, S., Malik, J., & Puzicha, J. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993558 · Cited by 6,306
Non-negative matrix factorization with sparseness constraints
Hoyer, P. O. (2004) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0408058 · Cited by 2,591
Behavior Recognition via Sparse Spatio-Temporal Features
Dollar, P., Rabaud, V., Cottrell, G., & Belongie, S. (2006) · 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance · doi:10.1109/vspets.2005.1570899 · Cited by 2,460
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667
Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words
Niebles, J. C., Wang, H., & Fei-Fei, L. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0122-4 · Cited by 1,475
Laptev, & Lindeberg (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238378 · Cited by 1,300
Recognizing action at a distance
Efros, Berg, Mori, & Malik (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238420 · Cited by 1,241
Cited by
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283
Pedestrian Detection: An Evaluation of the State of the Art
Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.155 · Cited by 3,190
SuperGlue: Learning Feature Matching With Graph Neural Networks
Sarlin, P.-E., DeTone, D., Malisiewicz, T., & Rabinovich, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00499 · Cited by 2,830
A survey on vision-based human action recognition
Poppe, R. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.11.014 · Cited by 2,242
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849
Attribute-Based Classification for Zero-Shot Visual Object Categorization
Lampert, C. H., Nickisch, H., & Harmeling, S. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.140 · Cited by 1,627
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206594 · Cited by 1,528
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
In defense of Nearest-Neighbor based image classification
Boiman, O., Shechtman, E., & Irani, M. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587598 · Cited by 1,072
Self-citationSeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights
Milford, M. J., & Wyeth, G. F. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6224623 · Cited by 981
SEMANTIC3D.NET: A NEW LARGE-SCALE POINT CLOUD CLASSIFICATION BENCHMARK
Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K., & Pollefeys, M. (2017) · ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprs-annals-iv-1-w1-91-2017 · Cited by 812
Detecting texts of arbitrary orientations in natural images
Yao, C., Bai, X., Liu, W., Ma, Y., & Tu, Z. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247787 · Cited by 765
Multiple kernels for object detection
Vedaldi, A., Gulshan, V., Varma, M., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459183 · Cited by 757
MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration
Heinrich, M. P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F. V., Brady, S. M., & Schnabel, J. A. (2012) · Medical Image Analysis · doi:10.1016/j.media.2012.05.008 · Cited by 707
Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation
Wei, Q., Bioucas-Dias, J., Dobigeon, N., & Tourneret, J.-Y. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2381272 · Cited by 698
RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform
Li, J., Hu, Q., & Ai, M. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2959244 · Cited by 651
Static and space-time visual saliency detection by self-resemblance
Seo, H. J., & Milanfar, P. (2009) · Journal of Vision · doi:10.1167/9.12.15 · Cited by 633
Progressive search space reduction for human pose estimation
Ferrari, V., Marin-Jimenez, M., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587468 · Cited by 612
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
Tumanyan, N., Geyer, M., Bagon, S., & Dekel, T. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00191 · Cited by 609
Violent flows: Real-time detection of violent crowd behavior
Hassner, T., Itcher, Y., & Kliper-Gross, O. (2012) · 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239348 · Cited by 587
PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval
Zhou, W., Newsam, S., Li, C., & Shao, Z. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.01.004 · Cited by 571
“proposed a local self-similarity (SSIM) descriptor [59] to measure the similarity between images or videos based on”
BackgroundA review of multimodal image matching: Methods and applications
Jiang, X., Ma, J., Xiao, G., Shao, Z., & Guo, X. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.012 · Cited by 532
Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity
Ye, Y., Shan, J., Bruzzone, L., & Shen, L. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2656380 · Cited by 487
Visual attribute transfer through deep image analogy
Liao, J., Yao, Y., Yuan, L., Hua, G., & Kang, S. B. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073683 · Cited by 487
New features and insights for pedestrian detection
Walk, S., Majer, N., Schindler, K., & Schiele, B. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540102 · Cited by 460
Pulling Things out of Perspective
Ladicky, L., Shi, J., & Pollefeys, M. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.19 · Cited by 457
Streetscore -- Predicting the Perceived Safety of One Million Streetscapes
Naik, N., Philipoom, J., Raskar, R., & Hidalgo, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.121 · Cited by 453
Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
Xia, L., & Aggarwal, J. K. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.365 · Cited by 433
Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection
Lim, J. J., Zitnick, C. L., & Dollar, P. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.406 · Cited by 427
What makes an image memorable?
Isola, P., Xiao, J., Torralba, A., & Oliva, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995721 · Cited by 418
Fast and Robust Matching for Multimodal Remote Sensing Image Registration
Ye, Y., Bruzzone, L., Shan, J., Bovolo, F., & Zhu, Q. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2924684 · Cited by 414
Efficient Object Category Recognition Using Classemes
Torresani, L., Szummer, M., & Fitzgibbon, A. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15549-9_56 · Cited by 409
On the burstiness of visual elements
Jegou, H., Douze, M., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206609 · Cited by 405
View-Independent Action Recognition from Temporal Self-Similarities
Junejo, I. N., Dexter, E., Laptev, I., & Pérez, P. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.68 · Cited by 392
Event Detection in Crowded Videos
Ke, Y., Sukthankar, R., & Hebert, M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409011 · Cited by 374
Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network
Lanaras, C., Bioucas-Dias, J., Galliani, S., Baltsavias, E., & Schindler, K. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.09.018 · Cited by 371
Nonparametric Scene Parsing via Label Transfer
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.131 · Cited by 368
Visual Classification With Multitask Joint Sparse Representation
Yuan, X.-T., Liu, X., & Yan, S. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2205006 · Cited by 364
“Double-DIP”: Unsupervised Image Decomposition via Coupled Deep-Image-Priors
Gandelsman, Y., Shocher, A., & Irani, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01128 · Cited by 355
Self-citationA performance evaluation of gradient field HOG descriptor for sketch based image retrieval
Hu, R., & Collomosse, J. (2013) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2013.02.005 · Cited by 353
AiATrack: Attention in Attention for Transformer Visual Tracking
Gao, S., Zhou, C., Ma, C., Wang, X., & Yuan, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20047-2_9 · Cited by 351
Local Trinary Patterns for human action recognition
Yeffet, L., & Wolf, L. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459201 · Cited by 339
Wolf, L., Hassner, T., & Taigman, Y. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.230 · Cited by 333
SUN Database: Exploring a Large Collection of Scene Categories
Xiao, J., Ehinger, K. A., Hays, J., Torralba, A., & Oliva, A. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-014-0748-y · Cited by 331
Learning Complexity-Aware Cascades for Deep Pedestrian Detection
Cai, Z., Saberian, M., & Vasconcelos, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.384 · Cited by 326
What Makes a Photograph Memorable?
Isola, P., Xiao, J., Parikh, D., Torralba, A., & Oliva, A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.200 · Cited by 322
Recognizing scene viewpoint using panoramic place representation
Xiao, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247991 · Cited by 318
Visual classification with multi-task joint sparse representation
Yuan, X.-T., & Yan, S. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539967 · Cited by 303
Cross-View Image Geolocalization
Lin, T.-Y., Belongie, S., & Hays, J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.120 · Cited by 291
Style Transfer by Relaxed Optimal Transport and Self-Similarity
Kolkin, N., Salavon, J., & Shakhnarovich, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01029 · Cited by 281
Deeply learned attributes for crowded scene understanding
Shao, J., Kang, K., Loy, C. C., & Wang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299097 · Cited by 280
Multiple Kernel Learning for Visual Object Recognition: A Review
(2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.212 · Cited by 274
Non-local Shape Descriptor: A New Similarity Metric for Deformable Multi-modal Registration
Heinrich, M. P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F. V., Brady, J. M., & Schnabel, J. A. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23629-7_66 · Cited by 268
Relational Embedding for Few-Shot Classification
Kang, D., Kwon, H., Min, J., & Cho, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00870 · Cited by 266
LBP-Based Edge-Texture Features for Object Recognition
Satpathy, A., Jiang, X., & Eng, H.-L. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2310123 · Cited by 257
Towards realtime multimodal fusion for image-guided interventions using self-similarities
Heinrich, M. P., Jenkinson, M., Papież, B. W., Brady, S. M., & Schnabel, J. A. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40811-3_24 · Cited by 176
Tsintotas, K. A., Bampis, L., & Gasteratos, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3175656 · Cited by 175
“During navigation in a changing environment, the topological information about the robot’s relative movement becomes more important as noise from the sensory inputs is accumulated to an overwhelming degree [296], [297].”
BackgroundSplicing ViT Features for Semantic Appearance Transfer
Tumanyan, N., Bar-Tal, O., Bagon, S., & Dekel, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01048 · Cited by 153
Liu, T., & Liu, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3091871 · Cited by 7
Metadata
- Authors
- Shechtman, E., & Irani, M.
- Venue
- 2007 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2007-06
- Pages
- 1-8
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2144502914
- Cited by
- 1,076
- References
- 27
- FWCI
- 32.57
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.