The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition
Abstract
We consider visual category recognition in the framework of measuring similarities, or equivalently perceptual distances, to prototype examples of categories. This approach is quite flexible, and permits recognition based on color, texture, and particularly shape, in a homogeneous framework. While nearest neighbor classifiers are natural in this setting, they suffer from the problem of high variance (in bias-variance decomposition) in the case of limited sampling. Alternatively, one could use support vector machines but they involve time-consuming optimization and computation of pairwise distances. We propose a hybrid of these two methods which deals naturally with the multiclass setting, has reasonable computational complexity both in training and at run time, and yields excellent results in practice. The basic idea is to find close neighbors to a query sample and train a local support vector machine that preserves the distance function on the collection of neighbors. Our method can be applied to large, multiclass data sets for which it outperforms nearest neighbor and support vector machines, and remains efficient when the problem becomes intractable for support vector machines. A wide variety of distance functions can be used and our experiments show state-of-the-art performance on a number of benchmark data sets for shape and texture classification (MNIST, USPS, CUReT) and object recognition (Caltech- 101). On Caltech-101 we achieved a correct classification rate of 59.05%(±0.56%) at 15 training images per class, and 66.23%(±0.48%) at 30 training images.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Shape matching and object recognition using shape contexts
Belongie, S., Malik, J., & Puzicha, J. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993558 · Cited by 6,306
Self-citationThompson, D. W. (2019) · Stripped Bare · doi:10.1515/9780691183978-018 · Cited by 4,177
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
Rosch, E. H. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90017-0 · Cited by 2,401
Fei-Fei, L., Fergus, R., & Perona, P. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2005.09.012 · Cited by 2,359
On the algorithmic implementation of multiclass kernel-based vector machines
Crammer, K., & Singer, Y. (2002) · Cited by 1,887
Cited by
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442
Locality-constrained Linear Coding for image classification
Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540018 · Cited by 3,100
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
Self-taught learning: transfer learning from unlabeled data
Raina, R., Battle, A., Lee, H., Packer, B., & Ng, A. Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273592 · Cited by 1,510
Image Classification using Random Forests and Ferns
Bosch, A., Zisserman, A., & Munoz, X. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409066 · Cited by 1,244
Monocular Pedestrian Detection: Survey and Experiments
Enzweiler, M., & Gavrila, D. M. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.260 · Cited by 1,213
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Kernelized locality-sensitive hashing for scalable image search
Kulis, B., & Grauman, K. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459466 · Cited by 890
Ensemble of exemplar-SVMs for object detection and beyond
Malisiewicz, T., Gupta, A., & Efros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126229 · Cited by 865
Why is Real-World Visual Object Recognition Hard?
Pinto, N., Cox, D. D., & DiCarlo, J. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040027 · Cited by 537
Unsupervised learning of hierarchical representations with convolutional deep belief networks
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2011) · Communications of the ACM · doi:10.1145/2001269.2001295 · Cited by 388
Machine learning in geo- and environmental sciences: From small to large scale
Tahmasebi, P., Kamrava, S., Bai, T., & Sahimi, M. (2020) · Advances in Water Resources · doi:10.1016/j.advwatres.2020.103619 · Cited by 293
A Nonlocal Weighted Joint Sparse Representation Classification Method for Hyperspectral Imagery
Zhang, H., Li, J., Huang, Y., & Zhang, L. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2013.2264720 · Cited by 284
Self-citationLR-SMOTE — An improved unbalanced data set oversampling based on K-means and SVM
Liang, X. W., Jiang, A. P., Li, T., Xue, Y. Y., & Wang, G. T. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105845 · Cited by 282
Similarity Scores Based on Background Samples
Wolf, L., Hassner, T., & Taigman, Y. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-12304-7_9 · Cited by 279
Adaptive Active Learning for Image Classification
Li, X., & Guo, Y. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.116 · Cited by 268
Fast Similarity Search for Learned Metrics
Kulis, B., Jain, P., & Grauman, K. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.151 · Cited by 256
Improving random forests by neighborhood projection for effective text classification
Salles, T., Gonçalves, M., Rodrigues, V., & Rocha, L. (2018) · Information Systems · doi:10.1016/j.is.2018.05.006 · Cited by 66
VFIRM: Verifiable Fine-Grained Encrypted Image Retrieval in Multi-owner Multi-user Settings
Tong, Q., Miao, Y., Chen, L., Weng, J., Liu, X., Choo, K.-K. R., & Deng, R. H. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3083512 · Cited by 58
A Systematic Review of Using Machine Learning and Natural Language Processing in Smart Policing
Sarzaeim, P., Mahmoud, Q. H., Azim, A., Bauer, G., & Bowles, I. (2023) · Computers · doi:10.3390/computers12120255 · Cited by 51
Gao, J., Yang, J., Wang, G., & Li, M. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.06.055 · Cited by 44
Zeng, H., He, X., & Pan, H. (2021) · Virtual Reality · doi:10.1007/s10055-020-00476-0 · Cited by 19
Titans: Learning to Memorize at Test Time
Behrouz, A., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2501.00663 · Cited by 24
“The key ideas of learning at test time or learning to learn (i.e., [115]) backs to very early studies on local learning [116], in which each test data sample is trained on its neighbors before making a prediction [117, 22].”
Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Sun, Y., Li, X., Dalal, K., Xu, J., Vikram, A., Zhang, G., Dubois, Y., Chen, X., Wang, X., Koyejo, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.04620 · Cited by 18
“This procedure has been e ff ectively applied to models ranging from SVMs [85] to modern LLMs [29].”
End-to-End Test-Time Training for Long Context
Tandon, A., Dalal, K., Li, X., Koceja, D., Rød, M., Buchanan, S., Wang, X., Leskovec, J., Koyejo, S., Hashimoto, T., et al. (2025) · arXiv
“…form of test-time training was called locally weighted regression in the 1970s [85, 18], local learning in the 1990s [12], and KNN-SVM in the 2000s [109]: Given a test instance, find its nearest neighbors in the training set, and then train (or fine-tune) the model on these neighbors before making…”
Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2504.13173 · Cited by 65
Learning to (Learn at Test Time)
Sun, Y., Li, X., Dalal, K., Hsu, C., Koyejo, S., Guestrin, C., Wang, X., Hashimoto, T., & Chen, X. (2023) · arXiv · doi:10.48550/arxiv.2310.13807
“Some other non-parametric learners are: support vector machines (SVMs), radial basis function networks, and kernel ridge regression.”
Influential
Metadata
- Authors
- Zhang, H., Berg, A. C., Maire, M., & Malik, J.
- Venue
- 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06)
- Publisher
- IEEE
- Published
- 2006-07-10
- Volume
- 2
- Pages
- 2126-2136
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2165828254
- Cited by
- 1,114
- References
- 42
- FWCI
- 48.91
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
