Prototype‐based models in machine learning
Open accessThe DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Abstract
An overview is given of prototype‐based models in machine learning. In this framework, observations, i.e., data, are stored in terms of typical representatives. Together with a suitable measure of similarity, the systems can be employed in the context of unsupervised and supervised analysis of potentially high‐dimensional, complex datasets. We discuss basic schemes of competitive vector quantization as well as the so‐called neural gas approach and Kohonen's topology‐preserving self‐organizing map. Supervised learning in prototype systems is exemplified in terms of learning vector quantization. Most frequently, the familiar Euclidean distance serves as a dissimilarity measure. We present extensions of the framework to nonstandard measures and give an introduction to the use of adaptive distances in relevance learning. WIREs Cogn Sci 2016, 7:92–111. doi: 10.1002/wcs.1378 This article is categorized under: Psychology > Development and Aging Psychology > Learning
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936 · cited 3×
“Before discussing LVQ, we provide an intuitive introduction in terms of the simplest and most popular data-driven classifier: the Nearest neighbor (NN) classifier or its extensions to k-nearest neighbor (kNN) schemes.(3,6,50) It is roughly the machine learning counterpart of the idea of exemplars used to represent obs…”
MethodKohonen, T. (2001) · Springer Series in Information Sciences · doi:10.1007/978-3-642-56927-2 · Cited by 16,019
Least squares quantization in PCM
Lloyd, S. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056489 · Cited by 15,836
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS
FISHER, R. A. (1936) · Annals of Eugenics · doi:10.1111/j.1469-1809.1936.tb02137.x · Cited by 15,227 · cited 2×
“For illustration purposes, we revisit the wellknown Iris flower data, which was already considered by Fisher.(24) In the dataset as available from the UCI repository,(25) 150 individual flowers are represented by four numerical features (sepal length, sepal width, petal length, petal width) and belong to three differe…”
MethodA Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643 · cited 1×
“As in any stochastic gradient descent, convergence of the prototype vectors has to be guaranteed by employing a time-dependent learning rate, which slowly approaches zero in the course of training.(9) For a discussion of stochastic gradient descent in machine learning, its convergence properties, the role of the step…”
BackgroundKernel Methods for Pattern Analysis
Shawe-Taylor, J., & Cristianini, N. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809682 · Cited by 6,587 · cited 5×
“A proper kernel function corresponds to an inner product of transformed feature vectors.(60,61,67) Consider the simple example of a quadratic kernel”
BackgroundMethodInfluentialChiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced Textbooks in Control and Signal Processing · doi:10.1007/978-1-4471-0347-9_3 · Cited by 5,003
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., & Saul, L. K. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1577078 · Cited by 4,044
The condensed nearest neighbor rule (Corresp.)
Hart, P. (1968) · IEEE Transactions on Information Theory · doi:10.1109/tit.1968.1054155 · Cited by 1,733
Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S. (2009) · doi:10.1002/9780470747278 · Cited by 1,540
Clustering with Bregman Divergences
Banerjee, A., Merugu, S., Dhillon, I., & Ghosh, J. (2004) · Proceedings of the 2004 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972740.22 · Cited by 1,472
'Neural-gas' network for vector quantization and its application to time-series prediction
Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238311 · Cited by 1,431 · cited 2×
“Therefore, the name Neural Gas Algorithm has been coined for this particular VQ scheme.(5) It is frequently used for the faithful representation of datasets by a relatively large number of prototypes, the positions of which represent the density of observations in feature space.”
BackgroundGTM: The Generative Topographic Mapping
Bishop, C. M., Svensén, M., & Williams, C. K. I. (1998) · Neural Computation · doi:10.1162/089976698300017953 · Cited by 1,388
Pattern Recognition and Machine Learning
Burr, T. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s236 · Cited by 1,102
Optimization for Machine Learning
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/8996.001.0001 · Cited by 875 · cited 1×
“75 Cost-function-based approaches can also employ nondifferentiable measures if one resorts to more sophisticated optimization strategies that do not require gradients.(10)”
BackgroundSOM-based data visualization methods
Vesanto, J. (1999) · Intelligent Data Analysis · doi:10.3233/ida-1999-3203 · Cited by 851
Urine steroid metabolomics as a biomarker tool for detecting malignancy in adrenal tumors
Arlt, W., Biehl, M., Taylor, A. E., Hahner, S., Libé, R., Hughes, B. A., Schneider, P., Smith, D. J., Stiekema, H., Krone, N., et al. (2011) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2011-1565 · Cited by 458
Self-citationProceedings of the International Conference on Artificial Neural Networks
Dorffner, G., Bischof, H., & Hornik, K. (2001) · Cited by 377
Proceedings - 2004 IEEE International Conference on Fuzzy Systems
(2004) · 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat No 04CH37542) FUZZY-04 · doi:10.1109/fuzzy.2004.1375451 · Cited by 366
Adaptive Relevance Matrices in Learning Vector Quantization
Schneider, P., Biehl, M., & Hammer, B. (2009) · Neural Computation · doi:10.1162/neco.2009.11-08-908 · Cited by 345 · cited 1×
“to X i Λii = X i, j Ω(2)ij = 1 is imposed after each update step.(77) Cost-function-based schemes can be extended by an adaptive relevance matrix, analogously; see Refs 77 and 87.”
BackgroundSelf-citationQuantifying the neighborhood preservation of self-organizing feature maps
Bauer, H.-U., & Pawelzik, K. R. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.143371 · Cited by 339
Topology preservation in self-organizing feature maps: exact definition and measurement
Villmann, T., Der, R., Herrmann, M., & Martinetz, T. M. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.557663 · Cited by 311
Self-citationHow to make large self-organizing maps for nonvectorial data
Kohonen, T., & Somervuo, P. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00069-2 · Cited by 248 · cited 1×
“Then, learning corresponds to a possibly approximate optimization of NG or SOM costs with respect to these discrete parameters.(27,47) Median variants display limited flexibility when only few data points are given, as the space of possible prototype locations is not very rich.”
BackgroundImproved k-nearest neighbor classification
Wu, Y., Ianakiev, K., & Govindaraju, V. (2002) · Pattern Recognition · doi:10.1016/s0031-3203(01)00132-7 · Cited by 200
Energy functions for self-organizing maps
Heskes, T. (1999) · Elsevier eBooks · doi:10.1016/b978-044450270-4/50024-3 · Cited by 200 · cited 1×
“However, this is possible for a modification of the update, which retains the basic concepts of Kohonen’s algorithm and can be derived from a suitable cost function.(19) Advanced Review wires.”
MethodProceedings International Conference on Artificial Neural Networks ICANN'95
Bishop, C. (1995) · Cited by 194
PicSOM-self-organizing image retrieval with MPEG-7 content descriptors
Laaksonen, J., Koskela, M., & Oja, E. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.1021885 · Cited by 193
Soft Learning Vector Quantization
Seo, S., & Obermayer, K. (2003) · Neural Computation · doi:10.1162/089976603321891819 · Cited by 190 · cited 1×
“Important examples are the so-called generalized LVQ (GLVQ), which is related to the concept of large margin classification(56,57) and robust soft LVQ (RSLVQ), which is based on the statistical modeling of the data.(58,59) The basic ingredients of most LVQ training prescriptions are, however, already apparent in heuri…”
Proceedings of the International Joint Conference on Neural Networks (IJCNN) 2005
(2005) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1556189 · Cited by 168
Supervised Neural Gas with General Similarity Measure
Hammer, B., Marc Strickert, A., & Villmann, T. (2005) · Neural Processing Letters · doi:10.1007/s11063-004-3255-2 · Cited by 153
Self-citationRecursive self-organizing network models
Hammer, B., Micheli, A., Sperduti, A., & Marc Strickert, A. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.06.009 · Cited by 146
Self-citationA review of learning vector quantization classifiers
Nova, D., & Estévez, P. A. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1535-3 · Cited by 142
The self-organizing map, the Geo-SOM, and relevant variants for geosciences
Bação, F., Lobo, V. J. A. S., & Paìnho, M. (2004) · Computers & Geosciences · doi:10.1016/j.cageo.2004.06.013 · Cited by 129
Batch kernel SOM and related Laplacian methods for social network analysis
Boulet, R., Jouve, B., Rossi, F., & Vialaneix, N. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.12.026 · Cited by 116
Limited Rank Matrix Learning, discriminative dimension reduction and visualization
Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., & Biehl, M. L. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.10.001 · Cited by 110 · cited 1×
“We refer the interested reader to the literature concerning these and many other variants of the basic scheme.(76,78,87) Ready-to-use MATLAB implementations of generalized matrix relevance LVQ (GMLVQ)-based on the GLVQ cost function and several variants thereof are available in Refs 88 and 89.”
MethodSelf-citationA general framework for unsupervised processing of structured data
Hammer, B., Micheli, A., Sperduti, A., & Marc Strickert, A. (2004) · Neurocomputing · doi:10.1016/j.neucom.2004.01.008 · Cited by 110
Self-citationHierarchical GTM: constructing localized nonlinear projection manifolds in a principled way
Tiňo, P., & Nabney, I. T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000238 · Cited by 107
Dynamics and Generalization Ability of LVQ Algorithms
Biehl, M. L., Ghosh, A., & Hammer, B. (2007) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 100
Self-citationData visualization by nonlinear dimensionality reduction
Gisbrecht, A., & Hammer, B. (2015) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1147 · Cited by 99
Self-citationTopographic Mapping of Large Dissimilarity Data Sets
Hammer, B., & Hasenfuß, A. (2010) · Neural Computation · doi:10.1162/neco_a_00012 · Cited by 99 · cited 1×
“Approximate schemes allow further speedup to linear time complexity.(45)”
BackgroundSelf-citationWebsom for Textual Data Mining
Lagus, K., Honkela, T., Samuel Kaski, A., & Kohonen, T. (1999) · Artificial Intelligence Review · doi:10.1023/a:1006586221250 · Cited by 96
Soft nearest prototype classification
Seo, S., Bode, M., & Obermayer, K. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.809407 · Cited by 93 · cited 1×
“Important examples are the so-called generalized LVQ (GLVQ), which is related to the concept of large margin classification(56,57) and robust soft LVQ (RSLVQ), which is based on the statistical modeling of the data.(58,59) The basic ingredients of most LVQ training prescriptions are, however, already apparent in heuri…”
European Symposium on Artificial Neural Networks (ESANN) 2015
Mwebaze, E., Bearda, G., Biehl, M. L., & Zühlke, D. (2015) · Cited by 90
Self-citationAdvances in neural information processing systems 8: Proceedings of the 1995 conference
(1997) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(97)90029-8 · Cited by 80
Generalization and similarity in exemplar models of categorization: Insights from machine learning
Jäkel, F., Schölkopf, B., & Wichmann, F. A. (2008) · Psychonomic Bulletin & Review · doi:10.3758/pbr.15.2.256 · Cited by 74
Self-Organizing Feature Maps for Modeling and Control of Robotic Manipulators
Barreto, G. A., Araújo, A. F. R., & Ritter, H. J. (2003) · Journal of Intelligent & Robotic Systems · doi:10.1023/a:1023641801514 · Cited by 73
A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case
Barreto, G. A., Araujo, A. F. R., & Kremer, S. C. (2003) · Neural Computation · doi:10.1162/089976603321780281 · Cited by 68
Learning effective color features for content based image retrieval in dermatology
Bunte, K., Biehl, M. L., Jonkman, M. F., & Petkov, N. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.10.024 · Cited by 67
Self-citationLearning vector quantization for (dis-)similarities
Hammer, B., Hofmann, D., Schleif, F.-M., & Zhu, X. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.05.054 · Cited by 64
Self-citationKernel neural gas algorithms with application to cluster analysis
Qin, A. K., & Suganthan, P. N. (2004) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2004.1333848 · Cited by 63
Analysis of Flow Cytometry Data by Matrix Relevance Learning Vector Quantization
Biehl, M. L., Bunte, K., & Schneider, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059401 · Cited by 57
Self-citationDivergence-based classification in learning vector quantization
Mwebaze, E., Schneider, P., Schleif, F.-M., Aduwo, J. R., Quinn, J. A., Haase, S., Villmann, T., & Biehl, M. L. (2011) · Neurocomputing · doi:10.1016/j.neucom.2010.10.016 · Cited by 55 · cited 1×
“only one of the two versions δγ(x, w) or δγ(w, x) consistently in training and working phase.(62,70) An application example for the use of nonsymmetric divergences in LVQ classifiers is presented in Ref 70.”
BackgroundSelf-citationDistance-based classification of handwritten symbols
Golubitsky, O., & Watt, S. M. (2010) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-009-0107-7 · Cited by 53
Online data visualization using the neural gas network
Estévez, P. A., & Figueroa, C. J. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.024 · Cited by 49
Classification of hyperspectral imagery with neural networks: comparison to conventional tools
Merényi, E., Farrand, W. H., Taranik, J. V., & Minor, T. B. (2014) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/1687-6180-2014-71 · Cited by 48
On the equivalence between kernel self-organising maps and self-organising mixture density networks
Yin, H. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.05.007 · Cited by 42
Distance Measures for Prototype Based Classification
Biehl, M. L., Hammer, B., & Villmann, T. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-12084-3_9 · Cited by 33 · cited 1×
“only one of the two versions δγ(x, w) or δγ(w, x) consistently in training and working phase.(62,70) An application example for the use of nonsymmetric divergences in LVQ classifiers is presented in Ref 70.”
BackgroundSelf-citationA Supervised Self-Organizing Map for Structured Data
Hagenbuchner, M., Tsoi, A. C., & Sperduti, A. (2001) · Springer eBooks · doi:10.1007/978-1-4471-0715-6_4 · Cited by 32
Anomaly detection in mobile communication networks using the self-organizing map
Frota, R. A., Barreto, G. A., & Mota, J. (2007) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/ifs-2007-00363 · Cited by 22
Boareto, M., Cesar, J. E. D. S., Leite, V. B. P., & Caticha, N. (2014) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2014.2377750 · Cited by 14
Forecasting the CATS benchmark with the Double Vector Quantization method
Simon, G., Lee, J. A., Cottrell, M., et al. (2007) · Neurocomputing · doi:10.1016/j.neucom.2005.12.137 · Cited by 12
Metadata
- Authors
- Biehl, M., Hammer, B., & Villmann, T.
- Venue
- WIREs Cognitive Science
- Publisher
- Wiley
- Published
- 2016-01-21
- Volume
- 7
- Issue
- 2
- Pages
- 92-111
- ISSN
- 1939-5078 (ISSN-L)
- 1939-5086
- OpenAlex
- W2272817333
- Cited by
- 160
- References
- 63
- FWCI
- 11.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.