The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Bayesian Framework for Least-Squares Support Vector Machine Classifiers, Gaussian Processes, and Kernel Fisher Discriminant Analysis
Abstract
The Bayesian evidence framework has been successfully applied to the design of multilayer perceptrons (MLPs) in the work of MacKay. Nevertheless, the training of MLPs suffers from drawbacks like the nonconvex optimization problem and the choice of the number of hidden units. In support vector machines (SVMs) for classification, as introduced by Vapnik, a nonlinear decision boundary is obtained by mapping the input vector first in a nonlinear way to a high-dimensional kernel-induced feature space in which a linear large margin classifier is constructed. Practical expressions are formulated in the dual space in terms of the related kernel function, and the solution follows from a (convex) quadratic programming (QP) problem. In least-squares SVMs (LS-SVMs), the SVM problem formulation is modified by introducing a least-squares cost function and equality instead of inequality constraints, and the solution follows from a linear system in the dual space. Implicitly, the least-squares formulation corresponds to a regression formulation and is also related to kernel Fisher discriminant analysis. The least-squares regression formulation has advantages for deriving analytic expressions in a Bayesian evidence framework, in contrast to the classification formulations used, for example, in gaussian processes (GPs). The LS-SVM formulation has clear primal-dual interpretations, and without the bias term, one explicitly constructs a model that yields the same expressions as have been obtained with GPs for regression. In this article, the Bayesian evidence frame-work is combined with the LS-SVM classifier formulation. Starting from the feature space formulation, analytic expressions are obtained in the dual space on the different levels of Bayesian inference, while posterior class probabilities are obtained by marginalizing over the model param-eters. Empirical results obtained on 10 public domain data sets show that the LS-SVM classifier designed within the Bayesian evidence framework consistently yields good generalization performances.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
UCI Repository of machine learning databases
Blake, C. L. (1998) · Medical Entomology and Zoology · Cited by 10,582
Least Squares Support Vector Machine Classifiers
Suykens, J. A. K., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,362
Self-citationHorn, R. A., & Johnson, C. R. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511840371 · Cited by 9,319
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
Schölkopf, B., Smola, A., & Müller, K.-R. (1998) · Neural Computation · doi:10.1162/089976698300017467 · Cited by 8,084
Pattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,459
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 6,385
An Introduction to Support Vector Machines
Cristianini, N., & Shawe-Taylor, J. S. (2000) · ePrints Soton (University of Southampton) · Cited by 4,115
Generalized Discriminant Analysis Using a Kernel Approach
Baudat, G., & Anouar, F. (2000) · Neural Computation · doi:10.1162/089976600300014980 · Cited by 1,673
Mackay, D. J. C. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/3/011 · Cited by 915
Ridge Regression Learning Algorithm in Dual Variables
Saunders, C. J., Gammerman, A., & Vovk, V. (1998) · ePrints Soton (University of Southampton) · Cited by 811
Bayesian classification with Gaussian processes
Williams, C. K. I., & Barber, D. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735807 · Cited by 766
The Evidence Framework Applied to Classification Networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.720 · Cited by 722
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
Williams, C. K. I. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_23 · Cited by 659
The connection between regularization operators and support vector kernels
Smola, A. J., Schölkopf, B., & Müller, K.-R. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00032-x · Cited by 625
Optimal control by least squares support vector machines
Suykens, J. A. K., Vandewalle, J., & De Moor, B. (2001) · Neural Networks · doi:10.1016/s0893-6080(00)00077-0 · Cited by 540
Self-citationVan Gestel, T., Suykens, J. A. K., Baestaens, D.-E., Lambrechts, A., Lanckriet, G., Vandaele, B., De Moor, B., & Vandewalle, J. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935093 · Cited by 499
Self-citationEvaluation of gaussian processes and other methods for non-linear regression
Hinton, G. E., & Rasmussen, C. E. (1997) · Library and Archives Canada (Government of Canada) · Cited by 429
Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification
Neal, R. M. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.physics/9701026 · Cited by 375
Recurrent least squares support vector machines
Suykens, J. A. K., & Vandewalle, J. (2000) · IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications · doi:10.1109/81.855471 · Cited by 338
Self-citationBayesian Inductive Inference and Maximum Entropy
Gull, S. F. (1988) · Maximum-Entropy and Bayesian Methods in Science and Engineering · doi:10.1007/978-94-009-3049-0_4 · Cited by 246
Comparison of Approximate Methods for Handling Hyperparameters
MacKay, D. J. C. (1999) · Neural Computation · doi:10.1162/089976699300016331 · Cited by 244
Support Vector Machine Classification of Microarray Data
Mukherjee, S., & Rifkin, R. M. (2001) · Cited by 179
Bayesian kernel based classification for financial distress detection
Gestel, T. V., Baesens, B., Suykens, J. A. K., Van den Poel, D., Baestaens, D.-E., & Willekens, M. (2006) · European Journal of Operational Research · doi:10.1016/j.ejor.2004.11.009 · Cited by 177
Self-citationModerating the outputs of support vector machine classifiers
Kwok, J. T.-Y. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788642 · Cited by 159
The evidence framework applied to support vector machines
Kwok, J. T.-Y. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.870047 · Cited by 157
A support vector machine formulation to pca analysis and its kernel version
Suykens, J. A. K., Gestel, T. V., Vandewalle, J. P. L., et al. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.809414 · Cited by 142
Self-citationSMO Algorithm for Least-Squares SVM Formulations
Keerthi, S. S., & Shevade, S. (2003) · Neural Computation · doi:10.1162/089976603762553013 · Cited by 127
Probabilistic Methods for Support Vector Machines
Sollich, P. (1999) · Cited by 63
Shashua, A. (1999) · Neural Processing Letters · doi:10.1023/a:1018677409366 · Cited by 52
Kernel-dependent support vector error bounds
Schölkopf, B. (1999) · International Conference on Artificial Neural Networks · doi:10.1049/cp:19991092 · Cited by 45
Centering and Scaling in Ridge Regression
Brown, P. J. (1977) · Technometrics · doi:10.1080/00401706.1977.10489495 · Cited by 39
Image representations for object detection using kernel classifiers
Evgeniou, E., Pontil, M., Papageorgiou, C., & Poggio, T. (2000) · UCL Discovery (University College London) · Cited by 37
Bioinformatics: Organisms from Venus, Technology from Jupiter, Algorithms from Mars
Moor, B. D., Marchal, K., Mathys, J., & Moreau, Y. (2003) · European Journal of Control · doi:10.3166/ejc.9.237-278 · Cited by 9
Self-citation
Cited by
An efficient P300-based brain–computer interface for disabled subjects
Hoffmann, U., Vesin, J.-M., Ebrahimi, T., & Diserens, K. (2008) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.03.005 · Cited by 859
Yang, J., Frangi, A. F., Yang, J.-Y., Zhang, D., & Jin, Z. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.33 · Cited by 826
Benchmarking Least Squares Support Vector Machine Classifiers
van Gestel, T., Suykens, J. A. K., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., de Moor, B., & Vandewalle, J. (2004) · Machine Learning · doi:10.1023/b:mach.0000008082.80494.e0 · Cited by 745
Self-citationMultiobjective Intelligent Energy Management for a Microgrid
Chaouachi, A., Kamel, R. M., Andoulsi, R., & Nagasaka, K. (2013) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2012.2188873 · Cited by 657
A new fuzzy support vector machine to evaluate credit risk
Wang, Y., Wang, S., & Lai, K. K. (2005) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2005.859320 · Cited by 383
Metadata
- Authors
- Gestel, T. V., Suykens, J. A. K., Lanckriet, G., Lambrechts, A., Moor, B. D., & Vandewalle, J.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-05-01
- Volume
- 14
- Issue
- 5
- Pages
- 1115-1147
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2029279587
- Cited by
- 291
- References
- 39
- FWCI
- 12.38
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.