The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Fast exact leave-one-out cross-validation of sparse least-squares support vector machines
Abstract
Leave-one-out cross-validation has been shown to give an almost unbiased estimator of the generalisation properties of statistical models, and therefore provides a sensible criterion for model selection and comparison. In this paper we show that exact leave-one-out cross-validation of sparse Least-Squares Support Vector Machines (LS-SVMs) can be implemented with a computational complexity of only O(ln2) floating point operations, rather than the O(l2n2) operations of a naïve implementation, where l is the number of training patterns and n is the number of basis vectors. As a result, leave-one-out cross-validation becomes a practical proposition for model selection in large scale applications. For clarity the exposition concentrates on sparse least-squares support vector machines in the context of non-linear regression, but is equally applicable in a pattern recognition setting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Least Squares Support Vector Machine Classifiers
Suykens, J. A. K., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,362
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Solutions of Ill-Posed Problems.
Bell, J. B., Tikhonov, A. N., & Arsenin, V. Y. (1978) · Mathematics of Computation · doi:10.2307/2006360 · Cited by 8,101
Golub, G. H., & Loan, C. F. V. (1996) · Johns Hopkins University Press eBooks · Cited by 6,027
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 1×
“…w$x i K bÞ 2 : (1) The objective function used in ridge regression (1) implements a form of Tikhonov regularisation (Tikhonov & Arsenin, 1977) of a sum-of-squares error metric, where g is a regularisation parameter controlling the bias-variance trade-off (Geman et al., 1992; Hoerl & Kennard, 1970).”
An Introduction to Support Vector Machines and Other Kernel‐based Learning Methods
(2001) · Kybernetes · doi:10.1108/k.2001.30.1.103.6 · Cited by 3,308
Networks for approximation and learning
Poggio, T., & Girosi, F. (1990) · Proceedings of the IEEE · doi:10.1109/5.58326 · Cited by 3,252
Beckman, R. J. (1987) · Technometrics · doi:10.1080/00401706.1987.10488195 · Cited by 2,835
Fisher discriminant analysis with kernels
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., & Mullers, K. R. (2003) · Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468) · doi:10.1109/nnsp.1999.788121 · Cited by 2,661
Residuals and Influence in Regression
Ling, R. F. (1984) · Technometrics · doi:10.1080/00401706.1984.10487996 · Cited by 2,642
Choosing Multiple Parameters for Support Vector Machines
Chapelle, O., Vapnik, V., Bousquet, O., & Mukherjee, S. (2002) · Machine Learning · doi:10.1023/a:1012450327387 · Cited by 2,206
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.2307/1267351 · Cited by 1,955
Hedonic housing prices and the demand for clean air
Harrison, D., & Rubinfeld, D. L. (1978) · Journal of Environmental Economics and Management · doi:10.1016/0095-0696(78)90006-2 · Cited by 1,854
Using the Nyström Method to Speed Up Kernel Machines
Williams, C. K. I., & Seeger, M. (2000) · Cited by 1,646
The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
Allen, D. M. (1974) · Technometrics · doi:10.1080/00401706.1974.10489157 · Cited by 1,469 · cited 1×
“For instance we might adopt Allen's PRESS (predicted residual sum of squares) statistic (Allen, 1974) PRESS Z X [ iZ1 fe ðiÞ g 2 i ; where {e (i) } i Zt i K{y (i) } i is the residual for the ith training pattern during the ith iteration of the leave-one-out crossvalidation procedure.”
Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix
Sherman, J., & Morrison, W. J. (1950) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729893 · Cited by 1,402
Chatfield, C., & Weisberg, S. (1986) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981572 · Cited by 1,365
Interpolation of Scattered Data: Distance Matrices and Conditionally Positive Definite Functions
Micchelli, C. A. (1984) · Approximation Theory and Spline Functions · doi:10.1007/978-94-009-6466-2_7 · Cited by 1,297
Weighted least squares support vector machines: robustness and sparse approximation
Suykens, J. A. K., De Brabanter, J., Lukas, L., & Vandewalle, J. (2002) · Neurocomputing · doi:10.1016/s0925-2312(01)00644-0 · Cited by 1,265
Some results on Tchebycheffian spline functions
Kimeldorf, G., & Wahba, G. (1971) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(71)90184-3 · Cited by 1,254
Interpolation of scattered data: Distance matrices and conditionally positive definite functions
Micchelli, C. A. (1986) · Constructive Approximation · doi:10.1007/bf01893414 · Cited by 1,247
Hassard, T. H., & Weisberg, S. (1981) · The Statistician · doi:10.2307/2987575 · Cited by 1,216
Some aspects of the spline smoothing approach to non-parametric regression curve fitting
Silverman, B. W. (1985) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1985.tb01327.x · Cited by 1,117 · cited 2×
“The Motorcycle benchmark consists of a sequence of accelerometer readings through time following a simulated motor-cycle crash during an experiment to determine the efficacy of crash-helmets (Silverman, 1985).”
Updating the Inverse of a Matrix
Hager, W. W. (1989) · SIAM Review · doi:10.1137/1031049 · Cited by 1,103
Ridge Regression Learning Algorithm in Dual Variables
Saunders, C. J., Gammerman, A., & Vovk, V. (1998) · ePrints Soton (University of Southampton) · Cited by 811
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
RSVM: Reduced Support Vector Machines
Lee, Y.-J., & Mangasarian, O. L. (2001) · Proceedings of the 2001 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972719.13 · Cited by 647
Bounds on Error Expectation for Support Vector Machines
Vapnik, V., & Chapelle, O. (2000) · Neural Computation · doi:10.1162/089976600300015042 · Cited by 624
Sparse Greedy Matrix Approximation for Machine Learning
Smola, A. J., & Schökopf, B. (2000) · doi:10.5555/645529.657980 · Cited by 574
An Inverse Matrix Adjustment Arising in Discriminant Analysis
Bartlett, M. S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729698 · Cited by 429
Simplified support vector decision rules
Burges, C. J. C. (1996) · Cited by 427
Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms
Keerthi, S. S. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.1031955 · Cited by 317
Sparse approximation using least squares support vector machines
Suykens, J. A. K., Lukas, L., & Vandewalle, J. (2002) · 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353) · doi:10.1109/iscas.2000.856439 · Cited by 316
Improving the accuracy and speed of support vector learning machines
Burges, C. J. C., Schölkopf, B., Mozer, Jordan, M. I., & Petsche, T. (1997) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 209
Exact simplification of support vector solutions
Downs, T., Gates, K. E., & Masters, A. L. (2002) · doi:10.5555/944790.944814 · Cited by 195
Kernel-based methods and function approximation
Baudat, G., & Anouar, F. (2002) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2001.939539 · Cited by 153
Kernel MSE algorithm: a unified framework for KFD, LS-SVM and KRR
Xu, J., Zhang, X., & Li, Y. (2002) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2001.939584 · Cited by 83
Improved sparse least-squares support vector machines
Cawley, G. C., & Talbot, N. L. C. (2002) · Neurocomputing · doi:10.1016/s0925-2312(02)00606-9 · Cited by 69
Self-citationA Greedy Training Algorithm for Sparse Least-Squares Support Vector Machines
Cawley, G. C., & Talbot, N. L. C. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46084-5_111 · Cited by 18
Self-citation(1998)
Cited by
A tutorial on support vector machine-based methods for classification problems in chemometrics
Luts, J., Ojeda, F., Van de Plas, R., De Moor, B., Van Huffel, S., & Suykens, J. A. K. (2010) · Analytica Chimica Acta · doi:10.1016/j.aca.2010.03.030 · Cited by 326
An, S., Liu, W., & Venkatesh, S. (2007) · Pattern Recognition · doi:10.1016/j.patcog.2006.12.015 · Cited by 301
Detection of early prognostic biomarkers for metastasis of Ewing's sarcoma in pediatric patients
Ezzeldin, S., Osama, A., Anwar, A. M., Mahgoub, S., Ahmed, E. A., Farid, N., Zamzam, M., El Ghoneimy, A., & Magdeldin, S. (2023) · Life Sciences · doi:10.1016/j.lfs.2023.122237 · Cited by 12
Metadata
- Authors
- Cawley, G. C., & Talbot, N. L. C.
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2004-12
- Volume
- 17
- Issue
- 10
- Pages
- 1467-1475
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2069659771
- Cited by
- 332
- References
- 50
- FWCI
- 7.46
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.