Algorithms for Learning Kernels Based on Centered Alignment
Open accessThe DOI matches its OpenAlex record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
This paper presents new and effective algorithms for learning kernels. In particular, as shown by our empirical results, these algorithms consistently outperform the so-called uniform combination solution that has proven to be difficult to improve upon in the past, as well as other algorithms for learning kernels based on convex combinations of base kernels in both classification and regression. Our algorithms are based on the notion of centered alignment which is used as a similarity measure between kernels or kernel matrices. We present a number of novel algorithmic, theoretical, and empirical results for learning kernels based on our notion of centered alignment. In particular, we describe efficient algorithms for learning a maximum alignment kernel by showing that the problem can be reduced to a simple QP and discuss a one-stage algorithm for learning both a kernel and a hypothesis based on that kernel using an alignment-based regularization. Our theoretical results include a novel concentration bound for centered alignment between kernel matrices, the proof of the existence of effective predictors for kernels with high alignment, both for classification and for regression, and the proof of stability-based generalization bounds for a broad family of algorithms for learning kernels based on centered alignment. We also report the results of experiments with our centered alignment-based algorithms in both classification and regression.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
Choosing Multiple Parameters for Support Vector Machines
Chapelle, O., Vapnik, V., Bousquet, O., & Mukherjee, S. (2002) · Machine Learning · doi:10.1023/a:1012450327387 · Cited by 2,206
(2007) · Cited by 2,018
Measuring Statistical Dependence with Hilbert-Schmidt Norms
Gretton, A., Bousquet, O., Smola, A., & Schölkopf, B. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564089_7 · Cited by 1,560
On the method of bounded differences
McDiarmid, C. (1989) · Surveys in Combinatorics, 1989 · doi:10.1017/cbo9781107359949.008 · Cited by 1,425
Large Scale Multiple Kernel Learning
Sonnenburg, S., Rätsch, G., Schäfer, C., & Schölkopf, B. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248604 · Cited by 1,379
Cristianini, N., Kandola, J., Elisseeff, A., & Shawe-Taylor, J. (2006) · Studies in Fuzziness and Soft Computing · doi:10.1007/3-540-33486-6_8 · Cited by 837
Ridge Regression Learning Algorithm in Dual Variables
Saunders, C. J., Gammerman, A., & Vovk, V. (1998) · ePrints Soton (University of Southampton) · Cited by 811
More generality in efficient multiple kernel learning
Varma, M., & Babu, B. R. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553510 · Cited by 398
Learning the Kernel Function via Regularization
Micchelli, C. A., & Pontil, M. (2005) · doi:10.5555/1046920.1088710 · Cited by 388
L2 Regularization for Learning Kernels
Cortes, C., Mohri, M., & Rostamizadeh, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2653 · Cited by 362
Self-citationLearning the Kernel with Hyperkernels
Ong, C. S., Smola, A. J., Williamson, R. C., & Herbrich, R. (2005) · MPG.PuRe (Max Planck Society) · Cited by 353
Learning Non-Linear Combinations of Kernels
Cortes, C., Mohri, M., & Rostamizadeh, A. (2009) · neural information processing systems · Cited by 326
Self-citationMulticlass multiple kernel learning
Zien, A., & Ong, C. S. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273646 · Cited by 266
Multi-task feature and kernel selection for SVMs
Jebara, T. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015426 · Cited by 218
Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning
Bach, F. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0809.1493 · Cited by 172
Two-Stage Learning Kernel Algorithms
Cortes, C., Mohri, M., & Rostamizadeh, A. (2010) · Cited by 170
Self-citationOptimal kernel selection in Kernel Fisher discriminant analysis
Kim, S.-J., Magnani, A., & Boyd, S. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143903 · Cited by 146
On the Impact of Kernel Approximation on Learning Accuracy
Cortes, C., Mohri, M., & Talwalkar, A. (2010) · Cited by 126
Self-citationA DC-programming algorithm for kernel selection
Argyriou, A., Hauser, R., Micchelli, C. A., et al. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143850 · Cited by 114
Learning Convex Combinations of Continuously Parameterized Basic Kernels
Argyriou, A., Micchelli, C. A., & Pontil, M. (2005) · Lecture notes in computer science · doi:10.1007/11503415_23 · Cited by 108
Generalization Bounds for Learning Kernels
Cortes, C., Mohri, M., & Rostamizadeh, A. (2010) · Cited by 97
Self-citationAlgorithmic Stability and Generalization Performance
Bousquet, O., Elisseeff, A., & Bioinformatics, B. (2000) · Cited by 87
Nonstationary kernel combination
Lewis, D. P., Jebara, T., & Noble, W. S. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143914 · Cited by 78
Learning Bounds for Support Vector Machines with Learned Kernels
Srebro, N., & Ben-David, S. (2006) · Lecture notes in computer science · doi:10.1007/11776420_15 · Cited by 77
On a theory of learning with similarity functions
Balcan, M.-F., & Blum, A. L. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143854 · Cited by 71
Cortes, C., Mohri, M., & Rostamizadeh, A. (2008) · Machine learning for signal processing ... · doi:10.1109/mlsp.2008.4685446 · Cited by 48
Self-citationData centering in feature space.
Meilă, M. (2003) · International Conference on Artificial Intelligence and Statistics · Cited by 35
Cited by
SIRIUS 4: a rapid tool for turning tandem mass spectra into metabolite structure information
Dührkop, K., Fleischauer, M., Ludwig, M., Aksenov, A. A., Melnik, A. V., Meusel, M., Dorrestein, P. C., Rousu, J., & Böcker, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0344-8 · Cited by 2,060
Searching molecular structure databases with tandem mass spectra using CSI:FingerID
Dührkop, K., Shen, H., Meusel, M., Rousu, J., & Böcker, S. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1509788112 · Cited by 1,190
Riemannian Approaches in Brain-Computer Interfaces: A Review
Yger, F., Berar, M., & Lotte, F. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2016.2627016 · Cited by 420
Multiple Kernel k-means with Incomplete Kernels
Liu, X., Gao, W., Zhu, X., Li, M., Wang, L., Zhu, E., Liu, T., Kloft, M., Shen, D., & Yin, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2892416 · Cited by 300
Nguyen, T., Raghu, M., & Kornblith, S. (2020) · arXiv · doi:10.48550/arxiv.2010.15327 · Cited by 112
Metadata
- Authors
- Cortes, C., Mohri, M., & Rostamizadeh, A.
- Venue
- arXiv (Cornell University)
- Publisher
- arXiv
- Published
- 2012
- Volume
- 13
- Issue
- 1
- Pages
- 795-828
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1203.0550
- OpenAlex
- W1920328734
- Cited by
- 375
- References
- 31
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.