Cited by 1,453Advanced Optimization Algorithms ResearchStochastic Gradient Optimization TechniquesMachine Learning and Algorithms

Multiple kernel learning, conic duality, and the SMO algorithm

Bach, Francis R., Lanckriet, Gert R. G., Jordan, Michael I.·2004·Twenty-first international conference on Machine learning - ICML '04

Abstract

While classical kernel-based classifiers are based on a single kernel, in practice it is often desirable to base classifiers on combinations of multiple kernels. Lanckriet et al. (2004) considered conic combinations of kernel matrices for the support vector machine (SVM), and showed that the optimization of the coefficients of such a combination reduces to a convex optimization problem known as a quadratically-constrained quadratic program (QCQP). Unfortunately, current convex optimization toolboxes can solve this problem only for a small number of kernels and a small number of data points; moreover, the sequential minimal optimization (SMO) techniques that are essential in large-scale implementations of the SVM cannot be applied because the cost function is non-differentiable. We propose a novel dual formulation of the QCQP as a second-order cone programming problem, and show how to exploit the technique of Moreau-Yosida regularization to yield a formulation to which SMO techniques can be applied. We present experimental results that show that our SMO-based algorithm is significantly more efficient than the general-purpose interior point methods available in current optimization toolboxes.

Integrity signals

Registry

Crossref · 10.1145/1015330.1015424

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • With a frozen value extraction function psi, training can make each memory unit statistically more efficient by tuning the key extraction function phi, i.e., learning a similarity metric (kernel).

    “Let us first assume that each memory unit has a frozen value extraction function ψ.The training procedure can still make each memory unit statistically more efficient by tuning the parameters of the key extraction function φ, that is, by learning how to compare the current prediction context (x T , x T -1 , x T -2 . . . ) with past prediction contexts (x t , x t-1 , x t-2 . . . ) for t < T .”

    From Memory Mosaics (2024) · cites this paper as Bach et al., 2004

Cites

Flagged first, then by citations

1 of the 13 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Bach, F. R., Lanckriet, G. R. G., & Jordan, M. I.
Affiliations
University of California, Berkeley (US)
Venue
Twenty-first international conference on Machine learning - ICML '04
Publisher
ACM Press
Published
2004
Pages
6
OpenAlex
W2031823405
Cited by
1,453
References
13
Citations by year
2026: 7 · 2025: 4 · 2024: 19 · 2023: 20 · 2022: 22 · 2021: 36 · 2020: 52 · 2019: 84 · 2018: 89 · 2017: 86 · 2016: 84 · 2015: 116 · 2014: 136 · 2013: 123 · 2012: 132
Type
conference abstract
Language
en
Open access
closed
Locations
International Conference on Machine Learning (publishedVersion)
Funders
  • Intel Corporation
  • Microsoft Research
Topics
  • Physical Sciences › Mathematics › Numerical Analysis › Advanced Optimization Algorithms Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
multiple kernel learning, support vector machine, sequential minimal optimization, second-order cone programming, Moreau-Yosida regularization, quadratically constrained quadratic programming, non-differentiable cost functions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review