Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1,246

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Online Learning and Online Convex Optimization

Shalev-Shwartz, Shai·2012·Foundations and Trends® in Machine Learning
DOI 10.1561/2200000018
Open this paper in CoChat — free account

Abstract

Online learning is a well established learning paradigm which has both theoretical and practical appeals. The goal of online learning is to make a sequence of accurate predictions given knowledge of the correct answer to previous prediction tasks and possibly additional available information. Online learning has been studied in several research fields including game theory, information theory, and machine learning. It also became of great interest to practitioners due the recent emergence of large scale applications such as online advertisement placement and online web ranking. In this survey we provide a modern overview of online learning. Our goal is to give the reader a sense of some of the interesting ideas and in particular to underscore the centrality of convexity in deriving efficient online learning algorithms. We do not mean to be comprehensive but rather to give a high-level, rigorous yet easy to follow, survey.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1561/2200000018

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 43

  1. Abernethy, 2008, Optimal strategies and minimax lower bounds for online convex games, Proceedings of the 21st Annual Conference on Learning Theory (COLT)
  2. Abernethy, 2008, Competing in the dark: An efficient algorithm for bandit linear optimization, Proceedings of the 21st Annual Conference on Learning Theory (COLT)
  3. Agmon, 1954, The relaxation method for linear inequalities, Canadian Journal of Mathematics, 6, 382, 10.4153/CJM-1954-037-2 · DOI
  4. Auer, 2002, The nonstochastic multiarmed bandit problem, SIAM Journal on Computing, 32, 48, 10.1137/S0097539701398375 · DOI
  5. Auer, 1998, Tracking the best disjunction, Machine Learning, 32, 127, 10.1023/A:1007472513967 · DOI
  6. Azoury, 2001, Relative loss bounds for on-line density estimation with the exponential family of distributions, Machine Learning, 43, 211, 10.1023/A:1010896012157 · DOI · Paper page
  7. Ben-David, 2009, Agnostic online learning, Proceedings of the 22nd Annual Conference on Learning Theory (COLT)
  8. Blum, 1998, Online Algorithms: The State of the Art, 306, 10.1007/BFb0029575 · DOI
  9. Borwein, 2006, Convex Analysis and Nonlinear Optimization: Theory and Examples, 10.1007/978-0-387-31256-9 · DOI
  10. Cesa-Bianchi, 2004, On the generalization ability of on-line learning algorithms, IEEE Transactions on Information Theory, 50, 2050, 10.1109/TIT.2004.833339 · DOI
  11. Cesa-Bianchi, 2006, Improved risk tail bounds for on-line algorithms, Advances in Neural Information Processing Systems 18 (NIPS 2005)
  12. Cesa-Bianchi, 2006, Prediction, Learning, and Games, 10.1017/CBO9780511546921 · DOI · Paper page
  13. Cover, 1965, Behavior of sequential predictors of binary sequences, Transactions of the Fourth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes, 263
  14. Cristianini, 2000, An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, 10.1017/CBO9780511801389 · DOI · Paper page
  15. Dani, 2008, The price of bandit information for online optimization, Advances in Neural Information Processing Systems 20, 345
  16. Flaxman, 2005, Online convex optimization in the bandit setting: Gradient descent without a gradient, Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 385
  17. Freund, 1999, Large margin classification using the perceptron algorithm, Machine Learning, 37, 277, 10.1023/A:1007662407062 · DOI
  18. Gentile, 2003, The robustness of the p-norm algorithms, Machine Learning, 53, 265, 10.1023/A:1026319107706 · DOI
  19. Gentile, 1999, The robustness of the p-norm algorithms, Proceedings of the 12th Annual Conference on Computational Learning Theory (COLT), 10.1145/307400.307405 · DOI
  20. Gordon, 1999, Regret bounds for prediction problems, Proceedings of the 12th Annual Conference on Computational Learning Theory (COLT), 29, 10.1145/307400.307410 · DOI
  21. Grove, 2001, General convergence results for linear discriminant updates, Machine Learning, 43, 173, 10.1023/A:1010844028087 · DOI
  22. Hazan, 2011, Optimization for Machine Learning
  23. Helmbold, 1995, On weak learning, Journal of Computer and System Sciences, 50, 551, 10.1006/jcss.1995.1044 · DOI
  24. Kakade, 2012, Regularization techniques for learning with matrices, Journal of Machine Learning Research, 13, 1865
  25. Kalai, 2005, Efficient algorithms for online decision problems, Journal of Computer and System Sciences, 71, 291, 10.1016/j.jcss.2004.10.016 · DOI · Paper page
  26. Kivinen, 1997, Exponentiated gradient versus gradient descent for linear predictors, Information and Computation, 132, 1, 10.1006/inco.1996.2612 · DOI
  27. Kivinen, 1995, Additive versus exponentiated gradient updates for linear prediction, Proceedings of the Twenty-Seventh Annual ACM Symposium on Theory of Computing (STOC), 209, 10.1145/225058.225121 · DOI
  28. Kivinen, 2001, Relative loss bounds for multidimensional regression problems, Machine Learning, 45, 301, 10.1023/A:1017938623079 · DOI
  29. Littlestone, 1988, Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm, Machine Learning, 2, 285, 10.1023/A:1022869011914 · DOI · Paper page
  30. Littlestone, 1989, From on-line to batch learning, Proceedings of the Second Annual Workshop on Computational Learning Theory (COLT), 269, 10.1016/B978-0-08-094829-4.50022-2 · DOI
  31. Littlestone, 1989, Ph.D. dissertation
  32. Littlestone, 1994, The weighted majority algorithm, Information and Computation, 108, 212, 10.1006/inco.1994.1009 · DOI
  33. Minsky, 1969, Perceptrons: An Introduction to Computational Geometry
  34. Rakhlin, 2010, Online learning: Random averages, combinatorial parameters, and learnability, Advances in Neural Information Processing Systems 23 (NIPS 2010), 1984
  35. Schölkopf, 2002, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
  36. Shalev-Shwartz, 2007, Online learning: Theory, algorithms, and applications, Ph.D. dissertation
  37. Shalev-Shwartz, 2007, A primal-dual perspective of online learning algorithms, Machine Learning, 69, 115, 10.1007/s10994-007-5014-x · DOI
  38. Spall, 2003, Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control, 10.1002/0471722138 · DOI · Paper page
  39. Vapnik, 1998, Statistical Learning Theory
  40. Vovk, 1990, Proceedings of the 3rd Annual Workshop on Computational Learning Theory (COLT), 371
  41. Zalinescu, 2002, Convex Analysis in General Vector Spaces, 10.1142/5021 · DOI · Paper page
  42. Zhang, 2005, Data dependent concentration bounds for sequential prediction algorithms, Proceedings of the 18th Annual Conference on Learning Theory (COLT), 173, 10.1007/11503415_12 · DOI
  43. Zinkevich, 2003, Online convex programming and generalized infinitesimal gradient ascent, Proceedings of the Twentieth International Conference on Machine Learning (ICML), 928

Cited by

See all 1,246

Metadata

Authors
Shalev-Shwartz, S.
Venue
Foundations and Trends® in Machine Learning
Publisher
Emerald
Published
2012-03-29
Volume
4
Issue
2
Pages
107-194
ISSN
  • 1935-8237 (ISSN-L)
  • 1935-8245
DOI
10.1561/2200000018
OpenAlex
W4292022450
Cited by
1,246
References
2
FWCI
31.83
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.