Tracking the Best Disjunction
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
Littlestone developed a simple deterministic on-line learning algorithm for learning k-literal disjunctions. This algorithm (called $${WINNOW}$$ ) keeps one weight for each of then variables and does multiplicative updates to its weights. We develop a randomized version of $${WINNOW} $$ and prove bounds for an adaptation of the algorithm for the case when the disjunction may change over time. In this case a possible target disjunction schedule $${\mathcal{T}} $$ is a sequence of disjunctions (one per trial) and the shift size is the total number of literals that are added/removed from the disjunctions as one progresses through the sequence. We develop an algorithm that predicts nearly as well as the best disjunction schedule for an arbitrary sequence of examples. This algorithm that allows us to track the predictions of the best disjunction is hardly more complex than the original version. However, the amortized analysis needed for obtaining worst-case mistake bounds requires new techniques. In some cases our lower bounds show that the upper bounds of our algorithm have the right constant in front of the leading term in the mistake bound and almost the right constant in front of the second leading term. Computer experiments support our theoretical findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Cited by
Prediction, Learning, and Games
Cesa-Bianchi, N., & Lugosi, G. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546921 · Cited by 3,162
Machine learning: a review of classification and combining techniques
Kotsiantis, S. B., Zaharakis, I. D., & Pintelas, P. E. (2006) · Artificial Intelligence Review · doi:10.1007/s10462-007-9052-3 · Cited by 1,582
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000018 · Cited by 1,246
Online Learning and Online Convex Optimization
Shalev-Shwartz, S. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985477 · Cited by 1,095
Kivinen, J., Smola, A. J., & Williamson, R. C. (2004) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2004.830991 · Cited by 1,004
Chakrabarti, D., Kumar, R., & Tomkins, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150467 · Cited by 661
Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts
Kolter, J. Z., & Maloof, M. A. (2007) · doi:10.5555/1314498.1390333 · Cited by 648
Herbster, M., & Warmuth, M. K. (1998) · Machine Learning · doi:10.1023/a:1007424614876 · Cited by 560
Self-citationOn the Generalization Ability of On-Line Learning Algorithms
Cesa-Bianchi, N., Conconi, A., & Gentile, C. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.833339 · Cited by 426
On‐Line Portfolio Selection Using Multiplicative Updates
Helmbold, D. P., Schapire, R. E., Singer, Y., & Warmuth, M. K. (1998) · Mathematical Finance · doi:10.1111/1467-9965.00058 · Cited by 387
Self-citationCompetitive On‐line Statistics
Vovk, V. (2001) · International Statistical Review · doi:10.1111/j.1751-5823.2001.tb00457.x · Cited by 216
“This is a special case of a more general problem of tracking the best expert, which originated in Littlestone and Warmuth [45] and was further studied by, among others, Herbster and Warmuth [36], Auer and Warmuth [2], Vovk [62].”
BackgroundInfluentialAgnostic Learning of Monomials by Halfspaces Is Hard
Feldman, V., Guruswami, V., Raghavendra, P., & Wu, Y. (2012) · SIAM Journal on Computing · doi:10.1137/120865094 · Cited by 143
Targeting Prospective Customers: Robustness of Machine-Learning Methods to Typical Data Challenges
Simester, D., Timoshenko, A., & Zoumpoulis, S. I. (2020) · Management Science · doi:10.1287/mnsc.2019.3308 · Cited by 133
Metadata
- Authors
- Auer, P., & Warmuth, M. K.
- Venue
- Machine Learning
- Published
- 1998-08
- Volume
- 32
- Issue
- 2
- Pages
- 127-150
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W260562682
- Cited by
- 110
- References
- 1
- FWCI
- 8.75
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.