The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Boosting and Maximum Likelihood for Exponential Models
Open accessAbstract
Recent research has considered the relationship between boosting and more standard statistical methods, such as logistic regression, concluding that AdaBoost is similar but somehow still very different from statistical methods in that it minimizes a different loss function. In this paper we derive an equivalence between AdaBoost and the dual of a convex optimization problem. In this setting, it is seen that the only difference between minimizing the exponential loss used by AdaBoost and maximum likelihood for exponential models is that the latter requires the model to be normalized to form a conditional probability distribution over labels; the two methods minimize the same Kullback-Leibler divergence objective function subject to identical feature constraints. In addition to establishing a simple and easily understood connection between the two methods, this framework enables us to derive new regularization procedures for boosting that directly correspond to penalized maximum likelihood. Experiments on UCI datasets, comparing exponential loss and maximum likelihood for parallel and sequential update algorithms, confirm our theoretical analysis, indicating that AdaBoost and maximum likelihood typically yield identical results as the number of features increases to allow the models to fit the training data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · cited 3×
“We note that throughout the paper we view boosting as a procedure for minimizing the exponential loss, using either parallel or sequential update algorithms as in [2], rather than as a forward stepwise procedure as presented in [7] or [6].”
BackgroundMethod
Cited by
The Boosting Approach to Machine Learning: An Overview
Schapire, R. E. (2003) · Lecture Notes in Statistics · doi:10.1007/978-0-387-21579-2_9 · Cited by 2,003
Foundations of Machine Learning
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018) · doi:10.7551/mitpress/10290.001.0001 · Cited by 1,122
Convexity, Classification, and Risk Bounds
Bartlett, P. L., Jordan, M. I., & McAuliffe, J. D. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000000907 · Cited by 1,037
Discriminative Reranking for Natural Language Parsing
Collins, M., & Koo, T. (2005) · Computational Linguistics · doi:10.1162/0891201053630273 · Cited by 710
An Introduction to Boosting and Leveraging
Meir, R., & Rätsch, G. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36434-x_4 · Cited by 447
Metadata
- Authors
- Lebanon, G., & Lafferty, J.
- Venue
- Advances in Neural Information Processing Systems 14
- Publisher
- The MIT Press
- Published
- 2002-11-08
- Pages
- 447-454
- OpenAlex
- W2157112198
- Cited by
- 163
- References
- 1
- FWCI
- 26.62
- Type
- book chapter
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.