The DOI matches its Crossref record; 39 of the 42 references it cites were found in a registry or an index, and 3 are not checked yet.
Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)
Abstract
Boosting is one of the most important recent developments in classification methodology. Boosting works by sequentially applying a classification algorithm to reweighted versions of the training data and then taking a weighted majority vote of the sequence of classifiers thus produced. For many classification algorithms, this simple strategy results in dramatic improvements in performance. We show that this seemingly mysterious phenomenon can be understood in terms of well-known statistical principles, namely additive modeling and maximum likelihood. For the two-class problem, boosting can be viewed as an approximation to additive modeling on the logistic scale using maximum Bernoulli likelihood as a criterion. We develop more direct approximations and show that they exhibit nearly identical results to boosting. Direct multiclass generalizations based on multinomial likelihood are derived that exhibit performance comparable to other recently proposed multiclass generalizations of boosting in most situations, and far superior in some. We suggest a minor modification to boosting that can reduce computation, often by factors of 10 to 50. Finally, we apply these insights to produce an alternative formulation of boosting decision trees. This approach, based on best-first truncated tree induction, often leads to better performance, and can provide interpretable descriptions of the aggregate decision rule. It is also much faster computationally, making it more suitable to large-scale data mining applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Friedman et al. (2000) andFriedman (2001) extend boosting to contexts beyond classification, eventually leading to the gradient boosted regression tree.”
From Empirical Asset Pricing via Machine Learning (2020) · cites this paper as Friedman et al. (2000)
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 7×
“Details of these generalizations may be found in Friedman (1999).”
BackgroundMethodInfluentialSelf-citationA decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936 · cited 1×
“Classifiers are hurt less by overfitting than other function estimators [e.g., the famous risk bound of the 1-nearest-neighbor classifier, Cover and Hart (1967)].”
BackgroundMcCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
Self-citationMatching pursuits with time-frequency dictionaries
Mallat, S. G., & Zhang, Z. (1993) · IEEE Transactions on Signal Processing · doi:10.1109/78.258082 · Cited by 9,112 · cited 2×
“2 Schapire & Singer (1998) give the interpretation that the weights are updated to make the new weighted problem maximally di cult for the next weak learner.”
BackgroundMethodMultivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
Self-citationRobust estimation of a location parameter
Huber, P. J. (1964) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177703732 · Cited by 7,228
The Strength of Weak Learnability
Schapire, R. E. (1990) · Machine Learning · doi:10.1007/bf00116037 · Cited by 4,600 · cited 4×
“" Boosting was proposed in the computational learning theory literature [Schapire (1990), Freund (1995), Freund and Schapire (1997)] and has since received much attention.”
BackgroundMethodInfluential
Research works printed but not linked: 13
- Breiman, L. (1996). Bagging predictors. Machine Learning 24 123-140. · doi:10.1023/a:1018054314350 · Paper page
- Ridgeway, G. (1999). The state of boosting. In Proceedings of the Thirty-first Symposium on the Interface 172-181.
- Ridgeway, G. (1999). The state of boosting. In Computing Science and Statistics 31 (K. Berk, M. Pourahmadi, eds.) Interface Foundation of North America, 172-181. Fairfax, VA.
- Freund, Y. and Schapire, R. E. (1997). A decision-theoretic generalization of online learning and an application to boosting. J. Comput. System Sciences 55. · DOI · Paper page
- Schapire, R., Freund, Y., Bartlett, P. and Lee, W. (1998). Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651-1686. · DOI · Paper page
- B ¨uhlmann, P. and Yu, B. (2000). Explaining bagging. Preprint.
- Gill, E. P., Murray, W. and Wright, M. H. (1981). Practical Optimization. Academic Press, New York.
- Breimann, L. (1996). Bagging predictors. Machine Learning 24 123-140. · doi:10.1023/a:1018054314350 · Paper page
- Breiman, L. (1998). Arcing classifiers. Ann. Statist. 26 801-849. · DOI · Paper page
- Freund, Y. and Mason, L. (1999). The alternating decision tree learning algorithm. In Machine Learning: Proceedings of the Sixteenth International Conference 124-133.
- Quinlan, J. (1996). Bagging, boosting, and C4.5. In Proceedings Thirteenth American Association for Artificial Intelligence National Conference on Artificial Intelligence 725-730. AAAI Press, Menlo Park, CA.
- Freund, Y. (1999). An adaptive version of boost by majority algorithm. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory. · DOI
- Ratsch, G. (1998). Ensemble learning methods for classification. Masters thesis, Dept. Computer Science, Univ. Potsdam.
Cited by
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
“This permits the replacement of the di cult minimization problem (9) by least{squares minimization (11).”
Self-citationData Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
LightGBM: A Highly Efficient Gradient Boosting Decision Tree
(2017) · HAL · Cited by 10,512
Dormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., Marquéz, J. R. G., Gruber, B., Lafourcade, B., Leitão, P. J., et al. (2013) · Ecography · doi:10.1111/j.1600-0587.2012.07348.x · Cited by 9,998
Metadata
- Authors
- Friedman, J., Hastie, T., & Tibshirani, R.
- Affiliations
- SLAC National Accelerator Laboratory (US)
- Stanford University (US)
- Venue
- The Annals of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2000-04-01
- Volume
- 28
- Issue
- 2
- Pages
- 337-407
- ISSN
- 0090-5364 (ISSN-L)
- 2168-8966
- OpenAlex
- W2024046085
- Cited by
- 6,884
- References
- 40
- Citations by year
- 2026: 148 · 2025: 211 · 2024: 242 · 2023: 282 · 2022: 349 · 2021: 434 · 2020: 372 · 2019: 365 · 2018: 285 · 2017: 286 · 2016: 268 · 2015: 334 · 2014: 348 · 2013: 331 · 2012: 356
- FWCI
- 87.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Annals of Statistics (publishedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- Funders
- National Science Foundation (DMS-97-64431)
- National Institutes of Health
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- boosting, multi-class classification, boosted decision trees, weighted majority voting, binary classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors). https://app.cochat.ai/papers/10.1214/aos/1016218223
- The paper
- Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors). The Annals of Statistics. https://doi.org/10.1214/aos/1016218223
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.